C++ Development Workflow

This page assumes you have already read Preliminaries and that you are adding a module to a repo which is considered part of NWChemEx (see software_stack_overview for more clarification on what’s part of NWChemEx and what’s not). If you are developing a plugin for NWChemEx you should follow the documentation at xxx.

Todo

Write plugin documentation and link to it

Adding new features to NWChemEx is accomplished by adding new modules. NWChemEx relies on a plugin-based architecture and module development follows a more-or-less traditional plugin development cycle. This means you write your module’s source code largely decoupled from the rest of NWChemEx. When you need to get a quantity that your module does not know how to compute (and is not an input) you call out to a submodule. You do not have to write the submodule you call (unless it’s not available in any other repo). Providing your module with the submodule will happen at runtime.

Module development looks a little different depending on whether your module needs to call a submodule or not. The former is the easier scenario

Developing a Module without Submodules

This can be done largely as you would expect. You add your source code to the repo, add a unit test for your module’s source code, and then make sure the unit test works.

Todo

Make this into a true tutorial.

Developing a Module with Submodule(s)

Development gets a bit more hairy when your module depends on a submodule (and those submodules are not part of the current repo). For concreteness let’s assume we are writing a module called JCanonical which does a naive canonical J build. The source file for such a module will live in the SCF repo and will have the path nwx_workspace/SCF/src/scf/j_canonical.cpp. The contents of this source file could be something like:

#include "scf/property_types.hpp" // List of property types used in SCF repo
#include "scf/scf_modules.hpp" // Declarations of SCF modules, including ours
#include "scf/types.hpp"       // types of objects used in SCF repo

namespace scf {


 MODULE_CTOR(JCanonical) {
     // Our module knows how to compute J
     using j_prop_type = pt::coulomb<double, type::derived_space_t<double>>;
     satisfies_property_type<j_prop_type>();

     add_submodule<pt::eri4c<double>>("ERI Builder")
     .set_description("Computes 4C ERI integrals");
 }

 MODULE_RUN(JCanonical) {
     using j_prop_type = pt::coulomb<double, type::derived_space_t<double>>;
     auto [mol, MOs, bra, ket] = j_prop_type::unwrap_inputs(inputs);

     auto& eri_mod = submods.at("ERI Builder");
     auto [ERI4] = eri_mod.run_as<pt::eri4c<double>>(bra, bra, ket, ket);

     type::tensor<double> J;
     type::tensor<double> rho;
     const auto& C = MOs.C();
     rho("mu,nu") = C("mu,i") * C("nu,i");
     J("mu, nu") = rho("lambda, sigma") * ERI4("lambda, sigma, mu, nu");

     auto rv = results();
     return j_prop_type::wrap_results(rv, J);
 }

} // namespace scf

Of importance for our current purposes is the fact that this module depends on a submodule (for computing the four-center, electron-repulsion integrals) which is not part of the SCF repo. Aside from that, there’s nothing too remarkable about the implementation of this module.

To finish off the implementation we also need to make sure JCanonical is declared in nwx_workspace/SCF/include/scf/scf_modules.hpp (this file should really live in src and may have been moved since this documentation was written; if it has and you’re reading this please file an issue/make a PR) and we need to make sure our new module is added to the module manager in nwx_workspace/SCF/src/scf/scf_mm.cpp. The former amounts to adding:

DECLARE_MODULE(JCanonical);

to nwx_workspace/SCF/include/scf/scf_modules.hpp and the latter requires adding:

mm.add_module<JCanonical>("A key users will use to request your module");

to nwx_workspace/SCF/src/scf/scf_mm.cpp.

Running the Module

Now that we wrote the module we need to test/run it. Since our module needs integrals, and integrals are not provided by the SCF repo, we can’t simply add a unit test to nwx_workspace/SCF/tests which calls our module (we’ll get to how to unit test the module, in the SCF repo, later) because our module won’t have integrals. How to proceed depends on whether you are ok with using a Python script to run the calculation or if you insist on the entire development (including running the calculation) occurring in C++ (the former is preferred and the latter will likely be deprecated at some point).

Running the module from Python

In theory you write a Python script which looks like:

import nwchemex as nwx

mm = nwx.sde.ModuleManager()
nwx.load_modules(mm)

#We need to tell our module which ERIs to use
key = "whatever key you used in SCF for your module"
eri_key = "ERI4" # or whatever 4-center ERIs you want to use
mm.change_submod(key, "ERI Builder", eri_key);

# Make the input for our module
mol = nwx.libchemist.Molecule() # Make a Molecule
aos = nwx.libchemist.apply_basis(mol, "sto-3g")
mos = nwx.libchemist.DerivedSpaceD # get MOs from somewhere

# Call our module and bask in the result
mod                = mm.at("the key you put your module under")
derived_space_type = nwx.scf.type.derived_space_t[double]
j_prop_type        = nwx.scf.pt.coulomb[double, derived_space_type]
J                  = mod.run_as[j_prop_type](mol, mos, aos, aos)
print(J)

Assuming nwx.load_modules() is written in Python, you then would simply need to recompile nwx_workspace/SCF and run the above Python script. Python would take care of all of the dynamic linking etc.

Todo

Finish/write this section when NWX’s Python bindings allow this workflow.

Running the module from C++

To run our module from C++ the eaiest way is to add a validation test to the NWChemEx repo, say nwx_workspace/NWChemEx/tests/j_canonical.cpp. The contents of this validation test look something like:

#include <catch2/catch.hpp>
#include <nwchemex/load_modules.hpp>
#include <scf/property_types.hpp>
#include <scf/types.hpp>

using namespace scf;
using j_prop_type = pt::coulomb<double, type::derived_space_t<double>>;

TEST_CASE("Canonical J"){
    sde::ModuleManager mm;
    nwx::load_modules(mm);

    // We need to tell our module which ERIs to use
    const auto key     = "whatever key you used in SCF for your module";
    const auto eri_key = "ERI4"; // or whatever 4-center ERIs you want
    mm.change_submod(key, "ERI Builder", eri_key);

    // Make the input for our module
    auto mol = ;// Make a Molecule
    auto aos = libchemist::apply_basis(mol, "sto-3g");
    auto mos = ;// get MOs from somewhere

    // Call our module and bask in the result
    auto mod = mm.at("the key you put your module under");
    auto [J] = mod.run_as<j_prop_type>(mol, mos, aos, aos);
    std::cout << J << std::endl;
}

With this validation test written, we then compile nwx_workspace/NWChemEx, and run the tests in nwx_workspace/NWChemEx. N.B., we are not compiling the SCF repo; if the toolchain file is setup correctly building nwx_workspace/NWChemEx will use our local, modified, copy of SCF.

Note

After development is complete you should add the mm.change_submod line to the NWChemEx/src/nwchemex/load_modules.cpp file. So that the module is ready to be used outside of just the validation test.

Unit Testing the Module

It’s sometimes easier to get a module working using “real” data, which is what the previous section focused on. That said the unit test for our module should live in the SCF repo and not be coupled to the module used to get the integrals, i.e., if the integrals module breaks/changes we don’t want it break our module’s unit test too. To avoid this coupling in our module’s unit test we use a lambda module, which wraps some hard-coded data.

To start with we create a source file nwx_workspace/SCF/tests/jcanonical.cpp with the contents:

#include <catch2/catch.hpp>
#include <scf/scf_mm.hpp>
#include <scf/property_types.hpp>
#include <scf/types.hpp>
#include <sde/lambda_module.hpp>

using namespace scf;
using j_prop_type   = pt::coulomb<double, type::derived_space_t<double>>;
using eri_prop_type = pt::eri4c<double>;

TEST_CASE("Canonical J"){
    sde::ModuleManager mm;
    scf::load_modules(mm);

    // Make the input for our module
    auto mol = ;// Make a Molecule
    auto aos = libchemist::apply_basis(mol, "sto-3g");
    auto mos = ;// get MOs from somewhere

    // Make the lambda module which will serve as the submodule
    type::tensor<double> eris; // hard-coded ERIS, in practice need state
    auto l = [=](auto& bra1, auto& bra2, auto& ket1, auto& ket2) {
        // Make sure our module passes the right info to the submodule
        REQUIRE(bra1 == aos);
        REQUIRE(bra2 == aos);
        REQUIRE(ket1 == aos);
        REQUIRE(ket2 == aos);

        // It did so return the hard coded result
        return eris;
    };
    auto submod = sde::make_lambda<eri_prop_type>(l);


    // Tell our module to use the lambda module
    const auto key     = "whatever key you used in SCF for your module";
    auto& mod = mm.at(key);
    mod.change_submod("ERI Builder", submod);

    // Call our module and compare J to the correct value
    auto [J] = mod.run_as<j_prop_type>(mol, mos, aos, aos);
    std::cout << J << std::endl;
}

We now build nwx_workspace/SCF and run its tests to ensure our module works correctly. In particular note that this test is self-contained in that it should only fail if the implementation of our module changes (ignoring infrastructure breaks).

Neding hard-coded data for unit tests is very common which is why we made the NWChemEx-Project/testing repo (https://github.com/NWChemEx-Project/testing). You are encouraged to use that data when it makes sense.

Note

For developers who prefer test-based development it’s entirely possible to start with these unit tests and then to proceed to the validation tests with real integrals modules.