active_learning_step package

Submodules

active_learning_step.active_learning module

Non-graphical part of the Active Learning step in a SEAMM flowchart

class active_learning_step.active_learning.ActiveLearning(flowchart=None, title='Active Learning', namespace='org.molssi.seamm.active_learning', extension=None, logger=<Logger active_learning_step.active_learning (WARNING)>)

Bases: Node

The non-graphical part of an Active Learning step in a flowchart.

parser

The parser object.

Type:

configargparse.ArgParser

options

It contains a two item tuple containing the populated namespace and the list of remaining argument strings.

Type:

tuple

subflowchart

A SEAMM Flowchart object that represents a subflowchart, if needed.

Type:

seamm.Flowchart

parameters

The control parameters for Active Learning.

Type:

ActiveLearningParameters

See also

TkActiveLearning, ActiveLearning, ActiveLearningParameters

analyze(indent='', **kwargs)

Do any analysis of the output from this step.

Also print important results to the local step.out file using “printer”.

Parameters:

indent (str) – An extra indentation for the output

description_text(P=None)

Create the text description of what this step will do. The dictionary of control values is passed in as P so that the code can test values, etc.

Parameters:

P (dict) – An optional dictionary of the current values of the control parameters.

Returns:

A description of the current step.

Return type:

str

property git_revision

The git version of this module.

run()

Run the Active Learning step: each sub-step in turn runs itself.

Returns:

The next node in the flowchart, after this step.

Return type:

seamm.Node

set_id(node_id)

Set the id for node to a given tuple

property version

The semantic version of this module.

active_learning_step.active_learning_step module

class active_learning_step.active_learning_step.ActiveLearningStep(flowchart=None, gui=None)

Bases: object

Helper class needed for the stevedore integration.

This must provide a description() method that returns a dict containing a description of this node, and create_node() and create_tk_node() methods for creating the graphical and non-graphical nodes.

The dictionary for the description is the class variable just below these comments. The felds are as follows:

my_description{str, str}

A human-readable description of this step. It can be several lines long, and needs to be clear to non-expert users. It contains the following keys: description, group, name.

my_description[“description”]tuple

A description of the Active Learning step. It must be clear to non-experts.

my_description[“group”]str

Which group in the menus to put this step. If the group does not exist it will be created. Common groups are “Building”, “Control”, “Custom”, “Data”, and “Simulations”.

my_description[“name”]str

The name of this step, to be displayed in the menus.

create_node(flowchart=None, **kwargs)

Create and return the new node object.

Parameters:

• flowchart (seamm.Node) – A non-graphical SEAMM node

• **kwargs (keyword arguments) – Various keyword arguments such as title, namespace or extension representing the title displayed in the flowchart, the namespace for the plugins of a subflowchart and the extension, respectively.

Return type:

ActiveLearning

create_tk_node(canvas=None, **kwargs)

Create and return the graphical Tk node object.

Parameters:

• canvas (tk.Canvas) – The Tk Canvas widget

• **kwargs (keyword arguments) – Various keyword arguments such as tk_flowchart, node, x, y, w, h representing a graphical flowchart object, a non-graphical node for a step, and dimensions of the graphical node.

Return type:

TkActiveLearning

description()

Return a description of what this step does.

Returns:

description

Return type:

dict(str, str)

my_description = {'description': 'Query-by-committee active learning for machine-learned force fields: train a committee, check it, score candidate structures and select the ones to label', 'group': 'Simulations', 'name': 'Active Learning'}

active_learning_step.check module

seamm-al check: does the committee’s spread predict its real error?

For a trained committee and labelled frames, every frame gets its committee spreads (rho_*, as seamm-al score) and its out-of-fold errors, the errors of the members that never trained on it:

• a frame held out by one member (its fold): that member’s error;

• a frame no member trained on (a common validation or test set): the mean of the members’ errors;

• a frame every member trained on is trained and left out of the statistics.

The errors are the force RMSE over the atoms (err_forces, eV/A, the RMS of the force-vector error), the energy per atom (err_energy, eV) and, for cells, the atomic and molecular pressures (err_pressure, err_pressure_molecular, atm). The label’s molecular pressure is computed from its stress and forces with molecular_pressure(). Alongside, the worst-member errors of Smith et al.’s definition (worst_error(): eps_energy per sqrt(atom), eps_forces, eps_pressure, eps_pressure_molecular) over all members, which --calibrate uses.

The committee directory is either one made by seamm-al train (manifest.json and frames.csv: frame i is held out by member fold[i], matched by configuration name) or one with a fold_manifest.json per member, listing tags under held_out, validation and test (mlff’s layout). tag_key names the info key holding a frame’s tag; by default its configuration name.

Outputs: a CSV with one row per frame, and the Spearman correlations of each spread with its error, per kind, over all out-of-fold frames and over the held-out ones only.

active_learning_step.check.PAIRS = {'cell': [('rho_forces', 'err_forces'), ('rho_forces_max_atom', 'err_forces'), ('rho_energy', 'err_energy'), ('rho_pressure_molecular', 'err_pressure_molecular'), ('rho_pressure', 'err_pressure')], 'cluster': [('rho_forces', 'err_forces'), ('rho_forces_max_atom', 'err_forces'), ('rho_energy', 'err_energy')]}

The spread and error compared, per kind.

active_learning_step.check.check_committee(directory, data, out_csv='check.csv', tag_key=None, energy_key='REF_energy', forces_key='REF_forces', stress_key='REF_stress', dispersion=False, device='cpu', dtype=None, checkpoint='best.pt', calibrate=(), summary_json=None, drop=(), flag_factor=1.5, members_csv=None, threads=None)

Spreads against out-of-fold errors for every labelled frame.

Parameters:

• directory (str) – The committee directory (see the module docstring).

• data (sequence of str) – Labelled extxyz files.

• out_csv (str, optional) – The per-frame CSV.

• tag_key (str, optional) – The info key holding a frame’s tag; by default its configuration name.

• energy_key (str, optional) – Where the labels are (info / arrays); by default REF_*.

• forces_key (str, optional) – Where the labels are (info / arrays); by default REF_*.

• stress_key (str, optional) – Where the labels are (info / arrays); by default REF_*.

• dispersion (bool, optional) – Add the checkpoints’ recorded dispersion (for labels that include it); by default False, for labels with the dispersion removed.

• device (optional) – As in seamm-al score.

• dtype (optional) – As in seamm-al score.

• checkpoint (str, optional) – The members’ checkpoint, by default best.pt.

• calibrate (sequence of str, optional) – NAME:FAIL_LEVEL[:CAPTURE] with NAME a spread, e.g. rho_forces:0.15 (eV/A) or rho_pressure_molecular:500:0.9 (atm).

• summary_json (str, optional) – Also write the correlations and calibrations here.

• drop (sequence of int, optional) – Members to leave out of the committee (the k of m<k>).

• flag_factor (float, optional) – Flag a member whose loss or error is more than this times the median of the others’, by default 1.5.

• threads (int, optional) – CPU threads for torch (set_threads()).

• members_csv (str, optional) – The member-health table; by default next to out_csv (<stem>_members.csv).

Returns:

The per-frame rows and the summary.

Return type:

(list of dict, dict)

active_learning_step.check.read_folds(directory, checkpoint='best.pt', drop=())

The members of a committee and which frames each never trained on.

Parameters:

• directory (str) – A seamm-al train directory, or one with m<k>/fold_manifest.json.

• checkpoint (str, optional) – The members’ checkpoint file, by default best.pt.

• drop (sequence of int, optional) – Members to leave out (the k of m<k>). The frames a dropped member held out were trained on by all the others, so they become trained.

Returns:

The kept members’ checkpoints, in order; for each tag either ("held", [i]) or ("unseen", [i, ...]) with i the position in that list; the layout, "train" or "fold_manifest"; and the kept members’ k.

Return type:

(list of str, dict, str, list of int)

active_learning_step.check.spearman(x, y)

Spearman’s rank correlation and its two-sided p-value.

The coefficient is the Pearson correlation of the ranks (ties averaged), as scipy.stats.spearmanr; the p-value is scipy’s when scipy is there, else from the t distribution’s normal approximation.

Returns:

The coefficient, the p-value and the number of pairs (nan with fewer than 3 finite pairs).

Return type:

(float, float, int)

active_learning_step.check_committee module

Non-graphical part of the Check Committee step in an Active Learning flowchart

class active_learning_step.check_committee.CheckCommittee(flowchart=None, title='Check Committee', extension=None, logger=<Logger active_learning_step.check_committee (WARNING)>)

Bases: Substep

The non-graphical part of a Check Committee step in a flowchart.

parser

The parser object.

Type:

configargparse.ArgParser

options

It contains a two item tuple containing the populated namespace and the list of remaining argument strings.

Type:

tuple

subflowchart

A SEAMM Flowchart object that represents a subflowchart, if needed.

Type:

seamm.Flowchart

parameters

The control parameters for Check Committee.

Type:

CheckCommitteeParameters

See also

TkCheckCommittee, CheckCommittee, CheckCommitteeParameters

analyze(indent='', summary=None, thresholds=None, P=None, **kwargs)

Print the check and store the results.

Parameters:

• indent (str) – An extra indentation for the output

• summary (dict) – The summary written by seamm-al check.

• thresholds (dict) – The calibrated thresholds, as KIND:NAME -> value.

• P (dict) – The parameters of the step.

description_text(P=None)

Create the text description of what this step will do.

Parameters:

P (dict) – An optional dictionary of the current values of the control parameters.

Returns:

A description of the current step.

Return type:

str

property git_revision

The git version of this module.

property header

A printable header for this section of output

run()

Check the committee.

Returns:

The next node object in the flowchart.

Return type:

seamm.Node

training_data(committee)

The labelled data files a committee was trained on, from its manifest.

property version

The semantic version of this module.

active_learning_step.check_committee_parameters module

Control parameters for the Check Committee step in a SEAMM flowchart

class active_learning_step.check_committee_parameters.CheckCommitteeParameters(defaults={}, data=None)

Bases: Parameters

The control parameters for Check Committee.

You need to replace the “time” entry in dictionary below these comments with the definitions of parameters to control this step. The keys are parameters for the current plugin,the values are dictionaries as outlined below.

Examples

parameters = {
    "time": {
        "default": 100.0,
        "kind": "float",
        "default_units": "ps",
        "enumeration": tuple(),
        "format_string": ".1f",
        "description": "Simulation time:",
        "help_text": ("The time to simulate in the dynamics run.")
    },
}

parameters{str: {str: str}}

A dictionary containing the parameters for the current step. Each key of the dictionary is a dictionary that contains the the following keys:

parameters[“default”] :

The default value of the parameter, used to reset it.

parameters[“kind”]enum()

Specifies the kind of a variable. One of “integer”, “float”, “string”, “boolean”, or “enum”

While the “kind” of a variable might be a numeric value, it may still have enumerated custom values meaningful to the user. For instance, if the parameter is a convergence criterion for an optimizer, custom values like “normal”, “precise”, etc, might be adequate. In addition, any parameter can be set to a variable of expression, indicated by having “$” as the first character in the field. For example, $OPTIMIZER_CONV.

parameters[“default_units”]str

The default units, used for resetting the value.

parameters[“enumeration”]: tuple

A tuple of enumerated values.

parameters[“format_string”]: str

A format string for “pretty” output.

parameters[“description”]: str

A short string used as a prompt in the GUI.

parameters[“help_text”]: str

A longer string to display as help for the user.

See also

CheckCommittee, TkCheckCommittee, CheckCommitteeParameters, CheckCommitteeStep

parameters = {'add dispersion': {'default': 'no', 'default_units': '', 'description': 'Add the dispersion:', 'enumeration': ('yes', 'no'), 'format_string': '', 'help_text': 'For models trained on labels with the dispersion removed, whether to add it back. It is the same for every member, so it does not change the spreads.', 'kind': 'boolean'}, 'calibrate': {'default': '', 'default_units': '', 'description': 'Calibrate thresholds:', 'enumeration': (), 'format_string': '', 'help_text': 'Spreads to calibrate a selection threshold for, as NAME:ERROR[:CAPTURE] separated by spaces. The threshold catches the fraction CAPTURE (default 0.98) of the frames whose error is above ERROR, e.g. rho_forces:0.15 for force errors above 0.15 eV/Å. Select Frames can use the thresholds.', 'kind': 'string'}, 'checkpoint': {'default': 'best.pt', 'default_units': '', 'description': 'Checkpoint:', 'enumeration': ('best.pt', 'last.pt'), 'format_string': '', 'help_text': 'Which checkpoint of each member to use: the best on its validation fold, or the last.', 'kind': 'string'}, 'committee': {'default': 'previous', 'default_units': '', 'description': 'Committee:', 'enumeration': ('previous',), 'format_string': '', 'help_text': "The committee: 'previous' for the one trained or used earlier in this job, or the directory of a committee made by Train Committee (it holds the members m0, m1, ...).", 'kind': 'string'}, 'device': {'default': 'default', 'default_units': '', 'description': 'Device:', 'enumeration': ('default', 'cpu', 'cuda', 'mps'), 'format_string': '', 'help_text': "The PyTorch device to run the models on: cpu, cuda (or cuda:N), or mps. 'default' uses the device in xnn.ini, else the CPU.", 'kind': 'string'}, 'energy key': {'default': 'REF_energy', 'default_units': '', 'description': 'Energy key:', 'enumeration': (), 'format_string': '', 'help_text': 'The key of the reference energy in the extended XYZ files.', 'kind': 'string'}, 'flag factor': {'default': 1.5, 'default_units': '', 'description': 'Flag members worse than:', 'enumeration': (), 'format_string': '.2f', 'help_text': 'Flag a member whose final validation loss is more than this times the median of the others.', 'kind': 'float'}, 'forces key': {'default': 'REF_forces', 'default_units': '', 'description': 'Forces key:', 'enumeration': (), 'format_string': '', 'help_text': 'The key of the reference forces in the extended XYZ files.', 'kind': 'string'}, 'labelled data': {'default': "the committee's training data", 'default_units': '', 'description': 'Labelled data:', 'enumeration': ("the committee's training data",), 'format_string': '', 'help_text': "Labelled structures to check on: the committee's own training data, where each member is checked on the fold it never saw, or extended XYZ files (names or patterns separated by spaces) of new structures that no member saw.", 'kind': 'string'}, 'leave out members': {'default': '', 'default_units': '', 'description': 'Leave out members:', 'enumeration': (), 'format_string': '', 'help_text': 'Members to leave out of the committee, by number (0, 1, ...), separated by spaces: for example, ones that Check Committee flagged.', 'kind': 'string'}, 'precision': {'default': "the model's", 'default_units': '', 'description': 'Precision:', 'enumeration': ("the model's", 'float32', 'float64'), 'format_string': '', 'help_text': 'The floating-point precision to evaluate the models in.', 'kind': 'string'}, 'results': {'default': {}, 'default_units': '', 'description': 'results', 'enumeration': (), 'format_string': '', 'help_text': 'The results to save to variables or in tables.', 'kind': 'dictionary'}, 'stress key': {'default': 'REF_stress', 'default_units': '', 'description': 'Stress key:', 'enumeration': (), 'format_string': '', 'help_text': 'The key of the reference stress in the extended XYZ files.', 'kind': 'string'}}

active_learning_step.check_committee_step module

class active_learning_step.check_committee_step.CheckCommitteeStep(flowchart=None, gui=None)

Bases: object

Helper class needed for the stevedore integration.

This must provide a description() method that returns a dict containing a description of this node, and create_node() and create_tk_node() methods for creating the graphical and non-graphical nodes.

The dictionary for the description is the class variable just below these comments. The felds are as follows:

my_description{str, str}

A human-readable description of this step. It can be several lines long, and needs to be clear to non-expert users. It contains the following keys: description, group, name.

my_description[“description”]tuple

A description of the Check Committee step. It must be clear to non-experts.

my_description[“group”]str

Which group in the menus to put this step. If the group does not exist it will be created. Common groups are “Building”, “Control”, “Custom”, “Data”, and “Simulations”.

my_description[“name”]str

The name of this step, to be displayed in the menus.

create_node(flowchart=None, **kwargs)

Create and return the new node object.

Parameters:

• flowchart (seamm.Node) – A non-graphical SEAMM node

• **kwargs (keyword arguments) – Various keyword arguments such as title, namespace or extension representing the title displayed in the flowchart, the namespace for the plugins of a subflowchart and the extension, respectively.

Return type:

CheckCommittee

create_tk_node(canvas=None, **kwargs)

Create and return the graphical Tk node object.

Parameters:

• canvas (tk.Canvas) – The Tk Canvas widget

• **kwargs (keyword arguments) – Various keyword arguments such as tk_flowchart, node, x, y, w, h representing a graphical flowchart object, a non-graphical node for a step, and dimensions of the graphical node.

Return type:

TkCheckCommittee

description()

Return a description of what this step does.

Returns:

description

Return type:

dict(str, str)

my_description = {'description': "Check that the committee's spread predicts its error on frames it never trained on", 'group': 'Calculations', 'name': 'Check Committee'}

active_learning_step.cli module

The seamm-al command: query-by-committee active learning from the shell.

seamm-al score -m m0/best.pt m1/best.pt ... [-o scores.csv] pool.extxyz [...]
seamm-al train -c recipe.yaml -o committee labelled.extxyz [--cell-share 0.15]
seamm-al select scores.csv --budget 75000 \
    --cost cell:EC=2800 --cost cell:FEC=3900 \
    --group-column liquid --cap temperature=6 --exclude labelled.txt

score, train, check and engine need torch and xnn: use SEAMM’s MLFF environment (xnn 0.13 or later; training requires it). select needs neither. Without installing the package, run python -m active_learning_step.cli with the repository on PYTHONPATH.

active_learning_step.cli.main(argv=None)

Run seamm-al.

active_learning_step.cli.parser()

The argument parser of seamm-al.

active_learning_step.committee module

Query-by-committee tools: ensembles of potentials, disagreement, thresholds.

The active-learning recipe of Smith et al. (J. Chem. Phys. 148, 241733, 2018): a committee of L potentials trained on different folds of the same data with different initialisations; a structure i is uncertain when the spread of the committee’s energies

\[\rho_i = \sigma_i / \sqrt{N_i}\]

(their eq 1; \(\sigma_i\) the standard deviation of the L energies, \(N_i\) the atom count) exceeds a threshold \(\hat\rho\). The threshold is calibrated on a labelled benchmark (their fig 1): it is the value that captures a chosen fraction (98 %) of the structures whose largest per-member error \(\epsilon_i = \max_l |E_l - E_{ref}| / \sqrt{N_i}\) exceeds a failure level. The mean of the L predictions is the committee’s prediction (their sec II E).

Condensed phases need more than the energy. In liquid water the energy error of clusters never predicted the pressure of the bulk, so force and pressure spreads are provided alongside, calibrated the same way:

• \(\rho_F\): the root mean square over the atoms of the committee standard deviation of the force (eV/A);

• \(\rho_P\): the committee standard deviation of the pressure (atm), atomic (from the stress) or molecular (the intramolecular virial removed, the pressure a rigid-molecule barostat feels).

Nothing here trains. The members are trained by the Trainer, one run per member, on the folds of kfold_indices(); Committee loads the resulting checkpoints and scores structures.

Origin and changes

Adapted from code by Sina Mostafanejad, written for liquid water and xnn 0.8.0. The API and the formulas are his; the changes generalise it to other systems:

• the molecules come from a covalent bond graph (molecules(), in active_learning_step.molecules) rather than atom index // 3; ions such as Li+ are molecules of their own, and the masses come from the elements (masses());

• a structure’s total_charge reaches the models (structure_from_atoms() puts it in the structure dict that xnn’s graph builder reads);

• Committee.from_checkpoints() refuses members that record different subtracted dispersions, as well as different cutoffs;

• the numpy-only functions (kfold_indices(), calibrate_threshold(), split_statistics(), select()) live in active_learning_step.thresholds, so that selecting needs no torch; they are re-exported here unchanged;

• besides rho_forces, disagreement() returns the per-atom spread rho_forces_atom and its maximum rho_forces_max_atom, because one bad ion environment is diluted in the RMS over a large cell.

xnn interface

Written for xnn 0.11.x. The xnn names used, all public:

• xnn.common.data: AtomicDataset, AtomicGraph, collate;

• xnn.common.models.hub: load_checkpoint, build_potential (in Committee.from_checkpoints());

• the checkpoint’s Config.subtracted_dispersion attribute, read with a default as xnn.common.models.registry.recorded_dispersion does;

• members are xnn.common.models.ForceStressOutput objects, as build_potential returns them.

active_learning_step.committee.ATM_PER_EV_A3 = 1581225.3974833458

atm per eV/A^3 (CODATA 2018 electronvolt, 101 325 Pa per atm).

class active_learning_step.committee.Committee(*args: Any, **kwargs: Any)

Bases: Module

A committee of potentials evaluated together.

Every member sees its own shallow copy of the batch, because the force / stress wrapper displaces positions and cell by the strain it differentiates, and returns detached predictions.

Parameters:

• members (sequence of torch.nn.Module) – The potentials, each returning energy (B,), forces (N, 3) and, for periodic batches, stress (B, 3, 3) (a ForceStressOutput built with compute_stress=True). They must share one cutoff, since one neighbour list serves all of them.

• subtracted_dispersion (sequence, optional) – Each member’s subtracted_dispersion record (what its training labels had removed); from_checkpoints() reads them from the checkpoints. They must all be equal: members of different label spaces disagree for a reason that is not uncertainty.

members

The potentials.

Type:

torch.nn.ModuleList

cutoff

Their common neighbour-list cutoff.

Type:

float

subtracted_dispersion

The members’ common record (None when not given or not recorded).

Type:

dict, str or None

Raises:

ValueError – With fewer than two members, with differing cutoffs, or with differing subtracted dispersions.

disagreement(data: xnn.common.data.AtomicGraph, molecule: torch.Tensor | None = None, masses: torch.Tensor | None = None) → Dict[str, torch.Tensor | None]

The spreads of a batch, with the members’ and the mean prediction.

Parameters:

• data (AtomicGraph) – A structure or batch on the members’ device.

• molecule (Tensor, optional) – Molecule index per atom (N,), unique across the batch. When given and the batch has a cell, the molecular pressure spread is added.

• masses (Tensor, optional) – Atomic masses (N,) for the molecular centres of mass.

Returns:

The entries of disagreement() (rho_energy, rho_forces, and rho_pressure / rho_pressure_molecular in atm for periodic batches), the members’ energy (L, B), forces (L, N, 3), stress, pressure (L, B) and pressure_molecular (L, B) in atm, their means under mean_<name>, and n_atoms (B,).

Return type:

dict of str to Tensor

forward(data: xnn.common.data.AtomicGraph) → Dict[str, torch.Tensor | None]

Every member’s prediction of a batch.

Parameters:

data (AtomicGraph) – A structure or batch on the members’ device.

Returns:

energy (L, B), forces (L, N, 3) and stress (L, B, 3, 3) (None when the batch has no cell), detached.

Return type:

dict of str to Tensor

classmethod from_checkpoints(paths: Iterable[str], device=None, dtype=None, dispersion=False, compute_stress: bool = True, **build_options) → Committee

Load the members from trainer checkpoints or model directories.

Parameters:

• paths (iterable of str) – Anything load_checkpoint() reads.

• device (torch.device or str, optional) – Where to serve the members.

• dtype (torch.dtype or str, optional) – Dtype to serve in; by default that of each checkpoint’s weights.

• dispersion (dict, str, bool or None, optional) – Passed to build_potential(). By default False: a recorded subtracted dispersion is a term shared by every member, so it adds nothing to the spread and is left out. Pass None to serve the full energy.

• compute_stress (bool, optional) – Build the stress head (needed for the pressure spread), by default True.

• **build_options – Further keywords of build_potential().

Return type:

Committee

Raises:

ValueError – When the checkpoints record different subtracted dispersions or the members differ in cutoff.

predict(data: xnn.common.data.AtomicGraph) → Dict[str, torch.Tensor | None]

The committee’s prediction: the mean over the members.

Parameters:

data (AtomicGraph) – A structure or batch on the members’ device.

Returns:

energy (B,), forces (N, 3), stress (B, 3, 3) or None.

Return type:

dict of str to Tensor

score(structures, batch_size: int = 8, molecule: Callable[[dict], ndarray] | None = None, masses: Callable[[dict], ndarray] | None = None, device=None, keys: Sequence[str] = ('rho_energy', 'rho_forces', 'rho_forces_max_atom', 'rho_pressure', 'rho_pressure_molecular', 'mean_energy', 'mean_pressure', 'mean_pressure_molecular', 'n_atoms')) → Dict[str, ndarray]

Score many structures: per-structure spreads as numpy arrays.

Parameters:

• structures (AtomicDataset or list of dict) – The structures; a list is wrapped in an AtomicDataset at the committee’s cutoff.

• batch_size (int, optional) – Structures per batch, by default 8.

• molecule (callable, optional) – f(structure) -> (n_i,) molecule index per atom of one structure, in general molecules() (or, for OHH waters only, lambda s: np.arange(len(s["pos"])) // 3). Enables the molecular pressure spread.

• masses (callable, optional) – f(structure) -> (n_i,) atomic masses, with molecule; in general the module’s masses().

• device (torch.device or str, optional) – Where the batches go; by default the first member’s device.

• keys (sequence of str, optional) – Which entries of disagreement() to collect: per-structure ones by default; add "rho_forces_atom" for the per-atom spread.

Returns:

Each requested key present in the output, concatenated over the structures in order: (n_structures,) for the per-structure entries, (total atoms,) for rho_forces_atom (split it with n_atoms). Pressure keys are absent when no structure has a cell, and nan for the structures of batches without one.

Return type:

dict of str to ndarray

property size: int

Number of members L.

active_learning_step.committee.calibrate_threshold(rho: ndarray, eps: ndarray, fail_level: float, capture: float = 0.98) → float

The inclusion threshold of Smith et al., fig 1.

The threshold is the largest value of rho such that at least capture of the structures whose error eps exceeds fail_level have rho >= threshold, so selecting by rho >= threshold catches that fraction of the failures.

Parameters:

• rho (ndarray) – Committee spread per structure (one entry of disagreement()).

• eps (ndarray) – Worst error per structure (the matching entry of worst_error()), on the same structures.

• fail_level (float) – Error above which a structure counts as a failure (the paper: 1.5 kcal/mol in its energy units).

• capture (float, optional) – Fraction of the failures the threshold must catch, by default 0.98.

Returns:

The threshold.

Return type:

float

Raises:

ValueError – If no structure fails, so there is nothing to calibrate on.

active_learning_step.committee.disagreement(energy: torch.Tensor, forces: torch.Tensor, n_atoms: torch.Tensor, batch: torch.Tensor, pressures: Dict[str, torch.Tensor] | None = None) → Dict[str, torch.Tensor]

The committee spreads of a batch of structures.

The standard deviations are population ones (ddof = 0), as in the reference implementation of Smith et al.

Parameters:

• energy (Tensor) – Member energies (L, B) in eV.

• forces (Tensor) – Member forces (L, N, 3) in eV/A.

• n_atoms (Tensor) – Atoms per structure (B,).

• batch (Tensor) – Structure index per atom (N,).

• pressures (dict of str to Tensor, optional) – Named member pressures (L, B) in atm, for example {"pressure": ..., "pressure_molecular": ...}; each adds a rho_<name> entry.

Returns:

rho_energy (B,) in eV: \(\sigma_E / \sqrt{N}\); rho_forces (B,) in eV/A: the RMS over atoms of the per-atom standard deviation of the force vector; rho_forces_atom (N,) in eV/A: that per-atom standard deviation, sqrt(sum_xyz var); rho_forces_max_atom (B,): its largest value in each structure; and rho_<name> (B,) per pressure given, in atm.

Return type:

dict of str to Tensor

active_learning_step.committee.kfold_indices(n: int, k: int, seed: int = 0) → list[tuple[ndarray, ndarray]]

The k folds of a committee’s training data.

Member l trains on every fold but fold l and validates on fold l, the 5-fold split of Smith et al. (k = 5). One permutation seeded by seed is cut into k nearly equal folds, so the folds are disjoint and cover the n indices exactly once.

Parameters:

• n (int) – Number of training structures.

• k (int) – Number of folds, which is the number of committee members.

• seed (int, optional) – Seed of the permutation, by default 0. This is the data seed; the initialisation seed of each member is the trainer’s cfg.seed.

Returns:

For each member, (train_indices, held_indices) into the training set, sorted.

Return type:

list of (ndarray, ndarray)

Raises:

ValueError – If k is smaller than 2 or larger than n.

active_learning_step.committee.masses(structure)

The atomic masses (amu) of a structure, from ASE’s table.

Parameters:

structure (ase.Atoms or dict) – The structure.

Returns:

(N,) masses.

Return type:

numpy.ndarray

active_learning_step.committee.molecular_pressure(stress: torch.Tensor, pos: torch.Tensor, forces: torch.Tensor, cell: torch.Tensor, batch: torch.Tensor, molecule: torch.Tensor, pbc: torch.Tensor | None = None, masses: torch.Tensor | None = None) → torch.Tensor

The molecular pressure P_atom - tr(W_intra) / (3 V) per structure.

The pressure of a rigid-molecule description of the same structure: the atomic pressure of the stress with the intramolecular virial of molecular_virial() removed. It is what the labelling pipeline of the water work reports and the quantity the committee spread is taken over in disagreement() when molecules are given.

Parameters:

• stress (Tensor) – Stress (B, 3, 3) in eV/A^3.

• pos – As in molecular_virial().

• forces – As in molecular_virial().

• cell – As in molecular_virial().

• batch – As in molecular_virial().

• molecule – As in molecular_virial().

• pbc – As in molecular_virial().

• masses – As in molecular_virial().

Returns:

Molecular pressure (B,) in eV/A^3. Non-periodic structures of a mixed batch get nan.

Return type:

Tensor

active_learning_step.committee.molecular_virial(pos: torch.Tensor, forces: torch.Tensor, batch: torch.Tensor, molecule: torch.Tensor, cell: torch.Tensor | None = None, pbc: torch.Tensor | None = None, masses: torch.Tensor | None = None, n_structures: int | None = None) → torch.Tensor

The intramolecular virial sum_i (x_i - R_m) . F_i per structure.

Each molecule is made whole by the minimum image of every atom with respect to its first atom (periodic structures only), R_m is its centre of mass (its geometric centre without masses), and the trace of the virial is summed over the molecules of each structure.

Parameters:

• pos (Tensor) – Positions (N, 3) in A.

• forces (Tensor) – Forces (N, 3) in eV/A.

• batch (Tensor) – Structure index per atom (N,).

• molecule (Tensor) – Molecule index per atom (N,), unique across the batch.

• cell (Tensor, optional) – Lattice vectors as rows (B, 3, 3); needed for the minimum image.

• pbc (Tensor, optional) – Periodic flags (B, 3); a structure is wrapped only when all three are set. None with a cell means every structure is periodic.

• masses (Tensor, optional) – Atomic masses (N,) for the centre of mass. None uses the geometric centre.

• n_structures (int, optional) – Number of structures B; by default batch.max() + 1.

Returns:

The virial trace (B,) in eV.

Return type:

Tensor

active_learning_step.committee.molecules(structure, **rule)

The molecule index of every atom: connected components of the bonds.

Parameters:

• structure (ase.Atoms or dict) – The structure (an xnn structure dict is accepted too). Periodic directions use every image.

• **rule – tolerance, radii, max_bonds, exclude, min_distance, as in bonds() (molsystem’s perceive_bonds).

Returns:

(N,) molecule index per atom, numbered 0, 1, … in the order of each molecule’s first atom.

Return type:

numpy.ndarray of int

active_learning_step.committee.pressure(stress: torch.Tensor) → torch.Tensor

The isotropic pressure of a stress tensor, -tr(sigma) / 3.

Parameters:

stress (Tensor) – Stress of shape (..., 3, 3) in eV/A^3 with the ASE / xnn sign (sigma = -P under hydrostatic pressure).

Returns:

Pressure of shape (...) in eV/A^3 (multiply by ATM_PER_EV_A3 for atm).

Return type:

Tensor

active_learning_step.committee.select(rhos: Dict[str, ndarray], thresholds: Dict[str, float]) → ndarray

Which structures any calibrated spread flags.

Parameters:

• rhos (dict of str to ndarray) – Spreads per structure by name (rho_energy, rho_forces, …).

• thresholds (dict of str to float) – Threshold per name; names absent from rhos are ignored, and a nan spread never selects.

Returns:

Boolean mask, true where rho >= threshold for at least one name.

Return type:

ndarray

active_learning_step.committee.split_statistics(rho: ndarray, err: ndarray, threshold: float) → Dict[str, float]

What a threshold separates: the error above and below it.

Parameters:

• rho (ndarray) – Committee spread per structure.

• err (ndarray) – An error per structure (the committee mean’s, or the worst member’s).

• threshold (float) – The threshold from calibrate_threshold().

Returns:

fraction_selected (rho >= threshold), n_above, n_below, and the RMS of err on each side (rmse_above, rmse_below; nan for an empty side).

Return type:

dict of str to float

active_learning_step.committee.structure_from_atoms(atoms, total_charge=None) → dict

The xnn structure dict of an ASE Atoms, for scoring.

Only the geometry and the total charge are taken: no labels, so nothing in the frame’s info or arrays can change what the models see.

Parameters:

• atoms (ase.Atoms) – The structure. It is periodic when any direction is.

• total_charge (float, optional) – The net charge; by default atoms.info["total_charge"] (or "charge"), and none (neutral to the models) when absent.

Returns:

pos, atomic_numbers, cell and pbc (both None for a non-periodic structure) and total_charge when known.

Return type:

dict

active_learning_step.committee.worst_error(energy: torch.Tensor, forces: torch.Tensor, n_atoms: torch.Tensor, batch: torch.Tensor, energy_ref: torch.Tensor, forces_ref: torch.Tensor, pressures: Dict[str, torch.Tensor] | None = None, pressures_ref: Dict[str, torch.Tensor] | None = None) → Dict[str, torch.Tensor]

The largest per-member error of each structure (the paper’s epsilon).

Parameters:

• energy – As in disagreement().

• forces – As in disagreement().

• n_atoms – As in disagreement().

• batch – As in disagreement().

• pressures – As in disagreement().

• energy_ref (Tensor) – Reference energies (B,).

• forces_ref (Tensor) – Reference forces (N, 3).

• pressures_ref (dict of str to Tensor, optional) – Reference pressures (B,) per name in pressures.

Returns:

eps_energy (B,): \(\max_l |E_l - E_{ref}| / \sqrt{N}\); eps_forces (B,): the largest over members of the RMS force error of the structure; eps_<name> (B,): the largest absolute pressure error.

Return type:

dict of str to Tensor

active_learning_step.covalent_radii module

Covalent radii (Å) as molsystem uses them for bond perception.

Generated from mendeleev (version 1.3.0): covalent_radius_pyykko, falling back to covalent_radius, divided by 100 (pm to Å), exactly as molsystem.topology.TopologyMixin._covalent_radii. Kept as a table so that active_learning_step.molecules needs neither mendeleev nor molsystem.

active_learning_step.covalent_radii.COVALENT_RADII = {1: 0.32, 2: 0.46, 3: 1.33, 4: 1.02, 5: 0.85, 6: 0.75, 7: 0.71, 8: 0.63, 9: 0.64, 10: 0.67, 11: 1.55, 12: 1.39, 13: 1.26, 14: 1.16, 15: 1.11, 16: 1.03, 17: 0.99, 18: 0.96, 19: 1.96, 20: 1.71, 21: 1.48, 22: 1.36, 23: 1.34, 24: 1.22, 25: 1.19, 26: 1.16, 27: 1.11, 28: 1.1, 29: 1.12, 30: 1.18, 31: 1.24, 32: 1.21, 33: 1.21, 34: 1.16, 35: 1.14, 36: 1.17, 37: 2.1, 38: 1.85, 39: 1.63, 40: 1.54, 41: 1.47, 42: 1.38, 43: 1.28, 44: 1.25, 45: 1.25, 46: 1.2, 47: 1.28, 48: 1.36, 49: 1.42, 50: 1.4, 51: 1.4, 52: 1.36, 53: 1.33, 54: 1.31, 55: 2.32, 56: 1.96, 57: 1.8, 58: 1.63, 59: 1.76, 60: 1.74, 61: 1.73, 62: 1.72, 63: 1.68, 64: 1.69, 65: 1.68, 66: 1.67, 67: 1.66, 68: 1.65, 69: 1.64, 70: 1.7, 71: 1.62, 72: 1.52, 73: 1.46, 74: 1.37, 75: 1.31, 76: 1.29, 77: 1.22, 78: 1.23, 79: 1.24, 80: 1.33, 81: 1.44, 82: 1.44, 83: 1.51, 84: 1.45, 85: 1.47, 86: 1.42, 87: 2.23, 88: 2.01, 89: 1.86, 90: 1.75, 91: 1.69, 92: 1.7, 93: 1.71, 94: 1.72, 95: 1.66, 96: 1.66, 97: 1.68, 98: 1.68, 99: 1.65, 100: 1.67, 101: 1.73, 102: 1.76, 103: 1.61, 104: 1.57, 105: 1.49, 106: 1.43, 107: 1.41, 108: 1.34, 109: 1.29, 110: 1.28, 111: 1.21, 112: 1.22, 113: 1.36, 114: 1.43, 115: 1.62, 116: 1.75, 117: 1.65, 118: 1.57}

Atomic number -> covalent radius in Å (None when mendeleev has none).

active_learning_step.engine module

seamm-al engine: a committee’s mean as one MDI engine, with its spreads.

The committee mean is served to an MDI driver (LAMMPS, fix mdi/qm) as a single potential, adapted from code by Sina Mostafanejad: the mean of the members’ short-range cores is one InteratomicPotential (CommitteeMeanCore); the dispersion recorded in the checkpoints is added once on top (the same term in every member, so it is not averaged); and the force and stress heads differentiate the sum. The mean of the members’ energies has the mean of their forces as its gradient, so this is the committee’s prediction at the cost of one backward pass.

What is added here: every every evaluations (--spreads FILE --every N) the engine also evaluates each member on its own (one forward and backward pass per member, only on the steps written) and appends the committee spreads to a CSV: rho_energy, rho_forces, rho_forces_max_atom and, for periodic systems, rho_pressure and rho_pressure_molecular (molecules from the bond graph of the first frame written). Each row also carries max_dF, the largest difference between the engine’s forces and the mean of the members’ forces (zero to rounding when no dispersion is added; the check that the mean core is the members’ mean). With --flag NAME=VALUE a frame whose spread reaches a value is written, with its step and spreads and the per-atom force spread, to an extxyz (--flagged); --frames writes every frame recorded.

The members and the mean share their modules: each member is built once and the mean core holds the members’ cores.

xnn interface (public names): xnn.common.models.InteratomicPotential, ForceStressOutput, add_dispersion; xnn.common.models.hub.load_checkpoint, build_potential; xnn.common.models.registry.resolve_dispersion; xnn.common.models.d4.enable_eeq_reuse; xnn.common.deploy.mdi_engine.MDIEngine, BOHR_TO_ANGSTROM, HARTREE_TO_EV; xnn.common.data.structure_to_graph, collate.

active_learning_step.engine.CommitteeEngine(*args, **kwargs)

Build the committee MDI engine (see _engine_class()).

class active_learning_step.engine.CommitteeMeanCore(*args: Any, **kwargs: Any)

Bases: InteratomicPotential

The mean energy of several short-range cores, differentiable.

Parameters:

cores (sequence of torch.nn.Module) – The members’ cores (the model inside each ForceStressOutput), returning node_energy and energy; one cutoff for all.

members

The cores.

Type:

torch.nn.ModuleList

cutoff

Their common neighbour-list cutoff.

Type:

float

node_feature_dim

That of the first core (the long-range and dispersion wrappers read it; the mean core exposes no features of its own).

Type:

int

Raises:

ValueError – With fewer than two cores or differing cutoffs.

forward(data: xnn.common.data.AtomicGraph) → Dict[str, torch.Tensor]

Mean node_energy and energy over the members.

Every member sees a shallow copy of the graph, as in Committee. The result stays attached to the graph’s positions and cell, so a wrapping ForceStressOutput differentiates it.

property size: int

Number of members.

active_learning_step.engine.build_committee(paths, device=None, dtype=None)

The members, each built once with forces and stress and no dispersion.

Returns:

The members and their subtracted_dispersion records.

Return type:

(list of ForceStressOutput, list)

Raises:

ValueError – When the records differ.

active_learning_step.engine.committee_mean_potential(paths=None, device=None, dtype=None, dispersion=None, eeq_reuse: bool = False, compute_stress: bool = True, members=None, records=None) → xnn.common.models.ForceStressOutput

Build the committee mean as one potential with forces and stress.

Parameters:

• paths (iterable of str, optional) – The members’ checkpoints; or give members (and records) from build_committee(), whose cores the mean then shares.

• device (torch.device or str, optional) – Where to serve.

• dtype (torch.dtype or str, optional) – Dtype to serve in; by default that of the first member’s weights.

• dispersion (dict, str, bool or None, optional) – The term added once on top of the mean, resolved against the checkpoints’ record as resolve_dispersion() does: None adds the recorded term, False none (the energy the labels had).

• eeq_reuse (bool, optional) – Carry the D4 EEQ solve between steps (molecular dynamics).

• compute_stress (bool, optional) – Build the stress head, by default True.

• members (optional) – Prebuilt members and their dispersion records.

• records (optional) – Prebuilt members and their dispersion records.

Returns:

ForceStressOutput(dispersion(CommitteeMeanCore(cores))) in eval mode; its cutoff is the dispersion term’s when one is added.

Return type:

ForceStressOutput

active_learning_step.engine.main(argv: Sequence[str] | None = None) → int

Serve a committee mean to an MDI driver, recording its spreads.

active_learning_step.lammps_data module

LAMMPS data files for the committee engine: what atom_style atomic can read.

An MLFF engine run with fix mdi/qm uses atom_style atomic, which reads only the Atoms, Velocities, Masses and pair-coefficient sections. A data file written from a molecular style (or by tools that add their own sections, such as a Molecules section) stops LAMMPS at read_data with “Unknown identifier in data file”, after the job has waited in the queue. check_data() finds such sections before a job is submitted and strip_data() writes a copy without them.

A section starts at a line whose first word is not a number; its lines run to the next such line. Header lines (1536 atoms, 2 atom types, box bounds) start with a number. Only the standard library is needed.

active_learning_step.lammps_data.ATOMIC_SECTIONS = ('Atoms', 'Velocities', 'Masses', 'Pair Coeffs', 'PairIJ Coeffs')

Sections atom_style atomic reads.

active_learning_step.lammps_data.MOLECULAR_COUNTS = ('bonds', 'angles', 'dihedrals', 'impropers', 'bond types', 'angle types', 'dihedral types', 'improper types', 'extra bond per atom', 'extra angle per atom', 'extra dihedral per atom', 'extra improper per atom', 'extra special per atom')

Header counts that need a molecular atom style when they are not zero.

active_learning_step.lammps_data.check_data(path)

What in a data file atom_style atomic cannot read.

Parameters:

path (str or pathlib.Path) – The LAMMPS data file.

Returns:

The section names found, and the problems (empty when the file is fine).

Return type:

(list of str, list of str)

active_learning_step.lammps_data.sections(lines)

The sections of a data file: (name, first line index, end index), in order.

The first line (the title) is never a section.

active_learning_step.lammps_data.strip_data(path, out)

Write a copy of a data file that atom_style atomic reads.

Drops every section ATOMIC_SECTIONS does not list and the header counts of MOLECULAR_COUNTS; everything else is copied unchanged.

Returns:

What was removed.

Return type:

list of str

active_learning_step.launch module

Launching the committee-mean MDI engine from SEAMM.

A committee is offered as one model chemistry by the xnn plug-in, which runs it with the command built here: seamm-al engine (active_learning_step.engine) in the MLFF environment, serving the mean of the members and writing their spreads.

The engine runs in the MLFF environment, which has torch and xnn but need not have this package. A copy of the package’s modules, made once per version of the code in ~/.seamm.d/active_learning/code/, is put on that environment’s PYTHONPATH by the command itself, so nothing is installed there.

active_learning_step.launch.SPREADS_FILE = 'committee_spreads.csv'

Where the engine writes the spreads, in its working directory

active_learning_step.launch.code_directory(cache=None)

A directory holding a copy of this package, for PYTHONPATH.

Parameters:

cache (str or pathlib.Path, optional) – Where the copies are kept, by default ~/.seamm.d/active_learning/code.

Returns:

The directory that holds active_learning_step/. It is named after the hash of the sources, so a new version of the code gets a new copy and a running engine’s copy is never changed.

Return type:

pathlib.Path

active_learning_step.launch.engine_command(python, members, mdi, *, device='cpu', charge=0, extra_args=(), spreads='committee_spreads.csv', every=10, cache=None)

The argv of the committee-mean MDI engine.

Parameters:

• python (str) – The MLFF environment’s Python (python when the environment is active).

• members ([str or pathlib.Path]) – The members’ checkpoints, or their directories.

• mdi (str) – The -mdi options, e.g. "-role ENGINE -name XNN -method MPI".

• device (str) – The torch device.

• charge (float) – The total charge, for models whose dispersion needs the EEQ charges.

• extra_args ([str]) – More engine options, e.g. ["--dtype", "float32", "--eeq-reuse"].

• spreads (str or None) – The CSV the spreads are written to, every every steps; None for none.

• every (int) – How often to write the spreads, in calls of the engine (MD steps).

• cache (str or pathlib.Path, optional) – Where the copy of the package is kept (see code_directory()).

Returns:

Starting with env PYTHONPATH=..., so that it works as a command, under conda run and in an mpirun MPMD line alike.

Return type:

[str]

active_learning_step.metadata module

This file contains metadata describing the results from ActiveLearning

active_learning_step.metadata.metadata = {'results': {'committee': {'calculation': ['train'], 'description': "The committee's directory", 'dimensionality': 'scalar', 'type': 'string'}, 'cost': {'calculation': ['select'], 'description': 'The core-hours to label the selected structures', 'dimensionality': 'scalar', 'type': 'float'}, 'flagged_members': {'calculation': ['check'], 'description': 'The members flagged as worse than the others', 'dimensionality': [-1], 'type': 'integer'}, 'max_rho_forces': {'calculation': ['score'], 'description': 'The largest force spread', 'dimensionality': 'scalar', 'type': 'float', 'units': 'eV/Å'}, 'median_rho_forces': {'calculation': ['score'], 'description': 'The median force spread', 'dimensionality': 'scalar', 'type': 'float', 'units': 'eV/Å'}, 'n_frames': {'calculation': ['train', 'score'], 'description': 'The number of structures', 'dimensionality': 'scalar', 'type': 'integer'}, 'n_members': {'calculation': ['train', 'check'], 'description': 'The number of members', 'dimensionality': 'scalar', 'type': 'integer'}, 'n_selected': {'calculation': ['select'], 'description': 'The number of structures selected', 'dimensionality': 'scalar', 'type': 'integer'}, 'scores': {'calculation': ['score'], 'description': 'The scores file', 'dimensionality': 'scalar', 'type': 'string'}, 'selected_file': {'calculation': ['select'], 'description': 'The file of the selected structures', 'dimensionality': 'scalar', 'type': 'string'}, 'spearman': {'calculation': ['check'], 'description': 'Rank correlations of the spreads with the errors (JSON)', 'dimensionality': 'scalar', 'type': 'string'}, 'thresholds': {'calculation': ['check'], 'description': 'The calibrated thresholds of the spreads (JSON)', 'dimensionality': 'scalar', 'type': 'string'}, 'val_loss': {'calculation': ['train'], 'description': "The members' best validation losses", 'dimensionality': [-1], 'type': 'float'}}}

Description of the computational models for ActiveLearning.

Hamiltonians, approximations, and basis set or parameterizations, only if appropriate for this code. For example:

metadata["computational models"] = {
    "Hartree-Fock": {
        "models": {
            "PM7": {
                "parameterizations": {
                    "PM7": {
                        "elements": "1-60,62-83",
                        "periodic": True,
                        "reactions": True,
                        "optimization": True,
                        "code": "mopac",
                    },
                    "PM7-TS": {
                        "elements": "1-60,62-83",
                        "periodic": True,
                        "reactions": True,
                        "optimization": False,
                        "code": "mopac",
                    },
                },
            },
        },
    },
}

active_learning_step.molecules module

Molecules of a structure from its covalent bonds, with molsystem’s rule.

The committee’s molecular pressure removes the intramolecular virial, so it needs to know which atoms form each molecule. SEAMM defines molecules through bonds: molsystem’s Configuration.perceive_bonds() and find_molecules(). This module applies the same rule to coordinates alone, so that the command line runs in an MLFF environment that has neither SEAMM nor molsystem, and gives the same molecules:

• two atoms are bonded when min_distance < d < tolerance * (r_i + r_j), with the Pyykkö covalent radii molsystem uses (covalent_radii) and tolerance 1.2, using every periodic image (ASE’s neighbour list);

• the alkali and alkaline-earth metals (IONIC_ELEMENTS) are never bonded, so each cation is a molecule of its own; halide anions come out as single atoms;

• an atom has at most max_bonds bonds by element (H: 1), the shortest kept;

• the molecules are the connected components, numbered in the order of their first atom, so O H H O H H ... water gives index // 3.

Inside SEAMM, molecules_of_configuration() uses a configuration’s own bonds when it has them. The test suite checks this module against molsystem wherever molsystem is installed, so the two cannot drift apart.

Only ASE and numpy are needed.

active_learning_step.molecules.BOND_TOLERANCE = 1.2

Bonds are distances below this multiple of the sum of the covalent radii.

active_learning_step.molecules.IONIC_ELEMENTS = ('Li', 'Na', 'K', 'Rb', 'Cs', 'Fr', 'Be', 'Mg', 'Ca', 'Sr', 'Ba', 'Ra')

the alkali and alkaline-earth metals, ions in molecular systems (as molsystem.topology.IONIC_ELEMENTS).

Type:

Elements never bonded

active_learning_step.molecules.MAX_BONDS = {'H': 1}

The most bonds an atom of an element may have (as molsystem.topology.MAX_BONDS).

active_learning_step.molecules.MIN_DISTANCE = 0.4

Pairs closer than this (Å) are ignored rather than bonded.

active_learning_step.molecules.as_atoms(structure)

An ASE Atoms from Atoms or an xnn structure dict.

Parameters:

structure (ase.Atoms or dict) – The structure; a dict needs pos and atomic_numbers and may have cell and pbc.

Return type:

ase.Atoms

active_learning_step.molecules.masses(structure)

The atomic masses (amu) of a structure, from ASE’s table.

Parameters:

structure (ase.Atoms or dict) – The structure.

Returns:

(N,) masses.

Return type:

numpy.ndarray

active_learning_step.molecules.molecules(structure, **rule)

The molecule index of every atom: connected components of the bonds.

Parameters:

• structure (ase.Atoms or dict) – The structure (an xnn structure dict is accepted too). Periodic directions use every image.

• **rule – tolerance, radii, max_bonds, exclude, min_distance, as in bonds() (molsystem’s perceive_bonds).

Returns:

(N,) molecule index per atom, numbered 0, 1, … in the order of each molecule’s first atom.

Return type:

numpy.ndarray of int

active_learning_step.molecules.molecules_of_configuration(configuration, **rule)

The molecule index of every atom of a SEAMM (molsystem) configuration.

Uses the configuration’s own bonds when it has any (find_molecules), so a structure keeps the molecules SEAMM gave it. Otherwise the bonds are perceived with molsystem’s rule (bonds()), without adding them to the configuration.

Parameters:

• configuration (molsystem configuration) – The structure.

• **rule – Options of bonds(), used only when the configuration has no bonds.

Returns:

(N,) molecule index per atom, in the order of each molecule’s first atom.

Return type:

numpy.ndarray of int

active_learning_step.pool module

Candidate pools: the frames of extxyz and SDF files, as records and as structures.

A pool file is split into records, the exact bytes of each frame, so that a selection can be written as a verbatim subset of the input. Rewriting frames through ASE would drop keys it does not understand (for example the DDEC6 charges and the SEAMM header keys), and the labelling flowcharts read these files directly.

• extxyz: a record is the atom-count line, the comment line and that many atom lines.

• SDF: a record runs through its $$$$ line; everything is kept, including the blank line that ends each property.

• Either may be gzipped (.extxyz.gz, .sdf.gz); the records are the uncompressed bytes.

Each record is also turned into an ase.Atoms with an info dictionary holding the configuration name and the total charge:

• extxyz frames are parsed by ASE, one record at a time, so the frame numbering always matches the records;

• SDF records written by SEAMM carry the full-precision coordinates (SEAMM|XYZ|json|), the cell (SEAMM|cell|json|, as a, b, c, alpha, beta, gamma) and the net charge (SEAMM|net charge|int|); other SDF files fall back to the V2000 atom block and the M  CHG lines.

Nothing here needs torch or xnn.

active_learning_step.pool.configuration_name(atoms)

The configuration name of a frame, or an empty string.

SEAMM/configuration_name in the extxyz info (or the SDF record’s name).

active_learning_step.pool.iter_frames(path)

The frames of a pool file.

Parameters:

path (str or pathlib.Path) – An extxyz or SDF file.

Yields:

(int, ase.Atoms) – The index of the frame in the file and the structure.

active_learning_step.pool.iter_records(path)

The records of a pool file, in order, as exact bytes.

Parameters:

path (str or pathlib.Path) – An extxyz or SDF file.

Yields:

bytes – One record per frame.

active_learning_step.pool.pool_format(path)

The format of a pool file from its suffix: "extxyz" or "sdf".

A .gz suffix (trajectory.extxyz.gz) is ignored: such files are read through gzip.

Parameters:

path (str or pathlib.Path) – The file.

Return type:

str

Raises:

ValueError – For any other suffix.

active_learning_step.pool.record_to_atoms(record, fmt)

Parse one record into an ase.Atoms.

Parameters:

• record (bytes) – The record, from iter_records().

• fmt (str) – "extxyz" or "sdf".

Returns:

With info from the file. SDF records also get total_charge, SEAMM/configuration_name and every data item.

Return type:

ase.Atoms

active_learning_step.pool.total_charge(atoms)

The total charge of a frame: total_charge (or charge) in its info.

Returns:

None when the frame records no charge (the model then assumes neutral).

Return type:

float or None

active_learning_step.samplers module

Batch samplers for training committee members: one kind per batch, a set cell share.

Adapted from code by Sina Mostafanejad, with the kinds named as in scores.csv (cell and cluster).

• KindBatchSampler never mixes clusters and periodic cells in a batch, and gives each kind its own batch size: a cell holds about 20 times the atoms and neighbours of a cluster, so one size cannot suit both.

• ShareBatchSampler keeps one pass over the clusters per epoch and draws the cell batches so that they make a fixed share of the epoch’s batches, cycling through the cells in a fresh order. Without it, the share of optimiser steps spent on cells is set by the selection, and a selection of many clusters and few cells hardly trains on cells.

Both are DDP-aware (every rank builds the same global batches from the same seed and takes its own slice of each) and reshuffle every epoch under any caller: the trainer tells them the epoch with set_epoch (xnn 0.12 does in every launch mode), and they also advance their own epoch after each pass, giving the order set_epoch would. So a sampler passed to an xnn 0.11 Trainer, which calls set_epoch only in distributed runs, still reshuffles.

Only torch is needed.

class active_learning_step.samplers.KindBatchSampler(kinds, batch_sizes, shuffle, seed=0, rank=0, world=1)

Bases: object

Batches of one kind each, shuffled per epoch, with a batch size per kind.

Parameters:

• kinds (sequence of str) – The kind of every dataset index.

• batch_sizes (dict of str to int) – The global batch size of each kind.

• shuffle (bool) – Shuffle within kinds and the order of the batches (training); a validation sampler gives the same batches every pass.

• seed (int, optional) – The seed; epoch e uses seed + 1000003 * e.

• rank (int, optional) – This rank and the number of ranks of a distributed run.

• world (int, optional) – This rank and the number of ranks of a distributed run.

set_epoch(epoch)

Set the epoch whose order the next pass gives.

class active_learning_step.samplers.ShareBatchSampler(kinds, batch_sizes, shuffle, seed=0, *, share, cell_kind='cell')

Bases: KindBatchSampler

Like KindBatchSampler, with cell batches a fixed share of every epoch.

The cluster batches of an epoch are one pass over the clusters; the cell batches are drawn from the cells in a fresh random order, repeated or left out across epoch boundaries as the share requires, so that every cell recurs as evenly as possible.

Parameters:

• kinds – As in KindBatchSampler (single process only).

• batch_sizes – As in KindBatchSampler (single process only).

• shuffle – As in KindBatchSampler (single process only).

• seed – As in KindBatchSampler (single process only).

• share (float) – The share of an epoch’s batches that are cell batches, in (0, 1).

• cell_kind (str, optional) – The kind that is cells, by default "cell".

active_learning_step.samplers.kinds_of(structures)

"cell" for periodic structures, "cluster" for the others.

Parameters:

structures (sequence of dict) – xnn structure dicts.

Return type:

list of str

active_learning_step.score module

seamm-al score: the committee’s spreads for every frame of a candidate pool.

Each frame of the pool files (extxyz, or SDF from SEAMM) becomes one row of scores.csv:

• where it is: file, index (0-based, in the file), configuration;

• what it is: kind (cell if periodic, else cluster), n_atoms, formula, total_charge, n_molecules (cells);

• the spreads: rho_energy (eV), rho_forces and rho_forces_max_atom (eV/A), and for cells rho_pressure and rho_pressure_molecular (atm);

• the committee average: energy_mean (eV), and for cells pressure_mean and pressure_molecular_mean (atm);

• any info keys asked for with carry.

Clusters are scored in batches; cells one at a time, with their molecules from the bond graph (molecules()). Frames are read and written in order, so memory stays small however large the pool.

active_learning_step.score.COLUMNS = ('file', 'index', 'configuration', 'kind', 'n_atoms', 'formula', 'total_charge', 'n_molecules', 'rho_energy', 'rho_forces', 'rho_forces_max_atom', 'rho_pressure', 'rho_pressure_molecular', 'energy_mean', 'pressure_mean', 'pressure_molecular_mean')

The columns of scores.csv, before any carried keys.

active_learning_step.score.score_pool(paths, models, out_csv='scores.csv', atoms_out=None, carry=(), dispersion=False, device='cpu', dtype=None, batch_size=16, all_models=None, dropped=(), threads=None)

Score the frames of pool files with a committee and write scores.csv.

Parameters:

• paths (sequence of str) – The pool files (.extxyz or .sdf).

• models (sequence of str or None) – The members’ checkpoints. None (seamm-al score --no-models) writes only the identity and carried columns, without torch or xnn: a pool listing for a random draw (seamm-al select --random) before any committee exists.

• out_csv (str, optional) – The scores file.

• atoms_out (str, optional) – If given, an extxyz file of every frame with its per-atom force spread (rho_forces_atom, eV/A) as a per-atom array, the molecule index of cells (molecule), and the frame’s scores in its info.

• carry (sequence of str, optional) – info keys copied into columns of the same name.

• dispersion (bool, optional) – Add the dispersion the checkpoints record as subtracted, by default False: it is shared by every member, so it adds nothing to the spreads, and the members are then served as they were trained.

• device (str, optional) – Torch device, by default "cpu".

• dtype (str, optional) – "float32" or "float64"; by default the weights’ own.

• batch_size (int, optional) – Clusters per batch, by default 16. Cells are scored one at a time.

• threads (int, optional) – CPU threads for torch (set_threads()).

• all_models (optional) – The committee before any members were dropped, and the positions dropped; recorded with the models used in <out_csv>.json.

• dropped (optional) – The committee before any members were dropped, and the positions dropped; recorded with the models used in <out_csv>.json.

Returns:

The number of frames scored.

Return type:

int

active_learning_step.score_pool module

Non-graphical part of the Score Pool step in an Active Learning flowchart

active_learning_step.score_pool.POOL_KEYS = ('system_id', 'configuration_id')

The keys that tie a scored frame to its configuration in the database

active_learning_step.score_pool.SPREADS = (('rho_energy', 'eV'), ('rho_forces', 'eV/Å'), ('rho_forces_max_atom', 'eV/Å'), ('rho_pressure', 'atm'), ('rho_pressure_molecular', 'atm'))

The spreads summarised, and their units

class active_learning_step.score_pool.ScorePool(flowchart=None, title='Score Pool', extension=None, logger=<Logger active_learning_step.score_pool (WARNING)>)

Bases: Substep

The non-graphical part of a Score Pool step in a flowchart.

parser

The parser object.

Type:

configargparse.ArgParser

options

It contains a two item tuple containing the populated namespace and the list of remaining argument strings.

Type:

tuple

subflowchart

A SEAMM Flowchart object that represents a subflowchart, if needed.

Type:

seamm.Flowchart

parameters

The control parameters for Score Pool.

Type:

ScorePoolParameters

See also

TkScorePool, ScorePool, ScorePoolParameters

analyze(indent='', scores=None, P=None, **kwargs)

Print a summary of the scores and store the results.

Parameters:

• indent (str) – An extra indentation for the output

• scores (pathlib.Path) – The scores file.

• P (dict) – The parameters of the step.

description_text(P=None)

Create the text description of what this step will do.

Parameters:

P (dict) – An optional dictionary of the current values of the control parameters.

Returns:

A description of the current step.

Return type:

str

property git_revision

The git version of this module.

property header

A printable header for this section of output

run()

Score the candidates.

Returns:

The next node object in the flowchart.

Return type:

seamm.Node

property version

The semantic version of this module.

active_learning_step.score_pool.write_pool(configurations, path)

Write SEAMM configurations to an extxyz file for scoring.

Each frame carries the configuration’s name, its total charge, and the ids of its system and configuration, so that the selected frames can be found again in the database.

Parameters:

• configurations ([molsystem._Configuration]) – The structures.

• path (pathlib.Path) – The extxyz file to write.

active_learning_step.score_pool_parameters module

Control parameters for the Score Pool step in a SEAMM flowchart

class active_learning_step.score_pool_parameters.ScorePoolParameters(defaults={}, data=None)

Bases: Parameters

The control parameters for Score Pool.

You need to replace the “time” entry in dictionary below these comments with the definitions of parameters to control this step. The keys are parameters for the current plugin,the values are dictionaries as outlined below.

Examples

parameters = {
    "time": {
        "default": 100.0,
        "kind": "float",
        "default_units": "ps",
        "enumeration": tuple(),
        "format_string": ".1f",
        "description": "Simulation time:",
        "help_text": ("The time to simulate in the dynamics run.")
    },
}

parameters{str: {str: str}}

A dictionary containing the parameters for the current step. Each key of the dictionary is a dictionary that contains the the following keys:

parameters[“default”] :

The default value of the parameter, used to reset it.

parameters[“kind”]enum()

Specifies the kind of a variable. One of “integer”, “float”, “string”, “boolean”, or “enum”

While the “kind” of a variable might be a numeric value, it may still have enumerated custom values meaningful to the user. For instance, if the parameter is a convergence criterion for an optimizer, custom values like “normal”, “precise”, etc, might be adequate. In addition, any parameter can be set to a variable of expression, indicated by having “$” as the first character in the field. For example, $OPTIMIZER_CONV.

parameters[“default_units”]str

The default units, used for resetting the value.

parameters[“enumeration”]: tuple

A tuple of enumerated values.

parameters[“format_string”]: str

A format string for “pretty” output.

parameters[“description”]: str

A short string used as a prompt in the GUI.

parameters[“help_text”]: str

A longer string to display as help for the user.

See also

ScorePool, TkScorePool, ScorePoolParameters, ScorePoolStep

parameters = {'add dispersion': {'default': 'no', 'default_units': '', 'description': 'Add the dispersion:', 'enumeration': ('yes', 'no'), 'format_string': '', 'help_text': 'For models trained on labels with the dispersion removed, whether to add it back. It is the same for every member, so it does not change the spreads.', 'kind': 'boolean'}, 'batch size': {'default': 16, 'default_units': '', 'description': 'Clusters per batch:', 'enumeration': (), 'format_string': 'd', 'help_text': 'The number of clusters scored together. Periodic cells are scored one at a time.', 'kind': 'integer'}, 'committee': {'default': 'previous', 'default_units': '', 'description': 'Committee:', 'enumeration': ('previous', 'none'), 'format_string': '', 'help_text': "The committee: 'previous' for the one trained or used earlier in this job, or the directory of a committee made by Train Committee (it holds the members m0, m1, ...). 'none' only lists the candidates, for a random draw before there is a committee.", 'kind': 'string'}, 'device': {'default': 'default', 'default_units': '', 'description': 'Device:', 'enumeration': ('default', 'cpu', 'cuda', 'mps'), 'format_string': '', 'help_text': "The PyTorch device to run the models on: cpu, cuda (or cuda:N), or mps. 'default' uses the device in xnn.ini, else the CPU.", 'kind': 'string'}, 'keep keys': {'default': '', 'default_units': '', 'description': 'Keep info keys:', 'enumeration': (), 'format_string': '', 'help_text': "Keys of the candidates' info to copy into the scores, separated by spaces, for example to set costs or to balance a selection by them.", 'kind': 'string'}, 'leave out members': {'default': '', 'default_units': '', 'description': 'Leave out members:', 'enumeration': (), 'format_string': '', 'help_text': 'Members to leave out of the committee, by number (0, 1, ...), separated by spaces: for example, ones that Check Committee flagged.', 'kind': 'string'}, 'per-atom spreads': {'default': 'no', 'default_units': '', 'description': 'Write per-atom spreads:', 'enumeration': ('yes', 'no'), 'format_string': '', 'help_text': 'Also write every candidate with the spread of the force on each atom (atoms.extxyz), to see where the committee disagrees.', 'kind': 'boolean'}, 'pool': {'default': 'structures in the database', 'default_units': '', 'description': 'Candidates:', 'enumeration': ('structures in the database', 'files'), 'format_string': '', 'help_text': "The candidate structures to score: structures in this job's database, or extended XYZ or SDF files, for example a trajectory.", 'kind': 'string'}, 'pool files': {'default': '', 'default_units': '', 'description': 'Files:', 'enumeration': (), 'format_string': '', 'help_text': "Extended XYZ (.extxyz, .xyz, optionally gzipped) or SDF files of candidates: names or patterns separated by spaces, relative to the job's directory.", 'kind': 'string'}, 'precision': {'default': "the model's", 'default_units': '', 'description': 'Precision:', 'enumeration': ("the model's", 'float32', 'float64'), 'format_string': '', 'help_text': 'The floating-point precision to evaluate the models in.', 'kind': 'string'}, 'results': {'default': {}, 'default_units': '', 'description': 'results', 'enumeration': (), 'format_string': '', 'help_text': 'The results to save to variables or in tables.', 'kind': 'dictionary'}, 'source configuration name': {'default': '', 'default_units': '', 'description': '', 'enumeration': (), 'format_string': 's', 'help_text': 'The configuration name, wildcard pattern or regular expression.', 'kind': 'string'}, 'source configurations': {'default': 'current', 'default_units': '', 'description': 'Configurations:', 'enumeration': ('current', 'all', 'last', 'first', 'name is', 'name matches', 'name regexp'), 'format_string': 's', 'help_text': 'Which configurations of each selected system to use: its current configuration, all of them, the last or first, or those whose name is / matches (shell wildcards) / matches the regular expression given.', 'kind': 'string'}, 'source system name': {'default': '', 'default_units': '', 'description': '', 'enumeration': (), 'format_string': 's', 'help_text': 'The system name, wildcard pattern or regular expression.', 'kind': 'string'}, 'source systems': {'default': 'current', 'default_units': '', 'description': 'Systems:', 'enumeration': ('current', 'all', 'name is', 'name matches', 'name regexp'), 'format_string': 's', 'help_text': 'Which systems to take the structures from: the current system, all systems, or those whose name is / matches (shell wildcards) / matches the regular expression given. A variable ($name) holding a list of configurations or of systems may also be given.', 'kind': 'string'}}

active_learning_step.score_pool_step module

class active_learning_step.score_pool_step.ScorePoolStep(flowchart=None, gui=None)

Bases: object

Helper class needed for the stevedore integration.

This must provide a description() method that returns a dict containing a description of this node, and create_node() and create_tk_node() methods for creating the graphical and non-graphical nodes.

The dictionary for the description is the class variable just below these comments. The felds are as follows:

my_description{str, str}

A human-readable description of this step. It can be several lines long, and needs to be clear to non-expert users. It contains the following keys: description, group, name.

my_description[“description”]tuple

A description of the Score Pool step. It must be clear to non-experts.

my_description[“group”]str

Which group in the menus to put this step. If the group does not exist it will be created. Common groups are “Building”, “Control”, “Custom”, “Data”, and “Simulations”.

my_description[“name”]str

The name of this step, to be displayed in the menus.

create_node(flowchart=None, **kwargs)

Create and return the new node object.

Parameters:

• flowchart (seamm.Node) – A non-graphical SEAMM node

• **kwargs (keyword arguments) – Various keyword arguments such as title, namespace or extension representing the title displayed in the flowchart, the namespace for the plugins of a subflowchart and the extension, respectively.

Return type:

ScorePool

create_tk_node(canvas=None, **kwargs)

Create and return the graphical Tk node object.

Parameters:

• canvas (tk.Canvas) – The Tk Canvas widget

• **kwargs (keyword arguments) – Various keyword arguments such as tk_flowchart, node, x, y, w, h representing a graphical flowchart object, a non-graphical node for a step, and dimensions of the graphical node.

Return type:

TkScorePool

description()

Return a description of what this step does.

Returns:

description

Return type:

dict(str, str)

my_description = {'description': 'Score candidate structures by how much the committee disagrees on them', 'group': 'Calculations', 'name': 'Score Pool'}

active_learning_step.select_frames module

Non-graphical part of the Select Frames step in an Active Learning flowchart

class active_learning_step.select_frames.SelectFrames(flowchart=None, title='Select Frames', extension=None, logger=<Logger active_learning_step.select_frames (WARNING)>)

Bases: Substep

The non-graphical part of a Select Frames step in a flowchart.

parser

The parser object.

Type:

configargparse.ArgParser

options

It contains a two item tuple containing the populated namespace and the list of remaining argument strings.

Type:

tuple

subflowchart

A SEAMM Flowchart object that represents a subflowchart, if needed.

Type:

seamm.Flowchart

parameters

The control parameters for Select Frames.

Type:

SelectFramesParameters

See also

TkSelectFrames, SelectFrames, SelectFramesParameters

analyze(indent='', selected=None, files=(), cost=0.0, P=None, **kwargs)

Store the results.

Parameters:

• indent (str) – An extra indentation for the output

• selected ([dict]) – The selected rows.

• files ([pathlib.Path]) – The files of the selected structures.

• cost (float) – Their cost, in core-hours.

• P (dict) – The parameters of the step.

database(selected, P)

Put configurations selected from the database in a variable or a system.

Only for candidates that came from the database, whose scores carry the configurations’ ids.

description_text(P=None)

Create the text description of what this step will do.

Parameters:

P (dict) – An optional dictionary of the current values of the control parameters.

Returns:

A description of the current step.

Return type:

str

property git_revision

The git version of this module.

property header

A printable header for this section of output

run()

Select the structures to label.

Returns:

The next node object in the flowchart.

Return type:

seamm.Node

property version

The semantic version of this module.

active_learning_step.select_frames.copy_to_new_system(system_db, configurations, name)

Copy configurations into a new system.

Parameters:

• system_db (molsystem.SystemDB) – The database.

• configurations ([molsystem._Configuration]) – The configurations to copy.

• name (str) – The name of the new system.

Returns:

The new system.

Return type:

molsystem._System

active_learning_step.select_frames_parameters module

Control parameters for the Select Frames step in a SEAMM flowchart

class active_learning_step.select_frames_parameters.SelectFramesParameters(defaults={}, data=None)

Bases: Parameters

The control parameters for Select Frames.

You need to replace the “time” entry in dictionary below these comments with the definitions of parameters to control this step. The keys are parameters for the current plugin,the values are dictionaries as outlined below.

Examples

parameters = {
    "time": {
        "default": 100.0,
        "kind": "float",
        "default_units": "ps",
        "enumeration": tuple(),
        "format_string": ".1f",
        "description": "Simulation time:",
        "help_text": ("The time to simulate in the dynamics run.")
    },
}

parameters{str: {str: str}}

A dictionary containing the parameters for the current step. Each key of the dictionary is a dictionary that contains the the following keys:

parameters[“default”] :

The default value of the parameter, used to reset it.

parameters[“kind”]enum()

Specifies the kind of a variable. One of “integer”, “float”, “string”, “boolean”, or “enum”

While the “kind” of a variable might be a numeric value, it may still have enumerated custom values meaningful to the user. For instance, if the parameter is a convergence criterion for an optimizer, custom values like “normal”, “precise”, etc, might be adequate. In addition, any parameter can be set to a variable of expression, indicated by having “$” as the first character in the field. For example, $OPTIMIZER_CONV.

parameters[“default_units”]str

The default units, used for resetting the value.

parameters[“enumeration”]: tuple

A tuple of enumerated values.

parameters[“format_string”]: str

A format string for “pretty” output.

parameters[“description”]: str

A short string used as a prompt in the GUI.

parameters[“help_text”]: str

A longer string to display as help for the user.

See also

SelectFrames, TkSelectFrames, SelectFramesParameters, SelectFramesStep

parameters = {'budget': {'default': 'unlimited', 'default_units': '', 'description': 'Budget (core-hours):', 'enumeration': ('unlimited',), 'format_string': '.1f', 'help_text': 'The core-hours available to label the selected structures.', 'kind': 'float'}, 'caps': {'default': '', 'default_units': '', 'description': 'At most per value:', 'enumeration': (), 'format_string': '', 'help_text': 'At most N structures for each value of a column, as COLUMN=N separated by spaces, e.g. source=3.', 'kind': 'string'}, 'combine thresholds': {'default': 'any', 'default_units': '', 'description': 'Thresholds to meet:', 'enumeration': ('any', 'all'), 'format_string': '', 'help_text': 'With several thresholds, whether any of them (as Smith et al.) or all of them must be met.', 'kind': 'string'}, 'costs': {'default': '', 'default_units': '', 'description': 'Cost per structure:', 'enumeration': (), 'format_string': '', 'help_text': 'The core-hours to label one structure, as KIND=COST or KIND:GROUP=COST separated by spaces, e.g. cluster=10 cell=2800. GROUP is a pattern matched against the group column. Without costs each structure counts one, and the budget is a number of structures.', 'kind': 'string'}, 'exclude': {'default': '', 'default_units': '', 'description': 'Already labelled:', 'enumeration': (), 'format_string': '', 'help_text': 'Files listing the names of structures already labelled, one per line, which are not selected again.', 'kind': 'string'}, 'fill the budget': {'default': 'no', 'default_units': '', 'description': 'Fill the budget:', 'enumeration': ('yes', 'no'), 'format_string': '', 'help_text': 'When a structure does not fit in what is left of the budget, skip it and keep going with cheaper ones, rather than stop.', 'kind': 'boolean'}, 'group column': {'default': '', 'default_units': '', 'description': 'Group column:', 'enumeration': (), 'format_string': '', 'help_text': 'The column of the scores that the cost groups match.', 'kind': 'string'}, 'maximum number': {'default': 'no limit', 'default_units': '', 'description': 'Most structures:', 'enumeration': ('no limit',), 'format_string': 'd', 'help_text': 'The most structures to select.', 'kind': 'integer'}, 'method': {'default': 'largest spreads', 'default_units': '', 'description': 'Select:', 'enumeration': ('largest spreads', 'random'), 'format_string': '', 'help_text': 'Select the candidates the committee disagrees on most, or draw them at random (the first round, before there is a committee).', 'kind': 'string'}, 'new system': {'default': '', 'default_units': '', 'description': 'Copy into a new system:', 'enumeration': (), 'format_string': '', 'help_text': 'For candidates from the database: also copy the selected configurations into a new system with this name. Leave it empty for no copy.', 'kind': 'string'}, 'random seed': {'default': 0, 'default_units': '', 'description': 'Random seed:', 'enumeration': (), 'format_string': 'd', 'help_text': 'The seed of the random draw.', 'kind': 'integer'}, 'ranking': {'default': 'default', 'default_units': '', 'description': 'Rank by:', 'enumeration': ('default',), 'format_string': '', 'help_text': 'The spreads that rank each kind of structure, as KIND=SPREAD[,SPREAD] separated by spaces. The default ranks cells by rho_pressure_molecular and rho_forces, and clusters by rho_forces; several spreads are combined by the sum of the ranks.', 'kind': 'string'}, 'results': {'default': {}, 'default_units': '', 'description': 'results', 'enumeration': (), 'format_string': '', 'help_text': 'The results to save to variables or in tables.', 'kind': 'dictionary'}, 'scores': {'default': 'previous', 'default_units': '', 'description': 'Scores:', 'enumeration': ('previous',), 'format_string': '', 'help_text': "The scores to select from: 'previous' for those of the last Score Pool in this job, or scores.csv files (names or patterns separated by spaces).", 'kind': 'string'}, 'selected file': {'default': 'selected', 'default_units': '', 'description': 'Selected structures file:', 'enumeration': (), 'format_string': '', 'help_text': "The name, without the extension, of the file of the selected structures, copied exactly from the candidates' files.", 'kind': 'string'}, 'spread the draw over': {'default': '', 'default_units': '', 'description': 'Spread the draw over:', 'enumeration': (), 'format_string': '', 'help_text': 'Columns of the scores to spread the random draw evenly over, separated by spaces, for example the composition or temperature.', 'kind': 'string'}, 'thresholds': {'default': 'none', 'default_units': '', 'description': 'Thresholds:', 'enumeration': ('none', 'from Check Committee'), 'format_string': '', 'help_text': 'Select only candidates whose spread reaches a threshold: none, those calibrated by the last Check Committee in this job, or NAME=VALUE or KIND:NAME=VALUE separated by spaces (e.g. cluster:rho_forces=0.1).', 'kind': 'string'}, 'variable': {'default': 'selected_configurations', 'default_units': '', 'description': 'Variable for the selection:', 'enumeration': (), 'format_string': '', 'help_text': 'For candidates from the database: the variable to hold the selected configurations, for later steps to use as their structures. Leave it empty for none.', 'kind': 'string'}}

active_learning_step.select_frames_step module

class active_learning_step.select_frames_step.SelectFramesStep(flowchart=None, gui=None)

Bases: object

Helper class needed for the stevedore integration.

This must provide a description() method that returns a dict containing a description of this node, and create_node() and create_tk_node() methods for creating the graphical and non-graphical nodes.

The dictionary for the description is the class variable just below these comments. The felds are as follows:

my_description{str, str}

A human-readable description of this step. It can be several lines long, and needs to be clear to non-expert users. It contains the following keys: description, group, name.

my_description[“description”]tuple

A description of the Select Frames step. It must be clear to non-experts.

my_description[“group”]str

Which group in the menus to put this step. If the group does not exist it will be created. Common groups are “Building”, “Control”, “Custom”, “Data”, and “Simulations”.

my_description[“name”]str

The name of this step, to be displayed in the menus.

create_node(flowchart=None, **kwargs)

Create and return the new node object.

Parameters:

• flowchart (seamm.Node) – A non-graphical SEAMM node

• **kwargs (keyword arguments) – Various keyword arguments such as title, namespace or extension representing the title displayed in the flowchart, the namespace for the plugins of a subflowchart and the extension, respectively.

Return type:

SelectFrames

create_tk_node(canvas=None, **kwargs)

Create and return the graphical Tk node object.

Parameters:

• canvas (tk.Canvas) – The Tk Canvas widget

• **kwargs (keyword arguments) – Various keyword arguments such as tk_flowchart, node, x, y, w, h representing a graphical flowchart object, a non-graphical node for a step, and dimensions of the graphical node.

Return type:

TkSelectFrames

description()

Return a description of what this step does.

Returns:

description

Return type:

dict(str, str)

my_description = {'description': 'Select the structures to label from the scores, within a budget', 'group': 'Calculations', 'name': 'Select Frames'}

active_learning_step.selection module

seamm-al select: choose frames from scores.csv within a core-hour budget.

The candidates are put in order and taken until the budget is spent:

• Ranking (the default). Within each kind, every frame is ranked on each of the kind’s ranking columns, largest spread first, and the ranks are summed. Cells use rho_pressure_molecular and rho_forces, clusters rho_forces; both can be changed (--rank cell=rho_forces_max_atom). Kinds are merged by the rank-sum divided by its largest possible value, so the best cell and the best cluster come first together.

• Thresholds (--threshold): only frames with a spread at least the threshold are candidates (the rule of Smith et al.; with several thresholds any one suffices, or all with --combine all). The candidates are then ordered by ranking as above.

Frames are taken in that order. --cap COLUMN=N allows at most N frames per value of a column; --exclude FILE drops configurations already labelled. A frame whose cost would take the total past --budget stops the selection, or with --fill is skipped while cheaper frames further down still fit.

Costs are core-hours per frame, by kind or by group: --cost cell=2800 or --cost cell:EC=2800 with --group-column naming the column that holds EC. Groups are shell-style patterns, and the first match wins.

The selected frames are written as a verbatim text subset of the input files: the records are copied byte for byte, in selection order, one output file per format.

active_learning_step.selection.DEFAULT_RANK = {'cell': ('rho_pressure_molecular', 'rho_forces'), 'cluster': ('rho_forces',)}

The default ranking columns per kind.

active_learning_step.selection.choose(rows, costs, budget=None, group_column=None, rank=None, thresholds=None, combine='any', caps=None, exclude=None, fill=False, max_frames=None, random_seed=None, stratify_columns=())

Choose frames from scored rows.

Parameters:

• rows (list of dict) – The rows of scores.csv.

• costs (list of (kind, group, cost)) – From parse_costs().

• budget (float, optional) – Core-hours available; no limit by default.

• group_column (str, optional) – The column the cost groups match.

• rank (dict, optional) – Ranking columns per kind, by default DEFAULT_RANK.

• thresholds (dict, optional) – From parse_thresholds(); turns on threshold mode.

• combine (str, optional) – "any" (the rule of Smith et al.) or "all" of the thresholds must be met.

• caps (dict, optional) – Most frames per value of a column.

• exclude (set of str, optional) – Configuration names already labelled.

• fill (bool, optional) – Skip a frame that does not fit the budget and go on, rather than stop.

• max_frames (int, optional) – Most frames to select.

• random_seed (int, optional) – Draw the frames in a random order (seeded) instead of ranking them: round 0, before any committee exists. Thresholds do not apply.

• stratify_columns (sequence of str, optional) – With random_seed: spread the draw evenly over the values of these columns, taking one frame from each stratum in turn (the strata in a shuffled order, the frames of each shuffled).

Returns:

The selected rows, in order, with order, cost, running_cost and reason; and the rejected rows with their reason.

Return type:

(list of dict, list of dict)

active_learning_step.selection.parse_caps(specs)

COLUMN=N -> {column: N}.

active_learning_step.selection.parse_costs(specs)

KIND=COREH or KIND:GROUP=COREH -> list of (kind, group, cost).

active_learning_step.selection.parse_rank(specs)

KIND=COL[,COL...] -> {kind: (cols)}, over the defaults.

active_learning_step.selection.parse_thresholds(specs)

NAME=VALUE or KIND:NAME=VALUE -> {kind or None: {name: value}}.

active_learning_step.selection.read_names(path)

Configuration names, one per line; blank lines and # comments ignored.

active_learning_step.selection.read_scores(path)

The rows of a scores.csv, as dicts of strings, in file order.

active_learning_step.selection.write_records(selected, prefix)

Copy the selected frames’ records, byte for byte, into prefix.<format>.

Parameters:

• selected (list of dict) – Rows with file and index, in the order to write.

• prefix (str) – Output path without suffix; .extxyz and/or .sdf are added.

Returns:

The file written per format.

Return type:

dict of str to str

Raises:

ValueError – If a file has fewer frames than a row’s index, or a record’s configuration name differs from the one scored (the pool file changed since scoring).

active_learning_step.substep module

Base class for substeps in the Active Learning step.

class active_learning_step.substep.Substep(flowchart=None, title='no title', extension=None, logger=<Logger active_learning_step.substep (WARNING)>, module=None)

Bases: Node

A base class for substeps in the Active Learning subflowchart.

The top-level ActiveLearning step iterates over the substeps of its subflowchart and calls each substep’s run() method. Each substep is responsible for preparing its own input, invoking Active Learning, analyzing its output, and storing its results.

This base class provides:

• is_runable - whether the top-level loop should call run() on this substep. Override to return False for substeps that only contribute input (e.g., an Initialization step) rather than invoking the code.

• input_only - a flag for “write input but don’t run” mode, useful for debugging and testing.

• options / global_options - delegated to the parent step so that command-line options reach substeps uniformly.

• header - a printable header for this section of step.out.

• version / git_revision - delegated to the plug-in package so that every substep reports the plug-in’s version, not seamm.Node’s.

committee_directory(P, key='committee')

The committee to use: the one trained or used earlier, or a directory.

Parameters:

• P (dict) – The step’s parameters.

• key (str) – The parameter naming the committee.

Return type:

pathlib.Path

device(P)

The torch device: the step’s, else the one in xnn.ini, else cpu.

property git_revision

The git revision of the active_learning_step package.

property global_options

The global SEAMM options, delegated to the parent step.

property header

A printable header for this section of output.

property input_only

Whether to write the input only, not run Active Learning.

property is_runable

Whether this substep should be run by the top-level loop.

Override to return False for substeps that only contribute input (e.g., an Initialization step) rather than invoking Active Learning themselves.

mlff_config()

How to run code in the MLFF environment: the xnn plug-in’s xnn.ini.

The committee needs torch and xnn, which live in the environment the xnn plug-in runs its models in, so this step uses the same one.

property options

The Active Learning options, delegated to the parent step.

resolve_files(patterns, what='file')

The files matching whitespace-separated names or glob patterns.

Relative names are taken from the job’s directory; job: references and absolute paths work as in other steps.

run_seamm_al(args)

Run seamm-al in the MLFF environment, in this step’s directory.

This package’s own modules are copied into _code/ and put on the PYTHONPATH, so the code that runs is the code of this SEAMM installation and nothing has to be installed in the MLFF environment. They need only numpy, ase, torch and xnn there; the step’s own classes, which need SEAMM, are imported only where SEAMM is.

Parameters:

args ([str]) – The arguments of seamm-al, e.g. ["score", "pool.extxyz"].

Returns:

The result; None when only writing the input. The output is in stdout.txt and stderr.txt, and everything seamm-al writes stays in the step’s directory.

Return type:

seamm_exec.TaskResult

property state

The active-learning results of this job so far, shared by the sub-steps.

committee is the last committee trained or used, scores the last scores files and pool how they were made ("database" or "file"). A later Active Learning step in the same job sees what an earlier one left, so a flowchart can train a committee, run molecular dynamics with it, then score and select.

property version

The semantic version of the active_learning_step package.

active_learning_step.threads module

How many CPU threads torch uses: no scheduler-specific code.

The thread count is, in order:

1. the one asked for (--threads);

2. OMP_NUM_THREADS, which SEAMM (seamm_exec) sets for the codes it runs from the job’s allocation, as do PBS and most launchers;

3. the CPUs this process may run on (its affinity mask), which Slurm and PBS cpusets, taskset and containers restrict; on a plain machine, all of them.

So it works the same under Slurm, PBS, SEAMM’s own JobServer or no scheduler at all, and never uses more cores than the process was given.

active_learning_step.threads.resolve_threads(threads=None, environ=None, affinity=None)

The number of threads to use, and where it came from.

Parameters:

• threads (int, optional) – The number asked for.

• environ (mapping, optional) – The environment, by default os.environ.

• affinity (int, optional) – The CPUs available, by default from the process’s affinity mask.

Returns:

The thread count and its source: "requested", "OMP_NUM_THREADS" or "affinity".

Return type:

(int, str)

active_learning_step.threads.set_threads(threads=None)

Set torch’s intra-op thread count by resolve_threads(); return it.

active_learning_step.thresholds module

The numpy-only parts of query by committee: data folds and thresholds.

Adapted from code by Sina Mostafanejad, unchanged except for being split off so that selection runs without torch or xnn. They are re-exported by active_learning_step.committee, which keeps his API.

active_learning_step.thresholds.calibrate_threshold(rho: ndarray, eps: ndarray, fail_level: float, capture: float = 0.98) → float

The inclusion threshold of Smith et al., fig 1.

The threshold is the largest value of rho such that at least capture of the structures whose error eps exceeds fail_level have rho >= threshold, so selecting by rho >= threshold catches that fraction of the failures.

Parameters:

• rho (ndarray) – Committee spread per structure (one entry of disagreement()).

• eps (ndarray) – Worst error per structure (the matching entry of worst_error()), on the same structures.

• fail_level (float) – Error above which a structure counts as a failure (the paper: 1.5 kcal/mol in its energy units).

• capture (float, optional) – Fraction of the failures the threshold must catch, by default 0.98.

Returns:

The threshold.

Return type:

float

Raises:

ValueError – If no structure fails, so there is nothing to calibrate on.

active_learning_step.thresholds.kfold_indices(n: int, k: int, seed: int = 0) → list[tuple[ndarray, ndarray]]

The k folds of a committee’s training data.

Member l trains on every fold but fold l and validates on fold l, the 5-fold split of Smith et al. (k = 5). One permutation seeded by seed is cut into k nearly equal folds, so the folds are disjoint and cover the n indices exactly once.

Parameters:

• n (int) – Number of training structures.

• k (int) – Number of folds, which is the number of committee members.

• seed (int, optional) – Seed of the permutation, by default 0. This is the data seed; the initialisation seed of each member is the trainer’s cfg.seed.

Returns:

For each member, (train_indices, held_indices) into the training set, sorted.

Return type:

list of (ndarray, ndarray)

Raises:

ValueError – If k is smaller than 2 or larger than n.

active_learning_step.thresholds.select(rhos: Dict[str, ndarray], thresholds: Dict[str, float]) → ndarray

Which structures any calibrated spread flags.

Parameters:

• rhos (dict of str to ndarray) – Spreads per structure by name (rho_energy, rho_forces, …).

• thresholds (dict of str to float) – Threshold per name; names absent from rhos are ignored, and a nan spread never selects.

Returns:

Boolean mask, true where rho >= threshold for at least one name.

Return type:

ndarray

active_learning_step.thresholds.split_statistics(rho: ndarray, err: ndarray, threshold: float) → Dict[str, float]

What a threshold separates: the error above and below it.

Parameters:

• rho (ndarray) – Committee spread per structure.

• err (ndarray) – An error per structure (the committee mean’s, or the worst member’s).

• threshold (float) – The threshold from calibrate_threshold().

Returns:

fraction_selected (rho >= threshold), n_above, n_below, and the RMS of err on each side (rmse_above, rmse_below; nan for an empty side).

Return type:

dict of str to float

active_learning_step.tk_active_learning module

The graphical part of an Active Learning step

class active_learning_step.tk_active_learning.TkActiveLearning(tk_flowchart=None, node=None, namespace='org.molssi.seamm.active_learning.tk', canvas=None, x=None, y=None, w=200, h=50)

Bases: TkNode

The graphical part of an Active Learning step in a flowchart.

tk_flowchart

The flowchart that we belong to.

Type:

TkFlowchart = None

node

The corresponding node of the non-graphical flowchart

Type:

Node = None

canvas

The Tk Canvas to draw on

Type:

tkCanvas = None

dialog

The Pmw dialog object

Type:

Dialog

x

The x-coordinate of the center of the picture of the node

Type:

int = None

y

The y-coordinate of the center of the picture of the node

Type:

int = None

w

The width in pixels of the picture of the node

Type:

int = 200

h

The height in pixels of the picture of the node

Type:

int = 50

self[widget]

A dictionary of tk widgets built using the information contained in Active Learning_parameters.py

Type:

dict

See also

ActiveLearning, TkActiveLearning, ActiveLearningParameters

create_dialog()

Create the dialog. A set of widgets will be chosen by default based on what is specified in the Active Learning_parameters module.

Parameters:

None

Return type:

None

See also

TkActiveLearning.reset_dialog

right_click(event)

Handles the right click event on the node.

Parameters:

event (Tk Event)

Return type:

None

See also

TkActiveLearning.edit

active_learning_step.tk_check_committee module

The graphical part of a Check Committee step

class active_learning_step.tk_check_committee.TkCheckCommittee(tk_flowchart=None, node=None, canvas=None, x=None, y=None, w=200, h=50)

Bases: TkSubstep

The graphical part of a Check Committee step in a flowchart.

The dialog shows only the parameters that apply to the current choices.

See also

CheckCommittee, CheckCommitteeParameters, TkSubstep

dialog_title = 'Check Committee'

The title of the dialog

triggers = ('labelled data',)

The parameters whose change lays the dialog out again

visible(key)

Whether the parameter key applies to the current choices.

active_learning_step.tk_score_pool module

The graphical part of a Score Pool step

class active_learning_step.tk_score_pool.TkScorePool(tk_flowchart=None, node=None, canvas=None, x=None, y=None, w=200, h=50)

Bases: TkSubstep

The graphical part of a Score Pool step in a flowchart.

The dialog shows only the parameters that apply to the current choices.

See also

ScorePool, ScorePoolParameters, TkSubstep

MODEL_KEYS = ('leave out members', 'add dispersion', 'device', 'precision', 'batch size', 'per-atom spreads')

The parameters that only apply when there is a committee

dialog_title = 'Score Pool'

The title of the dialog

triggers = ('committee', 'pool')

The parameters whose change lays the dialog out again

visible(key)

Whether the parameter key applies to the current choices.

active_learning_step.tk_select_frames module

The graphical part of a Select Frames step

class active_learning_step.tk_select_frames.TkSelectFrames(tk_flowchart=None, node=None, canvas=None, x=None, y=None, w=200, h=50)

Bases: TkSubstep

The graphical part of a Select Frames step in a flowchart.

The dialog shows only the parameters that apply to the current choices.

See also

SelectFrames, SelectFramesParameters, TkSubstep

dialog_title = 'Select Frames'

The title of the dialog

triggers = ('method', 'thresholds', 'budget')

The parameters whose change lays the dialog out again

visible(key)

Whether the parameter key applies to the current choices.

active_learning_step.tk_substep module

The graphical part shared by the sub-steps of the Active Learning step.

class active_learning_step.tk_substep.TkSubstep(tk_flowchart=None, node=None, canvas=None, x=None, y=None, w=200, h=50)

Bases: TkNode

A sub-step’s dialog: its parameters, shown only when they apply.

Subclasses set dialog_title, list in triggers the choices that change which parameters apply, and say which do in visible(), so a setting that does not apply to the current choices is never shown.

create_dialog()

Create the dialog, with a Parameters tab and the standard Results tab.

dialog_title = 'Active Learning'

The title of the dialog

reset_dialog(widget=None)

Lay out the parameters that apply to the current choices.

right_click(event)

Handle a right-click: add the Edit… item and post the menu.

property structure_selection

Whether the step has the standard structure-selection parameters.

triggers = ()

The parameters whose change lays the dialog out again

value(key)

The current value of a parameter’s widget, as text.

visible(key)

Whether the parameter key applies to the current choices.

active_learning_step.tk_train_committee module

The graphical part of a Train Committee step

class active_learning_step.tk_train_committee.TkTrainCommittee(tk_flowchart=None, node=None, canvas=None, x=None, y=None, w=200, h=50)

Bases: TkSubstep

The graphical part of a Train Committee step in a flowchart.

The dialog shows only the parameters that apply to the current choices.

See also

TrainCommittee, TrainCommitteeParameters, TkSubstep

dialog_title = 'Train Committee'

The title of the dialog

triggers = ('stratify',)

The parameters whose change lays the dialog out again

visible(key)

Whether the parameter key applies to the current choices.

active_learning_step.train module

seamm-al train: the k members of a committee, each on k - 1 folds of the data.

The labelled frames of the data files are cut into k folds (kfold_indices(), seeded by the data seed); member l trains on every fold but l and validates on fold l, from the initialisation seed init_seed + l. The recipe (model, loss weights, optimiser, epochs, the recorded subtracted dispersion) is an ordinary xnn YAML config; only the data, the folds, the seeds, the output directory and the batching come from here.

Batches never mix clusters and cells (samplers): clusters use the config’s data.batch_size, cells cell_batch_size. With cell_share the cell batches make that share of every epoch. With steps every member trains for that many optimiser steps whatever the size of the data (the epochs are the steps divided by the batches per epoch), so committees trained on different selections differ only in their data.

The output directory holds

• manifest.json: the folds’ sizes, the seeds, the data files with their SHA-256, a hash of the ordered frames, the recipe, the batching and the code versions. Every member checks it before training, so members trained in separate jobs are certain to share one fold plan; a different plan in the same directory is refused;

• frames.csv: every frame with its file, index, configuration name, kind and fold;

• m<l>/: each member’s best.pt, last.pt, history.json and member.json (its seeds, sizes, steps, timing and versions).

A member whose last.pt exists is skipped, so a job can be rerun after a failure.

active_learning_step.train.file_sha256(path, chunk=1048576)

The SHA-256 of a file, hex.

active_learning_step.train.fold_plan(n, k, data_seed)

The fold of every frame: fold[i] is the member that holds frame i out.

Plain k-fold: kfold_indices(), the water campaign’s split.

active_learning_step.train.load_frames(paths, energy_key='energy', forces_key='forces', stress_key='stress', columns=())

The labelled structures of data files, with where each came from.

Parameters:

• paths (sequence of str) – extxyz files (or SDF from SEAMM; their labels must be in xnn’s keys).

• energy_key (str, optional) – Where the labels are, as xnn’s data.energy_key etc.

• forces_key (str, optional) – Where the labels are, as xnn’s data.energy_key etc.

• stress_key (str, optional) – Where the labels are, as xnn’s data.energy_key etc.

• columns (sequence of str, optional) – info keys to keep for every frame (as strings; "" when absent).

Returns:

The xnn structure dicts, in file order, and for each the file, index, configuration and the requested columns.

Return type:

(list of dict, list of dict)

active_learning_step.train.make_trainer(Trainer, cfg, train_set, val_set, sampler, val_sampler, callbacks=())

An xnn Trainer that batches with our samplers and runs our callbacks.

The samplers go to the trainer’s train_batch_sampler and val_batch_sampler; it tells them the epoch before every pass. Needs xnn 0.13 or later (require_training_xnn()).

active_learning_step.train.require_training_xnn()

Refuse to train with an xnn older than MIN_TRAIN_XNN.

active_learning_step.train.source_sha256()

A SHA-256 over the package’s Python sources (name and contents, sorted).

It identifies the code however it was deployed, including a copy without git history, where the version reads 0+unknown.

active_learning_step.train.stratified_fold_plan(strata, k, data_seed)

Folds that split every stratum as evenly as possible.

The strata are taken in sorted order; the frames of each are shuffled (one generator seeded by data_seed) and dealt to the folds in turn, each stratum starting at the fold after the one the previous stratum ended on. So within a stratum the folds’ counts differ by at most one, and so do the folds’ sizes.

Parameters:

• strata (sequence of str) – The stratum of every frame.

• k (int) – Number of folds.

• data_seed (int) – Seed of the shuffles.

Returns:

The fold of every frame.

Return type:

numpy.ndarray of int

Raises:

ValueError – If k is smaller than 2 or larger than the number of frames.

active_learning_step.train.train_committee(config, data, out, k=5, members=None, data_seed=0, init_seed=1000, cell_share=None, cell_batch_size=4, steps=None, device=None, overrides=(), stratify='kind', stratify_columns=(), threads=None)

Train members of a k-fold committee.

Parameters:

• config (str) – The xnn YAML config: the recipe.

• data (sequence of str) – The labelled data files, in order.

• out (str) – The committee’s directory.

• k (int, optional) – Number of folds and members, by default 5.

• members (sequence of int, optional) – Which members to train here, by default all.

• data_seed (int, optional) – Seed of the fold permutation, by default 0.

• init_seed (int, optional) – Member l is initialised (and its batches shuffled) with init_seed + l; by default 1000.

• cell_share (float, optional) – Cell batches as this share of every epoch; by default one pass over the cells.

• cell_batch_size (int, optional) – Cells per batch, by default 4.

• steps (int, optional) – Train for this many optimiser steps instead of the config’s epochs.

• device (str, optional) – Overrides the config’s device.

• overrides (sequence of str, optional) – xnn-style a.b=value overrides of the config.

• stratify (str, optional) – "kind" (the default) splits clusters and cells separately, so every fold holds out its share of each; "none" is a plain k-fold (kfold_indices()).

• threads (int, optional) – CPU threads for torch (set_threads()).

• stratify_columns (sequence of str, optional) – info keys whose values also define strata (combined with the kind unless stratify="none"), e.g. liquid.

Returns:

The directories of the members trained.

Return type:

list of str

active_learning_step.train_committee module

Non-graphical part of the Train Committee step in an Active Learning flowchart

class active_learning_step.train_committee.TrainCommittee(flowchart=None, title='Train Committee', extension=None, logger=<Logger active_learning_step.train_committee (WARNING)>)

Bases: Substep

The non-graphical part of a Train Committee step in a flowchart.

parser

The parser object.

Type:

configargparse.ArgParser

options

It contains a two item tuple containing the populated namespace and the list of remaining argument strings.

Type:

tuple

subflowchart

A SEAMM Flowchart object that represents a subflowchart, if needed.

Type:

seamm.Flowchart

parameters

The control parameters for Train Committee.

Type:

TrainCommitteeParameters

See also

TkTrainCommittee, TrainCommittee, TrainCommitteeParameters

analyze(indent='', committee=None, P=None, **kwargs)

Print the members’ training and store the results.

Parameters:

• indent (str) – An extra indentation for the output

• committee (pathlib.Path) – The committee’s directory.

• P (dict) – The parameters of the step.

description_text(P=None)

Create the text description of what this step will do.

Parameters:

P (dict) – An optional dictionary of the current values of the control parameters.

Returns:

A description of the current step.

Return type:

str

property git_revision

The git version of this module.

property header

A printable header for this section of output

run()

Train the committee.

Returns:

The next node object in the flowchart.

Return type:

seamm.Node

save_as_model(committee, name)

Copy the committee’s members into the user’s xnn model directory.

Only what the engine needs is copied: each member’s best.pt, with its member.json and history.json, and the committee’s manifest.

Returns:

The new model directory.

Return type:

pathlib.Path

property version

The semantic version of this module.

active_learning_step.train_committee_parameters module

Control parameters for the Train Committee step in a SEAMM flowchart

class active_learning_step.train_committee_parameters.TrainCommitteeParameters(defaults={}, data=None)

Bases: Parameters

The control parameters for Train Committee.

You need to replace the “time” entry in dictionary below these comments with the definitions of parameters to control this step. The keys are parameters for the current plugin,the values are dictionaries as outlined below.

Examples

parameters = {
    "time": {
        "default": 100.0,
        "kind": "float",
        "default_units": "ps",
        "enumeration": tuple(),
        "format_string": ".1f",
        "description": "Simulation time:",
        "help_text": ("The time to simulate in the dynamics run.")
    },
}

parameters{str: {str: str}}

A dictionary containing the parameters for the current step. Each key of the dictionary is a dictionary that contains the the following keys:

parameters[“default”] :

The default value of the parameter, used to reset it.

parameters[“kind”]enum()

Specifies the kind of a variable. One of “integer”, “float”, “string”, “boolean”, or “enum”

While the “kind” of a variable might be a numeric value, it may still have enumerated custom values meaningful to the user. For instance, if the parameter is a convergence criterion for an optimizer, custom values like “normal”, “precise”, etc, might be adequate. In addition, any parameter can be set to a variable of expression, indicated by having “$” as the first character in the field. For example, $OPTIMIZER_CONV.

parameters[“default_units”]str

The default units, used for resetting the value.

parameters[“enumeration”]: tuple

A tuple of enumerated values.

parameters[“format_string”]: str

A format string for “pretty” output.

parameters[“description”]: str

A short string used as a prompt in the GUI.

parameters[“help_text”]: str

A longer string to display as help for the user.

See also

TrainCommittee, TkTrainCommittee, TrainCommitteeParameters, TrainCommitteeStep

parameters = {'cell batch size': {'default': 4, 'default_units': '', 'description': 'Cells per batch:', 'enumeration': (), 'format_string': 'd', 'help_text': "The number of periodic cells in a batch. Clusters use the recipe's batch size.", 'kind': 'integer'}, 'cell share': {'default': 'one pass over the data', 'default_units': '', 'description': 'Share of cell batches:', 'enumeration': ('one pass over the data',), 'format_string': '.2f', 'help_text': "The fraction of each epoch's batches that are periodic cells. By default each epoch goes once over the clusters and once over the cells; a set share draws the cells more or less often than that.", 'kind': 'float'}, 'data seed': {'default': 0, 'default_units': '', 'description': 'Seed for the folds:', 'enumeration': (), 'format_string': 'd', 'help_text': 'The random seed that assigns the structures to folds.', 'kind': 'integer'}, 'device': {'default': 'from the recipe', 'default_units': '', 'description': 'Device:', 'enumeration': ('from the recipe', 'cpu', 'cuda', 'mps'), 'format_string': '', 'help_text': "The PyTorch device to train on: cpu, cuda (or cuda:N), or mps; by default the recipe's.", 'kind': 'string'}, 'initial seed': {'default': 1000, 'default_units': '', 'description': 'Seed of member 0:', 'enumeration': (), 'format_string': 'd', 'help_text': "The random seed of the first member's weights; member n uses this plus n.", 'kind': 'integer'}, 'members to train': {'default': 'all', 'default_units': '', 'description': 'Members to train:', 'enumeration': ('all',), 'format_string': '', 'help_text': 'Which members to train in this step, by number (0, 1, ...) separated by spaces, or all. Members trained in other jobs into the same directory join the same committee.', 'kind': 'string'}, 'model name': {'default': '', 'default_units': '', 'description': 'Save as the model:', 'enumeration': (), 'format_string': '', 'help_text': 'Also save the committee among your xnn models with this name, so that the Model Chemistry step offers it as xnn:MLFF@<name>, for example to run molecular dynamics with it and record where its members disagree. Leave empty not to save it.', 'kind': 'string'}, 'number of members': {'default': 5, 'default_units': '', 'description': 'Number of members:', 'enumeration': (), 'format_string': 'd', 'help_text': 'The number of members, which is also the number of folds: each member trains on all but one fold, and never sees that fold.', 'kind': 'integer'}, 'number of steps': {'default': 'from the recipe', 'default_units': '', 'description': 'Optimizer steps:', 'enumeration': ('from the recipe',), 'format_string': 'd', 'help_text': "The number of optimizer steps for each member, instead of the recipe's number of epochs.", 'kind': 'integer'}, 'recipe': {'default': '', 'default_units': '', 'description': 'xnn recipe:', 'enumeration': (), 'format_string': '', 'help_text': 'The xnn training configuration (YAML): the model, its hyperparameters and the training schedule. Every member is trained with it, on a different part of the data.', 'kind': 'string'}, 'recipe changes': {'default': '', 'default_units': '', 'description': 'Changes to the recipe:', 'enumeration': (), 'format_string': '', 'help_text': 'Settings to change in the recipe, as KEY=VALUE separated by spaces, for example optim.lr=0.005 trainer.epochs=200.', 'kind': 'string'}, 'results': {'default': {}, 'default_units': '', 'description': 'results', 'enumeration': (), 'format_string': '', 'help_text': 'The results to save to variables or in tables.', 'kind': 'dictionary'}, 'stratify': {'default': 'clusters and cells separately', 'default_units': '', 'description': 'Folds:', 'enumeration': ('clusters and cells separately', 'no'), 'format_string': '', 'help_text': 'How the data is split into folds: with clusters and periodic cells split separately, so each fold has its share of both, or a plain random split.', 'kind': 'string'}, 'stratify by': {'default': '', 'default_units': '', 'description': 'Also balance by:', 'enumeration': (), 'format_string': '', 'help_text': "Keys in the structures' info to also balance the folds by, separated by spaces, for example the temperature or the composition.", 'kind': 'string'}, 'training data': {'default': '', 'default_units': '', 'description': 'Labelled data:', 'enumeration': (), 'format_string': '', 'help_text': "The labelled structures to train on: extended XYZ files with the reference energies and forces (for example from the Write Structure step). Give names or patterns such as data/*.extxyz, separated by spaces, relative to the job's directory.", 'kind': 'string'}}

active_learning_step.train_committee_step module

class active_learning_step.train_committee_step.TrainCommitteeStep(flowchart=None, gui=None)

Bases: object

Helper class needed for the stevedore integration.

This must provide a description() method that returns a dict containing a description of this node, and create_node() and create_tk_node() methods for creating the graphical and non-graphical nodes.

The dictionary for the description is the class variable just below these comments. The felds are as follows:

my_description{str, str}

A human-readable description of this step. It can be several lines long, and needs to be clear to non-expert users. It contains the following keys: description, group, name.

my_description[“description”]tuple

A description of the Train Committee step. It must be clear to non-experts.

my_description[“group”]str

Which group in the menus to put this step. If the group does not exist it will be created. Common groups are “Building”, “Control”, “Custom”, “Data”, and “Simulations”.

my_description[“name”]str

The name of this step, to be displayed in the menus.

create_node(flowchart=None, **kwargs)

Create and return the new node object.

Parameters:

• flowchart (seamm.Node) – A non-graphical SEAMM node

• **kwargs (keyword arguments) – Various keyword arguments such as title, namespace or extension representing the title displayed in the flowchart, the namespace for the plugins of a subflowchart and the extension, respectively.

Return type:

TrainCommittee

create_tk_node(canvas=None, **kwargs)

Create and return the graphical Tk node object.

Parameters:

• canvas (tk.Canvas) – The Tk Canvas widget

• **kwargs (keyword arguments) – Various keyword arguments such as tk_flowchart, node, x, y, w, h representing a graphical flowchart object, a non-graphical node for a step, and dimensions of the graphical node.

Return type:

TkTrainCommittee

description()

Return a description of what this step does.

Returns:

description

Return type:

dict(str, str)

my_description = {'description': 'Train a k-fold committee of xnn machine-learned force fields', 'group': 'Calculations', 'name': 'Train Committee'}

Module contents

active_learning_step A SEAMM plug-in for Active Learning