Command line: `seamm-al`#

seamm-al chooses which candidate frames to label next, by query by committee: a committee of models (typically five, each trained on four fifths of the data) is run on every candidate, and the frames where the members disagree most are labelled.

Environments#

score, train, check and engine need torch and xnn; use the MLFF environment SEAMM itself uses, the one named in the xnn.ini and lammps.ini of the installation, with xnn 0.13 or later. Training requires 0.13; scoring, checking and the engine give the same results with older releases, but a committee trained with 0.13 can only be read by 0.13 or later. select needs neither torch nor xnn and runs in any SEAMM environment. Without installing the package, run python -m active_learning_step.cli with the repository on PYTHONPATH.

On a CPU, torch uses --threads threads when given, otherwise OMP_NUM_THREADS (which SEAMM, PBS and most launchers set from the job’s allocation), otherwise as many as the CPUs the process may run on, which a scheduler’s cpuset or taskset limits. No scheduler is assumed, so the same commands work under Slurm, PBS, SEAMM’s own JobServer or none.

Scoring#

seamm-al score needs torch and xnn, so run it in an MLFF environment (see Environments):

seamm-al score -m m0/best.pt m1/best.pt m2/best.pt m3/best.pt m4/best.pt \
    -o scores.csv --carry temperature pool.extxyz

The pools are extxyz files or SDF files written by SEAMM, either of them optionally gzipped (trajectory.extxyz.gz). scores.csv has one row per frame:

column	meaning
file, index, configuration	where the frame is (index from 0 in its file)
kind	`cell` if periodic, else `cluster`
n_atoms, formula	the frame’s size and Hill formula
total_charge, n_molecules	the charge given to the models; molecules of a cell
rho_energy	committee standard deviation of the energy / sqrt(N), eV
rho_forces	RMS over atoms of the per-atom force spread, eV/Å
rho_forces_max_atom	the largest per-atom force spread, eV/Å
rho_pressure	spread of the atomic pressure, atm (cells)
rho_pressure_molecular	spread of the molecular pressure, atm (cells)
energy_mean, pressure_mean	the committee average; also pressure_molecular_mean

Molecules are found with SEAMM’s own rule, that of molsystem’s perceive_bonds: atoms closer than 1.2 times the sum of their (Pyykkö) covalent radii are bonded, the alkali and alkaline-earth metals never are, and a hydrogen has at most one bond. The rule is applied to the coordinates, so it gives the same molecules as SEAMM without needing SEAMM; for a SEAMM structure that has bonds of its own, those are used. The total charge is read from total_charge in the extxyz info or from SEAMM’s net charge in an SDF. By default the dispersion the checkpoints record as subtracted is not added back (--dispersion false): it is the same in every member and adds nothing to the spreads. --atoms frames.extxyz writes every frame with its per-atom force spread.

The models are served in the dtype of their weights, usually float32. Float32 reductions on CPUs are not reproducible from run to run: the same frame scored twice can differ by about 1e-5 relative in the pressure spreads (and much less in the energy and force spreads), which cannot change a ranking. --dtype float64 gives reproducible numbers at about twice the cost.

Training a committee#

seamm-al train trains the k members of a committee in an MLFF environment:

seamm-al train -c recipe.yaml -o committee labelled.extxyz [more.extxyz ...] \
    -k 5 --cell-share 0.15 --steps 52950 [--member 2]

The recipe is an ordinary xnn YAML config (model, loss weights, optimiser, epochs and the dispersion subtracted from the labels); seamm-al supplies the data, the folds, the seeds and the batching. The labelled frames are cut into k folds with the data seed (--data-seed, default 0), splitting clusters and cells separately so that every fold holds out its share of each (--stratify kind, the default; within a stratum the folds’ counts differ by at most one). --stratify-column liquid adds an info key to the strata, and --stratify none is a plain k-fold. Member l trains on the other folds, validates on fold l and is initialised from seed 1000 + l (--init-seed). Batches never mix clusters and cells: clusters use the recipe’s data.batch_size and cells --cell-batch-size (default 4). --cell-share makes cell batches that share of every epoch whatever the selection holds, and --steps trains every member for the same number of optimiser steps, so committees trained on different selections differ only in their data. avg_num_neighbors: auto in the recipe uses the mean neighbour count of each member’s training set.

The committee’s directory holds manifest.json (the folds, the seeds, the data files and their SHA-256, a hash of the ordered frames, the recipe, the batching and the code versions), frames.csv (every frame’s fold) and one directory per member with best.pt, last.pt, history.json and member.json. Members can be trained in separate jobs (--member): each checks the manifest, a directory holding a different plan is refused, and a member that has finished is skipped.

Training needs xnn 0.13 or later, which chooses best.pt, drives the plateau learning rate and logs on dataset-averaged epoch metrics, and tells the batch samplers the epoch every pass. score, check and engine work with older releases on checkpoints those releases wrote, but a checkpoint trained with xnn 0.13 can only be read by 0.13 or later (its pickled config), so a committee trained here is scored, checked and run in an xnn 0.13 environment. Each member’s epochs are recorded in its history.json through an xnn callback, and the recipe’s trainer section (loggers, early stopping, checkpoints every n epochs) applies as in xnn train.

Selecting#

seamm-al select needs neither torch nor xnn:

seamm-al select scores.csv --budget 75000 \
    --cost cell:EC=2800 --cost cell:FEC=3900 --group-column liquid \
    --cap temperature=6 --exclude labelled.txt

By default cells are ranked by the sum of their ranks in rho_pressure_molecular and rho_forces, clusters by rho_forces (change with --rank), and frames are taken in that order until the next would exceed the budget (--fill skips it and keeps going). --threshold rho_forces=0.0106 limits the candidates to frames whose spread reaches a threshold. The output is selection.csv, with the reason, the running cost and the frame’s rank within its kind (rank_in_kind of n_in_kind) for every frame chosen; the command also reports where the budget cut fell, and --rejected rejected.csv lists every frame not chosen with its rank and why, and the chosen frames copied byte for byte from the pool files into selected.extxyz and/or selected.sdf, ready for labelling.

Round 0: a random draw#

Before any committee exists, score --no-models lists a pool (identity and --carry columns only, no torch or xnn needed) and select --random SEED draws from it in a seeded random order, with the same costs, budget, caps, --exclude and --max-frames, and the same verbatim output:

seamm-al score --no-models --carry source --carry salt -o pool.csv clusters.extxyz
seamm-al select pool.csv --random 1 --stratify-column source --stratify-column salt \
    --cost cluster=4 --max-frames 50

--stratify-column spreads the draw evenly over the values of the columns: one frame from each stratum in turn, the strata in a shuffled order. selection.csv records the seed, each frame’s stratum and its place in the draw.

Checking a committee#

seamm-al check asks whether the committee’s spread predicts its real error, on labelled frames:

seamm-al check committee labelled.extxyz --tag-key frame \
    --calibrate rho_forces:0.14 --calibrate rho_pressure_molecular:1500:0.9

For each frame it gives the spreads and the out-of-fold errors: the error of the one member that held the frame out, or the mean over the members for a frame none trained on (a common validation or test set); frames every member trained on are left out. The errors are the force RMSE, the energy per atom and, for cells, the atomic and molecular pressures (the label’s molecular pressure from its stress and forces). The command prints, and --summary writes, the Spearman correlation of each spread with its error per kind, over all out-of-fold frames and the held-out ones only.

The committee directory is one made by seamm-al train (frames matched by configuration name) or one with a fold_manifest.json per member listing tags under held_out, validation and test; --tag-key names the info key of the tags. Labels are read from REF_energy, REF_forces and REF_stress (a 9-vector, a Voigt 6-vector or a matrix) unless --*-key says otherwise.

check also judges every member. For each it reports the final and best validation loss (from the member’s history.json when there is one), its mean error on the frames it held out and on the common validation and test frames, and the median leave-one-out ratio of each spread: the spread with the member over the spread without it, so a member that inflates the spreads stands out. A member more than --flag-factor (default 1.5) times the median of the others on any of these is flagged — as a note only, except for the final validation loss, which is the gate. A member that trailed the others for a while (validation losses smoothed over 9 epochs, at least 5 epochs above the factor) and then recovered is noted, not flagged. The table is written to <out>_members.csv. --drop-member K (repeatable) leaves member m<K> out, in check and in score (where K counts the --models from 0); the members used and dropped are recorded in the summary and in scores.csv.json.

--calibrate NAME:FAIL_LEVEL[:CAPTURE] calibrates a threshold as Smith et al. (fig. 1): the largest spread that still catches CAPTURE (default 0.98) of the frames whose worst-member error exceeds FAIL_LEVEL (eV/A for forces, atm for pressures, eV per sqrt(atom) for energies), with what it separates. Fewer than 10 failures earn a warning: such a threshold is not to be trusted.

Committee molecular dynamics#

seamm-al engine serves the committee’s mean to an MDI driver such as LAMMPS (fix mdi/qm) as one potential: the mean of the members’ short-range energies, with the dispersion the checkpoints record added once on top, differentiated once for the forces and stress. It takes the place of xnn mdi in an MDI pair:

mpirun --bind-to none \
    -np 1 seamm-al engine --members m0/best.pt m1/best.pt m2/best.pt m3/best.pt \
        m4/best.pt --device cuda:0 --eeq-reuse --spreads spreads.csv --every 10 \
        --flag rho_forces=0.05 --flagged flagged.extxyz \
        -mdi "-role ENGINE -name xnn -method MPI" \
  : -np 1 lmp -mdi "-role DRIVER -name LAMMPS -method MPI" -in input.dat

With --spreads the engine also evaluates every member on its own every --every force evaluations and appends the committee spreads (rho_energy, rho_forces, rho_forces_max_atom and, for periodic systems, rho_pressure and rho_pressure_molecular) to the CSV, with the committee-mean energy and max_dF, the largest difference between the forces served and the mean of the members’ forces (rounding only, without dispersion). --flag NAME=VALUE writes the frames whose spread reaches a value to --flagged with their step, spreads and per-atom force spread, ready for seamm-al score or labelling; --frames writes every frame recorded. --drop-member K leaves a member out and --no-dispersion serves the energy without the dispersion the checkpoints record as subtracted from their labels.

LAMMPS runs the engine with atom_style atomic, which reads only the Atoms, Velocities, Masses and pair-coefficient sections of a data file. Check a data file before submitting, and strip what it cannot read (a Molecules or Bonds section, bond counts in the header) into a copy:

seamm-al data water512.data                      # exit status 1 on a problem
seamm-al data water512.data --strip water512_atomic.data

On a CPU (8 threads, 64 waters, five MACE members) the mean costs five times one model, and spreads every 10 steps add about 11 %.

Command line: seamm-al#