seamm_exec package

Submodules

seamm_exec.base module

Virtual base class for the Exec classes.

class seamm_exec.base.Base(logger)

Bases: object

exec(config, cmd=[], directory=None, input_data=None, env={}, shell=False, ce={})

Virtual method for executing a command.

Parameters:

• cmd ([str]) – The command as a list of words.

• directory (str or Path) – The directory for the tasks files.

• input_data (str) – Data to be redirected to the stdin of the process.

• env ({str: str}) – Dictionary of environment variables to pass to the execution environment

• shell (bool = False) – Whether to use the shell when launching task

• ce (dict(str, str or int)) – Description of the computational enviroment

get_mode_info(filename, mode)

Get the mode information for ‘ls’ like listing

ls_format(path, files)

Format a list of files as in ‘ls’

property name

The name of this type of executor.

run(config, cmd=[], directory=None, input_data=None, files=None, env={}, return_files=[], shell=False, in_situ=None, ce={})

Execute a command directly on the current machine.

Parameters:

• config (configparser.config) – The configuration options for the code

• cmd ([str]) – The command as a list of words.

• directory (str or Path) – The directory for the tasks files.

• input_data (str) – Data to be redirected to the stdin of the process.

• files ({str: str or byte}) – Dictionary of file names and data to write before execution.

• env ({str: str}) – Dictionary of environment variables to pass to the execution environment

• return_files ([str]) – List of files to return/keep from the calculation

• shell (bool = False) – Whether to use the shell when launching task

• in_situ (bool or None = None) – Whether to run in the given directory, versus a temp directory, copying back only return_files afterwards. None (the default) auto-detects: run in a temp directory – which honors $TMPDIR, so it lands on node-local scratch under a scheduler that sets it – when running under a batch scheduler (currently SLURM), since the job directory there is typically NFS-mounted and unsafe for an MPI code’s scratch I/O (molssi-seamm/orca_step#20); otherwise run in place, as before.

• ce (dict(str, str or int)) – Description of the computational enviroment

Returns:

• dict – “stdout”, “stderr”, “returncode”, “in_situ”, “directory” (where it ran) and “files”, the names of the returned files; for each of those, result[name] = {"path": Path, "exception": str or None}. The contents of the files are not read – they can be gigabytes – so a caller reads what it needs from “path” ("@subdir+name" is put in subdir/name).

• This is the original, blocking interface. It runs one task through a

• TaskSet with a one-slot, synchronous

• LocalPool, without a manifest, so its

• behaviour and the files it leaves are exactly as before.

seamm_exec.bundle_runner module

Run a SEAMM-mode bundle inside an allocation (see task_worker.py).

Each task runs through a LocalPool sized to the allocation, with programs resolved on this machine, and the task’s marker directory receives DONE or FAILED.

seamm_exec.bundle_runner.run_seamm_bundle(bundle)

Run a SEAMM-mode bundle through a LocalPool. Returns the number failed.

seamm_exec.computational_environment module

Helper routines for determining the computational environment from queueuing systems.

seamm_exec.computational_environment.computational_environment(limits={})

Examine where we are running to get the computational environment.

Parameters:

limits (dict(str, any)) – Limits to impose on the number of tasks, gpus, etc.

Returns:

The attributes of the computational enviroment, limited by the imposed limits.

Return type:

dict(str, any)

seamm_exec.computational_environment.expand_hostlist(text)

SLURM’s compressed hostlist -> host names.

tc[053,059-061],gpu7 -> tc053 tc059 tc060 tc061 gpu7; zero padding is kept; a suffix after the brackets (tc[01-02]-ib) and several bracket groups (r[1-2]n[1-2], the product) are expanded as SLURM does.

seamm_exec.computational_environment.running_scheduler()

The name of the queueing system whose allocation this is, or None.

seamm_exec.computational_environment.scheduler_job_variables()

The environment variables that mark a batch job, one per scheduler.

seamm_exec.evaluator module

Evaluate a model chemistry at many structures, over MDI or as tasks.

A step that needs energies (and gradients, and stress) for many structures submits them to an Evaluator and iterates over the results. The evaluator, not the step or the user, chooses how they are computed:

• the batch path: each structure is a Task from the program’s get_task, run by a TaskSet (on the job’s target: the local pool or a queue, with restart), and read back by the program’s analyze_task;

• the MDI path: one warm MDI engine per group of structures with the same elements, charge, multiplicity and periodicity, from the program’s get_mdi_engine_command.

The rule (see choose_path()): the batch path when the job’s tasks go to a queue and the program has get_task; else MDI when the program has an MDI engine, unless it prefers the batch path (options["prefers_batch"], set by ORCA, whose engine runs a subprocess per structure); else the batch path in the local pool. Both paths return the same numbers for the same model chemistry.

The program’s contract (classmethods beside get_model_chemistry_options):

get_task(configuration, model_chemistry, *, key, properties, options, resources)
    -> seamm_exec.Task
analyze_task(result, model_chemistry, configuration, *, properties, options)
    -> {"energy": kJ/mol, "gradients": (n, 3) kJ/mol/Å, "stress": GPa, ...}
can_run_task(configuration, model_chemistry, *, options) -> bool   (optional)

A structure for which can_run_task is False (e.g. a periodic system for ORCA or MOPAC) goes to the program’s MDI engine, if it has one, even when the others run as tasks. If there is no engine, or it cannot start here (a queue target with the code only on the cluster), that structure gets a failed result; so does one whose get_task raises. Neither stops the rest.

analyze_task reads the returned files with result.read(name) (they are on disk or in the task archive, not in the result). It raises AnalysisError when a required property is missing; it never returns partial numbers. options passes what a consumer needs for a fragment: atom_indices, ghost_atoms, charge, multiplicity, an initial guess.

exception seamm_exec.evaluator.AnalysisError

Bases: RuntimeError

A task’s results lack a requested property.

class seamm_exec.evaluator.Evaluator(node, model_chemistry=None, *, properties=('energy', 'gradients'), path=None, directory=None, target=None, task_set_options=None, resources=None, name='SEAMM')

Bases: object

Evaluate a model chemistry at many structures.

Parameters:

• node (seamm.Node) – The step: its directory, flowchart (plug-ins, executor) and options.

• model_chemistry (dict, optional) – The _model_chemistry wrapper. Default: the node’s variable.

• properties ([str]) – “energy”, “gradients”, “stress”.

• path (str, optional) – “mdi” or “batch”, for tests only; the evaluator chooses otherwise.

• directory (str or Path, optional) – The batch path’s step directory (<directory>/tasks/...). Default: the node’s.

• target (seamm_scheduler.TargetSection, optional) – The job’s target. Default: found for the job.

• task_set_options (dict, optional) – Extra arguments for the TaskSet (bundle_tasks, archive, …).

• resources (seamm_exec.Resources, optional) – The resources of each calculation on the batch path (ranks, memory per rank), passed to the provider’s get_task. Default: the provider’s.

• name (str) – A name for the MDI engine.

cancel()

Stop this evaluator’s work, from any thread: the running TaskSet is cancelled (its tasks in flight killed, the held ones dropped, all marked cancelled for a fresh start later), and the MDI path stops before its next structure. Results already produced stand.

property cancelled

close()

property mdi_capable

Whether the program has an MDI engine for this model chemistry.

results()

Compute what has been submitted; yield an EvaluatorResult for each, as it is ready.

submit(configuration, key=None, *, options=None)

Add a structure; returns its key (unique, filesystem-safe).

static topology_key(configuration, options=None)

What must stay fixed for one MDI engine session.

class seamm_exec.evaluator.EvaluatorResult(key: str, ok: bool, energy: float | None = None, gradients: object = None, stress: object = None, reason: str | None = None, restored: bool = False, path: str = 'batch', data: dict = <factory>, elapsed: float = 0.0)

Bases: object

One structure’s results.

Variables:

• key (str)

• ok (bool)

• energy (float or None) – kJ/mol

• gradients (numpy.ndarray or None) – (n, 3), kJ/mol/Å

• stress (list or None) – GPa, as the program gives it

• reason (str or None) – Why it failed.

• restored (bool) – From an earlier run (batch path).

• path (str) – “mdi” or “batch”

• data (dict) – Everything the program returned.

• elapsed (float) – Seconds spent on it in this run (0 if restored).

data: dict

elapsed: float = 0.0

energy: float | None = None

gradients: object = None

key: str

ok: bool

path: str = 'batch'

reason: str | None = None

restored: bool = False

stress: object = None

class seamm_exec.evaluator.Geometry(atomic_numbers, coordinates, charge=0, multiplicity=1, cell=None, name=None)

Bases: object

A light stand-in for a molsystem configuration: elements, Cartesian coordinates (Å), charge, multiplicity and an optional cell. For structures a step makes on the fly, such as finite-difference displacements.

property n_atoms

seamm_exec.evaluator.REQUIRED = ('energy', 'gradients')

Properties a result must have when requested. Stress is required only for a periodic structure (check_properties(..., periodic=True), and the Evaluator checks it for every task); a molecule has none.

seamm_exec.evaluator.check_properties(data, properties, what, periodic=False)

Raise AnalysisError unless data has every required property in properties (and the stress, if requested, for a periodic structure). For programs’ analyze_task.

seamm_exec.evaluator.choose_path(model_chemistry, provider, target=None)

“mdi” or “batch” for a model chemistry, a program and the job’s target.

Raises:

ValueError – If the program offers neither path.

seamm_exec.evaluator.mdi_method_and_basis(model_chemistry)

The (method, basis) an MDI engine is launched with.

method is the program’s own keyword (options["mdi_method_arg"]), falling back to the model chemistry’s method; basis is options["mdi_basis_arg"] (the user’s basis, for programs that take one) or the model chemistry’s basis, or None for programs that take a method alone (MOPAC, xTB, an MLFF).

seamm_exec.evaluator.structure_data(configuration)

What a program needs from a configuration (or a Geometry).

Returns:

atomic_numbers, symbols, coordinates ((n, 3) Å), charge, multiplicity, periodicity and cell ((3, 3) Å or None).

Return type:

dict

seamm_exec.exec_flowchart module

The ExecWorkflow object does what it name implies: it executes, or runs, a given flowchart.

It provides the environment for running the computational tasks locally or remotely, using what is commonly called workflow management system (WMS). The WMS concept, as used here, means tools that run given tasks without knowing anything about chemistry. The chemistry specialization is contained in the Flowchart and the nodes that it contains.

class seamm_exec.exec_flowchart.ExecFlowchart(flowchart=None, cmdline=None, plan=None)

Bases: object

run(root=None, job_id=None)

seamm_exec.exec_flowchart.archive_previous_database(root, options)

Move a previous run’s job database into previous/<UTC time>/.

Only when the job database is <root>/seamm.db and it exists; the caller has decided not to resume from it (see plan_start()). The checkpoint mirror goes with it. Returns the directory it was moved to, or None.

class seamm_exec.exec_flowchart.cd(newPath)

Bases: object

Context manager for changing the current working directory

seamm_exec.exec_flowchart.comparable_command_line(cmdline)

The command line as recorded in a checkpoint: without –resume.

seamm_exec.exec_flowchart.credential_sections(root)

The seammrc sections to look in for datastore credentials, in order.

[Dashboard: <name>] where name is the installation’s root directory name (SEAMM_DEV, SEAMM_NEW, …), then, as before, dev for a root whose path contains “dev” or localhost otherwise, then this host’s name.

seamm_exec.exec_flowchart.flowchart_problems(flowchart)

Check the flowchart as a whole before running it.

Uses seamm.builder.validate, which checks the order of the steps and asks each step that can (e.g. an analysis step that needs the MD before it to sample the heat flux) to check itself. A failure of the check itself is logged, not reported, so that it never stops a job that could run.

Returns:

A description of each problem; empty if none.

Return type:

[str]

seamm_exec.exec_flowchart.get_job_id(filename)

Get the next job id from the given file.

This uses the fasteners module to provide locking so that only one job at a time can access the file, so that the job ids are unique and monotonically increasing.

seamm_exec.exec_flowchart.open_datastore(root, datastore, timeout=20.0)

Open the database via the datastore

seamm_exec.exec_flowchart.plan_start(root, options, flowchart, cmdline)

Decide how a run starts: resume from the checkpoint, or from the top.

Resuming is never the silent default for a rerun in place: the checkpoint cannot tell whether the files the flowchart reads have changed. A run that does not ask to resume sets a resumable checkpoint aside with the rest of the previous database, and says how to resume instead.

Parameters:

• root (str or pathlib.Path) – The job directory.

• options (dict) – The parsed options.

• flowchart (seamm.Flowchart) – The flowchart to run.

• cmdline ([str]) – The command-line arguments.

Returns:

resume: the checkpoint to resume from, or None; checkpointing: whether this run writes checkpoints (only for a job database in the job directory, writable and on disk); message: what to tell the user, or None.

Return type:

dict

seamm_exec.exec_flowchart.previous_directory(root)

<root>/previous/<UTC time>, where this run sets aside what an earlier run in the directory left: its job database and job.out. The same directory for everything this run sets aside.

seamm_exec.exec_flowchart.resume_requested(options)

Whether this run was asked to resume: –resume, or SEAMM_RESUME=1.

The JobServer asks with the environment variable when it resubmits a job that was lost, so that an evaluator too old to know about resuming simply starts from the top instead of failing on an unknown option.

seamm_exec.exec_flowchart.run(job_id=None, wdir=None, db_path=None, setup_logging=True, in_jobserver=False, cmdline=None)

The standalone flowchart app

seamm_exec.exec_flowchart.run_from_jobserver()

Helper routine to run from the JobServer.

Gets the arguments from the command line.

seamm_exec.iteration module

One iteration of a parallel Loop, run by an evaluator of its own (phase 6).

The parent’s Loop sets an iteration up, then writes, in the iteration’s _evaluator/ directory, a snapshot of the job database holding the checkpoint that resumes the flowchart into that iteration only. It runs the iteration as a Task of the program seamm, whose resolver here finds run_flowchart on the machine that runs it. The child keeps its step directories in the parent’s tree and its own files (database, output, job-level files) in _evaluator/.

When an iteration is done the parent merges what it did, in iteration order: the database (molsystem.snapshot.merge(), in the parent’s transaction), then, after that transaction is committed, its job-level files and its citations (plan_files(), merge_files(), merge_citations()).

See docs/developer_guide/campaigns/2026-10-02/NOTES_phase6.rst.

seamm_exec.iteration.APPENDED_RECORD = 'appended_files.json'

Write Structure’s record of the files it appended to, in its step directory

seamm_exec.iteration.CE_ENVIRONMENT = 'SEAMM_CE'

The child’s share of the machine, as JSON (see computational_environment)

seamm_exec.iteration.EVALUATOR_DIRECTORY = '_evaluator'

Where an iteration’s evaluator keeps its own files, in the iteration directory

seamm_exec.iteration.PARENT_JOB_ENVIRONMENT = 'SEAMM_PARENT_JOB'

The parent job’s directory, relative to the child’s, which puts an evaluator into child mode

seamm_exec.iteration.PROGRAM = 'seamm'

The program of an iteration’s task, and its resolver’s name

seamm_exec.iteration.READ_ENVIRONMENT = 'SEAMM_JOB_READ'

the enclosing iterations’ job directories, then the job’s (os.pathsep-separated, relative to the child’s directory)

Type:

Where a child reads job-level files it has not written

seamm_exec.iteration.decode_merge_state(data)

The inverse of encode_merge_state().

seamm_exec.iteration.encode_merge_state(state)

The merge’s state between iterations (tuple keys), for the checkpoint.

seamm_exec.iteration.export_path(filename, evaluator, job_directory)

Where a file the iteration wrote at filename belongs in the job.

seamm_exec.iteration.iteration_outcome(evaluator)

How an iteration ended, from its final checkpoint, or None if it did not.

Returns:

{"done": True, "break": bool, "skip": bool} and run_id.

Return type:

dict or None

seamm_exec.iteration.iteration_task(key, evaluator, job_directory, command_line=(), cores=1, memory=None, walltime=None, placement='inline', root_directory=None, read_directories=None, estimated_seconds=None)

The task that runs one iteration in its _evaluator directory.

Parameters:

• key (str) – The task’s key, unique in the Loop’s task set.

• evaluator (str or Path) – The iteration’s _evaluator directory, holding its snapshot.

• job_directory (str or Path) – The parent evaluator’s job directory, holding flowchart.flow (and target.json).

• command_line ([str]) – The parent’s command-line arguments (as in its checkpoint).

• cores (int) – Cores for the iteration.

• memory (int, optional) – Memory for the iteration, in bytes.

• walltime (float, optional) – An estimate of the iteration’s time, in seconds: the same as estimated_seconds, which it is kept for. (The Loop’s “time per iteration” is an estimate; as a hard walltime it got no margin, no guards, and the same walltime again after running out of time.)

• placement (str) – “inline”: the iteration’s codes run in its own share of the machine; “separate”: they go to the job’s target like any job’s.

• root_directory (str or Path, optional) – Where the steps’ directories are, by default job_directory: the job’s, also for a parallel loop nested in an iteration of another.

• read_directories ([str or Path], optional) – Where the child reads job-level files it has not written, innermost first; by default job_directory.

• estimated_seconds (float, optional) – An estimate of the iteration’s time, in seconds. A bundle asks for twice its tasks’ estimates plus ten minutes, within the target’s guards, and more after running out of time.

seamm_exec.iteration.merge_citations(evaluator, references, before=None)

Cite in the job’s references what the iteration cited.

Parameters:

• evaluator (str or Path) – The iteration’s _evaluator directory.

• references (reference_handler.Reference_Handler) – The job’s.

• before (dict, optional) – From plan_citations(): the counts are set to these plus the iteration’s, so doing it twice gives the same counts.

seamm_exec.iteration.merge_database(system_db, evaluator, state, iteration, later_wins=False)

Merge the iteration’s database into the job’s, in its open transaction.

See molsystem.snapshot.merge(); the result also has exports, the tables the iteration exported, {name: filename in the parent's job}.

seamm_exec.iteration.merge_files(evaluator, job_directory, plan)

Do a plan from plan_files(). Doing it again gives the same files.

seamm_exec.iteration.plan_citations(evaluator, references)

The job’s count of each citation the iteration made, before merging it.

Kept in the committed plan, so merging the citations again after an interruption sets the same counts rather than adding the iteration’s twice.

seamm_exec.iteration.plan_files(evaluator, job_directory, run_id)

What merging the iteration’s job-level files will do, before doing it.

The plan is kept in the parent’s checkpoint, committed before the files are touched, so a merge interrupted part way is redone exactly: each file the iteration appended to is first cut back to its size in the plan.

Returns:

{"append": {name: [offset in the iteration's file, size of the job's file before]}, "copy": [name, ...]}, names relative to the job.

Return type:

dict

seamm_exec.iteration.resolve(config, cmd, env, ce, root)

The resolver of the program seamm: an evaluator for one iteration.

code is this machine’s run_flowchart unless <root>/seamm.ini names one. The child’s share of the machine goes in its environment, so its own pool runs its codes within that share.

seamm_exec.iteration.run_flowchart_path()

run_flowchart of this Python, else the one on the PATH.

seamm_exec.local module

The Local object does what it name implies: it executes, or runs, an executable locally.

class seamm_exec.local.Local(logger=<Logger seamm-exec (WARNING)>)

Bases: Base

exec(config, cmd=[], directory=None, input_data=None, env={}, shell=False, ce={})

Execute a command directly on the current machine.

Parameters:

• config (dict(str: any)) – The configuration for the code to run

• cmd ([str]) – The command as a list of words.

• directory (str or Path) – The directory for the tasks files.

• input_data (str) – Data to be redirected to the stdin of the process.

• env ({str: str}) – Dictionary of environment variables to pass to the execution environment

• shell (bool = False) – Whether to use the shell when launching task

• ce (dict(str, str or int)) – Description of the computational enviroment

Returns:

{str – Dictionary with stdout, stderr, returncode, etc.

Return type:

str}

property name

The name of this type of executor.

seamm_exec.local_pool module

LocalPool: run tasks concurrently on this machine or in this allocation.

The pool’s capacity is the computational environment – the whole machine, or the SLURM allocation the evaluator runs in – and each task gets a share of it from its Resources. Tasks start, in the order submitted, as soon as enough cores and memory are free; a task larger than the pool is clamped to it and runs alone.

Each task runs through the executor’s _run_task() (the body of the original Base.run()), so the conda and modules handling, in_situ and the return-file contract are exactly those of Base.run().

class seamm_exec.local_pool.LocalPool(executor, *, root=None, ce=None, synchronous=False, max_concurrent=None, resolve_programs=False)

Bases: object

A back end that runs tasks as local subprocesses.

Parameters:

• executor (seamm_exec.Base) – Provides _run_task() and exec().

• root (str or Path, optional) – Where the <program>.ini files are, for tasks without config.

• ce (dict, optional) – The computational environment to share out. Default: computational_environment().

• synchronous (bool = False) – Run each task in submit(), in the caller’s thread, with the given ce unchanged, the environment untouched and exceptions propagated: exactly the original Base.run().

• max_concurrent (int, optional) – At most this many tasks at once.

• resolve_programs (bool = False) – Kept for compatibility; it changes nothing now. A task without config is always configured here, from <root>/<program>.ini and the program’s resolver (see seamm_exec.resolve); a task with config is run with it exactly as given.

cancel(ids)

capacity()

The cores, memory (bytes) and GPUs this pool shares out.

config_for(task)

The program’s configuration: task.config, else <program>.ini.

Adds code_dir, the directory holding code, for commands that run a code’s companion programs (e.g. ORCA’s orca_2aim), when code is a path rather than a bare name.

fetch(task, backend_id)

has_program(task)

Whether the task’s program can run here.

name = 'local'

reattach(records)

Tasks from an earlier evaluator: kill any still running here; lost.

A process that is not our child cannot be waited for, so instead of adopting it the pool kills its process group and the task is rerun. A pid is reused once its process has gone, so the group is killed only if its leader is the process the manifest recorded: same start time and working directory. (The host name check means a laptop that changed networks leaves the leftovers alone, the safe direction.)

status(ids)

submit(tasks, directories, on_start=None)

wait(ids, timeout=None)

Block until one of ids is finished, timeout seconds pass, or wake() is called.

wake()

Make a wait() return early (a TaskSet being cancelled).

seamm_exec.partial_outputs module

Copies of a running task’s outputs in its task directory.

A task that runs in a temporary directory – node-local scratch under a scheduler – has its returned files put in place only when it finishes. One stopped before then, by the job’s time limit or a cancel, loses them with the scratch: ten large ORCA calculations that timed out after 11-21 h left no orca.out to tell where the time went (MBE, 2026-10-23).

So while such a task runs, its returned files are copied to the task directory every DEFAULT_INTERVAL seconds as <name>.partial, and once more when the evaluator is told to stop (SIGTERM, see local_pool._on_signal). A task that finishes puts its files in place as before and removes the copies. Large files (over MAX_BYTES, e.g. MD trajectories) are not copied.

class seamm_exec.partial_outputs.Copier(source, destination, patterns, every=None, max_bytes=50000000)

Bases: object

Copy a running task’s returned files to its task directory as they grow.

Parameters:

• source (str or pathlib.Path) – Where the task runs.

• destination (str or pathlib.Path) – The task directory.

• patterns ([str]) – The task’s returned files (glob patterns).

• every (float or None) – Seconds between copies; None for interval(), 0 for none.

• max_bytes (int) – Larger files are not copied.

copy()

Copy the files that changed since the last copy; never raises.

remove()

Remove the copies, once the task’s own files are in place.

start()

Copy periodically until stop(), and on copy_all().

stop()

Stop copying; the copies stay (see remove()).

seamm_exec.partial_outputs.DEFAULT_INTERVAL = 600.0

Seconds between copies; the environment variable SEAMM_PARTIAL_INTERVAL overrides it, and 0 turns the periodic copies off (the copy on SIGTERM stays).

seamm_exec.partial_outputs.MAX_BYTES = 50000000

Files larger than this are not copied

seamm_exec.partial_outputs.SUFFIX = '.partial'

Appended to the name of each copy

seamm_exec.partial_outputs.copy_all()

Copy the outputs of every running task now, e.g. when told to stop.

seamm_exec.partial_outputs.interval()

Seconds between the copies, from SEAMM_PARTIAL_INTERVAL or the default.

seamm_exec.resolve module

Resolve a program on the machine that runs it.

A task names a program (“orca”, “mopac”, …). Where it runs, the program’s configuration comes from that machine’s <root>/<program>.ini (the section for the executor, normally [local]), and a plug-in may register a resolver that adjusts the configuration, the command and the environment for that machine and the task’s share of it: ORCA must be invoked by its full path, and a parallel run needs the OpenMPI it was built against on the paths.

Resolvers are entry points in the group org.molssi.seamm.exec.resolvers, named after the program:

[project.entry-points."org.molssi.seamm.exec.resolvers"]
orca = "orca_step.resolver:resolve"

with the signature

def resolve(config, cmd, env, ce, root) -> (config, cmd, env)

config is the program’s ini section as a dict (already a copy), cmd the task’s command template (a list), env its extra environment, ce the task’s computational environment (NTASKS, MEM_PER_CPU, …) and root the SEAMM root holding the ini files. The command is formatted with config and ce afterwards, by the executor, exactly as before.

Task.config is never sent to a remote back end (transport = ssh): it was resolved on the evaluator’s machine and names that machine’s paths, so the TaskSet keeps such a task in the evaluator’s own pool, with a warning. A task without it is resolved where it runs, here. On the local transport Task.config, when given, is used as it is, as in the LocalPool.

seamm_exec.resolve.available(program, root)

Whether program can run here without an ini file: its resolver says so through an optional available(root) attribute.

seamm_exec.resolve.has_resolver(program)

Whether program has a registered resolver.

seamm_exec.resolve.read_config(program, root, section='local')

The [section] of <root>/<program>.ini as a dict, or None.

seamm_exec.resolve.register(program, function)

Register a resolver by hand (tests, or a program without a plug-in).

seamm_exec.resolve.resolve(program, config, cmd, env, ce, root)

Apply the program’s resolver, if it has one.

Returns:

The configuration, command and environment to run with.

Return type:

(dict, list, dict)

seamm_exec.resolve.resolvers()

The registered resolvers, {program: callable}.

Built once, under a lock: pool worker threads may ask at the same time, and a second build must not replace the dict (dropping a register()).

seamm_exec.scheduler_backend module

SchedulerBackend: run tasks as batch jobs of a queueing system.

Each call to SchedulerBackend.submit() is one bundle: one batch job whose allocation runs the bundle’s tasks through the task worker (python -m seamm_exec.task_worker bundle.json) in SEAMM mode, that is, through a LocalPool inside the allocation, with each program resolved from the <program>.ini files of the machine it runs on. The TaskSet decides the bundles (bundle_tasks, bundle_walltime) and how many it keeps in the queue (room()).

Files: a bundle lives in <step dir>/tasks/_bundles/<bundle>.<n>/ (bundle.json, run.sh and the scheduler’s log), the tasks in their own directories, and the worker writes each task’s DONE or FAILED in <step dir>/tasks/<key>/. When the cluster does not share the evaluator’s filesystem the directories are pushed before submission and pulled back once the bundle’s job has ended, to <remote_root>/<job>/..., the same relative paths under the remote job directory.

Backend ids are <job id>#<bundle>.<n>#<key>, so a restarted evaluator can find a task’s job, bundle and files from its manifest record alone (adopt()) and poll it rather than submit it again.

exception seamm_exec.scheduler_backend.QueueFull(message, maybe_submitted=False)

Bases: RuntimeError

The queue will not take more jobs now (it is full, or cannot be reached); the TaskSet holds the bundle and tries again later.

maybe_submitted is True when the bundle may have reached the queue (the connection dropped during sbatch): its tasks must then stay recorded as queued, so that a restart looks for the job by name.

class seamm_exec.scheduler_backend.SchedulerBackend(queue, *, name='queue', job_directory, stager=None, remote_job_directory=None, directives=None, setup=None, python=None, root=None, executor='local', accepts_config=True, bundle_walltime=None, max_queued=None, poll_interval=30.0, job_name_prefix='seamm', max_walltime=None, estimate_scale=None, min_task_walltime=None)

Bases: object

A TaskBackend that submits bundles of tasks to a queueing system.

Parameters:

• queue (seamm_scheduler.QueueBackend) – Submits, polls and cancels jobs (a scheduler plus a transport).

• name (str) – The name recorded in the manifest, queue:<target>.

• job_directory (str or Path) – The job’s directory. Task directories must be inside it when the cluster does not share the filesystem.

• stager (seamm_scheduler.JobStager, optional) – RsyncStager when the cluster does not share the filesystem. None (or a LocalStager) means shared storage.

• remote_job_directory (str, optional) – Where the job directory is mirrored on the cluster, if not shared.

• directives (dict, optional) – Site defaults, in the scheduler’s own spelling (the target section’s directive keys: partition, account, qos, export, …).

• setup (str, optional) – Shell lines run in the batch script before the worker (module loads).

• python (str, optional) – A Python with seamm_exec where the tasks run. Default: this one.

• root (str, optional) – The SEAMM root with the <program>.ini files where the tasks run.

• accepts_config (bool = True) – Whether Task.config is valid where the tasks run (local transport). On a remote cluster it is not sent.

• bundle_walltime (float, optional) – Seconds to request for a bundle whose tasks give no walltime.

• estimate_scale (float, optional) – Multiplies each task’s estimate when a bundle’s walltime comes from the estimates (default 1).

• min_task_walltime (float, optional) – The least seconds a bundle gives each task whose walltime comes from its estimate.

• max_queued (int, optional) – The most jobs this user may have queued and running at once.

• poll_interval (float = 30) – Seconds between polls of the queue.

abandon(records, keep=())

Cancel the jobs of records (tasks whose inputs have changed) unless a task in keep (ids adopted) still runs in the same job.

adopt(task, directory, marker, record)

Take back a task an earlier evaluator submitted, from its manifest record. Returns its id, or None if the record is not one of ours.

bundles = True

The TaskSet submits one call per bundle.

cancel(ids)

check(task, directory, marker)

Fail early, before anything is submitted, for a task this back end cannot run (a directory outside the job, without shared storage).

fetch(task, backend_id)

forget(backend_id)

Drop a task the TaskSet will submit again.

classmethod from_target(section, *, job_directory, root=None, executor='local')

The back end a target section with tasks = queue describes.

reason(backend_id)

Why a task is lost, for the manifest.

relative(path)

path relative to the job directory.

room()

How many more bundles may be submitted now, or None for no limit.

property shared

status(ids)

submit(tasks, directories, on_start=None, bundle=None, markers=None, on_prepared=None, bundle_walltime=None, walltime_scale=1.0)

Submit tasks as one bundle: one batch job. Returns their ids.

bundle_walltime (seconds) is the caller’s limit on a bundle, used to bound its time when the tasks give none. walltime_scale multiplies the time estimated for tasks that give none, e.g. 2 for a task whose last attempt ran out of time (still within bundle_walltime).

on_prepared(tasks, info) is called before sbatch with the bundle’s unique job name and directory, so the caller can record them: a restart that finds no job id looks the job up by that name.

timed_out(backend_id)

Whether a lost task’s job was stopped for running out of time.

wait(ids, timeout=None)

Sleep until the next poll is due.

where(path)

The path the cluster sees for a local path.

seamm_exec.scheduler_backend.parse_id(backend_id)

<job id>#<bundle>.<n>#<key> -> (job id, bundle dir name, key).

seamm_exec.scheduler_backend.remote_name(job_directory)

The name of a job directory on the cluster: its own name and a hash of its full path. Job numbers are unique only within one datastore, and several installations (~/SEAMM, ~/SEAMM_DEV, ChemAI) may share a remote_root; a hand run’s directory may be called anything.

seamm_exec.seamm_exec module

Provide the primary functions.

seamm_exec.seamm_exec.canvas(with_attribution=True)

Placeholder function to show example docstring (NumPy format).

Replace this function and doc string for your own project.

Parameters:

with_attribution (bool, Optional, default: True) – Set whether or not to display who the quote is from.

Returns:

quote – Compiled string including quote and optional attribution.

Return type:

str

seamm_exec.targets module

Find the job’s target: where its tasks run.

A target is a section of the JobServer’s <root>/<jobserver-name>.ini (see seamm_scheduler.config). The evaluator finds it, in order:

1. given explicitly (TaskSet(target=...), a name or a TargetSection);

2. <job dir>/target.json, written by the JobServer when it starts the job, so an evaluator running as a batch job on a cluster, which cannot read the JobServer’s ini file, still knows its target;

3. the environment variable SEAMM_TARGET, naming a section of <root>/<hostname>.ini (or of the file named by SEAMM_TARGETS), for runs by hand with run_flowchart;

4. none: tasks run in the evaluator’s own LocalPool, as before targets existed.

seamm_exec.targets.find_target(target=None, *, job_directory=None, root=None)

The target section for the job, or None.

Parameters:

• target (str or TargetSection, optional) – A section, or the name of one in <root>/<hostname>.ini.

• job_directory (str or Path, optional) – Where to look for target.json. Default: the current directory.

• root (str or Path, optional) – The SEAMM root holding the JobServer’s ini file.

Return type:

seamm_scheduler.TargetSection or None

seamm_exec.targets.write_target(section, job_directory)

Write <job dir>/target.json for a section (what the JobServer does).

seamm_exec.task_worker module

Run a bundle of tasks, one after another, inside one allocation.

Usage:

python task_worker.py bundle.json

This is the generic worker that a scheduler back end submits for a bundle of small tasks. It is pure Python with no SEAMM imports, so it runs anywhere a python3 exists, and is run by path rather than imported.

bundle.json:

{
  "tasks": [
    {
      "key": "frag-0001",
      "directory": "/path/to/tasks/frag-0001",
      "command": "orca orca.inp > orca.out",   # fully resolved
      "shell": true,
      "env": {"OMP_NUM_THREADS": "1"},
      "input": null,
      "fingerprint": "sha256:...",
      "success_text": {"orca.out": "ORCA TERMINATED NORMALLY"}
    },
    ...
  ]
}

For each task, in order: if <directory>/DONE exists it is skipped; otherwise the command runs in the directory with its standard output and error in stdout.txt and stderr.txt, the return code goes in returncode, and a successful run – return code 0 and, if given, every success_text file containing its text – writes DONE (JSON). A failure does not stop the bundle. The worker exits 0 if every task succeeded and 1 otherwise.

The SEAMM mode

A bundle with "mode": "seamm" is run by a Python that has seamm_exec (python -m seamm_exec.task_worker bundle.json). Each task is then given as the task layer describes it – program, the cmd template, the names of its input files (already in its directory), return_files, resources, in_situ – and runs through a LocalPool sized to the allocation, exactly as the evaluator’s own pool would run it: the program resolved from <root>/<program>.ini on this machine (and its resolver, see seamm_exec.resolve), {code}/{NTASKS} from the task’s share of the allocation, scratch in $TMPDIR, and only return_files kept. Tasks that fit side by side in the allocation run concurrently.

Each task’s marker directory (<step dir>/tasks/<key>) receives DONE (JSON, in the task layer’s own form, including the list of returned files) or FAILED (JSON, with the reason). A task whose DONE exists is skipped, so a bundle that is submitted again – after a walltime limit, say – runs only what is left.

seamm_exec.task_worker.main(argv=None)

seamm_exec.task_worker.run_bundle(bundle)

Run the tasks of a bundle. Returns the number that failed.

seamm_exec.tasks module

Tasks: the units of work a step hands to a back end.

A step describes each external calculation as a Task – the program, a command template, the input files, the files to keep and the resources – and runs a group of them with a TaskSet, which submits what is not done to a back end (a TaskBackend, e.g. LocalPool), waits, and yields a TaskResult for each as it finishes.

With a step directory, the TaskSet keeps <step dir>/tasks/manifest.json and writes <step dir>/tasks/<key>/DONE when a task finishes, so a rerun of the step in the same directory never recomputes a finished task.

See docs/developer_guide/campaigns/2026-10-02 for the design.

class seamm_exec.tasks.Manifest(path, save_interval=1.0)

Bases: object

<step dir>/tasks/manifest.json: what the evaluator knows of its tasks.

One record per key: the back end, its id, the state, the fingerprint, the bundle, timestamps, the attempt count and history, and the archive.

flush(force=False)

Write the manifest if it changed, at most every save_interval.

The manifest is for reattaching and reporting; whether a task finished is recorded by its DONE file, which is written at once.

get(key)

save()

update(key, **values)

Change a record. It is written by the next flush().

class seamm_exec.tasks.Resources(ntasks: int | None = None, cpus_per_task: int = 1, mem_per_cpu: int | None = None, ngpus: int = 0, walltime: float | None = None, partition: str | None = None, account: str | None = None, qos: str | None = None, nodes: int | None = None)

Bases: object

What a task needs, in scheduler-neutral terms.

Each back end translates these: the LocalPool into a share of this machine or allocation, a scheduler into its directives.

ntasks = None means “all of the back end’s capacity” (for the LocalPool, the whole machine or allocation, as a code got before tasks). Memory is in bytes and walltime in seconds.

account: str | None = None

cpus_per_task: int = 1

mem_per_cpu: int | None = None

ngpus: int = 0

nodes: int | None = None

ntasks: int | None = None

partition: str | None = None

qos: str | None = None

walltime: float | None = None

class seamm_exec.tasks.Task(key: str, program: str, cmd: list = <factory>, files: dict | None = None, return_files: list = <factory>, resources: Resources = <factory>, env: dict = <factory>, in_situ: bool | None = None, shell: bool = False, input_data: str | None = None, estimated_seconds: float | None = None, estimate_source: str | None = None, target: str | None = None, directory: str | Path | None = None, config: dict | None = None, fingerprint: str | None = None, success_text: dict | None = None, keep: list = <factory>)

Bases: object

One external calculation.

Variables:

• key (str) – Unique within the step and stable across restarts (e.g. a fragment key). Letters, digits and ._+=@,-; it names the task’s directory.

• program (str) – The program’s identity (“orca”, “mopac”, …), which names its <root>/<program>.ini.

• cmd ([str]) – The command template; {code}, {code_dir}, {NTASKS}, … are filled in by the back end from the configuration and the task’s share of the resources.

• files ({str: str or bytes}) – Input files to write before running.

• return_files ([str]) – Globs of the files to keep; "@subdir+name" moves a file into a subdirectory, as with Base.run().

• resources (Resources)

• env ({str: str}) – Extra environment variables.

• in_situ (bool or None) – True runs in the task directory, leaving output there to watch; False in a temporary directory, copying back only return_files; None picks per Base.run() (scratch under a scheduler, in place otherwise).

• shell (bool)

• input_data (str or None) – Data for the standard input.

• estimated_seconds (float or None) – The step’s estimate of the cost, used by the inline rule for tiny tasks.

• estimate_source (str or None) – Where estimated_seconds came from: "model" (the fitted cost model), "formula" (the step’s own) or "none", as seamm_exec.timing_model.estimate() returns it; written in the timing record, so estimates from the fallback show.

• target (str or None) – Reserved: None is the job’s target.

• directory (str or Path or None) – Where the task runs and its results land. None means <step dir>/tasks/<key>/, the default for fan-out; a step running a single calculation passes its own directory so its output stays where it always was.

• config (dict or None) – A local override of the program’s configuration (the <program>.ini section), for steps that resolve it themselves. Remote back ends ignore it.

• fingerprint (str or None) – Identifies the inputs for restart. None hashes cmd and files; give one when the inputs carry run-dependent text (core counts, absolute paths).

• success_text ({str: str or [str]} or None) – For codes whose return code does not show failure (ORCA exits 0 after an error termination): each file must contain its text (or all of its texts), or the task failed, gets no DONE and is tried again on the next run.

• keep ([str]) – Paths in the task’s directory that the cleanup of a task run in place leaves as they are, without reading them as return_files are; "." keeps everything (an iteration of a parallel loop).

cmd: list

config: dict | None = None

digest()

The fingerprint of the inputs, used to detect a changed task.

directory: str | Path | None = None

env: dict

estimate_source: str | None = None

estimated_seconds: float | None = None

files: dict | None = None

fingerprint: str | None = None

in_situ: bool | None = None

input_data: str | None = None

keep: list

key: str

program: str

resources: Resources

return_files: list

shell: bool = False

success_text: dict | None = None

target: str | None = None

class seamm_exec.tasks.TaskBackend(*args, **kwargs)

Bases: Protocol

What runs tasks: the LocalPool now; a scheduler or TaskServer later.

A back end may also offer wait(ids, timeout) to block until one of ids changes state, reattach(records) -> {key: state} to recover tasks an earlier evaluator submitted, capacity() and has_program(task).

cancel(ids: list) → None

fetch(task: Task, backend_id: str) → TaskResult

name: str

status(ids: list) → dict

submit(tasks: list, directories: list, on_start=None) → list

Submit tasks; return their ids. on_start(task, info), if given, is called when a task’s process starts (info is what is needed to find it again, e.g. its process group).

class seamm_exec.tasks.TaskResult(key: str, state: str, returncode: int | None = None, stdout: str = '', stderr: str = '', directory: Path | None = None, files: dict = <factory>, attempts: int = 0, history: list = <factory>, in_situ: bool | None = None, run_directory: str | None = None, restored: bool = False, archive: Path | None = None, reason: str | None = None, timed_out: bool = False, started: float | None = None, finished: float | None = None, code: str | None = None, raw: dict | None = None)

Bases: object

The outcome of a task.

Variables:

• key (str)

• state (str) – finished | failed | cancelled | lost

• returncode (int or None)

• stderr (stdout,)

• directory (Path or None) – Where the task’s results are (for an archived task, where they were).

• files ({str: str}) – The returned files: for each name, where it is, relative to directory ("@subdir+name" is subdir/name). Their contents are not kept – use path() or read().

• attempts (int) – How many times the task has been submitted, over all runs.

• history ([dict]) – One record per attempt.

• in_situ (bool or None) – Whether it ran in place.

• run_directory (str or None) – Where it actually ran (a scratch directory when not in situ).

• restored (bool) – True when the result came from an earlier run rather than this one.

• reason (str or None) – Why a task failed: its return code, a failed success check, …

• archive (Path or None) – The tar holding the task’s directory, once archived.

• timed_out (bool) – Whether the queue stopped it for running past the time limit.

• code (str or None) – What ran the program – the modules loaded, the conda environment, or the command (see code_identity()) – recorded in DONE, so that a rerun can tell when reused results came from another version of the code.

• finished (started,) – When the task itself started and ended (epoch seconds), measured where it ran – for a task in a queue bundle, by the worker on the node. Its wall time for the timing records; None when the back end cannot tell.

• raw (dict or None) – The Base.run()-style dictionary, for this run’s results.

archive: Path | None = None

attempts: int = 0

code: str | None = None

directory: Path | None = None

files: dict

finished: float | None = None

classmethod from_raw(key, raw, directory=None)

Make a result from the dictionary Base._run_task() returns.

history: list

in_situ: bool | None = None

key: str

property ok

path(name)

Where a returned file is on disk, or None (missing, or archived).

raw: dict | None = None

read(name, binary=False)

The contents of a returned file, read now, from disk or its archive.

Parameters:

• name (str) – The name as returned, e.g. “orca.out” or “@2+md.trj”.

• binary (bool = False) – Return bytes rather than text.

Returns:

None if there is no such file.

Return type:

str, bytes or None

reason: str | None = None

restored: bool = False

returncode: int | None = None

run_directory: str | None = None

started: float | None = None

state: str

stderr: str = ''

stdout: str = ''

timed_out: bool = False

class seamm_exec.tasks.TaskSet(node=None, target=None, *, directory=None, executor=None, backend=None, local=None, root=None, manifest=True, archive=False, bundle_tasks=None, bundle_walltime=None, max_attempts=3, max_lost_retries=2, max_timeouts=4, inline_below=None, poll_interval=1.0)

Bases: object

What a step uses: add tasks, then iterate over their results.

run() yields finished tasks from earlier runs first (from their DONE markers, never recomputed), then submits the rest and yields each result as it completes. Within a run a task that fails (nonzero return code) is not retried; a lost one is, up to max_lost_retries. Across runs a failed or lost task is tried again until it has had max_attempts attempts in all. A local task that stopped because the evaluator stopped (killed, out of walltime) does not use up an attempt, nor does a task the step cancelled. Nor does running out of time, the first max_timeouts times: that is the estimate’s fault, not the task’s, and each retry asks for twice the time (within the queue’s limits). Later timeouts count, so a task that cannot finish even in the longest time allowed still stops.

Parameters:

• node (seamm.Node, optional) – The step. Gives the step directory, the executor, root and the job directory.

• target (str or seamm_scheduler.TargetSection, optional) – The job’s target. Default: found by seamm_exec.targets.find_target() (<job dir>/target.json, then $SEAMM_TARGET), else the local pool. Ignored when backend is given.

• directory (str or Path, optional) – The step directory, overriding node.directory.

• executor (seamm_exec.Base, optional) – Overrides node.flowchart.executor.

• backend (TaskBackend, optional) – Where tasks go. Default: the target’s back end (a SchedulerBackend for tasks = queue), else a LocalPool for this machine.

• local (LocalPool, optional) – The evaluator’s own pool, for the inline rule. Defaults to backend when that is a LocalPool, else one is made on demand.

• root (str or Path, optional) – Where the <program>.ini files are. Default node.global_options["root"].

• manifest (bool = True) – Keep the manifest and DONE markers. False only for Base.run().

• archive (bool = False) – Pack each bundle’s task directories into tasks/<bundle>.tar once all its tasks are done, and remove the directories.

• bundle_tasks (int, optional) – Tasks per bundle, in the order added. Default: the target’s bundle_tasks; else one bundle (one task per bundle on a queue without bundle_walltime).

• bundle_walltime (float, optional) – The longest walltime of a bundle, seconds: a new bundle starts when the time the bundle would ask for (bundle_request: the tasks’ walltimes, or twice their estimates and ten minutes) would be more. Default: the target’s.

• max_attempts (int = 3) – Attempts per task over all runs.

• max_lost_retries (int = 2) – Resubmissions of a lost task within one run.

• max_timeouts (int = 4) – Times a task may run out of time without using up an attempt; with the doubling, up to 16 times the first walltime.

• inline_below (float, optional) – Tasks estimated to take less than this many seconds run in the local pool rather than on a remote back end, if their program is installed here. Default: the target’s inline_below, else 60.

• poll_interval (float) – Seconds between status checks of a back end without wait().

add(task)

Add a task. Keys must be unique and filesystem-safe.

property backend

The back end for this TaskSet’s tasks (the job’s target).

cancel()

Stop this task set: may be called from another thread while run() is iterating (a sibling level failed, the step is being stopped).

run notices at its next poll, cancels every task in flight on its back end, drops those still held, marks them cancelled in the manifest – so a later run of the step submits them afresh rather than adopting a killed job – and yields a cancelled result for each before returning. Tasks that had already finished keep their results. Nothing is cancelled from the calling thread itself: the back ends’ bookkeeping belongs to the running loop.

property cancelled

Whether cancel() was called.

capacity()

The back end’s capacity, so a step can size its tasks beforehand.

Returns:

{"cores": int, "memory": bytes, "ngpus": int}

Return type:

dict

property local

The evaluator’s own LocalPool.

marker_directory(key)

<step dir>/tasks/<key>, holding the task’s DONE.

route(task)

The back end for a task: the job’s target, except for tiny tasks.

The inline rule: a task whose estimated_seconds is below inline_below runs in the evaluator’s own pool instead of a remote back end, provided its program is installed here and the task fits the pool: a task asking for more cores than the evaluator has goes to the back end however cheap it is, since its input may already say how many ranks to use (seamm_exec#38).

run() → Iterator[TaskResult]

Submit what is not done, wait, and yield results as they finish.

summary()

Counts by state, for the Dashboard and the step’s report.

Returns:

{"total": n, "finished": n, "failed": n, ...}

Return type:

dict

task_directory(task)

Where a task runs and its results land.

property tasks_directory

<step dir>/tasks

seamm_exec.tasks.WORKER_SCRIPT = PosixPath('/home/runner/work/seamm_exec/seamm_exec/seamm_exec/task_worker.py')

The pure-Python bundle worker, run by path (python task_worker.py bundle.json)

seamm_exec.tasks.run_task(task, node=None, **kwargs)

Run one task through a TaskSet and return its result.

The convenience for a step that runs a single calculation: it gets the manifest, restart and the pool’s handling of the process like any other task. kwargs go to TaskSet (e.g. directory= when the task runs somewhere other than node.directory).

Return type:

TaskResult

seamm_exec.testing module

Helpers for testing a code step’s run path end to end.

A test builds a small flowchart from a spec, as seamm-flowchart build does, and runs it with run_flowchart in a subprocess, the way a job runs. The code is either a fake – a script that writes a real run’s recorded output – so that the whole path runs in CI where the code is not installed, or the real program, found as SEAMM finds it, with the test skipped where it is not installed:

from seamm_exec.testing import fake_program, find_program, run_spec, table_value

SPEC = '''
title: MOPAC energy
steps:
- Water: {}
- MOPAC:
    steps:
    - Energy
'''

def test_with_a_fake(tmp_path):
    code = fake_program(tmp_path / "bin" / "mopac", files=DATA.glob("mopac.*"))
    job = run_spec(tmp_path, SPEC, inis={"mopac": {"code": code}})
    assert table_value(job / "job.out", "Enthalpy of Formation") == ...

Each run has its own HOME and SEAMM_ROOT under the test’s directory, so the user’s ~/.seamm.d (timing files, seammrc) and <root>/<code>.ini are never read or written, and the step’s <code>.ini is the one the test writes.

Water is a tiny step provided by run_spec() (on PYTHONPATH) that makes a water molecule at a fixed geometry, so that a code step’s test needs no structure step.

None of this imports pytest, so the module is importable anywhere; failures raise AssertionError with the end of the run’s output.

seamm_exec.testing.WATER_STEP = '\n"""A step for testing: water at a fixed geometry."""\n\nimport seamm\n\n\nclass Water(seamm.Node):\n    def __init__(self, flowchart=None, extension=None):\n        super().__init__(flowchart=flowchart, title="Water", extension=extension)\n\n    @property\n    def version(self):\n        return "0.1"\n\n    def description_text(self, P=None):\n        return self.header + "\\n    Water at a fixed geometry."\n\n    def run(self):\n        next_node = super().run(None)\n        db = self.get_variable("_system_db")\n        system = db.create_system(name="water")\n        configuration = system.create_configuration(name="water")\n        configuration.atoms.append(\n            x=[0.0, 0.757, -0.757],\n            y=[0.0, 0.586, 0.586],\n            z=[0.0, 0.0, 0.0],\n            symbol=["O", "H", "H"],\n        )\n        db.system = system\n        return next_node\n\n\nclass WaterStep:\n    my_description = {\n        "description": "Water for tests",\n        "group": "Building",\n        "name": "Water",\n    }\n\n    def __init__(self, flowchart=None, gui=None):\n        pass\n\n    def description(self):\n        return WaterStep.my_description\n\n    def create_node(self, flowchart=None, **kwargs):\n        return Water(flowchart=flowchart, **kwargs)\n\n    def create_tk_node(self, canvas=None, **kwargs):\n        raise NotImplementedError("no GUI for the test step")\n'

water at a fixed geometry (O at the origin).

Type:

Source of the Water step

seamm_exec.testing.fake_program(path, files=(), stdout=None, need_input=True)

Write an executable script at path that stands in for a program.

Parameters:

• path (str or Path) – Where to write it (its directory is created).

• files (iterable of paths) – Recorded output files, copied beside the input (the first argument that is an existing file), or into the working directory without one.

• stdout (path, optional) – A recorded file to print as the program’s standard output, for codes whose output the command redirects ({code} orca.inp > orca.out).

• need_input (bool) – Fail, as the program would, if the input file is missing or empty.

Returns:

The script’s path.

Return type:

str

seamm_exec.testing.find_program(program, ini=None, section='local', root=None)

The installed executable for program, or None, as SEAMM finds it.

In order: $<PROGRAM>_EXE; program on the PATH; the code named in the installation’s <ini>.ini [section] (an absolute path, or the executable in its conda-environment). The installation’s root is root, else $SEAMM_ROOT, else ~/SEAMM.

seamm_exec.testing.install_water_step(site)

Write the Water step and its entry points into the directory site, to be put on PYTHONPATH. Returns site.

seamm_exec.testing.run_spec(tmp_path, spec, inis=None, args=(), source=None, extra_path=(), timeout=600)

Build the flowchart spec and run it; return the job’s directory.

Parameters:

• tmp_path (str or Path) – A fresh directory for the run: job/ (the job), root/ (its SEAMM_ROOT, holding the ini files), home/ (its HOME) and site/ (the Water step).

• spec (str) – The flowchart spec, YAML as seamm-flowchart build takes it.

• inis ({str: {str: str}}, optional) – <code>.ini files to write in the root, each as the values of its [local] section, e.g. {"orca": {"code": "/path/to/orca"}} (installation = local is added unless given).

• args ([str]) – More arguments for run_flowchart, after the flowchart, e.g. ["--ncores", "1"].

• source (str or Path, optional) – A package’s source directory to put first on PYTHONPATH, so that the code under test is the one run.

• extra_path ([str or Path]) – More directories for PYTHONPATH.

• timeout (float) – Seconds allowed for each of the build and the run.

seamm_exec.testing.table_value(text_or_path, label)

The number in the last row of a results table whose label is label.

Steps print their results as tables, | label | value | units | (with | or the box-drawing │). text_or_path is the text or a file holding it, such as a job’s job.out.

seamm_exec.timing module

Timing records of the calculations steps run, shared by the plug-ins.

Each plug-in appends one row per calculation to ~/.seamm.d/timing/<program>.csv (the files Gaussian, LAMMPS, VASP and MOPAC already keep), which are fitted to estimate the cost of later calculations. Many runs append to the same file at once – on a cluster from many nodes, on a network file system – so rows are written under a lock and in one write, and the file is set aside when it grows past a size instead of growing without bound (vasp-step#18). A writer never waits long for the lock: if the file stays busy it writes the row to a file of its own in <program>.pending/, and moves its rows into the shared file the next time it gets the lock. Waiting on a network file system’s lock daemon held up every Evaluator of a cluster in turn, capping how fast it could take in its results. Rows should hold numbers that characterize the calculation (atoms, electrons, basis or cutoff, k-points, cores, node, wall time), not its input files.

A code step writes its row with record_task_timing() once it has run a seamm_exec.Task and parsed the output: the common columns (the machine class, the task’s resources, the wall time from the task manifest, the outcome) come from here, the descriptors of the calculation from the step. The design, the model fitted to the rows and the descriptors each code records are in docs/developer_guide/campaigns/2026-10-05.

seamm_exec.timing.COMMON_COLUMNS = ('schema', 'date', 'machine', 'cluster', 'partition', 'cpu_model', 'cpu_cores', 'gpu_model', 'host', 'program', 'ntasks', 'cpus_per_task', 'mem_per_cpu', 'ngpus', 'wall', 'estimated', 'estimate_source', 'state', 'timed_out', 'attempts', 'in_situ')

The common columns of a record, before the step’s descriptors

seamm_exec.timing.DEFAULT_DIRECTORY = PosixPath('~/.seamm.d/timing')

Where the timing files are

seamm_exec.timing.LOCK_WAIT = 1.0

How long a writer tries for the lock before writing to its own file, in seconds

seamm_exec.timing.MAX_BYTES = 52428800

A file larger than this is set aside (renamed with the date) and begun again

seamm_exec.timing.NEIGHBOUR_RADIUS = 8.0

The radius (Å) within which neighbours counts other atoms

seamm_exec.timing.NEIGHBOUR_SAMPLE = 256

At most this many atoms are the centres the mean is taken over

seamm_exec.timing.SCHEMA = 1

The version of the record written by record_task_timing()

seamm_exec.timing.append_timing(program, row, fieldnames=None, directory=None, max_bytes=52428800, lock_wait=1.0)

Append one timing row for program, safely with concurrent writers.

Parameters:

• program (str) – Names the file, <program>.csv.

• row (dict) – The row. Values are written as text; newlines in them are replaced by spaces, so a row is always one line.

• fieldnames ([str], optional) – The columns, in order, for a new file. Default: the row’s keys. A row is written with the file’s existing header, missing columns left empty. A row with columns the header lacks sets the file aside and begins a new one whose header is the old columns followed by the new, so a change of schema never loses columns and never mixes two schemas in one file.

• directory (str or Path, optional) – Default ~/.seamm.d/timing.

• max_bytes (int) – Set the file aside, as <program>-<YYYYmmdd-HHMMSS>[-n].csv, when it is larger than this.

• lock_wait (float) – How long to try for the lock, in seconds. If another writer holds it all that time, the row goes to this process’s own file in <program>.pending/ (see pending_paths()), which needs no lock. Rows waiting there are moved into the timing file, in the same single write, the next time this process gets the lock.

Returns:

The file written.

Return type:

pathlib.Path

Notes

The lock is a POSIX lockf lock, which network file systems honour through their lock daemon. It is only ever tried, never waited for: a waiting request is granted by a call back from the lock daemon, which on a busy cluster may not come for many seconds, and meanwhile the calling process – an Evaluator taking in results, say – does nothing else. The lock is the process’s, so the threads of a process take turns here.

seamm_exec.timing.machine_class(gpu_model=None)

The class of machine this process runs on: the columns that separate machines of different speed in the timing records.

Returns:

• dict – machine – the key, cluster:partition:cpu_model with empty parts omitted (and :gpu_model appended when given); plus cluster, partition, cpu_model, cpu_cores (physical cores of the node), gpu_model and host.

• The cluster and partition come from the scheduler’s environment; a laptop has

• neither, and its key is just the CPU model. The result is cached per process

• (except for the GPU, which depends on the task).

seamm_exec.timing.neighbour_count(coordinates, cell=None, radius=8.0)

The mean number of other atoms within radius Å of an atom.

How crowded a structure is: small and nearly constant along a chain, several times larger in dense 3D matter (a liquid, a protein, a crystal). The cost of a calculation with local orbitals or prescreened integrals – MOPAC’s MOZYME, a large ORCA basis – follows it as well as the size: a 1,000-atom water cluster took 8 times as long per atom with MOZYME as a 900-atom alkane chain.

coordinates: (n, 3) Å. cell: (3, 3) lattice vectors (rows, Å) for a periodic structure, else None. The mean is over up to NEIGHBOUR_SAMPLE atoms, spread evenly through the list, against all atoms, so it costs little even for 100,000 atoms. Returns a float, or None for a lone atom (no cell).

seamm_exec.timing.pending_directory(program, directory=None)

Where writers that could not get the lock put their rows, a file each.

seamm_exec.timing.pending_paths(program, directory=None)

The files of rows waiting to be moved into program’s timing file.

seamm_exec.timing.read_timings(program, directory=None, all_files=False)

The rows of program’s timing file, as dicts.

all_files=True also reads the files set aside (<program>-<stamp>.csv), oldest first, as a fit does; their columns may differ, so a row only has the keys of its own file. The rows still waiting in <program>.pending/ for writers to move them in (see append_timing()) are always read, after the timing file’s.

seamm_exec.timing.record_task_timing(task, result, descriptors=None, directory=None, spec=None)

Append the timing record of a task that has run.

Parameters:

• task (seamm_exec.Task) – The task as it was run: its program, resources and estimate.

• result (seamm_exec.TaskResult) – Its result: the wall time comes from the attempts’ history, the outcome from the state. A restored result (an earlier run’s, found again) is not recorded, and the function returns None.

• descriptors (dict, optional) – The step’s description of the calculation: numbers and short categorical values (see the campaign document for each code’s), written after the common columns. Keys that clash with a common column are ignored.

• directory (str or Path, optional) – The timing directory; default ~/.seamm.d/timing.

• spec (seamm_exec.timing_model.Spec or dict, optional) – The step’s spec of its cost model, written once beside the records.

Returns:

• pathlib.Path or None – The file written, or None if nothing was recorded.

• Never raises (a problem writing a timing record is logged, since it must not)

• stop the calculation that produced it.

seamm_exec.timing.record_timing(program, wall, descriptors=None, *, ntasks=None, cpus_per_task=None, mem_per_cpu=None, ngpus=0, gpu_model=None, estimated=None, state='finished', timed_out=False, attempts=1, in_situ=None, directory=None, spec=None, estimate_source=None)

Append the timing record of a run made outside the task layer.

For a step that still runs its code through executor.run and times it itself. The arguments are the common columns (see the module docstring); descriptors as for record_task_timing(); spec, the step’s seamm_exec.timing_model.Spec (or its dict) saying which descriptors the cost model is made of, written once beside the records; estimate_source, where estimated came from ("model", "formula"; see seamm_exec.timing_model.estimate()), else "none" without an estimate and empty when not known. Never raises.

seamm_exec.timing.structure_descriptors(configuration, atom_indices=None, ghost_atoms=None)

The descriptors of a structure every code’s record shares.

n_atoms and n_heavy (atomic number > 1) of the atoms the code was given (atom_indices, default all; ghost_atoms excluded and counted as n_ghosts), n_electrons (the atoms’ electrons less the charge), charge, multiplicity, periodicity and, for a periodic system, volume (Å^3); and neighbours, the mean number of other atoms within NEIGHBOUR_RADIUS (8 Å) of an atom – the code’s atoms, ghosts included, which have basis functions – how crowded the structure is (neighbour_count()). Never raises: a missing attribute leaves its key out.

seamm_exec.timing.task_wall_seconds(result)

The wall time of a task’s last attempt, from its history, or None.

seamm_exec.timing.timing_path(program, directory=None)

The timing file of program.

seamm_exec.timing_benchmark module

The seed benchmark: standard calculations that place a machine class in the cost model (campaign docs/developer_guide/campaigns/2026-10-05, section 5).

A machine the model has never seen gets its offset from a few minutes of standard runs; a sweep of core counts gives the parallel exponent that production records, whose core count usually follows the size, cannot. The runs are ordinary flowchart runs; their timing rows carry benchmark=<set> so a fit can tell them from production runs (they are chosen to span the space) and so a code or compiler change can be checked against an earlier set.

Each code step declares its own benchmark, as it declares its cost-model spec: a TIMING_BENCHMARK dict at the top of its package, found here through the org.molssi.seamm entry points. Nothing chemical lives in seamm_exec; a code knows what drives its own cost (molecules for ORCA, periodic boxes for LAMMPS, cells and k-points for VASP). The declaration:

TIMING_BENCHMARK = {
    "program": "orca",        # the timing records' program (orca.csv)
    "step": "ORCA",           # the step's name in a flowchart spec
    "section": "orca-step",   # its options section (default <program>-step)
    "parallel": True,         # sweep the core counts, or run once
    "systems": [              # in order of size; "size" is the number the
        {                     # limits below are compared with
            "name": "water",
            "size": 3,
            "steps": [{"FromSMILESStep": {"smiles string": "O"}}],
        },
        ...
    ],
    "chemistries": {          # each with the largest size it runs at, per tier
        "ORCA:DFT@B3LYP/def2-SVP": {"quick": 24, "full": 62},
        # optionally only some of the tasks below, e.g. a basis-set ladder
        "ORCA:DFT@B3LYP/def2-QZVPPD": {"quick": 15, "full": 24,
                                      "tasks": ["Energy"]},
        # and its own variants, replacing the task's below for it: e.g.
        # one option toggled ("NoCOSX" on and off), so that the fit can
        # tell what the option costs from what the molecule does
        "ORCA:DFT@B3LYP/def2-TZVP": {"quick": 15, "full": 24,
                                    "variants": {"Energy": [
                                        {}, {"extra keywords": "NoCOSX"}]}},
    },
    "parameter": None,        # or the sub-step parameter the chemistry sets
                              # (MOPAC's "hamiltonian") when the step does
                              # not take the Model Chemistry
    "tasks": {                # sub-steps, each with its largest size per tier
        "Energy": {"quick": 62, "full": 302},
    },
    "variants": {             # optional: a task run again with these
        "Energy": [{}, {"results": {"gradients": {}}}],   # parameters;
    },                        # "_min_size"/"_max_size" bound a variant
}

A system’s steps are the flowchart steps that build it, as a spec writes them; the quick tier keeps a code to a few minutes on one node. The flowchart is built from the declarations with the installed plug-ins and run once per core count with SEAMM_CE capping the cores the codes see:

python -m seamm_exec.timing_benchmark --codes orca,mopac --cores 1,4,8 --fit
python -m seamm_exec.timing_benchmark --build-only -o timing_benchmark.flow

seamm_exec.timing_benchmark.build_flowchart(spec_text, output)

Build the flowchart from the spec with seamm-flowchart build.

seamm_exec.timing_benchmark.build_spec(codes=None, tier='quick')

The YAML spec of the benchmark flowchart for codes (default every code with a declaration) at tier: each code’s systems in order of size, each built once, then its chemistries, tasks and variants up to the tier’s size limits.

seamm_exec.timing_benchmark.declarations(refresh=False)

{program: TIMING_BENCHMARK} of the installed code steps that declare one, found through the org.molssi.seamm entry points (a step’s package exports TIMING_BENCHMARK). Cached; refresh looks again.

seamm_exec.timing_benchmark.main(argv=None)

seamm_exec.timing_benchmark.performance_cores()

The cores a core sweep may use: on Apple silicon the performance cores only (hw.perflevel0.physicalcpu; a run on more adds efficiency cores, whose different speed would distort the parallel exponent), elsewhere the physical cores.

seamm_exec.timing_benchmark.run(codes=None, cores=(1, 4, 8, 16), tier='quick', directory=None, set_id=None, fit_after=False, dry_run=False)

Build the benchmark flowchart and run it once per core count.

SEAMM_CE caps the cores (and memory) the codes see in each run, and each code’s own ncores option is set on the command line (<code>-step --ncores N), since a machine’s seamm.ini may pin it; SEAMM_TIMING_BENCHMARK marks the timing rows. Core counts above this machine’s usable cores (its performance cores on Apple silicon, else its physical cores) are skipped; a code that is not parallel runs once.

seamm_exec.timing_check module

How well the estimates of recent runs matched their times.

Every timing record carries the estimate the task was given (estimated, before a target’s estimate_scale) and the time it took (wall). After an update, or whenever packing or walltimes look wrong, this compares them over the newest records, grouped by machine and calculation, and flags

• groups whose estimates are far from the times (by default more than 2x),

• groups with estimates from a step’s fallback formula rather than the model (the records’ estimate_source), and

• calculations estimated the same on machines whose times differ: the sign that the estimates came from a step’s fallback formula, which knows nothing of the machine, instead of from the fitted model.

Command line:

seamm-timing check [program ...] [--since 2h]

seamm_exec.timing_check.CALCULATION_COLUMNS = ('task', 'method', 'basis', 'n_atoms')

Columns that describe the calculation, when the records have them

seamm_exec.timing_check.DIFFERENT = 1.25

… while times further apart than this factor are different

seamm_exec.timing_check.LIMIT = 2.0

A group is flagged when its estimates are further than this factor from the times (the geometric mean of estimate / wall)

seamm_exec.timing_check.SAME = 0.01

Estimates closer than this fraction count as the same

seamm_exec.timing_check.check(program, directory=None, since=None, by=None, limit=2.0, rows=None)

Compare the estimates of program’s recent runs with their times.

Parameters:

• program (str)

• directory (str, optional) – The timing directory.

• since (datetime, optional) – Only runs recorded at or after this (aware) time.

• by (sequence of str, optional) – The columns describing a calculation; default the method class, task, method, basis and atom count, those the records have. The machine is always the first.

• limit (float) – The factor beyond which a group’s estimates are flagged.

• rows (list of dict, optional) – The records, instead of reading them.

Returns:

groups: a list of dicts, one per machine and calculation, with machine, key (the calculation’s values), rows, wall (mean seconds), estimated (mean seconds), ratio (geometric mean of estimate / wall), low and high (its 10th and 90th percentiles), distinct (the number of different estimates), sources (rows per estimate_source, when recorded) and flags; columns (the calculation’s columns), checked, no_estimate and timed_out (counts of runs).

Return type:

dict

seamm_exec.timing_check.parse_since(text, now=None)

The time text names, as an aware datetime.

text is either a span back from now – a number and m, h or d (90m, 2h, 1d) – or a date and time, 2026-10-16T10:30, in local time unless it gives its offset. The records’ dates are in UTC.

seamm_exec.timing_check.report(summary, min_rows=1)

The text of a check(): a table of the groups, flags last.

seamm_exec.timing_model module

Fit a cost model to the timing records and predict the time of a calculation.

The model (campaign docs/developer_guide/campaigns/2026-10-05) is a product of separable factors, fitted as a sum in log space:

log((t - t0) / units) = a_class + sum_i b_i log(size_i) - alpha log(cores)
                        + s_machine + task_offset + noise

• t0: a start-up constant per machine class, taken from the smallest runs and subtracted outside the log, so sub-second runs do not bend the power law.

• units: the iterations the run took (SCF runs, ionic steps, MD steps …), so the fit learns a unit cost; the distribution of units per task is kept and entered at the requested quantile when predicting.

• size_i: the program’s size variables (basis functions, electrons, atoms; for plane waves the valence electrons and the grid volume).

• alpha: the parallel exponent, fitted when the records span core counts.

• s_machine: a per-machine-class offset, shrunk toward zero when a class has few rows (the calibration a new machine gets from a few standard runs).

The fit is ridge regression in numpy; the model is a JSON file, ~/.seamm.d/timing/models/<program>.v<N>.json, and predict() evaluates it with numpy alone. The prediction is a quantile of the residual distribution (the 95th for a queue walltime), never the mean.

Command line (seamm-timing, or python -m seamm_exec.timing_model):

seamm-timing fit [program ...] [--directory DIR]
seamm-timing predict orca task=energy nbf=240 ... [-q 0.95]
seamm-timing check [program ...] [--since 2h]

seamm_exec.timing_model.CORES_SIZE_CORRELATION = 0.7

Ridge penalty on the slopes and class/task offsets |corr(log cores, log size)| above which the parallel exponent is not fitted

seamm_exec.timing_model.EXTRAPOLATION = 1.5

How far outside the fitted size range (as a factor) a prediction is still made

seamm_exec.timing_model.FALLBACK_SPECS = {'dftbplus': Spec(size=('n_atoms',), klass=('model',), task='task', units='scc_cycles', multiplier=None, default_alpha=0.8, slope_by=None, setup_by=None, flags=None), 'gaussian': Spec(size=('nbf', 'n_electrons', 'n_atoms'), klass=('method',), task='task', units='scf_runs', multiplier=None, default_alpha=0.8, slope_by=None, setup_by=None, flags=None), 'lammps': Spec(size=('n_atoms',), klass=(), task=None, units='md_steps', multiplier=None, default_alpha=0.8, slope_by=None, setup_by=None, flags=None), 'mopac': Spec(size=('n_basis', 'n_atoms'), klass=('hamiltonian', 'regime'), task='task', units='geometry_cycles', multiplier=None, default_alpha=0.0, slope_by='regime', setup_by='regime', flags=None), 'orca': Spec(size=('nbf', 'n_electrons', 'n_atoms'), klass=('method_class',), task='task', units='scf_runs', multiplier=None, default_alpha=0.8, slope_by=None, setup_by=None, flags=None), 'psi4': Spec(size=('nbf', 'n_electrons', 'n_atoms'), klass=('method',), task='task', units='scf_runs', multiplier=None, default_alpha=0.8, slope_by=None, setup_by=None, flags=None), 'vasp': Spec(size=('nelect', 'grid', 'volume'), klass=('model',), task='task', units='ionic_steps', multiplier='kpoints', default_alpha=0.5, slope_by=None, setup_by=None, flags=None)}

Specs for the code steps of 2026.10.6, which wrote records before the steps declared their own: used only when a program has no <program>.spec.json. Remove once every step writes its spec.

seamm_exec.timing_model.FLAG_MIN_CONTRASTS = 3

… and only when this many calculations – the same machine, method class, task and sizes – ran both with and without it. A flag that always comes with its own calculations (one campaign’s settings, a molecule’s elements) cannot be told from them: on ARC’s ORCA records every flag was like that, and the factors fitted anyway took machine and campaign differences (Owl’s production predictions fell from 96 % to 92 % within 2x). The timing benchmarks’ toggle ladders make the contrasts.

seamm_exec.timing_model.FLAG_MIN_ROWS = 10

A flag (Spec.flags) gets a factor only with this many runs with it, and as many without it …

seamm_exec.timing_model.FLAG_RIDGE = 0.01

a flag the records do not clearly need stays near no effect

Type:

The ridge penalty on a flag’s factor, per row

seamm_exec.timing_model.FLAG_UNKNOWN_SPREAD = 0.15

The added log-spread of a prediction (above the median) per flag the records have never seen, and its most

seamm_exec.timing_model.MACHINE_SHRINK = 5.0

Shrinkage (in rows) of a machine class’s offset toward the pooled fit

seamm_exec.timing_model.MODELS_SUBDIR = 'models'

<timing directory>/models/<program>.v<N>.json

Type:

Where the fitted models go

seamm_exec.timing_model.MODEL_VERSION = 3

The model file’s format, and the fit that made it. Each version keeps its own file, so that a process running an older seamm-exec (a long job pinned to an older environment) cannot replace a model with one from its older fit. Version 3: an assumed parallel exponent is applied in the fit as in predict().

seamm_exec.timing_model.PAIRED_MIN_GROUPS = 4

Paired runs – the same calculation on several core counts – needed to take the parallel exponent from them rather than from the main fit

seamm_exec.timing_model.PARALLEL_FRACTIONS = (0.0, 0.5, 0.75, 0.9)

The parallel start-up tried per machine, as fractions of the 5th percentile of the net times of its smallest parallel runs. A parallel run pays a fixed cost serial ones do not – launching the MPI processes, about 9 s on Owl and 3-4 s on TinkerCliffs for ORCA, nearly the same for 4 or 16 of them – which a power law in the cores cannot describe (tiny molecules ran slower on more cores). A large fraction on a machine whose smallest parallel runs are long hurts their fit and is not chosen.

seamm_exec.timing_model.QUANTILES = (0.5, 0.68, 0.84, 0.95, 0.99)

Residual quantiles stored in a model

seamm_exec.timing_model.REFIT_DAYS = 7.0

… or when it is older than this many days and the records have changed at all

seamm_exec.timing_model.REFIT_GROWTH = 0.2

A model is refitted by predict() when its records have grown by this fraction since the fit …

seamm_exec.timing_model.SETUP_CANDIDATES = (0.0, 0.5, 1.0, 2.0, 5.0, 10.0, 20.0, 50.0, 100.0, 200.0)

The setups tried for each group of Spec.setup_by, in iterations

class seamm_exec.timing_model.Spec(size: tuple = ('n_atoms',), klass: tuple = (), task: str | None = 'task', units: str | None = None, multiplier: str | None = None, default_alpha: float = 0.8, slope_by: str | None = None, setup_by: str | None = None, flags: dict | None = None)

Bases: object

What the cost model of a program is made of: plain data a code step declares and passes when it records a run (record_timing(..., spec=...)), written beside the records as <program>.spec.json so the fit reads it without importing the plug-in.

Variables:

• size (tuple of str) – Descriptor columns entering as log(size), most informative first (basis functions, electrons, atoms; valence electrons and the grid volume for plane waves – a computed variable is written as a descriptor by the step).

• klass (tuple of str) – Columns whose joined values are the method class (an intercept each).

• task (str or None) – The column naming the kind of task (an offset each, and the unit distribution).

• units (str or None) – The column holding the iterations the run took (SCF runs, ionic steps, MD steps). Missing or zero counts as one.

• multiplier (str or None) – A column the time is proportional to besides units (k-points).

• default_alpha (float) – The parallel exponent used when the records do not span core counts.

• slope_by (str or None) – A column whose values each get their own size exponents – for a code with regimes that scale differently, such as MOPAC’s MOZYME (roughly linear) and traditional SCF (roughly cubic). The class columns give each value its own intercept; this gives it its own slope as well.

• setup_by (str or None) – A column whose values each get a fixed cost per run, in units of one iteration: the work is units + setup. For a code whose first step is much dearer than the rest – MOPAC’s MOZYME localizes the orbitals once, then runs fast cycles – so a single point and a long optimization share one per-cycle cost.

• flags (dict or None) – The run’s options as flags, each with its own factor learned by the fit (shrunk to none unless the records support it): {"column": the column holding them, space separated (ORCA's '!' line); "drop": regular expressions of words that are not options (task keywords, basis sets); "drop_columns": columns whose values are dropped from the words (the method and basis, which the model has already)}. See flag_tokens().

default_alpha: float = 0.8

flags: dict | None = None

classmethod from_dict(data)

klass: tuple = ()

multiplier: str | None = None

setup_by: str | None = None

size: tuple = ('n_atoms',)

slope_by: str | None = None

task: str | None = 'task'

to_dict()

units: str | None = None

seamm_exec.timing_model.WEIGHT_POWER = 1.0

Each run’s weight in the power-law fit is its net time (wall less start-up) to this power, normalized. The cost exponent grows with size, so a single power law fitted equally to many small runs bends low at the large end – the runs that cost queue time and whose estimates set walltimes. Weighting by time puts the fit where the time is; small runs still set the start-up time.

seamm_exec.timing_model.estimate(program, descriptors, fallback=None, **kwargs)

A task’s estimated time, and where it came from.

The fitted model’s prediction (predict(), with kwargs) when it can make one, else fallback – the step’s own formula – and the reason the model could not be used, logged once per process and reason: as a warning when the descriptors lack a size variable the model was fitted with, which is a fault of the step (every estimate falls back to the formula without it), else at the info level (no model yet, a calculation outside the records).

Parameters:

• program (str)

• descriptors (dict) – As for predict().

• fallback (float, optional) – The step’s own estimate, in seconds.

• **kwargs – Passed to predict() (ntasks, quantile, …).

Returns:

seconds (None if there is neither), source – "model", "formula" or "none", for seamm_exec.Task.estimate_source – and reason (None when the model was used).

Return type:

dict

seamm_exec.timing_model.fit(program, directory=None, spec=None, min_rows=8, weight_power=None)

Fit the cost model of program to its records.

Returns:

The model (what save_model() writes), with a report entry; or None if there are fewer than min_rows usable records.

Return type:

dict

seamm_exec.timing_model.flag_tokens(record, flags)

The option flags of a run, as upper-case words, per a spec’s flags.

The words of flags["column"], less those matching a drop pattern and those that are another column’s value (drop_columns, compared ignoring case, with “_” and “/” the same, and also as the parts of a value like “R2SCAN-D4”). An empty set without a spec or a value.

seamm_exec.timing_model.load_model(program, directory=None)

The fitted model of program, or None if there is none.

seamm_exec.timing_model.load_rows(program, directory=None, spec=None)

The records of program usable for a fit: schema 1, finished, a wall time, as dicts with the model’s variables as floats added (_wall, _cores, _units, _mult, _size, _class, _task, _machine).

seamm_exec.timing_model.load_spec(program, directory=None)

The program’s spec: the one its step wrote beside the records, else the fallback for the 2026.10.6 steps, else the default (atoms only).

seamm_exec.timing_model.main(argv=None)

seamm_exec.timing_model.missing_sizes(spec, descriptors)

The size variables of spec that descriptors lack (or give as zero).

A step’s tests check that the descriptors of its estimate have every size variable of its TIMING_SPEC: its records have them, so the model may be fitted with any of them, and an estimate lacking one is never from the model.

seamm_exec.timing_model.model_path(program, directory=None)

seamm_exec.timing_model.models_directory(directory=None)

seamm_exec.timing_model.predict(program, descriptors, ntasks=1, cpus_per_task=1, machine=None, quantile=0.95, units=None, directory=None, model=None, refresh=True, why=None)

The predicted wall time of a calculation.

Parameters:

• program (str)

• descriptors (dict) – The calculation’s descriptors, as its timing record would carry them (the size columns, the class columns, the task; the multiplier).

• ntasks (int)

• cpus_per_task (int)

• machine (str, optional) – The machine class key; default this machine’s. An unknown class gets no offset and a wider spread.

• quantile (float) – Of the residual distribution: 0.5 for a median estimate (packing), 0.95 for a queue walltime.

• units (float, optional) – The iterations the run will take, if known (an optimization’s steps); else the task’s distribution at the same quantile.

• directory (optional) – The timing directory, or a model already loaded.

• model (optional) – The timing directory, or a model already loaded.

• refresh (bool) – Refit the model first if its records have grown by a fifth or it is a week old (refresh_if_stale()); a model given is used as it is.

• why (list, optional) – When there is no prediction, the reason is appended to it: no model, a size variable missing (named), a size outside the fitted range, or a regime, method class or task the records lack. See estimate().

Returns:

seconds (at the quantile), median, units, machine_known, quantile, spread (the log-space spread used), model (its date and rows); None when there is no model or the descriptors lack every size variable.

Return type:

dict or None

seamm_exec.timing_model.records_state(program, directory=None)

(bytes, mtime) over the program’s record files, current, set aside and pending: a cheap measure of how much the records have grown (no read).

seamm_exec.timing_model.refresh_if_stale(program, directory=None, growth=0.2, max_age_days=7.0, min_rows=8)

Refit the program’s model if its records have grown or it is old.

Stale: no model but enough records for one; the records’ bytes have grown by more than growth since the fit; or the model is older than max_age_days and the records have changed since. The refit runs under a non-blocking lock (another process refitting means this one uses the model as it is), and a new model replaces the old one only if it is fitted to at least as many rows and predicts within 2x at least as often, or nearly so. Never raises.

Returns:

"refitted", "fresh", "locked", "no records" or "failed".

Return type:

str

seamm_exec.timing_model.report_text(model)

seamm_exec.timing_model.save_model(model, directory=None)

Write the model, atomically (a reader never sees a partial file).

seamm_exec.timing_model.spec_path(program, directory=None)

seamm_exec.timing_model.write_spec(program, spec, directory=None)

Write a program’s spec beside its records, if it is not there or has changed. Called by record_timing when the step passes its spec; never raises.

Module contents

Classes to execute background codes for SEAMM

seamm_exec.get_executor(executor)

Return an object of the executor requested.

Parameters:

executor (str) – The name of the executor.

Return type:

instance of executor