golden_step package#

Submodules#

golden_step.compare module#

Pure-function comparator for Golden Test step output.

Compares an expected metrics dict (the reference, loaded from golden_expected.json) against an actual metrics dict (what the run produced, loaded from golden_output.json) and returns a structured report.

This module has no SEAMM dependencies and no I/O of its own, so it can be unit tested as pure data-in/data-out.

Comparison rules (see NOTES_golden_tests_design.rst for the design):

Walk expected; look up the matching node in actual.
Fields present in actual but not in expected are silently ignored (forward compatibility — new metrics never break old tests).
Fields present in expected but missing in actual are failures.
Lists named components match entries by smiles (or formula if smiles is null), not by position.
Booleans, integers and strings: exact match required.
Floats: tolerance-based. Defaults rtol = 1e-3, atol = 0.0.

Per-field tolerance overrides via a wrapper:

"density_g_per_mL": {"value": 1.21, "tol": {"rtol": 0.02}}

Lists of numbers: element-wise with the same tolerance rules.

golden_step.compare.compare(expected, actual)[source]#

Compare two metrics dicts.

Parameters:

expected (dict) – The reference metrics (loaded from golden_expected.json).
actual (dict) – The metrics produced by the run (loaded from golden_output.json).

Returns:

A report with keys passed (bool), schema_version, n_passes, n_failures, summary (human-readable string), and failures (list of per-failure dicts).

Return type:

dict

golden_step.golden module#

Non-graphical part of the Golden Test step in a SEAMM flowchart

class golden_step.golden.Golden(flowchart=None, title='Golden Test', extension=None, logger=<Logger golden_step.golden (WARNING)>)[source]#

Bases: Node

The non-graphical part of a Golden Test step in a flowchart.

parameters#

The control parameters for the Golden Test step.

Type:: GoldenParameters

See also

TkGolden, GoldenParameters

description_text(P=None)[source]#

Create the text description of what this step will do. The dictionary of control values is passed in as P so that the code can test values, etc.

Parameters:: P (dict) – An optional dictionary of the current values of the control parameters.
Returns:: A description of the current step.
Return type:: str

property git_revision#: The git version of this module.

run()[source]#

Run a Golden Test step.

Parameters:: None
Returns:: The next node object in the flowchart.
Return type:: seamm.Node

property version#: The semantic version of this module.

golden_step.golden_parameters module#

Control parameters for the Golden Test step in a SEAMM flowchart

class golden_step.golden_parameters.GoldenParameters(defaults={}, data=None)[source]#

Bases: Parameters

The control parameters for the Golden Test step.

See also

Golden, TkGolden, GoldenStep

parameters = {'case name': {'default': '', 'default_units': '', 'description': 'Case name:', 'enumeration': (), 'format_string': '', 'help_text': "Identifier for this test case, recorded in the results so it can be put into a table when many tests run in one flowchart. If left empty, the step's title is used.", 'kind': 'string'}, 'expected file': {'default': 'golden_expected.json', 'default_units': '', 'description': 'Expected file:', 'enumeration': ('golden_expected.json',), 'format_string': '', 'help_text': "Path to the reference JSON file used in 'verify' mode. If the path is relative it is resolved against the flowchart's directory.", 'kind': 'string'}, 'mode': {'default': 'record', 'default_units': '', 'description': 'Mode:', 'enumeration': ('record', 'verify', 'skip'), 'format_string': '', 'help_text': "What the Golden Test step should do. 'record' writes a metrics snapshot of the current system to a JSON file. 'verify' does the same and then compares the snapshot against a reference file. 'skip' does nothing.", 'kind': 'enum'}, 'on failure': {'default': 'continue', 'default_units': '', 'description': 'On verify failure:', 'enumeration': ('continue', 'stop'), 'format_string': '', 'help_text': "What to do if a 'verify'-mode comparison fails. 'continue' writes the result file and lets the flowchart proceed. 'stop' raises an error and halts the flowchart.", 'kind': 'enum'}, 'output file': {'default': 'golden_output.json', 'default_units': '', 'description': 'Output file:', 'enumeration': ('golden_output.json',), 'format_string': '', 'help_text': "Filename for the metrics snapshot written by this step in the step's working directory.", 'kind': 'string'}, 'results': {'default': {}, 'default_units': '', 'description': 'results', 'enumeration': (), 'format_string': '', 'help_text': 'The results to save to variables or in tables.', 'kind': 'dictionary'}}#

golden_step.golden_step module#

class golden_step.golden_step.GoldenStep(flowchart=None, gui=None)[source]#

Bases: object

Helper class needed for the stevedore integration.

This must provide a description() method that returns a dict containing a description of this node, and create_node() and create_tk_node() methods for creating the graphical and non-graphical nodes.

The dictionary for the description is the class variable just below these comments. The felds are as follows:

my_description{str, str}
A human-readable description of this step. It can be several lines long, and needs to be clear to non-expert users. It contains the following keys: description, group, name.

my_description[“description”]tuple
A description of the Golden Test step. It must be clear to non-experts.

my_description[“group”]str
Which group in the menus to put this step. If the group does not exist it will be created. Common groups are “Building”, “Control”, “Custom”, “Data”, and “Simulations”.

my_description[“name”]str
The name of this step, to be displayed in the menus.

create_node(flowchart=None, **kwargs)[source]#

Create and return the new node object.

Parameters:

flowchart (seamm.Node) – A non-graphical SEAMM node
**kwargs (keyword arguments) – Various keyword arguments such as title, namespace or extension representing the title displayed in the flowchart, the namespace for the plugins of a subflowchart and the extension, respectively.

Return type:

Golden

create_tk_node(canvas=None, **kwargs)[source]#

Create and return the graphical Tk node object.

Parameters:

canvas (tk.Canvas) – The Tk Canvas widget
**kwargs (keyword arguments) – Various keyword arguments such as tk_flowchart, node, x, y, w, h representing a graphical flowchart object, a non-graphical node for a step, and dimensions of the graphical node.

Return type:

TkGolden

description()[source]#

Return a description of what this step does.

Returns:: description
Return type:: dict(str, str)

my_description = {'description': 'Snapshot the current system to a JSON file and optionally verify it against a reference. Used for golden testing of SEAMM plug-ins.', 'group': 'Testing', 'name': 'Golden Test'}#

golden_step.metadata module#

This file contains metadata describing the results from Golden

golden_step.metadata.metadata = {'results': {'case_name': {'description': 'Name of the golden test case', 'dimensionality': 'scalar', 'type': 'string'}, 'mode': {'description': 'Mode the Golden Test step ran in', 'dimensionality': 'scalar', 'type': 'string'}, 'n_failures': {'description': 'Number of fields that failed to match', 'dimensionality': 'scalar', 'type': 'integer'}, 'n_passes': {'description': 'Number of fields that matched the reference', 'dimensionality': 'scalar', 'type': 'integer'}, 'passed': {'description': 'Whether the verify-mode comparison passed', 'dimensionality': 'scalar', 'type': 'boolean'}, 'summary': {'description': 'Human-readable summary of the test result', 'dimensionality': 'scalar', 'type': 'string'}}}#

Description of the computational models for Golden.

Hamiltonians, approximations, and basis set or parameterizations, only if appropriate for this code. For example:

metadata["computational models"] = {
    "Hartree-Fock": {
        "models": {
            "PM7": {
                "parameterizations": {
                    "PM7": {
                        "elements": "1-60,62-83",
                        "periodic": True,
                        "reactions": True,
                        "optimization": True,
                        "code": "mopac",
                    },
                    "PM7-TS": {
                        "elements": "1-60,62-83",
                        "periodic": True,
                        "reactions": True,
                        "optimization": False,
                        "code": "mopac",
                    },
                },
            },
        },
    },
}

golden_step.metrics module#

Pure-function builders for the Golden Test step’s metrics JSON.

These functions take a molsystem.Configuration and return JSON-serializable dicts. They have no side effects, do not write to disk, and have no SEAMM dependencies beyond molsystem and RDKit, so they can be unit tested without instantiating a flowchart.

The schema produced here is described in NOTES_golden_tests_design.rst. Bump SCHEMA_VERSION when making a backwards-incompatible change to the shape of the output.

golden_step.metrics.build_metrics(configuration, step_name='golden', schema_version=1)[source]#

Build a metrics dict from a molsystem Configuration.

Parameters:

configuration (molsystem.Configuration) – The current system/configuration.
step_name (str) – Name of the step producing these metrics. Recorded in the output as the step field. The Golden Test step itself passes "golden"; a hypothetical plug-in-internal use could pass its own name.
schema_version (int) – Schema version to tag the output with. Callers normally use the default; the parameter exists so tests can pin a version.

Returns:

A JSON-serializable dict with keys schema_version, step, system, components, derived.

Return type:

dict

golden_step.tk_golden module#

The graphical part of a Golden step

class golden_step.tk_golden.TkGolden(tk_flowchart=None, node=None, canvas=None, x=None, y=None, w=200, h=50)[source]#

Bases: TkNode

The graphical part of a Golden step in a flowchart.

tk_flowchart#

The flowchart that we belong to.

Type:: TkFlowchart = None

node#

The corresponding node of the non-graphical flowchart

Type:: Node = None

namespace#

The namespace of the current step.

Type:: str

tk_subflowchart#

A graphical Flowchart representing a subflowchart

Type:: TkFlowchart

canvas#

The Tk Canvas to draw on

Type:: tkCanvas = None

dialog#

The Pmw dialog object

Type:: Dialog

x#

The x-coordinate of the center of the picture of the node

Type:: int = None

y#

The y-coordinate of the center of the picture of the node

Type:: int = None

w#

The width in pixels of the picture of the node

Type:: int = 200

h#

The height in pixels of the picture of the node

Type:: int = 50

self[widget]#

A dictionary of tk widgets built using the information contained in Golden_parameters.py

Type:: dict

See also

Golden, TkGolden, GoldenParameters

create_dialog()[source]#

Create the dialog. A set of widgets will be chosen by default based on what is specified in the Golden_parameters module.

Parameters:: None
Return type:: None

See also

TkGolden.reset_dialog

reset_dialog(widget=None)[source]#

Layout the widgets in the dialog.

The widgets are chosen by default from the information in Golden_parameter.

This function simply lays them out row by row with aligned labels. You may wish a more complicated layout that is controlled by values of some of the control parameters. If so, edit or override this method

Parameters:: widget (Tk Widget = None)
Return type:: None

See also

TkGolden.create_dialog

right_click(event)[source]#

Handles the right click event on the node.

Parameters:: event (Tk Event)
Return type:: None

See also

TkGolden.edit

Module contents#

golden_step A SEAMM plug-in for Golden