Recipe structure¶
A recipe describes the work Momotor Engine should do to process a product into a result. It does not contain all assignment data and it does not usually implement the checks itself. Instead, it names the checklets that may be used, then defines the steps that run those checklets. The assignment specifics come from the config that is paired with the recipe when it runs.
The usual shape is:
metadata that identifies the recipe,
a list of checklet packages with versions,
a list of steps that reference those checklets,
dependencies between steps,
options that tell the scheduler and checklets what to do.
Declaring checklets¶
The checklets section is a list of Python packages that this recipe can use.
Each entry has a package name, a version range, and a local id. The local id is
what steps use later, so the recipe can refer to this id instead of
repeating the full package name. These Python packages need to be of a certain
shape to be usable as checklets, by inheriting from the right base package
which defines the expected interface.
The Momotor Checklets documentation introduces the checklets and their expected interface.
Declaring a checklet in the checklets section does not run it. It only makes
that checklet available to the steps in the recipe. The checklet package can also be
included inline in the recipe bundle, but that is not common for reusable recipes
and mostly used for testing and development.
Running steps¶
The steps section is the actual processing plan. Each step has its own id,
selects one checklet by reference, and passes options to that checklet. A step
can also list dependencies. Dependencies let Momotor Engine build a flow: check
the submitted files first, extract them only if that worked, build after
extraction, and so on.
Options can be grouped by domain. Options without a domain are normally meant
for the checklet. Options in the scheduler domain affect how Momotor Engine
schedules the step, such as whether it needs a tool or should be skipped when an
earlier step failed. Options in the tools domain tell the broker and checklet
which version of an external tool to use. The broker uses the tool version to
select a suitable work environment, and the checklet can use it to select the
correct tool version in that environment.
Config bundles and option overrides¶
A recipe is usually shared by many assignments. The config bundle contains the assignment-specific choices for one use of that recipe. In practice, that means the recipe can define the general workflow and default options, while the config bundle supplies details such as expected output, test data, limits, or tool versions.
When a checklet resolves an option, matching options from the config bundle generally take priority over options from the recipe. This lets a recipe provide reasonable defaults while individual assignments override only the values that need to differ. For example, a recipe can say that a step runs test cases, while the config bundle can say which test cases are present for a particular assignment.
The set of available options is not fixed by the recipe format itself. It depends on the checklets used by the recipe and on Momotor Engine options such as the scheduler and tools domains. A generated recipe documentation page normally lists the steps and the options accepted by the checklets in that recipe, which is the best place to look up what can be configured for a concrete recipe.
Simplified example¶
This example is based on the existing gcc-diff recipe, simplified to show the
structure without the full set of real options. The real recipe checks for a zip
submission, extracts it, builds C code with GCC, runs test cases, compares the
output, makes a report, and calculates the final outcome. See the
full gcc-diff recipe documentation
for the generated documentation of the complete gcc-diff recipe and its steps,
including the options that can be configured for each step in the config bundle.
<?xml version="1.0"?>
<recipe xmlns="http://momotor.org/1.0"
id="gcc-diff-simple">
<!-- This is a simplified version of the gcc-diff recipe for demonstration
purposes. Do not use this for real assignments. -->
<meta>
<name>gcc-diff-simple</name>
<version>1.0.0</version>
</meta>
<checklets>
<checklet id="files-chk"
name="mtrchk-org-momotor-check-files"
version="~=2.0"/>
<checklet id="zip-chk"
name="mtrchk-org-momotor-check-zip"
version="~=2.0"/>
<checklet id="build-chk"
name="mtrchk-org-momotor-lang-gcc-build"
version="~=2.0"/>
<checklet id="run-chk"
name="mtrchk-org-momotor-run-inout"
version="~=2.0"/>
<checklet id="evaluate-chk"
name="mtrchk-org-momotor-evaluate-compare"
version="~=2.0"/>
<checklet id="report-chk"
name="mtrchk-org-momotor-lti-stepreport"
version="~=2.0"/>
<checklet id="finalize-chk"
name="mtrchk-org-momotor-lti-finalize-condition"
version="~=3.0"/>
</checklets>
<steps>
<step id="check">
<checklet ref="files-chk"/>
<options>
<option name="stage" value="Check for expected files"/>
<option name="input-file-ref" value="@product"/>
<option name="match-default" value="*.zip"/>
</options>
</step>
<step id="extract">
<checklet ref="zip-chk"/>
<dependencies>
<depends step="check"/>
</dependencies>
<options domain="scheduler">
<option name="preflight" value="%notall pass => skip"/>
</options>
<options>
<option name="stage" value="Extract submitted files"/>
<option name="input-file-ref" value="@result#check:zip"/>
</options>
</step>
<step id="build">
<checklet ref="build-chk"/>
<dependencies>
<depends step="extract"/>
</dependencies>
<options domain="scheduler">
<option name="tools" value="gcc"/>
<option name="preflight" value="%notall pass => skip"/>
</options>
<options domain="tools">
<option name="gcc" value="gcc"/>
</options>
</step>
<step id="run">
<checklet ref="run-chk"/>
<dependencies>
<depends step="build"/>
</dependencies>
<options domain="scheduler">
<option name="tools" value="gcc"/>
</options>
<options>
<option name="exec-file-ref" value="@result#build:executable"/>
<option name="stdout-file-class" value="stdout"/>
</options>
</step>
<step id="evaluate">
<checklet ref="evaluate-chk"/>
<dependencies>
<depends step="run"/>
</dependencies>
<options>
<option name="left-file-ref" value="@result#run:stdout"/>
</options>
</step>
<step id="summary">
<checklet ref="report-chk"/>
<dependencies>
<depends step="evaluate"/>
</dependencies>
<options>
<option name="label" value="Summary"/>
</options>
</step>
<step id="finalize">
<checklet ref="finalize-chk"/>
<dependencies>
<depends step="check"/>
<depends step="extract"/>
<depends step="summary"/>
</dependencies>
<options>
<option name="pass-if" value="%all pass[#check,#extract,#summary]"/>
<option name="score-sum" value="prop[#summary:score]"/>
</options>
</step>
</steps>
</recipe>
How to read the example¶
The recipe starts by giving itself an id and metadata. This is how the recipe can be identified and versioned independently from the assignments that use it.
The checklet list declares reusable building blocks. In this example, one checklet checks the submitted files, another extracts a zip file, another builds with GCC, another runs the compiled program, and later checklets compare output, render a report, and decide the final result.
The steps define the order and data flow. check looks at the original
product. extract depends on check and reads the zip file reported by that
step. build depends on the extracted files and asks the scheduler for a
worker with the gcc tool. run depends on the build result and runs the
executable produced by the build step. evaluate compares the captured output.
summary creates user-facing feedback. finalize uses earlier outcomes to
decide the final pass or fail status and score.
The important idea is that the recipe coordinates checklets. The recipe decides which checklet runs when and which options it receives; each checklet decides how to perform its own specific task.
Recipe version management¶
Recipes define a version number in the meta section. For ease of use, this
version number will also be part of the recipe file name when downloading the
recipe from the releases section of the recipe’s repository.
Semantic versioning is used for the version numbering. A patch version update should only contain fixes or small improvements that do not change how assignments configure the recipe. A minor version update can add new functionality in a backwards compatible way. In practice, that new functionality might be available through a new option that an assignment can enable in its config bundle.
A major version update means the recipe can contain changes that are not backwards compatible. Existing config bundles might need to be reviewed and updated before they keep working with the new recipe version. This can include renamed options, changed defaults, different step behavior, or a new structure for the expected assignment data.
Check the available versions of a recipe regularly. Staying up to date helps assignments benefit from fixes, clearer feedback, new options, and improvements in the checklets used by the recipe. Changes in external dependencies might even make older recipe versions stop working. Outdated recipes due to changes in Momotor itself will generate deprecation warnings in the produced result, but we cannot guarantee that for external changes.
Customizing a recipe¶
A recipe should ideally be reusable across many assignments. The recipe defines the general workflow and default options, while the specific assignments can override these options to customize the behavior for their needs through the config bundle. Changes in external dependencies or Momotor itself might require updates to the recipe, and if each assignment has its own copy of the recipe, then all those copies need to be updated. This makes maintenance harder and increases the risk of errors.
By keeping a minimal set of shared recipes and using config bundles for customization, we can ensure that all assignments benefit from improvements and fixes in the recipe, checklets, and Momotor itself without needing to manually update each variant.
If a new feature is needed that cannot currently be provided by the recipe, then this should be discussed with the maintainers of the recipe to either incorporate the changes into the recipe in a backwards compatible way, or to create a new recipe that better fits the needs of the assignment.