Momotor lti-finalize-condition checklet

The mtrchk-org-momotor-lti-finalize-condition package provides a checklet to finalize the result based on step conditions.

Checklet

checklet mtrchk.org.momotor.lti.finalize.condition.FinalizeCondition(recipe, product, config, results, task_id, work_dir=None)

Finalize result based on step conditions.

This is for use as the special finalize step that Momotor LTI expects to be present in the result. The results of this step determine the final result of the submission.

The following properties are used by Momotor LTI as the submission result:

• outcome: the outcome of the submission

• score: the score of the submission to be reported to the learner

• score_reviewer: the score of the submission to be reported to reviewers (if not present, the score property is used)

• reason: the reason for failure (if any)

score-sum option

Score to report to learners. Can be an integer or float literal, or a reference value to a property of another step. Can also contain reference placeholders which will be expanded.

The %sum, %sumf, %sumr, %min and %max modifiers can be used to calculate the score based on scores of multiple steps.

Usage examples:

• %sum prop[:score] (the default) will sum the score property of all steps that are listed in the depends list for this step.

• %max prop[#run.0,#run.1:score] will use the maximum score of the explicitly named steps run.0 and run.1. [1]

[1]

As with any step, this step only has access to results of earlier steps that are listed as dependencies, so in the latter example, the run.0 and run.1 steps must be listed as dependencies.

Type:

Any

Required:

False

Multiple:

False

Location:

config, step, recipe

Default:

%sum prop[:score]

score-reviewer-sum option

Score to report to reviewers. Can be an integer or float literal, or a reference value to a property of another step. Can also contain reference placeholders which will be expanded.

See the score-sum option for usage examples. If this property is not provided, a single score is reported based on the score-sum option.

Type:

Any

Required:

False

Multiple:

False

Location:

config, step, recipe

Default:

No default

pass-with-score option

Minimal score-sum value required for a PASS outcome. Can be an integer or float, or a reference value to a property of another step. Can also contain reference placeholders which will be expanded.

When combined with the pass-if option, both conditions must be met for a PASS outcome.

Type:

Any

Required:

False

Multiple:

False

Location:

config, step, recipe

Default:

No default

pass-if option

Determines when the submission is considered to have passed.

Either the constant value always, a boolean value, or a match selector to match against all dependency steps. If the selector matches, the submission is considered to have passed. Can contain reference placeholders which will be expanded.

Usage examples:

• always or true: Always pass.

• false: Always fail.

• pass (the default): Pass if all depending steps have a PASS outcome.

• %any pass: Pass if any depending step has a PASS outcome.

• %all prop[score]>10: Pass if all depending steps have a score property with a value greater than 10.

This option is also used to determine the reason property when the step fails. The reason of the first step that fails the selector and has a reason property is used.

When combined with the pass-with-score option, both conditions must be met for a PASS outcome.

Type:

Any

Required:

False

Multiple:

False

Location:

config, step, recipe

Default:

pass

keep-score-on-fail option

Keep score on fail. If false, no score is reported when the submission fails.

Type:

boolean

Required:

False

Multiple:

False

Location:

config, step, recipe

Default:

False