Element reference syntax

References select bundle elements, such as properties or files. Selectors add conditions, and matches reduce those conditions to a boolean. Placeholders insert reference values into strings. Checklets and scheduler options use these syntaxes inside recipe and configuration option values.

Reference

A reference refers to one or more Result, Property, File or Option elements in bundles.

The syntax for a reference is:

typed_reference ::= type [ "[" reference ( "," reference )* "]" ]
type            ::= "file" | "prop" | "opt" | "result" | outcome | not_outcome
outcome         ::= "pass" | "fail" | "error" | "skip"
not_outcome     ::= "not-" outcome
reference       ::= [ provider_id ] [ ":" typeref ]
provider_id     ::= [ "@" provider ] [ "#" id ( "," id )* ]
provider        ::= "recipe" | "config" | "product" | "step" | "result"
typeref         ::= propref | fileref | optref
propref         ::= name
fileref         ::= class | [ class ] "#" name
optref          ::= name [ "@" domain ]
id              ::= result query, optionally containing task references
name            ::= element name, quoted when needed
class           ::= file class
domain          ::= option domain, optionally including a subdomain

The type defines the element type which is referenced. Choices are

type

element referenced

prop

One or more Property elements

file

One or more File elements

opt

One or more Option elements

result

One or more Result elements

pass / fail / skip / error

All Result elements with given outcome, e.g. pass selects all passed results

not-pass / not-fail / not-skip / not-error

All Result elements with a different outcome, e.g. not-pass selects all results that did not pass

The provider selects the bundle from which these elements are referenced. Choices are

provider

element referenced

@recipe

The RecipeBundle bundle

@config

The ConfigBundle bundle

@product

The ProductBundle bundle

@result

Result elements in a ResultsBundle bundle

@step

The current Step in the RecipeBundle bundle

Not all type / provider combinations are valid. For the prop, result, outcome and not_outcome types, only the @result provider is valid. Since there is only one provider valid for these types, specifying the provider is optional in this case. For the opt and file types, all providers are valid, but the provider must be explicit. For example, file[@product] selects all product files.

Since ResultsBundle bundles contain multiple results, one or more id tokens can be specified to limit the list of results. If no id is given, all results in the bundle are referenced. id is not used with other providers. The id can contain task id placeholders and these will be expanded with the task numbers for the currently active task.

The file reference type accepts an optional name or class filter. Without a filter, it selects all files from the provider. Names can contain glob-like wildcards. Names with spaces or special characters can be quoted.

The prop reference type requires an additional name to select the property.

The opt reference type requires an additional name with optional domain to select the option. If domain is not provided it defaults to checklet.

Similar to id, task number references are expanded in names, classes, and domains when a task id is available.

Result reference scope

Without a #id filter, result, pass, fail, skip, error, and their not- forms inspect all results in Providers.results. They do not read the current step’s <dependencies> declaration. This is also true when a recipe bundle is available.

A results bundle can contain dependencies of dependencies. For example, a summary step can depend on evaluate.*, while each evaluate task depends on build. If the supplied bundle contains the build result and skipped evaluate results, %all skip is false when build passed. %all skip[#evaluate.*] is true because the query excludes build.

The same distinction appears through the Python API:

>>> from momotor.bundles import ResultsBundle
>>> from momotor.bundles.elements.result import Outcome, Result
>>> from momotor.options.parser.selector import match_by_selector
>>> from momotor.options.providers import Providers
>>> results = ResultsBundle()
>>> _ = results.create(results=[
...     Result(results).create(step_id='build', outcome=Outcome.PASS),
...     Result(results).create(step_id='evaluate.0', outcome=Outcome.SKIP),
...     Result(results).create(step_id='evaluate.1', outcome=Outcome.SKIP),
... ])
>>> providers = Providers(results=results)
>>> match_by_selector('%all skip', providers)
(False, '')
>>> match_by_selector('%all skip[#evaluate.*]', providers)
(True, '')
>>> match_by_selector('%all skip[#missing.*]', providers)
(False, '')

A #id filter restricts the available results. It does not check whether every expected task has produced a result. The Result query syntax describes exact ids and wildcards.

Examples of references are:

reference

result

prop[:name1]

Selects properties with name name1 from all results in the results bundle

prop[#id1:name1]

Selects properties with name name1 from result with result id #id1

prop[@result#id1:name1]

Same as above (@result is optional and implied)

file[@config:class1#name1]

Selects the file with class class1 and exact name name1 from the config

file[@config:class1#*.txt]

Selects all files with class class1 and name ending with .txt from the config

file[@recipe:class1#doc.txt]

Selects all files with class class1 and exact name doc.txt from the recipe

file[@recipe:class1#"doc 1.txt"]

Selects all files with class class1 and exact name doc 1.txt from the recipe. Because of the whitespace, the name has to be quoted

opt[@step:name1]

Selects the option with name name1 in (default) domain checklet of the current step

opt[@step:name1@domain1]

Selects the option with name name1 in domain domain1 of the current step

result

Select all results from the results bundle

result[#id1]

Select result with id id1 from the results bundle

pass

Select all passed results from the results bundle

pass[#id1,#id2]

Select results with result id id1 and id2, if they passed

pass[@result#id1,#id2]

Same as above (@result is optional and implied)

not-pass[@result#id1,#id2]

Select results with result id id1 and id2, if they did not pass

Reference value

A reference value is a single value generated from a reference. Checklets can use reference values in options to resolve those options into values. Reference values are also used as part of the placeholder syntax.

A reference value is a reference, optionally prefixed with a modifier:

value_reference ::= [ "%" mod ] typed_reference
mod             ::= "all" | "any" | "notall" | "notany" | "not" | "sum"
                    | "sumf" | "sumr" | "max" | "min" | "cat" | "join"
                    | "joinc" | "joins" | "joincs" | "json" | "first" | "last"

What value is produced by a reference value is determined by the reference type:

type

value

prop / opt

The value attribute of the referenced Property or Option elements

file

The name attribute of the referenced File elements

result

The outcome attribute of the referenced Result elements [1]

pass / fail / skip / error / not-pass / not-fail / not-skip / not-error

The step_id attribute of the referenced Result elements [1]

The mod modifier indicates how the list of values produced by the reference is converted into a reference value. The default modifier is join, but this can be changed by the caller of the resolve_reference_value() method.

mod

result

%all

True if all values are considered True [2]

%any

True if at least one value is considered True [2]

%notall

False if all values are considered True [2]

%notany

False if at least one value is considered True [2]

%not

Alias for %notany

%sum

The sum of the values [3]

%sumf

The sum of the values, truncated toward zero [3]

%sumr

The sum of the values, rounded to the nearest int [3]

%max

The maximum of the values [3]

%min

The minimum of the values [3]

%cat

All values concatenated without any separator

%join

All values concatenated with a single comma, without spaces

%joinc

Alias for %join

%joins

All values concatenated with a space character

%joincs

All values concatenated with a comma followed by a space

%json

The values converted into a json list [4]

%first

The first value [5]

%last

The last value [5]

All other modifiers convert the values to a string before joining.

Selector

A selector filters references on the value. The value is one of the attributes of the referenced elements, the same attribute as used for value references.

A selector has the following syntax:

selector    ::= typed_reference [ selection ]
selection   ::= unary_oper | binary_oper value
unary_oper  ::= "?" | "!"
binary_oper ::= "==" | "!=" | ">" | ">=" | "<" | "<="
value       ::= quoted string or number

operator

action

(no selector)

Selects all elements (i.e. same as the reference)

?

Unary operator which selects the elements whose value is considered True [6]

!

Unary operator which selects the elements whose value is considered False [6]

== / != / > / >= / < / <=

Binary operators. Selects the elements whose value matches the equation. String values to compare with need to be quoted.

Example selectors:

selector

pass

Selects all passed results

result=="pass"

Also selects all passed results

prop[:score]

Selects all results containing a score property

prop[:score]>1

Selects all results with a score property greater than 1

Match

match ::= [ "%" mod ] selector
mod   ::= "all" | "any" | "not" | "notall" | "notany"

A match takes a selector and collapses it into a boolean, depending on the mod modifier.

mod

match

No modifier or %all

Matches if the selector is “true” for all referenced elements

%any

Matches if there is at least one selected element “true”

%notall

Matches if not all of the selected elements are “true”

%not or %notany

Matches if not any of the selected elements is “true”

Match results

Selected values

%all

%any

%notall

%notany

All comparisons true

True

True

False

False

All comparisons false

False

False

True

True

Mixed comparisons

False

True

True

False

No selected values

False

False

True

True

When no results match the result query, %all and %any are false. %notall and %notany are true. A result that lacks the requested property also makes %all false. These rules differ from reference-value modifiers, which can return None for empty input.

For example, %all skip[#evaluate.*] needs at least one evaluate result, and every selected evaluate result must have the skip outcome.

Example matches:

match

pass

Matches if all results passed

%any pass

Matches if at least one result passed

%notall pass

Matches if not all results passed

%notany pass

Matches if no result passed

prop[:score]

Matches if all results contain a score property

%any prop[:score]

Matches if at least one result contains a score property

prop[:score]>1

Matches if all results contain a score of more than 1

Reference placeholder

Reference placeholders can be used inside a longer string. The placeholder will be replaced by the value produced by the reference value.

placeholder ::= "${" value_reference "}"

Placeholders are reference values wrapped inside a ${...}. To include a literal ${ in the string, use $${ to escape the placeholder syntax.

replace_placeholders() recursively expands placeholders in resolved strings. When a valid placeholder resolves to None, its reference text is removed and ${} remains. With the default join modifier, missing properties produce an empty string, so their placeholders are removed entirely.

For the results bundle above, no result has a property named missing:

>>> from momotor.options.parser.placeholders import replace_placeholders
>>> replace_placeholders('x ${%first prop[:missing]} y', providers)
'x ${} y'
>>> replace_placeholders('x ${%sum prop[:missing]} y', providers)
'x ${} y'
>>> replace_placeholders('x ${prop[:missing]} y', providers)
'x  y'

Values that are not strings pass through unchanged. Option resolution does not expand placeholders automatically unless the caller explicitly invokes this function.

Task id placeholder

Task id placeholders can be used inside a longer string. The placeholder will be replaced by the sub-task number of the currently active task, either zero-based or one-based.

task_id_placeholder ::= "$" [ task_id_base ] "#"
task_id_base        ::= "0" | "1"

Task id placeholders are $#, $0# and $1#. The 0 and 1 indicate whether the task id is zero-based or one-based. If no base is given, the default is zero-based. If there is no task, the placeholder will be replaced by a -.

Examples, for a task with id task.0.1:

placeholder

replacement

$#

0.1

$0#

0.1

$1#

1.2