Reference and placeholder parsers

This page is the API reference for the parsers behind the reference syntax. Element reference syntax explains the syntax itself, with examples. Read this page when your code parses references, selectors, or placeholders directly.

References

class momotor.options.parser.reference.Reference(provider, id, name, _source)

Parsed provider, result query, and element filter from one reference.

provider=None uses the reference type’s default provider. id=None selects all available results for the result provider. name contains the property name, file filter, or option name and domain.

id: str | None

The id of the reference

name: str | None

The name of the reference, including the class or domain parts

provider: str | None

The provider of the references

class momotor.options.parser.reference.ReferenceMatch(provider, values)

One provider and the elements that match a reference within it.

values can be empty. For example, a pass reference produces an empty match for a failed result. Match selectors retain this information so that %all pass is false if any selected result did not pass.

provider: RecipeBundle | ConfigBundle | ProductBundle | Step | Result

The provider containing the elements referenced

values: tuple[Property | File | Option | Result, ...]

A tuple with the referenced elements

momotor.options.parser.reference.generate_reference(type_, refs, bundles)

Yield matches for parsed references against the supplied providers.

Without an id filter, result references inspect all of bundles.results. They do not use the recipe’s dependency declarations. Each selected result produces a match, including an empty match when its outcome or values do not satisfy the reference.

Parameters:
  • type – Reference type, such as prop, file, or skip.

  • refs (Sequence[Reference]) – Parsed provider, id, and name filters.

  • bundles (Providers) – Available bundles and current task.

Return type:

Generator[ReferenceMatch, None, None]

Returns:

An iterator of ReferenceMatch objects.

Raises:

ValueError – If a provider is unavailable or a reference is invalid. Validation happens as the iterator is consumed.

momotor.options.parser.reference.parse_reference(reference)

Parse a reference into its parts.

Parameters:

reference (str) – the reference to parse

Return type:

tuple[str, tuple[Reference, ...], str]

Returns:

a 3-tuple containing the type, a tuple of Reference objects, and a string with the rest of the reference string remaining after parsing.

Raises:

ValueError – the reference cannot be parsed

Examples:

>>> parse_reference("type")
('type', (Reference(provider=None, id=None, name=None, _source=''),), '')
>>> parse_reference("type rest")
('type', (Reference(provider=None, id=None, name=None, _source=''),), ' rest')
>>> parse_reference("type[] rest")
('type', (Reference(provider=None, id=None, name=None, _source=''),), ' rest')
>>> parse_reference("type[@provider] rest")
('type', (Reference(provider='provider', id=None, name=None, _source='@provider'),), ' rest')
>>> parse_reference("type[@provider#id:class#name] rest")
('type', (Reference(provider='provider', id='id', name='class#name', _source='@provider#id:class#name'),), ' rest')
>>> parse_reference("type[@provider#id,id2:class#name] rest")
('type', (Reference(provider='provider', id='id,id2', name='class#name',
                    _source='@provider#id,id2:class#name'),), ' rest')
>>> parse_reference("type[#wildcard.*:class] rest")
('type', (Reference(provider=None, id='wildcard.*', name='class', _source='#wildcard.*:class'),), ' rest')
momotor.options.parser.reference.resolve_reference_value(value_reference, bundles, *, default_mod='join')

Resolve a reference value string into the value

Parameters:
Return type:

tuple[str | int | float | bool | None, str]

Returns:

A pair containing the resolved value and the unconsumed input.

Raises:

ValueError – If the reference, provider, or modifier is invalid.

momotor.options.parser.reference.select_by_file_reference(reference, bundles)

Parse a file reference string and collect the referenced files.

This is similar to the file[...] reference syntax, but does not require the file nor the square brackets.

Parameters:
  • reference (str) – The reference to parse

  • bundles (Providers) – The bundles to resolve the references to

Return type:

tuple[tuple[ReferenceMatch, ...], str]

Returns:

a 2-tuple containing a tuple of ReferenceMatch objects, and a string with the rest of the reference string remaining after parsing.

momotor.options.parser.reference.select_by_opt_reference(reference, bundles)

Parse an option reference string and collect the referenced options.

This is similar to the opt[...] reference syntax, but does not require the opt nor the square brackets.

Parameters:
  • reference (str) – Provider, optional result query, and option name with domain.

  • bundles (Providers) – The bundles to resolve the references to

Return type:

tuple[tuple[ReferenceMatch, ...], str]

Returns:

a 2-tuple containing a tuple of ReferenceMatch objects, and a string with the rest of the reference string remaining after parsing.

momotor.options.parser.reference.select_by_prop_reference(reference, results=None, task_id=None)

Parse a property reference string and collect the referenced properties.

This is similar to the prop[...] reference syntax, but does not require the prop nor the square brackets.

Parameters:
  • reference (str) – The reference to parse

  • results (ResultsBundle | None) – The results bundle containing the properties

  • task_id (StepTaskId | None) – The task id to expand task references

Return type:

tuple[tuple[ReferenceMatch, ...], str]

Returns:

a 2-tuple containing a tuple of ReferenceMatch objects, and a string with the rest of the reference string remaining after parsing.

momotor.options.parser.reference.select_by_reference(reference, bundles)

Parse a reference string and collect the referenced items

Parameters:
  • reference (str) – The reference string to parse

  • bundles (Providers) – The providers providing the objects that are referenced

Return type:

tuple[str | None, tuple[ReferenceMatch, ...], str]

Returns:

a 3-tuple containing the type, a tuple of ReferenceMatch objects, and a string with the rest of the reference string remaining after parsing.

Raises:

ValueError – the reference cannot be parsed

Selectors

momotor.options.parser.selector.filter_by_selector(selector, bundles)

Return providers whose referenced values satisfy the selector.

For a property selector, the returned elements are the containing results, not the properties. Each matching provider is added once per reference match.

Parameters:
  • selector (str) – the selector to parse

  • bundles (Providers) – The bundles to resolve the references to

Return type:

tuple[tuple[Element, ...], str]

Returns:

a 2-tuple containing a tuple with the selected elements, and a string with the rest of the selector string remaining after parsing.

Raises:

ValueError – if the selector is not valid

momotor.options.parser.selector.match_by_selector(selector, bundles, *, default_mod='all')

Reduce a selector to a boolean match.

all requires at least one value and a successful comparison for every value from every selected provider. An empty match makes all false. any requires one successful comparison. notall and notany negate these results, and not is an alias for notany.

Without a result id filter, outcome selectors inspect all supplied results, including dependencies of dependencies. skip[#evaluate.*] restricts the check to available results whose ids match evaluate.*.

Parameters:
  • selector (str) – A selector with an optional % modifier.

  • bundles (Providers) – Available bundles and current task.

  • default_mod (str) – Modifier used when the selector omits one. Defaults to all.

Return type:

tuple[bool, str]

Returns:

A pair containing the match result and unconsumed input.

Raises:

ValueError – If the selector, provider, or modifier is invalid.

momotor.options.parser.selector.parse_selector(selector)

Parse a selector into its parts.

Parameters:

selector (str) – the selector to parse

Return type:

tuple[str, tuple[Reference, ...], str | None, str | int | float | None, str]

Returns:

a 5-tuple containing the type, a tuple of Reference objects, the operator, the value, and a string with the rest of the selector string remaining after parsing.

Raises:

ValueError – the selector cannot be parsed

Examples:

>>> parse_selector('pass')
('pass', (Reference(provider=None, id=None, name=None, _source=''),), None, None, '')
>>> parse_selector('prop[#id:name]!')
('prop', (Reference(provider=None, id='id', name='name', _source='#id:name'),), '!', None, '')
>>> parse_selector('prop[:test]?')
('prop', (Reference(provider=None, id=None, name='test', _source=':test'),), '?', None, '')
>>> parse_selector('prop[:test]?123')
('prop', (Reference(provider=None, id=None, name='test', _source=':test'),), '?', None, '123')
>>> parse_selector('prop[:test]>0')
('prop', (Reference(provider=None, id=None, name='test', _source=':test'),), '>', 0, '')
>>> parse_selector('prop[:test]>1_000')
('prop', (Reference(provider=None, id=None, name='test', _source=':test'),), '>', 1000, '')
>>> parse_selector('prop[:test]>1.5')
('prop', (Reference(provider=None, id=None, name='test', _source=':test'),), '>', 1.5, '')
>>> parse_selector('prop[:test]>-2')
('prop', (Reference(provider=None, id=None, name='test', _source=':test'),), '>', -2, '')
>>> parse_selector('prop[:test]>-2e2')
('prop', (Reference(provider=None, id=None, name='test', _source=':test'),), '>', -200.0, '')
>>> parse_selector('prop[:test]>2e-2')
('prop', (Reference(provider=None, id=None, name='test', _source=':test'),), '>', 0.02, '')
>>> parse_selector('prop[:test]=="test string"')
('prop', (Reference(provider=None, id=None, name='test', _source=':test'),), '==', 'test string', '')
>>> parse_selector('prop[:test]>0 123')
('prop', (Reference(provider=None, id=None, name='test', _source=':test'),), '>', 0, ' 123')
momotor.options.parser.selector.resolve_selector(selector, bundles)

Resolve all parts of a selector

Parameters:
  • selector (str) – the selector to parse

  • bundles (Providers) – The bundles to resolve the references to

Return type:

tuple[str, Iterable[ReferenceMatch], Callable[[Any, Any], bool], str | int | float | None, str]

Returns:

A 5-tuple containing the attribute to get the reference value from (based on type), an iterator for ReferenceMatch objects, the operator, the value, and a string with the rest of the selector string remaining after parsing.

Raises:

ValueError – if the selector is not valid

Reference placeholders

momotor.options.parser.placeholders.replace_placeholders(value, bundles, *, value_processor=None, mod='join')

Replace all placeholders in value with their resolved values. Placeholders are resolved recursively, i.e. if a resolved value contains more placeholders, these will be resolved as well.

When a valid placeholder resolves to None, its reference text is removed and ${} remains. With the default join modifier, missing properties produce an empty string, so their placeholders are removed entirely. $${ escapes a literal ${. Recursive expansion requires placeholders that eventually resolve without referring back to themselves.

Parameters:
  • value (TypeVar(VT) | str) – the string containing placeholders to resolve. If value is not a string, no processing is done and value is returned unmodified.

  • bundles (Providers) – the bundles to resolve the references to

  • value_processor (Callable[[Any], str] | None) – a callable that is called with every resolved value, can be used to modify placeholders. If not supplied, uses str to cast the returned value to a string.

  • mod (str) – the modifier to apply to the resolved values. Defaults to 'join'.

Return type:

TypeVar(VT) | str

Returns:

The input with resolved placeholders replaced.

Raises:

ValueError – If a reference is invalid or a resolved placeholder has no closing brace or contains unexpected trailing text.

Task id placeholders

momotor.options.parser.tasks.replace_task_placeholder(text, task_id)

Replace the task id placeholders in a string.

Replaces the $#, $0#, and $1# placeholders in a string with the sub-task number. $# and $0# are replaced with the zero-based task number, $1# is replaced with the one-based task number.

If the task id is None, or has no sub-tasks, the placeholders are replaced with -.

>>> replace_task_placeholder('Text $#', StepTaskId('task', (0, 1)))
'Text 0.1'
>>> replace_task_placeholder('Text $0#', StepTaskId('task', (0, 1)))
'Text 0.1'
>>> replace_task_placeholder('Text $1#', StepTaskId('task', (0, 1)))
'Text 1.2'
>>> replace_task_placeholder('Text $#', StepTaskId('task', None))
'Text -'
>>> replace_task_placeholder('Text $#', None)
'Text -'
Parameters:
  • text (str | None) – the string to replace the placeholders in

  • task_id (StepTaskId | None) – the task id to use

Return type:

str | None

Returns:

the string with the placeholders replaced

Added in version 2.0.0.

Modifiers

A modifier converts a list of values into a single value

momotor.options.parser.modifier.apply_combiner_modifier(mod, values)

Combine reference values using a named modifier.

Parameters:
Return type:

str | int | float | bool | None

Returns:

The combined value. Invalid numeric conversions and empty first or last selections return None. Join modifiers return an empty string for no values, and json returns the string null.

Raises:

ValueError – If the modifier name is unknown.

momotor.options.parser.modifier.cast_number(values)

Yield numeric values and strings that convert to integers or floats.

Parameters:

values (Iterable[Any]) – Numbers, numeric strings, or None.

Return type:

Generator[int | float, None, None]

Returns:

Converted numbers. None and invalid numeric strings are skipped.

Raises:

TypeError – If a value cannot be passed to the numeric constructors.

momotor.options.parser.modifier.exclude_none(values)

Yield values other than None, preserving their order.

Parameters:

values (Iterable[Any]) – Input values.

Return type:

Generator[Any, None, None]

Returns:

An iterator of non-None values.

momotor.options.parser.modifier.floor_safe(value)

Truncate floats toward zero.

Despite the function name, negative values do not round toward negative infinity.

Parameters:

value (Any) – Value to truncate.

Return type:

Any

Returns:

An integer for floats, or the unchanged value for other types.

momotor.options.parser.modifier.none_if_empty(fn, values)

Apply a function to a materialized list, or return None when empty.

Parameters:
Return type:

Any

Returns:

The function result, or None.

momotor.options.parser.modifier.parse_mod(modded_ref)

Split an optional leading %modifier from the remaining text.

This function parses the modifier name without checking whether it exists.

>>> parse_mod('')
(None, '')
>>> parse_mod(' 123')
(None, '123')
>>> parse_mod('%mod')
('mod', '')
>>> parse_mod('%mod 123')
('mod', '123')
>>> parse_mod(' %mod 123 ')
('mod', '123')
Parameters:

modded_ref (str) – Text with an optional modifier prefix.

Return type:

tuple[str | None, str]

Returns:

The modifier name without %, or None, and stripped remaining text.

momotor.options.parser.modifier.quote_str(value)

Quote strings containing spaces or quotes for a joined reference value.

Parameters:

value (Any) – Value to format. Other types are converted with str.

Return type:

str

Returns:

The formatted string. This is not shell escaping.

momotor.options.parser.modifier.round_safe(value)

Round floats to the nearest integer, with ties going to an even integer.

Parameters:

value (Any) – Value to round.

Return type:

Any

Returns:

The rounded integer, or the unchanged value for other types.

Key-value lists

momotor.options.parser.keyvalue.parse_key_value_list(s)

Parse a string of key=value pairs into a sequence of (key, value) pairs.

Value can be a string, integer, float or boolean. If value needs to contain spaces, it must be quoted.

If the value is empty, but the = is present, the value will be None. A single key (without =) will result in a value of True Constants ‘true’, ‘false’, ‘null’ and ‘none’ are converted to boolean or None values. Any other unquoted string is considered a string.

Parameters:

s (str) – Space-separated keys and key=value pairs.

Return type:

Generator[tuple[str, str | int | float | bool | None], None, None]

Returns:

An iterator of key and converted value pairs.

Raises:

ValueError – If a quoted value has no closing quote.

Examples:

>>> list(parse_key_value_list(""))
[]
>>> list(parse_key_value_list("key"))
[('key', True)]
>>> list(parse_key_value_list("key=value"))
[('key', 'value')]
>>> list(parse_key_value_list("key='value with space'"))
[('key', 'value with space')]
>>> list(parse_key_value_list('key="value with space"'))
[('key', 'value with space')]
>>> list(parse_key_value_list('key=0'))
[('key', 0)]
>>> list(parse_key_value_list('key=1.5'))
[('key', 1.5)]
>>> list(parse_key_value_list('key=true'))
[('key', True)]
>>> list(parse_key_value_list('key=false'))
[('key', False)]
>>> list(parse_key_value_list('key=null'))
[('key', None)]
>>> list(parse_key_value_list('key=none'))
[('key', None)]
>>> list(parse_key_value_list('key="true"'))
[('key', 'true')]
>>> list(parse_key_value_list('key1=value1 key2=value2 key3= key4'))
[('key1', 'value1'), ('key2', 'value2'), ('key3', None), ('key4', True)]
>>> list(parse_key_value_list('key1 key2 key3 key4'))
[('key1', True), ('key2', True), ('key3', True), ('key4', True)]
>>> type(next(parse_key_value_list('key=1'))[1])
<class 'int'>
>>> type(next(parse_key_value_list('key=1.0'))[1])
<class 'float'>
>>> type(next(parse_key_value_list('key=x'))[1])
<class 'str'>
>>> type(next(parse_key_value_list('key=true'))[1])
<class 'bool'>
>>> type(next(parse_key_value_list('key=null'))[1])
<class 'NoneType'>