Utility classes and functions¶
This appendix lists the helper classes and functions under
momotor.bundles.utils. Most code only needs the bundle arguments below
and the collections described in Elements and Sequence filtering. The
remaining helpers support the bundle implementation itself. They are
documented here for readers who follow a reference from another page.
Bundle arguments¶
Factory and export methods accept these immutable settings objects through
args=. Method keyword arguments override the corresponding fields.
Use an arguments object when you reuse the same settings for several calls.
- class momotor.bundles.utils.arguments.BundleConstructionArguments(use_lxml=Ellipsis, pretty_xml=False, encoding='utf-8', xml_name=None, sign_files=True, hashers=<factory>, optimize=True, generator_name=True, *, signing=None, attachment_prefix=<factory>, deduplicate=False, deduplicate_attachments=False)¶
Shared settings for XML serialization and attachment export.
- attachment_prefix: PurePosixPath¶
Attachment namespace used by nested self-test fixtures and reports.
- deduplicate: bool = False¶
Replace identical file, option and checklet XML nodes with scoped references. Independent of optimize.
Added in version 9.0.0.
- deduplicate_attachments: bool = False¶
Share identical external attachment files and complete directory trees. Requires deduplicate=True; timestamps do not participate in payload identity.
Added in version 9.0.0.
- generator_name: bool | str = True¶
add a generator meta node. If
True, uses this package’s name and version number. IfFalsedoes not add a generator node. If a string is provided, uses that as the generator name, and adds the package name. DefaultTrue
- hashers: Sequence[str]¶
hashing algorithms to use when
sign_filesis set toTrue. Default['sha1']
- optimize: bool = True¶
Decode and re-encode previously loaded inline content when writing the bundle. Default
True. Short printable option and property strings usevalueattributes. Content requiring encoding uses quoted-printable or Base64 where supported. IfFalse, preserve the existing inline representation where possible. Newly created values still use normal serialization. Re-encoding only applies while the decoded value has not been cached. Reading an element’s value beforehand can prevent re-encoding, and serialization updates the element’s cached representation. Independent of ZIP compression, attachment recompression, and deduplication.
- sign_files: bool = True¶
sign the <file> nodes. This calculates a hash of the attachments and adds it as an attribute in the XML
- signing: SigningOptions | None = None¶
Sign the finished serialized output.
Added in version 9.0.0.
- use_lxml: bool | None | EllipsisType = Ellipsis¶
compatibility argument. XML processing always uses lxml. Supplying this argument emits a warning, regardless of its value. This argument will be removed in momotor-bundles version 10.
- class momotor.bundles.utils.arguments.BundleFactoryArguments(xml_name=None, use_lxml=Ellipsis, validate_xml=None, location_base=None, validate_signature=True, *, verification=None, attachment_source=None)¶
Settings for bundle loading and type detection.
- attachment_source: str | PathLike[str] | Mapping[str, bytes] | None = None¶
Confined attachment source for standalone XML bytes.
Added in version 9.0.0.
- use_lxml: bool | None | EllipsisType = Ellipsis¶
compatibility argument. XML processing always uses lxml. Supplying this argument emits a warning, regardless of its value. This argument will be removed in momotor-bundles version 10.
- validate_xml: bool | None = None¶
enable (
True) or disable (False) XML validation. IfNone, uses default setting, which is to enable validation
- verification: VerificationOptions | None = None¶
Opt-in authentication, completed before ordinary source detection.
Added in version 9.0.0.
- class momotor.bundles.utils.arguments.DirectoryConstructionArguments(use_lxml=Ellipsis, pretty_xml=False, encoding='utf-8', xml_name=None, sign_files=True, hashers=<factory>, optimize=True, generator_name=True, dir_mode=448, *, signing=None, attachment_prefix=<factory>, deduplicate=False, deduplicate_attachments=False)¶
Settings for export to a directory containing XML and attachments.
- attachment_prefix: PurePosixPath¶
Attachment namespace used by nested self-test fixtures and reports.
- deduplicate: bool = False¶
Replace identical file, option and checklet XML nodes with scoped references. Independent of optimize.
Added in version 9.0.0.
- deduplicate_attachments: bool = False¶
Share identical external attachment files and complete directory trees. Requires deduplicate=True; timestamps do not participate in payload identity.
Added in version 9.0.0.
- dir_mode: int = 448¶
the mode bits (defaults to 0o700, making the directory only accessible to the current user)
- generator_name: bool | str = True¶
add a generator meta node. If
True, uses this package’s name and version number. IfFalsedoes not add a generator node. If a string is provided, uses that as the generator name, and adds the package name. DefaultTrue
- hashers: Sequence[str]¶
hashing algorithms to use when
sign_filesis set toTrue. Default['sha1']
- optimize: bool = True¶
Decode and re-encode previously loaded inline content when writing the bundle. Default
True. Short printable option and property strings usevalueattributes. Content requiring encoding uses quoted-printable or Base64 where supported. IfFalse, preserve the existing inline representation where possible. Newly created values still use normal serialization. Re-encoding only applies while the decoded value has not been cached. Reading an element’s value beforehand can prevent re-encoding, and serialization updates the element’s cached representation. Independent of ZIP compression, attachment recompression, and deduplication.
- sign_files: bool = True¶
sign the <file> nodes. This calculates a hash of the attachments and adds it as an attribute in the XML
- signing: SigningOptions | None = None¶
Sign the finished serialized output.
Added in version 9.0.0.
- use_lxml: bool | None | EllipsisType = Ellipsis¶
compatibility argument. XML processing always uses lxml. Supplying this argument emits a warning, regardless of its value. This argument will be removed in momotor-bundles version 10.
- class momotor.bundles.utils.arguments.FileConstructionArguments(use_lxml=Ellipsis, pretty_xml=False, encoding='utf-8', xml_name=None, sign_files=True, hashers=<factory>, optimize=True, generator_name=True, zip=False, compression=8, compresslevel=None, always_recompress=False, *, signing=None, attachment_prefix=<factory>, deduplicate=False, deduplicate_attachments=False)¶
Settings for XML or ZIP export to a file or binary buffer.
- always_recompress: bool = False¶
Recompress existing ZIP attachments using the requested compression settings. Defaults to False, allowing direct copying when the compression method matches.
Added in version 9.0.0.
- attachment_prefix: PurePosixPath¶
Attachment namespace used by nested self-test fixtures and reports.
- compression: int = 8¶
compression mode, see
zipfile.ZipFilefor possible values. (Defaults toZIP_DEFLATED, only used when generating a zipped bundle)
- compresslevel: int | None = None¶
compression level, see
zipfile.ZipFilefor possible values. Copied ZIP attachments retain their existing level unless always_recompress is True. (Only used when generating a zipped bundle)
- deduplicate: bool = False¶
Replace identical file, option and checklet XML nodes with scoped references. Independent of optimize.
Added in version 9.0.0.
- deduplicate_attachments: bool = False¶
Share identical external attachment files and complete directory trees. Requires deduplicate=True; timestamps do not participate in payload identity.
Added in version 9.0.0.
- generator_name: bool | str = True¶
add a generator meta node. If
True, uses this package’s name and version number. IfFalsedoes not add a generator node. If a string is provided, uses that as the generator name, and adds the package name. DefaultTrue
- hashers: Sequence[str]¶
hashing algorithms to use when
sign_filesis set toTrue. Default['sha1']
- optimize: bool = True¶
Decode and re-encode previously loaded inline content when writing the bundle. Default
True. Short printable option and property strings usevalueattributes. Content requiring encoding uses quoted-printable or Base64 where supported. IfFalse, preserve the existing inline representation where possible. Newly created values still use normal serialization. Re-encoding only applies while the decoded value has not been cached. Reading an element’s value beforehand can prevent re-encoding, and serialization updates the element’s cached representation. Independent of ZIP compression, attachment recompression, and deduplication.
- sign_files: bool = True¶
sign the <file> nodes. This calculates a hash of the attachments and adds it as an attribute in the XML
- signing: SigningOptions | None = None¶
Sign the finished serialized output.
Added in version 9.0.0.
- use_lxml: bool | None | EllipsisType = Ellipsis¶
compatibility argument. XML processing always uses lxml. Supplying this argument emits a warning, regardless of its value. This argument will be removed in momotor-bundles version 10.
KeyedTuple¶
An immutable tuple that also looks up items by their ID. Steps and results
use it, so bundle.results['first'] and bundle.results[0] both work.
- class momotor.bundles.utils.keyedtuple.KeyedTuple(items=None, *, key_attr=None)¶
A tuple of objects with an indexable attribute that acts as both an immutable sequence and a mapping. Elements can be accessed by their numeric index or the key attribute.
- Parameters:
items (ItemsType | None) – the initial items for the tuple. can be an iterable, mapping, another
KeyedTuple, or Nonekey_attr (str | None) – the attribute to index the items on. defaults to ‘id’
If items is another
KeyedTuple, key_attr must either be the same as the key_attr of items, or not provided.- copy()¶
Create a shallow copy
- Return type:
Self
- count(value)¶
Return number of occurrences of value
- get(key_or_index, default=None)¶
Get an item from the sequence by index, key or item
- index(value, start=0, stop=None)¶
Return first index of value
- Parameters:
value (
TypeVar(IT)) – the item to findstart – the index to start searching from
stop – the index to stop searching at
- Return type:
- Returns:
the index of the first occurrence
Added in version 8.0.0.
- values()¶
A set-like object providing a view on the values
- Return type:
ValuesView[TypeVar(IT)]
Assertions¶
Checks that elements belong to the bundle that uses them and that values have the expected types.
- momotor.bundles.utils.assertion.assert_element_mapping_instanceof(mapping, expected_key_type, expected_value_type, bundle)¶
Combines
assert_elements_bundle()andassert_mapping_instanceof()- Parameters:
mapping (
TypeVar(TME, bound=Mapping[str|int, Element]) |None) – A mapping containing items to testexpected_key_type (
type[Hashable]) – Expected type for the keysexpected_value_type (
type[TypeVar(TE, bound= Element)]) – Expected type for the valuesbundle (
Bundle) – expected bundle instance for each element
- Return type:
- momotor.bundles.utils.assertion.assert_elements_bundle(elements, bundle)¶
Assert that all elements are linked the correct bundle. Returns elements
- momotor.bundles.utils.assertion.assert_elements_instanceof(elements, expected_type, bundle)¶
Combines
assert_elements_bundle()andassert_sequence_instanceof()- Parameters:
- Return type:
- momotor.bundles.utils.assertion.assert_mapping_instanceof(mapping, expected_key_type, expected_value_type)¶
Assert that all items in the mapping have the expected key and value types. Returns mapping
- momotor.bundles.utils.assertion.assert_sequence_instanceof(items, expected_type)¶
Assert that all items are of the expected type. Returns items
Boolean¶
Conversion of XML text values to Python booleans.
- momotor.bundles.utils.boolean.to_bool(val)¶
Convert a representation of truth to True or False.
True values are ‘y’, ‘yes’, ‘t’, ‘true’, ‘on’, and ‘1’; false values are ‘n’, ‘no’, ‘f’, ‘false’, ‘off’, and ‘0’. Raises ValueError if ‘val’ is anything else.
- Return type:
>>> to_bool(True) True
>>> to_bool('y') True
>>> to_bool('yes') True
>>> to_bool('t') True
>>> to_bool('true') True
>>> to_bool('on') True
>>> to_bool('1') True
>>> to_bool(1) True
>>> to_bool(False) False
>>> to_bool('n') False
>>> to_bool('no') False
>>> to_bool('f') False
>>> to_bool('false') False
>>> to_bool('off') False
>>> to_bool('0') False
>>> to_bool(0) False
>>> to_bool(None) False
>>> to_bool('other') Traceback (most recent call last): ... ValueError: invalid truth value 'other'
Domains¶
Splitting of option domain names into a domain and subdomain.
- momotor.bundles.utils.domain.merge_domains(*domains)¶
Merge domains of format <domain>#<subdomain>.
For all domains, the first provided part is returned as part of the merged domain:
abc#def | uvw#xyz = abc#def abc#def | uvw = abc#def abc#def | #xyz = abc#def abc | uvw#xyz = abc#xyz abc | uvw = abc abc | #xyz = abc#xyz #def | uvw#xyz = uwv#def #def | uvw = uwv#def #def | #xyz = #def
>>> merge_domains() is None True
>>> merge_domains(None) is None True
>>> merge_domains('abc#def', 'uvw') 'abc#def'
>>> merge_domains('abc#def', '#xyz') 'abc#def'
>>> merge_domains('abc#def', 'uvw#xyz') 'abc#def'
>>> merge_domains('abc', 'uvw#xyz') 'abc#xyz'
>>> merge_domains('abc', 'uvw') 'abc'
>>> merge_domains('abc', '#xyz') 'abc#xyz'
>>> merge_domains('#def', 'uvw#xyz') 'uvw#def'
>>> merge_domains('#def', 'uvw') 'uvw#def'
>>> merge_domains('#def', '#xyz') '#def'
- momotor.bundles.utils.domain.split_domain(domain)¶
Split a domain name <domain>[#<subdomain>] into its parts.
>>> split_domain(None) (None, None)
>>> split_domain('') (None, None)
>>> split_domain('#') (None, None)
>>> split_domain('domain') ('domain', None)
>>> split_domain('domain#') ('domain', None)
>>> split_domain('#subdomain') (None, 'subdomain')
>>> split_domain('domain#subdomain') ('domain', 'subdomain')
- momotor.bundles.utils.domain.unsplit_domain(domain, subdomain)¶
Merge <domain> and <subdomain> parts into a <domain>#<subdomain> string, handling
NonevaluesThe reverse operation of
split_domain().>>> unsplit_domain(None, None) is None True
>>> unsplit_domain('domain', None) 'domain'
>>> unsplit_domain('domain', '') 'domain'
>>> unsplit_domain(None, 'subdomain') '#subdomain'
>>> unsplit_domain('domain', 'subdomain') 'domain#subdomain'
Encoding¶
Encoding and decoding of inline content, such as base64 and quoted-printable text.
Data encoding and decoding
- momotor.bundles.utils.encoding.ENCODINGS¶
List of all supported encodings
- momotor.bundles.utils.encoding.decode_data(data, encoding)¶
Decode encoded data.
- Parameters:
- Return type:
- Returns:
The decoded data
- Raises:
ValueError – For unsupported encodings
- momotor.bundles.utils.encoding.encode_data(data)¶
Encode provided data.
Printable data is encoded using Quoted-printable encoding, any other data is encoded using Base64 encoding.
- momotor.bundles.utils.encoding.encode_posix_path(path)¶
Encode non-ASCII path components with Punycode.
- Parameters:
path (
PurePosixPath) – Original POSIX path.- Return type:
- Returns:
A POSIX path containing only ASCII characters.
- momotor.bundles.utils.encoding.is_printable(s, low=32, allowed=None)¶
Detect if string is printable; it should not contain any control characters, except those in allowed, and less than 5% of the characters are allowed to be high-ascii
- Return type:
- momotor.bundles.utils.encoding.make_pure_ascii(s)¶
Encode non-ASCII parts of a period-separated string with Punycode.
- momotor.bundles.utils.encoding.quopri_encode(data)¶
Encode data using quoted printable method Uses a different algorithm than
quopri.encodestring():Encodes first space character on a line, but not the rest
Encodes newline, return characters and high-ascii
- Return type:
Grouping¶
Grouping of elements by an attribute value.
- momotor.bundles.utils.grouping.group_by_attr(items, attr1, attr2=None)¶
Group a list of elements by the value of one or more attributes
Immutable¶
Mappings that cannot be changed after creation.
- class momotor.bundles.utils.immutable.ImmutableDict(*args, **kwargs)¶
An immutable
dict- clear()¶
Reject changes to this immutable mapping.
- move_to_end(key, last=None)¶
Reject changes to this immutable mapping.
- pop(key, default=None)¶
Reject changes to this immutable mapping.
- popitem(last=True)¶
Reject changes to this immutable mapping.
- setdefault(key, default=None)¶
Reject changes to this immutable mapping.
- class momotor.bundles.utils.immutable.ImmutableOrderedDict(*args, **kwargs)¶
Deprecated alias class for
ImmutableDict.Construction emits
DeprecationWarning.ImmutableDictalready preserves insertion order.
Nodes¶
Traversal of nested generated XML node structures.
- momotor.bundles.utils.nodes.get_nested_complex_nodes(node, node_name, *sub_node_names)¶
Get all the nodes in a nested complex node structure.
For example, with the following XML:
<root> <level1> <level2> <level2> <level2> </level1> <level1> <level2> </level1> </root>
a call
get_nested_complex_list(root_node, 'level1', 'level2')yields a sequence of[('level1', level1-node), ('level2', level2-node), ('level2', level2-node), ('level2', level2-node), ('level1', level1-node), ('level2', level2-node) ]
Text¶
String splitting and unescaping helpers.
Useful functions for string processing, borrowed from Django (with a few small modifications)
- momotor.bundles.utils.text.smart_split(text)¶
Generator that splits a string by spaces, leaving quoted phrases together. Supports both single and double quotes, and supports escaping quotes with backslashes. In the output, strings will keep their initial and trailing quote marks and escaped quotes will remain escaped (the results can then be further processed with unescape_string_literal()).
>>> list(smart_split(r'This is "a person\'s" test.')) ['This', 'is', '"a person\\\'s"', 'test.'] >>> list(smart_split(r"Another 'person\'s' test.")) ['Another', "'person\\'s'", 'test.'] >>> list(smart_split(r'A "\"funky\" style" test.')) ['A', '"\\"funky\\" style"', 'test.']
- momotor.bundles.utils.text.unescape_string_literal(s)¶
Convert quoted string literals to unquoted strings with escaped quotes and backslashes unquoted:
>>> unescape_string_literal('"abc"') 'abc' >>> unescape_string_literal("'abc'") 'abc' >>> unescape_string_literal('"a \"bc\""') 'a "bc"' >>> unescape_string_literal("'\'ab\' c'") "'ab' c"
- Return type:
Zip¶
Opening ZIP archives on demand, and copying an attachment’s compressed bytes to a new archive without decompressing them first.
- class momotor.bundles.utils.zipwrapper.ZipWrapper(*, path=None, content=None)¶
Open a filesystem or in-memory ZIP archive on demand.
Nested context managers share one archive handle. Leaving the outermost context closes that handle. A later context reopens the archive from the same source.
from pathlib import Path from momotor.bundles.utils.zipwrapper import ZipWrapper wrapper = ZipWrapper(path=Path('recipe.zip')) with wrapper as archive: with archive.open('recipe.xml') as source: xml = source.read()
- Parameters:
path (
Path|None) – Filesystem ZIP path. Supply exactly one ofpathandcontent.content (
bytes|memoryview|None) – ZIP bytes or a memory view containing the entire archive.
- close()¶
Close the wrapped zip file
- content: bytes | memoryview | None¶
Copy ZIP payloads using the zipfile internals shared by Python 3.11 through 3.14.
- momotor.bundles.utils.zipcopy.copy_compressed_file(source, source_info, destination, target_info)¶
Copy a validated member’s compressed bytes and write new destination headers.
ZipFile.openvalidates the local header, name, encryption and overlapping entries. Its underlying reader is positioned at the compressed payload. Reading that reader avoids decompression while retaining these checks.- Parameters:
- Raises:
zipfile.BadZipFile – If the source header is invalid or the compressed payload is truncated
zipfile.LargeZipFile – If ZIP64 is required but disabled in the destination
ValueError – If the destination is closed or has an open write handle
- Return type:
Added in version 9.0.0.
Recursive glob matching¶
These functions match path strings without reading the filesystem.
* matches within one path component. ** can cross directory boundaries.
Filename matching with shell patterns.
rglob(FILENAME, PATTERN) matches according to the local convention. rglobcase(FILENAME, PATTERN) always takes case in account.
The functions operate by translating the pattern into a regular expression. They cache the compiled regular expressions for speed.
The function translate(PATTERN) returns a regular expression corresponding to PATTERN. (It does not compile it.)
—
Cloned from pywildcard <https://pypi.org/project/pywildcard/>,
modified to match behaviour of glob.glob() with recursive=True:
**/and**\match zero or more directories, instead of requiring at least a path separator to match? matches any character except the path separator, instead of matching any character
Fix incorrect matching on non-posix systems by properly escaping
\in the generated regular expression
- momotor.bundles.utils.rglob.filter(names, pat)¶
Return the subset of the list NAMES that match PAT.
- momotor.bundles.utils.rglob.rglob(name, pat)¶
Test whether FILENAME matches PATTERN.
Patterns are Unix shell style:
**matches across path components.*matches within one path component.?matches one character other than a path separator.[seq]matches one character inseq.[!seq]matches one character outsideseq.
An initial period in FILENAME is not special. Both FILENAME and PATTERN are first case-normalized if the operating system requires it. If you don’t want this, use rglobcase(FILENAME, PATTERN).
- momotor.bundles.utils.rglob.rglobcase(name, pat)¶
Test whether FILENAME matches PATTERN, including case.
This is a version of fnmatch() which doesn’t case-normalize its arguments.
- momotor.bundles.utils.rglob.translate(pat)¶
Translate a shell PATTERN to a regular expression.
There is no way to quote meta-characters.
Directory digests¶
One hash for a whole directory attachment, calculated from its files. Signing uses this digest to check directory attachments. See Bundle verification for the format.
Versioned directory digest framing shared by ordinary and verified readers.
Added in version 9.0.0.
- momotor.bundles.utils.hashes.hash_directory(leaves, algorithms)¶
Hash sorted directory leaf records using the directory-v1 encoding.