Utility classes and functions

class mtrchk.org.momotor.mixin.exec.capture.Capture(cap_stdin=True, cap_stdout=True, cap_stderr=True, *, limits=None, max_size=1048576)

Send and capture I/O on stdin/stdout/stderr and records timestamps when each line was sent or received.

Truncates stream if it gets too big. If a stream is truncated, the last record for that stream will contain the boolean value False.

Spools to a buffer in memory until total data size gets bigger than max_size

any_truncated()

Whether any stream was truncated

Return type:

bool

lines(t)

The total number of lines captured for the stream with id t, where t is one of STDIN, STDOUT or STDERR

Return type:

int

playback()

Start playback of I/O on stdin/stdout/stderr

Return type:

Iterable[dict]

record()

Start recording I/O on stdin/stdout/stderr

Return type:

CaptureRecorderBase

size(t)

The total number of bytes captured for the stream with id t, where t is one of STDIN, STDOUT or STDERR

Return type:

int

truncated(t)

Whether the stream with id t was truncated, where t is one of STDIN, STDOUT or STDERR

Return type:

bool

class mtrchk.org.momotor.mixin.exec.capture.CaptureLimits(max_stdout_lines=None, max_stdout_bytes=None, max_stderr_lines=None, max_stderr_bytes=None)

Limits for capturing stdout/stderr. The stream will be truncated (no more output captured) if any of the limits are exceeded.

max_stderr_bytes: int | None = None

Maximum number of bytes to capture from stderr

max_stderr_lines: int | None = None

Maximum number of lines to capture from stderr

max_stdout_bytes: int | None = None

Maximum number of bytes to capture from stdout

max_stdout_lines: int | None = None

Maximum number of lines to capture from stdout

class mtrchk.org.momotor.mixin.exec.capture.CapturePlayback(file)

A class for playing back I/O on stdin/stdout/stderr from a tempfile.SpooledTemporaryFile as recorded by CaptureRecorder.

class mtrchk.org.momotor.mixin.exec.capture.CapturePlaybackBase(file)

A base class for playing back I/O on stdin/stdout/stderr.

mtrchk.org.momotor.mixin.exec.capture.CaptureRecord

TypeAlias for a capture record. The dictionary contains the following keys:

  • @ (str): The timestamp in seconds since execution start. Contains a Decimal timestamp.

  • 0 (int): The data on stdin. Contains a str or bytes value.

  • 1 (int): The data on stdout. Contains a str or bytes value, or the boolean value False if the stream was truncated at this timestamp

  • 2 (int): The data on stderr. Contains a str or bytes value, or the boolean value False if the stream was truncated at this timestamp

class mtrchk.org.momotor.mixin.exec.capture.CaptureRecorder(file, capture, size, lines, trunc, limits)

A class for recording I/O on stdin/stdout/stderr to a tempfile.SpooledTemporaryFile.

Parameters:
  • file (SpooledTemporaryFile) – The file to write the capture to

  • capture (tuple[bool, bool, bool]) – A tuple of booleans indicating which streams to capture

  • size (list[int]) – A list of 3 integers where the size of the captured data for each stream is recorded

  • lines (list[int]) – A list of 3 integers where the number of lines of the captured data for each stream is recorded

  • trunc (list[bool]) – A list of 3 booleans indicating whether the stream was truncated

  • limits (CaptureLimits) – The limits for capturing stdout/stderr

add(*args)

Add data to the capture.

Expected to be called with 3 arguments, one for each stream. If a stream is not captured, the argument should be None.

TIMESTAMP_PRECISION: ClassVar[Decimal] = Decimal('0.001')

The precision to use for timestamps

class mtrchk.org.momotor.mixin.exec.capture.CaptureRecorderBase(file, capture, size, lines, trunc, limits)

A base class for recording I/O on stdin/stdout/stderr. The base class only records the size and number of lines of the captured data for each stream, and checks capture limits. It does not record the actual data.

Parameters:
  • file – The file to write the capture to

  • capture (tuple[bool, bool, bool]) – A tuple of booleans indicating which streams to capture

  • size (list[int]) – A list of 3 integers where the size of the captured data for each stream is recorded

  • lines (list[int]) – A list of 3 integers where the number of lines of the captured data for each stream is recorded

  • trunc (list[bool]) – A list of 3 booleans indicating whether the stream was truncated

  • limits (CaptureLimits) – The limits for capturing stdout/stderr

add(*args)

Add data to the capture.

Expected to be called with 3 arguments, one for each stream. If a stream is not captured, the argument should be None.

class mtrchk.org.momotor.mixin.exec.capture.JSONCaptureEncoder(*, ensure_ascii=False, sort_keys=False, **kwargs)

JSON encoder to convert CaptureRecord objects to JSON. Differs from the json.JSONEncoder in that it:

  • Generates more compact JSON, by overriding the item_separator and key_separator attributes

  • ensure_ascii defaults to False

  • sort_keys defaults to False

  • Converts decimal.Decimal objects to floats

default(o)

Convert the record to JSON

Return type:

Any

mtrchk.org.momotor.mixin.exec.capture.STDERR: Literal[0, 1, 2] = 2

Constant to reference stderr

mtrchk.org.momotor.mixin.exec.capture.STDIN: Literal[0, 1, 2] = 0

Constant to reference stdin

mtrchk.org.momotor.mixin.exec.capture.STDOUT: Literal[0, 1, 2] = 1

Constant to reference stdout

class mtrchk.org.momotor.mixin.exec.monitor.ChildUsage(utime=0.0, stime=0.0, mem=0)
mtrchk.org.momotor.mixin.exec.monitor.Limits

deprecated alias of RunLimits

class mtrchk.org.momotor.mixin.exec.monitor.Reason(*values)

An enum for the various reasons a process may be terminated

class mtrchk.org.momotor.mixin.exec.monitor.ResourceMonitor(parent_pid, process, limits, shell, *, mem_maps_filter=True)

A thread to monitor another process and enforce runtime limits.

join(timeout=None)

Wait until the thread terminates.

This blocks the calling thread until the thread whose join() method is called terminates – either normally or through an unhandled exception or until the optional timeout occurs.

When the timeout argument is present and not None, it should be a floating-point number specifying a timeout for the operation in seconds (or fractions thereof). As join() always returns None, you must call is_alive() after join() to decide whether a timeout happened – if the thread is still alive, the join() call timed out.

When the timeout argument is not present or None, the operation will block until the thread terminates.

A thread can be join()ed many times.

join() raises a RuntimeError if an attempt is made to join the current thread as that would cause a deadlock. It is also an error to join() a thread before it has been started and attempts to do so raises the same exception.

Return type:

None

run()

Check usage of child processes

stop()

Stop the monitor and kill the monitored processes

Return type:

None

reason: Reason | None

Termination reason

usage: Usage

Current usage of the process and it’s children

class mtrchk.org.momotor.mixin.exec.monitor.RunLimits(timeLimit=None, blockLimit=None, memoryLimit=None, processLimit=None)

Runtime limits for a process executed with MPopen

blockLimit: float | None = None

Maximum time a process may be blocked (in seconds)

memoryLimit: int | None = None

Maximum memory usage (in bytes)

processLimit: int | None = None

Maximum number of processes that can be spawned

timeLimit: float | None = None

Maximum time process is allowed to run (in seconds)

class mtrchk.org.momotor.mixin.exec.monitor.Usage(wctime=0.0, utime=0.0, stime=0.0, mem=0, proc=0)
mem: int = 0

Memory usage

proc: int = 0

Process count

stime: float = 0.0

CPU system time

utime: float = 0.0

CPU user time

wctime: float = 0.0

Wall clock time

mtrchk.org.momotor.mixin.exec.monitor.verbose_reason: dict[Reason, str] = {Reason.BLOCKLIMIT_EXCEEDED: 'Blocked or deadlocked', Reason.MEMORYLIMIT_EXCEEDED: 'Memory usage limit exceeded', Reason.PROCESSLIMIT_EXCEEDED: 'Too many processes started', Reason.TIMELIMIT_EXCEEDED: 'Time limit exceeded'}

Verbose termination reason strings for use in reports and as reason property

class mtrchk.org.momotor.mixin.exec.mpopen.MPopen(args, limits, mem_maps_filter=True, shell=False, **kwargs)

A monitored version of subprocess.Popen

Checklets should use the ExecutionMixin or CapturedExecutionMixin mixin instead of this class directly.

As with subprocess.Popen, supplying errors enables text mode, even without text or encoding.

communicate(input_file, *, input_delay_ms=None)

A generator variant of subprocess.Popen.communicate() that keeps the interleaving on stdin/stdout/stderr intact.

Input will be read from input_stream, output on stdout and stderr will be collected. In text mode, CRLF and CR output newlines are converted to LF, including CRLF split across reads. Each output stream keeps its own newline state. A trailing CR becomes LF at EOF. Binary output is unchanged.

Return type:

Generator[tuple[str | bytes, str | bytes, str | bytes], None, None]

communicate_readonly()

A generator variant of subprocess.Popen.communicate() that keeps the interleaving of output on stdout/stderr intact.

No input will be provided on stdin, output on stdout and stderr will be collected.

Return type:

Generator[tuple[str | bytes, str | bytes], None, None]

status_properties()

Get status to use in checklet properties.

Return type:

dict[str, Union[str, bool, None, NoContentType (object), bytes, int, float, Decimal, Sequence[str], set[str]]]

Returns:

A dict with the returncode, usage and reason values usable as checklet properties.

property outcome: Outcome

An Outcome value based on the reason and returncode.

property reason: Reason | None

The termination reason, if any

property returncode: int | None

The process return code as reported by subprocess.Popen

property usage: Usage | None

The process usage, or None if the monitor hasn’t run yet