Momotor Engine Protocol Library documentation

momotor-engine-proto defines how the parts of the Momotor engine talk to each other. Three kinds of programs use it:

  • Clients send jobs to the broker and collect the results.

  • The broker is the server. It receives jobs, assigns tasks to workers, and stores the files that move between them.

  • Workers ask the broker for tasks, run them, and return the results.

Programs that only submit jobs use the client side. Programs that run tasks use the worker side. New terms are defined in the Glossary.

What is in the package

The package has two layers.

The protocol definitions in Protocol definition are gRPC service and message definitions, plus the Python code generated from them. A stub is the generated client for one service. You create a stub from a channel and call its methods like local async functions.

The helper modules wrap common work that raw stubs do not cover:

  • Authentication opens an authenticated channel to the broker. Start here. Every other helper needs a channel or a stub built on one.

  • Assets uploads and downloads the files of a job, such as recipes and results.

  • Status streams job status updates.

  • Shared state gives workers locks that the broker coordinates.

  • Exception turns error fields in protocol messages into Python exceptions.

  • Message validation checks messages before a server acts on them.

  • Hash, Resources, Task, Tools, and Constants convert and identify the values that messages carry.

Install the package

pip install momotor-engine-proto

The package needs Python 3.11 or newer. Python 3.10 support ended in version 7.0.0.

A first example

This example opens an authenticated channel to a broker on the local machine. The returned channel is what you pass to a stub, such as the client stub that creates jobs.

from momotor.rpc.auth.client import get_authenticated_channel
from momotor.rpc.proto.client_grpc import ClientStub

channel, auth_stub = await get_authenticated_channel(
    'localhost', None, api_key='my-key', api_secret='my-secret',
)
client = ClientStub(channel)

Passing None as the port selects the default port, 50051 without TLS and 50052 with TLS. Authentication explains keys, secrets, and tokens. Client connection recovery explains how the channel detects a dead connection.

Contents

Indices and tables