Hash¶
The broker stores each asset once, no matter how many jobs use it. To do that, it identifies an asset by a hash of its content. Two uploads with the same content get the same hash, so the second upload sends no data.
Momotor uses the multihash format, encoded with base58. A multihash
carries the code of the hashing algorithm next to the digest. That lets a
client and a broker agree on an algorithm they both support, and it lets a
future version move to a stronger algorithm without changing the format.
SUPPORTED_HASH_FUNCS lists the algorithm
codes this version can use, most preferred first.
Very short content, shorter than
MAX_IDENTITY_LENGTH bytes, is its own hash.
The content bytes are stored inside the hash value, in what the multihash
format calls an identity hash. The broker then needs no separate transfer
for such assets.
You rarely call these functions directly. The asset helpers in Assets hash files for you. Use the functions here when you need the hash of a file before any transfer, for example to check whether the broker must receive it at all:
import hashlib
from momotor.rpc.hash.funcs import get_algorithm_by_code, get_code_by_hash_obj
algorithm = get_algorithm_by_code('sha2-256')
digest = algorithm(b'content')
print(get_code_by_hash_obj(digest) == get_code_by_hash_obj(hashlib.sha256()))
API reference¶
Functions¶
- momotor.rpc.hash.funcs.get_algorithm_by_code(code)¶
Get a PEP 452 compatible hashing object by its multihash code or name
- Parameters:
code (
str|int) – multihash code or name- Return type:
- Returns:
PEP 452 compatible hashing object
- momotor.rpc.hash.funcs.get_code_by_hash_obj(hash_obj)¶
Get the multihash code for a PEP 452 compatible hashing object
- Parameters:
hash_obj – PEP 452 compatible hashing object
- Return type:
- Returns:
- momotor.rpc.hash.funcs.SUPPORTED_HASH_FUNCS = [20, 21, 19, 22, 23, 18, 17]¶
Available multihash algorithm codes, from most to least preferred.
Identity hash¶
A PEP 452 compatible implementation of an identity hashing object.
The identity hash stores the literal content as a hash:
>>> IdentityHash(b'any random data').digest() == b'any random data'
True
- class momotor.rpc.hash.identity.IdentityHash(data=None)¶
- copy()¶
Create a copy of this hash object
- Return type:
- Returns:
Copy of the object
- hexdigest()¶
Get the current data as hex-encoded digest
- Return type:
- Returns:
the hex-encoded current data
- update(data)¶
Extend the hash with data
- Parameters:
data (
bytes) – data to concatenate to the existing data
- property block_size¶
- property digest_size¶
- name = 'identity'¶
identifier for the hashing function
- momotor.rpc.hash.identity.new(data=None)¶
Create a identity hash object
- Parameters:
data (
bytes|memoryview|None) – initial data for the hash- Return type:
- Returns:
the identity hash object
- momotor.rpc.hash.identity.digest_size = None¶
Digest size is variable