Sequence filtering#

The momotor.bundles.utils.filters package provides two filterable sequences, the FilterableList and FilterableTuple. Both add the same functions to the standard python list and tuple types.

The interface is loosely based on Django QuerySets field lookups.

The examples in this documentation use the following filterable lists:

>>> data_str = FilterableList([ItemA(a='abc'), ItemA(a='ABC'), ItemA(a='def'), ItemA(a='abc/def')])
>>> data_int = FilterableList([ItemA(a=1), ItemA(a=2), ItemA(a=3)])
>>> data_set = FilterableList([ItemA(a={1, 2}), ItemA(a={2, 3}), ItemA(a={3, 4})])
>>> data_ab = FilterableList([ItemAB(a=1, b=1), ItemAB(a=1, b=2), ItemAB(a=2, b=2), ItemAB(a=3, b=3)])

Where ItemA and ItemAB are datatypes having both an a attribute, and ItemAB also has a b attribute.

Filtering and excluding#

The most basic lookup looks like this:

>>> data_ab.filter(a=1)  # filter all items with a==1
[ItemAB(a=1, b=1), ItemAB(a=1, b=2)]

This filters data_ab for any objects containing an a attribute with value equal to 1.

The result of filter() is itself another filterable type, so filters can be chained:

>>> data_ab.filter(a=1).filter(b=2)  # filter all items with a==1, the filter the result for a==2
[ItemAB(a=1, b=2)]

FilterableList.exclude(...) excludes items:

>>> data_ab.exclude(a=1)  # exclude all items where a==1
[ItemAB(a=2, b=2), ItemAB(a=3, b=3)]

Providing multiple lookups in filter() or exclude() will filter or exclude objects matching all the lookups:

>>> data_ab.filter(a=1, b=2)  # filter all items with a==1 and b==2
[ItemAB(a=1, b=2)]

>>> data_ab.exclude(a=1, b=2)  # exclude all items with a==1 and b==2
[ItemAB(a=1, b=1), ItemAB(a=2, b=2), ItemAB(a=3, b=3)]

Not() object#

Lookups can be negated using a Not object:

>>> data_ab.filter(Not(a=1))  # filter all items where `a` is not 1
[ItemAB(a=2, b=2), ItemAB(a=3, b=3)]

FilterableList.exclude(...) is a shortcut for FilterableList.filter(Not(...))

All() object#

The All object can be used to make lookups with multiple arguments more explicit:

>>> data_ab.filter(All(a=1, b=2))  # filter all items with a==1 and b==2
[ItemAB(a=1, b=2)]

Any() object#

Lookups can also be combined using the Any object to find an object which matches any of the requested lookups:

>>> data_ab.filter(Any(a=1, b=2))  # filter all items with a==1 OR b==2
[ItemAB(a=1, b=1), ItemAB(a=1, b=2), ItemAB(a=2, b=2)]

Combining filter objects#

Lookup objects can be combined, for example:

>>> data_ab.filter(Any(All(a=1, b=2), b=3))  # Look for items with (a==1 AND b==2) OR b==3
[ItemAB(a=1, b=2), ItemAB(a=3, b=3)]

F() object#

Using keyword arguments it is not possible to filter for multiple values of the same property, since filter(a=1, a=3) has a repeated keyword a:

>>> data_ab.filter(a=1, a=3)  # this is a SyntaxError
Traceback (most recent call last):
...
SyntaxError: keyword argument repeated

To look for data_ab values for the same attribute, use the F object to wrap individual queries:

>>> data_ab.filter(F(a=1), F(a=3))  # filter all items with a==1 AND a==3 (which is nonsensical indeed, but this is just an example)
[]

This can also be combined with the other filter objects like Any

>>> data_ab.filter(Any(F(a=1), F(a=3)))  # filter items with a==1 OR a==3
[ItemAB(a=1, b=1), ItemAB(a=1, b=2), ItemAB(a=3, b=3)]

F objects can also be used to easily pass on filter queries as arguments to functions:

>>> def filter_data(f: F):
...     return data_ab.filter(f)

>>> for f in [F(a=1), F(a=2)]:
...     print(filter_data(f))
[ItemAB(a=1, b=1), ItemAB(a=1, b=2)]
[ItemAB(a=2, b=2)]

Any, All and Not are subclasses of F.

When mixing F arguments and keyword arguments, Python syntax requires the positional arguments to be provided before the keyword arguments. The following is invalid:

>>> data_ab.filter(a=1, Any(a=2, b=3))  # positional argument follows keyword argument
Traceback (most recent call last):
...
SyntaxError: positional argument follows keyword argument

Using an F object solves this issue:

>>> data_ab.filter(F(a=1), Any(a=2, b=3))
[]

Negation#

The Any and All classes support negation using the ~ operator, for example ~All(...) is the same as Not(All(...))

Lookup types#

Additionally to filtering or excluding exact values, there are several other lookup types. A lookup type is added to the lookup by separating the attribute name and the lookup with a double underscore __, eg. a__ne applies the not-equal lookup to the a attribute.

The following filter lookup operations are available:

Lookup type	Case sensitive lookup	Case insensitive lookup
Is-equal	(none) / eq / is	ieq / iis
Not-equal	ne	ine
Contains	contains	icontains
In	in	iin
Starts-with	startswith	istartswith
Ends-with	endswith	iendswith
Glob	glob	iglob
Recursive glob	rglob	irglob
Regular expression	re	ire

Note

Case insensitive lookups only accept strings or a sequence of strings. Any other value will raise a TypeError:

>>> data_int.filter(a__ine=1)
Traceback (most recent call last):
...
TypeError: Expected a string or sequence of strings, got <class 'int'>

Is-equal lookup#

Filters exact values.

Operator: No operator, is or eq

Examples:

>>> data_str.filter(a='abc')
[ItemA(a='abc')]

>>> data_str.filter(a__is='ABC')
[ItemA(a='ABC')]

>>> data_str.filter(a__eq='def')
[ItemA(a='def')]

>>> data_int.filter(a=1)
[ItemA(a=1)]

>>> data_int.filter(a='abc')
[]

Is-equal lookup (case insensitive)#

Filters string attributes case insensitive.

Lookup operator: ieq / iis

Examples:

>>> data_str.filter(a__ieq='abc')
[ItemA(a='abc'), ItemA(a='ABC')]

>>> data_int.filter(a__iis='abc')
[]

Not-equal lookup#

Lookup operator: ne

Usage example:

>>> data_str.filter(a__ne='abc')
[ItemA(a='ABC'), ItemA(a='def'), ItemA(a='abc/def')]

>>> data_int.filter(a__ne=1)
[ItemA(a=2), ItemA(a=3)]

Not-equal lookup (case insensitive)#

Lookup operator: ine

Usage example:

>>> data_str.filter(a__ine='abc')
[ItemA(a='def'), ItemA(a='abc/def')]

Contains lookup#

Lookup operator: contains

Works on Python types implementing object.__contains__() like str, sequences and sets.

Usage example:

>>> data_str.filter(a__contains='a')
[ItemA(a='abc'), ItemA(a='abc/def')]

>>> data_set.filter(a__contains=2)
[ItemA(a={1, 2}), ItemA(a={2, 3})]

Contains lookup (case insensitive)#

Lookup operator: icontains

Works on Python str and sets and sequences containing strings.

Usage example:

>>> data_str.filter(a__icontains='a')
[ItemA(a='abc'), ItemA(a='ABC'), ItemA(a='abc/def')]

In lookup#

Lookup operator: in

The reverse of contains. The lookup value must implement object.__contains__()

Usage example:

>>> data_str.filter(a__in={'abc', 'def', 'ghi'})
[ItemA(a='abc'), ItemA(a='def')]

>>> data_int.filter(a__in={0, 1, 2})
[ItemA(a=1), ItemA(a=2)]

In lookup (case insensitive)#

Lookup operator: iin

Usage example:

>>> data_str.filter(a__iin={'abc', 'def', 'ghi'})
[ItemA(a='abc'), ItemA(a='ABC'), ItemA(a='def')]

Starts-with lookup#

Lookup operator: startswith

Usage example:

>>> data_str.filter(a__startswith='a')
[ItemA(a='abc'), ItemA(a='abc/def')]

Starts-with lookup (case insensitive)#

Lookup operator: istartswith

Usage example:

>>> data_str.filter(a__istartswith='a')
[ItemA(a='abc'), ItemA(a='ABC'), ItemA(a='abc/def')]

Ends-with lookup#

Lookup operator: endswith

Usage example:

>>> data_str.filter(a__endswith='c')
[ItemA(a='abc')]

Ends-with lookup (case insensitive)#

Lookup operator: iendswith

Usage example:

>>> data_str.filter(a__iendswith='c')
[ItemA(a='abc'), ItemA(a='ABC')]

Glob lookup#

Lookup operator: glob

Matches a string against a pattern. The pattern is a string with special characters:

* matches any number of characters
? matches any single character
[seq] matches any character in seq
[!seq] matches any character not in seq

Unlike the rglob filter, path separators characters (/ and \) are not considered special and are matched by the * pattern.

Usage example:

>>> data_str.filter(a__glob='a*')
[ItemA(a='abc'), ItemA(a='abc/def')]

Glob lookup (case insensitive)#

Lookup operator: iglob

Matches a string against a pattern, case insensitive. The pattern is a string with special characters:

* matches any number of characters
? matches any single character
[seq] matches any character in seq
[!seq] matches any character not in seq

Unlike the irglob filter, path separators characters (/ and \) are not considered special and are matched by the * pattern.

Usage example:

>>> data_str.filter(a__iglob='a*')
[ItemA(a='abc'), ItemA(a='ABC'), ItemA(a='abc/def')]

Recursive glob lookup#

Lookup operator: rglob

Matches a string against a recursive glob pattern. The pattern is a string with special characters:

* matches any number of characters, excluding path separators
** matches any number of characters, including path separators (except a trailing path separator, see below)
? matches any single character
[seq] matches any character in seq
[!seq] matches any character not in seq

To match a string ending in a path separator (/ or \, indicating a directory path), the pattern must explicitly end in a path separator as well, i.e. ** alone will not match strings ending with a path separator, and **/ or **\ will only match strings ending with a path separator.

Usage example:

>>> data_str.filter(a__rglob='a*')
[ItemA(a='abc')]

>>> data_str.filter(a__rglob='a**')
[ItemA(a='abc'), ItemA(a='abc/def')]

Recursive glob lookup (case insensitive)#

Lookup operator: irglob

Matches a string against a recursive glob pattern, case insensitive. The pattern is a string with special characters:

* matches any number of characters, excluding path separators
** matches any number of characters, including path separators (except a trailing path separator, see below)
? matches any single character
[seq] matches any character in seq
[!seq] matches any character not in seq

To match a string ending in a path separator (/, indicating a directory path), the pattern must explicitly end in a path separator as well, i.e. ** alone will not match strings ending with a path separator, and **/ or **\ will only match strings ending with a path separator.

Usage example:

>>> data_str.filter(a__irglob='a*')
[ItemA(a='abc'), ItemA(a='ABC')]

>>> data_str.filter(a__irglob='a**')
[ItemA(a='abc'), ItemA(a='ABC'), ItemA(a='abc/def')]

Regular expression lookup#

Lookup operator: re

Usage example:

>>> data_str.filter(a__re=r'^.b.$')
[ItemA(a='abc')]

Regular expression lookup (case insensitive)#

Lookup operator: ire

Usage example:

>>> data_str.filter(a__ire=r'^.b.$')
[ItemA(a='abc'), ItemA(a='ABC')]

Additional methods#

FilterableList and FilterableTuple have several more functions:

ifilter(...)#: Same as filter(), but returning an iterable of items.

iexclude(...)#: Same as exclude(), but returning an iterable of items.

filter_with(func: Callable[[Any], bool])#: Filter the sequence using a filter function. The function receives an item and should return a boolean indicating if the item should be included in the result.

API#

class momotor.bundles.utils.filters.All(*filters, **filter_args)#

“All” filter operator

All the lookups in the argument list will need to match for an item to match this filter, see the All object documentation for an example.

(alias of F)

COMBINER#: alias of _All

NONE#: alias of _Always

class momotor.bundles.utils.filters.Any(*filters, **filter_args)#

“Any” filter operator

Any of the lookups in the argument list can match for an item to match this filter, see the Any object documentation for an example.

COMBINER#: alias of _Any

NONE#: alias of _Never

class momotor.bundles.utils.filters.F(*filters, **filter_args)#

Filter operator

Base for the other filter operators. Can be used to prevent multiple keywords SyntaxErrors in Python, see the F object documentation for an example.

COMBINER#: alias of _All

NONE#: alias of _Always

class momotor.bundles.utils.filters.FilterableList(iterable=(), /)#

A list with additional functions to filter the objects in the list

See Sequence filtering section in the documentation

append(object, /)#: Append object to the end of the list.

clear()#: Remove all items from list.

copy()#: Return a shallow copy of the list.

count(value, /)#: Return number of occurrences of value.

exclude(*args, **kwargs)#

Returns a filtered version of self that only includes elements that do not match the provided query

See Filtering and excluding

Return type:: TypeVar(TS, bound= Iterable)

extend(iterable, /)#: Extend list by appending elements from the iterable.

filter(*args, **kwargs)#

Returns a filtered version of self that only includes elements that match the provided query

See Filtering and excluding

Return type:: TypeVar(TS, bound= Iterable)

filter_with(func)#

Filter the items using a callable function

Parameters:: func (Callable[[TypeVar(TO, bound= object)], bool]) – A callable receiving an item and returning a boolean
Return type:: TypeVar(TS, bound= Iterable)

iexclude(*args, **kwargs)#

Iterator version of exclude()

Return type:: Iterable[TypeVar(TO, bound= object)]

ifilter(*args, **kwargs)#

Iterator version of filter()

Return type:: Iterable[TypeVar(TO, bound= object)]

index(value, start=0, stop=9223372036854775807, /)#

Return first index of value.

Raises ValueError if the value is not present.

insert(index, object, /)#: Insert object before index.

pop(index=-1, /)#

Remove and return item at index (default last).

Raises IndexError if list is empty or index is out of range.

remove(value, /)#

Remove first occurrence of value.

Raises ValueError if the value is not present.

reverse()#: Reverse IN PLACE.

sort(*, key=None, reverse=False)#

Sort the list in ascending order and return None.

The sort is in-place (i.e. the list itself is modified) and stable (i.e. the order of two equal elements is maintained).

If a key function is given, apply it once to each list item and sort them, ascending or descending, according to their function values.

The reverse flag can be set to sort in descending order.

class momotor.bundles.utils.filters.FilterableTuple(iterable=(), /)#

A tuple with additional functions to filter the objects in the tuple

See Sequence filtering section in the documentation

count(value, /)#: Return number of occurrences of value.

exclude(*args, **kwargs)#

Returns a filtered version of self that only includes elements that do not match the provided query

See Filtering and excluding

Return type:: TypeVar(TS, bound= Iterable)

filter(*args, **kwargs)#

Returns a filtered version of self that only includes elements that match the provided query

See Filtering and excluding

Return type:: TypeVar(TS, bound= Iterable)

filter_with(func)#

Filter the items using a callable function

Parameters:: func (Callable[[TypeVar(TO, bound= object)], bool]) – A callable receiving an item and returning a boolean
Return type:: TypeVar(TS, bound= Iterable)

iexclude(*args, **kwargs)#

Iterator version of exclude()

Return type:: Iterable[TypeVar(TO, bound= object)]

ifilter(*args, **kwargs)#

Iterator version of filter()

Return type:: Iterable[TypeVar(TO, bound= object)]

index(value, start=0, stop=9223372036854775807, /)#

Return first index of value.

Raises ValueError if the value is not present.

class momotor.bundles.utils.filters.Not(*filters, **filter_args)#

“Not” filter operator

Any item not matching all the lookup arguments will match this filter, see the Not object documentation for an example.

COMBINER#: alias of _All

NONE#: alias of _Always