Cache

class hybrid_learning.datasets.caching.Cache

Bases: ABC

Caching base handle. Put objects into the cache using put(), and load cached objects by their cache descriptor using load(). Derive custom caching handles from this class.

Public Methods:

put(descriptor, obj)	Store obj in this cache.
load(descriptor)	Load the object stored under key descriptor from cache.
put_batch(descriptors, objs)	Store a batch of objs in this cache using according descriptors.
load_batch(descriptors[, return_none_if])	Load a batch of objects.
clear()	Clear the current cache.
descriptors()	Return all descriptors for which an element is cached.
as_dict()	Return a dict with all cached descriptors and objects.
wrap(getitem[, descriptor_map])	Add this cache to the deterministic function getitem (which should have no side effects).

Special Methods:

__repr__()	Return repr(self).
__add__(other)	Return a (cascaded) cache which will first lookup self then other with default sync mode.
__radd__(other)	Return a (cascaded) cache which will first lookup other then self with default sync mode.

---

__add__(other)

Return a (cascaded) cache which will first lookup self then other with default sync mode. In case other is None or a dummy NoCache, return self.

Returns

one of the summands in case the other is a no-op, else a CacheCascade transforms.

Parameters

other (Optional[Cache]) –

Return type

Cache

__radd__(other)

Return a (cascaded) cache which will first lookup other then self with default sync mode. See __add__().

Parameters

other (Optional[Cache]) –

Return type

Cache

__repr__()

Return repr(self).

Return type

str

as_dict()

Return a dict with all cached descriptors and objects. Beware: This can be very large!

Return type

Dict

abstract clear()

Clear the current cache.

abstract descriptors()

Return all descriptors for which an element is cached.

Return type

Iterable

abstract load(descriptor)

Load the object stored under key descriptor from cache. None is returned if the object is not in the cache.

Parameters

descriptor (Hashable) –

Return type

Optional[Any]

load_batch(descriptors, return_none_if='any')

Load a batch of objects. Return None according to return_none_if.

Parameters

• descriptors (Iterable[Hashable]) – descriptors to load values for

• return_none_if (Union[str, int]) – may be "any", "all", "never"

Return type

Optional[Collection]

abstract put(descriptor, obj)

Store obj in this cache. In case it already exists, the existing object is overwritten.

Parameters

• descriptor (Hashable) – the descriptor key under which to store the object; used to access the object later

• obj (Any) – the object to put into cache; must not be None

put_batch(descriptors, objs)

Store a batch of objs in this cache using according descriptors.

Parameters

• descriptors (Iterable[Hashable]) –

• objs (Iterable[Any]) –

wrap(getitem, descriptor_map=None)

Add this cache to the deterministic function getitem (which should have no side effects). When the wrapped method is called, the desired item from cache is returned. Only if it does not exist in the cache, the original getitem is called, and its output cached and returned. The optional descriptor_map function should map a getitem-input to the hash value which is to be used for the cache. E.g. it could map an index to the underlying file name.

getitem should

• have no side effects, and

• have deterministic output, i.e. calls with equal input value will return equal output values.

descriptor_map should

• accept elements from the same domain as getitem,

• be injective, i.e. map each getitem-input to a unique descriptor value.

Parameters

• getitem (Callable[[int], Any]) – the function to wrap; for requirements see above

• descriptor_map (Optional[Callable[[int], Hashable]]) – optionally a map from getitem-input to descriptor

Return type

Callable[[int], Any]