CacheCascade

class hybrid_learning.datasets.caching.CacheCascade(*caches, sync_by='precedence')

Bases: Cache

Combine several caches by trying to load from first to last. In case of a put, all caches are updated. In case of a load, the object is collected from the first cache holding it. In case and all previous ones are updated to also hold it. If sync_by is True, then on load the first match is put to all other cache instances, not only the previous ones. I.e. the order of caches also determines the precedence.

Some use-cases:

• Combine in-memory and persistent cache: Combine a DictCache with a FileCache instance: Files are stored in file system for later runs, respectively loaded from previous runs, and additionally for even faster access stored in memory.

• Cache to different cache locations: Combine several FileCache caches with sync_by=True to write cache to several locations (can be used as sort of a lazy copy).

Public Methods:

put(descriptor, obj)	Put obj to all caches under key descriptor.
load(descriptor)	Load object stored under descriptor from the cache with highest precedence holding it.
clear()	Clears all caches in the cascade.
descriptors()	This returns the united descriptor lists of all sub-caches.
append(cache)	Append cache to the cascade and return self.
insert(i, cache)	Insert cache at position i in cascade and return self.
remove(i)	Remove the cache at position i in cascade and return self.

Inherited from : py: class:Cache

put(descriptor, obj)	Put obj to all caches under key descriptor.
load(descriptor)	Load object stored under descriptor from the cache with highest precedence holding it.
put_batch(descriptors, objs)	Store a batch of objs in this cache using according descriptors.
load_batch(descriptors[, return_none_if])	Load a batch of objects.
clear()	Clears all caches in the cascade.
descriptors()	This returns the united descriptor lists of all sub-caches.
as_dict()	Return a dict with all cached descriptors and objects.
wrap(getitem[, descriptor_map])	Add this cache to the deterministic function getitem (which should have no side effects).

Special Methods:

__init__(*caches[, sync_by])	Init.
__repr__()	Return repr(self).

Inherited from : py: class:Cache

__repr__()	Return repr(self).
__add__(other)	Return a (cascaded) cache which will first lookup self then other with default sync mode.
__radd__(other)	Return a (cascaded) cache which will first lookup other then self with default sync mode.

---

Parameters

• caches (Cache) –

• sync_by (Optional[Union[bool, str]]) –

__init__(*caches, sync_by='precedence')

Init.

Parameters

• caches (Cache) – cache instances in the correct load order

• sync_by (Optional[Union[bool, str]]) – synchronization mode; must be one of 'none' / False, 'precedence', 'all'; for details see sync_by

__repr__()

Return repr(self).

Return type

str

append(cache)

Append cache to the cascade and return self.

Parameters

cache (Cache) –

Return type

CacheCascade

clear()

Clears all caches in the cascade.

descriptors()

This returns the united descriptor lists of all sub-caches.

Warning

This may be very computationally expensive, depending on the size of the lists to merge into a set. So use with care!

Return type

Iterable

insert(i, cache)

Insert cache at position i in cascade and return self.

Parameters

• i (int) –

• cache (Cache) –

Return type

CacheCascade

load(descriptor)

Load object stored under descriptor from the cache with highest precedence holding it. Possibly update other cache instances according to sync_by mode.

Parameters

descriptor (Hashable) –

Return type

Optional[Any]

put(descriptor, obj)

Put obj to all caches under key descriptor. Beware that the key may be changed (e.g. transformed to string) in sub-caches, leading to non-unique descriptors.

Parameters

• descriptor (Hashable) –

• obj (Any) –

remove(i)

Remove the cache at position i in cascade and return self.

Parameters

i (int) –

Return type

CacheCascade

caches: List[Cache]

The list of caches to consult during load, ordered by descending precedence.

sync_by: bool

Synchronization mode for loading. Update other instances according to the following settings:

• False: no sync; simply load from the first cache holding an object without updating the others

• 'precedence': when loading, update all caches with higher precedence (earlier in the caches list) that do not hold the object

• 'all': put object to all other caches when a value was loaded from one