ActivationDatasetWrapper

class hybrid_learning.datasets.activations_handle.ActivationDatasetWrapper(dataset, act_map_gen=None, activations_root=None, device=None, **data_args)

Bases: DatasetWrapper

Wrapper for image datasets that will generate and yield activation maps. Behaves like a sequence of tuples of

• activation maps (original input transformed by

  generate_act_map) and

• original ground truth (e.g. mask)

The wrapper can handle torch.utils.data.Subset and hybrid_learning.datasets.base.BaseDataset instances.

Features:

• Option to enable efficient file caching of the generated activation maps in order to avoid costly re-evaluations. See act_maps_cache.

• Convenience functions for caching: existence checks and cache filling with progress bar.

• Replacement of the activation generator by the cache, i.e. no act_map_gen must be provided if activation maps for all indices are cached (make sure to not clear the cache then though).

Activation map caching is enabled by setting activations_root and disabled by setting activations_root to None. To fill the cache i.e. generate all activation maps, call fill_cache(). But be aware that this can be very time consuming depending on the generator.

Public Data Attributes:

activations_root

The activations root of the file cache if caching is enable.

Inherited from : py: class:DatasetWrapper

dataset

The wrapped dataset.

Inherited from : py: class:BaseDataset

settings

Settings of the instance.

Public Methods:

getitem(i)	Get activation map and original ground truth for item at index i.
act_map_filepath(i)	Return the path to the activation map file in the cache.
act_map_exists(i)	Check whether the activation map at index i is cached.
load_image(i)	Load the image/original input for index i.
fill_cache([force_rebuild, show_progress_bar])	Generate activation maps for all images.

Inherited from : py: class:DatasetWrapper

getitem(i)	Get activation map and original ground truth for item at index i.
descriptor(i)	Wrap descriptor method of wrapped dataset.

Inherited from : py: class:BaseDataset

getitem(i)	Get activation map and original ground truth for item at index i.
descriptor(i)	Wrap descriptor method of wrapped dataset.

Special Methods:

__init__(dataset[, act_map_gen, ...])

Init.

Inherited from : py: class:DatasetWrapper

__init__(dataset[, act_map_gen, ...])	Init.
__len__()	Length determined by the length of the wrapped dataset.

Inherited from : py: class:BaseDataset

__init__(dataset[, act_map_gen, ...])	Init.
__len__()	Length determined by the length of the wrapped dataset.
__getitem__(idx)	Get item from idx in dataset with transformations applied.
__repr__()	Nice printing function.

Inherited from : py: class:Dataset

__getitem__(idx)	Get item from idx in dataset with transformations applied.
__add__(other)

---

__init__(dataset, act_map_gen=None, activations_root=None, device=None, **data_args)

Init.

The base settings (dataset root, split) default to those of the wrapped dataset.

Parameters

• dataset (BaseDataset) – Dataset to wrap; must be a sequence of tuples of (image, ground_truth) with both image and ground-truth of type torch.Tensor; the default transformation assumes that the ground truth are masks (same sized images)

• act_map_gen (Optional[Module]) – torch module that accepts as input a batch of images and returns the activation maps to yield

• activations_root (Optional[str]) – root directory under which to store and find the activation maps if file caching shall be enabled

• device (Optional[Union[str, device]]) – the device on which to run act_map_gen; see hybrid_learning.datasets.transforms.image_transforms.ToActMap

act_map_exists(i)

Check whether the activation map at index i is cached.

Parameters

i (int) – index in dataset for which to check whether an activation map was created.

Return type

bool

act_map_filepath(i)

Return the path to the activation map file in the cache. The base directory is activations_root. The basename is determined by the act_maps_cache from the descriptor() for the index i.

Parameters

i (int) – index of the image to get activation map for.

Returns

(relative or absolute) path to the activation map for datum at index i

Return type

str

fill_cache(force_rebuild=False, show_progress_bar=True, **kwargs)

Generate activation maps for all images.

Parameters

• force_rebuild (bool) – whether to overwrite existing images or not

• show_progress_bar (bool) – whether to show the progress using tqdm.tqdm

• kwargs – further arguments to the progress bar

Return type

ActivationDatasetWrapper

getitem(i)

Get activation map and original ground truth for item at index i.

Used for __getitem__(). If the activation map does not exist and a generator is given in generate_act_map, generate and save the activation map.

Returns

tuple of the loaded or generated activation map and the target of the original dataset for that act map

Parameters

i (int) –

Return type

Tuple[Tensor, Any]

load_image(i)

Load the image/original input for index i.

Parameters

i (int) –

Return type

Tensor

__parameters__ = ()

act_maps_cache: PTCache

File cache for caching activations. Set to None in case the activations root is set to False during init.

property activations_root: Optional[str]

The activations root of the file cache if caching is enable. Enable caching by setting this to a file path, disable caching by setting this attribute to None.

generate_act_map: Callable[[Tensor], Tensor]

Transformation that returns an activation map given a valid input datum. Input data is assumed to origin from the original dataset. Used to generate missing activation maps in getitem().