ModelStump

class hybrid_learning.concepts.models.model_extension.ModelStump(model, stump_head)

Bases: HooksHandle

Obtain the intermediate output of a sub-module of a complete NN. This is a smaller version of the ActivationMapGrabber:

• It only handles the output of one sub-module, its stump head.

• It does not retrieve the output of the main model.

In the other points it is the same as ActivationMapGrabber.

Public Data Attributes:

stump_head

ID of the sub-module from which the activation maps are retrieved.

Inherited from : py: class:HooksHandle

registered_submodules

List of IDs of the registered sub-modules.

Inherited from : py: class:Module

dump_patches	This allows better BC support for load_state_dict().
T_destination	alias of TypeVar('T_destination', bound=Mapping[str, Tensor])

Public Methods:

register_submodule(module_id)	Register a sub-module hook.
unregister_submodule(module_id)	Unregister a submodule for intermediate output retrieval.
forward(*inps)	Pytorch forward method: Return intermediate output of stump head.

Inherited from : py: class:HooksHandle

register_submodule(module_id)	Register a sub-module hook.
unregister_submodule(module_id)	Unregister a submodule for intermediate output retrieval.
get_module_by_id(m_id)	Get actual sub-module object within wrapped model by module ID.
forward(*inps)	Pytorch forward method: Return intermediate output of stump head.

Inherited from : py: class:Module

forward(*inps)	Pytorch forward method: Return intermediate output of stump head.
register_buffer(name, tensor[, persistent])	Adds a buffer to the module.
register_parameter(name, param)	Adds a parameter to the module.
add_module(name, module)	Adds a child module to the current module.
get_submodule(target)	Returns the submodule given by target if it exists, otherwise throws an error.
get_parameter(target)	Returns the parameter given by target if it exists, otherwise throws an error.
get_buffer(target)	Returns the buffer given by target if it exists, otherwise throws an error.
apply(fn)	Applies fn recursively to every submodule (as returned by .children()) as well as self.
cuda([device])	Moves all model parameters and buffers to the GPU.
xpu([device])	Moves all model parameters and buffers to the XPU.
cpu()	Moves all model parameters and buffers to the CPU.
type(dst_type)	Casts all parameters and buffers to dst_type.
float()	Casts all floating point parameters and buffers to float datatype.
double()	Casts all floating point parameters and buffers to double datatype.
half()	Casts all floating point parameters and buffers to half datatype.
bfloat16()	Casts all floating point parameters and buffers to bfloat16 datatype.
to_empty(*, device)	Moves the parameters and buffers to the specified device without copying storage.
to(*args, **kwargs)	Moves and/or casts the parameters and buffers.
register_backward_hook(hook)	Registers a backward hook on the module.
register_full_backward_hook(hook)	Registers a backward hook on the module.
register_forward_pre_hook(hook)	Registers a forward pre-hook on the module.
register_forward_hook(hook)	Registers a forward hook on the module.
state_dict([destination, prefix, keep_vars])	Returns a dictionary containing a whole state of the module.
load_state_dict(state_dict[, strict])	Copies parameters and buffers from state_dict into this module and its descendants.
parameters([recurse])	Returns an iterator over module parameters.
named_parameters([prefix, recurse])	Returns an iterator over module parameters, yielding both the name of the parameter as well as the parameter itself.
buffers([recurse])	Returns an iterator over module buffers.
named_buffers([prefix, recurse])	Returns an iterator over module buffers, yielding both the name of the buffer as well as the buffer itself.
children()	Returns an iterator over immediate children modules.
named_children()	Returns an iterator over immediate children modules, yielding both the name of the module as well as the module itself.
modules()	Returns an iterator over all modules in the network.
named_modules([memo, prefix, remove_duplicate])	Returns an iterator over all modules in the network, yielding both the name of the module as well as the module itself.
train([mode])	Sets the module in training mode.
eval()	Sets the module in evaluation mode.
requires_grad_([requires_grad])	Change if autograd should record operations on parameters in this module.
zero_grad([set_to_none])	Sets gradients of all model parameters to zero.
share_memory()	See torch.Tensor.share_memory_()
extra_repr()	Set the extra representation of the module

Special Methods:

__init__(model, stump_head)

Init.

Inherited from : py: class:HooksHandle

__init__(model, stump_head)	Init.
__del__()	Unregister all hooks held by this handle on handle delete.

Inherited from : py: class:Module

__init__(model, stump_head)	Init.
__call__(*input, **kwargs)	Call self as a function.
__setstate__(state)
__getattr__(name)
__setattr__(name, value)	Implement setattr(self, name, value).
__delattr__(name)	Implement delattr(self, name).
__repr__()	Return repr(self).
__dir__()	Default dir() implementation.

---

Parameters

• model (Module) –

• stump_head (str) –

__init__(model, stump_head)

Init.

Parameters

• model (Module) – model to obtain intermediate output from.

• stump_head (str) –

  ID of the sub-module from which to obtain intermediate output.

  Note

  If the sub-module occurs several times, only the first output is collected.

forward(*inps)

Pytorch forward method: Return intermediate output of stump head. Provides __call__ functionality.

register_submodule(module_id)

Register a sub-module hook. If stump_head is unset, set it to this sub-module.

Parameters

module_id (str) –

Return type

None

unregister_submodule(module_id)

Unregister a submodule for intermediate output retrieval.

Parameters

module_id (str) –

Return type

None

hook_handles: Dict[str, torch.utils.hooks.RemovableHandle]

Dictionary of hooks; for each sub-module to grab output from, a hook is registered. On each forward, the hook for a sub-module of ID m writes the intermediate output of the sub-module into _intermediate_outs[m]. The dictionary saves for the sub-module ID the hook handle.

property stump_head: str

ID of the sub-module from which the activation maps are retrieved.

training: bool

wrapped_model: torch.nn.modules.module.Module

Original model from which intermediate and final output are retrieved.