BatchIntersectEncode2D

class hybrid_learning.datasets.transforms.encoder.BatchIntersectEncode2D(proto_shape=None, kernel_size=None, normalize_by='proto_shape')

Bases: BatchConvOp

Apply intersection encoding to batch of input masks of shape (batch, 1, height, width).

The idea of intersection encoding is to have a generalized and continuous bounding box score. The better a given proto-shape (e.g. a box) intersects with the ground truth shape, the higher the value in [0,1]. More precisely, in an IoU encoding of a segmentation target, a pixel holds the value how much of a pre-defined proto-shape centered at this pixel intersects with the ground truth segmentation.

Such a proto shape is defined by a (not necessarily binary) mask, which may be at most the size of the target mask.

Theoretical notes

The set intersection is implemented as fuzzy set intersection with respect to the product t-norm. This means, the intersection is element_wise_prod(A, B), and the value of a pixel in a mask is treated similar to probability that the pixel belongs to the mask. We use reduction by sum to calculate the area of a fuzzy mask.

Using sum reduction allows to use the intersection area value for union calculation: The product t-norm fuzzy union is defined as element-wise \(A + B - A \cdot B\) for sets \(A, B\). Thus, the union area is:

sum(A + B - A*B) = sum(A) + sum(B) - sum(A*B)

where sum is the reduction by sum and sum(A*B) is the intersection area.

The following holds for a factor c:

intersect(c*A, B) == c * intersect(A, B).

This is used to normalize the intersection value by the proto shape size if requested, i.e. c=area(proto shape).

Note

In the case that at least one of proto shape or the segmentation mask is binary with values in {0, 1}, the product t-norm fuzzy intersection is equivalent to the minimum t-norm fuzzy intersection, which is element_wise_min(A, B). If both proto shape and segmentation feature non-binary values, the product intersection can become much smaller than the minimum intersection.

Implementation Notes

The intersection calculation for each possible “center pixel”, i.e. proto shape location, is done by applying a convolution to the segmentation mask, the weights of which hold the mask.

To change from 2D to other image dimensionality, replace the padding and convolution layer and adapt AREA_DIMS accordingly.

Public Data Attributes:

proto_shape	The proto shape used for intersection calculation
settings	Settings to reproduce instance.

Inherited from : py: class:BatchConvOp

proto_shape	The proto shape used for intersection calculation
kernel_size	The kernel size of the proto-type shape.
settings	Settings to reproduce instance.

Inherited from : py: class:BatchWindowOp

AREA_DIMS	Indices of axes in which the image area is defined.
kernel_size	The kernel size of the proto-type shape.
settings	Settings to reproduce instance.

Inherited from : py: class:Module

dump_patches	This allows better BC support for load_state_dict().
T_destination	alias of TypeVar('T_destination', bound=Mapping[str, Tensor])

Public Methods:

conv_op(masks)

Encode given masks as IoU masks.

Inherited from : py: class:BatchWindowOp

forward(masks)	Wrapper for the convolutional operation on batch of masks.
conv_op(masks)	Encode given masks as IoU masks.

Inherited from : py: class:Module

forward(masks)	Wrapper for the convolutional operation on batch of masks.
register_buffer(name, tensor[, persistent])	Adds a buffer to the module.
register_parameter(name, param)	Adds a parameter to the module.
add_module(name, module)	Adds a child module to the current module.
get_submodule(target)	Returns the submodule given by target if it exists, otherwise throws an error.
get_parameter(target)	Returns the parameter given by target if it exists, otherwise throws an error.
get_buffer(target)	Returns the buffer given by target if it exists, otherwise throws an error.
apply(fn)	Applies fn recursively to every submodule (as returned by .children()) as well as self.
cuda([device])	Moves all model parameters and buffers to the GPU.
xpu([device])	Moves all model parameters and buffers to the XPU.
cpu()	Moves all model parameters and buffers to the CPU.
type(dst_type)	Casts all parameters and buffers to dst_type.
float()	Casts all floating point parameters and buffers to float datatype.
double()	Casts all floating point parameters and buffers to double datatype.
half()	Casts all floating point parameters and buffers to half datatype.
bfloat16()	Casts all floating point parameters and buffers to bfloat16 datatype.
to_empty(*, device)	Moves the parameters and buffers to the specified device without copying storage.
to(*args, **kwargs)	Moves and/or casts the parameters and buffers.
register_backward_hook(hook)	Registers a backward hook on the module.
register_full_backward_hook(hook)	Registers a backward hook on the module.
register_forward_pre_hook(hook)	Registers a forward pre-hook on the module.
register_forward_hook(hook)	Registers a forward hook on the module.
state_dict([destination, prefix, keep_vars])	Returns a dictionary containing a whole state of the module.
load_state_dict(state_dict[, strict])	Copies parameters and buffers from state_dict into this module and its descendants.
parameters([recurse])	Returns an iterator over module parameters.
named_parameters([prefix, recurse])	Returns an iterator over module parameters, yielding both the name of the parameter as well as the parameter itself.
buffers([recurse])	Returns an iterator over module buffers.
named_buffers([prefix, recurse])	Returns an iterator over module buffers, yielding both the name of the buffer as well as the buffer itself.
children()	Returns an iterator over immediate children modules.
named_children()	Returns an iterator over immediate children modules, yielding both the name of the module as well as the module itself.
modules()	Returns an iterator over all modules in the network.
named_modules([memo, prefix, remove_duplicate])	Returns an iterator over all modules in the network, yielding both the name of the module as well as the module itself.
train([mode])	Sets the module in training mode.
eval()	Sets the module in evaluation mode.
requires_grad_([requires_grad])	Change if autograd should record operations on parameters in this module.
zero_grad([set_to_none])	Sets gradients of all model parameters to zero.
share_memory()	See torch.Tensor.share_memory_()
extra_repr()	Set the extra representation of the module

Special Methods:

__init__([proto_shape, kernel_size, ...])	Init.
__repr__()	Representation based on this instances settings.

Inherited from : py: class:BatchWindowOp

__repr__()

Representation based on this instances settings.

Inherited from : py: class:Module

__init__([proto_shape, kernel_size, ...])	Init.
__call__(*input, **kwargs)	Call self as a function.
__setstate__(state)
__getattr__(name)
__setattr__(name, value)	Implement setattr(self, name, value).
__delattr__(name)	Implement delattr(self, name).
__repr__()	Representation based on this instances settings.
__dir__()	Default dir() implementation.

---

Parameters

• proto_shape (ndarray) –

• kernel_size (Tuple[int, ...]) –

• normalize_by (str) –

__init__(proto_shape=None, kernel_size=None, normalize_by='proto_shape')

Init.

Parameters

• proto_shape (Optional[ndarray]) – the proto shape definition in a form accepted by numpy.ndarray()

• kernel_size (Optional[Tuple[int, ...]]) – if proto_shape is None, use all-ones rectangular shape of kernel_size

• normalize_by (str) – whether to normalize the intersection output, i.e. divide it by the total area of the proto shape or the target, or none; allowed options: 'proto_shape', 'target', 'none'

__repr__()

Representation based on this instances settings.

conv_op(masks)

Encode given masks as IoU masks.

Parameters

masks (Tensor) – torch.Tensor of shape (batch_dim, 1, width, height) holding the segmentation masks for one batch.

Returns

tensor of the same size as masks tensor holding IoU encoding of the latter

Return type

Tensor

intersect_conv

Convolution to calculate the intersection for each location of proto_shape

normalize_by

Whether to normalize the intersection area. Divide the intersection area value by:

Value of normalize_by	Divisor = total area of …
'proto_shape'	… the proto shape
'target'	… the target.

padding

Padding to obtain same size as input after convolution

property proto_shape: numpy.ndarray

The proto shape used for intersection calculation

property settings: Dict[str, Any]

Settings to reproduce instance.

training: bool