IsPartOfA

class hybrid_learning.fuzzy_logic.predicates.custom_ops.IsPartOfA(in_key, *, kernel_size=10, thresh=0.1, logical_and=None, conv_hang_front=True, **other_setts)[source]

Bases: TorchOperation

Unary predicate that calculates for each pixel location whether it is part of an object segmentation. The predicate accepts as input the segmentation mask information for objects (encoding the predicate “pixel p is a A” = IsA(p)). It returns a mask that for each pixel holds the truth value, whether that pixel is part of an object, i.e. at each pixel position p the output is

\[\begin{split}\exists p_o: \text{IsA}(p_o) \wedge \text{IsPartOf}(p, p_o) \\ = \max_{p_o} \text{IsA}(p_o) \wedge \text{IsPartOf}(p, p_o)\end{split}\]

It accepts 1D or 2D object masks of the shape [[batch x [channels x]] height x] width, in any format that can be parsed to a torch.Tensor.

Fuzzification

As fuzzification of the exists quantifier, max is used. The logical_and defines the fuzzification of AND. The values of the predicate IsA(.) are given by the pixel values of the input segmentation mask. The fuzzy IsPartOf relation is chosen to calculate as a Gaussian distance, with a threshold of value thresh and distance thresh_radius=int(kernel_size - 1) / 2 for better computational performance:

\[\begin{split}d(p, p') = \exp(- \frac{\|p - p'\|_2^2}{2 \sigma^2}) \\ \text{IsPartOf}(p, p') := \begin{cases} d(p, p') \text{if} \|p, p'\|_1 <= \text{thresh\_radius} \\ 0 \text{else} \end{cases}\end{split}\]

with \(\sigma\) chosen such that \(d(p, p') = \text{thresh}\) for \(\|p - p'\|_2 = \text{thresh\_radius}\). The per-pixel values of the IsPartOf predicate are stored in the _ispartof_values mask which is then convoluted with the IsA values.

Note

An even kernel_size will have the same _ispartof_values pixel values as the odd kernel_size-1, only with a row and a column added (on top/left for conv_hang_front True, else on bottom/right).

Choosing the right ``kernel_size``

The shape of the used Gaussian can be defined by a pair of x-y-values (r, t). The Gaussian then looks like

\[G_{r,t}(x) = \exp\left(- \frac{ x^2 }{ \frac{r^2}{- \ln(t)}}} \right)\]

As thresh_radius, int(kernel_size - 1) / 2 is used, which also serves as the cutoff distance. To choose the kernel_size such that (1) the point (r, t) lies on the Gaussian, and (2) the kernel_size cuts at threshold thresh, solve \(G_{r,t}(\frac{\text{k}-1}{2}) = \text{thresh}\) for \(k\) and gets kernel_size=int(k). E.g. to match the constraints that at 4 pixel shift (r=4), \(G(r)\) should still be 0.8, and the kernel should only cut off at a threshold of 0.1, the kernel_size must be 2*int(12.4) + 1=25.

Visualization example

Visualization of the different logics:

>>> import hybrid_learning.fuzzy_logic as fl
>>> import torch, torchvision.transforms.functional as F, matplotlib.pyplot as plt
>>> # Builder for IsPartOf
>>> P = fl.predicates.custom_ops.IsPartOfA.with_(kernel_size=10, thresh=0.01, conv_hang_front=False)
>>> # The sample input mask
>>> t = torch.zeros([1, 1, 11, 11])
>>> t[:, :, 4, 4], t[:, :, 5, 8] = 0.75, 0.5
>>> # The tensors to compare
>>> figs = dict(segmask=t)  # input
>>> for logic in (fl.LukasiewiczLogic(), fl.ProductLogic(), fl.GoedelLogic()):
...    title = f'IsPartOfA ({logic.__class__.__name__.replace("Logic", "")})'
...    p = P("pedestrian", logical_and=logic.logical_('AND'))
...    figs[title] = p({"pedestrian": t})[p.out_key]
>>> # Plot
>>> fig, axes = plt.subplots(1, len(figs), figsize=(len(figs)*3, 3), squeeze=False)
>>> for i, (title, tens) in enumerate(figs.items()):
...     shown_img = axes[0, i].imshow(F.to_pil_image(tens.view(t.size()[2:])))
...     _ = axes[0, i].set_title(title)
>>> plt.subplots_adjust(bottom=0.1, right=0.8, top=0.9)
>>> cax = plt.axes([.85, .25, 0.01, 0.5])
>>> _ = plt.colorbar(shown_img, cax=cax, orientation='vertical')
>>> plt.show()

Public Data Attributes:

`SYMB`	The string symbol of this class (override for sub-classes).
`ARITY`	The arity of the operation.
`kernel_size`	The size of the kernel to use.
`thresh`	The value the Gaussian should have at `1/2 kernel_size`.
`conv_hang_front`	Whether the cutoff modelled by the `IsPartOfA` convolution should hang in the front dimensions in case of unequal padding.
`settings`	Settings to reproduce the instance.
`setting_defaults`	Defaults used for settings.

Inherited from : py: class:Merge

`SYMB`	The string symbol of this class (override for sub-classes).
`ARITY`	The arity of the operation.
`IS_COMMUTATIVE`	Whether instances are equivalent to ones with permuted `in_keys`.
`is_variadic`	Whether the instance is variadic.
`settings`	Settings to reproduce the instance.
`setting_defaults`	Defaults used for settings.
`pretty_op_symb`	Name of the operation symbol suitable for filenames etc.
`children`	The input keys which are child operations.
`all_children`	All children operations in the flattened computational tree, sorted depth first.
`consts`	The constant string keys in the input keys.
`operation_keys`	The list of keys used for this parent operation in original order (constants and children output keys).
`all_in_keys`	All string input keys both of self and of all child operations.
`all_out_keys`	Output keys of self and all child operations.

Inherited from : py: class:DictTransform

settings

Settings to reproduce the instance.

Inherited from : py: class:Transform

`IDENTITY_CLASS`	The identity class or classes for composition / addition.
`settings`	Settings to reproduce the instance.

Public Methods:

torch_operation(mask)

Calculate value of IsPartOfA predicate for given object segmentation mask.

Inherited from : py: class:TorchOperation

operation(annotation_vals)

Calculate the predicate output.

Inherited from : py: class:Merge

`to_infix_notation`([sort_key, ...])	Return an infix str encoding equal for differently sorted operations.
`to_str`(**infix_notation_kwargs)	Alias for `to_infix_notation()`.
`to_pretty_str`(**infix_notation_kwargs)	Same as `to_str()` but using pretty operation names suitable for filenames etc.
`to_repr`([settings, defaults, sort_key, ...])	Return str representation which can be used to reproduce and compare the instance.
`treerecurse_replace_keys`(**replace_map)	Return a new formula with all occurences of variables in `replace_map` replaced and else identical settings.
`treerecurse`(fun)	Apply the given function recursively to this and all children instances.
`apply_to`(annotations[, keep_keys])	Apply this operation to the `annotations` dict.
`variadic_apply_to`(annotations)	Return the result of operation on the values/items of a mapping or sequence of arbitrary length.
`operation`(annotation_vals)	Calculate the predicate output.

Inherited from : py: class:DictTransform

apply_to(annotations[, keep_keys])

Apply this operation to the annotations dict.

Inherited from : py: class:Transform

apply_to(annotations[, keep_keys])

Apply this operation to the annotations dict.

Special Methods:

__init__(in_key, *[, kernel_size, thresh, ...])

Init.

Inherited from : py: class:Merge

`__init__`(in_key, *[, kernel_size, thresh, ...])	Init.
`__str__`()	Return str(self).
`__repr__`()	Call `to_repr()` without sorting.
`__eq__`(other)	Two merge operations are considered equal, if their normalized representations coincide.
`__copy__`()	Return a deep copy of self using settings.
`__call__`(annotations[, keep_keys])	Call method modifying a given dictionary.

Inherited from : py: class:DictTransform

__call__(annotations[, keep_keys])

Call method modifying a given dictionary.

Inherited from : py: class:Transform

`__repr__`()	Call `to_repr()` without sorting.
`__eq__`(other)	Two merge operations are considered equal, if their normalized representations coincide.
`__copy__`()	Return a deep copy of self using settings.
`__add__`(other)	Return a flat composition of `self` with `other`.
`__radd__`(other)	Return a flat composition of `other` and `self`.
`__call__`(annotations[, keep_keys])	Call method modifying a given dictionary.

Parameters

in_key (Union[str, Merge]) –
kernel_size (int) –
thresh (float) –
logical_and (Callable[[List[Tensor]], Tensor]) –
conv_hang_front (bool) –

__init__(in_key, *, kernel_size=10, thresh=0.1, logical_and=None, conv_hang_front=True, **other_setts)[source]

Init.

The parameters kernel_size and thresh are specifiers for the distance function encoding the is-part-of relation. The logical_and is the logical AND callable that accepts a list of two tensors with values in [0, 1] and returns their pixel-wise logical AND. The in_key is the key of the fuzzy mask which encodes the per-pixel membership degree to the class of interest. E.g. if “is part of a pedestrian” should be calculated, "pedestrian" is a good choice.

Parameters

in_key (Union[str, Merge]) –
kernel_size (int) –
thresh (float) –
logical_and (Optional[Callable[[List[Tensor]], Tensor]]) –
conv_hang_front (bool) –

static create_ispartof_kernel(kernel_size, thresh, thresh_radius=None, conv_hang_front=True)[source]

Create the kernel tensor defining the IsPartOf relation. For details see _ispartof_values.

Parameters

kernel_size (int) –
thresh (float) –
thresh_radius (Optional[float]) –
conv_hang_front (bool) –

Return type

Tensor

torch_operation(mask)[source]

Calculate value of IsPartOfA predicate for given object segmentation mask. Allowed mask shapes (1D-4D): ([[batch, [1,]] height,] width) The output shape is that of the input mask. Invalid shapes raise a ValueError.

The device of mask is used for calculations and will be the device of the output tensor. As a side effect, the internal storage of the IsPartOf values is moved to that device, assuming that the device of inputs does not change frequently.

Parameters: mask (Tensor) –
Return type: Tensor

ARITY: int = 1: The arity of the operation. -1 means unlimited number of arguments possible.

SYMB: str = 'IsPartOfA': The string symbol of this class (override for sub-classes).

property conv_hang_front: bool: Whether the cutoff modelled by the IsPartOfA convolution should hang in the front dimensions in case of unequal padding. Internals: To achieve hanging towards the back, the padding must be chosen to hang front.

property kernel_size: int: The size of the kernel to use. The standard deviation of the gaussian is determined using this as radius and thresh.

logical_and: Callable[[List[Tensor]], Tensor]: Callable accepting a list of two tensors with values in [0, 1] and returning their pixel-wise logical AND. Mandatory argument.

property setting_defaults: Defaults used for settings.

property settings: Dict[str, Any]: Settings to reproduce the instance. (Mind that in_keys must be expanded! For direct reproduction use copy.)

property thresh: float: The value the Gaussian should have at 1/2 kernel_size.