Merge

class hybrid_learning.fuzzy_logic.logic_base.merge_operation.Merge(*in_keys, out_key=None, overwrite=True, skip_none=True, replace_none=None, symb=None, cache_duplicates=True, keep_keys=None, _variadic=False)[source]

Bases: DictTransform, ABC

Base class for operations and operation trees on dictionary inputs. Merge the masks or scalars values of the dict input according to the operation (tree) definition and store them under the specified output key. The merge operation may recursively have child merge operations as in_keys, which are evaluated on the given dictionary before the parent is.

Operation

The actual operation is hidden in the apply_to() method: It is given a dictionary of annotations of the form {ID: value} and will return the dict with the merged mask added as {out_key: value}. The intermediate outputs of child operations are by default only used for caching (see cache_duplicates) and then discarded. To include them into the final output, use the keep_keys argument to the operation call (see apply_to()). The benefit of caching duplicates is that results may be reused amongst different operations.

Initialization

During init, all non-keyword arguments serve as in_keys. These are used when the merge operation is called on a dict: The dict must provide items with these in_keys, and the values of these items are fed to the actual operation. Settings must be given as keyword arguments. To set default keyword arguments for the init call, use a MergeBuilder. See with_() for creating a MergeBuilder from a Merge class.

Example: Boolean Logic

To get all heads, noses, and mouths (binary masks) of real persons (binary masks) in bathrooms (boolean labels), call:

>>> from hybrid_learning.fuzzy_logic.tnorm_connectives.boolean import AND, OR, NOT, BooleanLogic
>>> op = AND("person", OR("head", "nose", "mouth"), NOT("bathroom"))
>>> op == BooleanLogic().parser()("person&&head||nose||mouth&&~bathroom")
True
>>> # Example with 1 pixel of a person mouth not in a bathroom:
>>> result = op({"person": 1, "head": 0, "nose": 0, "mouth": 1, "bathroom": False})
>>> result[op.out_key] == 1
True
>>> result
{'person': 1, 'head': 0, 'nose': 0, 'mouth': 1, 'bathroom': False,
'(head||mouth||nose)&&(~bathroom)&&person': 1}

To also inspect the intermediate output, use the keep_keys option:

>>> op({"person": 1, "head": 0, "nose": 0, "mouth": 1, "bathroom": False},
...    keep_keys=op.all_out_keys)
{'person': 1, 'head': 0, 'nose': 0, 'mouth': 1, 'bathroom': False,
'head||mouth||nose': True,
'~bathroom': True,
'(head||mouth||nose)&&(~bathroom)&&person': 1}

Note that the input dict must feature all in_keys of operations in the formula.

Subclassing

To implement your own merge operation

implement the operation()
specify your own SYMB (this must be unique within the logic you are using)
extend the settings and setting_defaults properties by new items if necessary

Format and String Parsing

The (recursive) merge operation best is specified in conjunctive normal form for uniqueness (thus comparability) and parsing compatibility. This is the form

AND(..., [NOT(...), ...], [OR(..., [NOT(..), ...])])

(see https://en.wikipedia.org/wiki/Conjunctive_normal_form). Exemplary available operations are the Boolean ones AND (intersection), OR (union), and NOT (inversion) that all operate pixel-wise. Boolean classification labels are treated as all-one-masks.

One can use a FormulaParser implementation to parse a string representation of an operation tree. Check the corresponding implementation for the operator precedence and examples. For parsing, used connectors of the logic must be encoded by their SYMB attribute, e.g. for the examples above:

AND: a&&b
OR: a||b
NOT (unary operation): ~a

Public Data Attributes:

`SYMB`	The string symbol of this class (override for sub-classes).
`ARITY`	The arity of the operation.
`IS_COMMUTATIVE`	Whether instances are equivalent to ones with permuted `in_keys`.
`is_variadic`	Whether the instance is variadic.
`settings`	Settings to reproduce the instance.
`setting_defaults`	Defaults used for `settings`.
`pretty_op_symb`	Name of the operation symbol suitable for filenames etc.
`children`	The input keys which are child operations.
`all_children`	All children operations in the flattened computational tree, sorted depth first.
`consts`	The constant string keys in the input keys.
`operation_keys`	The list of keys used for this parent operation in original order (constants and children output keys).
`all_in_keys`	All string input keys both of self and of all child operations.
`all_out_keys`	Output keys of self and all child operations.

Inherited from : py: class:DictTransform

settings

Settings to reproduce the instance.

Inherited from : py: class:Transform

`IDENTITY_CLASS`	The identity class or classes for composition / addition.
`settings`	Settings to reproduce the instance.

Public Methods:

`to_infix_notation`([sort_key, ...])	Return an infix str encoding equal for differently sorted operations.
`to_str`(**infix_notation_kwargs)	Alias for `to_infix_notation()`.
`to_pretty_str`(**infix_notation_kwargs)	Same as `to_str()` but using pretty operation names suitable for filenames etc.
`to_repr`([settings, defaults, sort_key, ...])	Return str representation which can be used to reproduce and compare the instance.
`treerecurse_replace_keys`(**replace_map)	Return a new formula with all occurences of variables in `replace_map` replaced and else identical settings.
`treerecurse`(fun)	Apply the given function recursively to this and all children instances.
`apply_to`(annotations[, keep_keys])	Apply this operation to the `annotations` dict.
`variadic_apply_to`(annotations)	Return the result of operation on the values/items of a mapping or sequence of arbitrary length.
`operation`(annotation_vals)	Actual merge operation on values of the input keys in annotations.

Inherited from : py: class:DictTransform

apply_to(annotations[, keep_keys])

Apply this operation to the annotations dict.

Inherited from : py: class:Transform

apply_to(annotations[, keep_keys])

Apply this operation to the annotations dict.

Special Methods:

`__init__`(*in_keys[, out_key, overwrite, ...])	Init.
`__str__`()	Return str(self).
`__repr__`()	Call `to_repr()` without sorting.
`__eq__`(other)	Two merge operations are considered equal, if their normalized representations coincide.
`__copy__`()	Return a deep copy of self using settings.
`__call__`(annotations[, keep_keys])	Call method modifying a given dictionary.

Inherited from : py: class:DictTransform

__call__(annotations[, keep_keys])

Call method modifying a given dictionary.

Inherited from : py: class:Transform

`__repr__`()	Call `to_repr()` without sorting.
`__eq__`(other)	Two merge operations are considered equal, if their normalized representations coincide.
`__copy__`()	Return a deep copy of self using settings.
`__add__`(other)	Return a flat composition of `self` with `other`.
`__radd__`(other)	Return a flat composition of `other` and `self`.
`__call__`(annotations[, keep_keys])	Call method modifying a given dictionary.

Parameters

in_keys (Union[str, Merge]) –
out_key (str) –
overwrite (bool) –
skip_none (bool) –
symb (str) –
cache_duplicates (bool) –
keep_keys (Collection[str]) –
_variadic (bool) –

__call__(annotations, keep_keys=None)[source]

Call method modifying a given dictionary.

Parameters

annotations (Union[Mapping[str, Any], Iterable]) –
keep_keys (Optional[Collection[str]]) –

Return type

Union[Mapping[str, Any], Any]

__copy__()[source]

Return a deep copy of self using settings.

Return type: Merge

__eq__(other)[source]

Two merge operations are considered equal, if their normalized representations coincide. (See to_repr()). This means, they recursively have the same children up to commutation.

Parameters: other (Merge) –
Return type: bool

__init__(*in_keys, out_key=None, overwrite=True, skip_none=True, replace_none=None, symb=None, cache_duplicates=True, keep_keys=None, _variadic=False)[source]

Init.

Hand over input keys either as str or as a Merge operation of str.

Parameters

in_keys (Union[str, Merge]) – sequence of either Merge operation instances or strings with placeholders for the input keys
out_key (Optional[str]) – key for the output of this operation; used to init out_key
overwrite (bool) – on call, whether to overwrite the value at out_key in the given dict if the key already exists; raise if key exists and overwrite is true; saved in overwrite.
replace_none – if not None, the value to replace any None values with; see replace_none
symb (Optional[str]) – override the SYMB for this instance
keep_keys (Optional[Collection[str]]) – intermediate output keys to add to call output; see keep_keys
cache_duplicates (bool) – whether outputs of children with identical keys should be cached and reused; see cache_duplicates
_variadic (bool) – the preferred way to specify this argument is variadic_(); see there for details
skip_none (bool) –

__repr__()[source]

Call to_repr() without sorting.

Return type: str

__str__()[source]: Return str(self).

apply_to(annotations, keep_keys=None)[source]

Apply this operation to the annotations dict. In case of a variadic_() instance, also a plain iterable may be given, see variadic_apply_to() which is called in that case. The operation of this instance is defined in operation. First apply all child operations to the dict. Hereby try to overwrite a value of annotations if its key correspond to an out_key of a child operation, but do not create the value of a key twice. Then apply operation on the originally given and generated values now stored in annotations and store the result also in annotations.

Warning

Annotations is inline updated. Especially, the out_key and keep_keys items are added, and children may apply inline operations to values!

Parameters

annotations (Union[MutableMapping[str, Any], Iterable]) – dict to modify by adding values for out_key and keep_keys
keep_keys (Optional[Collection[str]]) – the output keys in all_out_keys for which values shall be added to annotations in addition to keep_keys

Returns

modified annotations dict, extended by the keys from all_out_keys with the recursively generated values; variadic instances return the plain output of operation()

Return type

Union[Mapping[str, Any], Any]

operation(annotation_vals)[source]

Actual merge operation on values of the input keys in annotations. See in_keys. The annotation_vals must not contain None values, and their length must match the ARITY of this operation.

Parameters: annotation_vals (Sequence) –
Return type: Any

to_infix_notation(sort_key=None, use_whitespace=False, use_pretty_op_symb=False, precedence=None, brackets=('(', ')'))[source]

Return an infix str encoding equal for differently sorted operations. To define a custom sorting for children of commutative operations, hand over the sort_key argument for the builtin sorted. If no precedence is given, brackets are set around all child operations.

Parameters

sort_key (Optional[Callable]) – sort child operations by the given sort_key if the parent operation IS_COMMUTATIVE; defaults to alphabetical sorting
use_whitespace (bool) – separate infix operation symbols from their arguments by whitespace
use_pretty_op_symb (bool) – use the pretty_op_symb instead of SYMB for representation of this operation instance
precedence (Optional[Sequence[Merge]]) – apply brackets according to the given precedence; if not given, assume this operation is in normal form (no brackets) must be a list of Merge operation classes or instances in order of increasing precedence; their SYMB attribute is used to access the operation symbol
brackets (Tuple[str, str]) – tuple of the left and right bracket symbols to use if needed

Return type

str

to_pretty_str(**infix_notation_kwargs)[source]

Same as to_str() but using pretty operation names suitable for filenames etc.

Return type: str

to_repr(settings=None, defaults=None, sort_key=None, use_module_names=False, indent=None, indent_level=None, indent_str=' ', indent_first_child=None, _prepend_indent=True)[source]

Return str representation which can be used to reproduce and compare the instance.

Warning

Tautologies in the form of duplicate children are not filtered for now, e.g. >>> from hybrid_learning.fuzzy_logic.tnorm_connectives.boolean import AND >>> AND(“a”) == AND(“a”, “a”) False

Examples:

>>> from hybrid_learning.fuzzy_logic.tnorm_connectives.boolean import AND, OR
>>> obj = OR(AND("b", "c"), "a", symb="CustomAND", overwrite=False,)
>>> print(obj.to_repr())
OR('a', AND('b', 'c'), overwrite=False, symb='CustomAND')
>>> print(obj.to_repr(indent=True))
OR('a',
   AND('b',
       'c'),
   overwrite=False, symb='CustomAND')
>>> print(obj.to_repr(indent_first_child=True))
OR(
   'a',
   AND(
       'b',
       'c'),
   overwrite=False, symb='CustomAND')
>>> print(obj.to_repr(indent_level=1, indent_str='--'))
--OR('a',
----AND('b',
------'c'),
----overwrite=False, symb='CustomAND')

Parameters

settings (Optional[Dict[str, Any]]) – the settings dict to include as key-value pairs; defaults to settings (set e.g. to overwrite this method)
defaults (Optional[Dict[str, Any]]) – updates to setting_defaults; if a default for a key is given and the value equals the default, it is excluded from printing
sort_key (Optional[Callable]) – sort child operations by the given sort_key if the parent operation IS_COMMUTATIVE; defaults to alphabetical sorting
use_module_names (bool) – whether to use both module plus class names or just the class names
indent (Optional[str]) – if not None, print a tree-like view by putting each in_keys item in a new line with indent matching the class name length; takes precedence over indent_level and indent_str arguments
indent_level (Optional[int]) – if not None and indent is None, print a tree-like view by putting each in_keys item in a new line with indent of indent_str; if >0, indent_level*indent_str is prepended to every printed line.
indent_str (str) – the (whitespace) string representing one indentation level
indent_first_child (Optional[bool]) – whether to already indent the first child or not
_prepend_indent (bool) – whether to prepend the given indent to the output string (default: yes)

Return type

str

to_str(**infix_notation_kwargs)[source]

Alias for to_infix_notation().

Return type: str

treerecurse(fun)[source]

Apply the given function recursively to this and all children instances. If fun returns None, the operation is assumed to have been inline. A non-None return replaces the original root respectively in_keys item. Acting root before children and depth first.

Parameters: fun (Callable[[Union[Merge, str]], Optional[Merge]]) –
Return type: Merge

treerecurse_replace_keys(**replace_map)[source]

Return a new formula with all occurences of variables in replace_map replaced and else identical settings. The children of the new formula instance are new instances as well.

Parameters: replace_map (Dict[str, str]) – mapping {old_var_name: new_var_name}
Return type: Merge

classmethod variadic_(**kwargs)[source]

Return an instance with variadic __call__. It’s __call__ will accept maps or iterables of arbitrary length (for ARITY = -1) respectively of length matching ARITY. All values/elements are passed through to the operation(), and the plain output of operation() is returned (see also variadic_apply_to()). Use this e.g. to wrap the operation() into an object in a multiprocessing-safe manner. Example:

>>> from hybrid_learning.fuzzy_logic.tnorm_connectives.boolean import AND
>>> primitive_and = AND.variadic_()
>>> primitive_and({"a": 1, "b": True})
1
>>> primitive_and([1, True, 1.])
1

No in_keys may be given, and out_key is obsolete. The returned instance may not be used as child element of a formula.

variadic_apply_to(annotations)[source]

Return the result of operation on the values/items of a mapping or sequence of arbitrary length. Performs None check/replacement and ARITY check. In case of a ARITY of -1 and empty annotations list, or an annotations list length not matching the arity, an IndexError is raised.

Parameters: annotations (Union[Mapping[str, Any], Iterable]) –
Return type: Any

classmethod with_(**additional_args)[source]

Return a MergeBuilder with the same symbol but additional init args. Example usage (with changed symbol):

>>> from hybrid_learning.fuzzy_logic.tnorm_connectives.boolean import AND
>>> builder = AND.with_(skip_none=False, replace_none=0).symb_('&n&')
>>> builder.SYMB
'&n&'
>>> builder("a", "b")
AND('a', 'b', replace_none=0, skip_none=False, symb='&n&')

Return type: MergeBuilder

ARITY: int = -1: The arity of the operation. -1 means unlimited number of arguments possible.

IS_COMMUTATIVE: bool = False: Whether instances are equivalent to ones with permuted in_keys.

SYMB: str = None: The string symbol of this class (override for sub-classes).

__hash__ = None

property all_children: List[Merge]: All children operations in the flattened computational tree, sorted depth first. See children for getting only the direct children.

property all_in_keys: Set[str]: All string input keys both of self and of all child operations. (See in_keys.) These are the keys that must be present in an annotation when called on it. Should preserve the order in which keys and children occur in in_keys.

property all_out_keys: Set[str]: Output keys of self and all child operations. (See children). Should preserve the order in which children occur in in_keys.

cache_duplicates: bool: Whether to cache duplicate child operation outputs with duplicate out_key. If set to false, all children and children children are evaluated and the values of duplicate out_keys are evaluated several times and overwritten, possibly leading to more computational time while using less memory. Note that the order of children execution is determined by their order in in_keys, depth first for nested operations.

property children: List[Merge]: The input keys which are child operations. Input keys are stored in in_keys

property consts: Set[str]: The constant string keys in the input keys. The in_keys contains both the constant keys which are to be directly found in a given annotations dictionary, and child operations whose output is used. For getting the child operations stored in in_keys refer to children. Should preserve the order in which children occur in in_keys.

in_keys: Sequence[Union[str, Merge]]: The keys of segmentation masks to unite in given order. Keys are either constant strings or a merge operation.

property is_variadic: bool: Whether the instance is variadic. See variadic_().

keep_keys: Optional[Collection[str]]: The keys of intermediate outputs in all_out_keys which should be added to the return of a call. Default (None or empty collection): duplicate children outputs are cached but not returned to save memory.

property operation_keys: List[str]

The list of keys used for this parent operation in original order (constants and children output keys). These are all consts and the out_key of all children operations. Keys may be duplicate as e.g. in

>>> from hybrid_learning.fuzzy_logic.tnorm_connectives.boolean import OR, NOT
>>> OR("a", NOT("b"), "a", NOT("c", out_key="not_c")).operation_keys
['a', '~b', 'a', 'not_c']

out_key: str: The key to use to store the merge output in the annotations dict. Take care to not accidentally overwrite existing keys (cf. overwrite).

overwrite: Union[bool, Literal['noop']]: Whether to overwrite a value in the input dictionary when applying this operation. The operation is defined in operation(). The key that may be overwritten is stored in out_key. An exception is raised if this is False and the key exists. If set to 'noop' and out_key is in the given annotations dict, it is returned unchanged.

property pretty_op_symb: str: Name of the operation symbol suitable for filenames etc.

replace_none: Optional[Any]: If not None, any received None value is replaced by the given value. This is done only for computation, the None value in the received dict is left unchanged. Key-value pairs with None value may come from the input or from child operations.

property setting_defaults: Defaults used for settings.

property settings: Dict[str, Any]: Settings to reproduce the instance. (Mind that in_keys must be expanded! For direct reproduction use copy.)

skip_none: bool: If set to True, when a None input value is encountered simply None is returned. If False, an error is raised.