ConceptDataset

class hybrid_learning.datasets.custom.coco.mask_dataset.ConceptDataset(body_parts, force_rebuild=False, pt_radius=None, masks_root=None, masks_root_root=None, lazy_mask_generation=True, img_size=None, mask_size=None, device=None, **kwargs)[source]

Bases: COCODataset

Data handle for ground truth segmentations of visual concepts (body parts) generated from COCO keypoints. The concepts that can be handled are combinations of objects (body parts) marked by keypoints.

The input images the dataset contains are the original COCO images. The annotations are segmentation masks masking the desired concept which are generated from the keypoint annotations. The translation of keypoint annotations to masks can be found in annotations_to_mask().

The ground truth masks are generated and stored in a sub-directory masks_root:

<root root>
|
+---annotations
|   +---<annotations file>.json
|
+---images
|   +---dataset_root
|
+---masks
    +---masks_root
        (default: <dataset_root>_<concept desc>)

Public Data Attributes:

`MASKS_ROOT_ROOT`	Usual parent to all masks folders for all body parts; sibling to `images` folder
`settings`	Settings to reproduce the instance.

Inherited from : py: class:COCODataset

`COMMERCIAL_LICENSE_IDS`	IDs of COCO image licenses that allow for commercial use.
`DATASET_ROOT_TEMPL`	Default root directory template for image files that accepts the `split` (`'train'` or `'val'`).
`ANNOTATION_FP_TEMPL`	Default template for the annotation file path that accepts the `split` (`'train'` or `'val'`) and the `root` directory.
`DEFAULT_IMG_SIZE`	Default target size of images to use for the default transforms as `(height, width)`.
`settings`	Settings to reproduce the instance.
`license_mapping`	The mapping of image IDs to license descriptions and URLs.

Inherited from : py: class:BaseDataset

settings

Settings to reproduce the instance.

Public Methods:

`getitem`(i)	Get image and mask by index `i` in dataset.
`load_orig_mask`(i)	Load and return unmodified mask as image by index in dataset.
`mask_exists`(i)	Check whether a mask for the specified index already exists.
`mask_filepath`(i)	Provide the path under which the mask for the given image ID should lie.
`ANNOTATION_TRANSFORM`(*, annotations, ...[, ...])	Create a mask of the linked keypoints from the `annotations` list.
`annotations_to_mask`(orig_img_wh, annotations)	Create mask of all configured keypoints from `annotations`.
`generate_masks`([force_rebuild, ...])	Generate and save the masks for the images in this dataset.
`save_mask`(i, mask)	Save the given mask according to the given index in the dataset.

Inherited from : py: class:COCODataset

`subset`(*[, license_ids, body_parts, num, ...])	Restrict the items by the given selection criteria and an optional custom condition.
`shuffle`()	Wrapper around `subset()` that only shuffles the instance.
`to_raw_anns`([description, save_as])	Create the content of a new valid annotations file restricted to the current image IDs.
`copy_to`([root_root, description, overwrite, ...])	Create a new dataset by copying used images and annotations to new root folder.
`load_orig_image`(i)	Load unmodified image by index in dataset.
`descriptor`(i)	Return the image file name for the item at index `i`.
`image_filepath`(i)	Path to image file at index `i`.
`image_attribution`(i)	Get attribution information for image at index `i`.
`image_meta`(i)	Load the dict with meta information for image at index `i`.
`raw_anns`(i)	Return the list of raw annotations for image at index `i`.
`getitem`(i)	Get image and mask by index `i` in dataset.

Inherited from : py: class:BaseDataset

`getitem`(i)	Get image and mask by index `i` in dataset.
`descriptor`(i)	Return the image file name for the item at index `i`.

Special Methods:

__init__(body_parts[, force_rebuild, ...])

Init.

Inherited from : py: class:COCODataset

`__init__`(body_parts[, force_rebuild, ...])	Init.
`__len__`()	Length is given by the length of the index mapping.

Inherited from : py: class:BaseDataset

`__init__`(body_parts[, force_rebuild, ...])	Init.
`__len__`()	Length is given by the length of the index mapping.
`__getitem__`(idx)	Get item from `idx` in dataset with transformations applied.
`__repr__`()	Nice printing function.

Inherited from : py: class:Dataset

`__getitem__`(idx)	Get item from `idx` in dataset with transformations applied.
`__add__`(other)

ANNOTATION_TRANSFORM(*, annotations, keypoint_idxs, skeleton, pt_radius=0.025, link_width=None)

Callable used to transform a list of annotations into a mask. All keypoints belonging to any of the body_parts are marked, and keypoints belonging to a common body part are linked according to the person skeleton. For details see annotations_to_mask().

Parameters

annotation_wh (Tuple[int, int]) –
annotations (List[dict]) –
keypoint_idxs (Union[Sequence[int], Iterable[Iterable[int]]]) –
skeleton (Sequence[Tuple[int, int]]) –
pt_radius (float) –
link_width (Optional[float]) –

Return type

Image

__init__(body_parts, force_rebuild=False, pt_radius=None, masks_root=None, masks_root_root=None, lazy_mask_generation=True, img_size=None, mask_size=None, device=None, **kwargs)[source]

Init.

Parameters

body_parts (Sequence[Union[BodyParts, Sequence[str]]]) – see body_parts
force_rebuild (bool) – Whether to regenerate all masks during init, or to skip existing ones.
pt_radius (Optional[float]) – see pt_radius
masks_root (Optional[str]) – set the root directory under which to generate the masks; note that one should prefer setting masks_root_root instead
masks_root_root (Optional[str]) – the path to the root under which the masks folder is created; the masks folder’s name is uniquely determined by the mask generation settings; defaults to coco_root/MASKS_ROOT_ROOT where MASKS_ROOT_ROOT is the class default
lazy_mask_generation (bool) – whether to generate the masks lazily or directly during init; overwritten by force_rebuild: if this is true, masks are directly regenerated.
img_size (Optional[Tuple[int, int]]) – see super class
mask_size (Optional[Tuple[int, int]]) – if given, the size of the generated masks (annotations are scaled and padded to this size before mask generation)
kwargs – Any other arguments for the super class
device (Optional[Union[str, device]]) –

annotations_to_mask(orig_img_wh, annotations)[source]

Create mask of all configured keypoints from annotations. It is assumed that the original image considered in annotations has orig_img_wh. If mask_size is None, the mask size will be orig_img_wh. For details see ANNOTATION_TRANSFORM.

Parameters

orig_img_wh (Tuple[int, int]) –
annotations (List[dict]) –

Return type

Image

classmethod default_masks_root(*, dataset_root=None, body_parts=None, pt_radius=None, person_rel_size_range=None, img_size=None, mask_size=None, masks_root_root=None, split=None)[source]

Return default mask root directory for given settings. By default, a directory structure as follows is assumed:

coco_root
 |
 +-images_root_root
 |  +-os.path.basename(dataset_root)
 |
 +-masks_root_root
   +-os.path.basename(dataset_root)+spec_specifier

If optional specifiers are not given, they are omitted. For the details on the parameters, see settings_to_str().

Parameters

dataset_root (Optional[str]) – the root directory of the coco images; basename is used to prefix the mask folder; the folder is used to determine the default for masks_root_root
masks_root_root (Optional[str]) – the root folder under which to put the masks; defaults to dataset_root/../../MASKS_ROOT_ROOT;
body_parts (Optional[Iterable[Iterable[str]]]) –
pt_radius (Optional[float]) –
person_rel_size_range (Optional[Tuple[Optional[float], Optional[float]]]) –
img_size (Optional[Tuple[int, int]]) –
mask_size (Optional[Tuple[int, int]]) –
split (Optional[str]) –

Return type

str

generate_masks(force_rebuild=False, show_progress_bar=True, **kwargs)[source]

Generate and save the masks for the images in this dataset.

Parameters

force_rebuild (bool) – whether to overwrite masks that already exist.
show_progress_bar (bool) – whether to show the progress using tqdm.tqdm
kwargs – further arguments to the progress bar

Return type

None

classmethod get_default_transforms(img_size=None, mask_size=None, device=None)[source]

Default transformation that pads and resizes both images and masks, and binarizes the masks. If mask_size is None, the masks are scaled and padded to the original image, otherwise to mask_size.

Parameters

img_size (Optional[Tuple[int, int]]) –
mask_size (Optional[Tuple[int, int]]) –
device (Optional[Union[str, device]]) –

Return type

Callable[[Image, Image], Tuple[Tensor, Tensor]]

getitem(i)[source]

Get image and mask by index i in dataset. Used for __getitem__().

Parameters: i (int) –
Return type: Tuple[Image, Image]

load_orig_mask(i)[source]

Load and return unmodified mask as image by index in dataset. Image mode of returned masks is ‘L’. Note that the mask has mask_size if this is set, not the original image’s size (which is only changed to img_size during transformation).

Parameters: i (int) –
Return type: Image

mask_exists(i)[source]

Check whether a mask for the specified index already exists.

Parameters: i (int) –
Return type: bool

mask_filepath(i)[source]

Provide the path under which the mask for the given image ID should lie.

Parameters: i (int) –
Return type: str

save_mask(i, mask)[source]

Save the given mask according to the given index in the dataset.

Parameters

i (int) – ID of the image the mask belongs to.
mask (Image) – mask image

Return type

None

classmethod settings_to_str(body_parts=None, pt_radius=None, person_rel_size_range=None, img_size=None, mask_size=None, dataset_root=None, split=None)[source]

Return a string representation of the given mask generation settings. Intended to serve as unique folder name. If optional specifiers are not given, they are omitted.

Parameters

body_parts (Optional[Iterable[Iterable[str]]]) – see body_parts
pt_radius (Optional[float]) – see pt_radius
person_rel_size_range (Optional[Tuple[Optional[float], Optional[float]]]) – the minimum and maximum height of persons included, relative to the image height; provide as tuple (min, max) and set None to unset values
mask_size (Optional[Tuple[int, int]]) – the size of the created masks as (height, width); defaults to img_size
img_size (Optional[Tuple[int, int]]) – the image size for which the mask is created; should be given if one cannot assume the mask fits the original image size, or if e.g. person_rel_size_range is dependent on the image size; should be given as (height, width)
dataset_root (Optional[str]) – if given, used instead of split
split (Optional[Union[str, DatasetSplit]]) – the dataset split

Returns

string that is unique for the given settings and can be used as folder name

Return type

str

MASKS_ROOT_ROOT = 'masks': Usual parent to all masks folders for all body parts; sibling to images folder

__parameters__ = ()

body_parts: Sequence[Union[BodyParts, Sequence[str]]]: List of concepts to mask in selected images. See annotations_to_mask() for details.

mask_size: Optional[Tuple[int, int]]: If given, annotations are scaled and padded to this size before generating the masks (of this size). If left None, the generated masks have the same size as the original image.

masks_root: Folder name of the directory in which all generated masks are stored.

pt_radius: float: Radius of white circles to draw centered at the keypoint coordinates in the masks. Also used as half of the width of link lines between keypoints in the masks. Unit: proportion of height of annotated person in the image (defaults to the image height if person height cannot be estimated).

property settings: Dict[str, Any]: Settings to reproduce the instance.