ca_exp_eval

Description

Helper functions for evaluation of standard concept embedding analysis experiments. Experiment results are assumed to have a folder structure of <root>/layers/<layer_id>/<timestamp>/<concept_name>. For details see the respective sample Sacred experiment scripts.

Attributes

ALEXNET_LAYERS	Layer IDs of pytorch AlexNet model in correct order.
VGG16_LAYERS	Layer IDs of pytorch VGG16 model in correct order.
MASK_RCNN_LAYERS	Layer IDs of pytorch Mask R-CNN model in correct order.

Functions

analysis_root(layer_id, concept_name, root)	Get the analysis root folder path for given root, layer_id, and concept_name.
display_overview(best_set_iou_display)	Provide a nice display style of best_set_iou_display.
gather_stats([roots, root_templ, ...])	Gather dicts with all_stats, best_emb_stats, and merged_stats for different roots.
get_all_best_emb_stats(root, layers[, concepts])	Gather a DataFrame with the stats of all best embeddings.
get_all_stats(root, layers[, concepts])	Gather a DataFrame with the stats of all embedding runs.
get_best_emb(layer_id, root)	Best embedding for given layer.
get_best_emb_stats(root)	Stats of the best embeddings of an analysis as DataFrame.
get_common_concepts(root[, layers])	Get all the concept names available for ALL layers in the given experiment root.
get_concepts(root[, layers])	Get all the concept names available for the given experiment root.
get_embs(root)	Embeddings of an analysis as series indexed by layer_id.
get_layers(root[, model_layers, verbose])	Return all available layers for the given analysis root.
get_stats(root)	Stats of an analysis run as DataFrame.
get_vis_best_embedding(root)	Get the saved visualization of the best embedding.
highlight_max_blue(series)	Highlight the maximum in a Series with red font color.
merge_to_overview(best_ious, all_stats[, ...])	Merge the stats from all runs and from the best embeddings to an overview over the test values of metric.
plot_best_iou_comparison(metric_mean_map, ...)	Provide a plot of metric values against layers for one concept and all settings in best_emb_ious.
plot_best_ious_wt_std(best_emb_ious, iou_stds)	Provide a plot of metric values against layers for all concepts in best_emb_ious.
plot_overview(best_set_iou_display[, ...])	Plot and save the output of merge_to_overview using plot_best_ious_wt_std.
plot_overview_for_concept(merged_stats_map, ...)	Plot and save the output of merge_to_overview using plot_best_ious_wt_std.