mmdet.apis¶

mmdet.datasets¶

datasets¶

api_wrappers¶

class mmdet.datasets.api_wrappers.COCO(*args: Any, **kwargs: Any)[source]¶

This class is almost the same as official pycocotools package.

It implements some snake case function aliases. So that the COCO class has the same interface as LVIS class.

class mmdet.datasets.api_wrappers.COCOPanoptic(*args: Any, **kwargs: Any)[source]¶

This wrapper is for loading the panoptic style annotation file.

The format is shown in the CocoPanopticDataset class.

Parameters: annotation_file (str, optional) – Path of annotation file. Defaults to None.

createIndex() → None[source]¶: Create index.

load_anns(ids: Union[List[int], int] = []) → Optional[List[dict]][source]¶

Load anns with the specified ids.

self.anns is a list of annotation lists instead of a list of annotations.

Parameters: ids (Union[List[int], int]) – Integer ids specifying anns.
Returns: Loaded ann objects.
Return type: anns (List[dict], optional)

samplers¶

class mmdet.datasets.samplers.AspectRatioBatchSampler(sampler: torch.utils.data.sampler.Sampler, batch_size: int, drop_last: bool = False)[source]¶

A sampler wrapper for grouping images with similar aspect ratio (< 1 or.

>= 1) into a same batch.

Parameters

sampler (Sampler) – Base sampler.
batch_size (int) – Size of mini-batch.
drop_last (bool) – If True, the sampler will drop the last batch if its size would be less than batch_size.

class mmdet.datasets.samplers.ClassAwareSampler(dataset: mmengine.dataset.base_dataset.BaseDataset, seed: Optional[int] = None, num_sample_class: int = 1)[source]¶

Sampler that restricts data loading to the label of the dataset.

A class-aware sampling strategy to effectively tackle the non-uniform class distribution. The length of the training data is consistent with source data. Simple improvements based on Relay Backpropagation for Effective Learning of Deep Convolutional Neural Networks

The implementation logic is referred to https://github.com/Sense-X/TSD/blob/master/mmdet/datasets/samplers/distributed_classaware_sampler.py

Parameters

dataset – Dataset used for sampling.
seed (int, optional) – random seed used to shuffle the sampler. This number should be identical across all processes in the distributed group. Defaults to None.
num_sample_class (int) – The number of samples taken from each per-label list. Defaults to 1.

get_cat2imgs() → Dict[int, list][source]¶

Get a dict with class as key and img_ids as values.

Returns: A dict of per-label image list, the item of the dict indicates a label index, corresponds to the image index that contains the label.
Return type: dict[int, list]

set_epoch(epoch: int) → None[source]¶

Sets the epoch for this sampler.

When shuffle=True, this ensures all replicas use a different random ordering for each epoch. Otherwise, the next iteration of this sampler will yield the same ordering.

Parameters: epoch (int) – Epoch number.

class mmdet.datasets.samplers.GroupMultiSourceSampler(dataset: mmengine.dataset.base_dataset.BaseDataset, batch_size: int, source_ratio: List[Union[int, float]], shuffle: bool = True, seed: Optional[int] = None)[source]¶

Group Multi-Source Infinite Sampler.

According to the sampling ratio, sample data from different datasets but the same group to form batches.

Parameters

dataset (Sized) – The dataset.
batch_size (int) – Size of mini-batch.
source_ratio (list[int | float]) – The sampling ratio of different source datasets in a mini-batch.
shuffle (bool) – Whether shuffle the dataset or not. Defaults to True.
seed (int, optional) – Random seed. If None, set a random seed. Defaults to None.

class mmdet.datasets.samplers.MultiSourceSampler(dataset: Sized, batch_size: int, source_ratio: List[Union[int, float]], shuffle: bool = True, seed: Optional[int] = None)[source]¶

Multi-Source Infinite Sampler.

According to the sampling ratio, sample data from different datasets to form batches.

Parameters

dataset (Sized) – The dataset.
batch_size (int) – Size of mini-batch.
source_ratio (list[int | float]) – The sampling ratio of different source datasets in a mini-batch.
shuffle (bool) – Whether shuffle the dataset or not. Defaults to True.
seed (int, optional) – Random seed. If None, set a random seed. Defaults to None.

Examples

>>> dataset_type = 'ConcatDataset'
>>> sub_dataset_type = 'CocoDataset'
>>> data_root = 'data/coco/'
>>> sup_ann = '../coco_semi_annos/instances_train2017.1@10.json'
>>> unsup_ann = '../coco_semi_annos/' \
>>>             'instances_train2017.1@10-unlabeled.json'
>>> dataset = dict(type=dataset_type,
>>>     datasets=[
>>>         dict(
>>>             type=sub_dataset_type,
>>>             data_root=data_root,
>>>             ann_file=sup_ann,
>>>             data_prefix=dict(img='train2017/'),
>>>             filter_cfg=dict(filter_empty_gt=True, min_size=32),
>>>             pipeline=sup_pipeline),
>>>         dict(
>>>             type=sub_dataset_type,
>>>             data_root=data_root,
>>>             ann_file=unsup_ann,
>>>             data_prefix=dict(img='train2017/'),
>>>             filter_cfg=dict(filter_empty_gt=True, min_size=32),
>>>             pipeline=unsup_pipeline),
>>>         ])
>>>     train_dataloader = dict(
>>>         batch_size=5,
>>>         num_workers=5,
>>>         persistent_workers=True,
>>>         sampler=dict(type='MultiSourceSampler',
>>>             batch_size=5, source_ratio=[1, 4]),
>>>         batch_sampler=None,
>>>         dataset=dataset)

set_epoch(epoch: int) → None[source]¶: Not supported in `epoch-based runner.

class mmdet.datasets.samplers.TrackAspectRatioBatchSampler(sampler: torch.utils.data.sampler.Sampler, batch_size: int, drop_last: bool = False)[source]¶

A sampler wrapper for grouping images with similar aspect ratio (< 1 or.

>= 1) into a same batch.

Parameters

sampler (Sampler) – Base sampler.
batch_size (int) – Size of mini-batch.
drop_last (bool) – If True, the sampler will drop the last batch if its size would be less than batch_size.

class mmdet.datasets.samplers.TrackImgSampler(dataset: Sized, seed: Optional[int] = None)[source]¶

Sampler that providing image-level sampling outputs for video datasets in tracking tasks. It could be both used in both distributed and non-distributed environment. If using the default sampler in pytorch, the subsequent data receiver will get one video, which is not desired in some cases: (Take a non-distributed environment as an example) 1. In test mode, we want only one image is fed into the data pipeline. This is in consideration of memory usage since feeding the whole video commonly requires a large amount of memory (>=20G on MOTChallenge17 dataset), which is not available in some machines. 2. In training mode, we may want to make sure all the images in one video are randomly sampled once in one epoch and this can not be guaranteed in the default sampler in pytorch.

Parameters

dataset (Sized) – Dataset used for sampling.
seed (int, optional) – random seed used to shuffle the sampler. This number should be identical across all processes in the distributed group. Defaults to None.

transforms¶

mmdet.engine¶

hooks¶

class mmdet.engine.hooks.CheckInvalidLossHook(interval: int = 50)[source]¶

Check invalid loss hook.

This hook will regularly check whether the loss is valid during training.

Parameters: interval (int) – Checking interval (every k iterations). Default: 50.

after_train_iter(runner: mmengine.runner.runner.Runner, batch_idx: int, data_batch: Optional[dict] = None, outputs: Optional[dict] = None) → None[source]¶

Regularly check whether the loss is valid every n iterations.

Parameters

runner (Runner) – The runner of the training process.
batch_idx (int) – The index of the current batch in the train loop.
data_batch (dict, Optional) – Data from dataloader. Defaults to None.
outputs (dict, Optional) – Outputs from model. Defaults to None.

class mmdet.engine.hooks.DetVisualizationHook(draw: bool = False, interval: int = 50, score_thr: float = 0.3, show: bool = False, wait_time: float = 0.0, test_out_dir: Optional[str] = None, backend_args: Optional[dict] = None)[source]¶

Detection Visualization Hook. Used to visualize validation and testing process prediction results.

In the testing phase:

If show is True, it means that only the prediction results are
visualized without storing data, so vis_backends needs to be excluded.
If test_out_dir is specified, it means that the prediction results
need to be saved to test_out_dir. In order to avoid vis_backends also storing data, so vis_backends needs to be excluded.
vis_backends takes effect if the user does not specify show
and test_out_dir`. You can set vis_backends to WandbVisBackend or TensorboardVisBackend to store the prediction result in Wandb or Tensorboard.

Parameters

draw (bool) – whether to draw prediction results. If it is False, it means that no drawing will be done. Defaults to False.
interval (int) – The interval of visualization. Defaults to 50.
score_thr (float) – The threshold to visualize the bboxes and masks. Defaults to 0.3.
show (bool) – Whether to display the drawn image. Default to False.
wait_time (float) – The interval of show (s). Defaults to 0.
test_out_dir (str, optional) – directory where painted images will be saved in testing process.
backend_args (dict, optional) – Arguments to instantiate the corresponding backend. Defaults to None.

after_test_iter(runner: mmengine.runner.runner.Runner, batch_idx: int, data_batch: dict, outputs: Sequence[mmdet.structures.det_data_sample.DetDataSample]) → None[source]¶

Run after every testing iterations.

Parameters

runner (Runner) – The runner of the testing process.
batch_idx (int) – The index of the current batch in the val loop.
data_batch (dict) – Data from dataloader.
outputs (Sequence[DetDataSample]) – A batch of data samples that contain annotations and predictions.

after_val_iter(runner: mmengine.runner.runner.Runner, batch_idx: int, data_batch: dict, outputs: Sequence[mmdet.structures.det_data_sample.DetDataSample]) → None[source]¶

Run after every self.interval validation iterations.

Parameters

runner (Runner) – The runner of the validation process.
batch_idx (int) – The index of the current batch in the val loop.
data_batch (dict) – Data from dataloader.
outputs (Sequence[DetDataSample]]) – A batch of data samples that contain annotations and predictions.

class mmdet.engine.hooks.MeanTeacherHook(momentum: float = 0.001, interval: int = 1, skip_buffer=True)[source]¶

Mean Teacher Hook.

Mean Teacher is an efficient semi-supervised learning method in Mean Teacher. This method requires two models with exactly the same structure, as the student model and the teacher model, respectively. The student model updates the parameters through gradient descent, and the teacher model updates the parameters through exponential moving average of the student model. Compared with the student model, the teacher model is smoother and accumulates more knowledge.

Parameters

momentum (float) –

The momentum used for updating teacher’s parameter.
Teacher’s parameter are updated with the formula:

teacher = (1-momentum) * teacher + momentum * student.
Defaults to 0.001.
interval (int) – Update teacher’s parameter every interval iteration. Defaults to 1.
skip_buffers (bool) – Whether to skip the model buffers, such as batchnorm running stats (running_mean, running_var), it does not perform the ema operation. Default to True.

after_train_iter(runner: mmengine.runner.runner.Runner, batch_idx: int, data_batch: Optional[dict] = None, outputs: Optional[dict] = None) → None[source]¶: Update teacher’s parameter every self.interval iterations.

before_train(runner: mmengine.runner.runner.Runner) → None[source]¶: To check that teacher model and student model exist.

momentum_update(model: torch.nn.modules.module.Module, momentum: float) → None[source]¶: Compute the moving average of the parameters using exponential moving average.

class mmdet.engine.hooks.MemoryProfilerHook(interval: int = 50)[source]¶

Memory profiler hook recording memory information including virtual memory, swap memory, and the memory of the current process.

Parameters: interval (int) – Checking interval (every k iterations). Default: 50.

after_test_iter(runner: mmengine.runner.runner.Runner, batch_idx: int, data_batch: Optional[dict] = None, outputs: Optional[Sequence[mmdet.structures.det_data_sample.DetDataSample]] = None) → None[source]¶

Regularly record memory information.

Parameters

runner (Runner) – The runner of the testing process.
batch_idx (int) – The index of the current batch in the test loop.
data_batch (dict, optional) – Data from dataloader. Defaults to None.
outputs (Sequence[DetDataSample], optional) – Outputs from model. Defaults to None.

after_train_iter(runner: mmengine.runner.runner.Runner, batch_idx: int, data_batch: Optional[dict] = None, outputs: Optional[dict] = None) → None[source]¶

Regularly record memory information.

Parameters

runner (Runner) – The runner of the training process.
batch_idx (int) – The index of the current batch in the train loop.
data_batch (dict, optional) – Data from dataloader. Defaults to None.
outputs (dict, optional) – Outputs from model. Defaults to None.

after_val_iter(runner: mmengine.runner.runner.Runner, batch_idx: int, data_batch: Optional[dict] = None, outputs: Optional[Sequence[mmdet.structures.det_data_sample.DetDataSample]] = None) → None[source]¶

Regularly record memory information.

Parameters

runner (Runner) – The runner of the validation process.
batch_idx (int) – The index of the current batch in the val loop.
data_batch (dict, optional) – Data from dataloader. Defaults to None.
outputs (Sequence[DetDataSample], optional) – Outputs from model. Defaults to None.

class mmdet.engine.hooks.NumClassCheckHook[source]¶

Check whether the num_classes in head matches the length of classes in dataset.metainfo.

before_train_epoch(runner: mmengine.runner.runner.Runner) → None[source]¶

Check whether the training dataset is compatible with head.

Parameters: runner (Runner) – The runner of the training or evaluation process.

before_val_epoch(runner: mmengine.runner.runner.Runner) → None[source]¶

Check whether the dataset in val epoch is compatible with head.

Parameters: runner (Runner) – The runner of the training or evaluation process.

class mmdet.engine.hooks.PipelineSwitchHook(switch_epoch, switch_pipeline)[source]¶

Switch data pipeline at switch_epoch.

Parameters

switch_epoch (int) – switch pipeline at this epoch.
switch_pipeline (list[dict]) – the pipeline to switch to.

before_train_epoch(runner)[source]¶: switch pipeline.

class mmdet.engine.hooks.SetEpochInfoHook[source]¶

Set runner’s epoch information to the model.

before_train_epoch(runner)[source]¶

All subclasses should override this method, if they need any operations before each training epoch.

Parameters: runner (Runner) – The runner of the training process.

class mmdet.engine.hooks.SyncNormHook[source]¶

Synchronize Norm states before validation, currently used in YOLOX.

before_val_epoch(runner)[source]¶: Synchronizing norm.

class mmdet.engine.hooks.TrackVisualizationHook(draw: bool = False, frame_interval: int = 30, score_thr: float = 0.3, show: bool = False, wait_time: float = 0.0, test_out_dir: Optional[str] = None, backend_args: Optional[dict] = None)[source]¶

Tracking Visualization Hook. Used to visualize validation and testing process prediction results.

In the testing phase:

If show is True, it means that only the prediction results are
visualized without storing data, so vis_backends needs to be excluded.
If test_out_dir is specified, it means that the prediction results
need to be saved to test_out_dir. In order to avoid vis_backends also storing data, so vis_backends needs to be excluded.
vis_backends takes effect if the user does not specify show
and test_out_dir`. You can set vis_backends to WandbVisBackend or TensorboardVisBackend to store the prediction result in Wandb or Tensorboard.

Parameters

draw (bool) – whether to draw prediction results. If it is False, it means that no drawing will be done. Defaults to False.
frame_interval (int) – The interval of visualization. Defaults to 30.
score_thr (float) – The threshold to visualize the bboxes and masks. Defaults to 0.3.
show (bool) – Whether to display the drawn image. Default to False.
wait_time (float) – The interval of show (s). Defaults to 0.
test_out_dir (str, optional) – directory where painted images will be saved in testing process.
backend_args (dict) – Arguments to instantiate a file client. Defaults to None.

after_test_iter(runner: mmengine.runner.runner.Runner, batch_idx: int, data_batch: dict, outputs: Sequence[mmdet.structures.track_data_sample.TrackDataSample]) → None[source]¶

Run after every testing iteration.

Parameters

runner (Runner) – The runner of the testing process.
batch_idx (int) – The index of the current batch in the test loop.
data_batch (dict) – Data from dataloader.
outputs (Sequence[TrackDataSample]) – Outputs from model.

after_val_iter(runner: mmengine.runner.runner.Runner, batch_idx: int, data_batch: dict, outputs: Sequence[mmdet.structures.track_data_sample.TrackDataSample]) → None[source]¶

Run after every self.interval validation iteration.

Parameters

runner (Runner) – The runner of the validation process.
batch_idx (int) – The index of the current batch in the val loop.
data_batch (dict) – Data from dataloader.
outputs (Sequence[TrackDataSample]) – Outputs from model.

visualize_single_image(img_data_sample: mmdet.structures.det_data_sample.DetDataSample, step: int) → None[source]¶

Parameters

img_data_sample (DetDataSample) – single image output.
step (int) – The index of the current image.

class mmdet.engine.hooks.YOLOXModeSwitchHook(num_last_epochs: int = 15, skip_type_keys: Sequence[str] = ('Mosaic', 'RandomAffine', 'MixUp'))[source]¶

Switch the mode of YOLOX during training.

This hook turns off the mosaic and mixup data augmentation and switches to use L1 loss in bbox_head.

Parameters: num_last_epochs – The number of latter epochs in the end of the training to close the data augmentation and switch to L1 loss. Defaults to 15.

before_train_epoch(runner) → None[source]¶: Close mosaic and mixup augmentation and switches to use L1 loss.

optimizers¶

class mmdet.engine.optimizers.LearningRateDecayOptimizerConstructor(optim_wrapper_cfg: dict, paramwise_cfg: Optional[dict] = None)[source]¶

add_params(params: List[dict], module: torch.nn.modules.module.Module, **kwargs) → None[source]¶

Add all parameters of module to the params list.

The parameters of the given module will be added to the list of param groups, with specific rules defined by paramwise_cfg.

Parameters

params (list[dict]) – A list of param groups, it will be modified in place.
module (nn.Module) – The module to be added.

runner¶

class mmdet.engine.runner.TeacherStudentValLoop(runner, dataloader: Union[torch.utils.data.dataloader.DataLoader, Dict], evaluator: Union[mmengine.evaluator.evaluator.Evaluator, Dict, List], fp16: bool = False)[source]¶

Loop for validation of model teacher and student.

run()[source]¶: Launch validation for model teacher and student.

schedulers¶

class mmdet.engine.schedulers.QuadraticWarmupLR(optimizer, *args, **kwargs)[source]¶

Warm up the learning rate of each parameter group by quadratic formula.

Parameters

optimizer (Optimizer) – Wrapped optimizer.
begin (int) – Step at which to start updating the parameters. Defaults to 0.
end (int) – Step at which to stop updating the parameters. Defaults to INF.
last_step (int) – The index of last step. Used for resume without state dict. Defaults to -1.
by_epoch (bool) – Whether the scheduled parameters are updated by epochs. Defaults to True.
verbose (bool) – Whether to print the value for each update. Defaults to False.

class mmdet.engine.schedulers.QuadraticWarmupMomentum(optimizer, *args, **kwargs)[source]¶

Warm up the momentum value of each parameter group by quadratic formula.

Parameters

optimizer (Optimizer) – Wrapped optimizer.
begin (int) – Step at which to start updating the parameters. Defaults to 0.
end (int) – Step at which to stop updating the parameters. Defaults to INF.
last_step (int) – The index of last step. Used for resume without state dict. Defaults to -1.
by_epoch (bool) – Whether the scheduled parameters are updated by epochs. Defaults to True.
verbose (bool) – Whether to print the value for each update. Defaults to False.

class mmdet.engine.schedulers.QuadraticWarmupParamScheduler(optimizer: torch.optim.optimizer.Optimizer, param_name: str, begin: int = 0, end: int = 1000000000, last_step: int = - 1, by_epoch: bool = True, verbose: bool = False)[source]¶

Warm up the parameter value of each parameter group by quadratic formula:

\[X_{t} = X_{t-1} + \frac{2t+1}{{(end-begin)}^{2}} \times X_{base}\]

Parameters

optimizer (Optimizer) – Wrapped optimizer.
param_name (str) – Name of the parameter to be adjusted, such as lr, momentum.
begin (int) – Step at which to start updating the parameters. Defaults to 0.
end (int) – Step at which to stop updating the parameters. Defaults to INF.
last_step (int) – The index of last step. Used for resume without state dict. Defaults to -1.
by_epoch (bool) – Whether the scheduled parameters are updated by epochs. Defaults to True.
verbose (bool) – Whether to print the value for each update. Defaults to False.

classmethod build_iter_from_epoch(*args, begin=0, end=1000000000, by_epoch=True, epoch_length=None, **kwargs)[source]¶: Build an iter-based instance of this scheduler from an epoch-based config.

mmdet.evaluation¶

functional¶

mmdet.evaluation.functional.average_precision(recalls, precisions, mode='area')[source]¶

Calculate average precision (for single or multiple scales).

Parameters

recalls (ndarray) – shape (num_scales, num_dets) or (num_dets, )
precisions (ndarray) – shape (num_scales, num_dets) or (num_dets, )
mode (str) – ‘area’ or ‘11points’, ‘area’ means calculating the area under precision-recall curve, ‘11points’ means calculating the average precision of recalls at [0, 0.1, …, 1]

Returns

calculated average precision

Return type

float or ndarray

mmdet.evaluation.functional.bbox_overlaps(bboxes1, bboxes2, mode='iou', eps=1e-06, use_legacy_coordinate=False)[source]¶

Calculate the ious between each bbox of bboxes1 and bboxes2.

Parameters

bboxes1 (ndarray) – Shape (n, 4)
bboxes2 (ndarray) – Shape (k, 4)
mode (str) – IOU (intersection over union) or IOF (intersection over foreground)
use_legacy_coordinate (bool) – Whether to use coordinate system in mmdet v1.x. which means width, height should be calculated as ‘x2 - x1 + 1` and ‘y2 - y1 + 1’ respectively. Note when function is used in VOCDataset, it should be True to align with the official implementation http://host.robots.ox.ac.uk/pascal/VOC/voc2012/VOCdevkit_18-May-2011.tar Default: False.

Returns

Shape (n, k)

Return type

ious (ndarray)

mmdet.evaluation.functional.cityscapes_classes() → list[source]¶: Class names of Cityscapes.

mmdet.evaluation.functional.coco_classes() → list[source]¶: Class names of COCO.

mmdet.evaluation.functional.coco_panoptic_classes() → list[source]¶: Class names of COCO panoptic.

mmdet.evaluation.functional.eval_map(det_results, annotations, scale_ranges=None, iou_thr=0.5, ioa_thr=None, dataset=None, logger=None, tpfp_fn=None, nproc=4, use_legacy_coordinate=False, use_group_of=False, eval_mode='area')[source]¶

Evaluate mAP of a dataset.

Parameters

det_results (list[list]) – [[cls1_det, cls2_det, …], …]. The outer list indicates images, and the inner list indicates per-class detected bboxes.
annotations (list[dict]) –
Ground truth annotations where each item of the list indicates an image. Keys of annotations are:
- bboxes: numpy array of shape (n, 4)
- labels: numpy array of shape (n, )
- bboxes_ignore (optional): numpy array of shape (k, 4)
- labels_ignore (optional): numpy array of shape (k, )
scale_ranges (list[tuple] | None) – Range of scales to be evaluated, in the format [(min1, max1), (min2, max2), …]. A range of (32, 64) means the area range between (32**2, 64**2). Defaults to None.
iou_thr (float) – IoU threshold to be considered as matched. Defaults to 0.5.
ioa_thr (float | None) – IoA threshold to be considered as matched, which only used in OpenImages evaluation. Defaults to None.
dataset (list[str] | str | None) – Dataset name or dataset classes, there are minor differences in metrics for different datasets, e.g. “voc”, “imagenet_det”, etc. Defaults to None.
logger (logging.Logger | str | None) – The way to print the mAP summary. See mmengine.logging.print_log() for details. Defaults to None.
tpfp_fn (callable | None) – The function used to determine true/ false positives. If None, tpfp_default() is used as default unless dataset is ‘det’ or ‘vid’ (tpfp_imagenet() in this case). If it is given as a function, then this function is used to evaluate tp & fp. Default None.
nproc (int) – Processes used for computing TP and FP. Defaults to 4.
use_legacy_coordinate (bool) – Whether to use coordinate system in mmdet v1.x. which means width, height should be calculated as ‘x2 - x1 + 1` and ‘y2 - y1 + 1’ respectively. Defaults to False.
use_group_of (bool) – Whether to use group of when calculate TP and FP, which only used in OpenImages evaluation. Defaults to False.
eval_mode (str) – ‘area’ or ‘11points’, ‘area’ means calculating the area under precision-recall curve, ‘11points’ means calculating the average precision of recalls at [0, 0.1, …, 1], PASCAL VOC2007 uses 11points as default evaluate mode, while others are ‘area’. Defaults to ‘area’.

Returns

(mAP, [dict, dict, …])

Return type

tuple

mmdet.evaluation.functional.eval_recalls(gts, proposals, proposal_nums=None, iou_thrs=0.5, logger=None, use_legacy_coordinate=False)[source]¶

Calculate recalls.

Parameters

gts (list[ndarray]) – a list of arrays of shape (n, 4)
proposals (list[ndarray]) – a list of arrays of shape (k, 4) or (k, 5)
proposal_nums (int | Sequence[int]) – Top N proposals to be evaluated.
iou_thrs (float | Sequence[float]) – IoU thresholds. Default: 0.5.
logger (logging.Logger | str | None) – The way to print the recall summary. See mmengine.logging.print_log() for details. Default: None.
use_legacy_coordinate (bool) – Whether use coordinate system in mmdet v1.x. “1” was added to both height and width which means w, h should be computed as ‘x2 - x1 + 1` and ‘y2 - y1 + 1’. Default: False.

Returns

recalls of different ious and proposal nums

Return type

ndarray

mmdet.evaluation.functional.evaluateImgLists(prediction_list: list, groundtruth_list: list, args: object, backend_args: Optional[dict] = None, dump_matches: bool = False) → dict[source]¶

A wrapper of obj:``cityscapesscripts.evaluation.

evalInstanceLevelSemanticLabeling.evaluateImgLists``. Support loading groundtruth image from file backend. :param prediction_list: A list of prediction txt file. :type prediction_list: list :param groundtruth_list: A list of groundtruth image file. :type groundtruth_list: list :param args: A global object setting in

obj:cityscapesscripts.evaluation. evalInstanceLevelSemanticLabeling

Parameters

backend_args (dict, optional) – Arguments to instantiate the preifx of uri corresponding backend. Defaults to None.
dump_matches (bool) – whether dump matches.json. Defaults to False.

Returns

The computed metric.

Return type

dict

mmdet.evaluation.functional.get_classes(dataset) → list[source]¶: Get class names of a dataset.

mmdet.evaluation.functional.imagenet_det_classes() → list[source]¶: Class names of ImageNet Det.

mmdet.evaluation.functional.imagenet_vid_classes() → list[source]¶: Class names of ImageNet VID.

mmdet.evaluation.functional.objects365v1_classes() → list[source]¶: Class names of Objects365 V1.

mmdet.evaluation.functional.objects365v2_classes() → list[source]¶: Class names of Objects365 V2.

mmdet.evaluation.functional.oid_challenge_classes() → list[source]¶: Class names of Open Images Challenge.

mmdet.evaluation.functional.oid_v6_classes() → list[source]¶: Class names of Open Images V6.

mmdet.evaluation.functional.plot_iou_recall(recalls, iou_thrs)[source]¶

Plot IoU-Recalls curve.

Parameters

recalls (ndarray or list) – shape (k,)
iou_thrs (ndarray or list) – same shape as recalls

mmdet.evaluation.functional.plot_num_recall(recalls, proposal_nums)[source]¶

Plot Proposal_num-Recalls curve.

Parameters

recalls (ndarray or list) – shape (k,)
proposal_nums (ndarray or list) – same shape as recalls

mmdet.evaluation.functional.pq_compute_multi_core(matched_annotations_list, gt_folder, pred_folder, categories, backend_args=None, nproc=32)[source]¶

Evaluate the metrics of Panoptic Segmentation with multithreading.

Same as the function with the same name in panopticapi.

Parameters

matched_annotations_list (list) – The matched annotation list. Each element is a tuple of annotations of the same image with the format (gt_anns, pred_anns).
gt_folder (str) – The path of the ground truth images.
pred_folder (str) – The path of the prediction images.
categories (str) – The categories of the dataset.
backend_args (object) – The file client of the dataset. If None, the backend will be set to local.
nproc (int) – Number of processes for panoptic quality computing. Defaults to 32. When nproc exceeds the number of cpu cores, the number of cpu cores is used.

mmdet.evaluation.functional.pq_compute_single_core(proc_id, annotation_set, gt_folder, pred_folder, categories, backend_args=None, print_log=False)[source]¶

The single core function to evaluate the metric of Panoptic Segmentation.

Same as the function with the same name in panopticapi. Only the function to load the images is changed to use the file client.

Parameters

proc_id (int) – The id of the mini process.
gt_folder (str) – The path of the ground truth images.
pred_folder (str) – The path of the prediction images.
categories (str) – The categories of the dataset.
backend_args (object) – The Backend of the dataset. If None, the backend will be set to local.
print_log (bool) – Whether to print the log. Defaults to False.

mmdet.evaluation.functional.print_map_summary(mean_ap, results, dataset=None, scale_ranges=None, logger=None)[source]¶

Print mAP and results of each class.

A table will be printed to show the gts/dets/recall/AP of each class and the mAP.

Parameters

mean_ap (float) – Calculated from eval_map().
results (list[dict]) – Calculated from eval_map().
dataset (list[str] | str | None) – Dataset name or dataset classes.
scale_ranges (list[tuple] | None) – Range of scales to be evaluated.
logger (logging.Logger | str | None) – The way to print the mAP summary. See mmengine.logging.print_log() for details. Defaults to None.

mmdet.evaluation.functional.print_recall_summary(recalls, proposal_nums, iou_thrs, row_idxs=None, col_idxs=None, logger=None)[source]¶

Print recalls in a table.

Parameters

recalls (ndarray) – calculated from bbox_recalls
proposal_nums (ndarray or list) – top N proposals
iou_thrs (ndarray or list) – iou thresholds
row_idxs (ndarray) – which rows(proposal nums) to print
col_idxs (ndarray) – which cols(iou thresholds) to print
logger (logging.Logger | str | None) – The way to print the recall summary. See mmengine.logging.print_log() for details. Default: None.

mmdet.evaluation.functional.voc_classes() → list[source]¶: Class names of PASCAL VOC.

metrics¶

mmdet.models¶

backbones¶

data_preprocessors¶

dense_heads¶

detectors¶

layers¶

losses¶

necks¶

roi_heads¶

seg_heads¶

task_modules¶

test_time_augs¶

utils¶

mmdet.structures¶

structures¶

class mmdet.structures.DetDataSample(*, metainfo: Optional[dict] = None, **kwargs)[source]¶

A data structure interface of MMDetection. They are used as interfaces between different components.

The attributes in DetDataSample are divided into several parts:

``proposals``(InstanceData): Region proposals used in two-stage
detectors.

``gt_instances``(InstanceData): Ground truth of instance annotations.

``pred_instances``(InstanceData): Instances of detection predictions.

``pred_track_instances``(InstanceData): Instances of tracking
predictions.

``ignored_instances``(InstanceData): Instances to be ignored during
training/testing.

``gt_panoptic_seg``(PixelData): Ground truth of panoptic
segmentation.

``pred_panoptic_seg``(PixelData): Prediction of panoptic
segmentation.

``gt_sem_seg``(PixelData): Ground truth of semantic segmentation.

``pred_sem_seg``(PixelData): Prediction of semantic segmentation.

Examples

>>> import torch
>>> import numpy as np
>>> from mmengine.structures import InstanceData
>>> from mmdet.structures import DetDataSample

>>> data_sample = DetDataSample()
>>> img_meta = dict(img_shape=(800, 1196),
...                 pad_shape=(800, 1216))
>>> gt_instances = InstanceData(metainfo=img_meta)
>>> gt_instances.bboxes = torch.rand((5, 4))
>>> gt_instances.labels = torch.rand((5,))
>>> data_sample.gt_instances = gt_instances
>>> assert 'img_shape' in data_sample.gt_instances.metainfo_keys()
>>> len(data_sample.gt_instances)
5
>>> print(data_sample)

<DetDataSample(

META INFORMATION

DATA FIELDS gt_instances: <InstanceData(

META INFORMATION pad_shape: (800, 1216) img_shape: (800, 1196)

DATA FIELDS labels: tensor([0.8533, 0.1550, 0.5433, 0.7294, 0.5098]) bboxes: tensor([[9.7725e-01, 5.8417e-01, 1.7269e-01, 6.5694e-01],

[1.7894e-01, 5.1780e-01, 7.0590e-01, 4.8589e-01], [7.0392e-01, 6.6770e-01, 1.7520e-01, 1.4267e-01], [2.2411e-01, 5.1962e-01, 9.6953e-01, 6.6994e-01], [4.1338e-01, 2.1165e-01, 2.7239e-04, 6.8477e-01]])

) at 0x7f21fb1b9190>
) at 0x7f21fb1b9880>
>>> pred_instances = InstanceData(metainfo=img_meta)
>>> pred_instances.bboxes = torch.rand((5, 4))
>>> pred_instances.scores = torch.rand((5,))
>>> data_sample = DetDataSample(pred_instances=pred_instances)
>>> assert 'pred_instances' in data_sample
>>> pred_track_instances = InstanceData(metainfo=img_meta)
>>> pred_track_instances.bboxes = torch.rand((5, 4))
>>> pred_track_instances.scores = torch.rand((5,))
>>> data_sample = DetDataSample(
...    pred_track_instances=pred_track_instances)
>>> assert 'pred_track_instances' in data_sample
>>> data_sample = DetDataSample()
>>> gt_instances_data = dict(
...                        bboxes=torch.rand(2, 4),
...                        labels=torch.rand(2),
...                        masks=np.random.rand(2, 2, 2))
>>> gt_instances = InstanceData(**gt_instances_data)
>>> data_sample.gt_instances = gt_instances
>>> assert 'gt_instances' in data_sample
>>> assert 'masks' in data_sample.gt_instances
>>> data_sample = DetDataSample()
>>> gt_panoptic_seg_data = dict(panoptic_seg=torch.rand(2, 4))
>>> gt_panoptic_seg = PixelData(**gt_panoptic_seg_data)
>>> data_sample.gt_panoptic_seg = gt_panoptic_seg
>>> print(data_sample)
<DetDataSample(

META INFORMATION

DATA FIELDS _gt_panoptic_seg: <BaseDataElement(

META INFORMATION

DATA FIELDS panoptic_seg: tensor([[0.7586, 0.1262, 0.2892, 0.9341],

[0.3200, 0.7448, 0.1052, 0.5371]])

) at 0x7f66c2bb7730>

gt_panoptic_seg: <BaseDataElement(

META INFORMATION

DATA FIELDS panoptic_seg: tensor([[0.7586, 0.1262, 0.2892, 0.9341],

[0.3200, 0.7448, 0.1052, 0.5371]])

) at 0x7f66c2bb7730>

) at 0x7f66c2bb7280> >>> data_sample = DetDataSample() >>> gt_segm_seg_data = dict(segm_seg=torch.rand(2, 2, 2)) >>> gt_segm_seg = PixelData(**gt_segm_seg_data) >>> data_sample.gt_segm_seg = gt_segm_seg >>> assert ‘gt_segm_seg’ in data_sample >>> assert ‘segm_seg’ in data_sample.gt_segm_seg

class mmdet.structures.ReIDDataSample(*, metainfo: Optional[dict] = None, **kwargs)[source]¶

A data structure interface of ReID task.

It’s used as interfaces between different components.

Meta field:

img_shape (Tuple): The shape of the corresponding input image.: Used for visualization.
ori_shape (Tuple): The original shape of the corresponding image.: Used for visualization.
num_classes (int): The number of all categories.: Used for label format conversion.

Data field:

gt_label (LabelData): The ground truth label. pred_label (LabelData): The predicted label. scores (torch.Tensor): The outputs of model.

set_gt_label(value: Union[numpy.ndarray, torch.Tensor, Sequence[numbers.Number], numbers.Number]) → mmdet.structures.reid_data_sample.ReIDDataSample [source]¶: Set label of gt_label.

set_gt_score(value: torch.Tensor) → mmdet.structures.reid_data_sample.ReIDDataSample [source]¶: Set score of gt_label.

class mmdet.structures.TrackDataSample(*, metainfo: Optional[dict] = None, **kwargs)[source]¶

A data structure interface of tracking task in MMDetection. It is used as interfaces between different components.

This data structure can be viewd as a wrapper of multiple DetDataSample to some extent. Specifically, it only contains a property: video_data_samples which is a list of DetDataSample, each of which corresponds to a single frame. If you want to get the property of a single frame, you must first get the corresponding DetDataSample by indexing and then get the property of the frame, such as gt_instances, pred_instances and so on. As for metainfo, it differs from DetDataSample in that each value corresponds to the metainfo key is a list where each element corresponds to information of a single frame.

Examples

>>> import torch
>>> from mmengine.structures import InstanceData
>>> from mmdet.structures import DetDataSample, TrackDataSample
>>> track_data_sample = TrackDataSample()
>>> # set the 1st frame
>>> frame1_data_sample = DetDataSample(metainfo=dict(
...         img_shape=(100, 100), frame_id=0))
>>> frame1_gt_instances = InstanceData()
>>> frame1_gt_instances.bbox = torch.zeros([2, 4])
>>> frame1_data_sample.gt_instances = frame1_gt_instances
>>> # set the 2nd frame
>>> frame2_data_sample = DetDataSample(metainfo=dict(
...         img_shape=(100, 100), frame_id=1))
>>> frame2_gt_instances = InstanceData()
>>> frame2_gt_instances.bbox = torch.ones([3, 4])
>>> frame2_data_sample.gt_instances = frame2_gt_instances
>>> track_data_sample.video_data_samples = [frame1_data_sample,
...                                         frame2_data_sample]
>>> # set metainfo for track_data_sample
>>> track_data_sample.set_metainfo(dict(key_frames_inds=[0]))
>>> track_data_sample.set_metainfo(dict(ref_frames_inds=[1]))
>>> print(track_data_sample)
<TrackDataSample(

META INFORMATION key_frames_inds: [0] ref_frames_inds: [1]

DATA FIELDS video_data_samples: [<DetDataSample(

META INFORMATION img_shape: (100, 100)

DATA FIELDS gt_instances: <InstanceData(

META INFORMATION

DATA FIELDS bbox: tensor([[0., 0., 0., 0.],

[0., 0., 0., 0.]])

) at 0x7f639320dcd0>

) at 0x7f64bd223340>, <DetDataSample(

META INFORMATION img_shape: (100, 100)

DATA FIELDS gt_instances: <InstanceData(

META INFORMATION

DATA FIELDS bbox: tensor([[1., 1., 1., 1.],

[1., 1., 1., 1.], [1., 1., 1., 1.]])

) at 0x7f64bd128b20>

) at 0x7f64bd1346d0>]

) at 0x7f64bd2237f0> >>> print(len(track_data_sample)) 2 >>> key_data_sample = track_data_sample.get_key_frames() >>> print(key_data_sample[0].frame_id) 0 >>> ref_data_sample = track_data_sample.get_ref_frames() >>> print(ref_data_sample[0].frame_id) 1 >>> frame1_data_sample = track_data_sample[0] >>> print(frame1_data_sample.gt_instances.bbox) tensor([[0., 0., 0., 0.],

[0., 0., 0., 0.]])

>>> # Tensor-like methods
>>> cuda_track_data_sample = track_data_sample.to('cuda')
>>> cuda_track_data_sample = track_data_sample.cuda()
>>> cpu_track_data_sample = track_data_sample.cpu()
>>> cpu_track_data_sample = track_data_sample.to('cpu')
>>> fp16_instances = cuda_track_data_sample.to(
...     device=None, dtype=torch.float16, non_blocking=False,
...     copy=False, memory_format=torch.preserve_format)

clone() → mmengine.structures.base_data_element.BaseDataElement[source]¶

Deep copy the current data element.

Returns: The copy of current data element.
Return type: BaseDataElement

cpu() → mmengine.structures.base_data_element.BaseDataElement[source]¶: Convert all tensors to CPU in data.

cuda() → mmengine.structures.base_data_element.BaseDataElement[source]¶: Convert all tensors to GPU in data.

detach() → mmengine.structures.base_data_element.BaseDataElement[source]¶: Detach all tensors in data.

npu() → mmengine.structures.base_data_element.BaseDataElement[source]¶: Convert all tensors to NPU in data.

numpy() → mmengine.structures.base_data_element.BaseDataElement[source]¶: Convert all tensors to np.ndarray in data.

to(*args, **kwargs) → mmengine.structures.base_data_element.BaseDataElement[source]¶: Apply same name function to all tensors in data_fields.

to_tensor() → mmengine.structures.base_data_element.BaseDataElement[source]¶: Convert all np.ndarray to tensor in data.

bbox¶

mask¶

mmdet.testing¶

mmdet.visualization¶

mmdet.utils¶

class mmdet.utils.AvoidOOM(to_cpu=True, test=False)[source]¶

Try to convert inputs to FP16 and CPU if got a PyTorch’s CUDA Out of Memory error. It will do the following steps:

First retry after calling torch.cuda.empty_cache().

If that still fails, it will then retry by converting inputs

to FP16.

If that still fails trying to convert inputs to CPUs.

In this case, it expects the function to dispatch to CPU implementation.

Parameters

to_cpu (bool) – Whether to convert outputs to CPU if get an OOM error. This will slow down the code significantly. Defaults to True.
test (bool) – Skip _ignore_torch_cuda_oom operate that can use lightweight data in unit test, only used in test unit. Defaults to False.

Examples

>>> from mmdet.utils.memory import AvoidOOM
>>> AvoidCUDAOOM = AvoidOOM()
>>> output = AvoidOOM.retry_if_cuda_oom(
>>>     some_torch_function)(input1, input2)
>>> # To use as a decorator
>>> # from mmdet.utils import AvoidCUDAOOM
>>> @AvoidCUDAOOM.retry_if_cuda_oom
>>> def function(*args, **kwargs):
>>>     return None

```

Note

The output may be on CPU even if inputs are on GPU. Processing
on CPU will slow down the code significantly.
When converting inputs to CPU, it will only look at each argument
and check if it has .device and .to for conversion. Nested structures of tensors are not supported.
Since the function might be called more than once, it has to be
stateless.

retry_if_cuda_oom(func)[source]¶

Makes a function retry itself after encountering pytorch’s CUDA OOM error.

The implementation logic is referred to https://github.com/facebookresearch/detectron2/blob/main/detectron2/utils/memory.py

Parameters: func – a stateless callable that takes tensor-like objects as arguments.
Returns: a callable which retries func if OOM is encountered.
Return type: func

mmdet.utils.all_reduce_dict(py_dict, op='sum', group=None, to_float=True)[source]¶

Apply all reduce function for python dict object.

The code is modified from https://github.com/Megvii- BaseDetection/YOLOX/blob/main/yolox/utils/allreduce_norm.py.

NOTE: make sure that py_dict in different ranks has the same keys and the values should be in the same shape. Currently only supports nccl backend.

Parameters

py_dict (dict) – Dict to be applied all reduce op.
op (str) – Operator, could be ‘sum’ or ‘mean’. Default: ‘sum’
group (torch.distributed.group, optional) – Distributed group, Default: None.
to_float (bool) – Whether to convert all values of dict to float. Default: True.

Returns

reduced python dict object.

Return type

OrderedDict

mmdet.utils.allreduce_grads(params, coalesce=True, bucket_size_mb=- 1)[source]¶

Allreduce gradients.

Parameters

params (list[torch.Parameters]) – List of parameters of a model
coalesce (bool, optional) – Whether allreduce parameters as a whole. Defaults to True.
bucket_size_mb (int, optional) – Size of bucket, the unit is MB. Defaults to -1.

mmdet.utils.collect_env()[source]¶: Collect the information of the running environments.

mmdet.utils.compat_cfg(cfg)[source]¶

This function would modify some filed to keep the compatibility of config.

For example, it will move some args which will be deprecated to the correct fields.

mmdet.utils.find_latest_checkpoint(path, suffix='pth')[source]¶

Find the latest checkpoint from the working directory.

Parameters

path (str) – The path to find checkpoints.
suffix (str) – File extension. Defaults to pth.

Returns

File path of the latest checkpoint.

Return type

latest_path(str | None)

References

1: https://github.com/microsoft/SoftTeacher /blob/main/ssod/utils/patch.py

mmdet.utils.get_caller_name()[source]¶: Get name of caller method.

mmdet.utils.get_test_pipeline_cfg(cfg: Union[str, mmengine.config.config.ConfigDict]) → mmengine.config.config.ConfigDict[source]¶

Get the test dataset pipeline from entire config.

Parameters: cfg (str or ConfigDict) – the entire config. Can be a config file or a ConfigDict.
Returns: the config of test dataset.
Return type: ConfigDict

mmdet.utils.imshow_mot_errors(*args, backend: str = 'cv2', **kwargs)[source]¶

Show the wrong tracks on the input image.

Parameters: backend (str, optional) – Backend of visualization. Defaults to ‘cv2’.

mmdet.utils.log_img_scale(img_scale, shape_order='hw', skip_square=False)[source]¶

Log image size.

Parameters

img_scale (tuple) – Image size to be logged.
shape_order (str, optional) – The order of image shape. ‘hw’ for (height, width) and ‘wh’ for (width, height). Defaults to ‘hw’.
skip_square (bool, optional) – Whether to skip logging for square img_scale. Defaults to False.

Returns

Whether to have done logging.

Return type

bool

mmdet.utils.reduce_mean(tensor)[source]¶: “Obtain the mean of tensor on different GPUs.

mmdet.utils.register_all_modules(init_default_scope: bool = True) → None[source]¶

Register all modules in mmdet into the registries.

Parameters: init_default_scope (bool) – Whether initialize the mmdet default scope. When init_default_scope=True, the global default scope will be set to mmdet, and all registries will build modules from mmdet’s registry node. To understand more about the registry, please refer to https://github.com/open-mmlab/mmengine/blob/main/docs/en/tutorials/registry.md Defaults to True.

mmdet.utils.replace_cfg_vals(ori_cfg)[source]¶

Replace the string “${key}” with the corresponding value.

Replace the “${key}” with the value of ori_cfg.key in the config. And support replacing the chained ${key}. Such as, replace “${key0.key1}” with the value of cfg.key0.key1. Code is modified from `vars.py < https://github.com/microsoft/SoftTeacher/blob/main/ssod/utils/vars.py>`_ # noqa: E501

Parameters: ori_cfg (mmengine.config.Config) – The origin config with “${key}” generated from a file.
Returns: The config with “${key}” replaced by the corresponding value.
Return type: updated_cfg [mmengine.config.Config]

mmdet.utils.setup_cache_size_limit_of_dynamo()[source]¶

Setup cache size limit of dynamo.

Note: Due to the dynamic shape of the loss calculation and post-processing parts in the object detection algorithm, these functions must be compiled every time they are run. Setting a large value for torch._dynamo.config.cache_size_limit may result in repeated compilation, which can slow down training and testing speed. Therefore, we need to set the default value of cache_size_limit smaller. An empirical value is 4.

mmdet.utils.setup_multi_processes(cfg)[source]¶: Setup multi-processing environment variables.

mmdet.utils.split_batch(img, img_metas, kwargs)[source]¶

Split data_batch by tags.

Code is modified from <https://github.com/microsoft/SoftTeacher/blob/main/ssod/utils/structure_utils.py> # noqa: E501

Parameters

img (Tensor) – of shape (N, C, H, W) encoding input images. Typically these should be mean centered and std scaled.
img_metas (list[dict]) – List of image info dict where each dict has: ‘img_shape’, ‘scale_factor’, ‘flip’, and may also contain ‘filename’, ‘ori_shape’, ‘pad_shape’, and ‘img_norm_cfg’. For details on the values of these keys, see mmdet.datasets.pipelines.Collect.
kwargs (dict) – Specific to concrete implementation.

Returns

a dict that data_batch splited by tags,: such as ‘sup’, ‘unsup_teacher’, and ‘unsup_student’.

Return type

data_groups (dict)

mmdet.utils.sync_random_seed(seed=None, device='cuda')[source]¶

Make sure different ranks share the same seed.

All workers must call this function, otherwise it will deadlock. This method is generally used in DistributedSampler, because the seed should be identical across all processes in the distributed group.

In distributed sampling, different ranks should sample non-overlapped data in the dataset. Therefore, this function is used to make sure that each rank shuffles the data indices in the same order based on the same seed. Then different ranks could use different indices to select non-overlapped data from the same data list.

Parameters

seed (int, Optional) – The seed. Default to None.
device (str) – The device where the seed will be put on. Default to ‘cuda’.

Returns

Seed to be used.

Return type

int

mmdet.utils.update_data_root(cfg, logger=None)[source]¶

Update data root according to env MMDET_DATASETS.

If set env MMDET_DATASETS, update cfg.data_root according to MMDET_DATASETS. Otherwise, using cfg.data_root as default.

Parameters

cfg (Config) – The model config need to modify
logger (logging.Logger | str | None) – the way to print msg