Shortcuts

mmdet.apis

async mmdet.apis.async_inference_detector(model, imgs)[source]

Async inference image(s) with the detector.

Parameters
  • model (nn.Module) – The loaded detector.

  • img (str | ndarray) – Either image files or loaded images.

Returns

Awaitable detection results.

mmdet.apis.get_root_logger(log_file=None, log_level=20)[source]

Get root logger.

Parameters
  • log_file (str, optional) – File path of log. Defaults to None.

  • log_level (int, optional) – The level of logger. Defaults to logging.INFO.

Returns

The obtained logger

Return type

logging.Logger

mmdet.apis.inference_detector(model, imgs)[source]

Inference image(s) with the detector.

Parameters
  • model (nn.Module) – The loaded detector.

  • imgs (str/ndarray or list[str/ndarray] or tuple[str/ndarray]) – Either image files or loaded images.

Returns

If imgs is a list or tuple, the same length list type results will be returned, otherwise return the detection results directly.

mmdet.apis.init_detector(config, checkpoint=None, device='cuda:0', cfg_options=None)[source]

Initialize a detector from config file.

Parameters
  • config (str or mmcv.Config) – Config file path or the config object.

  • checkpoint (str, optional) – Checkpoint path. If left as None, the model will not load any weights.

  • cfg_options (dict) – Options to override some settings in the used config.

Returns

The constructed detector.

Return type

nn.Module

mmdet.apis.multi_gpu_test(model, data_loader, tmpdir=None, gpu_collect=False)[source]

Test model with multiple gpus.

This method tests model with multiple gpus and collects the results under two different modes: gpu and cpu modes. By setting ‘gpu_collect=True’ it encodes results to gpu tensors and use gpu communication for results collection. On cpu mode it saves the results on different gpus to ‘tmpdir’ and collects them by the rank 0 worker.

Parameters
  • model (nn.Module) – Model to be tested.

  • data_loader (nn.Dataloader) – Pytorch data loader.

  • tmpdir (str) – Path of directory to save the temporary results from different gpus under cpu mode.

  • gpu_collect (bool) – Option to use either gpu or cpu to collect results.

Returns

The prediction results.

Return type

list

mmdet.apis.set_random_seed(seed, deterministic=False)[source]

Set random seed.

Parameters
  • seed (int) – Seed to be used.

  • deterministic (bool) – Whether to set the deterministic option for CUDNN backend, i.e., set torch.backends.cudnn.deterministic to True and torch.backends.cudnn.benchmark to False. Default: False.

mmdet.apis.show_result_pyplot(model, img, result, score_thr=0.3, title='result', wait_time=0)[source]

Visualize the detection results on the image.

Parameters
  • model (nn.Module) – The loaded detector.

  • img (str or np.ndarray) – Image filename or loaded image.

  • result (tuple[list] or list) – The detection result, can be either (bbox, segm) or just bbox.

  • score_thr (float) – The threshold to visualize the bboxes and masks.

  • title (str) – Title of the pyplot figure.

  • wait_time (float) – Value of waitKey param. Default: 0.

mmdet.core

anchor

class mmdet.core.anchor.AnchorGenerator(strides, ratios, scales=None, base_sizes=None, scale_major=True, octave_base_scale=None, scales_per_octave=None, centers=None, center_offset=0.0)[source]

Standard anchor generator for 2D anchor-based detectors.

Parameters
  • strides (list[int] | list[tuple[int, int]]) – Strides of anchors in multiple feature levels in order (w, h).

  • ratios (list[float]) – The list of ratios between the height and width of anchors in a single level.

  • scales (list[int] | None) – Anchor scales for anchors in a single level. It cannot be set at the same time if octave_base_scale and scales_per_octave are set.

  • base_sizes (list[int] | None) – The basic sizes of anchors in multiple levels. If None is given, strides will be used as base_sizes. (If strides are non square, the shortest stride is taken.)

  • scale_major (bool) – Whether to multiply scales first when generating base anchors. If true, the anchors in the same row will have the same scales. By default it is True in V2.0

  • octave_base_scale (int) – The base scale of octave.

  • scales_per_octave (int) – Number of scales for each octave. octave_base_scale and scales_per_octave are usually used in retinanet and the scales should be None when they are set.

  • centers (list[tuple[float, float]] | None) – The centers of the anchor relative to the feature grid center in multiple feature levels. By default it is set to be None and not used. If a list of tuple of float is given, they will be used to shift the centers of anchors.

  • center_offset (float) – The offset of center in proportion to anchors’ width and height. By default it is 0 in V2.0.

Examples

>>> from mmdet.core import AnchorGenerator
>>> self = AnchorGenerator([16], [1.], [1.], [9])
>>> all_anchors = self.grid_anchors([(2, 2)], device='cpu')
>>> print(all_anchors)
[tensor([[-4.5000, -4.5000,  4.5000,  4.5000],
        [11.5000, -4.5000, 20.5000,  4.5000],
        [-4.5000, 11.5000,  4.5000, 20.5000],
        [11.5000, 11.5000, 20.5000, 20.5000]])]
>>> self = AnchorGenerator([16, 32], [1.], [1.], [9, 18])
>>> all_anchors = self.grid_anchors([(2, 2), (1, 1)], device='cpu')
>>> print(all_anchors)
[tensor([[-4.5000, -4.5000,  4.5000,  4.5000],
        [11.5000, -4.5000, 20.5000,  4.5000],
        [-4.5000, 11.5000,  4.5000, 20.5000],
        [11.5000, 11.5000, 20.5000, 20.5000]]),         tensor([[-9., -9., 9., 9.]])]
gen_base_anchors()[source]

Generate base anchors.

Returns

Base anchors of a feature grid in multiple feature levels.

Return type

list(torch.Tensor)

gen_single_level_base_anchors(base_size, scales, ratios, center=None)[source]

Generate base anchors of a single level.

Parameters
  • base_size (int | float) – Basic size of an anchor.

  • scales (torch.Tensor) – Scales of the anchor.

  • ratios (torch.Tensor) – The ratio between between the height and width of anchors in a single level.

  • center (tuple[float], optional) – The center of the base anchor related to a single feature grid. Defaults to None.

Returns

Anchors in a single-level feature maps.

Return type

torch.Tensor

grid_anchors(featmap_sizes, device='cuda')[source]

Generate grid anchors in multiple feature levels.

Parameters
  • featmap_sizes (list[tuple]) – List of feature map sizes in multiple feature levels.

  • device (str) – Device where the anchors will be put on.

Returns

Anchors in multiple feature levels. The sizes of each tensor should be [N, 4], where N = width * height * num_base_anchors, width and height are the sizes of the corresponding feature level, num_base_anchors is the number of anchors for that level.

Return type

list[torch.Tensor]

grid_priors(featmap_sizes, device='cuda')[source]

Generate grid anchors in multiple feature levels.

Parameters
  • featmap_sizes (list[tuple]) – List of feature map sizes in multiple feature levels.

  • device (str) – The device where the anchors will be put on.

Returns

Anchors in multiple feature levels. The sizes of each tensor should be [N, 4], where N = width * height * num_base_anchors, width and height are the sizes of the corresponding feature level, num_base_anchors is the number of anchors for that level.

Return type

list[torch.Tensor]

property num_base_anchors

total number of base anchors in a feature grid

Type

list[int]

property num_base_priors

The number of priors (anchors) at a point on the feature grid

Type

list[int]

property num_levels

number of feature levels that the generator will be applied

Type

int

single_level_grid_anchors(base_anchors, featmap_size, stride=(16, 16), device='cuda')[source]

Generate grid anchors of a single level.

Note

This function is usually called by method self.grid_anchors.

Parameters
  • base_anchors (torch.Tensor) – The base anchors of a feature grid.

  • featmap_size (tuple[int]) – Size of the feature maps.

  • stride (tuple[int], optional) – Stride of the feature map in order (w, h). Defaults to (16, 16).

  • device (str, optional) – Device the tensor will be put on. Defaults to ‘cuda’.

Returns

Anchors in the overall feature maps.

Return type

torch.Tensor

single_level_grid_priors(featmap_size, level_idx, device='cuda')[source]

Generate grid anchors of a single level.

Note

This function is usually called by method self.grid_priors.

Parameters
  • featmap_size (tuple[int]) – Size of the feature maps.

  • level_idx (int) – The index of corresponding feature map level.

  • device (str, optional) – The device the tensor will be put on. Defaults to ‘cuda’.

Returns

Anchors in the overall feature maps.

Return type

torch.Tensor

single_level_valid_flags(featmap_size, valid_size, num_base_anchors, device='cuda')[source]

Generate the valid flags of anchor in a single feature map.

Parameters
  • featmap_size (tuple[int]) – The size of feature maps, arrange as (h, w).

  • valid_size (tuple[int]) – The valid size of the feature maps.

  • num_base_anchors (int) – The number of base anchors.

  • device (str, optional) – Device where the flags will be put on. Defaults to ‘cuda’.

Returns

The valid flags of each anchor in a single level feature map.

Return type

torch.Tensor

sparse_priors(prior_idxs, featmap_size, level_idx, dtype=torch.float32, device='cuda')[source]

Generate sparse anchors according to the prior_idxs.

Parameters
  • prior_idxs (Tensor) – The index of corresponding anchors in the feature map.

  • featmap_size (tuple[int]) – feature map size arrange as (h, w).

  • level_idx (int) – The level index of corresponding feature map.

  • (obj (device) – torch.dtype): Date type of points.Defaults to torch.float32.

  • (objtorch.device): The device where the points is located.

Returns

Anchor with shape (N, 4), N should be equal to

the length of prior_idxs.

Return type

Tensor

valid_flags(featmap_sizes, pad_shape, device='cuda')[source]

Generate valid flags of anchors in multiple feature levels.

Parameters
  • featmap_sizes (list(tuple)) – List of feature map sizes in multiple feature levels.

  • pad_shape (tuple) – The padded shape of the image.

  • device (str) – Device where the anchors will be put on.

Returns

Valid flags of anchors in multiple levels.

Return type

list(torch.Tensor)

class mmdet.core.anchor.LegacyAnchorGenerator(strides, ratios, scales=None, base_sizes=None, scale_major=True, octave_base_scale=None, scales_per_octave=None, centers=None, center_offset=0.0)[source]

Legacy anchor generator used in MMDetection V1.x.

Note

Difference to the V2.0 anchor generator:

  1. The center offset of V1.x anchors are set to be 0.5 rather than 0.

  2. The width/height are minused by 1 when calculating the anchors’ centers and corners to meet the V1.x coordinate system.

  3. The anchors’ corners are quantized.

Parameters
  • strides (list[int] | list[tuple[int]]) – Strides of anchors in multiple feature levels.

  • ratios (list[float]) – The list of ratios between the height and width of anchors in a single level.

  • scales (list[int] | None) – Anchor scales for anchors in a single level. It cannot be set at the same time if octave_base_scale and scales_per_octave are set.

  • base_sizes (list[int]) – The basic sizes of anchors in multiple levels. If None is given, strides will be used to generate base_sizes.

  • scale_major (bool) – Whether to multiply scales first when generating base anchors. If true, the anchors in the same row will have the same scales. By default it is True in V2.0

  • octave_base_scale (int) – The base scale of octave.

  • scales_per_octave (int) – Number of scales for each octave. octave_base_scale and scales_per_octave are usually used in retinanet and the scales should be None when they are set.

  • centers (list[tuple[float, float]] | None) – The centers of the anchor relative to the feature grid center in multiple feature levels. By default it is set to be None and not used. It a list of float is given, this list will be used to shift the centers of anchors.

  • center_offset (float) – The offset of center in proportion to anchors’ width and height. By default it is 0.5 in V2.0 but it should be 0.5 in v1.x models.

Examples

>>> from mmdet.core import LegacyAnchorGenerator
>>> self = LegacyAnchorGenerator(
>>>     [16], [1.], [1.], [9], center_offset=0.5)
>>> all_anchors = self.grid_anchors(((2, 2),), device='cpu')
>>> print(all_anchors)
[tensor([[ 0.,  0.,  8.,  8.],
        [16.,  0., 24.,  8.],
        [ 0., 16.,  8., 24.],
        [16., 16., 24., 24.]])]
gen_single_level_base_anchors(base_size, scales, ratios, center=None)[source]

Generate base anchors of a single level.

Note

The width/height of anchors are minused by 1 when calculating the centers and corners to meet the V1.x coordinate system.

Parameters
  • base_size (int | float) – Basic size of an anchor.

  • scales (torch.Tensor) – Scales of the anchor.

  • ratios (torch.Tensor) – The ratio between between the height. and width of anchors in a single level.

  • center (tuple[float], optional) – The center of the base anchor related to a single feature grid. Defaults to None.

Returns

Anchors in a single-level feature map.

Return type

torch.Tensor

class mmdet.core.anchor.MlvlPointGenerator(strides, offset=0.5)[source]

Standard points generator for multi-level (Mlvl) feature maps in 2D points-based detectors.

Parameters
  • strides (list[int] | list[tuple[int, int]]) – Strides of anchors in multiple feature levels in order (w, h).

  • offset (float) – The offset of points, the value is normalized with corresponding stride. Defaults to 0.5.

grid_priors(featmap_sizes, device='cuda', with_stride=False)[source]

Generate grid points of multiple feature levels.

Parameters
  • featmap_sizes (list[tuple]) – List of feature map sizes in multiple feature levels, each size arrange as as (h, w).

  • device (str) – The device where the anchors will be put on.

  • with_stride (bool) – Whether to concatenate the stride to the last dimension of points.

Returns

Points of multiple feature levels. The sizes of each tensor should be (N, 2) when with stride is False, where N = width * height, width and height are the sizes of the corresponding feature level, and the last dimension 2 represent (coord_x, coord_y), otherwise the shape should be (N, 4), and the last dimension 4 represent (coord_x, coord_y, stride_w, stride_h).

Return type

list[torch.Tensor]

property num_base_priors

The number of priors (points) at a point on the feature grid

Type

list[int]

property num_levels

number of feature levels that the generator will be applied

Type

int

single_level_grid_priors(featmap_size, level_idx, device='cuda', with_stride=False)[source]

Generate grid Points of a single level.

Note

This function is usually called by method self.grid_priors.

Parameters
  • featmap_size (tuple[int]) – Size of the feature maps, arrange as (h, w).

  • level_idx (int) – The index of corresponding feature map level.

  • device (str, optional) – The device the tensor will be put on. Defaults to ‘cuda’.

  • with_stride (bool) – Concatenate the stride to the last dimension of points.

Returns

Points of single feature levels. The shape of tensor should be (N, 2) when with stride is False, where N = width * height, width and height are the sizes of the corresponding feature level, and the last dimension 2 represent (coord_x, coord_y), otherwise the shape should be (N, 4), and the last dimension 4 represent (coord_x, coord_y, stride_w, stride_h).

Return type

Tensor

single_level_valid_flags(featmap_size, valid_size, device='cuda')[source]

Generate the valid flags of points of a single feature map.

Parameters
  • featmap_size (tuple[int]) – The size of feature maps, arrange as as (h, w).

  • valid_size (tuple[int]) – The valid size of the feature maps. The size arrange as as (h, w).

  • device (str, optional) – The device where the flags will be put on. Defaults to ‘cuda’.

Returns

The valid flags of each points in a single level feature map.

Return type

torch.Tensor

sparse_priors(prior_idxs, featmap_size, level_idx, dtype=torch.float32, device='cuda')[source]

Generate sparse points according to the prior_idxs.

Parameters
  • prior_idxs (Tensor) – The index of corresponding anchors in the feature map.

  • featmap_size (tuple[int]) – feature map size arrange as (w, h).

  • level_idx (int) – The level index of corresponding feature map.

  • (obj (device) – torch.dtype): Date type of points. Defaults to torch.float32.

  • (objtorch.device): The device where the points is located.

Returns

Anchor with shape (N, 2), N should be equal to the length of prior_idxs. And last dimension 2 represent (coord_x, coord_y).

Return type

Tensor

valid_flags(featmap_sizes, pad_shape, device='cuda')[source]

Generate valid flags of points of multiple feature levels.

Parameters
  • featmap_sizes (list(tuple)) – List of feature map sizes in multiple feature levels, each size arrange as as (h, w).

  • pad_shape (tuple(int)) – The padded shape of the image, arrange as (h, w).

  • device (str) – The device where the anchors will be put on.

Returns

Valid flags of points of multiple levels.

Return type

list(torch.Tensor)

class mmdet.core.anchor.YOLOAnchorGenerator(strides, base_sizes)[source]

Anchor generator for YOLO.

Parameters
  • strides (list[int] | list[tuple[int, int]]) – Strides of anchors in multiple feature levels.

  • base_sizes (list[list[tuple[int, int]]]) – The basic sizes of anchors in multiple levels.

gen_base_anchors()[source]

Generate base anchors.

Returns

Base anchors of a feature grid in multiple feature levels.

Return type

list(torch.Tensor)

gen_single_level_base_anchors(base_sizes_per_level, center=None)[source]

Generate base anchors of a single level.

Parameters
  • base_sizes_per_level (list[tuple[int, int]]) – Basic sizes of anchors.

  • center (tuple[float], optional) – The center of the base anchor related to a single feature grid. Defaults to None.

Returns

Anchors in a single-level feature maps.

Return type

torch.Tensor

property num_levels

number of feature levels that the generator will be applied

Type

int

responsible_flags(featmap_sizes, gt_bboxes, device='cuda')[source]

Generate responsible anchor flags of grid cells in multiple scales.

Parameters
  • featmap_sizes (list(tuple)) – List of feature map sizes in multiple feature levels.

  • gt_bboxes (Tensor) – Ground truth boxes, shape (n, 4).

  • device (str) – Device where the anchors will be put on.

Returns

responsible flags of anchors in multiple level

Return type

list(torch.Tensor)

single_level_responsible_flags(featmap_size, gt_bboxes, stride, num_base_anchors, device='cuda')[source]

Generate the responsible flags of anchor in a single feature map.

Parameters
  • featmap_size (tuple[int]) – The size of feature maps.

  • gt_bboxes (Tensor) – Ground truth boxes, shape (n, 4).

  • stride (tuple(int)) – stride of current level

  • num_base_anchors (int) – The number of base anchors.

  • device (str, optional) – Device where the flags will be put on. Defaults to ‘cuda’.

Returns

The valid flags of each anchor in a single level feature map.

Return type

torch.Tensor

mmdet.core.anchor.anchor_inside_flags(flat_anchors, valid_flags, img_shape, allowed_border=0)[source]

Check whether the anchors are inside the border.

Parameters
  • flat_anchors (torch.Tensor) – Flatten anchors, shape (n, 4).

  • valid_flags (torch.Tensor) – An existing valid flags of anchors.

  • img_shape (tuple(int)) – Shape of current image.

  • allowed_border (int, optional) – The border to allow the valid anchor. Defaults to 0.

Returns

Flags indicating whether the anchors are inside a valid range.

Return type

torch.Tensor

mmdet.core.anchor.calc_region(bbox, ratio, featmap_size=None)[source]

Calculate a proportional bbox region.

The bbox center are fixed and the new h’ and w’ is h * ratio and w * ratio.

Parameters
  • bbox (Tensor) – Bboxes to calculate regions, shape (n, 4).

  • ratio (float) – Ratio of the output region.

  • featmap_size (tuple) – Feature map size used for clipping the boundary.

Returns

x1, y1, x2, y2

Return type

tuple

mmdet.core.anchor.images_to_levels(target, num_levels)[source]

Convert targets by image to targets by feature level.

[target_img0, target_img1] -> [target_level0, target_level1, …]

bbox

class mmdet.core.bbox.AssignResult(num_gts, gt_inds, max_overlaps, labels=None)[source]

Stores assignments between predicted and truth boxes.

num_gts

the number of truth boxes considered when computing this assignment

Type

int

gt_inds

for each predicted box indicates the 1-based index of the assigned truth box. 0 means unassigned and -1 means ignore.

Type

LongTensor

max_overlaps

the iou between the predicted box and its assigned truth box.

Type

FloatTensor

labels

If specified, for each predicted box indicates the category label of the assigned truth box.

Type

None | LongTensor

Example

>>> # An assign result between 4 predicted boxes and 9 true boxes
>>> # where only two boxes were assigned.
>>> num_gts = 9
>>> max_overlaps = torch.LongTensor([0, .5, .9, 0])
>>> gt_inds = torch.LongTensor([-1, 1, 2, 0])
>>> labels = torch.LongTensor([0, 3, 4, 0])
>>> self = AssignResult(num_gts, gt_inds, max_overlaps, labels)
>>> print(str(self))  # xdoctest: +IGNORE_WANT
<AssignResult(num_gts=9, gt_inds.shape=(4,), max_overlaps.shape=(4,),
              labels.shape=(4,))>
>>> # Force addition of gt labels (when adding gt as proposals)
>>> new_labels = torch.LongTensor([3, 4, 5])
>>> self.add_gt_(new_labels)
>>> print(str(self))  # xdoctest: +IGNORE_WANT
<AssignResult(num_gts=9, gt_inds.shape=(7,), max_overlaps.shape=(7,),
              labels.shape=(7,))>
add_gt_(gt_labels)[source]

Add ground truth as assigned results.

Parameters

gt_labels (torch.Tensor) – Labels of gt boxes

get_extra_property(key)[source]

Get user-defined property.

property info

a dictionary of info about the object

Type

dict

property num_preds

the number of predictions in this assignment

Type

int

classmethod random(**kwargs)[source]

Create random AssignResult for tests or debugging.

Parameters
  • num_preds – number of predicted boxes

  • num_gts – number of true boxes

  • p_ignore (float) – probability of a predicted box assigned to an ignored truth

  • p_assigned (float) – probability of a predicted box not being assigned

  • p_use_label (float | bool) – with labels or not

  • rng (None | int | numpy.random.RandomState) – seed or state

Returns

Randomly generated assign results.

Return type

AssignResult

Example

>>> from mmdet.core.bbox.assigners.assign_result import *  # NOQA
>>> self = AssignResult.random()
>>> print(self.info)
set_extra_property(key, value)[source]

Set user-defined new property.

class mmdet.core.bbox.BaseAssigner[source]

Base assigner that assigns boxes to ground truth boxes.

abstract assign(bboxes, gt_bboxes, gt_bboxes_ignore=None, gt_labels=None)[source]

Assign boxes to either a ground truth boxes or a negative boxes.

class mmdet.core.bbox.BaseBBoxCoder(**kwargs)[source]

Base bounding box coder.

abstract decode(bboxes, bboxes_pred)[source]

Decode the predicted bboxes according to prediction and base boxes.

abstract encode(bboxes, gt_bboxes)[source]

Encode deltas between bboxes and ground truth boxes.

class mmdet.core.bbox.BaseSampler(num, pos_fraction, neg_pos_ub=- 1, add_gt_as_proposals=True, **kwargs)[source]

Base class of samplers.

sample(assign_result, bboxes, gt_bboxes, gt_labels=None, **kwargs)[source]

Sample positive and negative bboxes.

This is a simple implementation of bbox sampling given candidates, assigning results and ground truth bboxes.

Parameters
  • assign_result (AssignResult) – Bbox assigning results.

  • bboxes (Tensor) – Boxes to be sampled from.

  • gt_bboxes (Tensor) – Ground truth bboxes.

  • gt_labels (Tensor, optional) – Class labels of ground truth bboxes.

Returns

Sampling result.

Return type

SamplingResult

Example

>>> from mmdet.core.bbox import RandomSampler
>>> from mmdet.core.bbox import AssignResult
>>> from mmdet.core.bbox.demodata import ensure_rng, random_boxes
>>> rng = ensure_rng(None)
>>> assign_result = AssignResult.random(rng=rng)
>>> bboxes = random_boxes(assign_result.num_preds, rng=rng)
>>> gt_bboxes = random_boxes(assign_result.num_gts, rng=rng)
>>> gt_labels = None
>>> self = RandomSampler(num=32, pos_fraction=0.5, neg_pos_ub=-1,
>>>                      add_gt_as_proposals=False)
>>> self = self.sample(assign_result, bboxes, gt_bboxes, gt_labels)
class mmdet.core.bbox.BboxOverlaps2D(scale=1.0, dtype=None)[source]

2D Overlaps (e.g. IoUs, GIoUs) Calculator.

class mmdet.core.bbox.CenterRegionAssigner(pos_scale, neg_scale, min_pos_iof=0.01, ignore_gt_scale=0.5, foreground_dominate=False, iou_calculator={'type': 'BboxOverlaps2D'})[source]

Assign pixels at the center region of a bbox as positive.

Each proposals will be assigned with -1, 0, or a positive integer indicating the ground truth index. - -1: negative samples - semi-positive numbers: positive sample, index (0-based) of assigned gt

Parameters
  • pos_scale (float) – Threshold within which pixels are labelled as positive.

  • neg_scale (float) – Threshold above which pixels are labelled as positive.

  • min_pos_iof (float) – Minimum iof of a pixel with a gt to be labelled as positive. Default: 1e-2

  • ignore_gt_scale (float) – Threshold within which the pixels are ignored when the gt is labelled as shadowed. Default: 0.5

  • foreground_dominate (bool) – If True, the bbox will be assigned as positive when a gt’s kernel region overlaps with another’s shadowed (ignored) region, otherwise it is set as ignored. Default to False.

assign(bboxes, gt_bboxes, gt_bboxes_ignore=None, gt_labels=None)[source]

Assign gt to bboxes.

This method assigns gts to every bbox (proposal/anchor), each bbox will be assigned with -1, or a semi-positive number. -1 means negative sample, semi-positive number is the index (0-based) of assigned gt.

Parameters
  • bboxes (Tensor) – Bounding boxes to be assigned, shape(n, 4).

  • gt_bboxes (Tensor) – Groundtruth boxes, shape (k, 4).

  • gt_bboxes_ignore (tensor, optional) – Ground truth bboxes that are labelled as ignored, e.g., crowd boxes in COCO.

  • gt_labels (tensor, optional) – Label of gt_bboxes, shape (num_gts,).

Returns

The assigned result. Note that shadowed_labels of shape (N, 2) is also added as an assign_result attribute. shadowed_labels is a tensor composed of N pairs of anchor_ind, class_label], where N is the number of anchors that lie in the outer region of a gt, anchor_ind is the shadowed anchor index and class_label is the shadowed class label.

Return type

AssignResult

Example

>>> self = CenterRegionAssigner(0.2, 0.2)
>>> bboxes = torch.Tensor([[0, 0, 10, 10], [10, 10, 20, 20]])
>>> gt_bboxes = torch.Tensor([[0, 0, 10, 10]])
>>> assign_result = self.assign(bboxes, gt_bboxes)
>>> expected_gt_inds = torch.LongTensor([1, 0])
>>> assert torch.all(assign_result.gt_inds == expected_gt_inds)
assign_one_hot_gt_indices(is_bbox_in_gt_core, is_bbox_in_gt_shadow, gt_priority=None)[source]

Assign only one gt index to each prior box.

Gts with large gt_priority are more likely to be assigned.

Parameters
  • is_bbox_in_gt_core (Tensor) – Bool tensor indicating the bbox center is in the core area of a gt (e.g. 0-0.2). Shape: (num_prior, num_gt).

  • is_bbox_in_gt_shadow (Tensor) – Bool tensor indicating the bbox center is in the shadowed area of a gt (e.g. 0.2-0.5). Shape: (num_prior, num_gt).

  • gt_priority (Tensor) – Priorities of gts. The gt with a higher priority is more likely to be assigned to the bbox when the bbox match with multiple gts. Shape: (num_gt, ).

Returns

Returns (assigned_gt_inds, shadowed_gt_inds).

  • assigned_gt_inds: The assigned gt index of each prior bbox (i.e. index from 1 to num_gts). Shape: (num_prior, ).

  • shadowed_gt_inds: shadowed gt indices. It is a tensor of shape (num_ignore, 2) with first column being the shadowed prior bbox indices and the second column the shadowed gt indices (1-based).

Return type

tuple

get_gt_priorities(gt_bboxes)[source]

Get gt priorities according to their areas.

Smaller gt has higher priority.

Parameters

gt_bboxes (Tensor) – Ground truth boxes, shape (k, 4).

Returns

The priority of gts so that gts with larger priority is more likely to be assigned. Shape (k, )

Return type

Tensor

class mmdet.core.bbox.CombinedSampler(pos_sampler, neg_sampler, **kwargs)[source]

A sampler that combines positive sampler and negative sampler.

class mmdet.core.bbox.DeltaXYWHBBoxCoder(target_means=(0.0, 0.0, 0.0, 0.0), target_stds=(1.0, 1.0, 1.0, 1.0), clip_border=True, add_ctr_clamp=False, ctr_clamp=32)[source]

Delta XYWH BBox coder.

Following the practice in R-CNN, this coder encodes bbox (x1, y1, x2, y2) into delta (dx, dy, dw, dh) and decodes delta (dx, dy, dw, dh) back to original bbox (x1, y1, x2, y2).

Parameters
  • target_means (Sequence[float]) – Denormalizing means of target for delta coordinates

  • target_stds (Sequence[float]) – Denormalizing standard deviation of target for delta coordinates

  • clip_border (bool, optional) – Whether clip the objects outside the border of the image. Defaults to True.

  • add_ctr_clamp (bool) – Whether to add center clamp, when added, the predicted box is clamped is its center is too far away from the original anchor’s center. Only used by YOLOF. Default False.

  • ctr_clamp (int) – the maximum pixel shift to clamp. Only used by YOLOF. Default 32.

decode(bboxes, pred_bboxes, max_shape=None, wh_ratio_clip=0.016)[source]

Apply transformation pred_bboxes to boxes.

Parameters
  • bboxes (torch.Tensor) – Basic boxes. Shape (B, N, 4) or (N, 4)

  • pred_bboxes (Tensor) – Encoded offsets with respect to each roi. Has shape (B, N, num_classes * 4) or (B, N, 4) or (N, num_classes * 4) or (N, 4). Note N = num_anchors * W * H when rois is a grid of anchors.Offset encoding follows 1.

  • (Sequence[int] or torch.Tensor or Sequence[ (max_shape) – Sequence[int]],optional): Maximum bounds for boxes, specifies (H, W, C) or (H, W). If bboxes shape is (B, N, 4), then the max_shape should be a Sequence[Sequence[int]] and the length of max_shape should also be B.

  • wh_ratio_clip (float, optional) – The allowed ratio between width and height.

Returns

Decoded boxes.

Return type

torch.Tensor

encode(bboxes, gt_bboxes)[source]

Get box regression transformation deltas that can be used to transform the bboxes into the gt_bboxes.

Parameters
  • bboxes (torch.Tensor) – Source boxes, e.g., object proposals.

  • gt_bboxes (torch.Tensor) – Target of the transformation, e.g., ground-truth boxes.

Returns

Box transformation deltas

Return type

torch.Tensor

class mmdet.core.bbox.InstanceBalancedPosSampler(num, pos_fraction, neg_pos_ub=- 1, add_gt_as_proposals=True, **kwargs)[source]

Instance balanced sampler that samples equal number of positive samples for each instance.

class mmdet.core.bbox.IoUBalancedNegSampler(num, pos_fraction, floor_thr=- 1, floor_fraction=0, num_bins=3, **kwargs)[source]

IoU Balanced Sampling.

arXiv: https://arxiv.org/pdf/1904.02701.pdf (CVPR 2019)

Sampling proposals according to their IoU. floor_fraction of needed RoIs are sampled from proposals whose IoU are lower than floor_thr randomly. The others are sampled from proposals whose IoU are higher than floor_thr. These proposals are sampled from some bins evenly, which are split by num_bins via IoU evenly.

Parameters
  • num (int) – number of proposals.

  • pos_fraction (float) – fraction of positive proposals.

  • floor_thr (float) – threshold (minimum) IoU for IoU balanced sampling, set to -1 if all using IoU balanced sampling.

  • floor_fraction (float) – sampling fraction of proposals under floor_thr.

  • num_bins (int) – number of bins in IoU balanced sampling.

sample_via_interval(max_overlaps, full_set, num_expected)[source]

Sample according to the iou interval.

Parameters
  • max_overlaps (torch.Tensor) – IoU between bounding boxes and ground truth boxes.

  • full_set (set(int)) – A full set of indices of boxes。

  • num_expected (int) – Number of expected samples。

Returns

Indices of samples

Return type

np.ndarray

class mmdet.core.bbox.MaxIoUAssigner(pos_iou_thr, neg_iou_thr, min_pos_iou=0.0, gt_max_assign_all=True, ignore_iof_thr=- 1, ignore_wrt_candidates=True, match_low_quality=True, gpu_assign_thr=- 1, iou_calculator={'type': 'BboxOverlaps2D'})[source]

Assign a corresponding gt bbox or background to each bbox.

Each proposals will be assigned with -1, or a semi-positive integer indicating the ground truth index.

  • -1: negative sample, no assigned gt

  • semi-positive integer: positive sample, index (0-based) of assigned gt

Parameters
  • pos_iou_thr (float) – IoU threshold for positive bboxes.

  • neg_iou_thr (float or tuple) – IoU threshold for negative bboxes.

  • min_pos_iou (float) – Minimum iou for a bbox to be considered as a positive bbox. Positive samples can have smaller IoU than pos_iou_thr due to the 4th step (assign max IoU sample to each gt).

  • gt_max_assign_all (bool) – Whether to assign all bboxes with the same highest overlap with some gt to that gt.

  • ignore_iof_thr (float) – IoF threshold for ignoring bboxes (if gt_bboxes_ignore is specified). Negative values mean not ignoring any bboxes.

  • ignore_wrt_candidates (bool) – Whether to compute the iof between bboxes and gt_bboxes_ignore, or the contrary.

  • match_low_quality (bool) – Whether to allow low quality matches. This is usually allowed for RPN and single stage detectors, but not allowed in the second stage. Details are demonstrated in Step 4.

  • gpu_assign_thr (int) – The upper bound of the number of GT for GPU assign. When the number of gt is above this threshold, will assign on CPU device. Negative values mean not assign on CPU.

assign(bboxes, gt_bboxes, gt_bboxes_ignore=None, gt_labels=None)[source]

Assign gt to bboxes.

This method assign a gt bbox to every bbox (proposal/anchor), each bbox will be assigned with -1, or a semi-positive number. -1 means negative sample, semi-positive number is the index (0-based) of assigned gt. The assignment is done in following steps, the order matters.

  1. assign every bbox to the background

  2. assign proposals whose iou with all gts < neg_iou_thr to 0

  3. for each bbox, if the iou with its nearest gt >= pos_iou_thr, assign it to that bbox

  4. for each gt bbox, assign its nearest proposals (may be more than one) to itself

Parameters
  • bboxes (Tensor) – Bounding boxes to be assigned, shape(n, 4).

  • gt_bboxes (Tensor) – Groundtruth boxes, shape (k, 4).

  • gt_bboxes_ignore (Tensor, optional) – Ground truth bboxes that are labelled as ignored, e.g., crowd boxes in COCO.

  • gt_labels (Tensor, optional) – Label of gt_bboxes, shape (k, ).

Returns

The assign result.

Return type

AssignResult

Example

>>> self = MaxIoUAssigner(0.5, 0.5)
>>> bboxes = torch.Tensor([[0, 0, 10, 10], [10, 10, 20, 20]])
>>> gt_bboxes = torch.Tensor([[0, 0, 10, 9]])
>>> assign_result = self.assign(bboxes, gt_bboxes)
>>> expected_gt_inds = torch.LongTensor([1, 0])
>>> assert torch.all(assign_result.gt_inds == expected_gt_inds)
assign_wrt_overlaps(overlaps, gt_labels=None)[source]

Assign w.r.t. the overlaps of bboxes with gts.

Parameters
  • overlaps (Tensor) – Overlaps between k gt_bboxes and n bboxes, shape(k, n).

  • gt_labels (Tensor, optional) – Labels of k gt_bboxes, shape (k, ).

Returns

The assign result.

Return type

AssignResult

class mmdet.core.bbox.OHEMSampler(num, pos_fraction, context, neg_pos_ub=- 1, add_gt_as_proposals=True, **kwargs)[source]

Online Hard Example Mining Sampler described in Training Region-based Object Detectors with Online Hard Example Mining.

class mmdet.core.bbox.PseudoBBoxCoder(**kwargs)[source]

Pseudo bounding box coder.

decode(bboxes, pred_bboxes)[source]

torch.Tensor: return the given pred_bboxes

encode(bboxes, gt_bboxes)[source]

torch.Tensor: return the given bboxes

class mmdet.core.bbox.PseudoSampler(**kwargs)[source]

A pseudo sampler that does not do sampling actually.

sample(assign_result, bboxes, gt_bboxes, **kwargs)[source]

Directly returns the positive and negative indices of samples.

Parameters
  • assign_result (AssignResult) – Assigned results

  • bboxes (torch.Tensor) – Bounding boxes

  • gt_bboxes (torch.Tensor) – Ground truth boxes

Returns

sampler results

Return type

SamplingResult

class mmdet.core.bbox.RandomSampler(num, pos_fraction, neg_pos_ub=- 1, add_gt_as_proposals=True, **kwargs)[source]

Random sampler.

Parameters
  • num (int) – Number of samples

  • pos_fraction (float) – Fraction of positive samples

  • neg_pos_up (int, optional) – Upper bound number of negative and positive samples. Defaults to -1.

  • add_gt_as_proposals (bool, optional) – Whether to add ground truth boxes as proposals. Defaults to True.

random_choice(gallery, num)[source]

Random select some elements from the gallery.

If gallery is a Tensor, the returned indices will be a Tensor; If gallery is a ndarray or list, the returned indices will be a ndarray.

Parameters
  • gallery (Tensor | ndarray | list) – indices pool.

  • num (int) – expected sample num.

Returns

sampled indices.

Return type

Tensor or ndarray

class mmdet.core.bbox.RegionAssigner(center_ratio=0.2, ignore_ratio=0.5)[source]

Assign a corresponding gt bbox or background to each bbox.

Each proposals will be assigned with -1, 0, or a positive integer indicating the ground truth index.

  • -1: don’t care

  • 0: negative sample, no assigned gt

  • positive integer: positive sample, index (1-based) of assigned gt

Parameters
  • center_ratio – ratio of the region in the center of the bbox to define positive sample.

  • ignore_ratio – ratio of the region to define ignore samples.

assign(mlvl_anchors, mlvl_valid_flags, gt_bboxes, img_meta, featmap_sizes, anchor_scale, anchor_strides, gt_bboxes_ignore=None, gt_labels=None, allowed_border=0)[source]

Assign gt to anchors.

This method assign a gt bbox to every bbox (proposal/anchor), each bbox will be assigned with -1, 0, or a positive number. -1 means don’t care, 0 means negative sample, positive number is the index (1-based) of assigned gt.

The assignment is done in following steps, and the order matters.

  1. Assign every anchor to 0 (negative)

  2. (For each gt_bboxes) Compute ignore flags based on ignore_region then assign -1 to anchors w.r.t. ignore flags

  3. (For each gt_bboxes) Compute pos flags based on center_region then assign gt_bboxes to anchors w.r.t. pos flags

  4. (For each gt_bboxes) Compute ignore flags based on adjacent anchor level then assign -1 to anchors w.r.t. ignore flags

  5. Assign anchor outside of image to -1

Parameters
  • mlvl_anchors (list[Tensor]) – Multi level anchors.

  • mlvl_valid_flags (list[Tensor]) – Multi level valid flags.

  • gt_bboxes (Tensor) – Ground truth bboxes of image

  • img_meta (dict) – Meta info of image.

  • featmap_sizes (list[Tensor]) – Feature mapsize each level

  • anchor_scale (int) – Scale of the anchor.

  • anchor_strides (list[int]) – Stride of the anchor.

  • gt_bboxes – Groundtruth boxes, shape (k, 4).

  • gt_bboxes_ignore (Tensor, optional) – Ground truth bboxes that are labelled as ignored, e.g., crowd boxes in COCO.

  • gt_labels (Tensor, optional) – Label of gt_bboxes, shape (k, ).

  • allowed_border (int, optional) – The border to allow the valid anchor. Defaults to 0.

Returns

The assign result.

Return type

AssignResult

class mmdet.core.bbox.SamplingResult(pos_inds, neg_inds, bboxes, gt_bboxes, assign_result, gt_flags)[source]

Bbox sampling result.

Example

>>> # xdoctest: +IGNORE_WANT
>>> from mmdet.core.bbox.samplers.sampling_result import *  # NOQA
>>> self = SamplingResult.random(rng=10)
>>> print(f'self = {self}')
self = <SamplingResult({
    'neg_bboxes': torch.Size([12, 4]),
    'neg_inds': tensor([ 0,  1,  2,  4,  5,  6,  7,  8,  9, 10, 11, 12]),
    'num_gts': 4,
    'pos_assigned_gt_inds': tensor([], dtype=torch.int64),
    'pos_bboxes': torch.Size([0, 4]),
    'pos_inds': tensor([], dtype=torch.int64),
    'pos_is_gt': tensor([], dtype=torch.uint8)
})>
property bboxes

concatenated positive and negative boxes

Type

torch.Tensor

property info

Returns a dictionary of info about the object.

classmethod random(rng=None, **kwargs)[source]
Parameters
  • rng (None | int | numpy.random.RandomState) – seed or state.

  • kwargs (keyword arguments) –

    • num_preds: number of predicted boxes

    • num_gts: number of true boxes

    • p_ignore (float): probability of a predicted box assigned to an ignored truth.

    • p_assigned (float): probability of a predicted box not being assigned.

    • p_use_label (float | bool): with labels or not.

Returns

Randomly generated sampling result.

Return type

SamplingResult

Example

>>> from mmdet.core.bbox.samplers.sampling_result import *  # NOQA
>>> self = SamplingResult.random()
>>> print(self.__dict__)
to(device)[source]

Change the device of the data inplace.

Example

>>> self = SamplingResult.random()
>>> print(f'self = {self.to(None)}')
>>> # xdoctest: +REQUIRES(--gpu)
>>> print(f'self = {self.to(0)}')
class mmdet.core.bbox.ScoreHLRSampler(num, pos_fraction, context, neg_pos_ub=- 1, add_gt_as_proposals=True, k=0.5, bias=0, score_thr=0.05, iou_thr=0.5, **kwargs)[source]

Importance-based Sample Reweighting (ISR_N), described in Prime Sample Attention in Object Detection.

Score hierarchical local rank (HLR) differentiates with RandomSampler in negative part. It firstly computes Score-HLR in a two-step way, then linearly maps score hlr to the loss weights.

Parameters
  • num (int) – Total number of sampled RoIs.

  • pos_fraction (float) – Fraction of positive samples.

  • context (BaseRoIHead) – RoI head that the sampler belongs to.

  • neg_pos_ub (int) – Upper bound of the ratio of num negative to num positive, -1 means no upper bound.

  • add_gt_as_proposals (bool) – Whether to add ground truth as proposals.

  • k (float) – Power of the non-linear mapping.

  • bias (float) – Shift of the non-linear mapping.

  • score_thr (float) – Minimum score that a negative sample is to be considered as valid bbox.

static random_choice(gallery, num)[source]

Randomly select some elements from the gallery.

If gallery is a Tensor, the returned indices will be a Tensor; If gallery is a ndarray or list, the returned indices will be a ndarray.

Parameters
  • gallery (Tensor | ndarray | list) – indices pool.

  • num (int) – expected sample num.

Returns

sampled indices.

Return type

Tensor or ndarray

sample(assign_result, bboxes, gt_bboxes, gt_labels=None, img_meta=None, **kwargs)[source]

Sample positive and negative bboxes.

This is a simple implementation of bbox sampling given candidates, assigning results and ground truth bboxes.

Parameters
  • assign_result (AssignResult) – Bbox assigning results.

  • bboxes (Tensor) – Boxes to be sampled from.

  • gt_bboxes (Tensor) – Ground truth bboxes.

  • gt_labels (Tensor, optional) – Class labels of ground truth bboxes.

Returns

Sampling result and negative

label weights.

Return type

tuple[SamplingResult, Tensor]

class mmdet.core.bbox.TBLRBBoxCoder(normalizer=4.0, clip_border=True)[source]

TBLR BBox coder.

Following the practice in FSAF, this coder encodes gt bboxes (x1, y1, x2, y2) into (top, bottom, left, right) and decode it back to the original.

Parameters
  • normalizer (list | float) – Normalization factor to be divided with when coding the coordinates. If it is a list, it should have length of 4 indicating normalization factor in tblr dims. Otherwise it is a unified float factor for all dims. Default: 4.0

  • clip_border (bool, optional) – Whether clip the objects outside the border of the image. Defaults to True.

decode(bboxes, pred_bboxes, max_shape=None)[source]

Apply transformation pred_bboxes to boxes.

Parameters
  • bboxes (torch.Tensor) – Basic boxes.Shape (B, N, 4) or (N, 4)

  • pred_bboxes (torch.Tensor) – Encoded boxes with shape (B, N, 4) or (N, 4)

  • (Sequence[int] or torch.Tensor or Sequence[ (max_shape) – Sequence[int]],optional): Maximum bounds for boxes, specifies (H, W, C) or (H, W). If bboxes shape is (B, N, 4), then the max_shape should be a Sequence[Sequence[int]] and the length of max_shape should also be B.

Returns

Decoded boxes.

Return type

torch.Tensor

encode(bboxes, gt_bboxes)[source]

Get box regression transformation deltas that can be used to transform the bboxes into the gt_bboxes in the (top, left, bottom, right) order.

Parameters
  • bboxes (torch.Tensor) – source boxes, e.g., object proposals.

  • gt_bboxes (torch.Tensor) – target of the transformation, e.g., ground truth boxes.

Returns

Box transformation deltas

Return type

torch.Tensor

mmdet.core.bbox.bbox2distance(points, bbox, max_dis=None, eps=0.1)[source]

Decode bounding box based on distances.

Parameters
  • points (Tensor) – Shape (n, 2), [x, y].

  • bbox (Tensor) – Shape (n, 4), “xyxy” format

  • max_dis (float) – Upper bound of the distance.

  • eps (float) – a small value to ensure target < max_dis, instead <=

Returns

Decoded distances.

Return type

Tensor

mmdet.core.bbox.bbox2result(bboxes, labels, num_classes)[source]

Convert detection results to a list of numpy arrays.

Parameters
  • bboxes (torch.Tensor | np.ndarray) – shape (n, 5)

  • labels (torch.Tensor | np.ndarray) – shape (n, )

  • num_classes (int) – class number, including background class

Returns

bbox results of each class

Return type

list(ndarray)

mmdet.core.bbox.bbox2roi(bbox_list)[source]

Convert a list of bboxes to roi format.

Parameters

bbox_list (list[Tensor]) – a list of bboxes corresponding to a batch of images.

Returns

shape (n, 5), [batch_ind, x1, y1, x2, y2]

Return type

Tensor

mmdet.core.bbox.bbox_cxcywh_to_xyxy(bbox)[source]

Convert bbox coordinates from (cx, cy, w, h) to (x1, y1, x2, y2).

Parameters

bbox (Tensor) – Shape (n, 4) for bboxes.

Returns

Converted bboxes.

Return type

Tensor

mmdet.core.bbox.bbox_flip(bboxes, img_shape, direction='horizontal')[source]

Flip bboxes horizontally or vertically.

Parameters
  • bboxes (Tensor) – Shape (…, 4*k)

  • img_shape (tuple) – Image shape.

  • direction (str) – Flip direction, options are “horizontal”, “vertical”, “diagonal”. Default: “horizontal”

Returns

Flipped bboxes.

Return type

Tensor

mmdet.core.bbox.bbox_mapping(bboxes, img_shape, scale_factor, flip, flip_direction='horizontal')[source]

Map bboxes from the original image scale to testing scale.

mmdet.core.bbox.bbox_mapping_back(bboxes, img_shape, scale_factor, flip, flip_direction='horizontal')[source]

Map bboxes from testing scale to original image scale.

mmdet.core.bbox.bbox_overlaps(bboxes1, bboxes2, mode='iou', is_aligned=False, eps=1e-06)[source]

Calculate overlap between two set of bboxes.

FP16 Contributed by https://github.com/open-mmlab/mmdetection/pull/4889 .. note:

Assume bboxes1 is M x 4, bboxes2 is N x 4, when mode is 'iou',
there are some new generated variable when calculating IOU
using bbox_overlaps function:

1) is_aligned is False
    area1: M x 1
    area2: N x 1
    lt: M x N x 2
    rb: M x N x 2
    wh: M x N x 2
    overlap: M x N x 1
    union: M x N x 1
    ious: M x N x 1

    Total memory:
        S = (9 x N x M + N + M) * 4 Byte,

    When using FP16, we can reduce:
        R = (9 x N x M + N + M) * 4 / 2 Byte
        R large than (N + M) * 4 * 2 is always true when N and M >= 1.
        Obviously, N + M <= N * M < 3 * N * M, when N >=2 and M >=2,
                   N + 1 < 3 * N, when N or M is 1.

    Given M = 40 (ground truth), N = 400000 (three anchor boxes
    in per grid, FPN, R-CNNs),
        R = 275 MB (one times)

    A special case (dense detection), M = 512 (ground truth),
        R = 3516 MB = 3.43 GB

    When the batch size is B, reduce:
        B x R

    Therefore, CUDA memory runs out frequently.

    Experiments on GeForce RTX 2080Ti (11019 MiB):

    |   dtype   |   M   |   N   |   Use    |   Real   |   Ideal   |
    |:----:|:----:|:----:|:----:|:----:|:----:|
    |   FP32   |   512 | 400000 | 8020 MiB |   --   |   --   |
    |   FP16   |   512 | 400000 |   4504 MiB | 3516 MiB | 3516 MiB |
    |   FP32   |   40 | 400000 |   1540 MiB |   --   |   --   |
    |   FP16   |   40 | 400000 |   1264 MiB |   276MiB   | 275 MiB |

2) is_aligned is True
    area1: N x 1
    area2: N x 1
    lt: N x 2
    rb: N x 2
    wh: N x 2
    overlap: N x 1
    union: N x 1
    ious: N x 1

    Total memory:
        S = 11 x N * 4 Byte

    When using FP16, we can reduce:
        R = 11 x N * 4 / 2 Byte

So do the 'giou' (large than 'iou').

Time-wise, FP16 is generally faster than FP32.

When gpu_assign_thr is not -1, it takes more time on cpu
but not reduce memory.
There, we can reduce half the memory and keep the speed.

If is_aligned is False, then calculate the overlaps between each bbox of bboxes1 and bboxes2, otherwise the overlaps between each aligned pair of bboxes1 and bboxes2.

Parameters
  • bboxes1 (Tensor) – shape (B, m, 4) in <x1, y1, x2, y2> format or empty.

  • bboxes2 (Tensor) – shape (B, n, 4) in <x1, y1, x2, y2> format or empty. B indicates the batch dim, in shape (B1, B2, …, Bn). If is_aligned is True, then m and n must be equal.

  • mode (str) – “iou” (intersection over union), “iof” (intersection over foreground) or “giou” (generalized intersection over union). Default “iou”.

  • is_aligned (bool, optional) – If True, then m and n must be equal. Default False.

  • eps (float, optional) – A value added to the denominator for numerical stability. Default 1e-6.

Returns

shape (m, n) if is_aligned is False else shape (m,)

Return type

Tensor

Example

>>> bboxes1 = torch.FloatTensor([
>>>     [0, 0, 10, 10],
>>>     [10, 10, 20, 20],
>>>     [32, 32, 38, 42],
>>> ])
>>> bboxes2 = torch.FloatTensor([
>>>     [0, 0, 10, 20],
>>>     [0, 10, 10, 19],
>>>     [10, 10, 20, 20],
>>> ])
>>> overlaps = bbox_overlaps(bboxes1, bboxes2)
>>> assert overlaps.shape == (3, 3)
>>> overlaps = bbox_overlaps(bboxes1, bboxes2, is_aligned=True)
>>> assert overlaps.shape == (3, )

Example

>>> empty = torch.empty(0, 4)
>>> nonempty = torch.FloatTensor([[0, 0, 10, 9]])
>>> assert tuple(bbox_overlaps(empty, nonempty).shape) == (0, 1)
>>> assert tuple(bbox_overlaps(nonempty, empty).shape) == (1, 0)
>>> assert tuple(bbox_overlaps(empty, empty).shape) == (0, 0)
mmdet.core.bbox.bbox_rescale(bboxes, scale_factor=1.0)[source]

Rescale bounding box w.r.t. scale_factor.

Parameters
  • bboxes (Tensor) – Shape (n, 4) for bboxes or (n, 5) for rois

  • scale_factor (float) – rescale factor

Returns

Rescaled bboxes.

Return type

Tensor

mmdet.core.bbox.bbox_xyxy_to_cxcywh(bbox)[source]

Convert bbox coordinates from (x1, y1, x2, y2) to (cx, cy, w, h).

Parameters

bbox (Tensor) – Shape (n, 4) for bboxes.

Returns

Converted bboxes.

Return type

Tensor

mmdet.core.bbox.build_assigner(cfg, **default_args)[source]

Builder of box assigner.

mmdet.core.bbox.build_bbox_coder(cfg, **default_args)[source]

Builder of box coder.

mmdet.core.bbox.build_sampler(cfg, **default_args)[source]

Builder of box sampler.

mmdet.core.bbox.distance2bbox(points, distance, max_shape=None)[source]

Decode distance prediction to bounding box.

Parameters
  • points (Tensor) – Shape (B, N, 2) or (N, 2).

  • distance (Tensor) – Distance from the given point to 4 boundaries (left, top, right, bottom). Shape (B, N, 4) or (N, 4)

  • (Sequence[int] or torch.Tensor or Sequence[ (max_shape) – Sequence[int]],optional): Maximum bounds for boxes, specifies (H, W, C) or (H, W). If priors shape is (B, N, 4), then the max_shape should be a Sequence[Sequence[int]] and the length of max_shape should also be B.

Returns

Boxes with shape (N, 4) or (B, N, 4)

Return type

Tensor

mmdet.core.bbox.roi2bbox(rois)[source]

Convert rois to bounding box format.

Parameters

rois (torch.Tensor) – RoIs with the shape (n, 5) where the first column indicates batch id of each RoI.

Returns

Converted boxes of corresponding rois.

Return type

list[torch.Tensor]

export

mmdet.core.export.add_dummy_nms_for_onnx(boxes, scores, max_output_boxes_per_class=1000, iou_threshold=0.5, score_threshold=0.05, pre_top_k=- 1, after_top_k=- 1, labels=None)[source]

Create a dummy onnx::NonMaxSuppression op while exporting to ONNX.

This function helps exporting to onnx with batch and multiclass NMS op. It only supports class-agnostic detection results. That is, the scores is of shape (N, num_bboxes, num_classes) and the boxes is of shape (N, num_boxes, 4).

Parameters
  • boxes (Tensor) – The bounding boxes of shape [N, num_boxes, 4]

  • scores (Tensor) – The detection scores of shape [N, num_boxes, num_classes]

  • max_output_boxes_per_class (int) – Maximum number of output boxes per class of nms. Defaults to 1000.

  • iou_threshold (float) – IOU threshold of nms. Defaults to 0.5

  • score_threshold (float) – score threshold of nms. Defaults to 0.05.

  • pre_top_k (bool) – Number of top K boxes to keep before nms. Defaults to -1.

  • after_top_k (int) – Number of top K boxes to keep after nms. Defaults to -1.

  • labels (Tensor, optional) – It not None, explicit labels would be used. Otherwise, labels would be automatically generated using num_classed. Defaults to None.

Returns

dets of shape [N, num_det, 5] and class labels

of shape [N, num_det].

Return type

tuple[Tensor, Tensor]

mmdet.core.export.build_model_from_cfg(config_path, checkpoint_path, cfg_options=None)[source]

Build a model from config and load the given checkpoint.

Parameters
  • config_path (str) – the OpenMMLab config for the model we want to export to ONNX

  • checkpoint_path (str) – Path to the corresponding checkpoint

Returns

the built model

Return type

torch.nn.Module

mmdet.core.export.dynamic_clip_for_onnx(x1, y1, x2, y2, max_shape)[source]

Clip boxes dynamically for onnx.

Since torch.clamp cannot have dynamic min and max, we scale the

boxes by 1/max_shape and clamp in the range [0, 1].

Parameters
  • x1 (Tensor) – The x1 for bounding boxes.

  • y1 (Tensor) – The y1 for bounding boxes.

  • x2 (Tensor) – The x2 for bounding boxes.

  • y2 (Tensor) – The y2 for bounding boxes.

  • max_shape (Tensor or torch.Size) – The (H,W) of original image.

Returns

The clipped x1, y1, x2, y2.

Return type

tuple(Tensor)

mmdet.core.export.generate_inputs_and_wrap_model(config_path, checkpoint_path, input_config, cfg_options=None)[source]

Prepare sample input and wrap model for ONNX export.

The ONNX export API only accept args, and all inputs should be torch.Tensor or corresponding types (such as tuple of tensor). So we should call this function before exporting. This function will:

  1. generate corresponding inputs which are used to execute the model.

  2. Wrap the model’s forward function.

For example, the MMDet models’ forward function has a parameter return_loss:bool. As we want to set it as False while export API supports neither bool type or kwargs. So we have to replace the forward method like model.forward = partial(model.forward, return_loss=False).

Parameters
  • config_path (str) – the OpenMMLab config for the model we want to export to ONNX

  • checkpoint_path (str) – Path to the corresponding checkpoint

  • input_config (dict) – the exactly data in this dict depends on the framework. For MMSeg, we can just declare the input shape, and generate the dummy data accordingly. However, for MMDet, we may pass the real img path, or the NMS will return None as there is no legal bbox.

Returns

(model, tensor_data) wrapped model which can be called by

model(*tensor_data) and a list of inputs which are used to execute the model while exporting.

Return type

tuple

mmdet.core.export.get_k_for_topk(k, size)[source]

Get k of TopK for onnx exporting.

The K of TopK in TensorRT should not be a Tensor, while in ONNX Runtime

it could be a Tensor.Due to dynamic shape feature, we have to decide whether to do TopK and what K it should be while exporting to ONNX.

If returned K is less than zero, it means we do not have to do

TopK operation.

Parameters
  • k (int or Tensor) – The set k value for nms from config file.

  • size (Tensor or torch.Size) – The number of elements of TopK’s input tensor

Returns

(int or Tensor): The final K for TopK.

Return type

tuple

mmdet.core.export.preprocess_example_input(input_config)[source]

Prepare an example input image for generate_inputs_and_wrap_model.

Parameters

input_config (dict) – customized config describing the example input.

Returns

(one_img, one_meta), tensor of the example input image and meta information for the example input image.

Return type

tuple

Examples

>>> from mmdet.core.export import preprocess_example_input
>>> input_config = {
>>>         'input_shape': (1,3,224,224),
>>>         'input_path': 'demo/demo.jpg',
>>>         'normalize_cfg': {
>>>             'mean': (123.675, 116.28, 103.53),
>>>             'std': (58.395, 57.12, 57.375)
>>>             }
>>>         }
>>> one_img, one_meta = preprocess_example_input(input_config)
>>> print(one_img.shape)
torch.Size([1, 3, 224, 224])
>>> print(one_meta)
{'img_shape': (224, 224, 3),
'ori_shape': (224, 224, 3),
'pad_shape': (224, 224, 3),
'filename': '<demo>.png',
'scale_factor': 1.0,
'flip': False}

mask

class mmdet.core.mask.BaseInstanceMasks[source]

Base class for instance masks.

abstract property areas

areas of each instance.

Type

ndarray

abstract crop(bbox)[source]

Crop each mask by the given bbox.

Parameters

bbox (ndarray) – Bbox in format [x1, y1, x2, y2], shape (4, ).

Returns

The cropped masks.

Return type

BaseInstanceMasks

abstract crop_and_resize(bboxes, out_shape, inds, device, interpolation='bilinear', binarize=True)[source]

Crop and resize masks by the given bboxes.

This function is mainly used in mask targets computation. It firstly align mask to bboxes by assigned_inds, then crop mask by the assigned bbox and resize to the size of (mask_h, mask_w)

Parameters
  • bboxes (Tensor) – Bboxes in format [x1, y1, x2, y2], shape (N, 4)

  • out_shape (tuple[int]) – Target (h, w) of resized mask

  • inds (ndarray) – Indexes to assign masks to each bbox, shape (N,) and values should be between [0, num_masks - 1].

  • device (str) – Device of bboxes

  • interpolation (str) – See mmcv.imresize

  • binarize (bool) – if True fractional values are rounded to 0 or 1 after the resize operation. if False and unsupported an error will be raised. Defaults to True.

Returns

the cropped and resized masks.

Return type

BaseInstanceMasks

abstract expand(expanded_h, expanded_w, top, left)[source]

see Expand.

abstract flip(flip_direction='horizontal')[source]

Flip masks alone the given direction.

Parameters

flip_direction (str) – Either ‘horizontal’ or ‘vertical’.

Returns

The flipped masks.

Return type

BaseInstanceMasks

abstract pad(out_shape, pad_val)[source]

Pad masks to the given size of (h, w).

Parameters
  • out_shape (tuple[int]) – Target (h, w) of padded mask.

  • pad_val (int) – The padded value.

Returns

The padded masks.

Return type

BaseInstanceMasks

abstract rescale(scale, interpolation='nearest')[source]

Rescale masks as large as possible while keeping the aspect ratio. For details can refer to mmcv.imrescale.

Parameters
  • scale (tuple[int]) – The maximum size (h, w) of rescaled mask.

  • interpolation (str) – Same as mmcv.imrescale().

Returns

The rescaled masks.

Return type

BaseInstanceMasks

abstract resize(out_shape, interpolation='nearest')[source]

Resize masks to the given out_shape.

Parameters
  • out_shape – Target (h, w) of resized mask.

  • interpolation (str) – See mmcv.imresize().

Returns

The resized masks.

Return type

BaseInstanceMasks

abstract rotate(out_shape, angle, center=None, scale=1.0, fill_val=0)[source]

Rotate the masks.

Parameters
  • out_shape (tuple[int]) – Shape for output mask, format (h, w).

  • angle (int | float) – Rotation angle in degrees. Positive values mean counter-clockwise rotation.

  • center (tuple[float], optional) – Center point (w, h) of the rotation in source image. If not specified, the center of the image will be used.

  • scale (int | float) – Isotropic scale factor.

  • fill_val (int | float) – Border value. Default 0 for masks.

Returns

Rotated masks.

shear(out_shape, magnitude, direction='horizontal', border_value=0, interpolation='bilinear')[source]

Shear the masks.

Parameters
  • out_shape (tuple[int]) – Shape for output mask, format (h, w).

  • magnitude (int | float) – The magnitude used for shear.

  • direction (str) – The shear direction, either “horizontal” or “vertical”.

  • border_value (int | tuple[int]) – Value used in case of a constant border. Default 0.

  • interpolation (str) – Same as in mmcv.imshear().

Returns

Sheared masks.

Return type

ndarray

abstract to_ndarray()[source]

Convert masks to the format of ndarray.

Returns

Converted masks in the format of ndarray.

Return type

ndarray

abstract to_tensor(dtype, device)[source]

Convert masks to the format of Tensor.

Parameters
  • dtype (str) – Dtype of converted mask.

  • device (torch.device) – Device of converted masks.

Returns

Converted masks in the format of Tensor.

Return type

Tensor

abstract translate(out_shape, offset, direction='horizontal', fill_val=0, interpolation='bilinear')[source]

Translate the masks.

Parameters
  • out_shape (tuple[int]) – Shape for output mask, format (h, w).

  • offset (int | float) – The offset for translate.

  • direction (str) – The translate direction, either “horizontal” or “vertical”.

  • fill_val (int | float) – Border value. Default 0.

  • interpolation (str) – Same as mmcv.imtranslate().

Returns

Translated masks.

class mmdet.core.mask.BitmapMasks(masks, height, width)[source]

This class represents masks in the form of bitmaps.

Parameters
  • masks (ndarray) – ndarray of masks in shape (N, H, W), where N is the number of objects.

  • height (int) – height of masks

  • width (int) – width of masks

Example

>>> from mmdet.core.mask.structures import *  # NOQA
>>> num_masks, H, W = 3, 32, 32
>>> rng = np.random.RandomState(0)
>>> masks = (rng.rand(num_masks, H, W) > 0.1).astype(np.int)
>>> self = BitmapMasks(masks, height=H, width=W)
>>> # demo crop_and_resize
>>> num_boxes = 5
>>> bboxes = np.array([[0, 0, 30, 10.0]] * num_boxes)
>>> out_shape = (14, 14)
>>> inds = torch.randint(0, len(self), size=(num_boxes,))
>>> device = 'cpu'
>>> interpolation = 'bilinear'
>>> new = self.crop_and_resize(
...     bboxes, out_shape, inds, device, interpolation)
>>> assert len(new) == num_boxes
>>> assert new.height, new.width == out_shape
property areas

See BaseInstanceMasks.areas.

crop(bbox)[source]

See BaseInstanceMasks.crop().

crop_and_resize(bboxes, out_shape, inds, device='cpu', interpolation='bilinear', binarize=True)[source]

See BaseInstanceMasks.crop_and_resize().

expand(expanded_h, expanded_w, top, left)[source]

See BaseInstanceMasks.expand().

flip(flip_direction='horizontal')[source]

See BaseInstanceMasks.flip().

pad(out_shape, pad_val=0)[source]

See BaseInstanceMasks.pad().

classmethod random(num_masks=3, height=32, width=32, dtype=<class 'numpy.uint8'>, rng=None)[source]

Generate random bitmap masks for demo / testing purposes.

Example

>>> from mmdet.core.mask.structures import BitmapMasks
>>> self = BitmapMasks.random()
>>> print('self = {}'.format(self))
self = BitmapMasks(num_masks=3, height=32, width=32)
rescale(scale, interpolation='nearest')[source]

See BaseInstanceMasks.rescale().

resize(out_shape, interpolation='nearest')[source]

See BaseInstanceMasks.resize().

rotate(out_shape, angle, center=None, scale=1.0, fill_val=0)[source]

Rotate the BitmapMasks.

Parameters
  • out_shape (tuple[int]) – Shape for output mask, format (h, w).

  • angle (int | float) – Rotation angle in degrees. Positive values mean counter-clockwise rotation.

  • center (tuple[float], optional) – Center point (w, h) of the rotation in source image. If not specified, the center of the image will be used.

  • scale (int | float) – Isotropic scale factor.

  • fill_val (int | float) – Border value. Default 0 for masks.

Returns

Rotated BitmapMasks.

Return type

BitmapMasks

shear(out_shape, magnitude, direction='horizontal', border_value=0, interpolation='bilinear')[source]

Shear the BitmapMasks.

Parameters
  • out_shape (tuple[int]) – Shape for output mask, format (h, w).

  • magnitude (int | float) – The magnitude used for shear.

  • direction (str) – The shear direction, either “horizontal” or “vertical”.

  • border_value (int | tuple[int]) – Value used in case of a constant border.

  • interpolation (str) – Same as in mmcv.imshear().

Returns

The sheared masks.

Return type

BitmapMasks

to_ndarray()[source]

See BaseInstanceMasks.to_ndarray().

to_tensor(dtype, device)[source]

See BaseInstanceMasks.to_tensor().

translate(out_shape, offset, direction='horizontal', fill_val=0, interpolation='bilinear')[source]

Translate the BitmapMasks.

Parameters
  • out_shape (tuple[int]) – Shape for output mask, format (h, w).

  • offset (int | float) – The offset for translate.

  • direction (str) – The translate direction, either “horizontal” or “vertical”.

  • fill_val (int | float) – Border value. Default 0 for masks.

  • interpolation (str) – Same as mmcv.imtranslate().

Returns

Translated BitmapMasks.

Return type

BitmapMasks

Example

>>> from mmdet.core.mask.structures import BitmapMasks
>>> self = BitmapMasks.random(dtype=np.uint8)
>>> out_shape = (32, 32)
>>> offset = 4
>>> direction = 'horizontal'
>>> fill_val = 0
>>> interpolation = 'bilinear'
>>> # Note, There seem to be issues when:
>>> # * out_shape is different than self's shape
>>> # * the mask dtype is not supported by cv2.AffineWarp
>>> new = self.translate(out_shape, offset, direction, fill_val,
>>>                      interpolation)
>>> assert len(new) == len(self)
>>> assert new.height, new.width == out_shape
class mmdet.core.mask.PolygonMasks(masks, height, width)[source]

This class represents masks in the form of polygons.

Polygons is a list of three levels. The first level of the list corresponds to objects, the second level to the polys that compose the object, the third level to the poly coordinates

Parameters
  • masks (list[list[ndarray]]) – The first level of the list corresponds to objects, the second level to the polys that compose the object, the third level to the poly coordinates

  • height (int) – height of masks

  • width (int) – width of masks

Example

>>> from mmdet.core.mask.structures import *  # NOQA
>>> masks = [
>>>     [ np.array([0, 0, 10, 0, 10, 10., 0, 10, 0, 0]) ]
>>> ]
>>> height, width = 16, 16
>>> self = PolygonMasks(masks, height, width)
>>> # demo translate
>>> new = self.translate((16, 16), 4., direction='horizontal')
>>> assert np.all(new.masks[0][0][1::2] == masks[0][0][1::2])
>>> assert np.all(new.masks[0][0][0::2] == masks[0][0][0::2] + 4)
>>> # demo crop_and_resize
>>> num_boxes = 3
>>> bboxes = np.array([[0, 0, 30, 10.0]] * num_boxes)
>>> out_shape = (16, 16)
>>> inds = torch.randint(0, len(self), size=(num_boxes,))
>>> device = 'cpu'
>>> interpolation = 'bilinear'
>>> new = self.crop_and_resize(
...     bboxes, out_shape, inds, device, interpolation)
>>> assert len(new) == num_boxes
>>> assert new.height, new.width == out_shape
property areas

Compute areas of masks.

This func is modified from detectron2. The function only works with Polygons using the shoelace formula.

Returns

areas of each instance

Return type

ndarray

crop(bbox)[source]

see BaseInstanceMasks.crop()

crop_and_resize(bboxes, out_shape, inds, device='cpu', interpolation='bilinear', binarize=True)[source]

see BaseInstanceMasks.crop_and_resize()

expand(*args, **kwargs)[source]

TODO: Add expand for polygon

flip(flip_direction='horizontal')[source]

see BaseInstanceMasks.flip()

pad(out_shape, pad_val=0)[source]

padding has no effect on polygons`

classmethod random(num_masks=3, height=32, width=32, n_verts=5, dtype=<class 'numpy.float32'>, rng=None)[source]

Generate random polygon masks for demo / testing purposes.

Adapted from 1

References

1(1,2)

https://gitlab.kitware.com/computer-vision/kwimage/-/blob/928cae35ca8/kwimage/structs/polygon.py#L379 # noqa: E501

Example

>>> from mmdet.core.mask.structures import PolygonMasks
>>> self = PolygonMasks.random()
>>> print('self = {}'.format(self))
rescale(scale, interpolation=None)[source]

see BaseInstanceMasks.rescale()

resize(out_shape, interpolation=None)[source]

see BaseInstanceMasks.resize()

rotate(out_shape, angle, center=None, scale=1.0, fill_val=0)[source]

See BaseInstanceMasks.rotate().

shear(out_shape, magnitude, direction='horizontal', border_value=0, interpolation='bilinear')[source]

See BaseInstanceMasks.shear().

to_bitmap()[source]

convert polygon masks to bitmap masks.

to_ndarray()[source]

Convert masks to the format of ndarray.

to_tensor(dtype, device)[source]

See BaseInstanceMasks.to_tensor().

translate(out_shape, offset, direction='horizontal', fill_val=None, interpolation=None)[source]

Translate the PolygonMasks.

Example

>>> self = PolygonMasks.random(dtype=np.int)
>>> out_shape = (self.height, self.width)
>>> new = self.translate(out_shape, 4., direction='horizontal')
>>> assert np.all(new.masks[0][0][1::2] == self.masks[0][0][1::2])
>>> assert np.all(new.masks[0][0][0::2] == self.masks[0][0][0::2] + 4)  # noqa: E501
mmdet.core.mask.encode_mask_results(mask_results)[source]

Encode bitmap mask to RLE code.

Parameters

mask_results (list | tuple[list]) – bitmap mask results. In mask scoring rcnn, mask_results is a tuple of (segm_results, segm_cls_score).

Returns

RLE encoded mask.

Return type

list | tuple

mmdet.core.mask.mask_target(pos_proposals_list, pos_assigned_gt_inds_list, gt_masks_list, cfg)[source]

Compute mask target for positive proposals in multiple images.

Parameters
  • pos_proposals_list (list[Tensor]) – Positive proposals in multiple images.

  • pos_assigned_gt_inds_list (list[Tensor]) – Assigned GT indices for each positive proposals.

  • gt_masks_list (list[BaseInstanceMasks]) – Ground truth masks of each image.

  • cfg (dict) – Config dict that specifies the mask size.

Returns

Mask target of each image.

Return type

list[Tensor]

Example

>>> import mmcv
>>> import mmdet
>>> from mmdet.core.mask import BitmapMasks
>>> from mmdet.core.mask.mask_target import *
>>> H, W = 17, 18
>>> cfg = mmcv.Config({'mask_size': (13, 14)})
>>> rng = np.random.RandomState(0)
>>> # Positive proposals (tl_x, tl_y, br_x, br_y) for each image
>>> pos_proposals_list = [
>>>     torch.Tensor([
>>>         [ 7.2425,  5.5929, 13.9414, 14.9541],
>>>         [ 7.3241,  3.6170, 16.3850, 15.3102],
>>>     ]),
>>>     torch.Tensor([
>>>         [ 4.8448, 6.4010, 7.0314, 9.7681],
>>>         [ 5.9790, 2.6989, 7.4416, 4.8580],
>>>         [ 0.0000, 0.0000, 0.1398, 9.8232],
>>>     ]),
>>> ]
>>> # Corresponding class index for each proposal for each image
>>> pos_assigned_gt_inds_list = [
>>>     torch.LongTensor([7, 0]),
>>>     torch.LongTensor([5, 4, 1]),
>>> ]
>>> # Ground truth mask for each true object for each image
>>> gt_masks_list = [
>>>     BitmapMasks(rng.rand(8, H, W), height=H, width=W),
>>>     BitmapMasks(rng.rand(6, H, W), height=H, width=W),
>>> ]
>>> mask_targets = mask_target(
>>>     pos_proposals_list, pos_assigned_gt_inds_list,
>>>     gt_masks_list, cfg)
>>> assert mask_targets.shape == (5,) + cfg['mask_size']
mmdet.core.mask.split_combined_polys(polys, poly_lens, polys_per_mask)[source]

Split the combined 1-D polys into masks.

A mask is represented as a list of polys, and a poly is represented as a 1-D array. In dataset, all masks are concatenated into a single 1-D tensor. Here we need to split the tensor into original representations.

Parameters
  • polys (list) – a list (length = image num) of 1-D tensors

  • poly_lens (list) – a list (length = image num) of poly length

  • polys_per_mask (list) – a list (length = image num) of poly number of each mask

Returns

a list (length = image num) of list (length = mask num) of list (length = poly num) of numpy array.

Return type

list

evaluation

class mmdet.core.evaluation.DistEvalHook(dataloader, start=None, interval=1, by_epoch=True, save_best=None, rule=None, test_fn=None, greater_keys=None, less_keys=None, broadcast_bn_buffer=True, tmpdir=None, gpu_collect=False, **eval_kwargs)[source]
class mmdet.core.evaluation.EvalHook(dataloader, start=None, interval=1, by_epoch=True, save_best=None, rule=None, test_fn=None, greater_keys=None, less_keys=None, **eval_kwargs)[source]
mmdet.core.evaluation.average_precision(recalls, precisions, mode='area')[source]

Calculate average precision (for single or multiple scales).

Parameters
  • recalls (ndarray) – shape (num_scales, num_dets) or (num_dets, )

  • precisions (ndarray) – shape (num_scales, num_dets) or (num_dets, )

  • mode (str) – ‘area’ or ‘11points’, ‘area’ means calculating the area under precision-recall curve, ‘11points’ means calculating the average precision of recalls at [0, 0.1, …, 1]

Returns

calculated average precision

Return type

float or ndarray

mmdet.core.evaluation.eval_map(det_results, annotations, scale_ranges=None, iou_thr=0.5, dataset=None, logger=None, tpfp_fn=None, nproc=4, use_legacy_coordinate=False)[source]

Evaluate mAP of a dataset.

Parameters
  • det_results (list[list]) – [[cls1_det, cls2_det, …], …]. The outer list indicates images, and the inner list indicates per-class detected bboxes.

  • annotations (list[dict]) –

    Ground truth annotations where each item of the list indicates an image. Keys of annotations are:

    • bboxes: numpy array of shape (n, 4)

    • labels: numpy array of shape (n, )

    • bboxes_ignore (optional): numpy array of shape (k, 4)

    • labels_ignore (optional): numpy array of shape (k, )

  • scale_ranges (list[tuple] | None) – Range of scales to be evaluated, in the format [(min1, max1), (min2, max2), …]. A range of (32, 64) means the area range between (32**2, 64**2). Default: None.

  • iou_thr (float) – IoU threshold to be considered as matched. Default: 0.5.

  • dataset (list[str] | str | None) – Dataset name or dataset classes, there are minor differences in metrics for different datasets, e.g. “voc07”, “imagenet_det”, etc. Default: None.

  • logger (logging.Logger | str | None) – The way to print the mAP summary. See mmcv.utils.print_log() for details. Default: None.

  • tpfp_fn (callable | None) – The function used to determine true/ false positives. If None, tpfp_default() is used as default unless dataset is ‘det’ or ‘vid’ (tpfp_imagenet() in this case). If it is given as a function, then this function is used to evaluate tp & fp. Default None.

  • nproc (int) – Processes used for computing TP and FP. Default: 4.

  • use_legacy_coordinate (bool) – Whether to use coordinate system in mmdet v1.x. which means width, height should be calculated as ‘x2 - x1 + 1` and ‘y2 - y1 + 1’ respectively. Default: False.

Returns

(mAP, [dict, dict, …])

Return type

tuple

mmdet.core.evaluation.eval_recalls(gts, proposals, proposal_nums=None, iou_thrs=0.5, logger=None, use_legacy_coordinate=False)[source]

Calculate recalls.

Parameters
  • gts (list[ndarray]) – a list of arrays of shape (n, 4)

  • proposals (list[ndarray]) – a list of arrays of shape (k, 4) or (k, 5)

  • proposal_nums (int | Sequence[int]) – Top N proposals to be evaluated.

  • iou_thrs (float | Sequence[float]) – IoU thresholds. Default: 0.5.

  • logger (logging.Logger | str | None) – The way to print the recall summary. See mmcv.utils.print_log() for details. Default: None.

  • use_legacy_coordinate (bool) – Whether use coordinate system in mmdet v1.x. “1” was added to both height and width which means w, h should be computed as ‘x2 - x1 + 1` and ‘y2 - y1 + 1’. Default: False.

Returns

recalls of different ious and proposal nums

Return type

ndarray

mmdet.core.evaluation.get_classes(dataset)[source]

Get class names of a dataset.

mmdet.core.evaluation.plot_iou_recall(recalls, iou_thrs)[source]

Plot IoU-Recalls curve.

Parameters
  • recalls (ndarray or list) – shape (k,)

  • iou_thrs (ndarray or list) – same shape as recalls

mmdet.core.evaluation.plot_num_recall(recalls, proposal_nums)[source]

Plot Proposal_num-Recalls curve.

Parameters
  • recalls (ndarray or list) – shape (k,)

  • proposal_nums (ndarray or list) – same shape as recalls

mmdet.core.evaluation.print_map_summary(mean_ap, results, dataset=None, scale_ranges=None, logger=None)[source]

Print mAP and results of each class.

A table will be printed to show the gts/dets/recall/AP of each class and the mAP.

Parameters
  • mean_ap (float) – Calculated from eval_map().

  • results (list[dict]) – Calculated from eval_map().

  • dataset (list[str] | str | None) – Dataset name or dataset classes.

  • scale_ranges (list[tuple] | None) – Range of scales to be evaluated.

  • logger (logging.Logger | str | None) – The way to print the mAP summary. See mmcv.utils.print_log() for details. Default: None.

mmdet.core.evaluation.print_recall_summary(recalls, proposal_nums, iou_thrs, row_idxs=None, col_idxs=None, logger=None)[source]

Print recalls in a table.

Parameters
  • recalls (ndarray) – calculated from bbox_recalls

  • proposal_nums (ndarray or list) – top N proposals

  • iou_thrs (ndarray or list) – iou thresholds

  • row_idxs (ndarray) – which rows(proposal nums) to print

  • col_idxs (ndarray) – which cols(iou thresholds) to print

  • logger (logging.Logger | str | None) – The way to print the recall summary. See mmcv.utils.print_log() for details. Default: None.

post_processing

mmdet.core.post_processing.fast_nms(multi_bboxes, multi_scores, multi_coeffs, score_thr, iou_thr, top_k, max_num=- 1)[source]

Fast NMS in YOLACT.

Fast NMS allows already-removed detections to suppress other detections so that every instance can be decided to be kept or discarded in parallel, which is not possible in traditional NMS. This relaxation allows us to implement Fast NMS entirely in standard GPU-accelerated matrix operations.

Parameters
  • multi_bboxes (Tensor) – shape (n, #class*4) or (n, 4)

  • multi_scores (Tensor) – shape (n, #class+1), where the last column contains scores of the background class, but this will be ignored.

  • multi_coeffs (Tensor) – shape (n, #class*coeffs_dim).

  • score_thr (float) – bbox threshold, bboxes with scores lower than it will not be considered.

  • iou_thr (float) – IoU threshold to be considered as conflicted.

  • top_k (int) – if there are more than top_k bboxes before NMS, only top top_k will be kept.

  • max_num (int) – if there are more than max_num bboxes after NMS, only top max_num will be kept. If -1, keep all the bboxes. Default: -1.

Returns

(dets, labels, coefficients), tensors of shape (k, 5), (k, 1),

and (k, coeffs_dim). Dets are boxes with scores. Labels are 0-based.

Return type

tuple

mmdet.core.post_processing.mask_matrix_nms(masks, labels, scores, filter_thr=- 1, nms_pre=- 1, max_num=- 1, kernel='gaussian', sigma=2.0, mask_area=None)[source]

Matrix NMS for multi-class masks.

Parameters
  • masks (Tensor) – Has shape (num_instances, h, w)

  • labels (Tensor) – Labels of corresponding masks, has shape (num_instances,).

  • scores (Tensor) – Mask scores of corresponding masks, has shape (num_instances).

  • filter_thr (float) – Score threshold to filter the masks after matrix nms. Default: -1, which means do not use filter_thr.

  • nms_pre (int) – The max number of instances to do the matrix nms. Default: -1, which means do not use nms_pre.

  • max_num (int, optional) – If there are more than max_num masks after matrix, only top max_num will be kept. Default: -1, which means do not use max_num.

  • kernel (str) – ‘linear’ or ‘gaussian’.

  • sigma (float) – std in gaussian method.

  • mask_area (Tensor) – The sum of seg_masks.

Returns

Processed mask results.

  • scores (Tensor): Updated scores, has shape (n,).

  • labels (Tensor): Remained labels, has shape (n,).

  • masks (Tensor): Remained masks, has shape (n, w, h).

  • keep_inds (Tensor): The indices number of

    the remaining mask in the input mask, has shape (n,).

Return type

tuple(Tensor)

mmdet.core.post_processing.merge_aug_bboxes(aug_bboxes, aug_scores, img_metas, rcnn_test_cfg)[source]

Merge augmented detection bboxes and scores.

Parameters
  • aug_bboxes (list[Tensor]) – shape (n, 4*#class)

  • aug_scores (list[Tensor] or None) – shape (n, #class)

  • img_shapes (list[Tensor]) – shape (3, ).

  • rcnn_test_cfg (dict) – rcnn test config.

Returns

(bboxes, scores)

Return type

tuple

mmdet.core.post_processing.merge_aug_masks(aug_masks, img_metas, rcnn_test_cfg, weights=None)[source]

Merge augmented mask prediction.

Parameters
  • aug_masks (list[ndarray]) – shape (n, #class, h, w)

  • img_shapes (list[ndarray]) – shape (3, ).

  • rcnn_test_cfg (dict) – rcnn test config.

Returns

(bboxes, scores)

Return type

tuple

mmdet.core.post_processing.merge_aug_proposals(aug_proposals, img_metas, cfg)[source]

Merge augmented proposals (multiscale, flip, etc.)

Parameters
  • aug_proposals (list[Tensor]) – proposals from different testing schemes, shape (n, 5). Note that they are not rescaled to the original image size.

  • img_metas (list[dict]) – list of image info dict where each dict has: ‘img_shape’, ‘scale_factor’, ‘flip’, and may also contain ‘filename’, ‘ori_shape’, ‘pad_shape’, and ‘img_norm_cfg’. For details on the values of these keys see mmdet/datasets/pipelines/formatting.py:Collect.

  • cfg (dict) – rpn test config.

Returns

shape (n, 4), proposals corresponding to original image scale.

Return type

Tensor

mmdet.core.post_processing.merge_aug_scores(aug_scores)[source]

Merge augmented bbox scores.

mmdet.core.post_processing.multiclass_nms(multi_bboxes, multi_scores, score_thr, nms_cfg, max_num=- 1, score_factors=None, return_inds=False)[source]

NMS for multi-class bboxes.

Parameters
  • multi_bboxes (Tensor) – shape (n, #class*4) or (n, 4)

  • multi_scores (Tensor) – shape (n, #class), where the last column contains scores of the background class, but this will be ignored.

  • score_thr (float) – bbox threshold, bboxes with scores lower than it will not be considered.

  • nms_thr (float) – NMS IoU threshold

  • max_num (int, optional) – if there are more than max_num bboxes after NMS, only top max_num will be kept. Default to -1.

  • score_factors (Tensor, optional) – The factors multiplied to scores before applying NMS. Default to None.

  • return_inds (bool, optional) – Whether return the indices of kept bboxes. Default to False.

Returns

(dets, labels, indices (optional)), tensors of shape (k, 5),

(k), and (k). Dets are boxes with scores. Labels are 0-based.

Return type

tuple

utils

class mmdet.core.utils.DistOptimizerHook(*args, **kwargs)[source]

Deprecated optimizer hook for distributed training.

mmdet.core.utils.all_reduce_dict(py_dict, op='sum', group=None, to_float=True)[source]

Apply all reduce function for python dict object.

The code is modified from https://github.com/Megvii- BaseDetection/YOLOX/blob/main/yolox/utils/allreduce_norm.py.

NOTE: make sure that py_dict in different ranks has the same keys and the values should be in the same shape.

Parameters
  • py_dict (dict) – Dict to be applied all reduce op.

  • op (str) – Operator, could be ‘sum’ or ‘mean’. Default: ‘sum’

  • group (torch.distributed.group, optional) – Distributed group, Default: None.

  • to_float (bool) – Whether to convert all values of dict to float. Default: True.

Returns

reduced python dict object.

Return type

OrderedDict

mmdet.core.utils.allreduce_grads(params, coalesce=True, bucket_size_mb=- 1)[source]

Allreduce gradients.

Parameters
  • params (list[torch.Parameters]) – List of parameters of a model

  • coalesce (bool, optional) – Whether allreduce parameters as a whole. Defaults to True.

  • bucket_size_mb (int, optional) – Size of bucket, the unit is MB. Defaults to -1.

mmdet.core.utils.center_of_mass(mask, esp=1e-06)[source]

Calculate the centroid coordinates of the mask.

Parameters
  • mask (Tensor) – The mask to be calculated, shape (h, w).

  • esp (float) – Avoid dividing by zero. Default: 1e-6.

Returns

the coordinates of the center point of the mask.

  • center_h (Tensor): the center point of the height.

  • center_w (Tensor): the center point of the width.

Return type

tuple[Tensor]

mmdet.core.utils.flip_tensor(src_tensor, flip_direction)[source]

flip tensor base on flip_direction.

Parameters
  • src_tensor (Tensor) – input feature map, shape (B, C, H, W).

  • flip_direction (str) – The flipping direction. Options are ‘horizontal’, ‘vertical’, ‘diagonal’.

Returns

Flipped tensor.

Return type

out_tensor (Tensor)

mmdet.core.utils.generate_coordinate(featmap_sizes, device='cuda')[source]

Generate the coordinate.

Parameters
  • featmap_sizes (tuple) – The feature to be calculated, of shape (N, C, W, H).

  • device (str) – The device where the feature will be put on.

Returns

The coordinate feature, of shape (N, 2, W, H).

Return type

coord_feat (Tensor)

mmdet.core.utils.mask2ndarray(mask)[source]

Convert Mask to ndarray..

:param mask (BitmapMasks or PolygonMasks or: :param torch.Tensor or np.ndarray): The mask to be converted.

Returns

Ndarray mask of shape (n, h, w) that has been converted

Return type

np.ndarray

mmdet.core.utils.multi_apply(func, *args, **kwargs)[source]

Apply function to a list of arguments.

Note

This function applies the func to multiple inputs and map the multiple outputs of the func into different list. Each list contains the same type of outputs corresponding to different inputs.

Parameters

func (Function) – A function that will be applied to a list of arguments

Returns

A tuple containing multiple list, each list contains a kind of returned results by the function

Return type

tuple(list)

mmdet.core.utils.reduce_mean(tensor)[source]

“Obtain the mean of tensor on different GPUs.

mmdet.core.utils.unmap(data, count, inds, fill=0)[source]

Unmap a subset of item (data) back to the original set of items (of size count)

mmdet.datasets

datasets

class mmdet.datasets.CityscapesDataset(ann_file, pipeline, classes=None, data_root=None, img_prefix='', seg_prefix=None, proposal_file=None, test_mode=False, filter_empty_gt=True)[source]
evaluate(results, metric='bbox', logger=None, outfile_prefix=None, classwise=False, proposal_nums=(100, 300, 1000), iou_thrs=array([0.5, 0.55, 0.6, 0.65, 0.7, 0.75, 0.8, 0.85, 0.9, 0.95]))[source]

Evaluation in Cityscapes/COCO protocol.

Parameters
  • results (list[list | tuple]) – Testing results of the dataset.

  • metric (str | list[str]) – Metrics to be evaluated. Options are ‘bbox’, ‘segm’, ‘proposal’, ‘proposal_fast’.

  • logger (logging.Logger | str | None) – Logger used for printing related information during evaluation. Default: None.

  • outfile_prefix (str | None) – The prefix of output file. It includes the file path and the prefix of filename, e.g., “a/b/prefix”. If results are evaluated with COCO protocol, it would be the prefix of output json file. For example, the metric is ‘bbox’ and ‘segm’, then json files would be “a/b/prefix.bbox.json” and “a/b/prefix.segm.json”. If results are evaluated with cityscapes protocol, it would be the prefix of output txt/png files. The output files would be png images under folder “a/b/prefix/xxx/” and the file name of images would be written into a txt file “a/b/prefix/xxx_pred.txt”, where “xxx” is the video name of cityscapes. If not specified, a temp file will be created. Default: None.

  • classwise (bool) – Whether to evaluating the AP for each class.

  • proposal_nums (Sequence[int]) – Proposal number used for evaluating recalls, such as recall@100, recall@1000. Default: (100, 300, 1000).

  • iou_thrs (Sequence[float]) – IoU threshold used for evaluating recalls. If set to a list, the average recall of all IoUs will also be computed. Default: 0.5.

Returns

COCO style evaluation metric or cityscapes mAP and AP@50.

Return type

dict[str, float]

format_results(results, txtfile_prefix=None)[source]

Format the results to txt (standard format for Cityscapes evaluation).

Parameters
  • results (list) – Testing results of the dataset.

  • txtfile_prefix (str | None) – The prefix of txt files. It includes the file path and the prefix of filename, e.g., “a/b/prefix”. If not specified, a temp file will be created. Default: None.

Returns

(result_files, tmp_dir), result_files is a dict containing the json filepaths, tmp_dir is the temporal directory created for saving txt/png files when txtfile_prefix is not specified.

Return type

tuple

results2txt(results, outfile_prefix)[source]

Dump the detection results to a txt file.

Parameters
  • results (list[list | tuple]) – Testing results of the dataset.

  • outfile_prefix (str) – The filename prefix of the json files. If the prefix is “somepath/xxx”, the txt files will be named “somepath/xxx.txt”.

Returns

Result txt files which contains corresponding instance segmentation images.

Return type

list[str]

class mmdet.datasets.ClassBalancedDataset(dataset, oversample_thr, filter_empty_gt=True)[source]

A wrapper of repeated dataset with repeat factor.

Suitable for training on class imbalanced datasets like LVIS. Following the sampling strategy in the paper, in each epoch, an image may appear multiple times based on its “repeat factor”. The repeat factor for an image is a function of the frequency the rarest category labeled in that image. The “frequency of category c” in [0, 1] is defined by the fraction of images in the training set (without repeats) in which category c appears. The dataset needs to instantiate self.get_cat_ids() to support ClassBalancedDataset.

The repeat factor is computed as followed.

  1. For each category c, compute the fraction # of images that contain it: \(f(c)\)

  2. For each category c, compute the category-level repeat factor: \(r(c) = max(1, sqrt(t/f(c)))\)

  3. For each image I, compute the image-level repeat factor: \(r(I) = max_{c in I} r(c)\)

Parameters
  • dataset (CustomDataset) – The dataset to be repeated.

  • oversample_thr (float) – frequency threshold below which data is repeated. For categories with f_c >= oversample_thr, there is no oversampling. For categories with f_c < oversample_thr, the degree of oversampling following the square-root inverse frequency heuristic above.

  • filter_empty_gt (bool, optional) – If set true, images without bounding boxes will not be oversampled. Otherwise, they will be categorized as the pure background class and involved into the oversampling. Default: True.

class mmdet.datasets.CocoDataset(ann_file, pipeline, classes=None, data_root=None, img_prefix='', seg_prefix=None, proposal_file=None, test_mode=False, filter_empty_gt=True)[source]
evaluate(results, metric='bbox', logger=None, jsonfile_prefix=None, classwise=False, proposal_nums=(100, 300, 1000), iou_thrs=None, metric_items=None)[source]

Evaluation in COCO protocol.

Parameters
  • results (list[list | tuple]) – Testing results of the dataset.

  • metric (str | list[str]) – Metrics to be evaluated. Options are ‘bbox’, ‘segm’, ‘proposal’, ‘proposal_fast’.

  • logger (logging.Logger | str | None) – Logger used for printing related information during evaluation. Default: None.

  • jsonfile_prefix (str | None) – The prefix of json files. It includes the file path and the prefix of filename, e.g., “a/b/prefix”. If not specified, a temp file will be created. Default: None.

  • classwise (bool) – Whether to evaluating the AP for each class.

  • proposal_nums (Sequence[int]) – Proposal number used for evaluating recalls, such as recall@100, recall@1000. Default: (100, 300, 1000).

  • iou_thrs (Sequence[float], optional) – IoU threshold used for evaluating recalls/mAPs. If set to a list, the average of all IoUs will also be computed. If not specified, [0.50, 0.55, 0.60, 0.65, 0.70, 0.75, 0.80, 0.85, 0.90, 0.95] will be used. Default: None.

  • metric_items (list[str] | str, optional) – Metric items that will be returned. If not specified, ['AR@100', 'AR@300', 'AR@1000', 'AR_s@1000', 'AR_m@1000', 'AR_l@1000' ] will be used when metric=='proposal', ['mAP', 'mAP_50', 'mAP_75', 'mAP_s', 'mAP_m', 'mAP_l'] will be used when metric=='bbox' or metric=='segm'.

Returns

COCO style evaluation metric.

Return type

dict[str, float]

format_results(results, jsonfile_prefix=None, **kwargs)[source]

Format the results to json (standard format for COCO evaluation).

Parameters
  • results (list[tuple | numpy.ndarray]) – Testing results of the dataset.

  • jsonfile_prefix (str | None) – The prefix of json files. It includes the file path and the prefix of filename, e.g., “a/b/prefix”. If not specified, a temp file will be created. Default: None.

Returns

(result_files, tmp_dir), result_files is a dict containing the json filepaths, tmp_dir is the temporal directory created for saving json files when jsonfile_prefix is not specified.

Return type

tuple

get_ann_info(idx)[source]

Get COCO annotation by index.

Parameters

idx (int) – Index of data.

Returns

Annotation info of specified index.

Return type

dict

get_cat_ids(idx)[source]

Get COCO category ids by index.

Parameters

idx (int) – Index of data.

Returns

All categories in the image of specified index.

Return type

list[int]

load_annotations(ann_file)[source]

Load annotation from COCO style annotation file.

Parameters

ann_file (str) – Path of annotation file.

Returns

Annotation info from COCO api.

Return type

list[dict]

results2json(results, outfile_prefix)[source]

Dump the detection results to a COCO style json file.

There are 3 types of results: proposals, bbox predictions, mask predictions, and they have different data types. This method will automatically recognize the type, and dump them to json files.

Parameters
  • results (list[list | tuple | ndarray]) – Testing results of the dataset.

  • outfile_prefix (str) – The filename prefix of the json files. If the prefix is “somepath/xxx”, the json files will be named “somepath/xxx.bbox.json”, “somepath/xxx.segm.json”, “somepath/xxx.proposal.json”.

Returns

str]: Possible keys are “bbox”, “segm”, “proposal”, and values are corresponding filenames.

Return type

dict[str

xyxy2xywh(bbox)[source]

Convert xyxy style bounding boxes to xywh style for COCO evaluation.

Parameters

bbox (numpy.ndarray) – The bounding boxes, shape (4, ), in xyxy order.

Returns

The converted bounding boxes, in xywh order.

Return type

list[float]

class mmdet.datasets.CocoPanopticDataset(ann_file, pipeline, classes=None, data_root=None, img_prefix='', seg_prefix=None, proposal_file=None, test_mode=False, filter_empty_gt=True)[source]

Coco dataset for Panoptic segmentation.

The annotation format is shown as follows. The ann field is optional for testing.

[
    {
        'filename': f'{image_id:012}.png',
        'image_id':9
        'segments_info': {
            [
                {
                    'id': 8345037, (segment_id in panoptic png,
                                    convert from rgb)
                    'category_id': 51,
                    'iscrowd': 0,
                    'bbox': (x1, y1, w, h),
                    'area': 24315,
                    'segmentation': list,(coded mask)
                },
                ...
            }
        }
    },
    ...
]
evaluate(results, metric='PQ', logger=None, jsonfile_prefix=None, classwise=False, **kwargs)[source]

Evaluation in COCO Panoptic protocol.

Parameters
  • results (list[dict]) – Testing results of the dataset.

  • metric (str | list[str]) – Metrics to be evaluated. Only support ‘PQ’ at present. ‘pq’ will be regarded as ‘PQ.

  • logger (logging.Logger | str | None) – Logger used for printing related information during evaluation. Default: None.

  • jsonfile_prefix (str | None) – The prefix of json files. It includes the file path and the prefix of filename, e.g., “a/b/prefix”. If not specified, a temp file will be created. Default: None.

  • classwise (bool) – Whether to print classwise evaluation results. Default: False.

Returns

COCO Panoptic style evaluation metric.

Return type

dict[str, float]

evaluate_pan_json(result_files, outfile_prefix, logger=None, classwise=False)[source]

Evaluate PQ according to the panoptic results json file.

get_ann_info(idx)[source]

Get COCO annotation by index.

Parameters

idx (int) – Index of data.

Returns

Annotation info of specified index.

Return type

dict

load_annotations(ann_file)[source]

Load annotation from COCO Panoptic style annotation file.

Parameters

ann_file (str) – Path of annotation file.

Returns

Annotation info from COCO api.

Return type

list[dict]

results2json(results, outfile_prefix)[source]

Dump the panoptic results to a COCO panoptic style json file.

Parameters
  • results (dict) – Testing results of the dataset.

  • outfile_prefix (str) – The filename prefix of the json files. If the prefix is “somepath/xxx”, the json files will be named “somepath/xxx.panoptic.json”

Returns

str]: The key is ‘panoptic’ and the value is

corresponding filename.

Return type

dict[str

class mmdet.datasets.ConcatDataset(datasets, separate_eval=True)[source]

A wrapper of concatenated dataset.

Same as torch.utils.data.dataset.ConcatDataset, but concat the group flag for image aspect ratio.

Parameters
  • datasets (list[Dataset]) – A list of datasets.

  • separate_eval (bool) – Whether to evaluate the results separately if it is used as validation dataset. Defaults to True.

evaluate(results, logger=None, **kwargs)[source]

Evaluate the results.

Parameters
  • results (list[list | tuple]) – Testing results of the dataset.

  • logger (logging.Logger | str | None) – Logger used for printing related information during evaluation. Default: None.

Returns

float]: AP results of the total dataset or each separate dataset if self.separate_eval=True.

Return type

dict[str

get_cat_ids(idx)[source]

Get category ids of concatenated dataset by index.

Parameters

idx (int) – Index of data.

Returns

All categories in the image of specified index.

Return type

list[int]

class mmdet.datasets.CustomDataset(ann_file, pipeline, classes=None, data_root=None, img_prefix='', seg_prefix=None, proposal_file=None, test_mode=False, filter_empty_gt=True)[source]

Custom dataset for detection.

The annotation format is shown as follows. The ann field is optional for testing.

[
    {
        'filename': 'a.jpg',
        'width': 1280,
        'height': 720,
        'ann': {
            'bboxes': <np.ndarray> (n, 4) in (x1, y1, x2, y2) order.
            'labels': <np.ndarray> (n, ),
            'bboxes_ignore': <np.ndarray> (k, 4), (optional field)
            'labels_ignore': <np.ndarray> (k, 4) (optional field)
        }
    },
    ...
]
Parameters
  • ann_file (str) – Annotation file path.

  • pipeline (list[dict]) – Processing pipeline.

  • classes (str | Sequence[str], optional) – Specify classes to load. If is None, cls.CLASSES will be used. Default: None.

  • data_root (str, optional) – Data root for ann_file, img_prefix, seg_prefix, proposal_file if specified.

  • test_mode (bool, optional) – If set True, annotation will not be loaded.

  • filter_empty_gt (bool, optional) – If set true, images without bounding boxes of the dataset’s classes will be filtered out. This option only works when test_mode=False, i.e., we never filter images during tests.

evaluate(results, metric='mAP', logger=None, proposal_nums=(100, 300, 1000), iou_thr=0.5, scale_ranges=None)[source]

Evaluate the dataset.

Parameters
  • results (list) – Testing results of the dataset.

  • metric (str | list[str]) – Metrics to be evaluated.

  • logger (logging.Logger | None | str) – Logger used for printing related information during evaluation. Default: None.

  • proposal_nums (Sequence[int]) – Proposal number used for evaluating recalls, such as recall@100, recall@1000. Default: (100, 300, 1000).

  • iou_thr (float | list[float]) – IoU threshold. Default: 0.5.

  • scale_ranges (list[tuple] | None) – Scale ranges for evaluating mAP. Default: None.

format_results(results, **kwargs)[source]

Place holder to format result to dataset specific output.

get_ann_info(idx)[source]

Get annotation by index.

Parameters

idx (int) – Index of data.

Returns

Annotation info of specified index.

Return type

dict

get_cat_ids(idx)[source]

Get category ids by index.

Parameters

idx (int) – Index of data.

Returns

All categories in the image of specified index.

Return type

list[int]

classmethod get_classes(classes=None)[source]

Get class names of current dataset.

Parameters

classes (Sequence[str] | str | None) – If classes is None, use default CLASSES defined by builtin dataset. If classes is a string, take it as a file name. The file contains the name of classes where each line contains one class name. If classes is a tuple or list, override the CLASSES defined by the dataset.

Returns

Names of categories of the dataset.

Return type

tuple[str] or list[str]

load_annotations(ann_file)[source]

Load annotation from annotation file.

load_proposals(proposal_file)[source]

Load proposal from proposal file.

pre_pipeline(results)[source]

Prepare results dict for pipeline.

prepare_test_img(idx)[source]

Get testing data after pipeline.

Parameters

idx (int) – Index of data.

Returns

Testing data after pipeline with new keys introduced by pipeline.

Return type

dict

prepare_train_img(idx)[source]

Get training data and annotations after pipeline.

Parameters

idx (int) – Index of data.

Returns

Training data and annotation after pipeline with new keys introduced by pipeline.

Return type

dict

class mmdet.datasets.DeepFashionDataset(ann_file, pipeline, classes=None, data_root=None, img_prefix='', seg_prefix=None, proposal_file=None, test_mode=False, filter_empty_gt=True)[source]
class mmdet.datasets.DistributedGroupSampler(dataset, samples_per_gpu=1, num_replicas=None, rank=None, seed=0)[source]

Sampler that restricts data loading to a subset of the dataset.

It is especially useful in conjunction with torch.nn.parallel.DistributedDataParallel. In such case, each process can pass a DistributedSampler instance as a DataLoader sampler, and load a subset of the original dataset that is exclusive to it.

Note

Dataset is assumed to be of constant size.

Parameters
  • dataset – Dataset used for sampling.

  • num_replicas (optional) – Number of processes participating in distributed training.

  • rank (optional) – Rank of the current process within num_replicas.

  • seed (int, optional) – random seed used to shuffle the sampler if shuffle=True. This number should be identical across all processes in the distributed group. Default: 0.

class mmdet.datasets.DistributedSampler(dataset, num_replicas=None, rank=None, shuffle=True, seed=0)[source]
class mmdet.datasets.GroupSampler(dataset, samples_per_gpu=1)[source]
mmdet.datasets.LVISDataset

alias of mmdet.datasets.lvis.LVISV05Dataset

class mmdet.datasets.LVISV05Dataset(ann_file, pipeline, classes=None, data_root=None, img_prefix='', seg_prefix=None, proposal_file=None, test_mode=False, filter_empty_gt=True)[source]
evaluate(results, metric='bbox', logger=None, jsonfile_prefix=None, classwise=False, proposal_nums=(100, 300, 1000), iou_thrs=array([0.5, 0.55, 0.6, 0.65, 0.7, 0.75, 0.8, 0.85, 0.9, 0.95]))[source]

Evaluation in LVIS protocol.

Parameters
  • results (list[list | tuple]) – Testing results of the dataset.

  • metric (str | list[str]) – Metrics to be evaluated. Options are ‘bbox’, ‘segm’, ‘proposal’, ‘proposal_fast’.

  • logger (logging.Logger | str | None) – Logger used for printing related information during evaluation. Default: None.

  • jsonfile_prefix (str | None) –

  • classwise (bool) – Whether to evaluating the AP for each class.

  • proposal_nums (Sequence[int]) – Proposal number used for evaluating recalls, such as recall@100, recall@1000. Default: (100, 300, 1000).

  • iou_thrs (Sequence[float]) – IoU threshold used for evaluating recalls. If set to a list, the average recall of all IoUs will also be computed. Default: 0.5.

Returns

LVIS style metrics.

Return type

dict[str, float]

load_annotations(ann_file)[source]

Load annotation from lvis style annotation file.

Parameters

ann_file (str) – Path of annotation file.

Returns

Annotation info from LVIS api.

Return type

list[dict]

class mmdet.datasets.LVISV1Dataset(ann_file, pipeline, classes=None, data_root=None, img_prefix='', seg_prefix=None, proposal_file=None, test_mode=False, filter_empty_gt=True)[source]
load_annotations(ann_file)[source]

Load annotation from lvis style annotation file.

Parameters

ann_file (str) – Path of annotation file.

Returns

Annotation info from LVIS api.

Return type

list[dict]

class mmdet.datasets.MultiImageMixDataset(dataset, pipeline, dynamic_scale=None, skip_type_keys=None)[source]

A wrapper of multiple images mixed dataset.

Suitable for training on multiple images mixed data augmentation like mosaic and mixup. For the augmentation pipeline of mixed image data, the get_indexes method needs to be provided to obtain the image indexes, and you can set skip_flags to change the pipeline running process. At the same time, we provide the dynamic_scale parameter to dynamically change the output image size.

Parameters
  • dataset (CustomDataset) – The dataset to be mixed.

  • pipeline (Sequence[dict]) – Sequence of transform object or config dict to be composed.

  • dynamic_scale (tuple[int], optional) – The image scale can be changed dynamically. Default to None.

  • skip_type_keys (list[str], optional) – Sequence of type string to be skip pipeline. Default to None.

update_dynamic_scale(dynamic_scale)[source]

Update dynamic_scale. It is called by an external hook.

Parameters

dynamic_scale (tuple[int]) – The image scale can be changed dynamically.

update_skip_type_keys(skip_type_keys)[source]

Update skip_type_keys. It is called by an external hook.

Parameters

skip_type_keys (list[str], optional) – Sequence of type string to be skip pipeline.

class mmdet.datasets.RepeatDataset(dataset, times)[source]

A wrapper of repeated dataset.

The length of repeated dataset will be times larger than the original dataset. This is useful when the data loading time is long but the dataset is small. Using RepeatDataset can reduce the data loading time between epochs.

Parameters
  • dataset (Dataset) – The dataset to be repeated.

  • times (int) – Repeat times.

get_cat_ids(idx)[source]

Get category ids of repeat dataset by index.

Parameters

idx (int) – Index of data.

Returns

All categories in the image of specified index.

Return type

list[int]

class mmdet.datasets.VOCDataset(**kwargs)[source]
evaluate(results, metric='mAP', logger=None, proposal_nums=(100, 300, 1000), iou_thr=0.5, scale_ranges=None)[source]

Evaluate in VOC protocol.

Parameters
  • results (list[list | tuple]) – Testing results of the dataset.

  • metric (str | list[str]) – Metrics to be evaluated. Options are ‘mAP’, ‘recall’.

  • logger (logging.Logger | str, optional) – Logger used for printing related information during evaluation. Default: None.

  • proposal_nums (Sequence[int]) – Proposal number used for evaluating recalls, such as recall@100, recall@1000. Default: (100, 300, 1000).

  • iou_thr (float | list[float]) – IoU threshold. Default: 0.5.

  • scale_ranges (list[tuple], optional) – Scale ranges for evaluating mAP. If not specified, all bounding boxes would be included in evaluation. Default: None.

Returns

AP/recall metrics.

Return type

dict[str, float]

class mmdet.datasets.WIDERFaceDataset(**kwargs)[source]

Reader for the WIDER Face dataset in PASCAL VOC format.

Conversion scripts can be found in https://github.com/sovrasov/wider-face-pascal-voc-annotations

load_annotations(ann_file)[source]

Load annotation from WIDERFace XML style annotation file.

Parameters

ann_file (str) – Path of XML file.

Returns

Annotation info from XML file.

Return type

list[dict]

class mmdet.datasets.XMLDataset(min_size=None, img_subdir='JPEGImages', ann_subdir='Annotations', **kwargs)[source]

XML dataset for detection.

Parameters
  • min_size (int | float, optional) – The minimum size of bounding boxes in the images. If the size of a bounding box is less than min_size, it would be add to ignored field.

  • img_subdir (str) – Subdir where images are stored. Default: JPEGImages.

  • ann_subdir (str) – Subdir where annotations are. Default: Annotations.

get_ann_info(idx)[source]

Get annotation from XML file by index.

Parameters

idx (int) – Index of data.

Returns

Annotation info of specified index.

Return type

dict

get_cat_ids(idx)[source]

Get category ids in XML file by index.

Parameters

idx (int) – Index of data.

Returns

All categories in the image of specified index.

Return type

list[int]

load_annotations(ann_file)[source]

Load annotation from XML style ann_file.

Parameters

ann_file (str) – Path of XML file.

Returns

Annotation info from XML file.

Return type

list[dict]

mmdet.datasets.build_dataloader(dataset, samples_per_gpu, workers_per_gpu, num_gpus=1, dist=True, shuffle=True, seed=None, **kwargs)[source]

Build PyTorch DataLoader.

In distributed training, each GPU/process has a dataloader. In non-distributed training, there is only one dataloader for all GPUs.

Parameters
  • dataset (Dataset) – A PyTorch dataset.

  • samples_per_gpu (int) – Number of training samples on each GPU, i.e., batch size of each GPU.

  • workers_per_gpu (int) – How many subprocesses to use for data loading for each GPU.

  • num_gpus (int) – Number of GPUs. Only used in non-distributed training.

  • dist (bool) – Distributed training/test or not. Default: True.

  • shuffle (bool) – Whether to shuffle the data at every epoch. Default: True.

  • kwargs – any keyword argument to be used to initialize DataLoader

Returns

A PyTorch dataloader.

Return type

DataLoader

mmdet.datasets.get_loading_pipeline(pipeline)[source]

Only keep loading image and annotations related configuration.

Parameters

pipeline (list[dict]) – Data pipeline configs.

Returns

The new pipeline list with only keep

loading image and annotations related configuration.

Return type

list[dict]

Examples

>>> pipelines = [
...    dict(type='LoadImageFromFile'),
...    dict(type='LoadAnnotations', with_bbox=True),
...    dict(type='Resize', img_scale=(1333, 800), keep_ratio=True),
...    dict(type='RandomFlip', flip_ratio=0.5),
...    dict(type='Normalize', **img_norm_cfg),
...    dict(type='Pad', size_divisor=32),
...    dict(type='DefaultFormatBundle'),
...    dict(type='Collect', keys=['img', 'gt_bboxes', 'gt_labels'])
...    ]
>>> expected_pipelines = [
...    dict(type='LoadImageFromFile'),
...    dict(type='LoadAnnotations', with_bbox=True)
...    ]
>>> assert expected_pipelines ==        ...        get_loading_pipeline(pipelines)
mmdet.datasets.replace_ImageToTensor(pipelines)[source]

Replace the ImageToTensor transform in a data pipeline to DefaultFormatBundle, which is normally useful in batch inference.

Parameters

pipelines (list[dict]) – Data pipeline configs.

Returns

The new pipeline list with all ImageToTensor replaced by

DefaultFormatBundle.

Return type

list

Examples

>>> pipelines = [
...    dict(type='LoadImageFromFile'),
...    dict(
...        type='MultiScaleFlipAug',
...        img_scale=(1333, 800),
...        flip=False,
...        transforms=[
...            dict(type='Resize', keep_ratio=True),
...            dict(type='RandomFlip'),
...            dict(type='Normalize', mean=[0, 0, 0], std=[1, 1, 1]),
...            dict(type='Pad', size_divisor=32),
...            dict(type='ImageToTensor', keys=['img']),
...            dict(type='Collect', keys=['img']),
...        ])
...    ]
>>> expected_pipelines = [
...    dict(type='LoadImageFromFile'),
...    dict(
...        type='MultiScaleFlipAug',
...        img_scale=(1333, 800),
...        flip=False,
...        transforms=[
...            dict(type='Resize', keep_ratio=True),
...            dict(type='RandomFlip'),
...            dict(type='Normalize', mean=[0, 0, 0], std=[1, 1, 1]),
...            dict(type='Pad', size_divisor=32),
...            dict(type='DefaultFormatBundle'),
...            dict(type='Collect', keys=['img']),
...        ])
...    ]
>>> assert expected_pipelines == replace_ImageToTensor(pipelines)

pipelines

class mmdet.datasets.pipelines.Albu(transforms, bbox_params=None, keymap=None, update_pad_shape=False, skip_img_without_anno=False)[source]

Albumentation augmentation.

Adds custom transformations from Albumentations library. Please, visit https://albumentations.readthedocs.io to get more information.

An example of transforms is as followed:

[
    dict(
        type='ShiftScaleRotate',
        shift_limit=0.0625,
        scale_limit=0.0,
        rotate_limit=0,
        interpolation=1,
        p=0.5),
    dict(
        type='RandomBrightnessContrast',
        brightness_limit=[0.1, 0.3],
        contrast_limit=[0.1, 0.3],
        p=0.2),
    dict(type='ChannelShuffle', p=0.1),
    dict(
        type='OneOf',
        transforms=[
            dict(type='Blur', blur_limit=3, p=1.0),
            dict(type='MedianBlur', blur_limit=3, p=1.0)
        ],
        p=0.1),
]
Parameters
  • transforms (list[dict]) – A list of albu transformations

  • bbox_params (dict) – Bbox_params for albumentation Compose

  • keymap (dict) – Contains {‘input key’:’albumentation-style key’}

  • skip_img_without_anno (bool) – Whether to skip the image if no ann left after aug

albu_builder(cfg)[source]

Import a module from albumentations.

It inherits some of build_from_cfg() logic.

Parameters

cfg (dict) – Config dict. It should at least contain the key “type”.

Returns

The constructed object.

Return type

obj

static mapper(d, keymap)[source]

Dictionary mapper. Renames keys according to keymap provided.

Parameters
  • d (dict) – old dict

  • keymap (dict) – {‘old_key’:’new_key’}

Returns

new dict.

Return type

dict

class mmdet.datasets.pipelines.AutoAugment(policies)[source]

Auto augmentation.

This data augmentation is proposed in Learning Data Augmentation Strategies for Object Detection.

TODO: Implement ‘Shear’, ‘Sharpness’ and ‘Rotate’ transforms

Parameters

policies (list[list[dict]]) – The policies of auto augmentation. Each policy in policies is a specific augmentation policy, and is composed by several augmentations (dict). When AutoAugment is called, a random policy in policies will be selected to augment images.

Examples

>>> replace = (104, 116, 124)
>>> policies = [
>>>     [
>>>         dict(type='Sharpness', prob=0.0, level=8),
>>>         dict(
>>>             type='Shear',
>>>             prob=0.4,
>>>             level=0,
>>>             replace=replace,
>>>             axis='x')
>>>     ],
>>>     [
>>>         dict(
>>>             type='Rotate',
>>>             prob=0.6,
>>>             level=10,
>>>             replace=replace),
>>>         dict(type='Color', prob=1.0, level=6)
>>>     ]
>>> ]
>>> augmentation = AutoAugment(policies)
>>> img = np.ones(100, 100, 3)
>>> gt_bboxes = np.ones(10, 4)
>>> results = dict(img=img, gt_bboxes=gt_bboxes)
>>> results = augmentation(results)
class mmdet.datasets.pipelines.BrightnessTransform(level, prob=0.5)[source]

Apply Brightness transformation to image. The bboxes, masks and segmentations are not modified.

Parameters
  • level (int | float) – Should be in range [0,_MAX_LEVEL].

  • prob (float) – The probability for performing Brightness transformation.

class mmdet.datasets.pipelines.Collect(keys, meta_keys=('filename', 'ori_filename', 'ori_shape', 'img_shape', 'pad_shape', 'scale_factor', 'flip', 'flip_direction', 'img_norm_cfg'))[source]

Collect data from the loader relevant to the specific task.

This is usually the last stage of the data loader pipeline. Typically keys is set to some subset of “img”, “proposals”, “gt_bboxes”, “gt_bboxes_ignore”, “gt_labels”, and/or “gt_masks”.

The “img_meta” item is always populated. The contents of the “img_meta” dictionary depends on “meta_keys”. By default this includes:

  • “img_shape”: shape of the image input to the network as a tuple (h, w, c). Note that images may be zero padded on the bottom/right if the batch tensor is larger than this shape.

  • “scale_factor”: a float indicating the preprocessing scale

  • “flip”: a boolean indicating if image flip transform was used

  • “filename”: path to the image file

  • “ori_shape”: original shape of the image as a tuple (h, w, c)

  • “pad_shape”: image shape after padding

  • “img_norm_cfg”: a dict of normalization information:

    • mean - per channel mean subtraction

    • std - per channel std divisor

    • to_rgb - bool indicating if bgr was converted to rgb

Parameters
  • keys (Sequence[str]) – Keys of results to be collected in data.

  • meta_keys (Sequence[str], optional) – Meta keys to be converted to mmcv.DataContainer and collected in data[img_metas]. Default: ('filename', 'ori_filename', 'ori_shape', 'img_shape', 'pad_shape', 'scale_factor', 'flip', 'flip_direction', 'img_norm_cfg')

class mmdet.datasets.pipelines.ColorTransform(level, prob=0.5)[source]

Apply Color transformation to image. The bboxes, masks, and segmentations are not modified.

Parameters
  • level (int | float) – Should be in range [0,_MAX_LEVEL].

  • prob (float) – The probability for performing Color transformation.

class mmdet.datasets.pipelines.Compose(transforms)[source]

Compose multiple transforms sequentially.

Parameters

transforms (Sequence[dict | callable]) – Sequence of transform object or config dict to be composed.

class mmdet.datasets.pipelines.ContrastTransform(level, prob=0.5)[source]

Apply Contrast transformation to image. The bboxes, masks and segmentations are not modified.

Parameters
  • level (int | float) – Should be in range [0,_MAX_LEVEL].

  • prob (float) – The probability for performing Contrast transformation.

class mmdet.datasets.pipelines.CutOut(n_holes, cutout_shape=None, cutout_ratio=None, fill_in=(0, 0, 0))[source]

CutOut operation.

Randomly drop some regions of image used in Cutout.

Parameters
  • n_holes (int | tuple[int, int]) – Number of regions to be dropped. If it is given as a list, number of holes will be randomly selected from the closed interval [n_holes[0], n_holes[1]].

  • cutout_shape (tuple[int, int] | list[tuple[int, int]]) – The candidate shape of dropped regions. It can be tuple[int, int] to use a fixed cutout shape, or list[tuple[int, int]] to randomly choose shape from the list.

  • cutout_ratio (tuple[float, float] | list[tuple[float, float]]) – The candidate ratio of dropped regions. It can be tuple[float, float] to use a fixed ratio or list[tuple[float, float]] to randomly choose ratio from the list. Please note that cutout_shape and cutout_ratio cannot be both given at the same time.

  • fill_in (tuple[float, float, float] | tuple[int, int, int]) – The value of pixel to fill in the dropped regions. Default: (0, 0, 0).

class mmdet.datasets.pipelines.DefaultFormatBundle[source]

Default formatting bundle.

It simplifies the pipeline of formatting common fields, including “img”, “proposals”, “gt_bboxes”, “gt_labels”, “gt_masks” and “gt_semantic_seg”. These fields are formatted as follows.

  • img: (1)transpose, (2)to tensor, (3)to DataContainer (stack=True)

  • proposals: (1)to tensor, (2)to DataContainer

  • gt_bboxes: (1)to tensor, (2)to DataContainer

  • gt_bboxes_ignore: (1)to tensor, (2)to DataContainer

  • gt_labels: (1)to tensor, (2)to DataContainer

  • gt_masks: (1)to tensor, (2)to DataContainer (cpu_only=True)

  • gt_semantic_seg: (1)unsqueeze dim-0 (2)to tensor, (3)to DataContainer (stack=True)

class mmdet.datasets.pipelines.EqualizeTransform(prob=0.5)[source]

Apply Equalize transformation to image. The bboxes, masks and segmentations are not modified.

Parameters

prob (float) – The probability for performing Equalize transformation.

class mmdet.datasets.pipelines.Expand(mean=(0, 0, 0), to_rgb=True, ratio_range=(1, 4), seg_ignore_label=None, prob=0.5)[source]

Random expand the image & bboxes.

Randomly place the original image on a canvas of ‘ratio’ x original image size filled with mean values. The ratio is in the range of ratio_range.

Parameters
  • mean (tuple) – mean value of dataset.

  • to_rgb (bool) – if need to convert the order of mean to align with RGB.

  • ratio_range (tuple) – range of expand ratio.

  • prob (float) – probability of applying this transformation

class mmdet.datasets.pipelines.ImageToTensor(keys)[source]

Convert image to torch.Tensor by given keys.

The dimension order of input image is (H, W, C). The pipeline will convert it to (C, H, W). If only 2 dimension (H, W) is given, the output would be (1, H, W).

Parameters

keys (Sequence[str]) – Key of images to be converted to Tensor.

class mmdet.datasets.pipelines.InstaBoost(action_candidate=('normal', 'horizontal', 'skip'), action_prob=(1, 0, 0), scale=(0.8, 1.2), dx=15, dy=15, theta=(- 1, 1), color_prob=0.5, hflag=False, aug_ratio=0.5)[source]

Data augmentation method in InstaBoost: Boosting Instance Segmentation Via Probability Map Guided Copy-Pasting.

Refer to https://github.com/GothicAi/Instaboost for implementation details.

Parameters
  • action_candidate (tuple) – Action candidates. “normal”, “horizontal”, “vertical”, “skip” are supported. Default: (‘normal’, ‘horizontal’, ‘skip’).

  • action_prob (tuple) – Corresponding action probabilities. Should be the same length as action_candidate. Default: (1, 0, 0).

  • scale (tuple) – (min scale, max scale). Default: (0.8, 1.2).

  • dx (int) – The maximum x-axis shift will be (instance width) / dx. Default 15.

  • dy (int) – The maximum y-axis shift will be (instance height) / dy. Default 15.

  • theta (tuple) – (min rotation degree, max rotation degree). Default: (-1, 1).

  • color_prob (float) – Probability of images for color augmentation. Default 0.5.

  • heatmap_flag (bool) – Whether to use heatmap guided. Default False.

  • aug_ratio (float) – Probability of applying this transformation. Default 0.5.

class mmdet.datasets.pipelines.LoadAnnotations(with_bbox=True, with_label=True, with_mask=False, with_seg=False, poly2mask=True, file_client_args={'backend': 'disk'})[source]

Load multiple types of annotations.

Parameters
  • with_bbox (bool) – Whether to parse and load the bbox annotation. Default: True.

  • with_label (bool) – Whether to parse and load the label annotation. Default: True.

  • with_mask (bool) – Whether to parse and load the mask annotation. Default: False.

  • with_seg (bool) – Whether to parse and load the semantic segmentation annotation. Default: False.

  • poly2mask (bool) – Whether to convert the instance masks from polygons to bitmaps. Default: True.

  • file_client_args (dict) – Arguments to instantiate a FileClient. See mmcv.fileio.FileClient for details. Defaults to dict(backend='disk').

process_polygons(polygons)[source]

Convert polygons to list of ndarray and filter invalid polygons.

Parameters

polygons (list[list]) – Polygons of one instance.

Returns

Processed polygons.

Return type

list[numpy.ndarray]

class mmdet.datasets.pipelines.LoadImageFromFile(to_float32=False, color_type='color', file_client_args={'backend': 'disk'})[source]

Load an image from file.

Required keys are “img_prefix” and “img_info” (a dict that must contain the key “filename”). Added or updated keys are “filename”, “img”, “img_shape”, “ori_shape” (same as img_shape), “pad_shape” (same as img_shape), “scale_factor” (1.0) and “img_norm_cfg” (means=0 and stds=1).

Parameters
  • to_float32 (bool) – Whether to convert the loaded image to a float32 numpy array. If set to False, the loaded image is an uint8 array. Defaults to False.

  • color_type (str) – The flag argument for mmcv.imfrombytes(). Defaults to ‘color’.

  • file_client_args (dict) – Arguments to instantiate a FileClient. See mmcv.fileio.FileClient for details. Defaults to dict(backend='disk').

class mmdet.datasets.pipelines.LoadImageFromWebcam(to_float32=False, color_type='color', file_client_args={'backend': 'disk'})[source]

Load an image from webcam.

Similar with LoadImageFromFile, but the image read from webcam is in results['img'].

class mmdet.datasets.pipelines.LoadMultiChannelImageFromFiles(to_float32=False, color_type='unchanged', file_client_args={'backend': 'disk'})[source]

Load multi-channel images from a list of separate channel files.

Required keys are “img_prefix” and “img_info” (a dict that must contain the key “filename”, which is expected to be a list of filenames). Added or updated keys are “filename”, “img”, “img_shape”, “ori_shape” (same as img_shape), “pad_shape” (same as img_shape), “scale_factor” (1.0) and “img_norm_cfg” (means=0 and stds=1).

Parameters
  • to_float32 (bool) – Whether to convert the loaded image to a float32 numpy array. If set to False, the loaded image is an uint8 array. Defaults to False.

  • color_type (str) – The flag argument for mmcv.imfrombytes(). Defaults to ‘color’.

  • file_client_args (dict) – Arguments to instantiate a FileClient. See mmcv.fileio.FileClient for details. Defaults to dict(backend='disk').

class mmdet.datasets.pipelines.LoadProposals(num_max_proposals=None)[source]

Load proposal pipeline.

Required key is “proposals”. Updated keys are “proposals”, “bbox_fields”.

Parameters

num_max_proposals (int, optional) – Maximum number of proposals to load. If not specified, all proposals will be loaded.

class mmdet.datasets.pipelines.MinIoURandomCrop(min_ious=(0.1, 0.3, 0.5, 0.7, 0.9), min_crop_size=0.3, bbox_clip_border=True)[source]

Random crop the image & bboxes, the cropped patches have minimum IoU requirement with original image & bboxes, the IoU threshold is randomly selected from min_ious.

Parameters
  • min_ious (tuple) – minimum IoU threshold for all intersections with

  • boxes (bounding) –

  • min_crop_size (float) – minimum crop’s size (i.e. h,w := a*h, a*w,

  • a >= min_crop_size) (where) –

  • bbox_clip_border (bool, optional) – Whether clip the objects outside the border of the image. Defaults to True.

Note

The keys for bboxes, labels and masks should be paired. That is, gt_bboxes corresponds to gt_labels and gt_masks, and gt_bboxes_ignore to gt_labels_ignore and gt_masks_ignore.

class mmdet.datasets.pipelines.MixUp(img_scale=(640, 640), ratio_range=(0.5, 1.5), flip_ratio=0.5, pad_val=114, max_iters=15, min_bbox_size=5, min_area_ratio=0.2, max_aspect_ratio=20)[source]

MixUp data augmentation.

Parameters
  • img_scale (Sequence[int]) – Image output size after mixup pipeline. Default: (640, 640).

  • ratio_range (Sequence[float]) – Scale ratio of mixup image. Default: (0.5, 1.5).

  • flip_ratio (float) – Horizontal flip ratio of mixup image. Default: 0.5.

  • pad_val (int) – Pad value. Default: 114.

  • max_iters (int) – The maximum number of iterations. If the number of iterations is greater than max_iters, but gt_bbox is still empty, then the iteration is terminated. Default: 15.

  • min_bbox_size (float) – Width and height threshold to filter bboxes. If the height or width of a box is smaller than this value, it will be removed. Default: 5.

  • min_area_ratio (float) – Threshold of area ratio between original bboxes and wrapped bboxes. If smaller than this value, the box will be removed. Default: 0.2.

  • max_aspect_ratio (float) – Aspect ratio of width and height threshold to filter bboxes. If max(h/w, w/h) larger than this value, the box will be removed. Default: 20.

get_indexes(dataset)[source]

Call function to collect indexes.

Parameters

dataset (MultiImageMixDataset) – The dataset.

Returns

indexes.

Return type

list

class mmdet.datasets.pipelines.Mosaic(img_scale=(640, 640), center_ratio_range=(0.5, 1.5), min_bbox_size=0, pad_val=114)[source]

Mosaic augmentation.

Given 4 images, mosaic transform combines them into one output image. The output image is composed of the parts from each sub- image.

                   mosaic transform
                      center_x
           +------------------------------+
           |       pad        |  pad      |
           |      +-----------+           |
           |      |           |           |
           |      |  image1   |--------+  |
           |      |           |        |  |
           |      |           | image2 |  |
center_y   |----+-------------+-----------|
           |    |   cropped   |           |
           |pad |   image3    |  image4   |
           |    |             |           |
           +----|-------------+-----------+
                |             |
                +-------------+

The mosaic transform steps are as follows:

    1. Choose the mosaic center as the intersections of 4 images
    2. Get the left top image according to the index, and randomly
       sample another 3 images from the custom dataset.
    3. Sub image will be cropped if image is larger than mosaic patch
Parameters
  • img_scale (Sequence[int]) – Image size after mosaic pipeline of single image. Default to (640, 640).

  • center_ratio_range (Sequence[float]) – Center ratio range of mosaic output. Default to (0.5, 1.5).

  • min_bbox_size (int | float) – The minimum pixel for filtering invalid bboxes after the mosaic pipeline. Default to 0.

  • pad_val (int) – Pad value. Default to 114.

get_indexes(dataset)[source]

Call function to collect indexes.

Parameters

dataset (MultiImageMixDataset) – The dataset.

Returns

indexes.

Return type

list

class mmdet.datasets.pipelines.MultiScaleFlipAug(transforms, img_scale=None, scale_factor=None, flip=False, flip_direction='horizontal')[source]

Test-time augmentation with multiple scales and flipping.

An example configuration is as followed:

img_scale=[(1333, 400), (1333, 800)],
flip=True,
transforms=[
    dict(type='Resize', keep_ratio=True),
    dict(type='RandomFlip'),
    dict(type='Normalize', **img_norm_cfg),
    dict(type='Pad', size_divisor=32),
    dict(type='ImageToTensor', keys=['img']),
    dict(type='Collect', keys=['img']),
]

After MultiScaleFLipAug with above configuration, the results are wrapped into lists of the same length as followed:

dict(
    img=[...],
    img_shape=[...],
    scale=[(1333, 400), (1333, 400), (1333, 800), (1333, 800)]
    flip=[False, True, False, True]
    ...
)
Parameters
  • transforms (list[dict]) – Transforms to apply in each augmentation.

  • img_scale (tuple | list[tuple] | None) – Images scales for resizing.

  • scale_factor (float | list[float] | None) – Scale factors for resizing.

  • flip (bool) – Whether apply flip augmentation. Default: False.

  • flip_direction (str | list[str]) – Flip augmentation directions, options are “horizontal”, “vertical” and “diagonal”. If flip_direction is a list, multiple flip augmentations will be applied. It has no effect when flip == False. Default: “horizontal”.

class mmdet.datasets.pipelines.Normalize(mean, std, to_rgb=True)[source]

Normalize the image.

Added key is “img_norm_cfg”.

Parameters
  • mean (sequence) – Mean values of 3 channels.

  • std (sequence) – Std values of 3 channels.

  • to_rgb (bool) – Whether to convert the image from BGR to RGB, default is true.

class mmdet.datasets.pipelines.Pad(size=None, size_divisor=None, pad_to_square=False, pad_val={'img': 0, 'masks': 0, 'seg': 255})[source]

Pad the image & masks & segmentation map.

There are two padding modes: (1) pad to a fixed size and (2) pad to the minimum size that is divisible by some number. Added keys are “pad_shape”, “pad_fixed_size”, “pad_size_divisor”,

Parameters
  • size (tuple, optional) – Fixed padding size.

  • size_divisor (int, optional) – The divisor of padded size.

  • pad_to_square (bool) – Whether to pad the image into a square. Currently only used for YOLOX. Default: False.

  • pad_val (dict, optional) – A dict for padding value, the default value is dict(img=0, masks=0, seg=255).

class mmdet.datasets.pipelines.PhotoMetricDistortion(brightness_delta=32, contrast_range=(0.5, 1.5), saturation_range=(0.5, 1.5), hue_delta=18)[source]

Apply photometric distortion to image sequentially, every transformation is applied with a probability of 0.5. The position of random contrast is in second or second to last.

  1. random brightness

  2. random contrast (mode 0)

  3. convert color from BGR to HSV

  4. random saturation

  5. random hue

  6. convert color from HSV to BGR

  7. random contrast (mode 1)

  8. randomly swap channels

Parameters
  • brightness_delta (int) – delta of brightness.

  • contrast_range (tuple) – range of contrast.

  • saturation_range (tuple) – range of saturation.

  • hue_delta (int) – delta of hue.

class mmdet.datasets.pipelines.RandomAffine(max_rotate_degree=10.0, max_translate_ratio=0.1, scaling_ratio_range=(0.5, 1.5), max_shear_degree=2.0, border=(0, 0), border_val=(114, 114, 114), min_bbox_size=2, min_area_ratio=0.2, max_aspect_ratio=20)[source]

Random affine transform data augmentation.

This operation randomly generates affine transform matrix which including rotation, translation, shear and scaling transforms.

Parameters
  • max_rotate_degree (float) – Maximum degrees of rotation transform. Default: 10.

  • max_translate_ratio (float) – Maximum ratio of translation. Default: 0.1.

  • scaling_ratio_range (tuple[float]) – Min and max ratio of scaling transform. Default: (0.5, 1.5).

  • max_shear_degree (float) – Maximum degrees of shear transform. Default: 2.

  • border (tuple[int]) – Distance from height and width sides of input image to adjust output shape. Only used in mosaic dataset. Default: (0, 0).

  • border_val (tuple[int]) – Border padding values of 3 channels. Default: (114, 114, 114).

  • min_bbox_size (float) – Width and height threshold to filter bboxes. If the height or width of a box is smaller than this value, it will be removed. Default: 2.

  • min_area_ratio (float) – Threshold of area ratio between original bboxes and wrapped bboxes. If smaller than this value, the box will be removed. Default: 0.2.

  • max_aspect_ratio (float) – Aspect ratio of width and height threshold to filter bboxes. If max(h/w, w/h) larger than this value, the box will be removed.

class mmdet.datasets.pipelines.RandomCenterCropPad(crop_size=None, ratios=(0.9, 1.0, 1.1), border=128, mean=None, std=None, to_rgb=None, test_mode=False, test_pad_mode=('logical_or', 127), test_pad_add_pix=0, bbox_clip_border=True)[source]

Random center crop and random around padding for CornerNet.

This operation generates randomly cropped image from the original image and pads it simultaneously. Different from RandomCrop, the output shape may not equal to crop_size strictly. We choose a random value from ratios and the output shape could be larger or smaller than crop_size. The padding operation is also different from Pad, here we use around padding instead of right-bottom padding.

The relation between output image (padding image) and original image:

                output image

       +----------------------------+
       |          padded area       |
+------|----------------------------|----------+
|      |         cropped area       |          |
|      |         +---------------+  |          |
|      |         |    .   center |  |          | original image
|      |         |        range  |  |          |
|      |         +---------------+  |          |
+------|----------------------------|----------+
       |          padded area       |
       +----------------------------+

There are 5 main areas in the figure:

  • output image: output image of this operation, also called padding image in following instruction.

  • original image: input image of this operation.

  • padded area: non-intersect area of output image and original image.

  • cropped area: the overlap of output image and original image.

  • center range: a smaller area where random center chosen from. center range is computed by border and original image’s shape to avoid our random center is too close to original image’s border.

Also this operation act differently in train and test mode, the summary pipeline is listed below.

Train pipeline:

  1. Choose a random_ratio from ratios, the shape of padding image will be random_ratio * crop_size.

  2. Choose a random_center in center range.

  3. Generate padding image with center matches the random_center.

  4. Initialize the padding image with pixel value equals to mean.

  5. Copy the cropped area to padding image.

  6. Refine annotations.

Test pipeline:

  1. Compute output shape according to test_pad_mode.

  2. Generate padding image with center matches the original image center.

  3. Initialize the padding image with pixel value equals to mean.

  4. Copy the cropped area to padding image.

Parameters
  • crop_size (tuple | None) – expected size after crop, final size will computed according to ratio. Requires (h, w) in train mode, and None in test mode.

  • ratios (tuple) – random select a ratio from tuple and crop image to (crop_size[0] * ratio) * (crop_size[1] * ratio). Only available in train mode.

  • border (int) – max distance from center select area to image border. Only available in train mode.

  • mean (sequence) – Mean values of 3 channels.

  • std (sequence) – Std values of 3 channels.

  • to_rgb (bool) – Whether to convert the image from BGR to RGB.

  • test_mode (bool) – whether involve random variables in transform. In train mode, crop_size is fixed, center coords and ratio is random selected from predefined lists. In test mode, crop_size is image’s original shape, center coords and ratio is fixed.

  • test_pad_mode (tuple) –

    padding method and padding shape value, only available in test mode. Default is using ‘logical_or’ with 127 as padding shape value.

    • ’logical_or’: final_shape = input_shape | padding_shape_value

    • ’size_divisor’: final_shape = int( ceil(input_shape / padding_shape_value) * padding_shape_value)

  • test_pad_add_pix (int) – Extra padding pixel in test mode. Default 0.

  • bbox_clip_border (bool, optional) – Whether clip the objects outside the border of the image. Defaults to True.

class mmdet.datasets.pipelines.RandomCrop(crop_size, crop_type='absolute', allow_negative_crop=False, recompute_bbox=False, bbox_clip_border=True)[source]

Random crop the image & bboxes & masks.

The absolute crop_size is sampled based on crop_type and image_size, then the cropped results are generated.

Parameters
  • crop_size (tuple) – The relative ratio or absolute pixels of height and width.

  • crop_type (str, optional) – one of “relative_range”, “relative”, “absolute”, “absolute_range”. “relative” randomly crops (h * crop_size[0], w * crop_size[1]) part from an input of size (h, w). “relative_range” uniformly samples relative crop size from range [crop_size[0], 1] and [crop_size[1], 1] for height and width respectively. “absolute” crops from an input with absolute size (crop_size[0], crop_size[1]). “absolute_range” uniformly samples crop_h in range [crop_size[0], min(h, crop_size[1])] and crop_w in range [crop_size[0], min(w, crop_size[1])]. Default “absolute”.

  • allow_negative_crop (bool, optional) – Whether to allow a crop that does not contain any bbox area. Default False.

  • recompute_bbox (bool, optional) – Whether to re-compute the boxes based on cropped instance masks. Default False.

  • bbox_clip_border (bool, optional) – Whether clip the objects outside the border of the image. Defaults to True.

Note

  • If the image is smaller than the absolute crop size, return the

    original image.

  • The keys for bboxes, labels and masks must be aligned. That is, gt_bboxes corresponds to gt_labels and gt_masks, and gt_bboxes_ignore corresponds to gt_labels_ignore and gt_masks_ignore.

  • If the crop does not contain any gt-bbox region and allow_negative_crop is set to False, skip this image.

class mmdet.datasets.pipelines.RandomFlip(flip_ratio=None, direction='horizontal')[source]

Flip the image & bbox & mask.

If the input dict contains the key “flip”, then the flag will be used, otherwise it will be randomly decided by a ratio specified in the init method.

When random flip is enabled, flip_ratio/direction can either be a float/string or tuple of float/string. There are 3 flip modes:

  • flip_ratio is float, direction is string: the image will be

    direction``ly flipped with probability of ``flip_ratio . E.g., flip_ratio=0.5, direction='horizontal', then image will be horizontally flipped with probability of 0.5.

  • flip_ratio is float, direction is list of string: the image will

    be direction[i]``ly flipped with probability of ``flip_ratio/len(direction). E.g., flip_ratio=0.5, direction=['horizontal', 'vertical'], then image will be horizontally flipped with probability of 0.25, vertically with probability of 0.25.

  • flip_ratio is list of float, direction is list of string:

    given len(flip_ratio) == len(direction), the image will be direction[i]``ly flipped with probability of ``flip_ratio[i]. E.g., flip_ratio=[0.3, 0.5], direction=['horizontal', 'vertical'], then image will be horizontally flipped with probability of 0.3, vertically with probability of 0.5.

Parameters
  • flip_ratio (float | list[float], optional) – The flipping probability. Default: None.

  • direction (str | list[str], optional) – The flipping direction. Options are ‘horizontal’, ‘vertical’, ‘diagonal’. Default: ‘horizontal’. If input is a list, the length must equal flip_ratio. Each element in flip_ratio indicates the flip probability of corresponding direction.

bbox_flip(bboxes, img_shape, direction)[source]

Flip bboxes horizontally.

Parameters
  • bboxes (numpy.ndarray) – Bounding boxes, shape (…, 4*k)

  • img_shape (tuple[int]) – Image shape (height, width)

  • direction (str) – Flip direction. Options are ‘horizontal’, ‘vertical’.

Returns

Flipped bounding boxes.

Return type

numpy.ndarray

class mmdet.datasets.pipelines.RandomShift(shift_ratio=0.5, max_shift_px=32, filter_thr_px=1)[source]

Shift the image and box given shift pixels and probability.

Parameters
  • shift_ratio (float) – Probability of shifts. Default 0.5.

  • max_shift_px (int) – The max pixels for shifting. Default 32.

  • filter_thr_px (int) – The width and height threshold for filtering. The bbox and the rest of the targets below the width and height threshold will be filtered. Default 1.

class mmdet.datasets.pipelines.Resize(img_scale=None, multiscale_mode='range', ratio_range=None, keep_ratio=True, bbox_clip_border=True, backend='cv2', override=False)[source]

Resize images & bbox & mask.

This transform resizes the input image to some scale. Bboxes and masks are then resized with the same scale factor. If the input dict contains the key “scale”, then the scale in the input dict is used, otherwise the specified scale in the init method is used. If the input dict contains the key “scale_factor” (if MultiScaleFlipAug does not give img_scale but scale_factor), the actual scale will be computed by image shape and scale_factor.

img_scale can either be a tuple (single-scale) or a list of tuple (multi-scale). There are 3 multiscale modes:

  • ratio_range is not None: randomly sample a ratio from the ratio range and multiply it with the image scale.

  • ratio_range is None and multiscale_mode == "range": randomly sample a scale from the multiscale range.

  • ratio_range is None and multiscale_mode == "value": randomly sample a scale from multiple scales.

Parameters
  • img_scale (tuple or list[tuple]) – Images scales for resizing.

  • multiscale_mode (str) – Either “range” or “value”.

  • ratio_range (tuple[float]) – (min_ratio, max_ratio)

  • keep_ratio (bool) – Whether to keep the aspect ratio when resizing the image.

  • bbox_clip_border (bool, optional) – Whether clip the objects outside the border of the image. Defaults to True.

  • backend (str) – Image resize backend, choices are ‘cv2’ and ‘pillow’. These two backends generates slightly different results. Defaults to ‘cv2’.

  • override (bool, optional) – Whether to override scale and scale_factor so as to call resize twice. Default False. If True, after the first resizing, the existed scale and scale_factor will be ignored so the second resizing can be allowed. This option is a work-around for multiple times of resize in DETR. Defaults to False.

static random_sample(img_scales)[source]

Randomly sample an img_scale when multiscale_mode=='range'.

Parameters

img_scales (list[tuple]) – Images scale range for sampling. There must be two tuples in img_scales, which specify the lower and upper bound of image scales.

Returns

Returns a tuple (img_scale, None), where img_scale is sampled scale and None is just a placeholder to be consistent with random_select().

Return type

(tuple, None)

static random_sample_ratio(img_scale, ratio_range)[source]

Randomly sample an img_scale when ratio_range is specified.

A ratio will be randomly sampled from the range specified by ratio_range. Then it would be multiplied with img_scale to generate sampled scale.

Parameters
  • img_scale (tuple) – Images scale base to multiply with ratio.

  • ratio_range (tuple[float]) – The minimum and maximum ratio to scale the img_scale.

Returns

Returns a tuple (scale, None), where scale is sampled ratio multiplied with img_scale and None is just a placeholder to be consistent with random_select().

Return type

(tuple, None)

static random_select(img_scales)[source]

Randomly select an img_scale from given candidates.

Parameters

img_scales (list[tuple]) – Images scales for selection.

Returns

Returns a tuple (img_scale, scale_dix), where img_scale is the selected image scale and scale_idx is the selected index in the given candidates.

Return type

(tuple, int)

class mmdet.datasets.pipelines.Rotate(level, scale=1, center=None, img_fill_val=128, seg_ignore_label=255, prob=0.5, max_rotate_angle=30, random_negative_prob=0.5)[source]

Apply Rotate Transformation to image (and its corresponding bbox, mask, segmentation).

Parameters
  • level (int | float) – The level should be in range (0,_MAX_LEVEL].

  • scale (int | float) – Isotropic scale factor. Same in mmcv.imrotate.

  • center (int | float | tuple[float]) – Center point (w, h) of the rotation in the source image. If None, the center of the image will be used. Same in mmcv.imrotate.

  • img_fill_val (int | float | tuple) – The fill value for image border. If float, the same value will be used for all the three channels of image. If tuple, the should be 3 elements (e.g. equals the number of channels for image).

  • seg_ignore_label (int) – The fill value used for segmentation map. Note this value must equals ignore_label in semantic_head of the corresponding config. Default 255.

  • prob (float) – The probability for perform transformation and should be in range 0 to 1.

  • max_rotate_angle (int | float) – The maximum angles for rotate transformation.

  • random_negative_prob (float) – The probability that turns the offset negative.

class mmdet.datasets.pipelines.SegRescale(scale_factor=1, backend='cv2')[source]

Rescale semantic segmentation maps.

Parameters
  • scale_factor (float) – The scale factor of the final output.

  • backend (str) – Image rescale backend, choices are ‘cv2’ and ‘pillow’. These two backends generates slightly different results. Defaults to ‘cv2’.

class mmdet.datasets.pipelines.Shear(level, img_fill_val=128, seg_ignore_label=255, prob=0.5, direction='horizontal', max_shear_magnitude=0.3, random_negative_prob=0.5, interpolation='bilinear')[source]

Apply Shear Transformation to image (and its corresponding bbox, mask, segmentation).

Parameters
  • level (int | float) – The level should be in range [0,_MAX_LEVEL].

  • img_fill_val (int | float | tuple) – The filled values for image border. If float, the same fill value will be used for all the three channels of image. If tuple, the should be 3 elements.

  • seg_ignore_label (int) – The fill value used for segmentation map. Note this value must equals ignore_label in semantic_head of the corresponding config. Default 255.

  • prob (float) – The probability for performing Shear and should be in range [0, 1].

  • direction (str) – The direction for shear, either “horizontal” or “vertical”.

  • max_shear_magnitude (float) – The maximum magnitude for Shear transformation.

  • random_negative_prob (float) – The probability that turns the offset negative. Should be in range [0,1]

  • interpolation (str) – Same as in mmcv.imshear().

class mmdet.datasets.pipelines.ToDataContainer(fields=({'key': 'img', 'stack': True}, {'key': 'gt_bboxes'}, {'key': 'gt_labels'}))[source]

Convert results to mmcv.DataContainer by given fields.

Parameters

fields (Sequence[dict]) – Each field is a dict like dict(key='xxx', **kwargs). The key in result will be converted to mmcv.DataContainer with **kwargs. Default: (dict(key='img', stack=True), dict(key='gt_bboxes'), dict(key='gt_labels')).

class mmdet.datasets.pipelines.ToTensor(keys)[source]

Convert some results to torch.Tensor by given keys.

Parameters

keys (Sequence[str]) – Keys that need to be converted to Tensor.

class mmdet.datasets.pipelines.Translate(level, prob=0.5, img_fill_val=128, seg_ignore_label=255, direction='horizontal', max_translate_offset=250.0, random_negative_prob=0.5, min_size=0)[source]

Translate the images, bboxes, masks and segmentation maps horizontally or vertically.

Parameters
  • level (int | float) – The level for Translate and should be in range [0,_MAX_LEVEL].

  • prob (float) – The probability for performing translation and should be in range [0, 1].

  • img_fill_val (int | float | tuple) – The filled value for image border. If float, the same fill value will be used for all the three channels of image. If tuple, the should be 3 elements (e.g. equals the number of channels for image).

  • seg_ignore_label (int) – The fill value used for segmentation map. Note this value must equals ignore_label in semantic_head of the corresponding config. Default 255.

  • direction (str) – The translate direction, either “horizontal” or “vertical”.

  • max_translate_offset (int | float) – The maximum pixel’s offset for Translate.

  • random_negative_prob (float) – The probability that turns the offset negative.

  • min_size (int | float) – The minimum pixel for filtering invalid bboxes after the translation.

class mmdet.datasets.pipelines.Transpose(keys, order)[source]

Transpose some results by given keys.

Parameters
  • keys (Sequence[str]) – Keys of results to be transposed.

  • order (Sequence[int]) – Order of transpose.

mmdet.datasets.pipelines.to_tensor(data)[source]

Convert objects of various python types to torch.Tensor.

Supported types are: numpy.ndarray, torch.Tensor, Sequence, int and float.

Parameters

data (torch.Tensor | numpy.ndarray | Sequence | int | float) – Data to be converted.

samplers

class mmdet.datasets.samplers.DistributedGroupSampler(dataset, samples_per_gpu=1, num_replicas=None, rank=None, seed=0)[source]

Sampler that restricts data loading to a subset of the dataset.

It is especially useful in conjunction with torch.nn.parallel.DistributedDataParallel. In such case, each process can pass a DistributedSampler instance as a DataLoader sampler, and load a subset of the original dataset that is exclusive to it.

Note

Dataset is assumed to be of constant size.

Parameters
  • dataset – Dataset used for sampling.

  • num_replicas (optional) – Number of processes participating in distributed training.

  • rank (optional) – Rank of the current process within num_replicas.

  • seed (int, optional) – random seed used to shuffle the sampler if shuffle=True. This number should be identical across all processes in the distributed group. Default: 0.

class mmdet.datasets.samplers.DistributedSampler(dataset, num_replicas=None, rank=None, shuffle=True, seed=0)[source]
class mmdet.datasets.samplers.GroupSampler(dataset, samples_per_gpu=1)[source]

api_wrappers

class mmdet.datasets.api_wrappers.COCO(*args: Any, **kwargs: Any)[source]

This class is almost the same as official pycocotools package.

It implements some snake case function aliases. So that the COCO class has the same interface as LVIS class.

mmdet.models

detectors

class mmdet.models.detectors.ATSS(backbone, neck, bbox_head, train_cfg=None, test_cfg=None, pretrained=None, init_cfg=None)[source]

Implementation of ATSS.

class mmdet.models.detectors.AutoAssign(backbone, neck, bbox_head, train_cfg=None, test_cfg=None, pretrained=None)[source]

Implementation of AutoAssign: Differentiable Label Assignment for Dense Object Detection.

class mmdet.models.detectors.BaseDetector(init_cfg=None)[source]

Base class for detectors.

abstract aug_test(imgs, img_metas, **kwargs)[source]

Test function with test time augmentation.

abstract extract_feat(imgs)[source]

Extract features from images.

extract_feats(imgs)[source]

Extract features from multiple images.

Parameters

imgs (list[torch.Tensor]) – A list of images. The images are augmented from the same image but in different ways.

Returns

Features of different images

Return type

list[torch.Tensor]

forward(img, img_metas, return_loss=True, **kwargs)[source]

Calls either forward_train() or forward_test() depending on whether return_loss is True.

Note this setting will change the expected inputs. When return_loss=True, img and img_meta are single-nested (i.e. Tensor and List[dict]), and when resturn_loss=False, img and img_meta should be double nested (i.e. List[Tensor], List[List[dict]]), with the outer list indicating test time augmentations.

forward_test(imgs, img_metas, **kwargs)[source]
Parameters
  • imgs (List[Tensor]) – the outer list indicates test-time augmentations and inner Tensor should have a shape NxCxHxW, which contains all images in the batch.

  • img_metas (List[List[dict]]) – the outer list indicates test-time augs (multiscale, flip, etc.) and the inner list indicates images in a batch.

forward_train(imgs, img_metas, **kwargs)[source]
Parameters
  • img (list[Tensor]) – List of tensors of shape (1, C, H, W). Typically these should be mean centered and std scaled.

  • img_metas (list[dict]) – List of image info dict where each dict has: ‘img_shape’, ‘scale_factor’, ‘flip’, and may also contain ‘filename’, ‘ori_shape’, ‘pad_shape’, and ‘img_norm_cfg’. For details on the values of these keys, see mmdet.datasets.pipelines.Collect.

  • kwargs (keyword arguments) – Specific to concrete implementation.

show_result(img, result, score_thr=0.3, bbox_color=(72, 101, 241), text_color=(72, 101, 241), mask_color=None, thickness=2, font_size=13, win_name='', show=False, wait_time=0, out_file=None)[source]

Draw result over img.

Parameters
  • img (str or Tensor) – The image to be displayed.

  • result (Tensor or tuple) – The results to draw over img bbox_result or (bbox_result, segm_result).

  • score_thr (float, optional) – Minimum score of bboxes to be shown. Default: 0.3.

  • bbox_color (str or tuple(int) or Color) – Color of bbox lines. The tuple of color should be in BGR order. Default: ‘green’

  • text_color (str or tuple(int) or Color) – Color of texts. The tuple of color should be in BGR order. Default: ‘green’

  • mask_color (None or str or tuple(int) or Color) – Color of masks. The tuple of color should be in BGR order. Default: None

  • thickness (int) – Thickness of lines. Default: 2

  • font_size (int) – Font size of texts. Default: 13

  • win_name (str) – The window name. Default: ‘’

  • wait_time (float) – Value of waitKey param. Default: 0.

  • show (bool) – Whether to show the image. Default: False.

  • out_file (str or None) – The filename to write the image. Default: None.

Returns

Only if not show or out_file

Return type

img (Tensor)

train_step(data, optimizer)[source]

The iteration step during training.

This method defines an iteration step during training, except for the back propagation and optimizer updating, which are done in an optimizer hook. Note that in some complicated cases or models, the whole process including back propagation and optimizer updating is also defined in this method, such as GAN.

Parameters
  • data (dict) – The output of dataloader.

  • optimizer (torch.optim.Optimizer | dict) – The optimizer of runner is passed to train_step(). This argument is unused and reserved.

Returns

It should contain at least 3 keys: loss, log_vars, num_samples.

  • loss is a tensor for back propagation, which can be a weighted sum of multiple losses.

  • log_vars contains all the variables to be sent to the logger.

  • num_samples indicates the batch size (when the model is DDP, it means the batch size on each GPU), which is used for averaging the logs.

Return type

dict

val_step(data, optimizer=None)[source]

The iteration step during validation.

This method shares the same signature as train_step(), but used during val epochs. Note that the evaluation after training epochs is not implemented with this method, but an evaluation hook.

property with_bbox

whether the detector has a bbox head

Type

bool

property with_mask

whether the detector has a mask head

Type

bool

property with_neck

whether the detector has a neck

Type

bool

property with_shared_head

whether the detector has a shared head in the RoI Head

Type

bool

class mmdet.models.detectors.CascadeRCNN(backbone, neck=None, rpn_head=None, roi_head=None, train_cfg=None, test_cfg=None, pretrained=None, init_cfg=None)[source]

Implementation of Cascade R-CNN: Delving into High Quality Object Detection

show_result(data, result, **kwargs)[source]

Show prediction results of the detector.

Parameters
  • data (str or np.ndarray) – Image filename or loaded image.

  • result (Tensor or tuple) – The results to draw over img bbox_result or (bbox_result, segm_result).

Returns

The image with bboxes drawn on it.

Return type

np.ndarray

class mmdet.models.detectors.CenterNet(backbone, neck, bbox_head, train_cfg=None, test_cfg=None, pretrained=None, init_cfg=None)[source]

Implementation of CenterNet(Objects as Points)

<https://arxiv.org/abs/1904.07850>.

aug_test(imgs, img_metas, rescale=True)[source]

Augment testing of CenterNet. Aug test must have flipped image pair, and unlike CornerNet, it will perform an averaging operation on the feature map instead of detecting bbox.

Parameters
  • imgs (list[Tensor]) – Augmented images.

  • img_metas (list[list[dict]]) – Meta information of each image, e.g., image size, scaling factor, etc.

  • rescale (bool) – If True, return boxes in original image space. Default: True.

Note

imgs must including flipped image pairs.

Returns

BBox results of each image and classes.

The outer list corresponds to each image. The inner list corresponds to each class.

Return type

list[list[np.ndarray]]

merge_aug_results(aug_results, with_nms)[source]

Merge augmented detection bboxes and score.

Parameters
  • aug_results (list[list[Tensor]]) – Det_bboxes and det_labels of each image.

  • with_nms (bool) – If True, do nms before return boxes.

Returns

(out_bboxes, out_labels)

Return type

tuple

class mmdet.models.detectors.CornerNet(backbone, neck, bbox_head, train_cfg=None, test_cfg=None, pretrained=None, init_cfg=None)[source]

CornerNet.

This detector is the implementation of the paper CornerNet: Detecting Objects as Paired Keypoints .

aug_test(imgs, img_metas, rescale=False)[source]

Augment testing of CornerNet.

Parameters
  • imgs (list[Tensor]) – Augmented images.

  • img_metas (list[list[dict]]) – Meta information of each image, e.g., image size, scaling factor, etc.

  • rescale (bool) – If True, return boxes in original image space. Default: False.

Note

imgs must including flipped image pairs.

Returns

BBox results of each image and classes.

The outer list corresponds to each image. The inner list corresponds to each class.

Return type

list[list[np.ndarray]]

merge_aug_results(aug_results, img_metas)[source]

Merge augmented detection bboxes and score.

Parameters
  • aug_results (list[list[Tensor]]) – Det_bboxes and det_labels of each image.

  • img_metas (list[list[dict]]) – Meta information of each image, e.g., image size, scaling factor, etc.

Returns

(bboxes, labels)

Return type

tuple

class mmdet.models.detectors.DETR(backbone, bbox_head, train_cfg=None, test_cfg=None, pretrained=None, init_cfg=None)[source]

Implementation of DETR: End-to-End Object Detection with Transformers

forward_dummy(img)[source]

Used for computing network flops.

See mmdetection/tools/analysis_tools/get_flops.py

onnx_export(img, img_metas)[source]

Test function for exporting to ONNX, without test time augmentation.

Parameters
  • img (torch.Tensor) – input images.

  • img_metas (list[dict]) – List of image information.

Returns

dets of shape [N, num_det, 5]

and class labels of shape [N, num_det].

Return type

tuple[Tensor, Tensor]

class mmdet.models.detectors.DeformableDETR(*args, **kwargs)[source]
class mmdet.models.detectors.FCOS(backbone, neck, bbox_head, train_cfg=None, test_cfg=None, pretrained=None, init_cfg=None)[source]

Implementation of FCOS

class mmdet.models.detectors.FOVEA(backbone, neck, bbox_head, train_cfg=None, test_cfg=None, pretrained=None, init_cfg=None)[source]

Implementation of FoveaBox

class mmdet.models.detectors.FSAF(backbone, neck, bbox_head, train_cfg=None, test_cfg=None, pretrained=None, init_cfg=None)[source]

Implementation of FSAF

class mmdet.models.detectors.FastRCNN(backbone, roi_head, train_cfg, test_cfg, neck=None, pretrained=None, init_cfg=None)[source]

Implementation of Fast R-CNN

forward_test(imgs, img_metas, proposals, **kwargs)[source]
Parameters
  • imgs (List[Tensor]) – the outer list indicates test-time augmentations and inner Tensor should have a shape NxCxHxW, which contains all images in the batch.

  • img_metas (List[List[dict]]) – the outer list indicates test-time augs (multiscale, flip, etc.) and the inner list indicates images in a batch.

  • proposals (List[List[Tensor]]) – the outer list indicates test-time augs (multiscale, flip, etc.) and the inner list indicates images in a batch. The Tensor should have a shape Px4, where P is the number of proposals.

class mmdet.models.detectors.FasterRCNN(backbone, rpn_head, roi_head, train_cfg, test_cfg, neck=None, pretrained=None, init_cfg=None)[source]

Implementation of Faster R-CNN

class mmdet.models.detectors.GFL(backbone, neck, bbox_head, train_cfg=None, test_cfg=None, pretrained=None, init_cfg=None)[source]
class mmdet.models.detectors.GridRCNN(backbone, rpn_head, roi_head, train_cfg, test_cfg, neck=None, pretrained=None, init_cfg=None)[source]

Grid R-CNN.

This detector is the implementation of: - Grid R-CNN (https://arxiv.org/abs/1811.12030) - Grid R-CNN Plus: Faster and Better (https://arxiv.org/abs/1906.05688)

class mmdet.models.detectors.HybridTaskCascade(**kwargs)[source]

Implementation of HTC

property with_semantic

whether the detector has a semantic head

Type

bool

class mmdet.models.detectors.KnowledgeDistillationSingleStageDetector(backbone, neck, bbox_head, teacher_config, teacher_ckpt=None, eval_teacher=True, train_cfg=None, test_cfg=None, pretrained=None)[source]

Implementation of Distilling the Knowledge in a Neural Network..

Parameters
  • teacher_config (str | dict) – Config file path or the config object of teacher model.

  • teacher_ckpt (str, optional) – Checkpoint path of teacher model. If left as None, the model will not load any weights.

cuda(device=None)[source]

Since teacher_model is registered as a plain object, it is necessary to put the teacher model to cuda when calling cuda function.

forward_train(img, img_metas, gt_bboxes, gt_labels, gt_bboxes_ignore=None)[source]
Parameters
  • img (Tensor) – Input images of shape (N, C, H, W). Typically these should be mean centered and std scaled.

  • img_metas (list[dict]) – A List of image info dict where each dict has: ‘img_shape’, ‘scale_factor’, ‘flip’, and may also contain ‘filename’, ‘ori_shape’, ‘pad_shape’, and ‘img_norm_cfg’. For details on the values of these keys see mmdet.datasets.pipelines.Collect.

  • gt_bboxes (list[Tensor]) – Each item are the truth boxes for each image in [tl_x, tl_y, br_x, br_y] format.

  • gt_labels (list[Tensor]) – Class indices corresponding to each box

  • gt_bboxes_ignore (None | list[Tensor]) – Specify which bounding boxes can be ignored when computing the loss.

Returns

A dictionary of loss components.

Return type

dict[str, Tensor]

train(mode=True)[source]

Set the same train mode for teacher and student model.

class mmdet.models.detectors.MaskRCNN(backbone, rpn_head, roi_head, train_cfg, test_cfg, neck=None, pretrained=None, init_cfg=None)[source]

Implementation of Mask R-CNN

class mmdet.models.detectors.MaskScoringRCNN(backbone, rpn_head, roi_head, train_cfg, test_cfg, neck=None, pretrained=None, init_cfg=None)[source]

Mask Scoring RCNN.

https://arxiv.org/abs/1903.00241

class mmdet.models.detectors.NASFCOS(backbone, neck, bbox_head, train_cfg=None, test_cfg=None, pretrained=None, init_cfg=None)[source]

NAS-FCOS: Fast Neural Architecture Search for Object Detection.

https://arxiv.org/abs/1906.0442

class mmdet.models.detectors.PAA(backbone, neck, bbox_head, train_cfg=None, test_cfg=None, pretrained=None, init_cfg=None)[source]

Implementation of PAA.

class mmdet.models.detectors.PanopticFPN(backbone, neck=None, rpn_head=None, roi_head=None, train_cfg=None, test_cfg=None, pretrained=None, init_cfg=None, semantic_head=None, panoptic_fusion_head=None)[source]

Implementation of Panoptic feature pyramid networks

class mmdet.models.detectors.PointRend(backbone, rpn_head, roi_head, train_cfg, test_cfg, neck=None, pretrained=None, init_cfg=None)[source]

PointRend: Image Segmentation as Rendering

This detector is the implementation of PointRend.

class mmdet.models.detectors.RPN(backbone, neck, rpn_head, train_cfg, test_cfg, pretrained=None, init_cfg=None)[source]

Implementation of Region Proposal Network.

aug_test(imgs, img_metas, rescale=False)[source]

Test function with test time augmentation.

Parameters
  • imgs (list[torch.Tensor]) – List of multiple images

  • img_metas (list[dict]) – List of image information.

  • rescale (bool, optional) – Whether to rescale the results. Defaults to False.

Returns

proposals

Return type

list[np.ndarray]

extract_feat(img)[source]

Extract features.

Parameters

img (torch.Tensor) – Image tensor with shape (n, c, h ,w).

Returns

Multi-level features that may have

different resolutions.

Return type

list[torch.Tensor]

forward_dummy(img)[source]

Dummy forward function.

forward_train(img, img_metas, gt_bboxes=None, gt_bboxes_ignore=None)[source]
Parameters
  • img (Tensor) – Input images of shape (N, C, H, W). Typically these should be mean centered and std scaled.

  • img_metas (list[dict]) – A List of image info dict where each dict has: ‘img_shape’, ‘scale_factor’, ‘flip’, and may also contain ‘filename’, ‘ori_shape’, ‘pad_shape’, and ‘img_norm_cfg’. For details on the values of these keys see mmdet.datasets.pipelines.Collect.

  • gt_bboxes (list[Tensor]) – Each item are the truth boxes for each image in [tl_x, tl_y, br_x, br_y] format.

  • gt_bboxes_ignore (None | list[Tensor]) – Specify which bounding boxes can be ignored when computing the loss.

Returns

A dictionary of loss components.

Return type

dict[str, Tensor]

show_result(data, result, top_k=20, **kwargs)[source]

Show RPN proposals on the image.

Parameters
  • data (str or np.ndarray) – Image filename or loaded image.

  • result (Tensor or tuple) – The results to draw over img bbox_result or (bbox_result, segm_result).

  • top_k (int) – Plot the first k bboxes only if set positive. Default: 20

Returns

The image with bboxes drawn on it.

Return type

np.ndarray

simple_test(img, img_metas, rescale=False)[source]

Test function without test time augmentation.

Parameters
  • imgs (list[torch.Tensor]) – List of multiple images

  • img_metas (list[dict]) – List of image information.

  • rescale (bool, optional) – Whether to rescale the results. Defaults to False.

Returns

proposals

Return type

list[np.ndarray]

class mmdet.models.detectors.RepPointsDetector(backbone, neck, bbox_head, train_cfg=None, test_cfg=None, pretrained=None, init_cfg=None)[source]

RepPoints: Point Set Representation for Object Detection.

This detector is the implementation of: - RepPoints detector (https://arxiv.org/pdf/1904.11490)

class mmdet.models.detectors.RetinaNet(backbone, neck, bbox_head, train_cfg=None, test_cfg=None, pretrained=None, init_cfg=None)[source]

Implementation of RetinaNet

class mmdet.models.detectors.SCNet(**kwargs)[source]

Implementation of SCNet

class mmdet.models.detectors.SOLO(backbone, neck=None, bbox_head=None, mask_head=None, train_cfg=None, test_cfg=None, init_cfg=None, pretrained=None)[source]

SOLO: Segmenting Objects by Locations

class mmdet.models.detectors.SingleStageDetector(backbone, neck=None, bbox_head=None, train_cfg=None, test_cfg=None, pretrained=None, init_cfg=None)[source]

Base class for single-stage detectors.

Single-stage detectors directly and densely predict bounding boxes on the output features of the backbone+neck.

aug_test(imgs, img_metas, rescale=False)[source]

Test function with test time augmentation.

Parameters
  • imgs (list[Tensor]) – the outer list indicates test-time augmentations and inner Tensor should have a shape NxCxHxW, which contains all images in the batch.

  • img_metas (list[list[dict]]) – the outer list indicates test-time augs (multiscale, flip, etc.) and the inner list indicates images in a batch. each dict has image information.

  • rescale (bool, optional) – Whether to rescale the results. Defaults to False.

Returns

BBox results of each image and classes.

The outer list corresponds to each image. The inner list corresponds to each class.

Return type

list[list[np.ndarray]]

extract_feat(img)[source]

Directly extract features from the backbone+neck.

forward_dummy(img)[source]

Used for computing network flops.

See mmdetection/tools/analysis_tools/get_flops.py

forward_train(img, img_metas, gt_bboxes, gt_labels, gt_bboxes_ignore=None)[source]
Parameters
  • img (Tensor) – Input images of shape (N, C, H, W). Typically these should be mean centered and std scaled.

  • img_metas (list[dict]) – A List of image info dict where each dict has: ‘img_shape’, ‘scale_factor’, ‘flip’, and may also contain ‘filename’, ‘ori_shape’, ‘pad_shape’, and ‘img_norm_cfg’. For details on the values of these keys see mmdet.datasets.pipelines.Collect.

  • gt_bboxes (list[Tensor]) – Each item are the truth boxes for each image in [tl_x, tl_y, br_x, br_y] format.

  • gt_labels (list[Tensor]) – Class indices corresponding to each box

  • gt_bboxes_ignore (None | list[Tensor]) – Specify which bounding boxes can be ignored when computing the loss.

Returns

A dictionary of loss components.

Return type

dict[str, Tensor]

onnx_export(img, img_metas)[source]

Test function without test time augmentation.

Parameters
  • img (torch.Tensor) – input images.

  • img_metas (list[dict]) – List of image information.

Returns

dets of shape [N, num_det, 5]

and class labels of shape [N, num_det].

Return type

tuple[Tensor, Tensor]

simple_test(img, img_metas, rescale=False)[source]

Test function without test-time augmentation.

Parameters
  • img (torch.Tensor) – Images with shape (N, C, H, W).

  • img_metas (list[dict]) – List of image information.

  • rescale (bool, optional) – Whether to rescale the results. Defaults to False.

Returns

BBox results of each image and classes.

The outer list corresponds to each image. The inner list corresponds to each class.

Return type

list[list[np.ndarray]]

class mmdet.models.detectors.SparseRCNN(*args, **kwargs)[source]

Implementation of Sparse R-CNN: End-to-End Object Detection with Learnable Proposals

forward_dummy(img)[source]

Used for computing network flops.

See mmdetection/tools/analysis_tools/get_flops.py

forward_train(img, img_metas, gt_bboxes, gt_labels, gt_bboxes_ignore=None, gt_masks=None, proposals=None, **kwargs)[source]

Forward function of SparseR-CNN in train stage.

Parameters
  • img (Tensor) – of shape (N, C, H, W) encoding input images. Typically these should be mean centered and std scaled.

  • img_metas (list[dict]) – list of image info dict where each dict has: ‘img_shape’, ‘scale_factor’, ‘flip’, and may also contain ‘filename’, ‘ori_shape’, ‘pad_shape’, and ‘img_norm_cfg’. For details on the values of these keys see mmdet.datasets.pipelines.Collect.

  • gt_bboxes (list[Tensor]) – Ground truth bboxes for each image with shape (num_gts, 4) in [tl_x, tl_y, br_x, br_y] format.

  • gt_labels (list[Tensor]) – class indices corresponding to each box

  • gt_bboxes_ignore (None | list[Tensor) – specify which bounding boxes can be ignored when computing the loss.

  • gt_masks (List[Tensor], optional) – Segmentation masks for each box. But we don’t support it in this architecture.

  • proposals (List[Tensor], optional) – override rpn proposals with custom proposals. Use when with_rpn is False.

Returns

a dictionary of loss components

Return type

dict[str, Tensor]

simple_test(img, img_metas, rescale=False)[source]

Test function without test time augmentation.

Parameters
  • imgs (list[torch.Tensor]) – List of multiple images

  • img_metas (list[dict]) – List of image information.

  • rescale (bool) – Whether to rescale the results. Defaults to False.

Returns

BBox results of each image and classes.

The outer list corresponds to each image. The inner list corresponds to each class.

Return type

list[list[np.ndarray]]

class mmdet.models.detectors.TridentFasterRCNN(backbone, rpn_head, roi_head, train_cfg, test_cfg, neck=None, pretrained=None, init_cfg=None)[source]

Implementation of TridentNet

aug_test(imgs, img_metas, rescale=False)[source]

Test with augmentations.

If rescale is False, then returned bboxes and masks will fit the scale of imgs[0].

forward_train(img, img_metas, gt_bboxes, gt_labels, **kwargs)[source]

make copies of img and gts to fit multi-branch.

simple_test(img, img_metas, proposals=None, rescale=False)[source]

Test without augmentation.

class mmdet.models.detectors.TwoStageDetector(backbone, neck=None, rpn_head=None, roi_head=None, train_cfg=None, test_cfg=None, pretrained=None, init_cfg=None)[source]

Base class for two-stage detectors.

Two-stage detectors typically consisting of a region proposal network and a task-specific regression head.

async async_simple_test(img, img_meta, proposals=None, rescale=False)[source]

Async test without augmentation.

aug_test(imgs, img_metas, rescale=False)[source]

Test with augmentations.

If rescale is False, then returned bboxes and masks will fit the scale of imgs[0].

extract_feat(img)[source]

Directly extract features from the backbone+neck.

forward_dummy(img)[source]

Used for computing network flops.

See mmdetection/tools/analysis_tools/get_flops.py

forward_train(img, img_metas, gt_bboxes, gt_labels, gt_bboxes_ignore=None, gt_masks=None, proposals=None, **kwargs)[source]
Parameters
  • img (Tensor) – of shape (N, C, H, W) encoding input images. Typically these should be mean centered and std scaled.

  • img_metas (list[dict]) – list of image info dict where each dict has: ‘img_shape’, ‘scale_factor’, ‘flip’, and may also contain ‘filename’, ‘ori_shape’, ‘pad_shape’, and ‘img_norm_cfg’. For details on the values of these keys see mmdet/datasets/pipelines/formatting.py:Collect.

  • gt_bboxes (list[Tensor]) – Ground truth bboxes for each image with shape (num_gts, 4) in [tl_x, tl_y, br_x, br_y] format.

  • gt_labels (list[Tensor]) – class indices corresponding to each box

  • gt_bboxes_ignore (None | list[Tensor]) – specify which bounding boxes can be ignored when computing the loss.

  • gt_masks (None | Tensor) – true segmentation masks for each box used if the architecture supports a segmentation task.

  • proposals – override rpn proposals with custom proposals. Use when with_rpn is False.

Returns

a dictionary of loss components

Return type

dict[str, Tensor]

simple_test(img, img_metas, proposals=None, rescale=False)[source]

Test without augmentation.

property with_roi_head

whether the detector has a RoI head

Type

bool

property with_rpn

whether the detector has RPN

Type

bool

class mmdet.models.detectors.TwoStagePanopticSegmentor(backbone, neck=None, rpn_head=None, roi_head=None, train_cfg=None, test_cfg=None, pretrained=None, init_cfg=None, semantic_head=None, panoptic_fusion_head=None)[source]

Base class of Two-stage Panoptic Segmentor.

As well as the components in TwoStageDetector, Panoptic Segmentor has extra semantic_head and panoptic_fusion_head.

forward_dummy(img)[source]

Used for computing network flops.

See mmdetection/tools/get_flops.py

forward_train(img, img_metas, gt_bboxes, gt_labels, gt_bboxes_ignore=None, gt_masks=None, gt_semantic_seg=None, proposals=None, **kwargs)[source]
Parameters
  • img (Tensor) – of shape (N, C, H, W) encoding input images. Typically these should be mean centered and std scaled.

  • img_metas (list[dict]) – list of image info dict where each dict has: ‘img_shape’, ‘scale_factor’, ‘flip’, and may also contain ‘filename’, ‘ori_shape’, ‘pad_shape’, and ‘img_norm_cfg’. For details on the values of these keys see mmdet/datasets/pipelines/formatting.py:Collect.

  • gt_bboxes (list[Tensor]) – Ground truth bboxes for each image with shape (num_gts, 4) in [tl_x, tl_y, br_x, br_y] format.

  • gt_labels (list[Tensor]) – class indices corresponding to each box

  • gt_bboxes_ignore (None | list[Tensor]) – specify which bounding boxes can be ignored when computing the loss.

  • gt_masks (None | Tensor) – true segmentation masks for each box used if the architecture supports a segmentation task.

  • proposals – override rpn proposals with custom proposals. Use when with_rpn is False.

Returns

a dictionary of loss components

Return type

dict[str, Tensor]

simple_test(img, img_metas, proposals=None, rescale=False)[source]

Test without Augmentation.

simple_test_mask(x, img_metas, det_bboxes, det_labels, rescale=False)[source]

Simple test for mask head without augmentation.

class mmdet.models.detectors.VFNet(backbone, neck, bbox_head, train_cfg=None, test_cfg=None, pretrained=None, init_cfg=None)[source]

Implementation of `VarifocalNet (VFNet).<https://arxiv.org/abs/2008.13367>`_

class mmdet.models.detectors.YOLACT(backbone, neck, bbox_head, segm_head, mask_head, train_cfg=None, test_cfg=None, pretrained=None, init_cfg=None)[source]

Implementation of YOLACT

aug_test(imgs, img_metas, rescale=False)[source]

Test with augmentations.

forward_dummy(img)[source]

Used for computing network flops.

See mmdetection/tools/analysis_tools/get_flops.py

forward_train(img, img_metas, gt_bboxes, gt_labels, gt_bboxes_ignore=None, gt_masks=None)[source]
Parameters
  • img (Tensor) – of shape (N, C, H, W) encoding input images. Typically these should be mean centered and std scaled.

  • img_metas (list[dict]) – list of image info dict where each dict has: ‘img_shape’, ‘scale_factor’, ‘flip’, and may also contain ‘filename’, ‘ori_shape’, ‘pad_shape’, and ‘img_norm_cfg’. For details on the values of these keys see mmdet/datasets/pipelines/formatting.py:Collect.

  • gt_bboxes (list[Tensor]) – Ground truth bboxes for each image with shape (num_gts, 4) in [tl_x, tl_y, br_x, br_y] format.

  • gt_labels (list[Tensor]) – class indices corresponding to each box

  • gt_bboxes_ignore (None | list[Tensor]) – specify which bounding boxes can be ignored when computing the loss.

  • gt_masks (None | Tensor) – true segmentation masks for each box used if the architecture supports a segmentation task.

Returns

a dictionary of loss components

Return type

dict[str, Tensor]

simple_test(img, img_metas, rescale=False)[source]

Test function without test-time augmentation.

class mmdet.models.detectors.YOLOF(backbone, neck, bbox_head, train_cfg=None, test_cfg=None, pretrained=None)[source]

Implementation of You Only Look One-level Feature

class mmdet.models.detectors.YOLOV3(backbone, neck, bbox_head, train_cfg=None, test_cfg=None, pretrained=None, init_cfg=None)[source]
onnx_export(img, img_metas)[source]

Test function for exporting to ONNX, without test time augmentation.

Parameters
  • img (torch.Tensor) – input images.

  • img_metas (list[dict]) – List of image information.

Returns

dets of shape [N, num_det, 5]

and class labels of shape [N, num_det].

Return type

tuple[Tensor, Tensor]

class mmdet.models.detectors.YOLOX(backbone, neck, bbox_head, train_cfg=None, test_cfg=None, pretrained=None, init_cfg=None)[source]

Implementation of YOLOX: Exceeding YOLO Series in 2021

backbones

class mmdet.models.backbones.CSPDarknet(arch='P5', deepen_factor=1.0, widen_factor=1.0, out_indices=(2, 3, 4), frozen_stages=- 1, use_depthwise=False, arch_ovewrite=None, spp_kernal_sizes=(5, 9, 13), conv_cfg=None, norm_cfg={'eps': 0.001, 'momentum': 0.03, 'type': 'BN'}, act_cfg={'type': 'Swish'}, norm_eval=False, init_cfg={'a': 2.23606797749979, 'distribution': 'uniform', 'layer': 'Conv2d', 'mode': 'fan_in', 'nonlinearity': 'leaky_relu', 'type': 'Kaiming'})[source]

CSP-Darknet backbone used in YOLOv5 and YOLOX.

Parameters
  • arch (str) – Architecture of CSP-Darknet, from {P5, P6}. Default: P5.

  • deepen_factor (float) – Depth multiplier, multiply number of channels in each layer by this amount. Default: 1.0.

  • widen_factor (float) – Width multiplier, multiply number of blocks in CSP layer by this amount. Default: 1.0.

  • out_indices (Sequence[int]) – Output from which stages. Default: (2, 3, 4).

  • frozen_stages (int) – Stages to be frozen (stop grad and set eval mode). -1 means not freezing any parameters. Default: -1.

  • use_depthwise (bool) – Whether to use depthwise separable convolution. Default: False.

  • arch_ovewrite (list) – Overwrite default arch settings. Default: None.

  • spp_kernal_sizes – (tuple[int]): Sequential of kernel sizes of SPP layers. Default: (5, 9, 13).

  • conv_cfg (dict) – Config dict for convolution layer. Default: None.

  • norm_cfg (dict) – Dictionary to construct and config norm layer. Default: dict(type=’BN’, requires_grad=True).

  • act_cfg (dict) – Config dict for activation layer. Default: dict(type=’LeakyReLU’, negative_slope=0.1).

  • norm_eval (bool) – Whether to set norm layers to eval mode, namely, freeze running stats (mean and var). Note: Effect on Batch Norm and its variants only.

  • init_cfg (dict or list[dict], optional) – Initialization config dict. Default: None.

Example

>>> from mmdet.models import CSPDarknet
>>> import torch
>>> self = CSPDarknet(depth=53)
>>> self.eval()
>>> inputs = torch.rand(1, 3, 416, 416)
>>> level_outputs = self.forward(inputs)
>>> for level_out in level_outputs:
...     print(tuple(level_out.shape))
...
(1, 256, 52, 52)
(1, 512, 26, 26)
(1, 1024, 13, 13)
forward(x)[source]

Defines the computation performed at every call.

Should be overridden by all subclasses.

Note

Although the recipe for forward pass needs to be defined within this function, one should call the Module instance afterwards instead of this since the former takes care of running the registered hooks while the latter silently ignores them.

train(mode=True)[source]

Sets the module in training mode.

This has any effect only on certain modules. See documentations of particular modules for details of their behaviors in training/evaluation mode, if they are affected, e.g. Dropout, BatchNorm, etc.

Parameters

mode (bool) – whether to set training mode (True) or evaluation mode (False). Default: True.

Returns

self

Return type

Module

class mmdet.models.backbones.Darknet(depth=53, out_indices=(3, 4, 5), frozen_stages=- 1, conv_cfg=None, norm_cfg={'requires_grad': True, 'type': 'BN'}, act_cfg={'negative_slope': 0.1, 'type': 'LeakyReLU'}, norm_eval=True, pretrained=None, init_cfg=None)[source]

Darknet backbone.

Parameters
  • depth (int) – Depth of Darknet. Currently only support 53.

  • out_indices (Sequence[int]) – Output from which stages.

  • frozen_stages (int) – Stages to be frozen (stop grad and set eval mode). -1 means not freezing any parameters. Default: -1.

  • conv_cfg (dict) – Config dict for convolution layer. Default: None.

  • norm_cfg (dict) – Dictionary to construct and config norm layer. Default: dict(type=’BN’, requires_grad=True)

  • act_cfg (dict) – Config dict for activation layer. Default: dict(type=’LeakyReLU’, negative_slope=0.1).

  • norm_eval (bool) – Whether to set norm layers to eval mode, namely, freeze running stats (mean and var). Note: Effect on Batch Norm and its variants only.

  • pretrained (str, optional) – model pretrained path. Default: None

  • init_cfg (dict or list[dict], optional) – Initialization config dict. Default: None

Example

>>> from mmdet.models import Darknet
>>> import torch
>>> self = Darknet(depth=53)
>>> self.eval()
>>> inputs = torch.rand(1, 3, 416, 416)
>>> level_outputs = self.forward(inputs)
>>> for level_out in level_outputs:
...     print(tuple(level_out.shape))
...
(1, 256, 52, 52)
(1, 512, 26, 26)
(1, 1024, 13, 13)
forward(x)[source]

Defines the computation performed at every call.

Should be overridden by all subclasses.

Note

Although the recipe for forward pass needs to be defined within this function, one should call the Module instance afterwards instead of this since the former takes care of running the registered hooks while the latter silently ignores them.

static make_conv_res_block(in_channels, out_channels, res_repeat, conv_cfg=None, norm_cfg={'requires_grad': True, 'type': 'BN'}, act_cfg={'negative_slope': 0.1, 'type': 'LeakyReLU'})[source]

In Darknet backbone, ConvLayer is usually followed by ResBlock. This function will make that. The Conv layers always have 3x3 filters with stride=2. The number of the filters in Conv layer is the same as the out channels of the ResBlock.

Parameters
  • in_channels (int) – The number of input channels.

  • out_channels (int) – The number of output channels.

  • res_repeat (int) – The number of ResBlocks.

  • conv_cfg (dict) – Config dict for convolution layer. Default: None.

  • norm_cfg (dict) – Dictionary to construct and config norm layer. Default: dict(type=’BN’, requires_grad=True)

  • act_cfg (dict) – Config dict for activation layer. Default: dict(type=’LeakyReLU’, negative_slope=0.1).

train(mode=True)[source]

Sets the module in training mode.

This has any effect only on certain modules. See documentations of particular modules for details of their behaviors in training/evaluation mode, if they are affected, e.g. Dropout, BatchNorm, etc.

Parameters

mode (bool) – whether to set training mode (True) or evaluation mode (False). Default: True.

Returns

self

Return type

Module

class mmdet.models.backbones.DetectoRS_ResNeXt(groups=1, base_width=4, **kwargs)[source]

ResNeXt backbone for DetectoRS.

Parameters
  • groups (int) – The number of groups in ResNeXt.

  • base_width (int) – The base width of ResNeXt.

make_res_layer(**kwargs)[source]

Pack all blocks in a stage into a ResLayer for DetectoRS.

class mmdet.models.backbones.DetectoRS_ResNet(sac=None, stage_with_sac=(False, False, False, False), rfp_inplanes=None, output_img=False, pretrained=None, init_cfg=None, **kwargs)[source]

ResNet backbone for DetectoRS.

Parameters
  • sac (dict, optional) – Dictionary to construct SAC (Switchable Atrous Convolution). Default: None.

  • stage_with_sac (list) – Which stage to use sac. Default: (False, False, False, False).

  • rfp_inplanes (int, optional) – The number of channels from RFP. Default: None. If specified, an additional conv layer will be added for rfp_feat. Otherwise, the structure is the same as base class.

  • output_img (bool) – If True, the input image will be inserted into the starting position of output. Default: False.

forward(x)[source]

Forward function.

init_weights()[source]

Initialize the weights.

make_res_layer(**kwargs)[source]

Pack all blocks in a stage into a ResLayer for DetectoRS.

rfp_forward(x, rfp_feats)[source]

Forward function for RFP.

class mmdet.models.backbones.HRNet(extra, in_channels=3, conv_cfg=None, norm_cfg={'type': 'BN'}, norm_eval=True, with_cp=False, zero_init_residual=False, multiscale_output=True, pretrained=None, init_cfg=None)[source]

HRNet backbone.

High-Resolution Representations for Labeling Pixels and Regions arXiv:.

Parameters
  • extra (dict) –

    Detailed configuration for each stage of HRNet. There must be 4 stages, the configuration for each stage must have 5 keys:

    • num_modules(int): The number of HRModule in this stage.

    • num_branches(int): The number of branches in the HRModule.

    • block(str): The type of convolution block.

    • num_blocks(tuple): The number of blocks in each branch.

      The length must be equal to num_branches.

    • num_channels(tuple): The number of channels in each branch.

      The length must be equal to num_branches.

  • in_channels (int) – Number of input image channels. Default: 3.

  • conv_cfg (dict) – Dictionary to construct and config conv layer.

  • norm_cfg (dict) – Dictionary to construct and config norm layer.

  • norm_eval (bool) – Whether to set norm layers to eval mode, namely, freeze running stats (mean and var). Note: Effect on Batch Norm and its variants only. Default: True.

  • with_cp (bool) – Use checkpoint or not. Using checkpoint will save some memory while slowing down the training speed. Default: False.

  • zero_init_residual (bool) – Whether to use zero init for last norm layer in resblocks to let them behave as identity. Default: False.

  • multiscale_output (bool) – Whether to output multi-level features produced by multiple branches. If False, only the first level feature will be output. Default: True.

  • pretrained (str, optional) – Model pretrained path. Default: None.

  • init_cfg (dict or list[dict], optional) – Initialization config dict. Default: None.

Example

>>> from mmdet.models import HRNet
>>> import torch
>>> extra = dict(
>>>     stage1=dict(
>>>         num_modules=1,
>>>         num_branches=1,
>>>         block='BOTTLENECK',
>>>         num_blocks=(4, ),
>>>         num_channels=(64, )),
>>>     stage2=dict(
>>>         num_modules=1,
>>>         num_branches=2,
>>>         block='BASIC',
>>>         num_blocks=(4, 4),
>>>         num_channels=(32, 64)),
>>>     stage3=dict(
>>>         num_modules=4,
>>>         num_branches=3,
>>>         block='BASIC',
>>>         num_blocks=(4, 4, 4),
>>>         num_channels=(32, 64, 128)),
>>>     stage4=dict(
>>>         num_modules=3,
>>>         num_branches=4,
>>>         block='BASIC',
>>>         num_blocks=(4, 4, 4, 4),
>>>         num_channels=(32, 64, 128, 256)))
>>> self = HRNet(extra, in_channels=1)
>>> self.eval()
>>> inputs = torch.rand(1, 1, 32, 32)
>>> level_outputs = self.forward(inputs)
>>> for level_out in level_outputs:
...     print(tuple(level_out.shape))
(1, 32, 8, 8)
(1, 64, 4, 4)
(1, 128, 2, 2)
(1, 256, 1, 1)
forward(x)[source]

Forward function.

property norm1

the normalization layer named “norm1”

Type

nn.Module

property norm2

the normalization layer named “norm2”

Type

nn.Module

train(mode=True)[source]

Convert the model into training mode will keeping the normalization layer freezed.

class mmdet.models.backbones.HourglassNet(downsample_times=5, num_stacks=2, stage_channels=(256, 256, 384, 384, 384, 512), stage_blocks=(2, 2, 2, 2, 2, 4), feat_channel=256, norm_cfg={'requires_grad': True, 'type': 'BN'}, pretrained=None, init_cfg=None)[source]

HourglassNet backbone.

Stacked Hourglass Networks for Human Pose Estimation. More details can be found in the paper .

Parameters
  • downsample_times (int) – Downsample times in a HourglassModule.

  • num_stacks (int) – Number of HourglassModule modules stacked, 1 for Hourglass-52, 2 for Hourglass-104.

  • stage_channels (list[int]) – Feature channel of each sub-module in a HourglassModule.

  • stage_blocks (list[int]) – Number of sub-modules stacked in a HourglassModule.

  • feat_channel (int) – Feature channel of conv after a HourglassModule.

  • norm_cfg (dict) – Dictionary to construct and config norm layer.

  • pretrained (str, optional) – model pretrained path. Default: None

  • init_cfg (dict or list[dict], optional) – Initialization config dict. Default: None

Example

>>> from mmdet.models import HourglassNet
>>> import torch
>>> self = HourglassNet()
>>> self.eval()
>>> inputs = torch.rand(1, 3, 511, 511)
>>> level_outputs = self.forward(inputs)
>>> for level_output in level_outputs:
...     print(tuple(level_output.shape))
(1, 256, 128, 128)
(1, 256, 128, 128)
forward(x)[source]

Forward function.

init_weights()[source]

Init module weights.

class mmdet.models.backbones.MobileNetV2(widen_factor=1.0, out_indices=(1, 2, 4, 7), frozen_stages=- 1, conv_cfg=None, norm_cfg={'type': 'BN'}, act_cfg={'type': 'ReLU6'}, norm_eval=False, with_cp=False, pretrained=None, init_cfg=None)[source]

MobileNetV2 backbone.

Parameters
  • widen_factor (float) – Width multiplier, multiply number of channels in each layer by this amount. Default: 1.0.

  • out_indices (Sequence[int], optional) – Output from which stages. Default: (1, 2, 4, 7).

  • frozen_stages (int) – Stages to be frozen (all param fixed). Default: -1, which means not freezing any parameters.

  • conv_cfg (dict, optional) – Config dict for convolution layer. Default: None, which means using conv2d.

  • norm_cfg (dict) – Config dict for normalization layer. Default: dict(type=’BN’).

  • act_cfg (dict) – Config dict for activation layer. Default: dict(type=’ReLU6’).

  • norm_eval (bool) – Whether to set norm layers to eval mode, namely, freeze running stats (mean and var). Note: Effect on Batch Norm and its variants only. Default: False.

  • with_cp (bool) – Use checkpoint or not. Using checkpoint will save some memory while slowing down the training speed. Default: False.

  • pretrained (str, optional) – model pretrained path. Default: None

  • init_cfg (dict or list[dict], optional) – Initialization config dict. Default: None

forward(x)[source]

Forward function.

make_layer(out_channels, num_blocks, stride, expand_ratio)[source]

Stack InvertedResidual blocks to build a layer for MobileNetV2.

Parameters
  • out_channels (int) – out_channels of block.

  • num_blocks (int) – number of blocks.

  • stride (int) – stride of the first block. Default: 1

  • expand_ratio (int) – Expand the number of channels of the hidden layer in InvertedResidual by this ratio. Default: 6.

train(mode=True)[source]

Convert the model into training mode while keep normalization layer frozen.

class mmdet.models.backbones.PyramidVisionTransformer(pretrain_img_size=224, in_channels=3, embed_dims=64, num_stages=4, num_layers=[3, 4, 6, 3], num_heads=[1, 2, 5, 8], patch_sizes=[4, 2, 2, 2], strides=[4, 2, 2, 2], paddings=[0, 0, 0, 0], sr_ratios=[8, 4, 2, 1], out_indices=(0, 1, 2, 3), mlp_ratios=[8, 8, 4, 4], qkv_bias=True, drop_rate=0.0, attn_drop_rate=0.0, drop_path_rate=0.1, use_abs_pos_embed=True, norm_after_stage=False, use_conv_ffn=False, act_cfg={'type': 'GELU'}, norm_cfg={'eps': 1e-06, 'type': 'LN'}, pretrained=None, convert_weights=True, init_cfg=None)[source]

Pyramid Vision Transformer (PVT)

Implementation of Pyramid Vision Transformer: A Versatile Backbone for Dense Prediction without Convolutions.

Parameters
  • pretrain_img_size (int | tuple[int]) – The size of input image when pretrain. Defaults: 224.

  • in_channels (int) – Number of input channels. Default: 3.

  • embed_dims (int) – Embedding dimension. Default: 64.

  • num_stags (int) – The num of stages. Default: 4.

  • num_layers (Sequence[int]) – The layer number of each transformer encode layer. Default: [3, 4, 6, 3].

  • num_heads (Sequence[int]) – The attention heads of each transformer encode layer. Default: [1, 2, 5, 8].

  • patch_sizes (Sequence[int]) – The patch_size of each patch embedding. Default: [4, 2, 2, 2].

  • strides (Sequence[int]) – The stride of each patch embedding. Default: [4, 2, 2, 2].

  • paddings (Sequence[int]) – The padding of each patch embedding. Default: [0, 0, 0, 0].

  • sr_ratios (Sequence[int]) – The spatial reduction rate of each transformer encode layer. Default: [8, 4, 2, 1].

  • out_indices (Sequence[int] | int) – Output from which stages. Default: (0, 1, 2, 3).

  • mlp_ratios (Sequence[int]) – The ratio of the mlp hidden dim to the embedding dim of each transformer encode layer. Default: [8, 8, 4, 4].

  • qkv_bias (bool) – Enable bias for qkv if True. Default: True.

  • drop_rate (float) – Probability of an element to be zeroed. Default 0.0.

  • attn_drop_rate (float) – The drop out rate for attention layer. Default 0.0.

  • drop_path_rate (float) – stochastic depth rate. Default 0.1.

  • use_abs_pos_embed (bool) – If True, add absolute position embedding to the patch embedding. Defaults: True.

  • use_conv_ffn (bool) – If True, use Convolutional FFN to replace FFN. Default: False.

  • act_cfg (dict) – The activation config for FFNs. Default: dict(type=’GELU’).

  • norm_cfg (dict) – Config dict for normalization layer. Default: dict(type=’LN’).

  • pretrained (str, optional) – model pretrained path. Default: None.

  • convert_weights (bool) – The flag indicates whether the pre-trained model is from the original repo. We may need to convert some keys to make it compatible. Default: True.

  • init_cfg (dict or list[dict], optional) – Initialization config dict. Default: None.

forward(x)[source]

Defines the computation performed at every call.

Should be overridden by all subclasses.

Note

Although the recipe for forward pass needs to be defined within this function, one should call the Module instance afterwards instead of this since the former takes care of running the registered hooks while the latter silently ignores them.

init_weights()[source]

Initialize the weights.

class mmdet.models.backbones.PyramidVisionTransformerV2(**kwargs)[source]

Implementation of PVTv2: Improved Baselines with Pyramid Vision Transformer.

class mmdet.models.backbones.RegNet(arch, in_channels=3, stem_channels=32, base_channels=32, strides=(2, 2, 2, 2), dilations=(1, 1, 1, 1), out_indices=(0, 1, 2, 3), style='pytorch', deep_stem=False, avg_down=False, frozen_stages=- 1, conv_cfg=None, norm_cfg={'requires_grad': True, 'type': 'BN'}, norm_eval=True, dcn=None, stage_with_dcn=(False, False, False, False), plugins=None, with_cp=False, zero_init_residual=True, pretrained=None, init_cfg=None)[source]

RegNet backbone.

More details can be found in paper .

Parameters
  • arch (dict) –

    The parameter of RegNets.

    • w0 (int): initial width

    • wa (float): slope of width

    • wm (float): quantization parameter to quantize the width

    • depth (int): depth of the backbone

    • group_w (int): width of group

    • bot_mul (float): bottleneck ratio, i.e. expansion of bottleneck.

  • strides (Sequence[int]) – Strides of the first block of each stage.

  • base_channels (int) – Base channels after stem layer.

  • in_channels (int) – Number of input image channels. Default: 3.

  • dilations (Sequence[int]) – Dilation of each stage.

  • out_indices (Sequence[int]) – Output from which stages.

  • style (str) – pytorch or caffe. If set to “pytorch”, the stride-two layer is the 3x3 conv layer, otherwise the stride-two layer is the first 1x1 conv layer.

  • frozen_stages (int) – Stages to be frozen (all param fixed). -1 means not freezing any parameters.

  • norm_cfg (dict) – dictionary to construct and config norm layer.

  • norm_eval (bool) – Whether to set norm layers to eval mode, namely, freeze running stats (mean and var). Note: Effect on Batch Norm and its variants only.

  • with_cp (bool) – Use checkpoint or not. Using checkpoint will save some memory while slowing down the training speed.

  • zero_init_residual (bool) – whether to use zero init for last norm layer in resblocks to let them behave as identity.

  • pretrained (str, optional) – model pretrained path. Default: None

  • init_cfg (dict or list[dict], optional) – Initialization config dict. Default: None

Example

>>> from mmdet.models import RegNet
>>> import torch
>>> self = RegNet(
        arch=dict(
            w0=88,
            wa=26.31,
            wm=2.25,
            group_w=48,
            depth=25,
            bot_mul=1.0))
>>> self.eval()
>>> inputs = torch.rand(1, 3, 32, 32)
>>> level_outputs = self.forward(inputs)
>>> for level_out in level_outputs:
...     print(tuple(level_out.shape))
(1, 96, 8, 8)
(1, 192, 4, 4)
(1, 432, 2, 2)
(1, 1008, 1, 1)
adjust_width_group(widths, bottleneck_ratio, groups)[source]

Adjusts the compatibility of widths and groups.

Parameters
  • widths (list[int]) – Width of each stage.

  • bottleneck_ratio (float) – Bottleneck ratio.

  • groups (int) – number of groups in each stage

Returns

The adjusted widths and groups of each stage.

Return type

tuple(list)

forward(x)[source]

Forward function.

generate_regnet(initial_width, width_slope, width_parameter, depth, divisor=8)[source]

Generates per block width from RegNet parameters.

Parameters
  • initial_width ([int]) – Initial width of the backbone

  • width_slope ([float]) – Slope of the quantized linear function

  • width_parameter ([int]) – Parameter used to quantize the width.

  • depth ([int]) – Depth of the backbone.

  • divisor (int, optional) – The divisor of channels. Defaults to 8.

Returns

return a list of widths of each stage and the number of stages

Return type

list, int

get_stages_from_blocks(widths)[source]

Gets widths/stage_blocks of network at each stage.

Parameters

widths (list[int]) – Width in each stage.

Returns

width and depth of each stage

Return type

tuple(list)

static quantize_float(number, divisor)[source]

Converts a float to closest non-zero int divisible by divisor.

Parameters
  • number (int) – Original number to be quantized.

  • divisor (int) – Divisor used to quantize the number.

Returns

quantized number that is divisible by devisor.

Return type

int

class mmdet.models.backbones.Res2Net(scales=4, base_width=26, style='pytorch', deep_stem=True, avg_down=True, pretrained=None, init_cfg=None, **kwargs)[source]

Res2Net backbone.

Parameters
  • scales (int) – Scales used in Res2Net. Default: 4

  • base_width (int) – Basic width of each scale. Default: 26

  • depth (int) – Depth of res2net, from {50, 101, 152}.

  • in_channels (int) – Number of input image channels. Default: 3.

  • num_stages (int) – Res2net stages. Default: 4.

  • strides (Sequence[int]) – Strides of the first block of each stage.

  • dilations (Sequence[int]) – Dilation of each stage.

  • out_indices (Sequence[int]) – Output from which stages.

  • style (str) – pytorch or caffe. If set to “pytorch”, the stride-two layer is the 3x3 conv layer, otherwise the stride-two layer is the first 1x1 conv layer.

  • deep_stem (bool) – Replace 7x7 conv in input stem with 3 3x3 conv

  • avg_down (bool) – Use AvgPool instead of stride conv when downsampling in the bottle2neck.

  • frozen_stages (int) – Stages to be frozen (stop grad and set eval mode). -1 means not freezing any parameters.

  • norm_cfg (dict) – Dictionary to construct and config norm layer.

  • norm_eval (bool) – Whether to set norm layers to eval mode, namely, freeze running stats (mean and var). Note: Effect on Batch Norm and its variants only.

  • plugins (list[dict]) –

    List of plugins for stages, each dict contains:

    • cfg (dict, required): Cfg dict to build plugin.

    • position (str, required): Position inside block to insert plugin, options are ‘after_conv1’, ‘after_conv2’, ‘after_conv3’.

    • stages (tuple[bool], optional): Stages to apply plugin, length should be same as ‘num_stages’.

  • with_cp (bool) – Use checkpoint or not. Using checkpoint will save some memory while slowing down the training speed.

  • zero_init_residual (bool) – Whether to use zero init for last norm layer in resblocks to let them behave as identity.

  • pretrained (str, optional) – model pretrained path. Default: None

  • init_cfg (dict or list[dict], optional) – Initialization config dict. Default: None

Example

>>> from mmdet.models import Res2Net
>>> import torch
>>> self = Res2Net(depth=50, scales=4, base_width=26)
>>> self.eval()
>>> inputs = torch.rand(1, 3, 32, 32)
>>> level_outputs = self.forward(inputs)
>>> for level_out in level_outputs:
...     print(tuple(level_out.shape))
(1, 256, 8, 8)
(1, 512, 4, 4)
(1, 1024, 2, 2)
(1, 2048, 1, 1)
make_res_layer(**kwargs)[source]

Pack all blocks in a stage into a ResLayer.

class mmdet.models.backbones.ResNeSt(groups=1, base_width=4, radix=2, reduction_factor=4, avg_down_stride=True, **kwargs)[source]

ResNeSt backbone.

Parameters
  • groups (int) – Number of groups of Bottleneck. Default: 1

  • base_width (int) – Base width of Bottleneck. Default: 4

  • radix (int) – Radix of SplitAttentionConv2d. Default: 2

  • reduction_factor (int) – Reduction factor of inter_channels in SplitAttentionConv2d. Default: 4.

  • avg_down_stride (bool) – Whether to use average pool for stride in Bottleneck. Default: True.

  • kwargs (dict) – Keyword arguments for ResNet.

make_res_layer(**kwargs)[source]

Pack all blocks in a stage into a ResLayer.

class mmdet.models.backbones.ResNeXt(groups=1, base_width=4, **kwargs)[source]

ResNeXt backbone.

Parameters
  • depth (int) – Depth of resnet, from {18, 34, 50, 101, 152}.

  • in_channels (int) – Number of input image channels. Default: 3.

  • num_stages (int) – Resnet stages. Default: 4.

  • groups (int) – Group of resnext.

  • base_width (int) – Base width of resnext.

  • strides (Sequence[int]) – Strides of the first block of each stage.

  • dilations (Sequence[int]) – Dilation of each stage.

  • out_indices (Sequence[int]) – Output from which stages.

  • style (str) – pytorch or caffe. If set to “pytorch”, the stride-two layer is the 3x3 conv layer, otherwise the stride-two layer is the first 1x1 conv layer.

  • frozen_stages (int) – Stages to be frozen (all param fixed). -1 means not freezing any parameters.

  • norm_cfg (dict) – dictionary to construct and config norm layer.

  • norm_eval (bool) – Whether to set norm layers to eval mode, namely, freeze running stats (mean and var). Note: Effect on Batch Norm and its variants only.

  • with_cp (bool) – Use checkpoint or not. Using checkpoint will save some memory while slowing down the training speed.

  • zero_init_residual (bool) – whether to use zero init for last norm layer in resblocks to let them behave as identity.

make_res_layer(**kwargs)[source]

Pack all blocks in a stage into a ResLayer

class mmdet.models.backbones.ResNet(depth, in_channels=3, stem_channels=None, base_channels=64, num_stages=4, strides=(1, 2, 2, 2), dilations=(1, 1, 1, 1), out_indices=(0, 1, 2, 3), style='pytorch', deep_stem=False, avg_down=False, frozen_stages=- 1, conv_cfg=None, norm_cfg={'requires_grad': True, 'type': 'BN'}, norm_eval=True, dcn=None, stage_with_dcn=(False, False, False, False), plugins=None, with_cp=False, zero_init_residual=True, pretrained=None, init_cfg=None)[source]

ResNet backbone.

Parameters
  • depth (int) – Depth of resnet, from {18, 34, 50, 101, 152}.

  • stem_channels (int | None) – Number of stem channels. If not specified, it will be the same as base_channels. Default: None.

  • base_channels (int) – Number of base channels of res layer. Default: 64.

  • in_channels (int) – Number of input image channels. Default: 3.

  • num_stages (int) – Resnet stages. Default: 4.

  • strides (Sequence[int]) – Strides of the first block of each stage.

  • dilations (Sequence[int]) – Dilation of each stage.

  • out_indices (Sequence[int]) – Output from which stages.

  • style (str) – pytorch or caffe. If set to “pytorch”, the stride-two layer is the 3x3 conv layer, otherwise the stride-two layer is the first 1x1 conv layer.

  • deep_stem (bool) – Replace 7x7 conv in input stem with 3 3x3 conv

  • avg_down (bool) – Use AvgPool instead of stride conv when downsampling in the bottleneck.

  • frozen_stages (int) – Stages to be frozen (stop grad and set eval mode). -1 means not freezing any parameters.

  • norm_cfg (dict) – Dictionary to construct and config norm layer.

  • norm_eval (bool) – Whether to set norm layers to eval mode, namely, freeze running stats (mean and var). Note: Effect on Batch Norm and its variants only.

  • plugins (list[dict]) –

    List of plugins for stages, each dict contains:

    • cfg (dict, required): Cfg dict to build plugin.

    • position (str, required): Position inside block to insert plugin, options are ‘after_conv1’, ‘after_conv2’, ‘after_conv3’.

    • stages (tuple[bool], optional): Stages to apply plugin, length should be same as ‘num_stages’.

  • with_cp (bool) – Use checkpoint or not. Using checkpoint will save some memory while slowing down the training speed.

  • zero_init_residual (bool) – Whether to use zero init for last norm layer in resblocks to let them behave as identity.

  • pretrained (str, optional) – model pretrained path. Default: None

  • init_cfg (dict or list[dict], optional) – Initialization config dict. Default: None

Example

>>> from mmdet.models import ResNet
>>> import torch
>>> self = ResNet(depth=18)
>>> self.eval()
>>> inputs = torch.rand(1, 3, 32, 32)
>>> level_outputs = self.forward(inputs)
>>> for level_out in level_outputs:
...     print(tuple(level_out.shape))
(1, 64, 8, 8)
(1, 128, 4, 4)
(1, 256, 2, 2)
(1, 512, 1, 1)
forward(x)[source]

Forward function.

make_res_layer(**kwargs)[source]

Pack all blocks in a stage into a ResLayer.

make_stage_plugins(plugins, stage_idx)[source]

Make plugins for ResNet stage_idx th stage.

Currently we support to insert context_block, empirical_attention_block, nonlocal_block into the backbone like ResNet/ResNeXt. They could be inserted after conv1/conv2/conv3 of Bottleneck.

An example of plugins format could be:

Examples

>>> plugins=[
...     dict(cfg=dict(type='xxx', arg1='xxx'),
...          stages=(False, True, True, True),
...          position='after_conv2'),
...     dict(cfg=dict(type='yyy'),
...          stages=(True, True, True, True),
...          position='after_conv3'),
...     dict(cfg=dict(type='zzz', postfix='1'),
...          stages=(True, True, True, True),
...          position='after_conv3'),
...     dict(cfg=dict(type='zzz', postfix='2'),
...          stages=(True, True, True, True),
...          position='after_conv3')
... ]
>>> self = ResNet(depth=18)
>>> stage_plugins = self.make_stage_plugins(plugins, 0)
>>> assert len(stage_plugins) == 3

Suppose stage_idx=0, the structure of blocks in the stage would be:

conv1-> conv2->conv3->yyy->zzz1->zzz2

Suppose ‘stage_idx=1’, the structure of blocks in the stage would be:

conv1-> conv2->xxx->conv3->yyy->zzz1->zzz2

If stages is missing, the plugin would be applied to all stages.

Parameters
  • plugins (list[dict]) – List of plugins cfg to build. The postfix is required if multiple same type plugins are inserted.

  • stage_idx (int) – Index of stage to build

Returns

Plugins for current stage

Return type

list[dict]

property norm1

the normalization layer named “norm1”

Type

nn.Module

train(mode=True)[source]

Convert the model into training mode while keep normalization layer freezed.

class mmdet.models.backbones.ResNetV1d(**kwargs)[source]

ResNetV1d variant described in Bag of Tricks.

Compared with default ResNet(ResNetV1b), ResNetV1d replaces the 7x7 conv in the input stem with three 3x3 convs. And in the downsampling block, a 2x2 avg_pool with stride 2 is added before conv, whose stride is changed to 1.

class mmdet.models.backbones.SSDVGG(depth, with_last_pool=False, ceil_mode=True, out_indices=(3, 4), out_feature_indices=(22, 34), pretrained=None, init_cfg=None, input_size=None, l2_norm_scale=None)[source]

VGG Backbone network for single-shot-detection.

Parameters
  • depth (int) – Depth of vgg, from {11, 13, 16, 19}.

  • with_last_pool (bool) – Whether to add a pooling layer at the last of the model

  • ceil_mode (bool) – When True, will use ceil instead of floor to compute the output shape.

  • out_indices (Sequence[int]) – Output from which stages.

  • out_feature_indices (Sequence[int]) – Output from which feature map.

  • pretrained (str, optional) – model pretrained path. Default: None

  • init_cfg (dict or list[dict], optional) – Initialization config dict. Default: None

  • input_size (int, optional) – Deprecated argumment. Width and height of input, from {300, 512}.

  • l2_norm_scale (float, optional) – Deprecated argumment. L2 normalization layer init scale.

Example

>>> self = SSDVGG(input_size=300, depth=11)
>>> self.eval()
>>> inputs = torch.rand(1, 3, 300, 300)
>>> level_outputs = self.forward(inputs)
>>> for level_out in level_outputs:
...     print(tuple(level_out.shape))
(1, 1024, 19, 19)
(1, 512, 10, 10)
(1, 256, 5, 5)
(1, 256, 3, 3)
(1, 256, 1, 1)
forward(x)[source]

Forward function.

init_weights(pretrained=None)[source]

Initialize the weights.

class mmdet.models.backbones.SwinTransformer(pretrain_img_size=224, in_channels=3, embed_dims=96, patch_size=4, window_size=7, mlp_ratio=4, depths=(2, 2, 6, 2), num_heads=(3, 6, 12, 24), strides=(4, 2, 2, 2), out_indices=(0, 1, 2, 3), qkv_bias=True, qk_scale=None, patch_norm=True, drop_rate=0.0, attn_drop_rate=0.0, drop_path_rate=0.1, use_abs_pos_embed=False, act_cfg={'type': 'GELU'}, norm_cfg={'type': 'LN'}, with_cp=False, pretrained=None, convert_weights=False, frozen_stages=- 1, init_cfg=None)[source]

Swin Transformer A PyTorch implement of : Swin Transformer: Hierarchical Vision Transformer using Shifted Windows -

Inspiration from https://github.com/microsoft/Swin-Transformer

Parameters
  • pretrain_img_size (int | tuple[int]) – The size of input image when pretrain. Defaults: 224.

  • in_channels (int) – The num of input channels. Defaults: 3.

  • embed_dims (int) – The feature dimension. Default: 96.

  • patch_size (int | tuple[int]) – Patch size. Default: 4.

  • window_size (int) – Window size. Default: 7.

  • mlp_ratio (int) – Ratio of mlp hidden dim to embedding dim. Default: 4.

  • depths (tuple[int]) – Depths of each Swin Transformer stage. Default: (2, 2, 6, 2).

  • num_heads (tuple[int]) – Parallel attention heads of each Swin Transformer stage. Default: (3, 6, 12, 24).

  • strides (tuple[int]) – The patch merging or patch embedding stride of each Swin Transformer stage. (In swin, we set kernel size equal to stride.) Default: (4, 2, 2, 2).

  • out_indices (tuple[int]) – Output from which stages. Default: (0, 1, 2, 3).

  • qkv_bias (bool, optional) – If True, add a learnable bias to query, key, value. Default: True

  • qk_scale (float | None, optional) – Override default qk scale of head_dim ** -0.5 if set. Default: None.

  • patch_norm (bool) – If add a norm layer for patch embed and patch merging. Default: True.

  • drop_rate (float) – Dropout rate. Defaults: 0.

  • attn_drop_rate (float) – Attention dropout rate. Default: 0.

  • drop_path_rate (float) – Stochastic depth rate. Defaults: 0.1.

  • use_abs_pos_embed (bool) – If True, add absolute position embedding to the patch embedding. Defaults: False.

  • act_cfg (dict) – Config dict for activation layer. Default: dict(type=’LN’).

  • norm_cfg (dict) – Config dict for normalization layer at output of backone. Defaults: dict(type=’LN’).

  • with_cp (bool, optional) – Use checkpoint or not. Using checkpoint will save some memory while slowing down the training speed. Default: False.

  • pretrained (str, optional) – model pretrained path. Default: None.

  • convert_weights (bool) – The flag indicates whether the pre-trained model is from the original repo. We may need to convert some keys to make it compatible. Default: False.

  • frozen_stages (int) – Stages to be frozen (stop grad and set eval mode). -1 means not freezing any parameters.

  • init_cfg (dict, optional) – The Config for initialization. Defaults to None.

forward(x)[source]

Defines the computation performed at every call.

Should be overridden by all subclasses.

Note

Although the recipe for forward pass needs to be defined within this function, one should call the Module instance afterwards instead of this since the former takes care of running the registered hooks while the latter silently ignores them.

init_weights()[source]

Initialize the weights.

train(mode=True)[source]

Convert the model into training mode while keep layers freezed.

class mmdet.models.backbones.TridentResNet(depth, num_branch, test_branch_idx, trident_dilations, **kwargs)[source]

The stem layer, stage 1 and stage 2 in Trident ResNet are identical to ResNet, while in stage 3, Trident BottleBlock is utilized to replace the normal BottleBlock to yield trident output. Different branch shares the convolution weight but uses different dilations to achieve multi-scale output.

/ stage3(b0) x - stem - stage1 - stage2 - stage3(b1) - output stage3(b2) /

Parameters
  • depth (int) – Depth of resnet, from {50, 101, 152}.

  • num_branch (int) – Number of branches in TridentNet.

  • test_branch_idx (int) – In inference, all 3 branches will be used if test_branch_idx==-1, otherwise only branch with index test_branch_idx will be used.

  • trident_dilations (tuple[int]) – Dilations of different trident branch. len(trident_dilations) should be equal to num_branch.

necks

class mmdet.models.necks.BFP(Balanced Feature Pyramids)[source]

BFP takes multi-level features as inputs and gather them into a single one, then refine the gathered feature and scatter the refined results to multi-level features. This module is used in Libra R-CNN (CVPR 2019), see the paper Libra R-CNN: Towards Balanced Learning for Object Detection for details.

Parameters
  • in_channels (int) – Number of input channels (feature maps of all levels should have the same channels).

  • num_levels (int) – Number of input feature levels.

  • conv_cfg (dict) – The config dict for convolution layers.

  • norm_cfg (dict) – The config dict for normalization layers.

  • refine_level (int) – Index of integration and refine level of BSF in multi-level features from bottom to top.

  • refine_type (str) – Type of the refine op, currently support [None, ‘conv’, ‘non_local’].

  • init_cfg (dict or list[dict], optional) – Initialization config dict.

forward(inputs)[source]

Forward function.

class mmdet.models.necks.CTResNetNeck(in_channel, num_deconv_filters, num_deconv_kernels, use_dcn=True, init_cfg=None)[source]

The neck used in CenterNet for object classification and box regression.

Parameters
  • in_channel (int) – Number of input channels.

  • num_deconv_filters (tuple[int]) – Number of filters per stage.

  • num_deconv_kernels (tuple[int]) – Number of kernels per stage.

  • use_dcn (bool) – If True, use DCNv2. Default: True.

  • init_cfg (dict or list[dict], optional) – Initialization config dict.

forward(inputs)[source]

Defines the computation performed at every call.

Should be overridden by all subclasses.

Note

Although the recipe for forward pass needs to be defined within this function, one should call the Module instance afterwards instead of this since the former takes care of running the registered hooks while the latter silently ignores them.

init_weights()[source]

Initialize the weights.

class mmdet.models.necks.ChannelMapper(in_channels, out_channels, kernel_size=3, conv_cfg=None, norm_cfg=None, act_cfg={'type': 'ReLU'}, num_outs=None, init_cfg={'distribution': 'uniform', 'layer': 'Conv2d', 'type': 'Xavier'})[source]

Channel Mapper to reduce/increase channels of backbone features.

This is used to reduce/increase channels of backbone features.

Parameters
  • in_channels (List[int]) – Number of input channels per scale.

  • out_channels (int) – Number of output channels (used at each scale).

  • kernel_size (int, optional) – kernel_size for reducing channels (used at each scale). Default: 3.

  • conv_cfg (dict, optional) – Config dict for convolution layer. Default: None.

  • norm_cfg (dict, optional) – Config dict for normalization layer. Default: None.

  • act_cfg (dict, optional) – Config dict for activation layer in ConvModule. Default: dict(type=’ReLU’).

  • num_outs (int, optional) – Number of output feature maps. There would be extra_convs when num_outs larger than the length of in_channels.

  • init_cfg (dict or list[dict], optional) – Initialization config dict.

Example

>>> import torch
>>> in_channels = [2, 3, 5, 7]
>>> scales = [340, 170, 84, 43]
>>> inputs = [torch.rand(1, c, s, s)
...           for c, s in zip(in_channels, scales)]
>>> self = ChannelMapper(in_channels, 11, 3).eval()
>>> outputs = self.forward(inputs)
>>> for i in range(len(outputs)):
...     print(f'outputs[{i}].shape = {outputs[i].shape}')
outputs[0].shape = torch.Size([1, 11, 340, 340])
outputs[1].shape = torch.Size([1, 11, 170, 170])
outputs[2].shape = torch.Size([1, 11, 84, 84])
outputs[3].shape = torch.Size([1, 11, 43, 43])
forward(inputs)[source]

Forward function.

class mmdet.models.necks.DilatedEncoder(in_channels, out_channels, block_mid_channels, num_residual_blocks)[source]

Dilated Encoder for YOLOF <https://arxiv.org/abs/2103.09460>`.

This module contains two types of components:
  • the original FPN lateral convolution layer and fpn convolution layer,

    which are 1x1 conv + 3x3 conv

  • the dilated residual block

Parameters
  • in_channels (int) – The number of input channels.

  • out_channels (int) – The number of output channels.

  • block_mid_channels (int) – The number of middle block output channels

  • num_residual_blocks (int) – The number of residual blocks.

forward(feature)[source]

Defines the computation performed at every call.

Should be overridden by all subclasses.

Note

Although the recipe for forward pass needs to be defined within this function, one should call the Module instance afterwards instead of this since the former takes care of running the registered hooks while the latter silently ignores them.

class mmdet.models.necks.FPG(in_channels, out_channels, num_outs, stack_times, paths, inter_channels=None, same_down_trans=None, same_up_trans={'kernel_size': 3, 'padding': 1, 'stride': 2, 'type': 'conv'}, across_lateral_trans={'kernel_size': 1, 'type': 'conv'}, across_down_trans={'kernel_size': 3, 'type': 'conv'}, across_up_trans=None, across_skip_trans={'type': 'identity'}, output_trans={'kernel_size': 3, 'type': 'last_conv'}, start_level=0, end_level=- 1, add_extra_convs=False, norm_cfg=None, skip_inds=None, init_cfg=[{'type': 'Caffe2Xavier', 'layer': 'Conv2d'}, {'type': 'Constant', 'layer': ['_BatchNorm', '_InstanceNorm', 'GroupNorm', 'LayerNorm'], 'val': 1.0}])[source]

FPG.

Implementation of Feature Pyramid Grids (FPG). This implementation only gives the basic structure stated in the paper. But users can implement different type of transitions to fully explore the the potential power of the structure of FPG.

Parameters
  • in_channels (int) – Number of input channels (feature maps of all levels should have the same channels).

  • out_channels (int) – Number of output channels (used at each scale)

  • num_outs (int) – Number of output scales.

  • stack_times (int) – The number of times the pyramid architecture will be stacked.

  • paths (list[str]) – Specify the path order of each stack level. Each element in the list should be either ‘bu’ (bottom-up) or ‘td’ (top-down).

  • inter_channels (int) – Number of inter channels.

  • same_up_trans (dict) – Transition that goes down at the same stage.

  • same_down_trans (dict) – Transition that goes up at the same stage.

  • across_lateral_trans (dict) – Across-pathway same-stage

  • across_down_trans (dict) – Across-pathway bottom-up connection.

  • across_up_trans (dict) – Across-pathway top-down connection.

  • across_skip_trans (dict) – Across-pathway skip connection.

  • output_trans (dict) – Transition that trans the output of the last stage.

  • start_level (int) – Index of the start input backbone level used to build the feature pyramid. Default: 0.

  • end_level (int) – Index of the end input backbone level (exclusive) to build the feature pyramid. Default: -1, which means the last level.

  • add_extra_convs (bool) – It decides whether to add conv layers on top of the original feature maps. Default to False. If True, its actual mode is specified by extra_convs_on_inputs.

  • norm_cfg (dict) – Config dict for normalization layer. Default: None.

  • init_cfg (dict or list[dict], optional) – Initialization config dict.

forward(inputs)[source]

Defines the computation performed at every call.

Should be overridden by all subclasses.

Note

Although the recipe for forward pass needs to be defined within this function, one should call the Module instance afterwards instead of this since the former takes care of running the registered hooks while the latter silently ignores them.

class mmdet.models.necks.FPN(in_channels, out_channels, num_outs, start_level=0, end_level=- 1, add_extra_convs=False, relu_before_extra_convs=False, no_norm_on_lateral=False, conv_cfg=None, norm_cfg=None, act_cfg=None, upsample_cfg={'mode': 'nearest'}, init_cfg={'distribution': 'uniform', 'layer': 'Conv2d', 'type': 'Xavier'})[source]

Feature Pyramid Network.

This is an implementation of paper Feature Pyramid Networks for Object Detection.

Parameters
  • in_channels (List[int]) – Number of input channels per scale.

  • out_channels (int) – Number of output channels (used at each scale)

  • num_outs (int) – Number of output scales.

  • start_level (int) – Index of the start input backbone level used to build the feature pyramid. Default: 0.

  • end_level (int) – Index of the end input backbone level (exclusive) to build the feature pyramid. Default: -1, which means the last level.

  • add_extra_convs (bool | str) –

    If bool, it decides whether to add conv layers on top of the original feature maps. Default to False. If True, it is equivalent to add_extra_convs=’on_input’. If str, it specifies the source feature map of the extra convs. Only the following options are allowed

    • ’on_input’: Last feat map of neck inputs (i.e. backbone feature).

    • ’on_lateral’: Last feature map after lateral convs.

    • ’on_output’: The last output feature map after fpn convs.

  • relu_before_extra_convs (bool) – Whether to apply relu before the extra conv. Default: False.

  • no_norm_on_lateral (bool) – Whether to apply norm on lateral. Default: False.

  • conv_cfg (dict) – Config dict for convolution layer. Default: None.

  • norm_cfg (dict) – Config dict for normalization layer. Default: None.

  • act_cfg (str) – Config dict for activation layer in ConvModule. Default: None.

  • upsample_cfg (dict) – Config dict for interpolate layer. Default: dict(mode=’nearest’)

  • init_cfg (dict or list[dict], optional) – Initialization config dict.

Example

>>> import torch
>>> in_channels = [2, 3, 5, 7]
>>> scales = [340, 170, 84, 43]
>>> inputs = [torch.rand(1, c, s, s)
...           for c, s in zip(in_channels, scales)]
>>> self = FPN(in_channels, 11, len(in_channels)).eval()
>>> outputs = self.forward(inputs)
>>> for i in range(len(outputs)):
...     print(f'outputs[{i}].shape = {outputs[i].shape}')
outputs[0].shape = torch.Size([1, 11, 340, 340])
outputs[1].shape = torch.Size([1, 11, 170, 170])
outputs[2].shape = torch.Size([1, 11, 84, 84])
outputs[3].shape = torch.Size([1, 11, 43, 43])
forward(inputs)[source]

Forward function.

class mmdet.models.necks.FPN_CARAFE(in_channels, out_channels, num_outs, start_level=0, end_level=- 1, norm_cfg=None, act_cfg=None, order=('conv', 'norm', 'act'), upsample_cfg={'encoder_dilation': 1, 'encoder_kernel': 3, 'type': 'carafe', 'up_group': 1, 'up_kernel': 5}, init_cfg=None)[source]

FPN_CARAFE is a more flexible implementation of FPN. It allows more choice for upsample methods during the top-down pathway.

It can reproduce the performance of ICCV 2019 paper CARAFE: Content-Aware ReAssembly of FEatures Please refer to https://arxiv.org/abs/1905.02188 for more details.

Parameters
  • in_channels (list[int]) – Number of channels for each input feature map.

  • out_channels (int) – Output channels of feature pyramids.

  • num_outs (int) – Number of output stages.

  • start_level (int) – Start level of feature pyramids. (Default: 0)

  • end_level (int) – End level of feature pyramids. (Default: -1 indicates the last level).

  • norm_cfg (dict) – Dictionary to construct and config norm layer.

  • activate (str) – Type of activation function in ConvModule (Default: None indicates w/o activation).

  • order (dict) – Order of components in ConvModule.

  • upsample (str) – Type of upsample layer.

  • upsample_cfg (dict) – Dictionary to construct and config upsample layer.

  • init_cfg (dict or list[dict], optional) – Initialization config dict. Default: None

forward(inputs)[source]

Forward function.

init_weights()[source]

Initialize the weights of module.

slice_as(src, dst)[source]

Slice src as dst

Note

src should have the same or larger size than dst.

Parameters
  • src (torch.Tensor) – Tensors to be sliced.

  • dst (torch.Tensor) – src will be sliced to have the same size as dst.

Returns

Sliced tensor.

Return type

torch.Tensor

tensor_add(a, b)[source]

Add tensors a and b that might have different sizes.

class mmdet.models.necks.HRFPN(High Resolution Feature Pyramids)[source]

paper: High-Resolution Representations for Labeling Pixels and Regions.

Parameters
  • in_channels (list) – number of channels for each branch.

  • out_channels (int) – output channels of feature pyramids.

  • num_outs (int) – number of output stages.

  • pooling_type (str) – pooling for generating feature pyramids from {MAX, AVG}.

  • conv_cfg (dict) – dictionary to construct and config conv layer.

  • norm_cfg (dict) – dictionary to construct and config norm layer.

  • with_cp (bool) – Use checkpoint or not. Using checkpoint will save some memory while slowing down the training speed.

  • stride (int) – stride of 3x3 convolutional layers

  • init_cfg (dict or list[dict], optional) – Initialization config dict.

forward(inputs)[source]

Forward function.

class mmdet.models.necks.NASFCOS_FPN(in_channels, out_channels, num_outs, start_level=1, end_level=- 1, add_extra_convs=False, conv_cfg=None, norm_cfg=None, init_cfg=None)[source]

FPN structure in NASFPN.

Implementation of paper NAS-FCOS: Fast Neural Architecture Search for Object Detection

Parameters
  • in_channels (List[int]) – Number of input channels per scale.

  • out_channels (int) – Number of output channels (used at each scale)

  • num_outs (int) – Number of output scales.

  • start_level (int) – Index of the start input backbone level used to build the feature pyramid. Default: 0.

  • end_level (int) – Index of the end input backbone level (exclusive) to build the feature pyramid. Default: -1, which means the last level.

  • add_extra_convs (bool) – It decides whether to add conv layers on top of the original feature maps. Default to False. If True, its actual mode is specified by extra_convs_on_inputs.

  • conv_cfg (dict) – dictionary to construct and config conv layer.

  • norm_cfg (dict) – dictionary to construct and config norm layer.

  • init_cfg (dict or list[dict], optional) – Initialization config dict. Default: None

forward(inputs)[source]

Forward function.

init_weights()[source]

Initialize the weights of module.

class mmdet.models.necks.NASFPN(in_channels, out_channels, num_outs, stack_times, start_level=0, end_level=- 1, add_extra_convs=False, norm_cfg=None, init_cfg={'layer': 'Conv2d', 'type': 'Caffe2Xavier'})[source]

NAS-FPN.

Implementation of NAS-FPN: Learning Scalable Feature Pyramid Architecture for Object Detection

Parameters
  • in_channels (List[int]) – Number of input channels per scale.

  • out_channels (int) – Number of output channels (used at each scale)

  • num_outs (int) – Number of output scales.

  • stack_times (int) – The number of times the pyramid architecture will be stacked.

  • start_level (int) – Index of the start input backbone level used to build the feature pyramid. Default: 0.

  • end_level (int) – Index of the end input backbone level (exclusive) to build the feature pyramid. Default: -1, which means the last level.

  • add_extra_convs (bool) – It decides whether to add conv layers on top of the original feature maps. Default to False. If True, its actual mode is specified by extra_convs_on_inputs.

  • init_cfg (dict or list[dict], optional) – Initialization config dict.

forward(inputs)[source]

Forward function.

class mmdet.models.necks.PAFPN(in_channels, out_channels, num_outs, start_level=0, end_level=- 1, add_extra_convs=False, relu_before_extra_convs=False, no_norm_on_lateral=False, conv_cfg=None, norm_cfg=None, act_cfg=None, init_cfg={'distribution': 'uniform', 'layer': 'Conv2d', 'type': 'Xavier'})[source]

Path Aggregation Network for Instance Segmentation.

This is an implementation of the PAFPN in Path Aggregation Network.

Parameters
  • in_channels (List[int]) – Number of input channels per scale.

  • out_channels (int) – Number of output channels (used at each scale)

  • num_outs (int) – Number of output scales.

  • start_level (int) – Index of the start input backbone level used to build the feature pyramid. Default: 0.

  • end_level (int) – Index of the end input backbone level (exclusive) to build the feature pyramid. Default: -1, which means the last level.

  • add_extra_convs (bool | str) –

    If bool, it decides whether to add conv layers on top of the original feature maps. Default to False. If True, it is equivalent to add_extra_convs=’on_input’. If str, it specifies the source feature map of the extra convs. Only the following options are allowed

    • ’on_input’: Last feat map of neck inputs (i.e. backbone feature).

    • ’on_lateral’: Last feature map after lateral convs.

    • ’on_output’: The last output feature map after fpn convs.

  • relu_before_extra_convs (bool) – Whether to apply relu before the extra conv. Default: False.

  • no_norm_on_lateral (bool) – Whether to apply norm on lateral. Default: False.

  • conv_cfg (dict) – Config dict for convolution layer. Default: None.

  • norm_cfg (dict) – Config dict for normalization layer. Default: None.

  • act_cfg (str) – Config dict for activation layer in ConvModule. Default: None.

  • init_cfg (dict or list[dict], optional) – Initialization config dict.

forward(inputs)[source]

Forward function.

class mmdet.models.necks.RFP(Recursive Feature Pyramid)[source]

This is an implementation of RFP in DetectoRS. Different from standard FPN, the input of RFP should be multi level features along with origin input image of backbone.

Parameters
  • rfp_steps (int) – Number of unrolled steps of RFP.

  • rfp_backbone (dict) – Configuration of the backbone for RFP.

  • aspp_out_channels (int) – Number of output channels of ASPP module.

  • aspp_dilations (tuple[int]) – Dilation rates of four branches. Default: (1, 3, 6, 1)

  • init_cfg (dict or list[dict], optional) – Initialization config dict. Default: None

forward(inputs)[source]

Forward function.

init_weights()[source]

Initialize the weights.

class mmdet.models.necks.SSDNeck(in_channels, out_channels, level_strides, level_paddings, l2_norm_scale=20.0, last_kernel_size=3, use_depthwise=False, conv_cfg=None, norm_cfg=None, act_cfg={'type': 'ReLU'}, init_cfg=[{'type': 'Xavier', 'distribution': 'uniform', 'layer': 'Conv2d'}, {'type': 'Constant', 'val': 1, 'layer': 'BatchNorm2d'}])[source]

Extra layers of SSD backbone to generate multi-scale feature maps.

Parameters
  • in_channels (Sequence[int]) – Number of input channels per scale.

  • out_channels (Sequence[int]) – Number of output channels per scale.

  • level_strides (Sequence[int]) – Stride of 3x3 conv per level.

  • level_paddings (Sequence[int]) – Padding size of 3x3 conv per level.

  • l2_norm_scale (float|None) – L2 normalization layer init scale. If None, not use L2 normalization on the first input feature.

  • last_kernel_size (int) – Kernel size of the last conv layer. Default: 3.

  • use_depthwise (bool) – Whether to use DepthwiseSeparableConv. Default: False.

  • conv_cfg (dict) – Config dict for convolution layer. Default: None.

  • norm_cfg (dict) – Dictionary to construct and config norm layer. Default: None.

  • act_cfg (dict) – Config dict for activation layer. Default: dict(type=’ReLU’).

  • init_cfg (dict or list[dict], optional) – Initialization config dict.

forward(inputs)[source]

Forward function.

class mmdet.models.necks.YOLOV3Neck(num_scales, in_channels, out_channels, conv_cfg=None, norm_cfg={'requires_grad': True, 'type': 'BN'}, act_cfg={'negative_slope': 0.1, 'type': 'LeakyReLU'}, init_cfg=None)[source]

The neck of YOLOV3.

It can be treated as a simplified version of FPN. It will take the result from Darknet backbone and do some upsampling and concatenation. It will finally output the detection result.

Note

The input feats should be from top to bottom.

i.e., from high-lvl to low-lvl

But YOLOV3Neck will process them in reversed order.

i.e., from bottom (high-lvl) to top (low-lvl)

Parameters
  • num_scales (int) – The number of scales / stages.

  • in_channels (List[int]) – The number of input channels per scale.

  • out_channels (List[int]) – The number of output channels per scale.

  • conv_cfg (dict, optional) – Config dict for convolution layer. Default: None.

  • norm_cfg (dict, optional) – Dictionary to construct and config norm layer. Default: dict(type=’BN’, requires_grad=True)

  • act_cfg (dict, optional) – Config dict for activation layer. Default: dict(type=’LeakyReLU’, negative_slope=0.1).

  • init_cfg (dict or list[dict], optional) – Initialization config dict. Default: None

forward(feats)[source]

Defines the computation performed at every call.

Should be overridden by all subclasses.

Note

Although the recipe for forward pass needs to be defined within this function, one should call the Module instance afterwards instead of this since the former takes care of running the registered hooks while the latter silently ignores them.

class mmdet.models.necks.YOLOXPAFPN(in_channels, out_channels, num_csp_blocks=3, use_depthwise=False, upsample_cfg={'mode': 'nearest', 'scale_factor': 2}, conv_cfg=None, norm_cfg={'eps': 0.001, 'momentum': 0.03, 'type': 'BN'}, act_cfg={'type': 'Swish'}, init_cfg={'a': 2.23606797749979, 'distribution': 'uniform', 'layer': 'Conv2d', 'mode': 'fan_in', 'nonlinearity': 'leaky_relu', 'type': 'Kaiming'})[source]

Path Aggregation Network used in YOLOX.

Parameters
  • in_channels (List[int]) – Number of input channels per scale.

  • out_channels (int) – Number of output channels (used at each scale)

  • num_csp_blocks (int) – Number of bottlenecks in CSPLayer. Default: 3

  • use_depthwise (bool) – Whether to depthwise separable convolution in blocks. Default: False

  • upsample_cfg (dict) – Config dict for interpolate layer. Default: dict(scale_factor=2, mode=’nearest’)

  • conv_cfg (dict, optional) – Config dict for convolution layer. Default: None, which means using conv2d.

  • norm_cfg (dict) – Config dict for normalization layer. Default: dict(type=’BN’)

  • act_cfg (dict) – Config dict for activation layer. Default: dict(type=’Swish’)

  • init_cfg (dict or list[dict], optional) – Initialization config dict. Default: None.

forward(inputs)[source]
Parameters

inputs (tuple[Tensor]) – input features.

Returns

YOLOXPAFPN features.

Return type

tuple[Tensor]

dense_heads

class mmdet.models.dense_heads.ATSSHead(num_classes, in_channels, stacked_convs=4, conv_cfg=None, norm_cfg={'num_groups': 32, 'requires_grad': True, 'type': 'GN'}, loss_centerness={'loss_weight': 1.0, 'type': 'CrossEntropyLoss', 'use_sigmoid': True}, init_cfg={'layer': 'Conv2d',