mmdet.apis

mmdet.core

anchor

class mmdet.core.anchor.AnchorGenerator(strides, ratios, scales=None, base_sizes=None, scale_major=True, octave_base_scale=None, scales_per_octave=None, centers=None, center_offset=0.0)

Standard anchor generator for 2D anchor-based detectors.

Parameters

• strides (list[int] | list[tuple[int, int]]) – Strides of anchors in multiple feature levels in order (w, h).

• ratios (list[float]) – The list of ratios between the height and width of anchors in a single level.

• scales (list[int] | None) – Anchor scales for anchors in a single level. It cannot be set at the same time if octave_base_scale and scales_per_octave are set.

• base_sizes (list[int] | None) – The basic sizes of anchors in multiple levels. If None is given, strides will be used as base_sizes. (If strides are non square, the shortest stride is taken.)

• scale_major (bool) – Whether to multiply scales first when generating base anchors. If true, the anchors in the same row will have the same scales. By default it is True in V2.0

• octave_base_scale (int) – The base scale of octave.

• scales_per_octave (int) – Number of scales for each octave. octave_base_scale and scales_per_octave are usually used in retinanet and the scales should be None when they are set.

• centers (list[tuple[float, float]] | None) – The centers of the anchor relative to the feature grid center in multiple feature levels. By default it is set to be None and not used. If a list of tuple of float is given, they will be used to shift the centers of anchors.

• center_offset (float) – The offset of center in proportion to anchors’ width and height. By default it is 0 in V2.0.

Examples

>>> from mmdet.core import AnchorGenerator
>>> self = AnchorGenerator([16], [1.], [1.], [9])
>>> all_anchors = self.grid_priors([(2, 2)], device='cpu')
>>> print(all_anchors)
[tensor([[-4.5000, -4.5000,  4.5000,  4.5000],
        [11.5000, -4.5000, 20.5000,  4.5000],
        [-4.5000, 11.5000,  4.5000, 20.5000],
        [11.5000, 11.5000, 20.5000, 20.5000]])]
>>> self = AnchorGenerator([16, 32], [1.], [1.], [9, 18])
>>> all_anchors = self.grid_priors([(2, 2), (1, 1)], device='cpu')
>>> print(all_anchors)
[tensor([[-4.5000, -4.5000,  4.5000,  4.5000],
        [11.5000, -4.5000, 20.5000,  4.5000],
        [-4.5000, 11.5000,  4.5000, 20.5000],
        [11.5000, 11.5000, 20.5000, 20.5000]]),         tensor([[-9., -9., 9., 9.]])]

gen_base_anchors()

Generate base anchors.

Returns

Base anchors of a feature grid in multiple feature levels.

Return type

list(torch.Tensor)

gen_single_level_base_anchors(base_size, scales, ratios, center=None)

Generate base anchors of a single level.

Parameters

• base_size (int | float) – Basic size of an anchor.

• scales (torch.Tensor) – Scales of the anchor.

• ratios (torch.Tensor) – The ratio between between the height and width of anchors in a single level.

• center (tuple[float], optional) – The center of the base anchor related to a single feature grid. Defaults to None.

Returns

Anchors in a single-level feature maps.

Return type

torch.Tensor

grid_anchors(featmap_sizes, device='cuda')

Generate grid anchors in multiple feature levels.

Parameters

• featmap_sizes (list[tuple]) – List of feature map sizes in multiple feature levels.

• device (str) – Device where the anchors will be put on.

Returns

Anchors in multiple feature levels. The sizes of each tensor should be [N, 4], where N = width * height * num_base_anchors, width and height are the sizes of the corresponding feature level, num_base_anchors is the number of anchors for that level.

Return type

list[torch.Tensor]

grid_priors(featmap_sizes, dtype=torch.float32, device='cuda')

Generate grid anchors in multiple feature levels.

Parameters

• featmap_sizes (list[tuple]) – List of feature map sizes in multiple feature levels.

• dtype (torch.dtype) – Dtype of priors. Default: torch.float32.

• device (str) – The device where the anchors will be put on.

Returns

Anchors in multiple feature levels. The sizes of each tensor should be [N, 4], where N = width * height * num_base_anchors, width and height are the sizes of the corresponding feature level, num_base_anchors is the number of anchors for that level.

Return type

list[torch.Tensor]

property num_base_anchors

total number of base anchors in a feature grid

Type

list[int]

property num_base_priors

The number of priors (anchors) at a point on the feature grid

Type

list[int]

property num_levels

number of feature levels that the generator will be applied

Type

int

single_level_grid_anchors(base_anchors, featmap_size, stride=(16, 16), device='cuda')

Generate grid anchors of a single level.

Note

This function is usually called by method self.grid_anchors.

Parameters

• base_anchors (torch.Tensor) – The base anchors of a feature grid.

• featmap_size (tuple[int]) – Size of the feature maps.

• stride (tuple[int], optional) – Stride of the feature map in order (w, h). Defaults to (16, 16).

• device (str, optional) – Device the tensor will be put on. Defaults to ‘cuda’.

Returns

Anchors in the overall feature maps.

Return type

torch.Tensor

single_level_grid_priors(featmap_size, level_idx, dtype=torch.float32, device='cuda')

Generate grid anchors of a single level.

Note

This function is usually called by method self.grid_priors.

Parameters

• featmap_size (tuple[int]) – Size of the feature maps.

• level_idx (int) – The index of corresponding feature map level.

• (obj (dtype) – torch.dtype): Date type of points.Defaults to torch.float32.

• device (str, optional) – The device the tensor will be put on. Defaults to ‘cuda’.

Returns

Anchors in the overall feature maps.

Return type

torch.Tensor

single_level_valid_flags(featmap_size, valid_size, num_base_anchors, device='cuda')

Generate the valid flags of anchor in a single feature map.

Parameters

• featmap_size (tuple[int]) – The size of feature maps, arrange as (h, w).

• valid_size (tuple[int]) – The valid size of the feature maps.

• num_base_anchors (int) – The number of base anchors.

• device (str, optional) – Device where the flags will be put on. Defaults to ‘cuda’.

Returns

The valid flags of each anchor in a single level feature map.

Return type

torch.Tensor

sparse_priors(prior_idxs, featmap_size, level_idx, dtype=torch.float32, device='cuda')

Generate sparse anchors according to the prior_idxs.

Parameters

• prior_idxs (Tensor) – The index of corresponding anchors in the feature map.

• featmap_size (tuple[int]) – feature map size arrange as (h, w).

• level_idx (int) – The level index of corresponding feature map.

• (obj (device) – torch.dtype): Date type of points.Defaults to torch.float32.

• (obj – torch.device): The device where the points is located.

Returns

Anchor with shape (N, 4), N should be equal to

the length of prior_idxs.

Return type

Tensor

valid_flags(featmap_sizes, pad_shape, device='cuda')

Generate valid flags of anchors in multiple feature levels.

Parameters

• featmap_sizes (list(tuple)) – List of feature map sizes in multiple feature levels.

• pad_shape (tuple) – The padded shape of the image.

• device (str) – Device where the anchors will be put on.

Returns

Valid flags of anchors in multiple levels.

Return type

list(torch.Tensor)

class mmdet.core.anchor.LegacyAnchorGenerator(strides, ratios, scales=None, base_sizes=None, scale_major=True, octave_base_scale=None, scales_per_octave=None, centers=None, center_offset=0.0)

Legacy anchor generator used in MMDetection V1.x.

Note

Difference to the V2.0 anchor generator:

1. The center offset of V1.x anchors are set to be 0.5 rather than 0.

2. The width/height are minused by 1 when calculating the anchors’ centers and corners to meet the V1.x coordinate system.

3. The anchors’ corners are quantized.

Parameters

• strides (list[int] | list[tuple[int]]) – Strides of anchors in multiple feature levels.

• ratios (list[float]) – The list of ratios between the height and width of anchors in a single level.

• scales (list[int] | None) – Anchor scales for anchors in a single level. It cannot be set at the same time if octave_base_scale and scales_per_octave are set.

• base_sizes (list[int]) – The basic sizes of anchors in multiple levels. If None is given, strides will be used to generate base_sizes.

• scale_major (bool) – Whether to multiply scales first when generating base anchors. If true, the anchors in the same row will have the same scales. By default it is True in V2.0

• octave_base_scale (int) – The base scale of octave.

• scales_per_octave (int) – Number of scales for each octave. octave_base_scale and scales_per_octave are usually used in retinanet and the scales should be None when they are set.

• centers (list[tuple[float, float]] | None) – The centers of the anchor relative to the feature grid center in multiple feature levels. By default it is set to be None and not used. It a list of float is given, this list will be used to shift the centers of anchors.

• center_offset (float) – The offset of center in proportion to anchors’ width and height. By default it is 0.5 in V2.0 but it should be 0.5 in v1.x models.

Examples

>>> from mmdet.core import LegacyAnchorGenerator
>>> self = LegacyAnchorGenerator(
>>>     [16], [1.], [1.], [9], center_offset=0.5)
>>> all_anchors = self.grid_anchors(((2, 2),), device='cpu')
>>> print(all_anchors)
[tensor([[ 0.,  0.,  8.,  8.],
        [16.,  0., 24.,  8.],
        [ 0., 16.,  8., 24.],
        [16., 16., 24., 24.]])]

gen_single_level_base_anchors(base_size, scales, ratios, center=None)

Generate base anchors of a single level.

Note

The width/height of anchors are minused by 1 when calculating the centers and corners to meet the V1.x coordinate system.

Parameters

• base_size (int | float) – Basic size of an anchor.

• scales (torch.Tensor) – Scales of the anchor.

• ratios (torch.Tensor) – The ratio between between the height. and width of anchors in a single level.

• center (tuple[float], optional) – The center of the base anchor related to a single feature grid. Defaults to None.

Returns

Anchors in a single-level feature map.

Return type

torch.Tensor

class mmdet.core.anchor.MlvlPointGenerator(strides, offset=0.5)

Standard points generator for multi-level (Mlvl) feature maps in 2D points-based detectors.

Parameters

• strides (list[int] | list[tuple[int, int]]) – Strides of anchors in multiple feature levels in order (w, h).

• offset (float) – The offset of points, the value is normalized with corresponding stride. Defaults to 0.5.

grid_priors(featmap_sizes, dtype=torch.float32, device='cuda', with_stride=False)

Generate grid points of multiple feature levels.

Parameters

• featmap_sizes (list[tuple]) – List of feature map sizes in multiple feature levels, each size arrange as as (h, w).

• dtype (dtype) – Dtype of priors. Default: torch.float32.

• device (str) – The device where the anchors will be put on.

• with_stride (bool) – Whether to concatenate the stride to the last dimension of points.

Returns

Points of multiple feature levels. The sizes of each tensor should be (N, 2) when with stride is False, where N = width * height, width and height are the sizes of the corresponding feature level, and the last dimension 2 represent (coord_x, coord_y), otherwise the shape should be (N, 4), and the last dimension 4 represent (coord_x, coord_y, stride_w, stride_h).

Return type

list[torch.Tensor]

property num_base_priors

The number of priors (points) at a point on the feature grid

Type

list[int]

property num_levels

number of feature levels that the generator will be applied

Type

int

single_level_grid_priors(featmap_size, level_idx, dtype=torch.float32, device='cuda', with_stride=False)

Generate grid Points of a single level.

Note

This function is usually called by method self.grid_priors.

Parameters

• featmap_size (tuple[int]) – Size of the feature maps, arrange as (h, w).

• level_idx (int) – The index of corresponding feature map level.

• dtype (dtype) – Dtype of priors. Default: torch.float32.

• device (str, optional) – The device the tensor will be put on. Defaults to ‘cuda’.

• with_stride (bool) – Concatenate the stride to the last dimension of points.

Returns

Points of single feature levels. The shape of tensor should be (N, 2) when with stride is False, where N = width * height, width and height are the sizes of the corresponding feature level, and the last dimension 2 represent (coord_x, coord_y), otherwise the shape should be (N, 4), and the last dimension 4 represent (coord_x, coord_y, stride_w, stride_h).

Return type

Tensor

single_level_valid_flags(featmap_size, valid_size, device='cuda')

Generate the valid flags of points of a single feature map.

Parameters

• featmap_size (tuple[int]) – The size of feature maps, arrange as as (h, w).

• valid_size (tuple[int]) – The valid size of the feature maps. The size arrange as as (h, w).

• device (str, optional) – The device where the flags will be put on. Defaults to ‘cuda’.

Returns

The valid flags of each points in a single level feature map.

Return type

torch.Tensor

sparse_priors(prior_idxs, featmap_size, level_idx, dtype=torch.float32, device='cuda')

Generate sparse points according to the prior_idxs.

Parameters

• prior_idxs (Tensor) – The index of corresponding anchors in the feature map.

• featmap_size (tuple[int]) – feature map size arrange as (w, h).

• level_idx (int) – The level index of corresponding feature map.

• (obj (device) – torch.dtype): Date type of points. Defaults to torch.float32.

• (obj – torch.device): The device where the points is located.

Returns

Anchor with shape (N, 2), N should be equal to the length of prior_idxs. And last dimension 2 represent (coord_x, coord_y).

Return type

Tensor

valid_flags(featmap_sizes, pad_shape, device='cuda')

Generate valid flags of points of multiple feature levels.

Parameters

• featmap_sizes (list(tuple)) – List of feature map sizes in multiple feature levels, each size arrange as as (h, w).

• pad_shape (tuple(int)) – The padded shape of the image, arrange as (h, w).

• device (str) – The device where the anchors will be put on.

Returns

Valid flags of points of multiple levels.

Return type

list(torch.Tensor)

class mmdet.core.anchor.YOLOAnchorGenerator(strides, base_sizes)

Anchor generator for YOLO.

Parameters

• strides (list[int] | list[tuple[int, int]]) – Strides of anchors in multiple feature levels.

• base_sizes (list[list[tuple[int, int]]]) – The basic sizes of anchors in multiple levels.

gen_base_anchors()

Generate base anchors.

Returns

Base anchors of a feature grid in multiple feature levels.

Return type

list(torch.Tensor)

gen_single_level_base_anchors(base_sizes_per_level, center=None)

Generate base anchors of a single level.

Parameters

• base_sizes_per_level (list[tuple[int, int]]) – Basic sizes of anchors.

• center (tuple[float], optional) – The center of the base anchor related to a single feature grid. Defaults to None.

Returns

Anchors in a single-level feature maps.

Return type

torch.Tensor

property num_levels

number of feature levels that the generator will be applied

Type

int

responsible_flags(featmap_sizes, gt_bboxes, device='cuda')

Generate responsible anchor flags of grid cells in multiple scales.

Parameters

• featmap_sizes (list(tuple)) – List of feature map sizes in multiple feature levels.

• gt_bboxes (Tensor) – Ground truth boxes, shape (n, 4).

• device (str) – Device where the anchors will be put on.

Returns

responsible flags of anchors in multiple level

Return type

list(torch.Tensor)

single_level_responsible_flags(featmap_size, gt_bboxes, stride, num_base_anchors, device='cuda')

Generate the responsible flags of anchor in a single feature map.

Parameters

• featmap_size (tuple[int]) – The size of feature maps.

• gt_bboxes (Tensor) – Ground truth boxes, shape (n, 4).

• stride (tuple(int)) – stride of current level

• num_base_anchors (int) – The number of base anchors.

• device (str, optional) – Device where the flags will be put on. Defaults to ‘cuda’.

Returns

The valid flags of each anchor in a single level feature map.

Return type

torch.Tensor

mmdet.core.anchor.anchor_inside_flags(flat_anchors, valid_flags, img_shape, allowed_border=0)

Check whether the anchors are inside the border.

Parameters

• flat_anchors (torch.Tensor) – Flatten anchors, shape (n, 4).

• valid_flags (torch.Tensor) – An existing valid flags of anchors.

• img_shape (tuple(int)) – Shape of current image.

• allowed_border (int, optional) – The border to allow the valid anchor. Defaults to 0.

Returns

Flags indicating whether the anchors are inside a valid range.

Return type

torch.Tensor

mmdet.core.anchor.calc_region(bbox, ratio, featmap_size=None)

Calculate a proportional bbox region.

The bbox center are fixed and the new h’ and w’ is h * ratio and w * ratio.

Parameters

• bbox (Tensor) – Bboxes to calculate regions, shape (n, 4).

• ratio (float) – Ratio of the output region.

• featmap_size (tuple) – Feature map size used for clipping the boundary.

Returns

x1, y1, x2, y2

Return type

tuple

mmdet.core.anchor.images_to_levels(target, num_levels)

Convert targets by image to targets by feature level.

[target_img0, target_img1] -> [target_level0, target_level1, …]

bbox

export

mask

evaluation

post_processing

utils

mmdet.datasets

datasets

pipelines

samplers

api_wrappers

mmdet.models

detectors

backbones

class mmdet.models.backbones.CSPDarknet(arch='P5', deepen_factor=1.0, widen_factor=1.0, out_indices=(2, 3, 4), frozen_stages=-1, use_depthwise=False, arch_ovewrite=None, spp_kernal_sizes=(5, 9, 13), conv_cfg=None, norm_cfg={'eps': 0.001, 'momentum': 0.03, 'type': 'BN'}, act_cfg={'type': 'Swish'}, norm_eval=False, init_cfg={'a': 2.23606797749979, 'distribution': 'uniform', 'layer': 'Conv2d', 'mode': 'fan_in', 'nonlinearity': 'leaky_relu', 'type': 'Kaiming'})

CSP-Darknet backbone used in YOLOv5 and YOLOX.

Parameters

• arch (str) – Architecture of CSP-Darknet, from {P5, P6}. Default: P5.

• deepen_factor (float) – Depth multiplier, multiply number of blocks in CSP layer by this amount. Default: 1.0.

• widen_factor (float) – Width multiplier, multiply number of channels in each layer by this amount. Default: 1.0.

• out_indices (Sequence[int]) – Output from which stages. Default: (2, 3, 4).

• frozen_stages (int) – Stages to be frozen (stop grad and set eval mode). -1 means not freezing any parameters. Default: -1.

• use_depthwise (bool) – Whether to use depthwise separable convolution. Default: False.

• arch_ovewrite (list) – Overwrite default arch settings. Default: None.

• spp_kernal_sizes – (tuple[int]): Sequential of kernel sizes of SPP layers. Default: (5, 9, 13).

• conv_cfg (dict) – Config dict for convolution layer. Default: None.

• norm_cfg (dict) – Dictionary to construct and config norm layer. Default: dict(type=’BN’, requires_grad=True).

• act_cfg (dict) – Config dict for activation layer. Default: dict(type=’LeakyReLU’, negative_slope=0.1).

• norm_eval (bool) – Whether to set norm layers to eval mode, namely, freeze running stats (mean and var). Note: Effect on Batch Norm and its variants only.

• init_cfg (dict or list[dict], optional) – Initialization config dict. Default: None.

Example

>>> from mmdet.models import CSPDarknet
>>> import torch
>>> self = CSPDarknet(depth=53)
>>> self.eval()
>>> inputs = torch.rand(1, 3, 416, 416)
>>> level_outputs = self.forward(inputs)
>>> for level_out in level_outputs:
...     print(tuple(level_out.shape))
...
(1, 256, 52, 52)
(1, 512, 26, 26)
(1, 1024, 13, 13)

forward(x)

Defines the computation performed at every call.

Should be overridden by all subclasses.

Note

Although the recipe for forward pass needs to be defined within this function, one should call the Module instance afterwards instead of this since the former takes care of running the registered hooks while the latter silently ignores them.

train(mode=True)

Sets the module in training mode.

This has any effect only on certain modules. See documentations of particular modules for details of their behaviors in training/evaluation mode, if they are affected, e.g. Dropout, BatchNorm, etc.

Parameters

mode (bool) – whether to set training mode (True) or evaluation mode (False). Default: True.

Returns

self

Return type

Module

class mmdet.models.backbones.Darknet(depth=53, out_indices=(3, 4, 5), frozen_stages=-1, conv_cfg=None, norm_cfg={'requires_grad': True, 'type': 'BN'}, act_cfg={'negative_slope': 0.1, 'type': 'LeakyReLU'}, norm_eval=True, pretrained=None, init_cfg=None)

Darknet backbone.

Parameters

• depth (int) – Depth of Darknet. Currently only support 53.

• out_indices (Sequence[int]) – Output from which stages.

• frozen_stages (int) – Stages to be frozen (stop grad and set eval mode). -1 means not freezing any parameters. Default: -1.

• conv_cfg (dict) – Config dict for convolution layer. Default: None.

• norm_cfg (dict) – Dictionary to construct and config norm layer. Default: dict(type=’BN’, requires_grad=True)

• act_cfg (dict) – Config dict for activation layer. Default: dict(type=’LeakyReLU’, negative_slope=0.1).

• norm_eval (bool) – Whether to set norm layers to eval mode, namely, freeze running stats (mean and var). Note: Effect on Batch Norm and its variants only.

• pretrained (str, optional) – model pretrained path. Default: None

• init_cfg (dict or list[dict], optional) – Initialization config dict. Default: None

Example

>>> from mmdet.models import Darknet
>>> import torch
>>> self = Darknet(depth=53)
>>> self.eval()
>>> inputs = torch.rand(1, 3, 416, 416)
>>> level_outputs = self.forward(inputs)
>>> for level_out in level_outputs:
...     print(tuple(level_out.shape))
...
(1, 256, 52, 52)
(1, 512, 26, 26)
(1, 1024, 13, 13)

forward(x)

Defines the computation performed at every call.

Should be overridden by all subclasses.

Note

Although the recipe for forward pass needs to be defined within this function, one should call the Module instance afterwards instead of this since the former takes care of running the registered hooks while the latter silently ignores them.

static make_conv_res_block(in_channels, out_channels, res_repeat, conv_cfg=None, norm_cfg={'requires_grad': True, 'type': 'BN'}, act_cfg={'negative_slope': 0.1, 'type': 'LeakyReLU'})

In Darknet backbone, ConvLayer is usually followed by ResBlock. This function will make that. The Conv layers always have 3x3 filters with stride=2. The number of the filters in Conv layer is the same as the out channels of the ResBlock.

Parameters

• in_channels (int) – The number of input channels.

• out_channels (int) – The number of output channels.

• res_repeat (int) – The number of ResBlocks.

• conv_cfg (dict) – Config dict for convolution layer. Default: None.

• norm_cfg (dict) – Dictionary to construct and config norm layer. Default: dict(type=’BN’, requires_grad=True)

• act_cfg (dict) – Config dict for activation layer. Default: dict(type=’LeakyReLU’, negative_slope=0.1).

train(mode=True)

Sets the module in training mode.

This has any effect only on certain modules. See documentations of particular modules for details of their behaviors in training/evaluation mode, if they are affected, e.g. Dropout, BatchNorm, etc.

Parameters

mode (bool) – whether to set training mode (True) or evaluation mode (False). Default: True.

Returns

self

Return type

Module

class mmdet.models.backbones.DetectoRS_ResNeXt(groups=1, base_width=4, **kwargs)

ResNeXt backbone for DetectoRS.

Parameters

• groups (int) – The number of groups in ResNeXt.

• base_width (int) – The base width of ResNeXt.

make_res_layer(**kwargs)

Pack all blocks in a stage into a ResLayer for DetectoRS.

class mmdet.models.backbones.DetectoRS_ResNet(sac=None, stage_with_sac=(False, False, False, False), rfp_inplanes=None, output_img=False, pretrained=None, init_cfg=None, **kwargs)

ResNet backbone for DetectoRS.

Parameters

• sac (dict, optional) – Dictionary to construct SAC (Switchable Atrous Convolution). Default: None.

• stage_with_sac (list) – Which stage to use sac. Default: (False, False, False, False).

• rfp_inplanes (int, optional) – The number of channels from RFP. Default: None. If specified, an additional conv layer will be added for rfp_feat. Otherwise, the structure is the same as base class.

• output_img (bool) – If True, the input image will be inserted into the starting position of output. Default: False.

forward(x)

Forward function.

init_weights()

Initialize the weights.

make_res_layer(**kwargs)

Pack all blocks in a stage into a ResLayer for DetectoRS.

rfp_forward(x, rfp_feats)

Forward function for RFP.

class mmdet.models.backbones.EfficientNet(arch='b0', drop_path_rate=0.0, out_indices=(6,), frozen_stages=0, conv_cfg={'type': 'Conv2dAdaptivePadding'}, norm_cfg={'eps': 0.001, 'type': 'BN'}, act_cfg={'type': 'Swish'}, norm_eval=False, with_cp=False, init_cfg=[{'type': 'Kaiming', 'layer': 'Conv2d'}, {'type': 'Constant', 'layer': ['_BatchNorm', 'GroupNorm'], 'val': 1}])

EfficientNet backbone.

Parameters

• arch (str) – Architecture of efficientnet. Defaults to b0.

• out_indices (Sequence[int]) – Output from which stages. Defaults to (6, ).

• frozen_stages (int) – Stages to be frozen (all param fixed). Defaults to 0, which means not freezing any parameters.

• conv_cfg (dict) – Config dict for convolution layer. Defaults to None, which means using conv2d.

• norm_cfg (dict) – Config dict for normalization layer. Defaults to dict(type=’BN’).

• act_cfg (dict) – Config dict for activation layer. Defaults to dict(type=’Swish’).

• norm_eval (bool) – Whether to set norm layers to eval mode, namely, freeze running stats (mean and var). Note: Effect on Batch Norm and its variants only. Defaults to False.

• with_cp (bool) – Use checkpoint or not. Using checkpoint will save some memory while slowing down the training speed. Defaults to False.

forward(x)

Defines the computation performed at every call.

Should be overridden by all subclasses.

Note

Although the recipe for forward pass needs to be defined within this function, one should call the Module instance afterwards instead of this since the former takes care of running the registered hooks while the latter silently ignores them.

train(mode=True)

Sets the module in training mode.

This has any effect only on certain modules. See documentations of particular modules for details of their behaviors in training/evaluation mode, if they are affected, e.g. Dropout, BatchNorm, etc.

Parameters

mode (bool) – whether to set training mode (True) or evaluation mode (False). Default: True.

Returns

self

Return type

Module

class mmdet.models.backbones.HRNet(extra, in_channels=3, conv_cfg=None, norm_cfg={'type': 'BN'}, norm_eval=True, with_cp=False, zero_init_residual=False, multiscale_output=True, pretrained=None, init_cfg=None)

HRNet backbone.

High-Resolution Representations for Labeling Pixels and Regions arXiv:.

Parameters

• extra (dict) –

  Detailed configuration for each stage of HRNet. There must be 4 stages, the configuration for each stage must have 5 keys:

  • num_modules(int): The number of HRModule in this stage.

  • num_branches(int): The number of branches in the HRModule.

  • block(str): The type of convolution block.

  • num_blocks(tuple): The number of blocks in each branch.

    The length must be equal to num_branches.

  • num_channels(tuple): The number of channels in each branch.

    The length must be equal to num_branches.

• in_channels (int) – Number of input image channels. Default: 3.

• conv_cfg (dict) – Dictionary to construct and config conv layer.

• norm_cfg (dict) – Dictionary to construct and config norm layer.

• norm_eval (bool) – Whether to set norm layers to eval mode, namely, freeze running stats (mean and var). Note: Effect on Batch Norm and its variants only. Default: True.

• with_cp (bool) – Use checkpoint or not. Using checkpoint will save some memory while slowing down the training speed. Default: False.

• zero_init_residual (bool) – Whether to use zero init for last norm layer in resblocks to let them behave as identity. Default: False.

• multiscale_output (bool) – Whether to output multi-level features produced by multiple branches. If False, only the first level feature will be output. Default: True.

• pretrained (str, optional) – Model pretrained path. Default: None.

• init_cfg (dict or list[dict], optional) – Initialization config dict. Default: None.

Example

>>> from mmdet.models import HRNet
>>> import torch
>>> extra = dict(
>>>     stage1=dict(
>>>         num_modules=1,
>>>         num_branches=1,
>>>         block='BOTTLENECK',
>>>         num_blocks=(4, ),
>>>         num_channels=(64, )),
>>>     stage2=dict(
>>>         num_modules=1,
>>>         num_branches=2,
>>>         block='BASIC',
>>>         num_blocks=(4, 4),
>>>         num_channels=(32, 64)),
>>>     stage3=dict(
>>>         num_modules=4,
>>>         num_branches=3,
>>>         block='BASIC',
>>>         num_blocks=(4, 4, 4),
>>>         num_channels=(32, 64, 128)),
>>>     stage4=dict(
>>>         num_modules=3,
>>>         num_branches=4,
>>>         block='BASIC',
>>>         num_blocks=(4, 4, 4, 4),
>>>         num_channels=(32, 64, 128, 256)))
>>> self = HRNet(extra, in_channels=1)
>>> self.eval()
>>> inputs = torch.rand(1, 1, 32, 32)
>>> level_outputs = self.forward(inputs)
>>> for level_out in level_outputs:
...     print(tuple(level_out.shape))
(1, 32, 8, 8)
(1, 64, 4, 4)
(1, 128, 2, 2)
(1, 256, 1, 1)

forward(x)

Forward function.

property norm1

the normalization layer named “norm1”

Type

nn.Module

property norm2

the normalization layer named “norm2”

Type

nn.Module

train(mode=True)

Convert the model into training mode will keeping the normalization layer freezed.

class mmdet.models.backbones.HourglassNet(downsample_times=5, num_stacks=2, stage_channels=(256, 256, 384, 384, 384, 512), stage_blocks=(2, 2, 2, 2, 2, 4), feat_channel=256, norm_cfg={'requires_grad': True, 'type': 'BN'}, pretrained=None, init_cfg=None)

HourglassNet backbone.

Stacked Hourglass Networks for Human Pose Estimation. More details can be found in the paper .

Parameters

• downsample_times (int) – Downsample times in a HourglassModule.

• num_stacks (int) – Number of HourglassModule modules stacked, 1 for Hourglass-52, 2 for Hourglass-104.

• stage_channels (list[int]) – Feature channel of each sub-module in a HourglassModule.

• stage_blocks (list[int]) – Number of sub-modules stacked in a HourglassModule.

• feat_channel (int) – Feature channel of conv after a HourglassModule.

• norm_cfg (dict) – Dictionary to construct and config norm layer.

• pretrained (str, optional) – model pretrained path. Default: None

• init_cfg (dict or list[dict], optional) – Initialization config dict. Default: None

Example

>>> from mmdet.models import HourglassNet
>>> import torch
>>> self = HourglassNet()
>>> self.eval()
>>> inputs = torch.rand(1, 3, 511, 511)
>>> level_outputs = self.forward(inputs)
>>> for level_output in level_outputs:
...     print(tuple(level_output.shape))
(1, 256, 128, 128)
(1, 256, 128, 128)

forward(x)

Forward function.

init_weights()

Init module weights.

class mmdet.models.backbones.MobileNetV2(widen_factor=1.0, out_indices=(1, 2, 4, 7), frozen_stages=-1, conv_cfg=None, norm_cfg={'type': 'BN'}, act_cfg={'type': 'ReLU6'}, norm_eval=False, with_cp=False, pretrained=None, init_cfg=None)

MobileNetV2 backbone.

Parameters

• widen_factor (float) – Width multiplier, multiply number of channels in each layer by this amount. Default: 1.0.

• out_indices (Sequence[int], optional) – Output from which stages. Default: (1, 2, 4, 7).

• frozen_stages (int) – Stages to be frozen (all param fixed). Default: -1, which means not freezing any parameters.

• conv_cfg (dict, optional) – Config dict for convolution layer. Default: None, which means using conv2d.

• norm_cfg (dict) – Config dict for normalization layer. Default: dict(type=’BN’).

• act_cfg (dict) – Config dict for activation layer. Default: dict(type=’ReLU6’).

• norm_eval (bool) – Whether to set norm layers to eval mode, namely, freeze running stats (mean and var). Note: Effect on Batch Norm and its variants only. Default: False.

• with_cp (bool) – Use checkpoint or not. Using checkpoint will save some memory while slowing down the training speed. Default: False.

• pretrained (str, optional) – model pretrained path. Default: None

• init_cfg (dict or list[dict], optional) – Initialization config dict. Default: None

forward(x)

Forward function.

make_layer(out_channels, num_blocks, stride, expand_ratio)

Stack InvertedResidual blocks to build a layer for MobileNetV2.

Parameters

• out_channels (int) – out_channels of block.

• num_blocks (int) – number of blocks.

• stride (int) – stride of the first block. Default: 1

• expand_ratio (int) – Expand the number of channels of the hidden layer in InvertedResidual by this ratio. Default: 6.

train(mode=True)

Convert the model into training mode while keep normalization layer frozen.

class mmdet.models.backbones.PyramidVisionTransformer(pretrain_img_size=224, in_channels=3, embed_dims=64, num_stages=4, num_layers=[3, 4, 6, 3], num_heads=[1, 2, 5, 8], patch_sizes=[4, 2, 2, 2], strides=[4, 2, 2, 2], paddings=[0, 0, 0, 0], sr_ratios=[8, 4, 2, 1], out_indices=(0, 1, 2, 3), mlp_ratios=[8, 8, 4, 4], qkv_bias=True, drop_rate=0.0, attn_drop_rate=0.0, drop_path_rate=0.1, use_abs_pos_embed=True, norm_after_stage=False, use_conv_ffn=False, act_cfg={'type': 'GELU'}, norm_cfg={'eps': 1e-06, 'type': 'LN'}, pretrained=None, convert_weights=True, init_cfg=None)

Pyramid Vision Transformer (PVT)

Implementation of Pyramid Vision Transformer: A Versatile Backbone for Dense Prediction without Convolutions.

Parameters

• pretrain_img_size (int | tuple[int]) – The size of input image when pretrain. Defaults: 224.

• in_channels (int) – Number of input channels. Default: 3.

• embed_dims (int) – Embedding dimension. Default: 64.

• num_stags (int) – The num of stages. Default: 4.

• num_layers (Sequence[int]) – The layer number of each transformer encode layer. Default: [3, 4, 6, 3].

• num_heads (Sequence[int]) – The attention heads of each transformer encode layer. Default: [1, 2, 5, 8].

• patch_sizes (Sequence[int]) – The patch_size of each patch embedding. Default: [4, 2, 2, 2].

• strides (Sequence[int]) – The stride of each patch embedding. Default: [4, 2, 2, 2].

• paddings (Sequence[int]) – The padding of each patch embedding. Default: [0, 0, 0, 0].

• sr_ratios (Sequence[int]) – The spatial reduction rate of each transformer encode layer. Default: [8, 4, 2, 1].

• out_indices (Sequence[int] | int) – Output from which stages. Default: (0, 1, 2, 3).

• mlp_ratios (Sequence[int]) – The ratio of the mlp hidden dim to the embedding dim of each transformer encode layer. Default: [8, 8, 4, 4].

• qkv_bias (bool) – Enable bias for qkv if True. Default: True.

• drop_rate (float) – Probability of an element to be zeroed. Default 0.0.

• attn_drop_rate (float) – The drop out rate for attention layer. Default 0.0.

• drop_path_rate (float) – stochastic depth rate. Default 0.1.

• use_abs_pos_embed (bool) – If True, add absolute position embedding to the patch embedding. Defaults: True.

• use_conv_ffn (bool) – If True, use Convolutional FFN to replace FFN. Default: False.

• act_cfg (dict) – The activation config for FFNs. Default: dict(type=’GELU’).

• norm_cfg (dict) – Config dict for normalization layer. Default: dict(type=’LN’).

• pretrained (str, optional) – model pretrained path. Default: None.

• convert_weights (bool) – The flag indicates whether the pre-trained model is from the original repo. We may need to convert some keys to make it compatible. Default: True.

• init_cfg (dict or list[dict], optional) – Initialization config dict. Default: None.

forward(x)

Defines the computation performed at every call.

Should be overridden by all subclasses.

Note

Although the recipe for forward pass needs to be defined within this function, one should call the Module instance afterwards instead of this since the former takes care of running the registered hooks while the latter silently ignores them.

init_weights()

Initialize the weights.

class mmdet.models.backbones.PyramidVisionTransformerV2(**kwargs)

Implementation of PVTv2: Improved Baselines with Pyramid Vision Transformer.

class mmdet.models.backbones.RegNet(arch, in_channels=3, stem_channels=32, base_channels=32, strides=(2, 2, 2, 2), dilations=(1, 1, 1, 1), out_indices=(0, 1, 2, 3), style='pytorch', deep_stem=False, avg_down=False, frozen_stages=-1, conv_cfg=None, norm_cfg={'requires_grad': True, 'type': 'BN'}, norm_eval=True, dcn=None, stage_with_dcn=(False, False, False, False), plugins=None, with_cp=False, zero_init_residual=True, pretrained=None, init_cfg=None)

RegNet backbone.

More details can be found in paper .

Parameters

• arch (dict) –

  The parameter of RegNets.

  • w0 (int): initial width

  • wa (float): slope of width

  • wm (float): quantization parameter to quantize the width

  • depth (int): depth of the backbone

  • group_w (int): width of group

  • bot_mul (float): bottleneck ratio, i.e. expansion of bottleneck.

• strides (Sequence[int]) – Strides of the first block of each stage.

• base_channels (int) – Base channels after stem layer.

• in_channels (int) – Number of input image channels. Default: 3.

• dilations (Sequence[int]) – Dilation of each stage.

• out_indices (Sequence[int]) – Output from which stages.

• style (str) – pytorch or caffe. If set to “pytorch”, the stride-two layer is the 3x3 conv layer, otherwise the stride-two layer is the first 1x1 conv layer.

• frozen_stages (int) – Stages to be frozen (all param fixed). -1 means not freezing any parameters.

• norm_cfg (dict) – dictionary to construct and config norm layer.

• norm_eval (bool) – Whether to set norm layers to eval mode, namely, freeze running stats (mean and var). Note: Effect on Batch Norm and its variants only.

• with_cp (bool) – Use checkpoint or not. Using checkpoint will save some memory while slowing down the training speed.

• zero_init_residual (bool) – whether to use zero init for last norm layer in resblocks to let them behave as identity.

• pretrained (str, optional) – model pretrained path. Default: None

• init_cfg (dict or list[dict], optional) – Initialization config dict. Default: None

Example

>>> from mmdet.models import RegNet
>>> import torch
>>> self = RegNet(
        arch=dict(
            w0=88,
            wa=26.31,
            wm=2.25,
            group_w=48,
            depth=25,
            bot_mul=1.0))
>>> self.eval()
>>> inputs = torch.rand(1, 3, 32, 32)
>>> level_outputs = self.forward(inputs)
>>> for level_out in level_outputs:
...     print(tuple(level_out.shape))
(1, 96, 8, 8)
(1, 192, 4, 4)
(1, 432, 2, 2)
(1, 1008, 1, 1)

adjust_width_group(widths, bottleneck_ratio, groups)

Adjusts the compatibility of widths and groups.

Parameters

• widths (list[int]) – Width of each stage.

• bottleneck_ratio (float) – Bottleneck ratio.

• groups (int) – number of groups in each stage

Returns

The adjusted widths and groups of each stage.

Return type

tuple(list)

forward(x)

Forward function.

generate_regnet(initial_width, width_slope, width_parameter, depth, divisor=8)

Generates per block width from RegNet parameters.

Parameters

• initial_width ([int]) – Initial width of the backbone

• width_slope ([float]) – Slope of the quantized linear function

• width_parameter ([int]) – Parameter used to quantize the width.

• depth ([int]) – Depth of the backbone.

• divisor (int, optional) – The divisor of channels. Defaults to 8.

Returns

return a list of widths of each stage and the number of stages

Return type

list, int

get_stages_from_blocks(widths)

Gets widths/stage_blocks of network at each stage.

Parameters

widths (list[int]) – Width in each stage.

Returns

width and depth of each stage

Return type

tuple(list)

static quantize_float(number, divisor)

Converts a float to closest non-zero int divisible by divisor.

Parameters

• number (int) – Original number to be quantized.

• divisor (int) – Divisor used to quantize the number.

Returns

quantized number that is divisible by devisor.

Return type

int

class mmdet.models.backbones.Res2Net(scales=4, base_width=26, style='pytorch', deep_stem=True, avg_down=True, pretrained=None, init_cfg=None, **kwargs)

Res2Net backbone.

Parameters

• scales (int) – Scales used in Res2Net. Default: 4

• base_width (int) – Basic width of each scale. Default: 26

• depth (int) – Depth of res2net, from {50, 101, 152}.

• in_channels (int) – Number of input image channels. Default: 3.

• num_stages (int) – Res2net stages. Default: 4.

• strides (Sequence[int]) – Strides of the first block of each stage.

• dilations (Sequence[int]) – Dilation of each stage.

• out_indices (Sequence[int]) – Output from which stages.

• style (str) – pytorch or caffe. If set to “pytorch”, the stride-two layer is the 3x3 conv layer, otherwise the stride-two layer is the first 1x1 conv layer.

• deep_stem (bool) – Replace 7x7 conv in input stem with 3 3x3 conv

• avg_down (bool) – Use AvgPool instead of stride conv when downsampling in the bottle2neck.

• frozen_stages (int) – Stages to be frozen (stop grad and set eval mode). -1 means not freezing any parameters.

• norm_cfg (dict) – Dictionary to construct and config norm layer.

• norm_eval (bool) – Whether to set norm layers to eval mode, namely, freeze running stats (mean and var). Note: Effect on Batch Norm and its variants only.

• plugins (list[dict]) –

  List of plugins for stages, each dict contains:

  • cfg (dict, required): Cfg dict to build plugin.

  • position (str, required): Position inside block to insert plugin, options are ‘after_conv1’, ‘after_conv2’, ‘after_conv3’.

  • stages (tuple[bool], optional): Stages to apply plugin, length should be same as ‘num_stages’.

• with_cp (bool) – Use checkpoint or not. Using checkpoint will save some memory while slowing down the training speed.

• zero_init_residual (bool) – Whether to use zero init for last norm layer in resblocks to let them behave as identity.

• pretrained (str, optional) – model pretrained path. Default: None

• init_cfg (dict or list[dict], optional) – Initialization config dict. Default: None

Example

>>> from mmdet.models import Res2Net
>>> import torch
>>> self = Res2Net(depth=50, scales=4, base_width=26)
>>> self.eval()
>>> inputs = torch.rand(1, 3, 32, 32)
>>> level_outputs = self.forward(inputs)
>>> for level_out in level_outputs:
...     print(tuple(level_out.shape))
(1, 256, 8, 8)
(1, 512, 4, 4)
(1, 1024, 2, 2)
(1, 2048, 1, 1)

make_res_layer(**kwargs)

Pack all blocks in a stage into a ResLayer.

class mmdet.models.backbones.ResNeSt(groups=1, base_width=4, radix=2, reduction_factor=4, avg_down_stride=True, **kwargs)

ResNeSt backbone.

Parameters

• groups (int) – Number of groups of Bottleneck. Default: 1

• base_width (int) – Base width of Bottleneck. Default: 4

• radix (int) – Radix of SplitAttentionConv2d. Default: 2

• reduction_factor (int) – Reduction factor of inter_channels in SplitAttentionConv2d. Default: 4.

• avg_down_stride (bool) – Whether to use average pool for stride in Bottleneck. Default: True.

• kwargs (dict) – Keyword arguments for ResNet.

make_res_layer(**kwargs)

Pack all blocks in a stage into a ResLayer.

class mmdet.models.backbones.ResNeXt(groups=1, base_width=4, **kwargs)

ResNeXt backbone.

Parameters

• depth (int) – Depth of resnet, from {18, 34, 50, 101, 152}.

• in_channels (int) – Number of input image channels. Default: 3.

• num_stages (int) – Resnet stages. Default: 4.

• groups (int) – Group of resnext.

• base_width (int) – Base width of resnext.

• strides (Sequence[int]) – Strides of the first block of each stage.

• dilations (Sequence[int]) – Dilation of each stage.

• out_indices (Sequence[int]) – Output from which stages.

• style (str) – pytorch or caffe. If set to “pytorch”, the stride-two layer is the 3x3 conv layer, otherwise the stride-two layer is the first 1x1 conv layer.

• frozen_stages (int) – Stages to be frozen (all param fixed). -1 means not freezing any parameters.

• norm_cfg (dict) – dictionary to construct and config norm layer.

• norm_eval (bool) – Whether to set norm layers to eval mode, namely, freeze running stats (mean and var). Note: Effect on Batch Norm and its variants only.

• with_cp (bool) – Use checkpoint or not. Using checkpoint will save some memory while slowing down the training speed.

• zero_init_residual (bool) – whether to use zero init for last norm layer in resblocks to let them behave as identity.

make_res_layer(**kwargs)

Pack all blocks in a stage into a ResLayer

class mmdet.models.backbones.ResNet(depth, in_channels=3, stem_channels=None, base_channels=64, num_stages=4, strides=(1, 2, 2, 2), dilations=(1, 1, 1, 1), out_indices=(0, 1, 2, 3), style='pytorch', deep_stem=False, avg_down=False, frozen_stages=-1, conv_cfg=None, norm_cfg={'requires_grad': True, 'type': 'BN'}, norm_eval=True, dcn=None, stage_with_dcn=(False, False, False, False), plugins=None, with_cp=False, zero_init_residual=True, pretrained=None, init_cfg=None)

ResNet backbone.

Parameters

• depth (int) – Depth of resnet, from {18, 34, 50, 101, 152}.

• stem_channels (int | None) – Number of stem channels. If not specified, it will be the same as base_channels. Default: None.

• base_channels (int) – Number of base channels of res layer. Default: 64.

• in_channels (int) – Number of input image channels. Default: 3.

• num_stages (int) – Resnet stages. Default: 4.

• strides (Sequence[int]) – Strides of the first block of each stage.

• dilations (Sequence[int]) – Dilation of each stage.

• out_indices (Sequence[int]) – Output from which stages.

• style (str) – pytorch or caffe. If set to “pytorch”, the stride-two layer is the 3x3 conv layer, otherwise the stride-two layer is the first 1x1 conv layer.

• deep_stem (bool) – Replace 7x7 conv in input stem with 3 3x3 conv

• avg_down (bool) – Use AvgPool instead of stride conv when downsampling in the bottleneck.

• frozen_stages (int) – Stages to be frozen (stop grad and set eval mode). -1 means not freezing any parameters.

• norm_cfg (dict) – Dictionary to construct and config norm layer.

• norm_eval (bool) – Whether to set norm layers to eval mode, namely, freeze running stats (mean and var). Note: Effect on Batch Norm and its variants only.

• plugins (list[dict]) –

  List of plugins for stages, each dict contains:

  • cfg (dict, required): Cfg dict to build plugin.

  • position (str, required): Position inside block to insert plugin, options are ‘after_conv1’, ‘after_conv2’, ‘after_conv3’.

  • stages (tuple[bool], optional): Stages to apply plugin, length should be same as ‘num_stages’.

• with_cp (bool) – Use checkpoint or not. Using checkpoint will save some memory while slowing down the training speed.

• zero_init_residual (bool) – Whether to use zero init for last norm layer in resblocks to let them behave as identity.

• pretrained (str, optional) – model pretrained path. Default: None

• init_cfg (dict or list[dict], optional) – Initialization config dict. Default: None

Example

>>> from mmdet.models import ResNet
>>> import torch
>>> self = ResNet(depth=18)
>>> self.eval()
>>> inputs = torch.rand(1, 3, 32, 32)
>>> level_outputs = self.forward(inputs)
>>> for level_out in level_outputs:
...     print(tuple(level_out.shape))
(1, 64, 8, 8)
(1, 128, 4, 4)
(1, 256, 2, 2)
(1, 512, 1, 1)

forward(x)

Forward function.

make_res_layer(**kwargs)

Pack all blocks in a stage into a ResLayer.

make_stage_plugins(plugins, stage_idx)

Make plugins for ResNet stage_idx th stage.

Currently we support to insert context_block, empirical_attention_block, nonlocal_block into the backbone like ResNet/ResNeXt. They could be inserted after conv1/conv2/conv3 of Bottleneck.

An example of plugins format could be:

Examples

>>> plugins=[
...     dict(cfg=dict(type='xxx', arg1='xxx'),
...          stages=(False, True, True, True),
...          position='after_conv2'),
...     dict(cfg=dict(type='yyy'),
...          stages=(True, True, True, True),
...          position='after_conv3'),
...     dict(cfg=dict(type='zzz', postfix='1'),
...          stages=(True, True, True, True),
...          position='after_conv3'),
...     dict(cfg=dict(type='zzz', postfix='2'),
...          stages=(True, True, True, True),
...          position='after_conv3')
... ]
>>> self = ResNet(depth=18)
>>> stage_plugins = self.make_stage_plugins(plugins, 0)
>>> assert len(stage_plugins) == 3

Suppose stage_idx=0, the structure of blocks in the stage would be:

conv1-> conv2->conv3->yyy->zzz1->zzz2

Suppose ‘stage_idx=1’, the structure of blocks in the stage would be:

conv1-> conv2->xxx->conv3->yyy->zzz1->zzz2

If stages is missing, the plugin would be applied to all stages.

Parameters

• plugins (list[dict]) – List of plugins cfg to build. The postfix is required if multiple same type plugins are inserted.

• stage_idx (int) – Index of stage to build

Returns

Plugins for current stage

Return type

list[dict]

property norm1

the normalization layer named “norm1”

Type

nn.Module

train(mode=True)

Convert the model into training mode while keep normalization layer freezed.

class mmdet.models.backbones.ResNetV1d(**kwargs)

ResNetV1d variant described in Bag of Tricks.

Compared with default ResNet(ResNetV1b), ResNetV1d replaces the 7x7 conv in the input stem with three 3x3 convs. And in the downsampling block, a 2x2 avg_pool with stride 2 is added before conv, whose stride is changed to 1.

class mmdet.models.backbones.SSDVGG(depth, with_last_pool=False, ceil_mode=True, out_indices=(3, 4), out_feature_indices=(22, 34), pretrained=None, init_cfg=None, input_size=None, l2_norm_scale=None)

VGG Backbone network for single-shot-detection.

Parameters

• depth (int) – Depth of vgg, from {11, 13, 16, 19}.

• with_last_pool (bool) – Whether to add a pooling layer at the last of the model

• ceil_mode (bool) – When True, will use ceil instead of floor to compute the output shape.

• out_indices (Sequence[int]) – Output from which stages.

• out_feature_indices (Sequence[int]) – Output from which feature map.

• pretrained (str, optional) – model pretrained path. Default: None

• init_cfg (dict or list[dict], optional) – Initialization config dict. Default: None

• input_size (int, optional) – Deprecated argumment. Width and height of input, from {300, 512}.

• l2_norm_scale (float, optional) – Deprecated argumment. L2 normalization layer init scale.

Example

>>> self = SSDVGG(input_size=300, depth=11)
>>> self.eval()
>>> inputs = torch.rand(1, 3, 300, 300)
>>> level_outputs = self.forward(inputs)
>>> for level_out in level_outputs:
...     print(tuple(level_out.shape))
(1, 1024, 19, 19)
(1, 512, 10, 10)
(1, 256, 5, 5)
(1, 256, 3, 3)
(1, 256, 1, 1)

forward(x)

Forward function.

init_weights(pretrained=None)

Initialize the weights.

class mmdet.models.backbones.SwinTransformer(pretrain_img_size=224, in_channels=3, embed_dims=96, patch_size=4, window_size=7, mlp_ratio=4, depths=(2, 2, 6, 2), num_heads=(3, 6, 12, 24), strides=(4, 2, 2, 2), out_indices=(0, 1, 2, 3), qkv_bias=True, qk_scale=None, patch_norm=True, drop_rate=0.0, attn_drop_rate=0.0, drop_path_rate=0.1, use_abs_pos_embed=False, act_cfg={'type': 'GELU'}, norm_cfg={'type': 'LN'}, with_cp=False, pretrained=None, convert_weights=False, frozen_stages=-1, init_cfg=None)

Swin Transformer A PyTorch implement of : Swin Transformer: Hierarchical Vision Transformer using Shifted Windows -

https://arxiv.org/abs/2103.14030

Inspiration from https://github.com/microsoft/Swin-Transformer

Parameters

• pretrain_img_size (int | tuple[int]) – The size of input image when pretrain. Defaults: 224.

• in_channels (int) – The num of input channels. Defaults: 3.

• embed_dims (int) – The feature dimension. Default: 96.

• patch_size (int | tuple[int]) – Patch size. Default: 4.

• window_size (int) – Window size. Default: 7.

• mlp_ratio (int) – Ratio of mlp hidden dim to embedding dim. Default: 4.

• depths (tuple[int]) – Depths of each Swin Transformer stage. Default: (2, 2, 6, 2).

• num_heads (tuple[int]) – Parallel attention heads of each Swin Transformer stage. Default: (3, 6, 12, 24).

• strides (tuple[int]) – The patch merging or patch embedding stride of each Swin Transformer stage. (In swin, we set kernel size equal to stride.) Default: (4, 2, 2, 2).

• out_indices (tuple[int]) – Output from which stages. Default: (0, 1, 2, 3).

• qkv_bias (bool, optional) – If True, add a learnable bias to query, key, value. Default: True

• qk_scale (float | None, optional) – Override default qk scale of head_dim ** -0.5 if set. Default: None.

• patch_norm (bool) – If add a norm layer for patch embed and patch merging. Default: True.

• drop_rate (float) – Dropout rate. Defaults: 0.

• attn_drop_rate (float) – Attention dropout rate. Default: 0.

• drop_path_rate (float) – Stochastic depth rate. Defaults: 0.1.

• use_abs_pos_embed (bool) – If True, add absolute position embedding to the patch embedding. Defaults: False.

• act_cfg (dict) – Config dict for activation layer. Default: dict(type=’GELU’).

• norm_cfg (dict) – Config dict for normalization layer at output of backone. Defaults: dict(type=’LN’).

• with_cp (bool, optional) – Use checkpoint or not. Using checkpoint will save some memory while slowing down the training speed. Default: False.

• pretrained (str, optional) – model pretrained path. Default: None.

• convert_weights (bool) – The flag indicates whether the pre-trained model is from the original repo. We may need to convert some keys to make it compatible. Default: False.

• frozen_stages (int) – Stages to be frozen (stop grad and set eval mode). Default: -1 (-1 means not freezing any parameters).

• init_cfg (dict, optional) – The Config for initialization. Defaults to None.

forward(x)

Defines the computation performed at every call.

Should be overridden by all subclasses.

Note

Although the recipe for forward pass needs to be defined within this function, one should call the Module instance afterwards instead of this since the former takes care of running the registered hooks while the latter silently ignores them.

init_weights()

Initialize the weights.

train(mode=True)

Convert the model into training mode while keep layers freezed.

class mmdet.models.backbones.TridentResNet(depth, num_branch, test_branch_idx, trident_dilations, **kwargs)

The stem layer, stage 1 and stage 2 in Trident ResNet are identical to ResNet, while in stage 3, Trident BottleBlock is utilized to replace the normal BottleBlock to yield trident output. Different branch shares the convolution weight but uses different dilations to achieve multi-scale output.

/ stage3(b0) x - stem - stage1 - stage2 - stage3(b1) - output stage3(b2) /

Parameters

• depth (int) – Depth of resnet, from {50, 101, 152}.

• num_branch (int) – Number of branches in TridentNet.

• test_branch_idx (int) – In inference, all 3 branches will be used if test_branch_idx==-1, otherwise only branch with index test_branch_idx will be used.

• trident_dilations (tuple[int]) – Dilations of different trident branch. len(trident_dilations) should be equal to num_branch.

necks

class mmdet.models.necks.BFP(Balanced Feature Pyramids)

BFP takes multi-level features as inputs and gather them into a single one, then refine the gathered feature and scatter the refined results to multi-level features. This module is used in Libra R-CNN (CVPR 2019), see the paper Libra R-CNN: Towards Balanced Learning for Object Detection for details.

Parameters

• in_channels (int) – Number of input channels (feature maps of all levels should have the same channels).

• num_levels (int) – Number of input feature levels.

• conv_cfg (dict) – The config dict for convolution layers.

• norm_cfg (dict) – The config dict for normalization layers.

• refine_level (int) – Index of integration and refine level of BSF in multi-level features from bottom to top.

• refine_type (str) – Type of the refine op, currently support [None, ‘conv’, ‘non_local’].

• init_cfg (dict or list[dict], optional) – Initialization config dict.

forward(inputs)

Forward function.

class mmdet.models.necks.CTResNetNeck(in_channel, num_deconv_filters, num_deconv_kernels, use_dcn=True, init_cfg=None)

The neck used in CenterNet for object classification and box regression.

Parameters

• in_channel (int) – Number of input channels.

• num_deconv_filters (tuple[int]) – Number of filters per stage.

• num_deconv_kernels (tuple[int]) – Number of kernels per stage.

• use_dcn (bool) – If True, use DCNv2. Default: True.

• init_cfg (dict or list[dict], optional) – Initialization config dict.

forward(inputs)

Defines the computation performed at every call.

Should be overridden by all subclasses.

Note

Although the recipe for forward pass needs to be defined within this function, one should call the Module instance afterwards instead of this since the former takes care of running the registered hooks while the latter silently ignores them.

init_weights()

Initialize the weights.

class mmdet.models.necks.ChannelMapper(in_channels, out_channels, kernel_size=3, conv_cfg=None, norm_cfg=None, act_cfg={'type': 'ReLU'}, num_outs=None, init_cfg={'distribution': 'uniform', 'layer': 'Conv2d', 'type': 'Xavier'})

Channel Mapper to reduce/increase channels of backbone features.

This is used to reduce/increase channels of backbone features.

Parameters

• in_channels (List[int]) – Number of input channels per scale.

• out_channels (int) – Number of output channels (used at each scale).

• kernel_size (int, optional) – kernel_size for reducing channels (used at each scale). Default: 3.

• conv_cfg (dict, optional) – Config dict for convolution layer. Default: None.

• norm_cfg (dict, optional) – Config dict for normalization layer. Default: None.

• act_cfg (dict, optional) – Config dict for activation layer in ConvModule. Default: dict(type=’ReLU’).

• num_outs (int, optional) – Number of output feature maps. There would be extra_convs when num_outs larger than the length of in_channels.

• init_cfg (dict or list[dict], optional) – Initialization config dict.

Example

>>> import torch
>>> in_channels = [2, 3, 5, 7]
>>> scales = [340, 170, 84, 43]
>>> inputs = [torch.rand(1, c, s, s)
...           for c, s in zip(in_channels, scales)]
>>> self = ChannelMapper(in_channels, 11, 3).eval()
>>> outputs = self.forward(inputs)
>>> for i in range(len(outputs)):
...     print(f'outputs[{i}].shape = {outputs[i].shape}')
outputs[0].shape = torch.Size([1, 11, 340, 340])
outputs[1].shape = torch.Size([1, 11, 170, 170])
outputs[2].shape = torch.Size([1, 11, 84, 84])
outputs[3].shape = torch.Size([1, 11, 43, 43])

forward(inputs)

Forward function.

class mmdet.models.necks.DilatedEncoder(in_channels, out_channels, block_mid_channels, num_residual_blocks, block_dilations)

Dilated Encoder for YOLOF <https://arxiv.org/abs/2103.09460>`.

This module contains two types of components:

• the original FPN lateral convolution layer and fpn convolution layer,

  which are 1x1 conv + 3x3 conv

• the dilated residual block

Parameters

• in_channels (int) – The number of input channels.

• out_channels (int) – The number of output channels.

• block_mid_channels (int) – The number of middle block output channels

• num_residual_blocks (int) – The number of residual blocks.

• block_dilations (list) – The list of residual blocks dilation.

forward(feature)

Defines the computation performed at every call.

Should be overridden by all subclasses.

Note

Although the recipe for forward pass needs to be defined within this function, one should call the Module instance afterwards instead of this since the former takes care of running the registered hooks while the latter silently ignores them.

class mmdet.models.necks.DyHead(in_channels, out_channels, num_blocks=6, zero_init_offset=True, init_cfg=None)

DyHead neck consisting of multiple DyHead Blocks.

See Dynamic Head: Unifying Object Detection Heads with Attentions for details.

Parameters

• in_channels (int) – Number of input channels.

• out_channels (int) – Number of output channels.

• num_blocks (int, optional) – Number of DyHead Blocks. Default: 6.

• zero_init_offset (bool, optional) – Whether to use zero init for spatial_conv_offset. Default: True.

• init_cfg (dict or list[dict], optional) – Initialization config dict. Default: None.

forward(inputs)

Forward function.

class mmdet.models.necks.FPG(in_channels, out_channels, num_outs, stack_times, paths, inter_channels=None, same_down_trans=None, same_up_trans={'kernel_size': 3, 'padding': 1, 'stride': 2, 'type': 'conv'}, across_lateral_trans={'kernel_size': 1, 'type': 'conv'}, across_down_trans={'kernel_size': 3, 'type': 'conv'}, across_up_trans=None, across_skip_trans={'type': 'identity'}, output_trans={'kernel_size': 3, 'type': 'last_conv'}, start_level=0, end_level=-1, add_extra_convs=False, norm_cfg=None, skip_inds=None, init_cfg=[{'type': 'Caffe2Xavier', 'layer': 'Conv2d'}, {'type': 'Constant', 'layer': ['_BatchNorm', '_InstanceNorm', 'GroupNorm', 'LayerNorm'], 'val': 1.0}])

FPG.

Implementation of Feature Pyramid Grids (FPG). This implementation only gives the basic structure stated in the paper. But users can implement different type of transitions to fully explore the the potential power of the structure of FPG.

Parameters

• in_channels (int) – Number of input channels (feature maps of all levels should have the same channels).

• out_channels (int) – Number of output channels (used at each scale)

• num_outs (int) – Number of output scales.

• stack_times (int) – The number of times the pyramid architecture will be stacked.

• paths (list[str]) – Specify the path order of each stack level. Each element in the list should be either ‘bu’ (bottom-up) or ‘td’ (top-down).

• inter_channels (int) – Number of inter channels.

• same_up_trans (dict) – Transition that goes down at the same stage.

• same_down_trans (dict) – Transition that goes up at the same stage.

• across_lateral_trans (dict) – Across-pathway same-stage

• across_down_trans (dict) – Across-pathway bottom-up connection.

• across_up_trans (dict) – Across-pathway top-down connection.

• across_skip_trans (dict) – Across-pathway skip connection.

• output_trans (dict) – Transition that trans the output of the last stage.

• start_level (int) – Index of the start input backbone level used to build the feature pyramid. Default: 0.

• end_level (int) – Index of the end input backbone level (exclusive) to build the feature pyramid. Default: -1, which means the last level.

• add_extra_convs (bool) – It decides whether to add conv layers on top of the original feature maps. Default to False. If True, its actual mode is specified by extra_convs_on_inputs.

• norm_cfg (dict) – Config dict for normalization layer. Default: None.

• init_cfg (dict or list[dict], optional) – Initialization config dict.

forward(inputs)

Defines the computation performed at every call.

Should be overridden by all subclasses.

Note

Although the recipe for forward pass needs to be defined within this function, one should call the Module instance afterwards instead of this since the former takes care of running the registered hooks while the latter silently ignores them.

class mmdet.models.necks.FPN(in_channels, out_channels, num_outs, start_level=0, end_level=-1, add_extra_convs=False, relu_before_extra_convs=False, no_norm_on_lateral=False, conv_cfg=None, norm_cfg=None, act_cfg=None, upsample_cfg={'mode': 'nearest'}, init_cfg={'distribution': 'uniform', 'layer': 'Conv2d', 'type': 'Xavier'})

Feature Pyramid Network.

This is an implementation of paper Feature Pyramid Networks for Object Detection.

Parameters

• in_channels (list[int]) – Number of input channels per scale.

• out_channels (int) – Number of output channels (used at each scale).

• num_outs (int) – Number of output scales.

• start_level (int) – Index of the start input backbone level used to build the feature pyramid. Default: 0.

• end_level (int) – Index of the end input backbone level (exclusive) to build the feature pyramid. Default: -1, which means the last level.

• add_extra_convs (bool | str) –

  If bool, it decides whether to add conv layers on top of the original feature maps. Default to False. If True, it is equivalent to add_extra_convs=’on_input’. If str, it specifies the source feature map of the extra convs. Only the following options are allowed

  • ’on_input’: Last feat map of neck inputs (i.e. backbone feature).

  • ’on_lateral’: Last feature map after lateral convs.

  • ’on_output’: The last output feature map after fpn convs.

• relu_before_extra_convs (bool) – Whether to apply relu before the extra conv. Default: False.

• no_norm_on_lateral (bool) – Whether to apply norm on lateral. Default: False.

• conv_cfg (dict) – Config dict for convolution layer. Default: None.

• norm_cfg (dict) – Config dict for normalization layer. Default: None.

• act_cfg (dict) – Config dict for activation layer in ConvModule. Default: None.

• upsample_cfg (dict) – Config dict for interpolate layer. Default: dict(mode=’nearest’).

• init_cfg (dict or list[dict], optional) – Initialization config dict.

Example

>>> import torch
>>> in_channels = [2, 3, 5, 7]
>>> scales = [340, 170, 84, 43]
>>> inputs = [torch.rand(1, c, s, s)
...           for c, s in zip(in_channels, scales)]
>>> self = FPN(in_channels, 11, len(in_channels)).eval()
>>> outputs = self.forward(inputs)
>>> for i in range(len(outputs)):
...     print(f'outputs[{i}].shape = {outputs[i].shape}')
outputs[0].shape = torch.Size([1, 11, 340, 340])
outputs[1].shape = torch.Size([1, 11, 170, 170])
outputs[2].shape = torch.Size([1, 11, 84, 84])
outputs[3].shape = torch.Size([1, 11, 43, 43])

forward(inputs)

Forward function.

class mmdet.models.necks.FPN_CARAFE(in_channels, out_channels, num_outs, start_level=0, end_level=-1, norm_cfg=None, act_cfg=None, order=('conv', 'norm', 'act'), upsample_cfg={'encoder_dilation': 1, 'encoder_kernel': 3, 'type': 'carafe', 'up_group': 1, 'up_kernel': 5}, init_cfg=None)

FPN_CARAFE is a more flexible implementation of FPN. It allows more choice for upsample methods during the top-down pathway.

It can reproduce the performance of ICCV 2019 paper CARAFE: Content-Aware ReAssembly of FEatures Please refer to https://arxiv.org/abs/1905.02188 for more details.

Parameters

• in_channels (list[int]) – Number of channels for each input feature map.

• out_channels (int) – Output channels of feature pyramids.

• num_outs (int) – Number of output stages.

• start_level (int) – Start level of feature pyramids. (Default: 0)

• end_level (int) – End level of feature pyramids. (Default: -1 indicates the last level).

• norm_cfg (dict) – Dictionary to construct and config norm layer.

• activate (str) – Type of activation function in ConvModule (Default: None indicates w/o activation).

• order (dict) – Order of components in ConvModule.

• upsample (str) – Type of upsample layer.

• upsample_cfg (dict) – Dictionary to construct and config upsample layer.

• init_cfg (dict or list[dict], optional) – Initialization config dict. Default: None

forward(inputs)

Forward function.

init_weights()

Initialize the weights of module.

slice_as(src, dst)

Slice src as dst

Note

src should have the same or larger size than dst.

Parameters

• src (torch.Tensor) – Tensors to be sliced.

• dst (torch.Tensor) – src will be sliced to have the same size as dst.

Returns

Sliced tensor.

Return type

torch.Tensor

tensor_add(a, b)

Add tensors a and b that might have different sizes.

class mmdet.models.necks.HRFPN(High Resolution Feature Pyramids)

paper: High-Resolution Representations for Labeling Pixels and Regions.

Parameters

• in_channels (list) – number of channels for each branch.

• out_channels (int) – output channels of feature pyramids.

• num_outs (int) – number of output stages.

• pooling_type (str) – pooling for generating feature pyramids from {MAX, AVG}.

• conv_cfg (dict) – dictionary to construct and config conv layer.

• norm_cfg (dict) – dictionary to construct and config norm layer.

• with_cp (bool) – Use checkpoint or not. Using checkpoint will save some memory while slowing down the training speed.

• stride (int) – stride of 3x3 convolutional layers

• init_cfg (dict or list[dict], optional) – Initialization config dict.

forward(inputs)

Forward function.

class mmdet.models.necks.NASFCOS_FPN(in_channels, out_channels, num_outs, start_level=1, end_level=-1, add_extra_convs=False, conv_cfg=None, norm_cfg=None, init_cfg=None)

FPN structure in NASFPN.

Implementation of paper NAS-FCOS: Fast Neural Architecture Search for Object Detection

Parameters

• in_channels (List[int]) – Number of input channels per scale.

• out_channels (int) – Number of output channels (used at each scale)

• num_outs (int) – Number of output scales.

• start_level (int) – Index of the start input backbone level used to build the feature pyramid. Default: 0.

• end_level (int) – Index of the end input backbone level (exclusive) to build the feature pyramid. Default: -1, which means the last level.

• add_extra_convs (bool) – It decides whether to add conv layers on top of the original feature maps. Default to False. If True, its actual mode is specified by extra_convs_on_inputs.

• conv_cfg (dict) – dictionary to construct and config conv layer.

• norm_cfg (dict) – dictionary to construct and config norm layer.

• init_cfg (dict or list[dict], optional) – Initialization config dict. Default: None

forward(inputs)

Forward function.

init_weights()

Initialize the weights of module.

class mmdet.models.necks.NASFPN(in_channels, out_channels, num_outs, stack_times, start_level=0, end_level=-1, add_extra_convs=False, norm_cfg=None, init_cfg={'layer': 'Conv2d', 'type': 'Caffe2Xavier'})

NAS-FPN.

Implementation of NAS-FPN: Learning Scalable Feature Pyramid Architecture for Object Detection

Parameters

• in_channels (List[int]) – Number of input channels per scale.

• out_channels (int) – Number of output channels (used at each scale)

• num_outs (int) – Number of output scales.

• stack_times (int) – The number of times the pyramid architecture will be stacked.

• start_level (int) – Index of the start input backbone level used to build the feature pyramid. Default: 0.

• end_level (int) – Index of the end input backbone level (exclusive) to build the feature pyramid. Default: -1, which means the last level.

• add_extra_convs (bool) – It decides whether to add conv layers on top of the original feature maps. Default to False. If True, its actual mode is specified by extra_convs_on_inputs.

• init_cfg (dict or list[dict], optional) – Initialization config dict.

forward(inputs)

Forward function.

class mmdet.models.necks.PAFPN(in_channels, out_channels, num_outs, start_level=0, end_level=-1, add_extra_convs=False, relu_before_extra_convs=False, no_norm_on_lateral=False, conv_cfg=None, norm_cfg=None, act_cfg=None, init_cfg={'distribution': 'uniform', 'layer': 'Conv2d', 'type': 'Xavier'})

Path Aggregation Network for Instance Segmentation.

This is an implementation of the PAFPN in Path Aggregation Network.

Parameters

• in_channels (List[int]) – Number of input channels per scale.

• out_channels (int) – Number of output channels (used at each scale)

• num_outs (int) – Number of output scales.

• start_level (int) – Index of the start input backbone level used to build the feature pyramid. Default: 0.

• end_level (int) – Index of the end input backbone level (exclusive) to build the feature pyramid. Default: -1, which means the last level.

• add_extra_convs (bool | str) –

  If bool, it decides whether to add conv layers on top of the original feature maps. Default to False. If True, it is equivalent to add_extra_convs=’on_input’. If str, it specifies the source feature map of the extra convs. Only the following options are allowed

  • ’on_input’: Last feat map of neck inputs (i.e. backbone feature).

  • ’on_lateral’: Last feature map after lateral convs.

  • ’on_output’: The last output feature map after fpn convs.

• relu_before_extra_convs (bool) – Whether to apply relu before the extra conv. Default: False.

• no_norm_on_lateral (bool) – Whether to apply norm on lateral. Default: False.

• conv_cfg (dict) – Config dict for convolution layer. Default: None.

• norm_cfg (dict) – Config dict for normalization layer. Default: None.

• act_cfg (str) – Config dict for activation layer in ConvModule. Default: None.

• init_cfg (dict or list[dict], optional) – Initialization config dict.

forward(inputs)

Forward function.

class mmdet.models.necks.RFP(Recursive Feature Pyramid)

This is an implementation of RFP in DetectoRS. Different from standard FPN, the input of RFP should be multi level features along with origin input image of backbone.

Parameters

• rfp_steps (int) – Number of unrolled steps of RFP.

• rfp_backbone (dict) – Configuration of the backbone for RFP.

• aspp_out_channels (int) – Number of output channels of ASPP module.

• aspp_dilations (tuple[int]) – Dilation rates of four branches. Default: (1, 3, 6, 1)

• init_cfg (dict or list[dict], optional) – Initialization config dict. Default: None

forward(inputs)

Forward function.

init_weights()

Initialize the weights.

class mmdet.models.necks.SSDNeck(in_channels, out_channels, level_strides, level_paddings, l2_norm_scale=20.0, last_kernel_size=3, use_depthwise=False, conv_cfg=None, norm_cfg=None, act_cfg={'type': 'ReLU'}, init_cfg=[{'type': 'Xavier', 'distribution': 'uniform', 'layer': 'Conv2d'}, {'type': 'Constant', 'val': 1, 'layer': 'BatchNorm2d'}])

Extra layers of SSD backbone to generate multi-scale feature maps.

Parameters

• in_channels (Sequence[int]) – Number of input channels per scale.

• out_channels (Sequence[int]) – Number of output channels per scale.

• level_strides (Sequence[int]) – Stride of 3x3 conv per level.

• level_paddings (Sequence[int]) – Padding size of 3x3 conv per level.

• l2_norm_scale (float|None) – L2 normalization layer init scale. If None, not use L2 normalization on the first input feature.

• last_kernel_size (int) – Kernel size of the last conv layer. Default: 3.

• use_depthwise (bool) – Whether to use DepthwiseSeparableConv. Default: False.

• conv_cfg (dict) – Config dict for convolution layer. Default: None.

• norm_cfg (dict) – Dictionary to construct and config norm layer. Default: None.

• act_cfg (dict) – Config dict for activation layer. Default: dict(type=’ReLU’).

• init_cfg (dict or list[dict], optional) – Initialization config dict.

forward(inputs)

Forward function.

class mmdet.models.necks.YOLOV3Neck(num_scales, in_channels, out_channels, conv_cfg=None, norm_cfg={'requires_grad': True, 'type': 'BN'}, act_cfg={'negative_slope': 0.1, 'type': 'LeakyReLU'}, init_cfg=None)

The neck of YOLOV3.

It can be treated as a simplified version of FPN. It will take the result from Darknet backbone and do some upsampling and concatenation. It will finally output the detection result.

Note

The input feats should be from top to bottom.

i.e., from high-lvl to low-lvl

But YOLOV3Neck will process them in reversed order.

i.e., from bottom (high-lvl) to top (low-lvl)

Parameters

• num_scales (int) – The number of scales / stages.

• in_channels (List[int]) – The number of input channels per scale.

• out_channels (List[int]) – The number of output channels per scale.

• conv_cfg (dict, optional) – Config dict for convolution layer. Default: None.

• norm_cfg (dict, optional) – Dictionary to construct and config norm layer. Default: dict(type=’BN’, requires_grad=True)

• act_cfg (dict, optional) – Config dict for activation layer. Default: dict(type=’LeakyReLU’, negative_slope=0.1).

• init_cfg (dict or list[dict], optional) – Initialization config dict. Default: None

forward(feats)

Defines the computation performed at every call.

Should be overridden by all subclasses.

Note

Although the recipe for forward pass needs to be defined within this function, one should call the Module instance afterwards instead of this since the former takes care of running the registered hooks while the latter silently ignores them.

class mmdet.models.necks.YOLOXPAFPN(in_channels, out_channels, num_csp_blocks=3, use_depthwise=False, upsample_cfg={'mode': 'nearest', 'scale_factor': 2}, conv_cfg=None, norm_cfg={'eps': 0.001, 'momentum': 0.03, 'type': 'BN'}, act_cfg={'type': 'Swish'}, init_cfg={'a': 2.23606797749979, 'distribution': 'uniform', 'layer': 'Conv2d', 'mode': 'fan_in', 'nonlinearity': 'leaky_relu', 'type': 'Kaiming'})

Path Aggregation Network used in YOLOX.

Parameters

• in_channels (List[int]) – Number of input channels per scale.

• out_channels (int) – Number of output channels (used at each scale)

• num_csp_blocks (int) – Number of bottlenecks in CSPLayer. Default: 3

• use_depthwise (bool) – Whether to depthwise separable convolution in blocks. Default: False

• upsample_cfg (dict) – Config dict for interpolate layer. Default: dict(scale_factor=2, mode=’nearest’)

• conv_cfg (dict, optional) – Config dict for convolution layer. Default: None, which means using conv2d.

• norm_cfg (dict) – Config dict for normalization layer. Default: dict(type=’BN’)

• act_cfg (dict) – Config dict for activation layer. Default: dict(type=’Swish’)

• init_cfg (dict or list[dict], optional) – Initialization config dict. Default: None.

forward(inputs)

Parameters

inputs (tuple[Tensor]) – input features.

Returns

YOLOXPAFPN features.

Return type

tuple[Tensor]

dense_heads

roi_heads

losses

utils

class mmdet.models.utils.AdaptiveAvgPool2d(output_size: Union[int, None, Tuple[Optional[int], ...]])

Handle empty batch dimension to AdaptiveAvgPool2d.

forward(x)

Defines the computation performed at every call.

Should be overridden by all subclasses.

Note

Although the recipe for forward pass needs to be defined within this function, one should call the Module instance afterwards instead of this since the former takes care of running the registered hooks while the latter silently ignores them.

class mmdet.models.utils.CSPLayer(in_channels, out_channels, expand_ratio=0.5, num_blocks=1, add_identity=True, use_depthwise=False, conv_cfg=None, norm_cfg={'eps': 0.001, 'momentum': 0.03, 'type': 'BN'}, act_cfg={'type': 'Swish'}, init_cfg=None)

Cross Stage Partial Layer.

Parameters

• in_channels (int) – The input channels of the CSP layer.

• out_channels (int) – The output channels of the CSP layer.

• expand_ratio (float) – Ratio to adjust the number of channels of the hidden layer. Default: 0.5

• num_blocks (int) – Number of blocks. Default: 1

• add_identity (bool) – Whether to add identity in blocks. Default: True

• use_depthwise (bool) – Whether to depthwise separable convolution in blocks. Default: False

• conv_cfg (dict, optional) – Config dict for convolution layer. Default: None, which means using conv2d.

• norm_cfg (dict) – Config dict for normalization layer. Default: dict(type=’BN’)

• act_cfg (dict) – Config dict for activation layer. Default: dict(type=’Swish’)

forward(x)

Defines the computation performed at every call.

Should be overridden by all subclasses.

Note

Although the recipe for forward pass needs to be defined within this function, one should call the Module instance afterwards instead of this since the former takes care of running the registered hooks while the latter silently ignores them.

class mmdet.models.utils.ConvUpsample(in_channels, inner_channels, num_layers=1, num_upsample=None, conv_cfg=None, norm_cfg=None, init_cfg=None, **kwargs)

ConvUpsample performs 2x upsampling after Conv.

There are several ConvModule layers. In the first few layers, upsampling will be applied after each layer of convolution. The number of upsampling must be no more than the number of ConvModule layers.

Parameters

• in_channels (int) – Number of channels in the input feature map.

• inner_channels (int) – Number of channels produced by the convolution.

• num_layers (int) – Number of convolution layers.

• num_upsample (int | optional) – Number of upsampling layer. Must be no more than num_layers. Upsampling will be applied after the first num_upsample layers of convolution. Default: num_layers.

• conv_cfg (dict) – Config dict for convolution layer. Default: None, which means using conv2d.

• norm_cfg (dict) – Config dict for normalization layer. Default: None.

• init_cfg (dict) – Config dict for initialization. Default: None.

• kwargs (key word augments) – Other augments used in ConvModule.

forward(x)

Defines the computation performed at every call.

Should be overridden by all subclasses.

Note

Although the recipe for forward pass needs to be defined within this function, one should call the Module instance afterwards instead of this since the former takes care of running the registered hooks while the latter silently ignores them.

class mmdet.models.utils.DetrTransformerDecoder(*args, post_norm_cfg={'type': 'LN'}, return_intermediate=False, **kwargs)

Implements the decoder in DETR transformer.

Parameters

• return_intermediate (bool) – Whether to return intermediate outputs.

• post_norm_cfg (dict) – Config of last normalization layer. Default： LN.

forward(query, *args, **kwargs)

Forward function for TransformerDecoder.

Parameters

query (Tensor) – Input query with shape (num_query, bs, embed_dims).

Returns

Results with shape [1, num_query, bs, embed_dims] when

return_intermediate is False, otherwise it has shape [num_layers, num_query, bs, embed_dims].

Return type

Tensor

class mmdet.models.utils.DetrTransformerDecoderLayer(attn_cfgs, feedforward_channels, ffn_dropout=0.0, operation_order=None, act_cfg={'inplace': True, 'type': 'ReLU'}, norm_cfg={'type': 'LN'}, ffn_num_fcs=2, **kwargs)

Implements decoder layer in DETR transformer.

Parameters

• attn_cfgs (list[mmcv.ConfigDict] | list[dict] | dict )) – Configs for self_attention or cross_attention, the order should be consistent with it in operation_order. If it is a dict, it would be expand to the number of attention in operation_order.

• feedforward_channels (int) – The hidden dimension for FFNs.

• ffn_dropout (float) – Probability of an element to be zeroed in ffn. Default 0.0.

• operation_order (tuple[str]) – The execution order of operation in transformer. Such as (‘self_attn’, ‘norm’, ‘ffn’, ‘norm’). Default：None

• act_cfg (dict) – The activation config for FFNs. Default: LN

• norm_cfg (dict) – Config dict for normalization layer. Default: LN.

• ffn_num_fcs (int) – The number of fully-connected layers in FFNs. Default：2.

class mmdet.models.utils.DyReLU(channels, ratio=4, conv_cfg=None, act_cfg=({'type': 'ReLU'}, {'type': 'HSigmoid', 'bias': 3.0, 'divisor': 6.0}), init_cfg=None)

Dynamic ReLU (DyReLU) module.

See Dynamic ReLU for details. Current implementation is specialized for task-aware attention in DyHead. HSigmoid arguments in default act_cfg follow DyHead official code. https://github.com/microsoft/DynamicHead/blob/master/dyhead/dyrelu.py

Parameters

• channels (int) – The input (and output) channels of DyReLU module.

• ratio (int) – Squeeze ratio in Squeeze-and-Excitation-like module, the intermediate channel will be int(channels/ratio). Default: 4.

• conv_cfg (None or dict) – Config dict for convolution layer. Default: None, which means using conv2d.

• act_cfg (dict or Sequence[dict]) – Config dict for activation layer. If act_cfg is a dict, two activation layers will be configurated by this dict. If act_cfg is a sequence of dicts, the first activation layer will be configurated by the first dict and the second activation layer will be configurated by the second dict. Default: (dict(type=’ReLU’), dict(type=’HSigmoid’, bias=3.0, divisor=6.0))

• init_cfg (dict or list[dict], optional) – Initialization config dict. Default: None

forward(x)

Forward function.

class mmdet.models.utils.DynamicConv(in_channels=256, feat_channels=64, out_channels=None, input_feat_shape=7, with_proj=True, act_cfg={'inplace': True, 'type': 'ReLU'}, norm_cfg={'type': 'LN'}, init_cfg=None)

Implements Dynamic Convolution.

This module generate parameters for each sample and use bmm to implement 1*1 convolution. Code is modified from the official github repo .

Parameters

• in_channels (int) – The input feature channel. Defaults to 256.

• feat_channels (int) – The inner feature channel. Defaults to 64.

• out_channels (int, optional) – The output feature channel. When not specified, it will be set to in_channels by default

• input_feat_shape (int) – The shape of input feature. Defaults to 7.

• with_proj (bool) – Project two-dimentional feature to one-dimentional feature. Default to True.

• act_cfg (dict) – The activation config for DynamicConv.

• norm_cfg (dict) – Config dict for normalization layer. Default layer normalization.

• (obj (init_cfg) – mmcv.ConfigDict): The Config for initialization. Default: None.

forward(param_feature, input_feature)

Forward function for DynamicConv.

Parameters

• param_feature (Tensor) – The feature can be used to generate the parameter, has shape (num_all_proposals, in_channels).

• input_feature (Tensor) – Feature that interact with parameters, has shape (num_all_proposals, in_channels, H, W).

Returns

The output feature has shape (num_all_proposals, out_channels).

Return type

Tensor

class mmdet.models.utils.InvertedResidual(in_channels, out_channels, mid_channels, kernel_size=3, stride=1, se_cfg=None, with_expand_conv=True, conv_cfg=None, norm_cfg={'type': 'BN'}, act_cfg={'type': 'ReLU'}, drop_path_rate=0.0, with_cp=False, init_cfg=None)

Inverted Residual Block.

Parameters

• in_channels (int) – The input channels of this Module.

• out_channels (int) – The output channels of this Module.

• mid_channels (int) – The input channels of the depthwise convolution.

• kernel_size (int) – The kernel size of the depthwise convolution. Default: 3.

• stride (int) – The stride of the depthwise convolution. Default: 1.

• se_cfg (dict) – Config dict for se layer. Default: None, which means no se layer.

• with_expand_conv (bool) – Use expand conv or not. If set False, mid_channels must be the same with in_channels. Default: True.

• conv_cfg (dict) – Config dict for convolution layer. Default: None, which means using conv2d.

• norm_cfg (dict) – Config dict for normalization layer. Default: dict(type=’BN’).

• act_cfg (dict) – Config dict for activation layer. Default: dict(type=’ReLU’).

• drop_path_rate (float) – stochastic depth rate. Defaults to 0.

• with_cp (bool) – Use checkpoint or not. Using checkpoint will save some memory while slowing down the training speed. Default: False.

• init_cfg (dict or list[dict], optional) – Initialization config dict. Default: None

Returns

The output tensor.

Return type

Tensor

forward(x)

Defines the computation performed at every call.

Should be overridden by all subclasses.

Note

Although the recipe for forward pass needs to be defined within this function, one should call the Module instance afterwards instead of this since the former takes care of running the registered hooks while the latter silently ignores them.

class mmdet.models.utils.LearnedPositionalEncoding(num_feats, row_num_embed=50, col_num_embed=50, init_cfg={'layer': 'Embedding', 'type': 'Uniform'})

Position embedding with learnable embedding weights.

Parameters

• num_feats (int) – The feature dimension for each position along x-axis or y-axis. The final returned dimension for each position is 2 times of this value.

• row_num_embed (int, optional) – The dictionary size of row embeddings. Default 50.

• col_num_embed (int, optional) – The dictionary size of col embeddings. Default 50.

• init_cfg (dict or list[dict], optional) – Initialization config dict.

forward(mask)

Forward function for LearnedPositionalEncoding.

Parameters

mask (Tensor) – ByteTensor mask. Non-zero values representing ignored positions, while zero values means valid positions for this image. Shape [bs, h, w].

Returns

Returned position embedding with shape

[bs, num_feats*2, h, w].

Return type

pos (Tensor)

class mmdet.models.utils.NormedConv2d(*args, tempearture=20, power=1.0, eps=1e-06, norm_over_kernel=False, **kwargs)

Normalized Conv2d Layer.

Parameters

• tempeature (float, optional) – Tempeature term. Default to 20.

• power (int, optional) – Power term. Default to 1.0.

• eps (float, optional) – The minimal value of divisor to keep numerical stability. Default to 1e-6.

• norm_over_kernel (bool, optional) – Normalize over kernel. Default to False.

forward(x)

Defines the computation performed at every call.

Should be overridden by all subclasses.

Note

Although the recipe for forward pass needs to be defined within this function, one should call the Module instance afterwards instead of this since the former takes care of running the registered hooks while the latter silently ignores them.

class mmdet.models.utils.NormedLinear(*args, tempearture=20, power=1.0, eps=1e-06, **kwargs)

Normalized Linear Layer.

Parameters

• tempeature (float, optional) – Tempeature term. Default to 20.

• power (int, optional) – Power term. Default to 1.0.

• eps (float, optional) – The minimal value of divisor to keep numerical stability. Default to 1e-6.

forward(x)

Defines the computation performed at every call.

Should be overridden by all subclasses.

Note

Although the recipe for forward pass needs to be defined within this function, one should call the Module instance afterwards instead of this since the former takes care of running the registered hooks while the latter silently ignores them.

class mmdet.models.utils.PatchEmbed(in_channels=3, embed_dims=768, conv_type='Conv2d', kernel_size=16, stride=16, padding='corner', dilation=1, bias=True, norm_cfg=None, input_size=None, init_cfg=None)

Image to Patch Embedding.

We use a conv layer to implement PatchEmbed.

Parameters

• in_channels (int) – The num of input channels. Default: 3

• embed_dims (int) – The dimensions of embedding. Default: 768

• conv_type (str) – The config dict for embedding conv layer type selection. Default: “Conv2d.

• kernel_size (int) – The kernel_size of embedding conv. Default: 16.

• stride (int) – The slide stride of embedding conv. Default: None (Would be set as kernel_size).

• padding (int | tuple | string) – The padding length of embedding conv. When it is a string, it means the mode of adaptive padding, support “same” and “corner” now. Default: “corner”.

• dilation (int) – The dilation rate of embedding conv. Default: 1.

• bias (bool) – Bias of embed conv. Default: True.

• norm_cfg (dict, optional) – Config dict for normalization layer. Default: None.

• input_size (int | tuple | None) – The size of input, which will be used to calculate the out size. Only work when dynamic_size is False. Default: None.

• init_cfg (mmcv.ConfigDict, optional) – The Config for initialization. Default: None.

forward(x)

Parameters

x (Tensor) – Has shape (B, C, H, W). In most case, C is 3.

Returns

Contains merged results and its spatial shape.

• x (Tensor): Has shape (B, out_h * out_w, embed_dims)

• out_size (tuple[int]): Spatial shape of x, arrange as

  (out_h, out_w).

Return type

tuple

class mmdet.models.utils.ResLayer(block, inplanes, planes, num_blocks, stride=1, avg_down=False, conv_cfg=None, norm_cfg={'type': 'BN'}, downsample_first=True, **kwargs)

ResLayer to build ResNet style backbone.

Parameters

• block (nn.Module) – block used to build ResLayer.

• inplanes (int) – inplanes of block.

• planes (int) – planes of block.

• num_blocks (int) – number of blocks.

• stride (int) – stride of the first block. Default: 1

• avg_down (bool) – Use AvgPool instead of stride conv when downsampling in the bottleneck. Default: False

• conv_cfg (dict) – dictionary to construct and config conv layer. Default: None

• norm_cfg (dict) – dictionary to construct and config norm layer. Default: dict(type=’BN’)

• downsample_first (bool) – Downsample at the first block or last block. False for Hourglass, True for ResNet. Default: True

class mmdet.models.utils.SELayer(channels, ratio=16, conv_cfg=None, act_cfg=({'type': 'ReLU'}, {'type': 'Sigmoid'}), init_cfg=None)

Squeeze-and-Excitation Module.

Parameters

• channels (int) – The input (and output) channels of the SE layer.

• ratio (int) – Squeeze ratio in SELayer, the intermediate channel will be int(channels/ratio). Default: 16.

• conv_cfg (None or dict) – Config dict for convolution layer. Default: None, which means using conv2d.

• act_cfg (dict or Sequence[dict]) – Config dict for activation layer. If act_cfg is a dict, two activation layers will be configurated by this dict. If act_cfg is a sequence of dicts, the first activation layer will be configurated by the first dict and the second activation layer will be configurated by the second dict. Default: (dict(type=’ReLU’), dict(type=’Sigmoid’))

• init_cfg (dict or list[dict], optional) – Initialization config dict. Default: None

forward(x)

Defines the computation performed at every call.

Should be overridden by all subclasses.

Note

Although the recipe for forward pass needs to be defined within this function, one should call the Module instance afterwards instead of this since the former takes care of running the registered hooks while the latter silently ignores them.

class mmdet.models.utils.SimplifiedBasicBlock(inplanes, planes, stride=1, dilation=1, downsample=None, style='pytorch', with_cp=False, conv_cfg=None, norm_cfg={'type': 'BN'}, dcn=None, plugins=None, init_fg=None)

Simplified version of original basic residual block. This is used in SCNet.

• Norm layer is now optional

• Last ReLU in forward function is removed

forward(x)

Forward function.

property norm1

normalization layer after the first convolution layer

Type

nn.Module

property norm2

normalization layer after the second convolution layer

Type

nn.Module

class mmdet.models.utils.SinePositionalEncoding(num_feats, temperature=10000, normalize=False, scale=6.283185307179586, eps=1e-06, offset=0.0, init_cfg=None)

Position encoding with sine and cosine functions.

See End-to-End Object Detection with Transformers for details.

Parameters

• num_feats (int) – The feature dimension for each position along x-axis or y-axis. Note the final returned dimension for each position is 2 times of this value.

• temperature (int, optional) – The temperature used for scaling the position embedding. Defaults to 10000.

• normalize (bool, optional) – Whether to normalize the position embedding. Defaults to False.

• scale (float, optional) – A scale factor that scales the position embedding. The scale will be used only when normalize is True. Defaults to 2*pi.

• eps (float, optional) – A value added to the denominator for numerical stability. Defaults to 1e-6.

• offset (float) – offset add to embed when do the normalization. Defaults to 0.

• init_cfg (dict or list[dict], optional) – Initialization config dict. Default: None

forward(mask)

Forward function for SinePositionalEncoding.

Parameters

mask (Tensor) – ByteTensor mask. Non-zero values representing ignored positions, while zero values means valid positions for this image. Shape [bs, h, w].

Returns

Returned position embedding with shape

[bs, num_feats*2, h, w].

Return type

pos (Tensor)

class mmdet.models.utils.Transformer(encoder=None, decoder=None, init_cfg=None)

Implements the DETR transformer.

Following the official DETR implementation, this module copy-paste from torch.nn.Transformer with modifications:

• positional encodings are passed in MultiheadAttention

• extra LN at the end of encoder is removed

• decoder returns a stack of activations from all decoding layers

See paper: End-to-End Object Detection with Transformers for details.

Parameters

• encoder (mmcv.ConfigDict | Dict) – Config of TransformerEncoder. Defaults to None.

• decoder ((mmcv.ConfigDict | Dict)) – Config of TransformerDecoder. Defaults to None

• (obj (init_cfg) – mmcv.ConfigDict): The Config for initialization. Defaults to None.

forward(x, mask, query_embed, pos_embed)

Forward function for Transformer.

Parameters

• x (Tensor) – Input query with shape [bs, c, h, w] where c = embed_dims.

• mask (Tensor) – The key_padding_mask used for encoder and decoder, with shape [bs, h, w].

• query_embed (Tensor) – The query embedding for decoder, with shape [num_query, c].

• pos_embed (Tensor) – The positional encoding for encoder and decoder, with the same shape as x.

Returns

results of decoder containing the following tensor.

• out_dec: Output from decoder. If return_intermediate_dec is True output has shape [num_dec_layers, bs,

  num_query, embed_dims], else has shape [1, bs, num_query, embed_dims].

• memory: Output results from encoder, with shape [bs, embed_dims, h, w].

Return type

tuple[Tensor]

init_weights()

Initialize the weights.

mmdet.models.utils.adaptive_avg_pool2d(input, output_size)

Handle empty batch dimension to adaptive_avg_pool2d.

Parameters

• input (tensor) – 4D tensor.

• output_size (int, tuple[int,int]) – the target output size.

mmdet.models.utils.build_linear_layer(cfg, *args, **kwargs)

Build linear layer. :param cfg: The linear layer config, which should contain:

• type (str): Layer type.

• layer args: Args needed to instantiate an linear layer.

Parameters

• args (argument list) – Arguments passed to the __init__ method of the corresponding linear layer.

• kwargs (keyword arguments) – Keyword arguments passed to the __init__ method of the corresponding linear layer.

Returns

Created linear layer.

Return type

nn.Module

mmdet.models.utils.build_transformer(cfg, default_args=None)

Builder for Transformer.

mmdet.models.utils.gaussian_radius(det_size, min_overlap)

Generate 2D gaussian radius.

This function is modified from the official github repo.

Given min_overlap, radius could computed by a quadratic equation according to Vieta’s formulas.

There are 3 cases for computing gaussian radius, details are following:

• Explanation of figure: lt and br indicates the left-top and bottom-right corner of ground truth box. x indicates the generated corner at the limited position when radius=r.

• Case1: one corner is inside the gt box and the other is outside.

|<   width   >|

lt-+----------+         -
|  |          |         ^
+--x----------+--+
|  |          |  |
|  |          |  |    height
|  | overlap  |  |
|  |          |  |
|  |          |  |      v
+--+---------br--+      -
   |          |  |
   +----------+--x

To ensure IoU of generated box and gt box is larger than min_overlap:

\[\begin{split}\cfrac{(w-r)*(h-r)}{w*h+(w+h)r-r^2} \ge {iou} \quad\Rightarrow\quad {r^2-(w+h)r+\cfrac{1-iou}{1+iou}*w*h} \ge 0 \\ {a} = 1,\quad{b} = {-(w+h)},\quad{c} = {\cfrac{1-iou}{1+iou}*w*h} \\ {r} \le \cfrac{-b-\sqrt{b^2-4*a*c}}{2*a}\end{split}\]

• Case2: both two corners are inside the gt box.

|<   width   >|

lt-+----------+         -
|  |          |         ^
+--x-------+  |
|  |       |  |
|  |overlap|  |       height
|  |       |  |
|  +-------x--+
|          |  |         v
+----------+-br         -

To ensure IoU of generated box and gt box is larger than min_overlap:

\[\begin{split}\cfrac{(w-2*r)*(h-2*r)}{w*h} \ge {iou} \quad\Rightarrow\quad {4r^2-2(w+h)r+(1-iou)*w*h} \ge 0 \\ {a} = 4,\quad {b} = {-2(w+h)},\quad {c} = {(1-iou)*w*h} \\ {r} \le \cfrac{-b-\sqrt{b^2-4*a*c}}{2*a}\end{split}\]

• Case3: both two corners are outside the gt box.

   |<   width   >|

x--+----------------+
|  |                |
+-lt-------------+  |   -
|  |             |  |   ^
|  |             |  |
|  |   overlap   |  | height
|  |             |  |
|  |             |  |   v
|  +------------br--+   -
|                |  |
+----------------+--x

To ensure IoU of generated box and gt box is larger than min_overlap:

\[\begin{split}\cfrac{w*h}{(w+2*r)*(h+2*r)} \ge {iou} \quad\Rightarrow\quad {4*iou*r^2+2*iou*(w+h)r+(iou-1)*w*h} \le 0 \\ {a} = {4*iou},\quad {b} = {2*iou*(w+h)},\quad {c} = {(iou-1)*w*h} \\ {r} \le \cfrac{-b+\sqrt{b^2-4*a*c}}{2*a}\end{split}\]

Parameters

• det_size (list[int]) – Shape of object.

• min_overlap (float) – Min IoU with ground truth for boxes generated by keypoints inside the gaussian kernel.

Returns

Radius of gaussian kernel.

Return type

radius (int)

mmdet.models.utils.gen_gaussian_target(heatmap, center, radius, k=1)

Generate 2D gaussian heatmap.

Parameters

• heatmap (Tensor) – Input heatmap, the gaussian kernel will cover on it and maintain the max value.

• center (list[int]) – Coord of gaussian kernel’s center.

• radius (int) – Radius of gaussian kernel.

• k (int) – Coefficient of gaussian kernel. Default: 1.

Returns

Updated heatmap covered by gaussian kernel.

Return type

out_heatmap (Tensor)

mmdet.models.utils.get_uncertain_point_coords_with_randomness(mask_pred, labels, num_points, oversample_ratio, importance_sample_ratio)

Get num_points most uncertain points with random points during train.

Sample points in [0, 1] x [0, 1] coordinate space based on their uncertainty. The uncertainties are calculated for each point using ‘get_uncertainty()’ function that takes point’s logit prediction as input.

Parameters

• mask_pred (Tensor) – A tensor of shape (num_rois, num_classes, mask_height, mask_width) for class-specific or class-agnostic prediction.

• labels (list) – The ground truth class for each instance.

• num_points (int) – The number of points to sample.

• oversample_ratio (int) – Oversampling parameter.

• importance_sample_ratio (float) – Ratio of points that are sampled via importnace sampling.

Returns

A tensor of shape (num_rois, num_points, 2)

that contains the coordinates sampled points.

Return type

point_coords (Tensor)

mmdet.models.utils.get_uncertainty(mask_pred, labels)

Estimate uncertainty based on pred logits.

We estimate uncertainty as L1 distance between 0.0 and the logits prediction in ‘mask_pred’ for the foreground class in classes.

Parameters

• mask_pred (Tensor) – mask predication logits, shape (num_rois, num_classes, mask_height, mask_width).

• labels (list[Tensor]) – Either predicted or ground truth label for each predicted mask, of length num_rois.

Returns

Uncertainty scores with the most uncertain

locations having the highest uncertainty score, shape (num_rois, 1, mask_height, mask_width)

Return type

scores (Tensor)

mmdet.models.utils.interpolate_as(source, target, mode='bilinear', align_corners=False)

Interpolate the source to the shape of the target.

The source must be a Tensor, but the target can be a Tensor or a np.ndarray with the shape (…, target_h, target_w).

Parameters

• source (Tensor) – A 3D/4D Tensor with the shape (N, H, W) or (N, C, H, W).

• target (Tensor | np.ndarray) – The interpolation target with the shape (…, target_h, target_w).

• mode (str) – Algorithm used for interpolation. The options are the same as those in F.interpolate(). Default: 'bilinear'.

• align_corners (bool) – The same as the argument in F.interpolate().

Returns

The interpolated source Tensor.

Return type

Tensor

mmdet.models.utils.make_divisible(value, divisor, min_value=None, min_ratio=0.9)

Make divisible function.

This function rounds the channel number to the nearest value that can be divisible by the divisor. It is taken from the original tf repo. It ensures that all layers have a channel number that is divisible by divisor. It can be seen here: https://github.com/tensorflow/models/blob/master/research/slim/nets/mobilenet/mobilenet.py # noqa

Parameters

• value (int) – The original channel number.

• divisor (int) – The divisor to fully divide the channel number.

• min_value (int) – The minimum value of the output channel. Default: None, means that the minimum value equal to the divisor.

• min_ratio (float) – The minimum ratio of the rounded channel number to the original channel number. Default: 0.9.

Returns

The modified output channel number.

Return type

int

mmdet.models.utils.nchw_to_nlc(x)

Flatten [N, C, H, W] shape tensor to [N, L, C] shape tensor.

Parameters

x (Tensor) – The input tensor of shape [N, C, H, W] before conversion.

Returns

The output tensor of shape [N, L, C] after conversion.

Return type

Tensor

mmdet.models.utils.nlc_to_nchw(x, hw_shape)

Convert [N, L, C] shape tensor to [N, C, H, W] shape tensor.

Parameters

• x (Tensor) – The input tensor of shape [N, L, C] before conversion.

• hw_shape (Sequence[int]) – The height and width of output feature map.

Returns

The output tensor of shape [N, C, H, W] after conversion.

Return type

Tensor

mmdet.models.utils.preprocess_panoptic_gt(gt_labels, gt_masks, gt_semantic_seg, num_things, num_stuff, img_metas)

Preprocess the ground truth for a image.

Parameters

• gt_labels (Tensor) – Ground truth labels of each bbox, with shape (num_gts, ).

• gt_masks (BitmapMasks) – Ground truth masks of each instances of a image, shape (num_gts, h, w).

• gt_semantic_seg (Tensor | None) – Ground truth of semantic segmentation with the shape (1, h, w). [0, num_thing_class - 1] means things, [num_thing_class, num_class-1] means stuff, 255 means VOID. It’s None when training instance segmentation.

• img_metas (dict) – List of image meta information.

Returns

a tuple containing the following targets.

• labels (Tensor): Ground truth class indices for a

  image, with shape (n, ), n is the sum of number of stuff type and number of instance in a image.

• masks (Tensor): Ground truth mask for a image, with

  shape (n, h, w). Contains stuff and things when training panoptic segmentation, and things only when training instance segmentation.

Return type

tuple

mmdet.utils