API Reference¶
mmdet.apis¶
-
mmdet.apis.
get_root_logger
(log_file=None, log_level=20)[source]¶ Get root logger.
Parameters: - log_file (str, optional) – File path of log. Defaults to None.
- log_level (int, optional) – The level of logger. Defaults to logging.INFO.
Returns: The obtained logger
Return type: logging.Logger
-
mmdet.apis.
set_random_seed
(seed, deterministic=False)[source]¶ Set random seed.
Parameters: - seed (int) – Seed to be used.
- deterministic (bool) – Whether to set the deterministic option for CUDNN backend, i.e., set torch.backends.cudnn.deterministic to True and torch.backends.cudnn.benchmark to False. Default: False.
-
mmdet.apis.
init_detector
(config, checkpoint=None, device='cuda:0', cfg_options=None)[source]¶ Initialize a detector from config file.
Parameters: - config (str or
mmcv.Config
) – Config file path or the config object. - checkpoint (str, optional) – Checkpoint path. If left as None, the model will not load any weights.
- cfg_options (dict) – Options to override some settings in the used config.
Returns: The constructed detector.
Return type: nn.Module
- config (str or
-
mmdet.apis.
async_inference_detector
(model, imgs)[source]¶ Async inference image(s) with the detector.
Parameters: - model (nn.Module) – The loaded detector.
- img (str | ndarray) – Either image files or loaded images.
Returns: Awaitable detection results.
-
mmdet.apis.
inference_detector
(model, imgs)[source]¶ Inference image(s) with the detector.
Parameters: - model (nn.Module) – The loaded detector.
- imgs (str/ndarray or list[str/ndarray] or tuple[str/ndarray]) – Either image files or loaded images.
Returns: If imgs is a list or tuple, the same length list type results will be returned, otherwise return the detection results directly.
-
mmdet.apis.
show_result_pyplot
(model, img, result, score_thr=0.3, title='result', wait_time=0)[source]¶ Visualize the detection results on the image.
Parameters: - model (nn.Module) – The loaded detector.
- img (str or np.ndarray) – Image filename or loaded image.
- result (tuple[list] or list) – The detection result, can be either (bbox, segm) or just bbox.
- score_thr (float) – The threshold to visualize the bboxes and masks.
- title (str) – Title of the pyplot figure.
- wait_time (float) – Value of waitKey param. Default: 0.
-
mmdet.apis.
multi_gpu_test
(model, data_loader, tmpdir=None, gpu_collect=False)[source]¶ Test model with multiple gpus.
This method tests model with multiple gpus and collects the results under two different modes: gpu and cpu modes. By setting ‘gpu_collect=True’ it encodes results to gpu tensors and use gpu communication for results collection. On cpu mode it saves the results on different gpus to ‘tmpdir’ and collects them by the rank 0 worker.
Parameters: - model (nn.Module) – Model to be tested.
- data_loader (nn.Dataloader) – Pytorch data loader.
- tmpdir (str) – Path of directory to save the temporary results from different gpus under cpu mode.
- gpu_collect (bool) – Option to use either gpu or cpu to collect results.
Returns: The prediction results.
Return type: list
mmdet.core¶
anchor¶
-
class
mmdet.core.anchor.
AnchorGenerator
(strides, ratios, scales=None, base_sizes=None, scale_major=True, octave_base_scale=None, scales_per_octave=None, centers=None, center_offset=0.0)[source]¶ Standard anchor generator for 2D anchor-based detectors.
Parameters: - strides (list[int] | list[tuple[int, int]]) – Strides of anchors in multiple feature levels in order (w, h).
- ratios (list[float]) – The list of ratios between the height and width of anchors in a single level.
- scales (list[int] | None) – Anchor scales for anchors in a single level. It cannot be set at the same time if octave_base_scale and scales_per_octave are set.
- base_sizes (list[int] | None) – The basic sizes of anchors in multiple levels. If None is given, strides will be used as base_sizes. (If strides are non square, the shortest stride is taken.)
- scale_major (bool) – Whether to multiply scales first when generating base anchors. If true, the anchors in the same row will have the same scales. By default it is True in V2.0
- octave_base_scale (int) – The base scale of octave.
- scales_per_octave (int) – Number of scales for each octave. octave_base_scale and scales_per_octave are usually used in retinanet and the scales should be None when they are set.
- centers (list[tuple[float, float]] | None) – The centers of the anchor relative to the feature grid center in multiple feature levels. By default it is set to be None and not used. If a list of tuple of float is given, they will be used to shift the centers of anchors.
- center_offset (float) – The offset of center in proportion to anchors’ width and height. By default it is 0 in V2.0.
Examples
>>> from mmdet.core import AnchorGenerator >>> self = AnchorGenerator([16], [1.], [1.], [9]) >>> all_anchors = self.grid_anchors([(2, 2)], device='cpu') >>> print(all_anchors) [tensor([[-4.5000, -4.5000, 4.5000, 4.5000], [11.5000, -4.5000, 20.5000, 4.5000], [-4.5000, 11.5000, 4.5000, 20.5000], [11.5000, 11.5000, 20.5000, 20.5000]])] >>> self = AnchorGenerator([16, 32], [1.], [1.], [9, 18]) >>> all_anchors = self.grid_anchors([(2, 2), (1, 1)], device='cpu') >>> print(all_anchors) [tensor([[-4.5000, -4.5000, 4.5000, 4.5000], [11.5000, -4.5000, 20.5000, 4.5000], [-4.5000, 11.5000, 4.5000, 20.5000], [11.5000, 11.5000, 20.5000, 20.5000]]), tensor([[-9., -9., 9., 9.]])]
-
gen_base_anchors
()[source]¶ Generate base anchors.
Returns: Base anchors of a feature grid in multiple feature levels. Return type: list(torch.Tensor)
-
gen_single_level_base_anchors
(base_size, scales, ratios, center=None)[source]¶ Generate base anchors of a single level.
Parameters: - base_size (int | float) – Basic size of an anchor.
- scales (torch.Tensor) – Scales of the anchor.
- ratios (torch.Tensor) – The ratio between between the height and width of anchors in a single level.
- center (tuple[float], optional) – The center of the base anchor related to a single feature grid. Defaults to None.
Returns: Anchors in a single-level feature maps.
Return type: torch.Tensor
-
grid_anchors
(featmap_sizes, device='cuda')[source]¶ Generate grid anchors in multiple feature levels.
Parameters: - featmap_sizes (list[tuple]) – List of feature map sizes in multiple feature levels.
- device (str) – Device where the anchors will be put on.
Returns: Anchors in multiple feature levels. The sizes of each tensor should be [N, 4], where N = width * height * num_base_anchors, width and height are the sizes of the corresponding feature level, num_base_anchors is the number of anchors for that level.
Return type: list[torch.Tensor]
-
grid_priors
(featmap_sizes, device='cuda')[source]¶ Generate grid anchors in multiple feature levels.
Parameters: - featmap_sizes (list[tuple]) – List of feature map sizes in multiple feature levels.
- device (str) – The device where the anchors will be put on.
Returns: Anchors in multiple feature levels. The sizes of each tensor should be [N, 4], where N = width * height * num_base_anchors, width and height are the sizes of the corresponding feature level, num_base_anchors is the number of anchors for that level.
Return type: list[torch.Tensor]
-
num_base_anchors
¶ total number of base anchors in a feature grid
Type: list[int]
-
num_base_priors
¶ The number of priors (anchors) at a point on the feature grid
Type: list[int]
-
num_levels
¶ number of feature levels that the generator will be applied
Type: int
-
single_level_grid_anchors
(base_anchors, featmap_size, stride=(16, 16), device='cuda')[source]¶ Generate grid anchors of a single level.
Note
This function is usually called by method
self.grid_anchors
.Parameters: - base_anchors (torch.Tensor) – The base anchors of a feature grid.
- featmap_size (tuple[int]) – Size of the feature maps.
- stride (tuple[int], optional) – Stride of the feature map in order (w, h). Defaults to (16, 16).
- device (str, optional) – Device the tensor will be put on. Defaults to ‘cuda’.
Returns: Anchors in the overall feature maps.
Return type: torch.Tensor
-
single_level_grid_priors
(featmap_size, level_idx, device='cuda')[source]¶ Generate grid anchors of a single level.
Note
This function is usually called by method
self.grid_priors
.Parameters: - featmap_size (tuple[int]) – Size of the feature maps.
- level_idx (int) – The index of corresponding feature map level.
- device (str, optional) – The device the tensor will be put on. Defaults to ‘cuda’.
Returns: Anchors in the overall feature maps.
Return type: torch.Tensor
-
single_level_valid_flags
(featmap_size, valid_size, num_base_anchors, device='cuda')[source]¶ Generate the valid flags of anchor in a single feature map.
Parameters: - featmap_size (tuple[int]) – The size of feature maps, arrange as (h, w).
- valid_size (tuple[int]) – The valid size of the feature maps.
- num_base_anchors (int) – The number of base anchors.
- device (str, optional) – Device where the flags will be put on. Defaults to ‘cuda’.
Returns: The valid flags of each anchor in a single level feature map.
Return type: torch.Tensor
-
sparse_priors
(prior_idxs, featmap_size, level_idx, dtype=torch.float32, device='cuda')[source]¶ Generate sparse anchors according to the
prior_idxs
.Parameters: - prior_idxs (Tensor) – The index of corresponding anchors in the feature map.
- featmap_size (tuple[int]) – feature map size arrange as (h, w).
- level_idx (int) – The level index of corresponding feature map.
- (obj (device) – torch.dtype): Date type of points.Defaults to
torch.float32
. - (obj – torch.device): The device where the points is located.
Returns: - Anchor with shape (N, 4), N should be equal to
the length of
prior_idxs
.
Return type: Tensor
-
valid_flags
(featmap_sizes, pad_shape, device='cuda')[source]¶ Generate valid flags of anchors in multiple feature levels.
Parameters: - featmap_sizes (list(tuple)) – List of feature map sizes in multiple feature levels.
- pad_shape (tuple) – The padded shape of the image.
- device (str) – Device where the anchors will be put on.
Returns: Valid flags of anchors in multiple levels.
Return type: list(torch.Tensor)
-
class
mmdet.core.anchor.
LegacyAnchorGenerator
(strides, ratios, scales=None, base_sizes=None, scale_major=True, octave_base_scale=None, scales_per_octave=None, centers=None, center_offset=0.0)[source]¶ Legacy anchor generator used in MMDetection V1.x.
Note
Difference to the V2.0 anchor generator:
- The center offset of V1.x anchors are set to be 0.5 rather than 0.
- The width/height are minused by 1 when calculating the anchors’ centers and corners to meet the V1.x coordinate system.
- The anchors’ corners are quantized.
Parameters: - strides (list[int] | list[tuple[int]]) – Strides of anchors in multiple feature levels.
- ratios (list[float]) – The list of ratios between the height and width of anchors in a single level.
- scales (list[int] | None) – Anchor scales for anchors in a single level. It cannot be set at the same time if octave_base_scale and scales_per_octave are set.
- base_sizes (list[int]) – The basic sizes of anchors in multiple levels. If None is given, strides will be used to generate base_sizes.
- scale_major (bool) – Whether to multiply scales first when generating base anchors. If true, the anchors in the same row will have the same scales. By default it is True in V2.0
- octave_base_scale (int) – The base scale of octave.
- scales_per_octave (int) – Number of scales for each octave. octave_base_scale and scales_per_octave are usually used in retinanet and the scales should be None when they are set.
- centers (list[tuple[float, float]] | None) – The centers of the anchor relative to the feature grid center in multiple feature levels. By default it is set to be None and not used. It a list of float is given, this list will be used to shift the centers of anchors.
- center_offset (float) – The offset of center in propotion to anchors’ width and height. By default it is 0.5 in V2.0 but it should be 0.5 in v1.x models.
Examples
>>> from mmdet.core import LegacyAnchorGenerator >>> self = LegacyAnchorGenerator( >>> [16], [1.], [1.], [9], center_offset=0.5) >>> all_anchors = self.grid_anchors(((2, 2),), device='cpu') >>> print(all_anchors) [tensor([[ 0., 0., 8., 8.], [16., 0., 24., 8.], [ 0., 16., 8., 24.], [16., 16., 24., 24.]])]
-
gen_single_level_base_anchors
(base_size, scales, ratios, center=None)[source]¶ Generate base anchors of a single level.
Note
The width/height of anchors are minused by 1 when calculating the centers and corners to meet the V1.x coordinate system.
Parameters: - base_size (int | float) – Basic size of an anchor.
- scales (torch.Tensor) – Scales of the anchor.
- ratios (torch.Tensor) – The ratio between between the height. and width of anchors in a single level.
- center (tuple[float], optional) – The center of the base anchor related to a single feature grid. Defaults to None.
Returns: Anchors in a single-level feature map.
Return type: torch.Tensor
-
mmdet.core.anchor.
anchor_inside_flags
(flat_anchors, valid_flags, img_shape, allowed_border=0)[source]¶ Check whether the anchors are inside the border.
Parameters: - flat_anchors (torch.Tensor) – Flatten anchors, shape (n, 4).
- valid_flags (torch.Tensor) – An existing valid flags of anchors.
- img_shape (tuple(int)) – Shape of current image.
- allowed_border (int, optional) – The border to allow the valid anchor. Defaults to 0.
Returns: Flags indicating whether the anchors are inside a valid range.
Return type: torch.Tensor
-
mmdet.core.anchor.
images_to_levels
(target, num_levels)[source]¶ Convert targets by image to targets by feature level.
[target_img0, target_img1] -> [target_level0, target_level1, …]
-
mmdet.core.anchor.
calc_region
(bbox, ratio, featmap_size=None)[source]¶ Calculate a proportional bbox region.
The bbox center are fixed and the new h’ and w’ is h * ratio and w * ratio.
Parameters: - bbox (Tensor) – Bboxes to calculate regions, shape (n, 4).
- ratio (float) – Ratio of the output region.
- featmap_size (tuple) – Feature map size used for clipping the boundary.
Returns: x1, y1, x2, y2
Return type: tuple
-
class
mmdet.core.anchor.
YOLOAnchorGenerator
(strides, base_sizes)[source]¶ Anchor generator for YOLO.
Parameters: - strides (list[int] | list[tuple[int, int]]) – Strides of anchors in multiple feature levels.
- base_sizes (list[list[tuple[int, int]]]) – The basic sizes of anchors in multiple levels.
-
gen_base_anchors
()[source]¶ Generate base anchors.
Returns: Base anchors of a feature grid in multiple feature levels. Return type: list(torch.Tensor)
-
gen_single_level_base_anchors
(base_sizes_per_level, center=None)[source]¶ Generate base anchors of a single level.
Parameters: - base_sizes_per_level (list[tuple[int, int]]) – Basic sizes of anchors.
- center (tuple[float], optional) – The center of the base anchor related to a single feature grid. Defaults to None.
Returns: Anchors in a single-level feature maps.
Return type: torch.Tensor
-
num_levels
¶ number of feature levels that the generator will be applied
Type: int
-
responsible_flags
(featmap_sizes, gt_bboxes, device='cuda')[source]¶ Generate responsible anchor flags of grid cells in multiple scales.
Parameters: - featmap_sizes (list(tuple)) – List of feature map sizes in multiple feature levels.
- gt_bboxes (Tensor) – Ground truth boxes, shape (n, 4).
- device (str) – Device where the anchors will be put on.
Returns: responsible flags of anchors in multiple level
Return type: list(torch.Tensor)
-
single_level_responsible_flags
(featmap_size, gt_bboxes, stride, num_base_anchors, device='cuda')[source]¶ Generate the responsible flags of anchor in a single feature map.
Parameters: - featmap_size (tuple[int]) – The size of feature maps.
- gt_bboxes (Tensor) – Ground truth boxes, shape (n, 4).
- stride (tuple(int)) – stride of current level
- num_base_anchors (int) – The number of base anchors.
- device (str, optional) – Device where the flags will be put on. Defaults to ‘cuda’.
Returns: The valid flags of each anchor in a single level feature map.
Return type: torch.Tensor
-
class
mmdet.core.anchor.
MlvlPointGenerator
(strides, offset=0.5)[source]¶ Standard points generator for multi-level (Mlvl) feature maps in 2D points-based detectors.
Parameters: - strides (list[int] | list[tuple[int, int]]) – Strides of anchors in multiple feature levels in order (w, h).
- offset (float) – The offset of points, the value is normalized with corresponding stride. Defaults to 0.5.
-
grid_priors
(featmap_sizes, device='cuda', with_stride=False)[source]¶ Generate grid points of multiple feature levels.
Parameters: - featmap_sizes (list[tuple]) – List of feature map sizes in multiple feature levels, each size arrange as as (h, w).
- device (str) – The device where the anchors will be put on.
- with_stride (bool) – Whether to concatenate the stride to the last dimension of points.
Returns: Points of multiple feature levels. The sizes of each tensor should be (N, 2) when with stride is
False
, where N = width * height, width and height are the sizes of the corresponding feature level, and the last dimension 2 represent (coord_x, coord_y), otherwise the shape should be (N, 4), and the last dimension 4 represent (coord_x, coord_y, stride_w, stride_h).Return type: list[torch.Tensor]
-
num_base_priors
¶ The number of priors (points) at a point on the feature grid
Type: list[int]
-
num_levels
¶ number of feature levels that the generator will be applied
Type: int
-
single_level_grid_priors
(featmap_size, level_idx, device='cuda', with_stride=False)[source]¶ Generate grid Points of a single level.
Note
This function is usually called by method
self.grid_priors
.Parameters: - featmap_size (tuple[int]) – Size of the feature maps, arrange as (h, w).
- level_idx (int) – The index of corresponding feature map level.
- device (str, optional) – The device the tensor will be put on. Defaults to ‘cuda’.
- with_stride (bool) – Concatenate the stride to the last dimension of points.
Returns: Points of single feature levels. The shape of tensor should be (N, 2) when with stride is
False
, where N = width * height, width and height are the sizes of the corresponding feature level, and the last dimension 2 represent (coord_x, coord_y), otherwise the shape should be (N, 4), and the last dimension 4 represent (coord_x, coord_y, stride_w, stride_h).Return type: Tensor
-
single_level_valid_flags
(featmap_size, valid_size, device='cuda')[source]¶ Generate the valid flags of points of a single feature map.
Parameters: - featmap_size (tuple[int]) – The size of feature maps, arrange as as (h, w).
- valid_size (tuple[int]) – The valid size of the feature maps. The size arrange as as (h, w).
- device (str, optional) – The device where the flags will be put on. Defaults to ‘cuda’.
Returns: The valid flags of each points in a single level feature map.
Return type: torch.Tensor
-
sparse_priors
(prior_idxs, featmap_size, level_idx, dtype=torch.float32, device='cuda')[source]¶ Generate sparse points according to the
prior_idxs
.Parameters: - prior_idxs (Tensor) – The index of corresponding anchors in the feature map.
- featmap_size (tuple[int]) – feature map size arrange as (w, h).
- level_idx (int) – The level index of corresponding feature map.
- (obj (device) – torch.dtype): Date type of points. Defaults to
torch.float32
. - (obj – torch.device): The device where the points is located.
Returns: Anchor with shape (N, 2), N should be equal to the length of
prior_idxs
. And last dimension 2 represent (coord_x, coord_y).Return type: Tensor
-
valid_flags
(featmap_sizes, pad_shape, device='cuda')[source]¶ Generate valid flags of points of multiple feature levels.
Parameters: - featmap_sizes (list(tuple)) – List of feature map sizes in multiple feature levels, each size arrange as as (h, w).
- pad_shape (tuple(int)) – The padded shape of the image, arrange as (h, w).
- device (str) – The device where the anchors will be put on.
Returns: Valid flags of points of multiple levels.
Return type: list(torch.Tensor)
bbox¶
-
mmdet.core.bbox.
bbox_overlaps
(bboxes1, bboxes2, mode='iou', is_aligned=False, eps=1e-06)[source]¶ Calculate overlap between two set of bboxes.
FP16 Contributed by https://github.com/open-mmlab/mmdetection/pull/4889 .. note:
Assume bboxes1 is M x 4, bboxes2 is N x 4, when mode is 'iou', there are some new generated variable when calculating IOU using bbox_overlaps function: 1) is_aligned is False area1: M x 1 area2: N x 1 lt: M x N x 2 rb: M x N x 2 wh: M x N x 2 overlap: M x N x 1 union: M x N x 1 ious: M x N x 1 Total memory: S = (9 x N x M + N + M) * 4 Byte, When using FP16, we can reduce: R = (9 x N x M + N + M) * 4 / 2 Byte R large than (N + M) * 4 * 2 is always true when N and M >= 1. Obviously, N + M <= N * M < 3 * N * M, when N >=2 and M >=2, N + 1 < 3 * N, when N or M is 1. Given M = 40 (ground truth), N = 400000 (three anchor boxes in per grid, FPN, R-CNNs), R = 275 MB (one times) A special case (dense detection), M = 512 (ground truth), R = 3516 MB = 3.43 GB When the batch size is B, reduce: B x R Therefore, CUDA memory runs out frequently. Experiments on GeForce RTX 2080Ti (11019 MiB): | dtype | M | N | Use | Real | Ideal | |:----:|:----:|:----:|:----:|:----:|:----:| | FP32 | 512 | 400000 | 8020 MiB | -- | -- | | FP16 | 512 | 400000 | 4504 MiB | 3516 MiB | 3516 MiB | | FP32 | 40 | 400000 | 1540 MiB | -- | -- | | FP16 | 40 | 400000 | 1264 MiB | 276MiB | 275 MiB | 2) is_aligned is True area1: N x 1 area2: N x 1 lt: N x 2 rb: N x 2 wh: N x 2 overlap: N x 1 union: N x 1 ious: N x 1 Total memory: S = 11 x N * 4 Byte When using FP16, we can reduce: R = 11 x N * 4 / 2 Byte So do the 'giou' (large than 'iou'). Time-wise, FP16 is generally faster than FP32. When gpu_assign_thr is not -1, it takes more time on cpu but not reduce memory. There, we can reduce half the memory and keep the speed.
If
is_aligned
isFalse
, then calculate the overlaps between each bbox of bboxes1 and bboxes2, otherwise the overlaps between each aligned pair of bboxes1 and bboxes2.Parameters: - bboxes1 (Tensor) – shape (B, m, 4) in <x1, y1, x2, y2> format or empty.
- bboxes2 (Tensor) – shape (B, n, 4) in <x1, y1, x2, y2> format or empty.
B indicates the batch dim, in shape (B1, B2, …, Bn).
If
is_aligned
isTrue
, then m and n must be equal. - mode (str) – “iou” (intersection over union), “iof” (intersection over foreground) or “giou” (generalized intersection over union). Default “iou”.
- is_aligned (bool, optional) – If True, then m and n must be equal. Default False.
- eps (float, optional) – A value added to the denominator for numerical stability. Default 1e-6.
Returns: shape (m, n) if
is_aligned
is False else shape (m,)Return type: Tensor
Example
>>> bboxes1 = torch.FloatTensor([ >>> [0, 0, 10, 10], >>> [10, 10, 20, 20], >>> [32, 32, 38, 42], >>> ]) >>> bboxes2 = torch.FloatTensor([ >>> [0, 0, 10, 20], >>> [0, 10, 10, 19], >>> [10, 10, 20, 20], >>> ]) >>> overlaps = bbox_overlaps(bboxes1, bboxes2) >>> assert overlaps.shape == (3, 3) >>> overlaps = bbox_overlaps(bboxes1, bboxes2, is_aligned=True) >>> assert overlaps.shape == (3, )
Example
>>> empty = torch.empty(0, 4) >>> nonempty = torch.FloatTensor([[0, 0, 10, 9]]) >>> assert tuple(bbox_overlaps(empty, nonempty).shape) == (0, 1) >>> assert tuple(bbox_overlaps(nonempty, empty).shape) == (1, 0) >>> assert tuple(bbox_overlaps(empty, empty).shape) == (0, 0)
-
class
mmdet.core.bbox.
BboxOverlaps2D
(scale=1.0, dtype=None)[source]¶ 2D Overlaps (e.g. IoUs, GIoUs) Calculator.
-
class
mmdet.core.bbox.
MaxIoUAssigner
(pos_iou_thr, neg_iou_thr, min_pos_iou=0.0, gt_max_assign_all=True, ignore_iof_thr=-1, ignore_wrt_candidates=True, match_low_quality=True, gpu_assign_thr=-1, iou_calculator={'type': 'BboxOverlaps2D'})[source]¶ Assign a corresponding gt bbox or background to each bbox.
Each proposals will be assigned with -1, or a semi-positive integer indicating the ground truth index.
- -1: negative sample, no assigned gt
- semi-positive integer: positive sample, index (0-based) of assigned gt
Parameters: - pos_iou_thr (float) – IoU threshold for positive bboxes.
- neg_iou_thr (float or tuple) – IoU threshold for negative bboxes.
- min_pos_iou (float) – Minimum iou for a bbox to be considered as a positive bbox. Positive samples can have smaller IoU than pos_iou_thr due to the 4th step (assign max IoU sample to each gt).
- gt_max_assign_all (bool) – Whether to assign all bboxes with the same highest overlap with some gt to that gt.
- ignore_iof_thr (float) – IoF threshold for ignoring bboxes (if gt_bboxes_ignore is specified). Negative values mean not ignoring any bboxes.
- ignore_wrt_candidates (bool) – Whether to compute the iof between bboxes and gt_bboxes_ignore, or the contrary.
- match_low_quality (bool) – Whether to allow low quality matches. This is usually allowed for RPN and single stage detectors, but not allowed in the second stage. Details are demonstrated in Step 4.
- gpu_assign_thr (int) – The upper bound of the number of GT for GPU assign. When the number of gt is above this threshold, will assign on CPU device. Negative values mean not assign on CPU.
-
assign
(bboxes, gt_bboxes, gt_bboxes_ignore=None, gt_labels=None)[source]¶ Assign gt to bboxes.
This method assign a gt bbox to every bbox (proposal/anchor), each bbox will be assigned with -1, or a semi-positive number. -1 means negative sample, semi-positive number is the index (0-based) of assigned gt. The assignment is done in following steps, the order matters.
- assign every bbox to the background
- assign proposals whose iou with all gts < neg_iou_thr to 0
- for each bbox, if the iou with its nearest gt >= pos_iou_thr, assign it to that bbox
- for each gt bbox, assign its nearest proposals (may be more than one) to itself
Parameters: - bboxes (Tensor) – Bounding boxes to be assigned, shape(n, 4).
- gt_bboxes (Tensor) – Groundtruth boxes, shape (k, 4).
- gt_bboxes_ignore (Tensor, optional) – Ground truth bboxes that are labelled as ignored, e.g., crowd boxes in COCO.
- gt_labels (Tensor, optional) – Label of gt_bboxes, shape (k, ).
Returns: The assign result.
Return type: Example
>>> self = MaxIoUAssigner(0.5, 0.5) >>> bboxes = torch.Tensor([[0, 0, 10, 10], [10, 10, 20, 20]]) >>> gt_bboxes = torch.Tensor([[0, 0, 10, 9]]) >>> assign_result = self.assign(bboxes, gt_bboxes) >>> expected_gt_inds = torch.LongTensor([1, 0]) >>> assert torch.all(assign_result.gt_inds == expected_gt_inds)
-
class
mmdet.core.bbox.
AssignResult
(num_gts, gt_inds, max_overlaps, labels=None)[source]¶ Stores assignments between predicted and truth boxes.
-
num_gts
¶ the number of truth boxes considered when computing this assignment
Type: int
-
gt_inds
¶ for each predicted box indicates the 1-based index of the assigned truth box. 0 means unassigned and -1 means ignore.
Type: LongTensor
-
max_overlaps
¶ the iou between the predicted box and its assigned truth box.
Type: FloatTensor
-
labels
¶ If specified, for each predicted box indicates the category label of the assigned truth box.
Type: None | LongTensor
Example
>>> # An assign result between 4 predicted boxes and 9 true boxes >>> # where only two boxes were assigned. >>> num_gts = 9 >>> max_overlaps = torch.LongTensor([0, .5, .9, 0]) >>> gt_inds = torch.LongTensor([-1, 1, 2, 0]) >>> labels = torch.LongTensor([0, 3, 4, 0]) >>> self = AssignResult(num_gts, gt_inds, max_overlaps, labels) >>> print(str(self)) # xdoctest: +IGNORE_WANT <AssignResult(num_gts=9, gt_inds.shape=(4,), max_overlaps.shape=(4,), labels.shape=(4,))> >>> # Force addition of gt labels (when adding gt as proposals) >>> new_labels = torch.LongTensor([3, 4, 5]) >>> self.add_gt_(new_labels) >>> print(str(self)) # xdoctest: +IGNORE_WANT <AssignResult(num_gts=9, gt_inds.shape=(7,), max_overlaps.shape=(7,), labels.shape=(7,))>
-
add_gt_
(gt_labels)[source]¶ Add ground truth as assigned results.
Parameters: gt_labels (torch.Tensor) – Labels of gt boxes
-
info
¶ a dictionary of info about the object
Type: dict
-
num_preds
¶ the number of predictions in this assignment
Type: int
-
classmethod
random
(**kwargs)[source]¶ Create random AssignResult for tests or debugging.
Parameters: - num_preds – number of predicted boxes
- num_gts – number of true boxes
- p_ignore (float) – probability of a predicted box assigned to an ignored truth
- p_assigned (float) – probability of a predicted box not being assigned
- p_use_label (float | bool) – with labels or not
- rng (None | int | numpy.random.RandomState) – seed or state
Returns: Randomly generated assign results.
Return type: Example
>>> from mmdet.core.bbox.assigners.assign_result import * # NOQA >>> self = AssignResult.random() >>> print(self.info)
-
-
class
mmdet.core.bbox.
BaseSampler
(num, pos_fraction, neg_pos_ub=-1, add_gt_as_proposals=True, **kwargs)[source]¶ Base class of samplers.
-
sample
(assign_result, bboxes, gt_bboxes, gt_labels=None, **kwargs)[source]¶ Sample positive and negative bboxes.
This is a simple implementation of bbox sampling given candidates, assigning results and ground truth bboxes.
Parameters: - assign_result (
AssignResult
) – Bbox assigning results. - bboxes (Tensor) – Boxes to be sampled from.
- gt_bboxes (Tensor) – Ground truth bboxes.
- gt_labels (Tensor, optional) – Class labels of ground truth bboxes.
Returns: Sampling result.
Return type: Example
>>> from mmdet.core.bbox import RandomSampler >>> from mmdet.core.bbox import AssignResult >>> from mmdet.core.bbox.demodata import ensure_rng, random_boxes >>> rng = ensure_rng(None) >>> assign_result = AssignResult.random(rng=rng) >>> bboxes = random_boxes(assign_result.num_preds, rng=rng) >>> gt_bboxes = random_boxes(assign_result.num_gts, rng=rng) >>> gt_labels = None >>> self = RandomSampler(num=32, pos_fraction=0.5, neg_pos_ub=-1, >>> add_gt_as_proposals=False) >>> self = self.sample(assign_result, bboxes, gt_bboxes, gt_labels)
- assign_result (
-
-
class
mmdet.core.bbox.
PseudoSampler
(**kwargs)[source]¶ A pseudo sampler that does not do sampling actually.
-
sample
(assign_result, bboxes, gt_bboxes, **kwargs)[source]¶ Directly returns the positive and negative indices of samples.
Parameters: - assign_result (
AssignResult
) – Assigned results - bboxes (torch.Tensor) – Bounding boxes
- gt_bboxes (torch.Tensor) – Ground truth boxes
Returns: sampler results
Return type: - assign_result (
-
-
class
mmdet.core.bbox.
RandomSampler
(num, pos_fraction, neg_pos_ub=-1, add_gt_as_proposals=True, **kwargs)[source]¶ Random sampler.
Parameters: - num (int) – Number of samples
- pos_fraction (float) – Fraction of positive samples
- neg_pos_up (int, optional) – Upper bound number of negative and positive samples. Defaults to -1.
- add_gt_as_proposals (bool, optional) – Whether to add ground truth boxes as proposals. Defaults to True.
-
random_choice
(gallery, num)[source]¶ Random select some elements from the gallery.
If gallery is a Tensor, the returned indices will be a Tensor; If gallery is a ndarray or list, the returned indices will be a ndarray.
Parameters: - gallery (Tensor | ndarray | list) – indices pool.
- num (int) – expected sample num.
Returns: sampled indices.
Return type: Tensor or ndarray
-
class
mmdet.core.bbox.
InstanceBalancedPosSampler
(num, pos_fraction, neg_pos_ub=-1, add_gt_as_proposals=True, **kwargs)[source]¶ Instance balanced sampler that samples equal number of positive samples for each instance.
-
class
mmdet.core.bbox.
IoUBalancedNegSampler
(num, pos_fraction, floor_thr=-1, floor_fraction=0, num_bins=3, **kwargs)[source]¶ IoU Balanced Sampling.
arXiv: https://arxiv.org/pdf/1904.02701.pdf (CVPR 2019)
Sampling proposals according to their IoU. floor_fraction of needed RoIs are sampled from proposals whose IoU are lower than floor_thr randomly. The others are sampled from proposals whose IoU are higher than floor_thr. These proposals are sampled from some bins evenly, which are split by num_bins via IoU evenly.
Parameters: - num (int) – number of proposals.
- pos_fraction (float) – fraction of positive proposals.
- floor_thr (float) – threshold (minimum) IoU for IoU balanced sampling, set to -1 if all using IoU balanced sampling.
- floor_fraction (float) – sampling fraction of proposals under floor_thr.
- num_bins (int) – number of bins in IoU balanced sampling.
-
sample_via_interval
(max_overlaps, full_set, num_expected)[source]¶ Sample according to the iou interval.
Parameters: - max_overlaps (torch.Tensor) – IoU between bounding boxes and ground truth boxes.
- full_set (set(int)) – A full set of indices of boxes。
- num_expected (int) – Number of expected samples。
Returns: Indices of samples
Return type: np.ndarray
-
class
mmdet.core.bbox.
CombinedSampler
(pos_sampler, neg_sampler, **kwargs)[source]¶ A sampler that combines positive sampler and negative sampler.
-
class
mmdet.core.bbox.
OHEMSampler
(num, pos_fraction, context, neg_pos_ub=-1, add_gt_as_proposals=True, **kwargs)[source]¶ Online Hard Example Mining Sampler described in Training Region-based Object Detectors with Online Hard Example Mining.
-
class
mmdet.core.bbox.
SamplingResult
(pos_inds, neg_inds, bboxes, gt_bboxes, assign_result, gt_flags)[source]¶ Bbox sampling result.
Example
>>> # xdoctest: +IGNORE_WANT >>> from mmdet.core.bbox.samplers.sampling_result import * # NOQA >>> self = SamplingResult.random(rng=10) >>> print(f'self = {self}') self = <SamplingResult({ 'neg_bboxes': torch.Size([12, 4]), 'neg_inds': tensor([ 0, 1, 2, 4, 5, 6, 7, 8, 9, 10, 11, 12]), 'num_gts': 4, 'pos_assigned_gt_inds': tensor([], dtype=torch.int64), 'pos_bboxes': torch.Size([0, 4]), 'pos_inds': tensor([], dtype=torch.int64), 'pos_is_gt': tensor([], dtype=torch.uint8) })>
-
bboxes
¶ concatenated positive and negative boxes
Type: torch.Tensor
-
info
¶ Returns a dictionary of info about the object.
-
classmethod
random
(rng=None, **kwargs)[source]¶ Parameters: - rng (None | int | numpy.random.RandomState) – seed or state.
- kwargs (keyword arguments) –
- num_preds: number of predicted boxes
- num_gts: number of true boxes
- p_ignore (float): probability of a predicted box assigned to an ignored truth.
- p_assigned (float): probability of a predicted box not being assigned.
- p_use_label (float | bool): with labels or not.
Returns: Randomly generated sampling result.
Return type: Example
>>> from mmdet.core.bbox.samplers.sampling_result import * # NOQA >>> self = SamplingResult.random() >>> print(self.__dict__)
-
-
class
mmdet.core.bbox.
ScoreHLRSampler
(num, pos_fraction, context, neg_pos_ub=-1, add_gt_as_proposals=True, k=0.5, bias=0, score_thr=0.05, iou_thr=0.5, **kwargs)[source]¶ Importance-based Sample Reweighting (ISR_N), described in Prime Sample Attention in Object Detection.
Score hierarchical local rank (HLR) differentiates with RandomSampler in negative part. It firstly computes Score-HLR in a two-step way, then linearly maps score hlr to the loss weights.
Parameters: - num (int) – Total number of sampled RoIs.
- pos_fraction (float) – Fraction of positive samples.
- context (
BaseRoIHead
) – RoI head that the sampler belongs to. - neg_pos_ub (int) – Upper bound of the ratio of num negative to num positive, -1 means no upper bound.
- add_gt_as_proposals (bool) – Whether to add ground truth as proposals.
- k (float) – Power of the non-linear mapping.
- bias (float) – Shift of the non-linear mapping.
- score_thr (float) – Minimum score that a negative sample is to be considered as valid bbox.
-
static
random_choice
(gallery, num)[source]¶ Randomly select some elements from the gallery.
If gallery is a Tensor, the returned indices will be a Tensor; If gallery is a ndarray or list, the returned indices will be a ndarray.
Parameters: - gallery (Tensor | ndarray | list) – indices pool.
- num (int) – expected sample num.
Returns: sampled indices.
Return type: Tensor or ndarray
-
sample
(assign_result, bboxes, gt_bboxes, gt_labels=None, img_meta=None, **kwargs)[source]¶ Sample positive and negative bboxes.
This is a simple implementation of bbox sampling given candidates, assigning results and ground truth bboxes.
Parameters: - assign_result (
AssignResult
) – Bbox assigning results. - bboxes (Tensor) – Boxes to be sampled from.
- gt_bboxes (Tensor) – Ground truth bboxes.
- gt_labels (Tensor, optional) – Class labels of ground truth bboxes.
Returns: - Sampling result and negative
label weights.
Return type: tuple[
SamplingResult
, Tensor]- assign_result (
-
mmdet.core.bbox.
bbox_flip
(bboxes, img_shape, direction='horizontal')[source]¶ Flip bboxes horizontally or vertically.
Parameters: - bboxes (Tensor) – Shape (…, 4*k)
- img_shape (tuple) – Image shape.
- direction (str) – Flip direction, options are “horizontal”, “vertical”, “diagonal”. Default: “horizontal”
Returns: Flipped bboxes.
Return type: Tensor
-
mmdet.core.bbox.
bbox_mapping
(bboxes, img_shape, scale_factor, flip, flip_direction='horizontal')[source]¶ Map bboxes from the original image scale to testing scale.
-
mmdet.core.bbox.
bbox_mapping_back
(bboxes, img_shape, scale_factor, flip, flip_direction='horizontal')[source]¶ Map bboxes from testing scale to original image scale.
-
mmdet.core.bbox.
bbox2roi
(bbox_list)[source]¶ Convert a list of bboxes to roi format.
Parameters: bbox_list (list[Tensor]) – a list of bboxes corresponding to a batch of images. Returns: shape (n, 5), [batch_ind, x1, y1, x2, y2] Return type: Tensor
-
mmdet.core.bbox.
roi2bbox
(rois)[source]¶ Convert rois to bounding box format.
Parameters: rois (torch.Tensor) – RoIs with the shape (n, 5) where the first column indicates batch id of each RoI. Returns: Converted boxes of corresponding rois. Return type: list[torch.Tensor]
-
mmdet.core.bbox.
bbox2result
(bboxes, labels, num_classes)[source]¶ Convert detection results to a list of numpy arrays.
Parameters: - bboxes (torch.Tensor | np.ndarray) – shape (n, 5)
- labels (torch.Tensor | np.ndarray) – shape (n, )
- num_classes (int) – class number, including background class
Returns: bbox results of each class
Return type: list(ndarray)
-
mmdet.core.bbox.
distance2bbox
(points, distance, max_shape=None)[source]¶ Decode distance prediction to bounding box.
Parameters: - points (Tensor) – Shape (B, N, 2) or (N, 2).
- distance (Tensor) – Distance from the given point to 4 boundaries (left, top, right, bottom). Shape (B, N, 4) or (N, 4)
- (Sequence[int] or torch.Tensor or Sequence[ (max_shape) – Sequence[int]],optional): Maximum bounds for boxes, specifies (H, W, C) or (H, W). If priors shape is (B, N, 4), then the max_shape should be a Sequence[Sequence[int]] and the length of max_shape should also be B.
Returns: Boxes with shape (N, 4) or (B, N, 4)
Return type: Tensor
-
mmdet.core.bbox.
bbox2distance
(points, bbox, max_dis=None, eps=0.1)[source]¶ Decode bounding box based on distances.
Parameters: - points (Tensor) – Shape (n, 2), [x, y].
- bbox (Tensor) – Shape (n, 4), “xyxy” format
- max_dis (float) – Upper bound of the distance.
- eps (float) – a small value to ensure target < max_dis, instead <=
Returns: Decoded distances.
Return type: Tensor
-
class
mmdet.core.bbox.
BaseBBoxCoder
(**kwargs)[source]¶ Base bounding box coder.
-
class
mmdet.core.bbox.
DeltaXYWHBBoxCoder
(target_means=(0.0, 0.0, 0.0, 0.0), target_stds=(1.0, 1.0, 1.0, 1.0), clip_border=True, add_ctr_clamp=False, ctr_clamp=32)[source]¶ Delta XYWH BBox coder.
Following the practice in R-CNN, this coder encodes bbox (x1, y1, x2, y2) into delta (dx, dy, dw, dh) and decodes delta (dx, dy, dw, dh) back to original bbox (x1, y1, x2, y2).
Parameters: - target_means (Sequence[float]) – Denormalizing means of target for delta coordinates
- target_stds (Sequence[float]) – Denormalizing standard deviation of target for delta coordinates
- clip_border (bool, optional) – Whether clip the objects outside the border of the image. Defaults to True.
- add_ctr_clamp (bool) – Whether to add center clamp, when added, the predicted box is clamped is its center is too far away from the original anchor’s center. Only used by YOLOF. Default False.
- ctr_clamp (int) – the maximum pixel shift to clamp. Only used by YOLOF. Default 32.
-
decode
(bboxes, pred_bboxes, max_shape=None, wh_ratio_clip=0.016)[source]¶ Apply transformation pred_bboxes to boxes.
Parameters: - bboxes (torch.Tensor) – Basic boxes. Shape (B, N, 4) or (N, 4)
- pred_bboxes (Tensor) – Encoded offsets with respect to each roi. Has shape (B, N, num_classes * 4) or (B, N, 4) or (N, num_classes * 4) or (N, 4). Note N = num_anchors * W * H when rois is a grid of anchors.Offset encoding follows [1].
- (Sequence[int] or torch.Tensor or Sequence[ (max_shape) – Sequence[int]],optional): Maximum bounds for boxes, specifies (H, W, C) or (H, W). If bboxes shape is (B, N, 4), then the max_shape should be a Sequence[Sequence[int]] and the length of max_shape should also be B.
- wh_ratio_clip (float, optional) – The allowed ratio between width and height.
Returns: Decoded boxes.
Return type: torch.Tensor
-
encode
(bboxes, gt_bboxes)[source]¶ Get box regression transformation deltas that can be used to transform the
bboxes
into thegt_bboxes
.Parameters: - bboxes (torch.Tensor) – Source boxes, e.g., object proposals.
- gt_bboxes (torch.Tensor) – Target of the transformation, e.g., ground-truth boxes.
Returns: Box transformation deltas
Return type: torch.Tensor
-
class
mmdet.core.bbox.
TBLRBBoxCoder
(normalizer=4.0, clip_border=True)[source]¶ TBLR BBox coder.
Following the practice in FSAF, this coder encodes gt bboxes (x1, y1, x2, y2) into (top, bottom, left, right) and decode it back to the original.
Parameters: - normalizer (list | float) – Normalization factor to be divided with when coding the coordinates. If it is a list, it should have length of 4 indicating normalization factor in tblr dims. Otherwise it is a unified float factor for all dims. Default: 4.0
- clip_border (bool, optional) – Whether clip the objects outside the border of the image. Defaults to True.
-
decode
(bboxes, pred_bboxes, max_shape=None)[source]¶ Apply transformation pred_bboxes to boxes.
Parameters: - bboxes (torch.Tensor) – Basic boxes.Shape (B, N, 4) or (N, 4)
- pred_bboxes (torch.Tensor) – Encoded boxes with shape (B, N, 4) or (N, 4)
- (Sequence[int] or torch.Tensor or Sequence[ (max_shape) – Sequence[int]],optional): Maximum bounds for boxes, specifies (H, W, C) or (H, W). If bboxes shape is (B, N, 4), then the max_shape should be a Sequence[Sequence[int]] and the length of max_shape should also be B.
Returns: Decoded boxes.
Return type: torch.Tensor
-
encode
(bboxes, gt_bboxes)[source]¶ Get box regression transformation deltas that can be used to transform the
bboxes
into thegt_bboxes
in the (top, left, bottom, right) order.Parameters: - bboxes (torch.Tensor) – source boxes, e.g., object proposals.
- gt_bboxes (torch.Tensor) – target of the transformation, e.g., ground truth boxes.
Returns: Box transformation deltas
Return type: torch.Tensor
-
class
mmdet.core.bbox.
CenterRegionAssigner
(pos_scale, neg_scale, min_pos_iof=0.01, ignore_gt_scale=0.5, foreground_dominate=False, iou_calculator={'type': 'BboxOverlaps2D'})[source]¶ Assign pixels at the center region of a bbox as positive.
Each proposals will be assigned with -1, 0, or a positive integer indicating the ground truth index. - -1: negative samples - semi-positive numbers: positive sample, index (0-based) of assigned gt
Parameters: - pos_scale (float) – Threshold within which pixels are labelled as positive.
- neg_scale (float) – Threshold above which pixels are labelled as positive.
- min_pos_iof (float) – Minimum iof of a pixel with a gt to be labelled as positive. Default: 1e-2
- ignore_gt_scale (float) – Threshold within which the pixels are ignored when the gt is labelled as shadowed. Default: 0.5
- foreground_dominate (bool) – If True, the bbox will be assigned as positive when a gt’s kernel region overlaps with another’s shadowed (ignored) region, otherwise it is set as ignored. Default to False.
-
assign
(bboxes, gt_bboxes, gt_bboxes_ignore=None, gt_labels=None)[source]¶ Assign gt to bboxes.
This method assigns gts to every bbox (proposal/anchor), each bbox will be assigned with -1, or a semi-positive number. -1 means negative sample, semi-positive number is the index (0-based) of assigned gt.
Parameters: - bboxes (Tensor) – Bounding boxes to be assigned, shape(n, 4).
- gt_bboxes (Tensor) – Groundtruth boxes, shape (k, 4).
- gt_bboxes_ignore (tensor, optional) – Ground truth bboxes that are labelled as ignored, e.g., crowd boxes in COCO.
- gt_labels (tensor, optional) – Label of gt_bboxes, shape (num_gts,).
Returns: The assigned result. Note that shadowed_labels of shape (N, 2) is also added as an assign_result attribute. shadowed_labels is a tensor composed of N pairs of anchor_ind, class_label], where N is the number of anchors that lie in the outer region of a gt, anchor_ind is the shadowed anchor index and class_label is the shadowed class label.
Return type: Example
>>> self = CenterRegionAssigner(0.2, 0.2) >>> bboxes = torch.Tensor([[0, 0, 10, 10], [10, 10, 20, 20]]) >>> gt_bboxes = torch.Tensor([[0, 0, 10, 10]]) >>> assign_result = self.assign(bboxes, gt_bboxes) >>> expected_gt_inds = torch.LongTensor([1, 0]) >>> assert torch.all(assign_result.gt_inds == expected_gt_inds)
-
assign_one_hot_gt_indices
(is_bbox_in_gt_core, is_bbox_in_gt_shadow, gt_priority=None)[source]¶ Assign only one gt index to each prior box.
Gts with large gt_priority are more likely to be assigned.
Parameters: - is_bbox_in_gt_core (Tensor) – Bool tensor indicating the bbox center is in the core area of a gt (e.g. 0-0.2). Shape: (num_prior, num_gt).
- is_bbox_in_gt_shadow (Tensor) – Bool tensor indicating the bbox center is in the shadowed area of a gt (e.g. 0.2-0.5). Shape: (num_prior, num_gt).
- gt_priority (Tensor) – Priorities of gts. The gt with a higher priority is more likely to be assigned to the bbox when the bbox match with multiple gts. Shape: (num_gt, ).
Returns: Returns (assigned_gt_inds, shadowed_gt_inds).
- assigned_gt_inds: The assigned gt index of each prior bbox (i.e. index from 1 to num_gts). Shape: (num_prior, ).
- shadowed_gt_inds: shadowed gt indices. It is a tensor of shape (num_ignore, 2) with first column being the shadowed prior bbox indices and the second column the shadowed gt indices (1-based).
Return type: tuple
-
get_gt_priorities
(gt_bboxes)[source]¶ Get gt priorities according to their areas.
Smaller gt has higher priority.
Parameters: gt_bboxes (Tensor) – Ground truth boxes, shape (k, 4). Returns: The priority of gts so that gts with larger priority is more likely to be assigned. Shape (k, ) Return type: Tensor
-
mmdet.core.bbox.
bbox_rescale
(bboxes, scale_factor=1.0)[source]¶ Rescale bounding box w.r.t. scale_factor.
Parameters: - bboxes (Tensor) – Shape (n, 4) for bboxes or (n, 5) for rois
- scale_factor (float) – rescale factor
Returns: Rescaled bboxes.
Return type: Tensor
-
mmdet.core.bbox.
bbox_cxcywh_to_xyxy
(bbox)[source]¶ Convert bbox coordinates from (cx, cy, w, h) to (x1, y1, x2, y2).
Parameters: bbox (Tensor) – Shape (n, 4) for bboxes. Returns: Converted bboxes. Return type: Tensor
-
mmdet.core.bbox.
bbox_xyxy_to_cxcywh
(bbox)[source]¶ Convert bbox coordinates from (x1, y1, x2, y2) to (cx, cy, w, h).
Parameters: bbox (Tensor) – Shape (n, 4) for bboxes. Returns: Converted bboxes. Return type: Tensor
-
class
mmdet.core.bbox.
RegionAssigner
(center_ratio=0.2, ignore_ratio=0.5)[source]¶ Assign a corresponding gt bbox or background to each bbox.
Each proposals will be assigned with -1, 0, or a positive integer indicating the ground truth index.
- -1: don’t care
- 0: negative sample, no assigned gt
- positive integer: positive sample, index (1-based) of assigned gt
Parameters: - center_ratio – ratio of the region in the center of the bbox to define positive sample.
- ignore_ratio – ratio of the region to define ignore samples.
-
assign
(mlvl_anchors, mlvl_valid_flags, gt_bboxes, img_meta, featmap_sizes, anchor_scale, anchor_strides, gt_bboxes_ignore=None, gt_labels=None, allowed_border=0)[source]¶ Assign gt to anchors.
This method assign a gt bbox to every bbox (proposal/anchor), each bbox will be assigned with -1, 0, or a positive number. -1 means don’t care, 0 means negative sample, positive number is the index (1-based) of assigned gt.
The assignment is done in following steps, and the order matters.
- Assign every anchor to 0 (negative)
- (For each gt_bboxes) Compute ignore flags based on ignore_region then assign -1 to anchors w.r.t. ignore flags
- (For each gt_bboxes) Compute pos flags based on center_region then assign gt_bboxes to anchors w.r.t. pos flags
- (For each gt_bboxes) Compute ignore flags based on adjacent anchor level then assign -1 to anchors w.r.t. ignore flags
- Assign anchor outside of image to -1
Parameters: - mlvl_anchors (list[Tensor]) – Multi level anchors.
- mlvl_valid_flags (list[Tensor]) – Multi level valid flags.
- gt_bboxes (Tensor) – Ground truth bboxes of image
- img_meta (dict) – Meta info of image.
- featmap_sizes (list[Tensor]) – Feature mapsize each level
- anchor_scale (int) – Scale of the anchor.
- anchor_strides (list[int]) – Stride of the anchor.
- gt_bboxes – Groundtruth boxes, shape (k, 4).
- gt_bboxes_ignore (Tensor, optional) – Ground truth bboxes that are labelled as ignored, e.g., crowd boxes in COCO.
- gt_labels (Tensor, optional) – Label of gt_bboxes, shape (k, ).
- allowed_border (int, optional) – The border to allow the valid anchor. Defaults to 0.
Returns: The assign result.
Return type:
export¶
-
mmdet.core.export.
build_model_from_cfg
(config_path, checkpoint_path, cfg_options=None)[source]¶ Build a model from config and load the given checkpoint.
Parameters: - config_path (str) – the OpenMMLab config for the model we want to export to ONNX
- checkpoint_path (str) – Path to the corresponding checkpoint
Returns: the built model
Return type: torch.nn.Module
-
mmdet.core.export.
generate_inputs_and_wrap_model
(config_path, checkpoint_path, input_config, cfg_options=None)[source]¶ Prepare sample input and wrap model for ONNX export.
The ONNX export API only accept args, and all inputs should be torch.Tensor or corresponding types (such as tuple of tensor). So we should call this function before exporting. This function will:
- generate corresponding inputs which are used to execute the model.
- Wrap the model’s forward function.
For example, the MMDet models’ forward function has a parameter
return_loss:bool
. As we want to set it as False while export API supports neither bool type or kwargs. So we have to replace the forward method likemodel.forward = partial(model.forward, return_loss=False)
.Parameters: - config_path (str) – the OpenMMLab config for the model we want to export to ONNX
- checkpoint_path (str) – Path to the corresponding checkpoint
- input_config (dict) – the exactly data in this dict depends on the framework. For MMSeg, we can just declare the input shape, and generate the dummy data accordingly. However, for MMDet, we may pass the real img path, or the NMS will return None as there is no legal bbox.
Returns: - (model, tensor_data) wrapped model which can be called by
model(*tensor_data)
and a list of inputs which are used to execute the model while exporting.
Return type: tuple
-
mmdet.core.export.
preprocess_example_input
(input_config)[source]¶ Prepare an example input image for
generate_inputs_and_wrap_model
.Parameters: input_config (dict) – customized config describing the example input. Returns: (one_img, one_meta), tensor of the example input image and meta information for the example input image. Return type: tuple Examples
>>> from mmdet.core.export import preprocess_example_input >>> input_config = { >>> 'input_shape': (1,3,224,224), >>> 'input_path': 'demo/demo.jpg', >>> 'normalize_cfg': { >>> 'mean': (123.675, 116.28, 103.53), >>> 'std': (58.395, 57.12, 57.375) >>> } >>> } >>> one_img, one_meta = preprocess_example_input(input_config) >>> print(one_img.shape) torch.Size([1, 3, 224, 224]) >>> print(one_meta) {'img_shape': (224, 224, 3), 'ori_shape': (224, 224, 3), 'pad_shape': (224, 224, 3), 'filename': '<demo>.png', 'scale_factor': 1.0, 'flip': False}
-
mmdet.core.export.
get_k_for_topk
(k, size)[source]¶ Get k of TopK for onnx exporting.
- The K of TopK in TensorRT should not be a Tensor, while in ONNX Runtime
- it could be a Tensor.Due to dynamic shape feature, we have to decide whether to do TopK and what K it should be while exporting to ONNX.
- If returned K is less than zero, it means we do not have to do
- TopK operation.
Parameters: - k (int or Tensor) – The set k value for nms from config file.
- size (Tensor or torch.Size) – The number of elements of TopK’s input tensor
Returns: (int or Tensor): The final K for TopK.
Return type: tuple
-
mmdet.core.export.
add_dummy_nms_for_onnx
(boxes, scores, max_output_boxes_per_class=1000, iou_threshold=0.5, score_threshold=0.05, pre_top_k=-1, after_top_k=-1, labels=None)[source]¶ Create a dummy onnx::NonMaxSuppression op while exporting to ONNX.
This function helps exporting to onnx with batch and multiclass NMS op. It only supports class-agnostic detection results. That is, the scores is of shape (N, num_bboxes, num_classes) and the boxes is of shape (N, num_boxes, 4).
Parameters: - boxes (Tensor) – The bounding boxes of shape [N, num_boxes, 4]
- scores (Tensor) – The detection scores of shape [N, num_boxes, num_classes]
- max_output_boxes_per_class (int) – Maximum number of output boxes per class of nms. Defaults to 1000.
- iou_threshold (float) – IOU threshold of nms. Defaults to 0.5
- score_threshold (float) – score threshold of nms. Defaults to 0.05.
- pre_top_k (bool) – Number of top K boxes to keep before nms. Defaults to -1.
- after_top_k (int) – Number of top K boxes to keep after nms. Defaults to -1.
- labels (Tensor, optional) – It not None, explicit labels would be used. Otherwise, labels would be automatically generated using num_classed. Defaults to None.
Returns: - dets of shape [N, num_det, 5] and class labels
of shape [N, num_det].
Return type: tuple[Tensor, Tensor]
-
mmdet.core.export.
dynamic_clip_for_onnx
(x1, y1, x2, y2, max_shape)[source]¶ Clip boxes dynamically for onnx.
- Since torch.clamp cannot have dynamic min and max, we scale the
- boxes by 1/max_shape and clamp in the range [0, 1].
Parameters: - x1 (Tensor) – The x1 for bounding boxes.
- y1 (Tensor) – The y1 for bounding boxes.
- x2 (Tensor) – The x2 for bounding boxes.
- y2 (Tensor) – The y2 for bounding boxes.
- max_shape (Tensor or torch.Size) – The (H,W) of original image.
Returns: The clipped x1, y1, x2, y2.
Return type: tuple(Tensor)
mask¶
-
mmdet.core.mask.
split_combined_polys
(polys, poly_lens, polys_per_mask)[source]¶ Split the combined 1-D polys into masks.
A mask is represented as a list of polys, and a poly is represented as a 1-D array. In dataset, all masks are concatenated into a single 1-D tensor. Here we need to split the tensor into original representations.
Parameters: - polys (list) – a list (length = image num) of 1-D tensors
- poly_lens (list) – a list (length = image num) of poly length
- polys_per_mask (list) – a list (length = image num) of poly number of each mask
Returns: a list (length = image num) of list (length = mask num) of list (length = poly num) of numpy array.
Return type: list
-
mmdet.core.mask.
mask_target
(pos_proposals_list, pos_assigned_gt_inds_list, gt_masks_list, cfg)[source]¶ Compute mask target for positive proposals in multiple images.
Parameters: - pos_proposals_list (list[Tensor]) – Positive proposals in multiple images.
- pos_assigned_gt_inds_list (list[Tensor]) – Assigned GT indices for each positive proposals.
- gt_masks_list (list[
BaseInstanceMasks
]) – Ground truth masks of each image. - cfg (dict) – Config dict that specifies the mask size.
Returns: Mask target of each image.
Return type: list[Tensor]
Example
>>> import mmcv >>> import mmdet >>> from mmdet.core.mask import BitmapMasks >>> from mmdet.core.mask.mask_target import * >>> H, W = 17, 18 >>> cfg = mmcv.Config({'mask_size': (13, 14)}) >>> rng = np.random.RandomState(0) >>> # Positive proposals (tl_x, tl_y, br_x, br_y) for each image >>> pos_proposals_list = [ >>> torch.Tensor([ >>> [ 7.2425, 5.5929, 13.9414, 14.9541], >>> [ 7.3241, 3.6170, 16.3850, 15.3102], >>> ]), >>> torch.Tensor([ >>> [ 4.8448, 6.4010, 7.0314, 9.7681], >>> [ 5.9790, 2.6989, 7.4416, 4.8580], >>> [ 0.0000, 0.0000, 0.1398, 9.8232], >>> ]), >>> ] >>> # Corresponding class index for each proposal for each image >>> pos_assigned_gt_inds_list = [ >>> torch.LongTensor([7, 0]), >>> torch.LongTensor([5, 4, 1]), >>> ] >>> # Ground truth mask for each true object for each image >>> gt_masks_list = [ >>> BitmapMasks(rng.rand(8, H, W), height=H, width=W), >>> BitmapMasks(rng.rand(6, H, W), height=H, width=W), >>> ] >>> mask_targets = mask_target( >>> pos_proposals_list, pos_assigned_gt_inds_list, >>> gt_masks_list, cfg) >>> assert mask_targets.shape == (5,) + cfg['mask_size']
-
class
mmdet.core.mask.
BaseInstanceMasks
[source]¶ Base class for instance masks.
-
areas
¶ areas of each instance.
Type: ndarray
-
crop
(bbox)[source]¶ Crop each mask by the given bbox.
Parameters: bbox (ndarray) – Bbox in format [x1, y1, x2, y2], shape (4, ). Returns: The cropped masks. Return type: BaseInstanceMasks
-
crop_and_resize
(bboxes, out_shape, inds, device, interpolation='bilinear', binarize=True)[source]¶ Crop and resize masks by the given bboxes.
This function is mainly used in mask targets computation. It firstly align mask to bboxes by assigned_inds, then crop mask by the assigned bbox and resize to the size of (mask_h, mask_w)
Parameters: - bboxes (Tensor) – Bboxes in format [x1, y1, x2, y2], shape (N, 4)
- out_shape (tuple[int]) – Target (h, w) of resized mask
- inds (ndarray) – Indexes to assign masks to each bbox, shape (N,) and values should be between [0, num_masks - 1].
- device (str) – Device of bboxes
- interpolation (str) – See mmcv.imresize
- binarize (bool) – if True fractional values are rounded to 0 or 1 after the resize operation. if False and unsupported an error will be raised. Defaults to True.
Returns: the cropped and resized masks.
Return type:
-
flip
(flip_direction='horizontal')[source]¶ Flip masks alone the given direction.
Parameters: flip_direction (str) – Either ‘horizontal’ or ‘vertical’. Returns: The flipped masks. Return type: BaseInstanceMasks
-
pad
(out_shape, pad_val)[source]¶ Pad masks to the given size of (h, w).
Parameters: - out_shape (tuple[int]) – Target (h, w) of padded mask.
- pad_val (int) – The padded value.
Returns: The padded masks.
Return type:
-
rescale
(scale, interpolation='nearest')[source]¶ Rescale masks as large as possible while keeping the aspect ratio. For details can refer to mmcv.imrescale.
Parameters: - scale (tuple[int]) – The maximum size (h, w) of rescaled mask.
- interpolation (str) – Same as
mmcv.imrescale()
.
Returns: The rescaled masks.
Return type:
-
resize
(out_shape, interpolation='nearest')[source]¶ Resize masks to the given out_shape.
Parameters: - out_shape – Target (h, w) of resized mask.
- interpolation (str) – See
mmcv.imresize()
.
Returns: The resized masks.
Return type:
-
rotate
(out_shape, angle, center=None, scale=1.0, fill_val=0)[source]¶ Rotate the masks.
Parameters: - out_shape (tuple[int]) – Shape for output mask, format (h, w).
- angle (int | float) – Rotation angle in degrees. Positive values mean counter-clockwise rotation.
- center (tuple[float], optional) – Center point (w, h) of the rotation in source image. If not specified, the center of the image will be used.
- scale (int | float) – Isotropic scale factor.
- fill_val (int | float) – Border value. Default 0 for masks.
Returns: Rotated masks.
-
shear
(out_shape, magnitude, direction='horizontal', border_value=0, interpolation='bilinear')[source]¶ Shear the masks.
Parameters: - out_shape (tuple[int]) – Shape for output mask, format (h, w).
- magnitude (int | float) – The magnitude used for shear.
- direction (str) – The shear direction, either “horizontal” or “vertical”.
- border_value (int | tuple[int]) – Value used in case of a constant border. Default 0.
- interpolation (str) – Same as in
mmcv.imshear()
.
Returns: Sheared masks.
Return type: ndarray
-
to_ndarray
()[source]¶ Convert masks to the format of ndarray.
Returns: Converted masks in the format of ndarray. Return type: ndarray
-
to_tensor
(dtype, device)[source]¶ Convert masks to the format of Tensor.
Parameters: - dtype (str) – Dtype of converted mask.
- device (torch.device) – Device of converted masks.
Returns: Converted masks in the format of Tensor.
Return type: Tensor
-
translate
(out_shape, offset, direction='horizontal', fill_val=0, interpolation='bilinear')[source]¶ Translate the masks.
Parameters: - out_shape (tuple[int]) – Shape for output mask, format (h, w).
- offset (int | float) – The offset for translate.
- direction (str) – The translate direction, either “horizontal” or “vertical”.
- fill_val (int | float) – Border value. Default 0.
- interpolation (str) – Same as
mmcv.imtranslate()
.
Returns: Translated masks.
-
-
class
mmdet.core.mask.
BitmapMasks
(masks, height, width)[source]¶ This class represents masks in the form of bitmaps.
Parameters: - masks (ndarray) – ndarray of masks in shape (N, H, W), where N is the number of objects.
- height (int) – height of masks
- width (int) – width of masks
Example
>>> from mmdet.core.mask.structures import * # NOQA >>> num_masks, H, W = 3, 32, 32 >>> rng = np.random.RandomState(0) >>> masks = (rng.rand(num_masks, H, W) > 0.1).astype(np.int) >>> self = BitmapMasks(masks, height=H, width=W)
>>> # demo crop_and_resize >>> num_boxes = 5 >>> bboxes = np.array([[0, 0, 30, 10.0]] * num_boxes) >>> out_shape = (14, 14) >>> inds = torch.randint(0, len(self), size=(num_boxes,)) >>> device = 'cpu' >>> interpolation = 'bilinear' >>> new = self.crop_and_resize( ... bboxes, out_shape, inds, device, interpolation) >>> assert len(new) == num_boxes >>> assert new.height, new.width == out_shape
-
areas
¶
-
crop_and_resize
(bboxes, out_shape, inds, device='cpu', interpolation='bilinear', binarize=True)[source]¶
-
classmethod
random
(num_masks=3, height=32, width=32, dtype=<class 'numpy.uint8'>, rng=None)[source]¶ Generate random bitmap masks for demo / testing purposes.
Example
>>> from mmdet.core.mask.structures import BitmapMasks >>> self = BitmapMasks.random() >>> print('self = {}'.format(self)) self = BitmapMasks(num_masks=3, height=32, width=32)
-
rotate
(out_shape, angle, center=None, scale=1.0, fill_val=0)[source]¶ Rotate the BitmapMasks.
Parameters: - out_shape (tuple[int]) – Shape for output mask, format (h, w).
- angle (int | float) – Rotation angle in degrees. Positive values mean counter-clockwise rotation.
- center (tuple[float], optional) – Center point (w, h) of the rotation in source image. If not specified, the center of the image will be used.
- scale (int | float) – Isotropic scale factor.
- fill_val (int | float) – Border value. Default 0 for masks.
Returns: Rotated BitmapMasks.
Return type:
-
shear
(out_shape, magnitude, direction='horizontal', border_value=0, interpolation='bilinear')[source]¶ Shear the BitmapMasks.
Parameters: - out_shape (tuple[int]) – Shape for output mask, format (h, w).
- magnitude (int | float) – The magnitude used for shear.
- direction (str) – The shear direction, either “horizontal” or “vertical”.
- border_value (int | tuple[int]) – Value used in case of a constant border.
- interpolation (str) – Same as in
mmcv.imshear()
.
Returns: The sheared masks.
Return type:
-
translate
(out_shape, offset, direction='horizontal', fill_val=0, interpolation='bilinear')[source]¶ Translate the BitmapMasks.
Parameters: - out_shape (tuple[int]) – Shape for output mask, format (h, w).
- offset (int | float) – The offset for translate.
- direction (str) – The translate direction, either “horizontal” or “vertical”.
- fill_val (int | float) – Border value. Default 0 for masks.
- interpolation (str) – Same as
mmcv.imtranslate()
.
Returns: Translated BitmapMasks.
Return type: Example
>>> from mmdet.core.mask.structures import BitmapMasks >>> self = BitmapMasks.random(dtype=np.uint8) >>> out_shape = (32, 32) >>> offset = 4 >>> direction = 'horizontal' >>> fill_val = 0 >>> interpolation = 'bilinear' >>> # Note, There seem to be issues when: >>> # * out_shape is different than self's shape >>> # * the mask dtype is not supported by cv2.AffineWarp >>> new = self.translate(out_shape, offset, direction, fill_val, >>> interpolation) >>> assert len(new) == len(self) >>> assert new.height, new.width == out_shape
-
class
mmdet.core.mask.
PolygonMasks
(masks, height, width)[source]¶ This class represents masks in the form of polygons.
Polygons is a list of three levels. The first level of the list corresponds to objects, the second level to the polys that compose the object, the third level to the poly coordinates
Parameters: - masks (list[list[ndarray]]) – The first level of the list corresponds to objects, the second level to the polys that compose the object, the third level to the poly coordinates
- height (int) – height of masks
- width (int) – width of masks
Example
>>> from mmdet.core.mask.structures import * # NOQA >>> masks = [ >>> [ np.array([0, 0, 10, 0, 10, 10., 0, 10, 0, 0]) ] >>> ] >>> height, width = 16, 16 >>> self = PolygonMasks(masks, height, width)
>>> # demo translate >>> new = self.translate((16, 16), 4., direction='horizontal') >>> assert np.all(new.masks[0][0][1::2] == masks[0][0][1::2]) >>> assert np.all(new.masks[0][0][0::2] == masks[0][0][0::2] + 4)
>>> # demo crop_and_resize >>> num_boxes = 3 >>> bboxes = np.array([[0, 0, 30, 10.0]] * num_boxes) >>> out_shape = (16, 16) >>> inds = torch.randint(0, len(self), size=(num_boxes,)) >>> device = 'cpu' >>> interpolation = 'bilinear' >>> new = self.crop_and_resize( ... bboxes, out_shape, inds, device, interpolation) >>> assert len(new) == num_boxes >>> assert new.height, new.width == out_shape
-
areas
¶ Compute areas of masks.
This func is modified from detectron2. The function only works with Polygons using the shoelace formula.
Returns: areas of each instance Return type: ndarray
-
crop_and_resize
(bboxes, out_shape, inds, device='cpu', interpolation='bilinear', binarize=True)[source]¶
-
classmethod
random
(num_masks=3, height=32, width=32, n_verts=5, dtype=<class 'numpy.float32'>, rng=None)[source]¶ Generate random polygon masks for demo / testing purposes.
Adapted from [1]
References
[1] (1, 2) https://gitlab.kitware.com/computer-vision/kwimage/-/blob/928cae35ca8/kwimage/structs/polygon.py#L379 # noqa: E501 Example
>>> from mmdet.core.mask.structures import PolygonMasks >>> self = PolygonMasks.random() >>> print('self = {}'.format(self))
-
shear
(out_shape, magnitude, direction='horizontal', border_value=0, interpolation='bilinear')[source]¶
-
translate
(out_shape, offset, direction='horizontal', fill_val=None, interpolation=None)[source]¶ Translate the PolygonMasks.
Example
>>> self = PolygonMasks.random(dtype=np.int) >>> out_shape = (self.height, self.width) >>> new = self.translate(out_shape, 4., direction='horizontal') >>> assert np.all(new.masks[0][0][1::2] == self.masks[0][0][1::2]) >>> assert np.all(new.masks[0][0][0::2] == self.masks[0][0][0::2] + 4) # noqa: E501
evaluation¶
-
class
mmdet.core.evaluation.
DistEvalHook
(dataloader, start=None, interval=1, by_epoch=True, save_best=None, rule=None, test_fn=None, greater_keys=None, less_keys=None, broadcast_bn_buffer=True, tmpdir=None, gpu_collect=False, **eval_kwargs)[source]¶
-
class
mmdet.core.evaluation.
EvalHook
(dataloader, start=None, interval=1, by_epoch=True, save_best=None, rule=None, test_fn=None, greater_keys=None, less_keys=None, **eval_kwargs)[source]¶
-
mmdet.core.evaluation.
average_precision
(recalls, precisions, mode='area')[source]¶ Calculate average precision (for single or multiple scales).
Parameters: - recalls (ndarray) – shape (num_scales, num_dets) or (num_dets, )
- precisions (ndarray) – shape (num_scales, num_dets) or (num_dets, )
- mode (str) – ‘area’ or ‘11points’, ‘area’ means calculating the area under precision-recall curve, ‘11points’ means calculating the average precision of recalls at [0, 0.1, …, 1]
Returns: calculated average precision
Return type: float or ndarray
-
mmdet.core.evaluation.
eval_map
(det_results, annotations, scale_ranges=None, iou_thr=0.5, dataset=None, logger=None, tpfp_fn=None, nproc=4)[source]¶ Evaluate mAP of a dataset.
Parameters: - det_results (list[list]) – [[cls1_det, cls2_det, …], …]. The outer list indicates images, and the inner list indicates per-class detected bboxes.
- annotations (list[dict]) –
Ground truth annotations where each item of the list indicates an image. Keys of annotations are:
- bboxes: numpy array of shape (n, 4)
- labels: numpy array of shape (n, )
- bboxes_ignore (optional): numpy array of shape (k, 4)
- labels_ignore (optional): numpy array of shape (k, )
- scale_ranges (list[tuple] | None) – Range of scales to be evaluated, in the format [(min1, max1), (min2, max2), …]. A range of (32, 64) means the area range between (32**2, 64**2). Default: None.
- iou_thr (float) – IoU threshold to be considered as matched. Default: 0.5.
- dataset (list[str] | str | None) – Dataset name or dataset classes, there are minor differences in metrics for different datsets, e.g. “voc07”, “imagenet_det”, etc. Default: None.
- logger (logging.Logger | str | None) – The way to print the mAP summary. See mmcv.utils.print_log() for details. Default: None.
- tpfp_fn (callable | None) – The function used to determine true/
false positives. If None,
tpfp_default()
is used as default unless dataset is ‘det’ or ‘vid’ (tpfp_imagenet()
in this case). If it is given as a function, then this function is used to evaluate tp & fp. Default None. - nproc (int) – Processes used for computing TP and FP. Default: 4.
Returns: (mAP, [dict, dict, …])
Return type: tuple
-
mmdet.core.evaluation.
print_map_summary
(mean_ap, results, dataset=None, scale_ranges=None, logger=None)[source]¶ Print mAP and results of each class.
A table will be printed to show the gts/dets/recall/AP of each class and the mAP.
Parameters: - mean_ap (float) – Calculated from eval_map().
- results (list[dict]) – Calculated from eval_map().
- dataset (list[str] | str | None) – Dataset name or dataset classes.
- scale_ranges (list[tuple] | None) – Range of scales to be evaluated.
- logger (logging.Logger | str | None) – The way to print the mAP summary. See mmcv.utils.print_log() for details. Default: None.
-
mmdet.core.evaluation.
eval_recalls
(gts, proposals, proposal_nums=None, iou_thrs=0.5, logger=None)[source]¶ Calculate recalls.
Parameters: - gts (list[ndarray]) – a list of arrays of shape (n, 4)
- proposals (list[ndarray]) – a list of arrays of shape (k, 4) or (k, 5)
- proposal_nums (int | Sequence[int]) – Top N proposals to be evaluated.
- iou_thrs (float | Sequence[float]) – IoU thresholds. Default: 0.5.
- logger (logging.Logger | str | None) – The way to print the recall summary. See mmcv.utils.print_log() for details. Default: None.
Returns: recalls of different ious and proposal nums
Return type: ndarray
-
mmdet.core.evaluation.
print_recall_summary
(recalls, proposal_nums, iou_thrs, row_idxs=None, col_idxs=None, logger=None)[source]¶ Print recalls in a table.
Parameters: - recalls (ndarray) – calculated from bbox_recalls
- proposal_nums (ndarray or list) – top N proposals
- iou_thrs (ndarray or list) – iou thresholds
- row_idxs (ndarray) – which rows(proposal nums) to print
- col_idxs (ndarray) – which cols(iou thresholds) to print
- logger (logging.Logger | str | None) – The way to print the recall summary. See mmcv.utils.print_log() for details. Default: None.
post_processing¶
-
mmdet.core.post_processing.
multiclass_nms
(multi_bboxes, multi_scores, score_thr, nms_cfg, max_num=-1, score_factors=None, return_inds=False)[source]¶ NMS for multi-class bboxes.
Parameters: - multi_bboxes (Tensor) – shape (n, #class*4) or (n, 4)
- multi_scores (Tensor) – shape (n, #class), where the last column contains scores of the background class, but this will be ignored.
- score_thr (float) – bbox threshold, bboxes with scores lower than it will not be considered.
- nms_thr (float) – NMS IoU threshold
- max_num (int, optional) – if there are more than max_num bboxes after NMS, only top max_num will be kept. Default to -1.
- score_factors (Tensor, optional) – The factors multiplied to scores before applying NMS. Default to None.
- return_inds (bool, optional) – Whether return the indices of kept bboxes. Default to False.
Returns: - (dets, labels, indices (optional)), tensors of shape (k, 5),
(k), and (k). Dets are boxes with scores. Labels are 0-based.
Return type: tuple
-
mmdet.core.post_processing.
merge_aug_proposals
(aug_proposals, img_metas, cfg)[source]¶ Merge augmented proposals (multiscale, flip, etc.)
Parameters: - aug_proposals (list[Tensor]) – proposals from different testing schemes, shape (n, 5). Note that they are not rescaled to the original image size.
- img_metas (list[dict]) – list of image info dict where each dict has: ‘img_shape’, ‘scale_factor’, ‘flip’, and may also contain ‘filename’, ‘ori_shape’, ‘pad_shape’, and ‘img_norm_cfg’. For details on the values of these keys see mmdet/datasets/pipelines/formatting.py:Collect.
- cfg (dict) – rpn test config.
Returns: shape (n, 4), proposals corresponding to original image scale.
Return type: Tensor
-
mmdet.core.post_processing.
merge_aug_bboxes
(aug_bboxes, aug_scores, img_metas, rcnn_test_cfg)[source]¶ Merge augmented detection bboxes and scores.
Parameters: - aug_bboxes (list[Tensor]) – shape (n, 4*#class)
- aug_scores (list[Tensor] or None) – shape (n, #class)
- img_shapes (list[Tensor]) – shape (3, ).
- rcnn_test_cfg (dict) – rcnn test config.
Returns: (bboxes, scores)
Return type: tuple
-
mmdet.core.post_processing.
merge_aug_masks
(aug_masks, img_metas, rcnn_test_cfg, weights=None)[source]¶ Merge augmented mask prediction.
Parameters: - aug_masks (list[ndarray]) – shape (n, #class, h, w)
- img_shapes (list[ndarray]) – shape (3, ).
- rcnn_test_cfg (dict) – rcnn test config.
Returns: (bboxes, scores)
Return type: tuple
-
mmdet.core.post_processing.
fast_nms
(multi_bboxes, multi_scores, multi_coeffs, score_thr, iou_thr, top_k, max_num=-1)[source]¶ Fast NMS in YOLACT.
Fast NMS allows already-removed detections to suppress other detections so that every instance can be decided to be kept or discarded in parallel, which is not possible in traditional NMS. This relaxation allows us to implement Fast NMS entirely in standard GPU-accelerated matrix operations.
Parameters: - multi_bboxes (Tensor) – shape (n, #class*4) or (n, 4)
- multi_scores (Tensor) – shape (n, #class+1), where the last column contains scores of the background class, but this will be ignored.
- multi_coeffs (Tensor) – shape (n, #class*coeffs_dim).
- score_thr (float) – bbox threshold, bboxes with scores lower than it will not be considered.
- iou_thr (float) – IoU threshold to be considered as conflicted.
- top_k (int) – if there are more than top_k bboxes before NMS, only top top_k will be kept.
- max_num (int) – if there are more than max_num bboxes after NMS, only top max_num will be kept. If -1, keep all the bboxes. Default: -1.
Returns: - (dets, labels, coefficients), tensors of shape (k, 5), (k, 1),
and (k, coeffs_dim). Dets are boxes with scores. Labels are 0-based.
Return type: tuple
utils¶
-
mmdet.core.utils.
allreduce_grads
(params, coalesce=True, bucket_size_mb=-1)[source]¶ Allreduce gradients.
Parameters: - params (list[torch.Parameters]) – List of parameters of a model
- coalesce (bool, optional) – Whether allreduce parameters as a whole. Defaults to True.
- bucket_size_mb (int, optional) – Size of bucket, the unit is MB. Defaults to -1.
-
class
mmdet.core.utils.
DistOptimizerHook
(*args, **kwargs)[source]¶ Deprecated optimizer hook for distributed training.
-
mmdet.core.utils.
multi_apply
(func, *args, **kwargs)[source]¶ Apply function to a list of arguments.
Note
This function applies the
func
to multiple inputs and map the multiple outputs of thefunc
into different list. Each list contains the same type of outputs corresponding to different inputs.Parameters: func (Function) – A function that will be applied to a list of arguments Returns: A tuple containing multiple list, each list contains a kind of returned results by the function Return type: tuple(list)
-
mmdet.core.utils.
unmap
(data, count, inds, fill=0)[source]¶ Unmap a subset of item (data) back to the original set of items (of size count)
-
mmdet.core.utils.
mask2ndarray
(mask)[source]¶ Convert Mask to ndarray..
:param mask (
BitmapMasks
orPolygonMasks
or: :param torch.Tensor or np.ndarray): The mask to be converted.Returns: Ndarray mask of shape (n, h, w) that has been converted Return type: np.ndarray
-
mmdet.core.utils.
flip_tensor
(src_tensor, flip_direction)[source]¶ flip tensor base on flip_direction.
Parameters: - src_tensor (Tensor) – input feature map, shape (B, C, H, W).
- flip_direction (str) – The flipping direction. Options are ‘horizontal’, ‘vertical’, ‘diagonal’.
Returns: Flipped tensor.
Return type: out_tensor (Tensor)
-
mmdet.core.utils.
all_reduce_dict
(py_dict, op='sum', group=None, to_float=True)[source]¶ Apply all reduce function for python dict object.
The code is modified from https://github.com/Megvii- BaseDetection/YOLOX/blob/main/yolox/utils/allreduce_norm.py.
NOTE: make sure that py_dict in different ranks has the same keys and the values should be in the same shape.
Parameters: - py_dict (dict) – Dict to be applied all reduce op.
- op (str) – Operator, could be ‘sum’ or ‘mean’. Default: ‘sum’
- group (
torch.distributed.group
, optional) – Distributed group, Default: None. - to_float (bool) – Whether to convert all values of dict to float. Default: True.
Returns: reduced python dict object.
Return type: OrderedDict
mmdet.datasets¶
datasets¶
-
class
mmdet.datasets.
CustomDataset
(ann_file, pipeline, classes=None, data_root=None, img_prefix='', seg_prefix=None, proposal_file=None, test_mode=False, filter_empty_gt=True)[source]¶ Custom dataset for detection.
The annotation format is shown as follows. The ann field is optional for testing.
[ { 'filename': 'a.jpg', 'width': 1280, 'height': 720, 'ann': { 'bboxes': <np.ndarray> (n, 4) in (x1, y1, x2, y2) order. 'labels': <np.ndarray> (n, ), 'bboxes_ignore': <np.ndarray> (k, 4), (optional field) 'labels_ignore': <np.ndarray> (k, 4) (optional field) } }, ... ]
Parameters: - ann_file (str) – Annotation file path.
- pipeline (list[dict]) – Processing pipeline.
- classes (str | Sequence[str], optional) – Specify classes to load.
If is None,
cls.CLASSES
will be used. Default: None. - data_root (str, optional) – Data root for
ann_file
,img_prefix
,seg_prefix
,proposal_file
if specified. - test_mode (bool, optional) – If set True, annotation will not be loaded.
- filter_empty_gt (bool, optional) – If set true, images without bounding boxes of the dataset’s classes will be filtered out. This option only works when test_mode=False, i.e., we never filter images during tests.
-
evaluate
(results, metric='mAP', logger=None, proposal_nums=(100, 300, 1000), iou_thr=0.5, scale_ranges=None)[source]¶ Evaluate the dataset.
Parameters: - results (list) – Testing results of the dataset.
- metric (str | list[str]) – Metrics to be evaluated.
- logger (logging.Logger | None | str) – Logger used for printing related information during evaluation. Default: None.
- proposal_nums (Sequence[int]) – Proposal number used for evaluating recalls, such as recall@100, recall@1000. Default: (100, 300, 1000).
- iou_thr (float | list[float]) – IoU threshold. Default: 0.5.
- scale_ranges (list[tuple] | None) – Scale ranges for evaluating mAP. Default: None.
-
format_results
(results, **kwargs)[source]¶ Place holder to format result to dataset specific output.
-
get_ann_info
(idx)[source]¶ Get annotation by index.
Parameters: idx (int) – Index of data. Returns: Annotation info of specified index. Return type: dict
-
get_cat_ids
(idx)[source]¶ Get category ids by index.
Parameters: idx (int) – Index of data. Returns: All categories in the image of specified index. Return type: list[int]
-
classmethod
get_classes
(classes=None)[source]¶ Get class names of current dataset.
Parameters: classes (Sequence[str] | str | None) – If classes is None, use default CLASSES defined by builtin dataset. If classes is a string, take it as a file name. The file contains the name of classes where each line contains one class name. If classes is a tuple or list, override the CLASSES defined by the dataset. Returns: Names of categories of the dataset. Return type: tuple[str] or list[str]
-
class
mmdet.datasets.
XMLDataset
(min_size=None, **kwargs)[source]¶ XML dataset for detection.
Parameters: min_size (int | float, optional) – The minimum size of bounding boxes in the images. If the size of a bounding box is less than min_size
, it would be add to ignored field.-
get_ann_info
(idx)[source]¶ Get annotation from XML file by index.
Parameters: idx (int) – Index of data. Returns: Annotation info of specified index. Return type: dict
-
-
class
mmdet.datasets.
CocoDataset
(ann_file, pipeline, classes=None, data_root=None, img_prefix='', seg_prefix=None, proposal_file=None, test_mode=False, filter_empty_gt=True)[source]¶ -
evaluate
(results, metric='bbox', logger=None, jsonfile_prefix=None, classwise=False, proposal_nums=(100, 300, 1000), iou_thrs=None, metric_items=None)[source]¶ Evaluation in COCO protocol.
Parameters: - results (list[list | tuple]) – Testing results of the dataset.
- metric (str | list[str]) – Metrics to be evaluated. Options are ‘bbox’, ‘segm’, ‘proposal’, ‘proposal_fast’.
- logger (logging.Logger | str | None) – Logger used for printing related information during evaluation. Default: None.
- jsonfile_prefix (str | None) – The prefix of json files. It includes the file path and the prefix of filename, e.g., “a/b/prefix”. If not specified, a temp file will be created. Default: None.
- classwise (bool) – Whether to evaluating the AP for each class.
- proposal_nums (Sequence[int]) – Proposal number used for evaluating recalls, such as recall@100, recall@1000. Default: (100, 300, 1000).
- iou_thrs (Sequence[float], optional) – IoU threshold used for evaluating recalls/mAPs. If set to a list, the average of all IoUs will also be computed. If not specified, [0.50, 0.55, 0.60, 0.65, 0.70, 0.75, 0.80, 0.85, 0.90, 0.95] will be used. Default: None.
- metric_items (list[str] | str, optional) – Metric items that will
be returned. If not specified,
['AR@100', 'AR@300', 'AR@1000', 'AR_s@1000', 'AR_m@1000', 'AR_l@1000' ]
will be used whenmetric=='proposal'
,['mAP', 'mAP_50', 'mAP_75', 'mAP_s', 'mAP_m', 'mAP_l']
will be used whenmetric=='bbox' or metric=='segm'
.
Returns: COCO style evaluation metric.
Return type: dict[str, float]
-
format_results
(results, jsonfile_prefix=None, **kwargs)[source]¶ Format the results to json (standard format for COCO evaluation).
Parameters: - results (list[tuple | numpy.ndarray]) – Testing results of the dataset.
- jsonfile_prefix (str | None) – The prefix of json files. It includes the file path and the prefix of filename, e.g., “a/b/prefix”. If not specified, a temp file will be created. Default: None.
Returns: (result_files, tmp_dir), result_files is a dict containing the json filepaths, tmp_dir is the temporal directory created for saving json files when jsonfile_prefix is not specified.
Return type: tuple
-
get_ann_info
(idx)[source]¶ Get COCO annotation by index.
Parameters: idx (int) – Index of data. Returns: Annotation info of specified index. Return type: dict
-
get_cat_ids
(idx)[source]¶ Get COCO category ids by index.
Parameters: idx (int) – Index of data. Returns: All categories in the image of specified index. Return type: list[int]
-
load_annotations
(ann_file)[source]¶ Load annotation from COCO style annotation file.
Parameters: ann_file (str) – Path of annotation file. Returns: Annotation info from COCO api. Return type: list[dict]
-
results2json
(results, outfile_prefix)[source]¶ Dump the detection results to a COCO style json file.
There are 3 types of results: proposals, bbox predictions, mask predictions, and they have different data types. This method will automatically recognize the type, and dump them to json files.
Parameters: - results (list[list | tuple | ndarray]) – Testing results of the dataset.
- outfile_prefix (str) – The filename prefix of the json files. If the prefix is “somepath/xxx”, the json files will be named “somepath/xxx.bbox.json”, “somepath/xxx.segm.json”, “somepath/xxx.proposal.json”.
Returns: str]: Possible keys are “bbox”, “segm”, “proposal”, and values are corresponding filenames.
Return type: dict[str
-
-
class
mmdet.datasets.
DeepFashionDataset
(ann_file, pipeline, classes=None, data_root=None, img_prefix='', seg_prefix=None, proposal_file=None, test_mode=False, filter_empty_gt=True)[source]¶
-
class
mmdet.datasets.
VOCDataset
(**kwargs)[source]¶ -
evaluate
(results, metric='mAP', logger=None, proposal_nums=(100, 300, 1000), iou_thr=0.5, scale_ranges=None)[source]¶ Evaluate in VOC protocol.
Parameters: - results (list[list | tuple]) – Testing results of the dataset.
- metric (str | list[str]) – Metrics to be evaluated. Options are ‘mAP’, ‘recall’.
- logger (logging.Logger | str, optional) – Logger used for printing related information during evaluation. Default: None.
- proposal_nums (Sequence[int]) – Proposal number used for evaluating recalls, such as recall@100, recall@1000. Default: (100, 300, 1000).
- iou_thr (float | list[float]) – IoU threshold. Default: 0.5.
- scale_ranges (list[tuple], optional) – Scale ranges for evaluating mAP. If not specified, all bounding boxes would be included in evaluation. Default: None.
Returns: AP/recall metrics.
Return type: dict[str, float]
-
-
class
mmdet.datasets.
CityscapesDataset
(ann_file, pipeline, classes=None, data_root=None, img_prefix='', seg_prefix=None, proposal_file=None, test_mode=False, filter_empty_gt=True)[source]¶ -
evaluate
(results, metric='bbox', logger=None, outfile_prefix=None, classwise=False, proposal_nums=(100, 300, 1000), iou_thrs=array([0.5, 0.55, 0.6, 0.65, 0.7, 0.75, 0.8, 0.85, 0.9, 0.95]))[source]¶ Evaluation in Cityscapes/COCO protocol.
Parameters: - results (list[list | tuple]) – Testing results of the dataset.
- metric (str | list[str]) – Metrics to be evaluated. Options are ‘bbox’, ‘segm’, ‘proposal’, ‘proposal_fast’.
- logger (logging.Logger | str | None) – Logger used for printing related information during evaluation. Default: None.
- outfile_prefix (str | None) – The prefix of output file. It includes the file path and the prefix of filename, e.g., “a/b/prefix”. If results are evaluated with COCO protocol, it would be the prefix of output json file. For example, the metric is ‘bbox’ and ‘segm’, then json files would be “a/b/prefix.bbox.json” and “a/b/prefix.segm.json”. If results are evaluated with cityscapes protocol, it would be the prefix of output txt/png files. The output files would be png images under folder “a/b/prefix/xxx/” and the file name of images would be written into a txt file “a/b/prefix/xxx_pred.txt”, where “xxx” is the video name of cityscapes. If not specified, a temp file will be created. Default: None.
- classwise (bool) – Whether to evaluating the AP for each class.
- proposal_nums (Sequence[int]) – Proposal number used for evaluating recalls, such as recall@100, recall@1000. Default: (100, 300, 1000).
- iou_thrs (Sequence[float]) – IoU threshold used for evaluating recalls. If set to a list, the average recall of all IoUs will also be computed. Default: 0.5.
Returns: COCO style evaluation metric or cityscapes mAP and AP@50.
Return type: dict[str, float]
-
format_results
(results, txtfile_prefix=None)[source]¶ Format the results to txt (standard format for Cityscapes evaluation).
Parameters: - results (list) – Testing results of the dataset.
- txtfile_prefix (str | None) – The prefix of txt files. It includes the file path and the prefix of filename, e.g., “a/b/prefix”. If not specified, a temp file will be created. Default: None.
Returns: (result_files, tmp_dir), result_files is a dict containing the json filepaths, tmp_dir is the temporal directory created for saving txt/png files when txtfile_prefix is not specified.
Return type: tuple
-
results2txt
(results, outfile_prefix)[source]¶ Dump the detection results to a txt file.
Parameters: - results (list[list | tuple]) – Testing results of the dataset.
- outfile_prefix (str) – The filename prefix of the json files. If the prefix is “somepath/xxx”, the txt files will be named “somepath/xxx.txt”.
Returns: Result txt files which contains corresponding instance segmentation images.
Return type: list[str]
-
-
mmdet.datasets.
LVISDataset
¶ alias of
mmdet.datasets.lvis.LVISV05Dataset
-
class
mmdet.datasets.
LVISV05Dataset
(ann_file, pipeline, classes=None, data_root=None, img_prefix='', seg_prefix=None, proposal_file=None, test_mode=False, filter_empty_gt=True)[source]¶ -
evaluate
(results, metric='bbox', logger=None, jsonfile_prefix=None, classwise=False, proposal_nums=(100, 300, 1000), iou_thrs=array([0.5, 0.55, 0.6, 0.65, 0.7, 0.75, 0.8, 0.85, 0.9, 0.95]))[source]¶ Evaluation in LVIS protocol.
Parameters: - results (list[list | tuple]) – Testing results of the dataset.
- metric (str | list[str]) – Metrics to be evaluated. Options are ‘bbox’, ‘segm’, ‘proposal’, ‘proposal_fast’.
- logger (logging.Logger | str | None) – Logger used for printing related information during evaluation. Default: None.
- jsonfile_prefix (str | None) –
- classwise (bool) – Whether to evaluating the AP for each class.
- proposal_nums (Sequence[int]) – Proposal number used for evaluating recalls, such as recall@100, recall@1000. Default: (100, 300, 1000).
- iou_thrs (Sequence[float]) – IoU threshold used for evaluating recalls. If set to a list, the average recall of all IoUs will also be computed. Default: 0.5.
Returns: LVIS style metrics.
Return type: dict[str, float]
-
-
class
mmdet.datasets.
LVISV1Dataset
(ann_file, pipeline, classes=None, data_root=None, img_prefix='', seg_prefix=None, proposal_file=None, test_mode=False, filter_empty_gt=True)[source]¶
-
class
mmdet.datasets.
DistributedGroupSampler
(dataset, samples_per_gpu=1, num_replicas=None, rank=None, seed=0)[source]¶ Sampler that restricts data loading to a subset of the dataset.
It is especially useful in conjunction with
torch.nn.parallel.DistributedDataParallel
. In such case, each process can pass a DistributedSampler instance as a DataLoader sampler, and load a subset of the original dataset that is exclusive to it.Note
Dataset is assumed to be of constant size.
Parameters: - dataset – Dataset used for sampling.
- num_replicas (optional) – Number of processes participating in distributed training.
- rank (optional) – Rank of the current process within num_replicas.
- seed (int, optional) – random seed used to shuffle the sampler if
shuffle=True
. This number should be identical across all processes in the distributed group. Default: 0.
-
class
mmdet.datasets.
DistributedSampler
(dataset, num_replicas=None, rank=None, shuffle=True, seed=0)[source]¶
-
mmdet.datasets.
build_dataloader
(dataset, samples_per_gpu, workers_per_gpu, num_gpus=1, dist=True, shuffle=True, seed=None, **kwargs)[source]¶ Build PyTorch DataLoader.
In distributed training, each GPU/process has a dataloader. In non-distributed training, there is only one dataloader for all GPUs.
Parameters: - dataset (Dataset) – A PyTorch dataset.
- samples_per_gpu (int) – Number of training samples on each GPU, i.e., batch size of each GPU.
- workers_per_gpu (int) – How many subprocesses to use for data loading for each GPU.
- num_gpus (int) – Number of GPUs. Only used in non-distributed training.
- dist (bool) – Distributed training/test or not. Default: True.
- shuffle (bool) – Whether to shuffle the data at every epoch. Default: True.
- kwargs – any keyword argument to be used to initialize DataLoader
Returns: A PyTorch dataloader.
Return type: DataLoader
-
class
mmdet.datasets.
ConcatDataset
(datasets, separate_eval=True)[source]¶ A wrapper of concatenated dataset.
Same as
torch.utils.data.dataset.ConcatDataset
, but concat the group flag for image aspect ratio.Parameters: - datasets (list[
Dataset
]) – A list of datasets. - separate_eval (bool) – Whether to evaluate the results separately if it is used as validation dataset. Defaults to True.
-
evaluate
(results, logger=None, **kwargs)[source]¶ Evaluate the results.
Parameters: - results (list[list | tuple]) – Testing results of the dataset.
- logger (logging.Logger | str | None) – Logger used for printing related information during evaluation. Default: None.
Returns: float]: AP results of the total dataset or each separate dataset if self.separate_eval=True.
Return type: dict[str
- datasets (list[
-
class
mmdet.datasets.
RepeatDataset
(dataset, times)[source]¶ A wrapper of repeated dataset.
The length of repeated dataset will be times larger than the original dataset. This is useful when the data loading time is long but the dataset is small. Using RepeatDataset can reduce the data loading time between epochs.
Parameters: - dataset (
Dataset
) – The dataset to be repeated. - times (int) – Repeat times.
- dataset (
-
class
mmdet.datasets.
ClassBalancedDataset
(dataset, oversample_thr, filter_empty_gt=True)[source]¶ A wrapper of repeated dataset with repeat factor.
Suitable for training on class imbalanced datasets like LVIS. Following the sampling strategy in the paper, in each epoch, an image may appear multiple times based on its “repeat factor”. The repeat factor for an image is a function of the frequency the rarest category labeled in that image. The “frequency of category c” in [0, 1] is defined by the fraction of images in the training set (without repeats) in which category c appears. The dataset needs to instantiate
self.get_cat_ids()
to support ClassBalancedDataset.The repeat factor is computed as followed.
- For each category c, compute the fraction # of images that contain it: \(f(c)\)
- For each category c, compute the category-level repeat factor: \(r(c) = max(1, sqrt(t/f(c)))\)
- For each image I, compute the image-level repeat factor: \(r(I) = max_{c in I} r(c)\)
Parameters: - dataset (
CustomDataset
) – The dataset to be repeated. - oversample_thr (float) – frequency threshold below which data is
repeated. For categories with
f_c >= oversample_thr
, there is no oversampling. For categories withf_c < oversample_thr
, the degree of oversampling following the square-root inverse frequency heuristic above. - filter_empty_gt (bool, optional) – If set true, images without bounding boxes will not be oversampled. Otherwise, they will be categorized as the pure background class and involved into the oversampling. Default: True.
-
class
mmdet.datasets.
WIDERFaceDataset
(**kwargs)[source]¶ Reader for the WIDER Face dataset in PASCAL VOC format.
Conversion scripts can be found in https://github.com/sovrasov/wider-face-pascal-voc-annotations
-
mmdet.datasets.
replace_ImageToTensor
(pipelines)[source]¶ Replace the ImageToTensor transform in a data pipeline to DefaultFormatBundle, which is normally useful in batch inference.
Parameters: pipelines (list[dict]) – Data pipeline configs. Returns: - The new pipeline list with all ImageToTensor replaced by
- DefaultFormatBundle.
Return type: list Examples
>>> pipelines = [ ... dict(type='LoadImageFromFile'), ... dict( ... type='MultiScaleFlipAug', ... img_scale=(1333, 800), ... flip=False, ... transforms=[ ... dict(type='Resize', keep_ratio=True), ... dict(type='RandomFlip'), ... dict(type='Normalize', mean=[0, 0, 0], std=[1, 1, 1]), ... dict(type='Pad', size_divisor=32), ... dict(type='ImageToTensor', keys=['img']), ... dict(type='Collect', keys=['img']), ... ]) ... ] >>> expected_pipelines = [ ... dict(type='LoadImageFromFile'), ... dict( ... type='MultiScaleFlipAug', ... img_scale=(1333, 800), ... flip=False, ... transforms=[ ... dict(type='Resize', keep_ratio=True), ... dict(type='RandomFlip'), ... dict(type='Normalize', mean=[0, 0, 0], std=[1, 1, 1]), ... dict(type='Pad', size_divisor=32), ... dict(type='DefaultFormatBundle'), ... dict(type='Collect', keys=['img']), ... ]) ... ] >>> assert expected_pipelines == replace_ImageToTensor(pipelines)
-
mmdet.datasets.
get_loading_pipeline
(pipeline)[source]¶ Only keep loading image and annotations related configuration.
Parameters: pipeline (list[dict]) – Data pipeline configs. Returns: - The new pipeline list with only keep
- loading image and annotations related configuration.
Return type: list[dict] Examples
>>> pipelines = [ ... dict(type='LoadImageFromFile'), ... dict(type='LoadAnnotations', with_bbox=True), ... dict(type='Resize', img_scale=(1333, 800), keep_ratio=True), ... dict(type='RandomFlip', flip_ratio=0.5), ... dict(type='Normalize', **img_norm_cfg), ... dict(type='Pad', size_divisor=32), ... dict(type='DefaultFormatBundle'), ... dict(type='Collect', keys=['img', 'gt_bboxes', 'gt_labels']) ... ] >>> expected_pipelines = [ ... dict(type='LoadImageFromFile'), ... dict(type='LoadAnnotations', with_bbox=True) ... ] >>> assert expected_pipelines == ... get_loading_pipeline(pipelines)
-
class
mmdet.datasets.
CocoPanopticDataset
(ann_file, pipeline, classes=None, data_root=None, img_prefix='', seg_prefix=None, proposal_file=None, test_mode=False, filter_empty_gt=True)[source]¶ Coco dataset for Panoptic segmentation.
The annotation format is shown as follows. The ann field is optional for testing.
[ { 'filename': f'{image_id:012}.png', 'image_id':9 'segments_info': { [ { 'id': 8345037, (segment_id in panoptic png, convert from rgb) 'category_id': 51, 'iscrowd': 0, 'bbox': (x1, y1, w, h), 'area': 24315, 'segmentation': list,(coded mask) }, ... } } }, ... ]
-
evaluate
(results, metric='pq', logger=None, jsonfile_prefix=None, **kwargs)[source]¶ Evaluation in COCO Panoptic protocol.
Parameters: - results (list[dict]) – Testing results of the dataset.
- metric (str | list[str]) – Metrics to be evaluated. Only support ‘pq’ at present.
- logger (logging.Logger | str | None) – Logger used for printing related information during evaluation. Default: None.
- jsonfile_prefix (str | None) – The prefix of json files. It includes the file path and the prefix of filename, e.g., “a/b/prefix”. If not specified, a temp file will be created. Default: None.
Returns: COCO Panoptic style evaluation metric.
Return type: dict[str, float]
-
evaluate_pan_json
(result_files, outfile_prefix, logger=None)[source]¶ Evaluate PQ according to the panoptic results json file.
-
get_ann_info
(idx)[source]¶ Get COCO annotation by index.
Parameters: idx (int) – Index of data. Returns: Annotation info of specified index. Return type: dict
-
load_annotations
(ann_file)[source]¶ Load annotation from COCO Panoptic style annotation file.
Parameters: ann_file (str) – Path of annotation file. Returns: Annotation info from COCO api. Return type: list[dict]
-
results2json
(results, outfile_prefix)[source]¶ Dump the panoptic results to a COCO panoptic style json file.
Parameters: - results (dict) – Testing results of the dataset.
- outfile_prefix (str) – The filename prefix of the json files. If the prefix is “somepath/xxx”, the json files will be named “somepath/xxx.panoptic.json”
Returns: - str]: The key is ‘panoptic’ and the value is
corresponding filename.
Return type: dict[str
-
-
class
mmdet.datasets.
MultiImageMixDataset
(dataset, pipeline, dynamic_scale=None, skip_type_keys=None)[source]¶ A wrapper of multiple images mixed dataset.
Suitable for training on multiple images mixed data augmentation like mosaic and mixup. For the augmentation pipeline of mixed image data, the get_indexes method needs to be provided to obtain the image indexes, and you can set skip_flags to change the pipeline running process. At the same time, we provide the dynamic_scale parameter to dynamically change the output image size.
Parameters: - dataset (
CustomDataset
) – The dataset to be mixed. - pipeline (Sequence[dict]) – Sequence of transform object or config dict to be composed.
- dynamic_scale (tuple[int], optional) – The image scale can be changed dynamically. Default to None.
- skip_type_keys (list[str], optional) – Sequence of type string to be skip pipeline. Default to None.
- dataset (
pipelines¶
-
class
mmdet.datasets.pipelines.
Compose
(transforms)[source]¶ Compose multiple transforms sequentially.
Parameters: transforms (Sequence[dict | callable]) – Sequence of transform object or config dict to be composed.
-
mmdet.datasets.pipelines.
to_tensor
(data)[source]¶ Convert objects of various python types to
torch.Tensor
.Supported types are:
numpy.ndarray
,torch.Tensor
,Sequence
,int
andfloat
.Parameters: data (torch.Tensor | numpy.ndarray | Sequence | int | float) – Data to be converted.
-
class
mmdet.datasets.pipelines.
ToTensor
(keys)[source]¶ Convert some results to
torch.Tensor
by given keys.Parameters: keys (Sequence[str]) – Keys that need to be converted to Tensor.
-
class
mmdet.datasets.pipelines.
ImageToTensor
(keys)[source]¶ Convert image to
torch.Tensor
by given keys.The dimension order of input image is (H, W, C). The pipeline will convert it to (C, H, W). If only 2 dimension (H, W) is given, the output would be (1, H, W).
Parameters: keys (Sequence[str]) – Key of images to be converted to Tensor.
-
class
mmdet.datasets.pipelines.
ToDataContainer
(fields=({'key': 'img', 'stack': True}, {'key': 'gt_bboxes'}, {'key': 'gt_labels'}))[source]¶ Convert results to
mmcv.DataContainer
by given fields.Parameters: fields (Sequence[dict]) – Each field is a dict like dict(key='xxx', **kwargs)
. Thekey
in result will be converted tommcv.DataContainer
with**kwargs
. Default:(dict(key='img', stack=True), dict(key='gt_bboxes'), dict(key='gt_labels'))
.
-
class
mmdet.datasets.pipelines.
Transpose
(keys, order)[source]¶ Transpose some results by given keys.
Parameters: - keys (Sequence[str]) – Keys of results to be transposed.
- order (Sequence[int]) – Order of transpose.
-
class
mmdet.datasets.pipelines.
Collect
(keys, meta_keys=('filename', 'ori_filename', 'ori_shape', 'img_shape', 'pad_shape', 'scale_factor', 'flip', 'flip_direction', 'img_norm_cfg'))[source]¶ Collect data from the loader relevant to the specific task.
This is usually the last stage of the data loader pipeline. Typically keys is set to some subset of “img”, “proposals”, “gt_bboxes”, “gt_bboxes_ignore”, “gt_labels”, and/or “gt_masks”.
The “img_meta” item is always populated. The contents of the “img_meta” dictionary depends on “meta_keys”. By default this includes:
“img_shape”: shape of the image input to the network as a tuple (h, w, c). Note that images may be zero padded on the bottom/right if the batch tensor is larger than this shape.
“scale_factor”: a float indicating the preprocessing scale
“flip”: a boolean indicating if image flip transform was used
“filename”: path to the image file
“ori_shape”: original shape of the image as a tuple (h, w, c)
“pad_shape”: image shape after padding
“img_norm_cfg”: a dict of normalization information:
- mean - per channel mean subtraction
- std - per channel std divisor
- to_rgb - bool indicating if bgr was converted to rgb
Parameters: - keys (Sequence[str]) – Keys of results to be collected in
data
. - meta_keys (Sequence[str], optional) – Meta keys to be converted to
mmcv.DataContainer
and collected indata[img_metas]
. Default:('filename', 'ori_filename', 'ori_shape', 'img_shape', 'pad_shape', 'scale_factor', 'flip', 'flip_direction', 'img_norm_cfg')
-
class
mmdet.datasets.pipelines.
DefaultFormatBundle
[source]¶ Default formatting bundle.
It simplifies the pipeline of formatting common fields, including “img”, “proposals”, “gt_bboxes”, “gt_labels”, “gt_masks” and “gt_semantic_seg”. These fields are formatted as follows.
- img: (1)transpose, (2)to tensor, (3)to DataContainer (stack=True)
- proposals: (1)to tensor, (2)to DataContainer
- gt_bboxes: (1)to tensor, (2)to DataContainer
- gt_bboxes_ignore: (1)to tensor, (2)to DataContainer
- gt_labels: (1)to tensor, (2)to DataContainer
- gt_masks: (1)to tensor, (2)to DataContainer (cpu_only=True)
- gt_semantic_seg: (1)unsqueeze dim-0 (2)to tensor, (3)to DataContainer (stack=True)
-
class
mmdet.datasets.pipelines.
LoadAnnotations
(with_bbox=True, with_label=True, with_mask=False, with_seg=False, poly2mask=True, file_client_args={'backend': 'disk'})[source]¶ Load multiple types of annotations.
Parameters: - with_bbox (bool) – Whether to parse and load the bbox annotation. Default: True.
- with_label (bool) – Whether to parse and load the label annotation. Default: True.
- with_mask (bool) – Whether to parse and load the mask annotation. Default: False.
- with_seg (bool) – Whether to parse and load the semantic segmentation annotation. Default: False.
- poly2mask (bool) – Whether to convert the instance masks from polygons to bitmaps. Default: True.
- file_client_args (dict) – Arguments to instantiate a FileClient.
See
mmcv.fileio.FileClient
for details. Defaults todict(backend='disk')
.
-
class
mmdet.datasets.pipelines.
LoadImageFromFile
(to_float32=False, color_type='color', file_client_args={'backend': 'disk'})[source]¶ Load an image from file.
Required keys are “img_prefix” and “img_info” (a dict that must contain the key “filename”). Added or updated keys are “filename”, “img”, “img_shape”, “ori_shape” (same as img_shape), “pad_shape” (same as img_shape), “scale_factor” (1.0) and “img_norm_cfg” (means=0 and stds=1).
Parameters: - to_float32 (bool) – Whether to convert the loaded image to a float32 numpy array. If set to False, the loaded image is an uint8 array. Defaults to False.
- color_type (str) – The flag argument for
mmcv.imfrombytes()
. Defaults to ‘color’. - file_client_args (dict) – Arguments to instantiate a FileClient.
See
mmcv.fileio.FileClient
for details. Defaults todict(backend='disk')
.
-
class
mmdet.datasets.pipelines.
LoadImageFromWebcam
(to_float32=False, color_type='color', file_client_args={'backend': 'disk'})[source]¶ Load an image from webcam.
Similar with
LoadImageFromFile
, but the image read from webcam is inresults['img']
.
-
class
mmdet.datasets.pipelines.
LoadMultiChannelImageFromFiles
(to_float32=False, color_type='unchanged', file_client_args={'backend': 'disk'})[source]¶ Load multi-channel images from a list of separate channel files.
Required keys are “img_prefix” and “img_info” (a dict that must contain the key “filename”, which is expected to be a list of filenames). Added or updated keys are “filename”, “img”, “img_shape”, “ori_shape” (same as img_shape), “pad_shape” (same as img_shape), “scale_factor” (1.0) and “img_norm_cfg” (means=0 and stds=1).
Parameters: - to_float32 (bool) – Whether to convert the loaded image to a float32 numpy array. If set to False, the loaded image is an uint8 array. Defaults to False.
- color_type (str) – The flag argument for
mmcv.imfrombytes()
. Defaults to ‘color’. - file_client_args (dict) – Arguments to instantiate a FileClient.
See
mmcv.fileio.FileClient
for details. Defaults todict(backend='disk')
.
-
class
mmdet.datasets.pipelines.
LoadProposals
(num_max_proposals=None)[source]¶ Load proposal pipeline.
Required key is “proposals”. Updated keys are “proposals”, “bbox_fields”.
Parameters: num_max_proposals (int, optional) – Maximum number of proposals to load. If not specified, all proposals will be loaded.
-
class
mmdet.datasets.pipelines.
MultiScaleFlipAug
(transforms, img_scale=None, scale_factor=None, flip=False, flip_direction='horizontal')[source]¶ Test-time augmentation with multiple scales and flipping.
An example configuration is as followed:
After MultiScaleFLipAug with above configuration, the results are wrapped into lists of the same length as followed:
Parameters: - transforms (list[dict]) – Transforms to apply in each augmentation.
- img_scale (tuple | list[tuple] | None) – Images scales for resizing.
- scale_factor (float | list[float] | None) – Scale factors for resizing.
- flip (bool) – Whether apply flip augmentation. Default: False.
- flip_direction (str | list[str]) – Flip augmentation directions, options are “horizontal”, “vertical” and “diagonal”. If flip_direction is a list, multiple flip augmentations will be applied. It has no effect when flip == False. Default: “horizontal”.
-
class
mmdet.datasets.pipelines.
Resize
(img_scale=None, multiscale_mode='range', ratio_range=None, keep_ratio=True, bbox_clip_border=True, backend='cv2', override=False)[source]¶ Resize images & bbox & mask.
This transform resizes the input image to some scale. Bboxes and masks are then resized with the same scale factor. If the input dict contains the key “scale”, then the scale in the input dict is used, otherwise the specified scale in the init method is used. If the input dict contains the key “scale_factor” (if MultiScaleFlipAug does not give img_scale but scale_factor), the actual scale will be computed by image shape and scale_factor.
img_scale can either be a tuple (single-scale) or a list of tuple (multi-scale). There are 3 multiscale modes:
ratio_range is not None
: randomly sample a ratio from the ratio range and multiply it with the image scale.ratio_range is None
andmultiscale_mode == "range"
: randomly sample a scale from the multiscale range.ratio_range is None
andmultiscale_mode == "value"
: randomly sample a scale from multiple scales.
Parameters: - img_scale (tuple or list[tuple]) – Images scales for resizing.
- multiscale_mode (str) – Either “range” or “value”.
- ratio_range (tuple[float]) – (min_ratio, max_ratio)
- keep_ratio (bool) – Whether to keep the aspect ratio when resizing the image.
- bbox_clip_border (bool, optional) – Whether clip the objects outside the border of the image. Defaults to True.
- backend (str) – Image resize backend, choices are ‘cv2’ and ‘pillow’. These two backends generates slightly different results. Defaults to ‘cv2’.
- override (bool, optional) – Whether to override scale and scale_factor so as to call resize twice. Default False. If True, after the first resizing, the existed scale and scale_factor will be ignored so the second resizing can be allowed. This option is a work-around for multiple times of resize in DETR. Defaults to False.
-
static
random_sample
(img_scales)[source]¶ Randomly sample an img_scale when
multiscale_mode=='range'
.Parameters: img_scales (list[tuple]) – Images scale range for sampling. There must be two tuples in img_scales, which specify the lower and upper bound of image scales. Returns: Returns a tuple (img_scale, None)
, whereimg_scale
is sampled scale and None is just a placeholder to be consistent withrandom_select()
.Return type: (tuple, None)
-
static
random_sample_ratio
(img_scale, ratio_range)[source]¶ Randomly sample an img_scale when
ratio_range
is specified.A ratio will be randomly sampled from the range specified by
ratio_range
. Then it would be multiplied withimg_scale
to generate sampled scale.Parameters: - img_scale (tuple) – Images scale base to multiply with ratio.
- ratio_range (tuple[float]) – The minimum and maximum ratio to scale
the
img_scale
.
Returns: Returns a tuple
(scale, None)
, wherescale
is sampled ratio multiplied withimg_scale
and None is just a placeholder to be consistent withrandom_select()
.Return type: (tuple, None)
-
static
random_select
(img_scales)[source]¶ Randomly select an img_scale from given candidates.
Parameters: img_scales (list[tuple]) – Images scales for selection. Returns: Returns a tuple (img_scale, scale_dix)
, whereimg_scale
is the selected image scale andscale_idx
is the selected index in the given candidates.Return type: (tuple, int)
-
class
mmdet.datasets.pipelines.
RandomFlip
(flip_ratio=None, direction='horizontal')[source]¶ Flip the image & bbox & mask.
If the input dict contains the key “flip”, then the flag will be used, otherwise it will be randomly decided by a ratio specified in the init method.
When random flip is enabled,
flip_ratio
/direction
can either be a float/string or tuple of float/string. There are 3 flip modes:flip_ratio
is float,direction
is string: the image will bedirection``ly flipped with probability of ``flip_ratio
. E.g.,flip_ratio=0.5
,direction='horizontal'
, then image will be horizontally flipped with probability of 0.5.
flip_ratio
is float,direction
is list of string: the image wil- be
direction[i]``ly flipped with probability of ``flip_ratio/len(direction)
. E.g.,flip_ratio=0.5
,direction=['horizontal', 'vertical']
, then image will be horizontally flipped with probability of 0.25, vertically with probability of 0.25.
flip_ratio
is list of float,direction
is list of string:- given
len(flip_ratio) == len(direction)
, the image wil bedirection[i]``ly flipped with probability of ``flip_ratio[i]
. E.g.,flip_ratio=[0.3, 0.5]
,direction=['horizontal', 'vertical']
, then image will be horizontally flipped with probability of 0.3, vertically with probability of 0.5.
Parameters: - flip_ratio (float | list[float], optional) – The flipping probability. Default: None.
- direction (str | list[str], optional) – The flipping direction. Options
are ‘horizontal’, ‘vertical’, ‘diagonal’. Default: ‘horizontal’.
If input is a list, the length must equal
flip_ratio
. Each element inflip_ratio
indicates the flip probability of corresponding direction.
-
bbox_flip
(bboxes, img_shape, direction)[source]¶ Flip bboxes horizontally.
Parameters: - bboxes (numpy.ndarray) – Bounding boxes, shape (…, 4*k)
- img_shape (tuple[int]) – Image shape (height, width)
- direction (str) – Flip direction. Options are ‘horizontal’, ‘vertical’.
Returns: Flipped bounding boxes.
Return type: numpy.ndarray
-
class
mmdet.datasets.pipelines.
Pad
(size=None, size_divisor=None, pad_to_square=False, pad_val=0)[source]¶ Pad the image & mask.
There are two padding modes: (1) pad to a fixed size and (2) pad to the minimum size that is divisible by some number. Added keys are “pad_shape”, “pad_fixed_size”, “pad_size_divisor”,
Parameters: - size (tuple, optional) – Fixed padding size.
- size_divisor (int, optional) – The divisor of padded size.
- pad_to_square (bool) – Whether to pad the image into a square. Currently only used for YOLOX. Default: False.
- pad_val (float, optional) – Padding value, 0 by default.
-
class
mmdet.datasets.pipelines.
RandomCrop
(crop_size, crop_type='absolute', allow_negative_crop=False, bbox_clip_border=True)[source]¶ Random crop the image & bboxes & masks.
The absolute crop_size is sampled based on crop_type and image_size, then the cropped results are generated.
Parameters: - crop_size (tuple) – The relative ratio or absolute pixels of height and width.
- crop_type (str, optional) – one of “relative_range”, “relative”, “absolute”, “absolute_range”. “relative” randomly crops (h * crop_size[0], w * crop_size[1]) part from an input of size (h, w). “relative_range” uniformly samples relative crop size from range [crop_size[0], 1] and [crop_size[1], 1] for height and width respectively. “absolute” crops from an input with absolute size (crop_size[0], crop_size[1]). “absolute_range” uniformly samples crop_h in range [crop_size[0], min(h, crop_size[1])] and crop_w in range [crop_size[0], min(w, crop_size[1])]. Default “absolute”.
- allow_negative_crop (bool, optional) – Whether to allow a crop that does not contain any bbox area. Default False.
- bbox_clip_border (bool, optional) – Whether clip the objects outside the border of the image. Defaults to True.
Note
- If the image is smaller than the absolute crop size, return the
- original image.
- The keys for bboxes, labels and masks must be aligned. That is, gt_bboxes corresponds to gt_labels and gt_masks, and gt_bboxes_ignore corresponds to gt_labels_ignore and gt_masks_ignore.
- If the crop does not contain any gt-bbox region and allow_negative_crop is set to False, skip this image.
-
class
mmdet.datasets.pipelines.
Normalize
(mean, std, to_rgb=True)[source]¶ Normalize the image.
Added key is “img_norm_cfg”.
Parameters: - mean (sequence) – Mean values of 3 channels.
- std (sequence) – Std values of 3 channels.
- to_rgb (bool) – Whether to convert the image from BGR to RGB, default is true.
-
class
mmdet.datasets.pipelines.
SegRescale
(scale_factor=1, backend='cv2')[source]¶ Rescale semantic segmentation maps.
Parameters: - scale_factor (float) – The scale factor of the final output.
- backend (str) – Image rescale backend, choices are ‘cv2’ and ‘pillow’. These two backends generates slightly different results. Defaults to ‘cv2’.
-
class
mmdet.datasets.pipelines.
MinIoURandomCrop
(min_ious=(0.1, 0.3, 0.5, 0.7, 0.9), min_crop_size=0.3, bbox_clip_border=True)[source]¶ Random crop the image & bboxes, the cropped patches have minimum IoU requirement with original image & bboxes, the IoU threshold is randomly selected from min_ious.
Parameters: - min_ious (tuple) – minimum IoU threshold for all intersections with
- boxes (bounding) –
- min_crop_size (float) – minimum crop’s size (i.e. h,w := a*h, a*w,
- a >= min_crop_size) (where) –
- bbox_clip_border (bool, optional) – Whether clip the objects outside the border of the image. Defaults to True.
Note
The keys for bboxes, labels and masks should be paired. That is, gt_bboxes corresponds to gt_labels and gt_masks, and gt_bboxes_ignore to gt_labels_ignore and gt_masks_ignore.
-
class
mmdet.datasets.pipelines.
Expand
(mean=(0, 0, 0), to_rgb=True, ratio_range=(1, 4), seg_ignore_label=None, prob=0.5)[source]¶ Random expand the image & bboxes.
Randomly place the original image on a canvas of ‘ratio’ x original image size filled with mean values. The ratio is in the range of ratio_range.
Parameters: - mean (tuple) – mean value of dataset.
- to_rgb (bool) – if need to convert the order of mean to align with RGB.
- ratio_range (tuple) – range of expand ratio.
- prob (float) – probability of applying this transformation
-
class
mmdet.datasets.pipelines.
PhotoMetricDistortion
(brightness_delta=32, contrast_range=(0.5, 1.5), saturation_range=(0.5, 1.5), hue_delta=18)[source]¶ Apply photometric distortion to image sequentially, every transformation is applied with a probability of 0.5. The position of random contrast is in second or second to last.
- random brightness
- random contrast (mode 0)
- convert color from BGR to HSV
- random saturation
- random hue
- convert color from HSV to BGR
- random contrast (mode 1)
- randomly swap channels
Parameters: - brightness_delta (int) – delta of brightness.
- contrast_range (tuple) – range of contrast.
- saturation_range (tuple) – range of saturation.
- hue_delta (int) – delta of hue.
-
class
mmdet.datasets.pipelines.
Albu
(transforms, bbox_params=None, keymap=None, update_pad_shape=False, skip_img_without_anno=False)[source]¶ Albumentation augmentation.
Adds custom transformations from Albumentations library. Please, visit https://albumentations.readthedocs.io to get more information.
An example of
transforms
is as followed:Parameters: - transforms (list[dict]) – A list of albu transformations
- bbox_params (dict) – Bbox_params for albumentation Compose
- keymap (dict) – Contains {‘input key’:’albumentation-style key’}
- skip_img_without_anno (bool) – Whether to skip the image if no ann left after aug
-
class
mmdet.datasets.pipelines.
InstaBoost
(action_candidate=('normal', 'horizontal', 'skip'), action_prob=(1, 0, 0), scale=(0.8, 1.2), dx=15, dy=15, theta=(-1, 1), color_prob=0.5, hflag=False, aug_ratio=0.5)[source]¶ Data augmentation method in InstaBoost: Boosting Instance Segmentation Via Probability Map Guided Copy-Pasting.
Refer to https://github.com/GothicAi/Instaboost for implementation details.
Parameters: - action_candidate (tuple) – Action candidates. “normal”, “horizontal”, “vertical”, “skip” are supported. Default: (‘normal’, ‘horizontal’, ‘skip’).
- action_prob (tuple) – Corresponding action probabilities. Should be the same length as action_candidate. Default: (1, 0, 0).
- scale (tuple) – (min scale, max scale). Default: (0.8, 1.2).
- dx (int) – The maximum x-axis shift will be (instance width) / dx. Default 15.
- dy (int) – The maximum y-axis shift will be (instance height) / dy. Default 15.
- theta (tuple) – (min rotation degree, max rotation degree). Default: (-1, 1).
- color_prob (float) – Probability of images for color augmentation. Default 0.5.
- heatmap_flag (bool) – Whether to use heatmap guided. Default False.
- aug_ratio (float) – Probability of applying this transformation. Default 0.5.
-
class
mmdet.datasets.pipelines.
RandomCenterCropPad
(crop_size=None, ratios=(0.9, 1.0, 1.1), border=128, mean=None, std=None, to_rgb=None, test_mode=False, test_pad_mode=('logical_or', 127), test_pad_add_pix=0, bbox_clip_border=True)[source]¶ Random center crop and random around padding for CornerNet.
This operation generates randomly cropped image from the original image and pads it simultaneously. Different from
RandomCrop
, the output shape may not equal tocrop_size
strictly. We choose a random value fromratios
and the output shape could be larger or smaller thancrop_size
. The padding operation is also different fromPad
, here we use around padding instead of right-bottom padding.The relation between output image (padding image) and original image:
output image +----------------------------+ | padded area | +------|----------------------------|----------+ | | cropped area | | | | +---------------+ | | | | | . center | | | original image | | | range | | | | | +---------------+ | | +------|----------------------------|----------+ | padded area | +----------------------------+
There are 5 main areas in the figure:
- output image: output image of this operation, also called padding image in following instruction.
- original image: input image of this operation.
- padded area: non-intersect area of output image and original image.
- cropped area: the overlap of output image and original image.
- center range: a smaller area where random center chosen from.
center range is computed by
border
and original image’s shape to avoid our random center is too close to original image’s border.
Also this operation act differently in train and test mode, the summary pipeline is listed below.
Train pipeline:
- Choose a
random_ratio
fromratios
, the shape of padding image will berandom_ratio * crop_size
. - Choose a
random_center
in center range. - Generate padding image with center matches the
random_center
. - Initialize the padding image with pixel value equals to
mean
. - Copy the cropped area to padding image.
- Refine annotations.
Test pipeline:
- Compute output shape according to
test_pad_mode
. - Generate padding image with center matches the original image center.
- Initialize the padding image with pixel value equals to
mean
. - Copy the
cropped area
to padding image.
Parameters: - crop_size (tuple | None) – expected size after crop, final size will computed according to ratio. Requires (h, w) in train mode, and None in test mode.
- ratios (tuple) – random select a ratio from tuple and crop image to (crop_size[0] * ratio) * (crop_size[1] * ratio). Only available in train mode.
- border (int) – max distance from center select area to image border. Only available in train mode.
- mean (sequence) – Mean values of 3 channels.
- std (sequence) – Std values of 3 channels.
- to_rgb (bool) – Whether to convert the image from BGR to RGB.
- test_mode (bool) – whether involve random variables in transform. In train mode, crop_size is fixed, center coords and ratio is random selected from predefined lists. In test mode, crop_size is image’s original shape, center coords and ratio is fixed.
- test_pad_mode (tuple) –
padding method and padding shape value, only available in test mode. Default is using ‘logical_or’ with 127 as padding shape value.
- ’logical_or’: final_shape = input_shape | padding_shape_value
- ’size_divisor’: final_shape = int( ceil(input_shape / padding_shape_value) * padding_shape_value)
- test_pad_add_pix (int) – Extra padding pixel in test mode. Default 0.
- bbox_clip_border (bool, optional) – Whether clip the objects outside the border of the image. Defaults to True.
-
class
mmdet.datasets.pipelines.
AutoAugment
(policies)[source]¶ Auto augmentation.
This data augmentation is proposed in Learning Data Augmentation Strategies for Object Detection.
TODO: Implement ‘Shear’, ‘Sharpness’ and ‘Rotate’ transforms
Parameters: policies (list[list[dict]]) – The policies of auto augmentation. Each policy in policies
is a specific augmentation policy, and is composed by several augmentations (dict). When AutoAugment is called, a random policy inpolicies
will be selected to augment images.Examples
>>> replace = (104, 116, 124) >>> policies = [ >>> [ >>> dict(type='Sharpness', prob=0.0, level=8), >>> dict( >>> type='Shear', >>> prob=0.4, >>> level=0, >>> replace=replace, >>> axis='x') >>> ], >>> [ >>> dict( >>> type='Rotate', >>> prob=0.6, >>> level=10, >>> replace=replace), >>> dict(type='Color', prob=1.0, level=6) >>> ] >>> ] >>> augmentation = AutoAugment(policies) >>> img = np.ones(100, 100, 3) >>> gt_bboxes = np.ones(10, 4) >>> results = dict(img=img, gt_bboxes=gt_bboxes) >>> results = augmentation(results)
-
class
mmdet.datasets.pipelines.
CutOut
(n_holes, cutout_shape=None, cutout_ratio=None, fill_in=(0, 0, 0))[source]¶ CutOut operation.
Randomly drop some regions of image used in Cutout.
Parameters: - n_holes (int | tuple[int, int]) – Number of regions to be dropped. If it is given as a list, number of holes will be randomly selected from the closed interval [n_holes[0], n_holes[1]].
- cutout_shape (tuple[int, int] | list[tuple[int, int]]) – The candidate shape of dropped regions. It can be tuple[int, int] to use a fixed cutout shape, or list[tuple[int, int]] to randomly choose shape from the list.
- cutout_ratio (tuple[float, float] | list[tuple[float, float]]) – The candidate ratio of dropped regions. It can be tuple[float, float] to use a fixed ratio or list[tuple[float, float]] to randomly choose ratio from the list. Please note that cutout_shape and cutout_ratio cannot be both given at the same time.
- fill_in (tuple[float, float, float] | tuple[int, int, int]) – The value of pixel to fill in the dropped regions. Default: (0, 0, 0).
-
class
mmdet.datasets.pipelines.
Shear
(level, img_fill_val=128, seg_ignore_label=255, prob=0.5, direction='horizontal', max_shear_magnitude=0.3, random_negative_prob=0.5, interpolation='bilinear')[source]¶ Apply Shear Transformation to image (and its corresponding bbox, mask, segmentation).
Parameters: - level (int | float) – The level should be in range [0,_MAX_LEVEL].
- img_fill_val (int | float | tuple) – The filled values for image border. If float, the same fill value will be used for all the three channels of image. If tuple, the should be 3 elements.
- seg_ignore_label (int) – The fill value used for segmentation map.
Note this value must equals
ignore_label
insemantic_head
of the corresponding config. Default 255. - prob (float) – The probability for performing Shear and should be in range [0, 1].
- direction (str) – The direction for shear, either “horizontal” or “vertical”.
- max_shear_magnitude (float) – The maximum magnitude for Shear transformation.
- random_negative_prob (float) – The probability that turns the offset negative. Should be in range [0,1]
- interpolation (str) – Same as in
mmcv.imshear()
.
-
class
mmdet.datasets.pipelines.
Rotate
(level, scale=1, center=None, img_fill_val=128, seg_ignore_label=255, prob=0.5, max_rotate_angle=30, random_negative_prob=0.5)[source]¶ Apply Rotate Transformation to image (and its corresponding bbox, mask, segmentation).
Parameters: - level (int | float) – The level should be in range (0,_MAX_LEVEL].
- scale (int | float) – Isotropic scale factor. Same in
mmcv.imrotate
. - center (int | float | tuple[float]) – Center point (w, h) of the
rotation in the source image. If None, the center of the
image will be used. Same in
mmcv.imrotate
. - img_fill_val (int | float | tuple) – The fill value for image border. If float, the same value will be used for all the three channels of image. If tuple, the should be 3 elements (e.g. equals the number of channels for image).
- seg_ignore_label (int) – The fill value used for segmentation map.
Note this value must equals
ignore_label
insemantic_head
of the corresponding config. Default 255. - prob (float) – The probability for perform transformation and should be in range 0 to 1.
- max_rotate_angle (int | float) – The maximum angles for rotate transformation.
- random_negative_prob (float) – The probability that turns the offset negative.
-
class
mmdet.datasets.pipelines.
ColorTransform
(level, prob=0.5)[source]¶ Apply Color transformation to image. The bboxes, masks, and segmentations are not modified.
Parameters: - level (int | float) – Should be in range [0,_MAX_LEVEL].
- prob (float) – The probability for performing Color transformation.
-
class
mmdet.datasets.pipelines.
EqualizeTransform
(prob=0.5)[source]¶ Apply Equalize transformation to image. The bboxes, masks and segmentations are not modified.
Parameters: prob (float) – The probability for performing Equalize transformation.
-
class
mmdet.datasets.pipelines.
BrightnessTransform
(level, prob=0.5)[source]¶ Apply Brightness transformation to image. The bboxes, masks and segmentations are not modified.
Parameters: - level (int | float) – Should be in range [0,_MAX_LEVEL].
- prob (float) – The probability for performing Brightness transformation.
-
class
mmdet.datasets.pipelines.
ContrastTransform
(level, prob=0.5)[source]¶ Apply Contrast transformation to image. The bboxes, masks and segmentations are not modified.
Parameters: - level (int | float) – Should be in range [0,_MAX_LEVEL].
- prob (float) – The probability for performing Contrast transformation.
-
class
mmdet.datasets.pipelines.
Translate
(level, prob=0.5, img_fill_val=128, seg_ignore_label=255, direction='horizontal', max_translate_offset=250.0, random_negative_prob=0.5, min_size=0)[source]¶ Translate the images, bboxes, masks and segmentation maps horizontally or vertically.
Parameters: - level (int | float) – The level for Translate and should be in range [0,_MAX_LEVEL].
- prob (float) – The probability for performing translation and should be in range [0, 1].
- img_fill_val (int | float | tuple) – The filled value for image border. If float, the same fill value will be used for all the three channels of image. If tuple, the should be 3 elements (e.g. equals the number of channels for image).
- seg_ignore_label (int) – The fill value used for segmentation map.
Note this value must equals
ignore_label
insemantic_head
of the corresponding config. Default 255. - direction (str) – The translate direction, either “horizontal” or “vertical”.
- max_translate_offset (int | float) – The maximum pixel’s offset for Translate.
- random_negative_prob (float) – The probability that turns the offset negative.
- min_size (int | float) – The minimum pixel for filtering invalid bboxes after the translation.
-
class
mmdet.datasets.pipelines.
RandomShift
(shift_ratio=0.5, max_shift_px=32, filter_thr_px=1)[source]¶ Shift the image and box given shift pixels and probability.
Parameters: - shift_ratio (float) – Probability of shifts. Default 0.5.
- max_shift_px (int) – The max pixels for shifting. Default 32.
- filter_thr_px (int) – The width and height threshold for filtering. The bbox and the rest of the targets below the width and height threshold will be filtered. Default 1.
-
class
mmdet.datasets.pipelines.
Mosaic
(img_scale=(640, 640), center_ratio_range=(0.5, 1.5), pad_val=114)[source]¶ Mosaic augmentation.
Given 4 images, mosaic transform combines them into one output image. The output image is composed of the parts from each sub- image.
mosaic transform center_x +------------------------------+ | pad | pad | | +-----------+ | | | | | | | image1 |--------+ | | | | | | | | | image2 | | center_y |----+-------------+-----------| | | cropped | | |pad | image3 | image4 | | | | | +----|-------------+-----------+ | | +-------------+ The mosaic transform steps are as follows: 1. Choose the mosaic center as the intersections of 4 images 2. Get the left top image according to the index, and randomly sample another 3 images from the custom dataset. 3. Sub image will be cropped if image is larger than mosaic patch
Parameters: - img_scale (Sequence[int]) – Image size after mosaic pipeline of single image. Default to (640, 640).
- center_ratio_range (Sequence[float]) – Center ratio range of mosaic output. Default to (0.5, 1.5).
- pad_val (int) – Pad value. Default to 114.
-
class
mmdet.datasets.pipelines.
MixUp
(img_scale=(640, 640), ratio_range=(0.5, 1.5), flip_ratio=0.5, pad_val=114, max_iters=15, min_bbox_size=5, min_area_ratio=0.2, max_aspect_ratio=20)[source]¶ MixUp data augmentation.
Parameters: - img_scale (Sequence[int]) – Image output size after mixup pipeline. Default: (640, 640).
- ratio_range (Sequence[float]) – Scale ratio of mixup image. Default: (0.5, 1.5).
- flip_ratio (float) – Horizontal flip ratio of mixup image. Default: 0.5.
- pad_val (int) – Pad value. Default: 114.
- max_iters (int) – The maximum number of iterations. If the number of iterations is greater than max_iters, but gt_bbox is still empty, then the iteration is terminated. Default: 15.
- min_bbox_size (float) – Width and height threshold to filter bboxes. If the height or width of a box is smaller than this value, it will be removed. Default: 5.
- min_area_ratio (float) – Threshold of area ratio between original bboxes and wrapped bboxes. If smaller than this value, the box will be removed. Default: 0.2.
- max_aspect_ratio (float) – Aspect ratio of width and height threshold to filter bboxes. If max(h/w, w/h) larger than this value, the box will be removed. Default: 20.
-
class
mmdet.datasets.pipelines.
RandomAffine
(max_rotate_degree=10.0, max_translate_ratio=0.1, scaling_ratio_range=(0.5, 1.5), max_shear_degree=2.0, border=(0, 0), border_val=(114, 114, 114), min_bbox_size=2, min_area_ratio=0.2, max_aspect_ratio=20)[source]¶ Random affine transform data augmentation.
This operation randomly generates affine transform matrix which including rotation, translation, shear and scaling transforms.
Parameters: - max_rotate_degree (float) – Maximum degrees of rotation transform. Default: 10.
- max_translate_ratio (float) – Maximum ratio of translation. Default: 0.1.
- scaling_ratio_range (tuple[float]) – Min and max ratio of scaling transform. Default: (0.5, 1.5).
- max_shear_degree (float) – Maximum degrees of shear transform. Default: 2.
- border (tuple[int]) – Distance from height and width sides of input image to adjust output shape. Only used in mosaic dataset. Default: (0, 0).
- border_val (tuple[int]) – Border padding values of 3 channels. Default: (114, 114, 114).
- min_bbox_size (float) – Width and height threshold to filter bboxes. If the height or width of a box is smaller than this value, it will be removed. Default: 2.
- min_area_ratio (float) – Threshold of area ratio between original bboxes and wrapped bboxes. If smaller than this value, the box will be removed. Default: 0.2.
- max_aspect_ratio (float) – Aspect ratio of width and height threshold to filter bboxes. If max(h/w, w/h) larger than this value, the box will be removed.
samplers¶
-
class
mmdet.datasets.samplers.
DistributedSampler
(dataset, num_replicas=None, rank=None, shuffle=True, seed=0)[source]¶
-
class
mmdet.datasets.samplers.
DistributedGroupSampler
(dataset, samples_per_gpu=1, num_replicas=None, rank=None, seed=0)[source]¶ Sampler that restricts data loading to a subset of the dataset.
It is especially useful in conjunction with
torch.nn.parallel.DistributedDataParallel
. In such case, each process can pass a DistributedSampler instance as a DataLoader sampler, and load a subset of the original dataset that is exclusive to it.Note
Dataset is assumed to be of constant size.
Parameters: - dataset – Dataset used for sampling.
- num_replicas (optional) – Number of processes participating in distributed training.
- rank (optional) – Rank of the current process within num_replicas.
- seed (int, optional) – random seed used to shuffle the sampler if
shuffle=True
. This number should be identical across all processes in the distributed group. Default: 0.
mmdet.models¶
detectors¶
-
class
mmdet.models.detectors.
ATSS
(backbone, neck, bbox_head, train_cfg=None, test_cfg=None, pretrained=None, init_cfg=None)[source]¶ Implementation of ATSS.
-
class
mmdet.models.detectors.
BaseDetector
(init_cfg=None)[source]¶ Base class for detectors.
-
extract_feats
(imgs)[source]¶ Extract features from multiple images.
Parameters: imgs (list[torch.Tensor]) – A list of images. The images are augmented from the same image but in different ways. Returns: Features of different images Return type: list[torch.Tensor]
-
forward
(img, img_metas, return_loss=True, **kwargs)[source]¶ Calls either
forward_train()
orforward_test()
depending on whetherreturn_loss
isTrue
.Note this setting will change the expected inputs. When
return_loss=True
, img and img_meta are single-nested (i.e. Tensor and List[dict]), and whenresturn_loss=False
, img and img_meta should be double nested (i.e. List[Tensor], List[List[dict]]), with the outer list indicating test time augmentations.
-
forward_test
(imgs, img_metas, **kwargs)[source]¶ Parameters: - imgs (List[Tensor]) – the outer list indicates test-time augmentations and inner Tensor should have a shape NxCxHxW, which contains all images in the batch.
- img_metas (List[List[dict]]) – the outer list indicates test-time augs (multiscale, flip, etc.) and the inner list indicates images in a batch.
-
forward_train
(imgs, img_metas, **kwargs)[source]¶ Parameters: - img (list[Tensor]) – List of tensors of shape (1, C, H, W). Typically these should be mean centered and std scaled.
- img_metas (list[dict]) – List of image info dict where each dict
has: ‘img_shape’, ‘scale_factor’, ‘flip’, and may also contain
‘filename’, ‘ori_shape’, ‘pad_shape’, and ‘img_norm_cfg’.
For details on the values of these keys, see
mmdet.datasets.pipelines.Collect
. - kwargs (keyword arguments) – Specific to concrete implementation.
-
show_result
(img, result, score_thr=0.3, bbox_color=(72, 101, 241), text_color=(72, 101, 241), mask_color=None, thickness=2, font_size=13, win_name='', show=False, wait_time=0, out_file=None)[source]¶ Draw result over img.
Parameters: - img (str or Tensor) – The image to be displayed.
- result (Tensor or tuple) – The results to draw over img bbox_result or (bbox_result, segm_result).
- score_thr (float, optional) – Minimum score of bboxes to be shown. Default: 0.3.
- bbox_color (str or tuple(int) or
Color
) – Color of bbox lines. The tuple of color should be in BGR order. Default: ‘green’ - text_color (str or tuple(int) or
Color
) – Color of texts. The tuple of color should be in BGR order. Default: ‘green’ - mask_color (None or str or tuple(int) or
Color
) – Color of masks. The tuple of color should be in BGR order. Default: None - thickness (int) – Thickness of lines. Default: 2
- font_size (int) – Font size of texts. Default: 13
- win_name (str) – The window name. Default: ‘’
- wait_time (float) – Value of waitKey param. Default: 0.
- show (bool) – Whether to show the image. Default: False.
- out_file (str or None) – The filename to write the image. Default: None.
Returns: Only if not show or out_file
Return type: img (Tensor)
-
train_step
(data, optimizer)[source]¶ The iteration step during training.
This method defines an iteration step during training, except for the back propagation and optimizer updating, which are done in an optimizer hook. Note that in some complicated cases or models, the whole process including back propagation and optimizer updating is also defined in this method, such as GAN.
Parameters: - data (dict) – The output of dataloader.
- optimizer (
torch.optim.Optimizer
| dict) – The optimizer of runner is passed totrain_step()
. This argument is unused and reserved.
Returns: It should contain at least 3 keys:
loss
,log_vars
,num_samples
.loss
is a tensor for back propagation, which can be a weighted sum of multiple losses.log_vars
contains all the variables to be sent to the logger.num_samples
indicates the batch size (when the model is DDP, it means the batch size on each GPU), which is used for averaging the logs.
Return type: dict
-
val_step
(data, optimizer=None)[source]¶ The iteration step during validation.
This method shares the same signature as
train_step()
, but used during val epochs. Note that the evaluation after training epochs is not implemented with this method, but an evaluation hook.
-
with_bbox
¶ whether the detector has a bbox head
Type: bool
-
with_mask
¶ whether the detector has a mask head
Type: bool
-
with_neck
¶ whether the detector has a neck
Type: bool
whether the detector has a shared head in the RoI Head
Type: bool
-
-
class
mmdet.models.detectors.
SingleStageDetector
(backbone, neck=None, bbox_head=None, train_cfg=None, test_cfg=None, pretrained=None, init_cfg=None)[source]¶ Base class for single-stage detectors.
Single-stage detectors directly and densely predict bounding boxes on the output features of the backbone+neck.
-
aug_test
(imgs, img_metas, rescale=False)[source]¶ Test function with test time augmentation.
Parameters: - imgs (list[Tensor]) – the outer list indicates test-time augmentations and inner Tensor should have a shape NxCxHxW, which contains all images in the batch.
- img_metas (list[list[dict]]) – the outer list indicates test-time augs (multiscale, flip, etc.) and the inner list indicates images in a batch. each dict has image information.
- rescale (bool, optional) – Whether to rescale the results. Defaults to False.
Returns: - BBox results of each image and classes.
The outer list corresponds to each image. The inner list corresponds to each class.
Return type: list[list[np.ndarray]]
-
forward_dummy
(img)[source]¶ Used for computing network flops.
See mmdetection/tools/analysis_tools/get_flops.py
-
forward_train
(img, img_metas, gt_bboxes, gt_labels, gt_bboxes_ignore=None)[source]¶ Parameters: - img (Tensor) – Input images of shape (N, C, H, W). Typically these should be mean centered and std scaled.
- img_metas (list[dict]) – A List of image info dict where each dict
has: ‘img_shape’, ‘scale_factor’, ‘flip’, and may also contain
‘filename’, ‘ori_shape’, ‘pad_shape’, and ‘img_norm_cfg’.
For details on the values of these keys see
mmdet.datasets.pipelines.Collect
. - gt_bboxes (list[Tensor]) – Each item are the truth boxes for each image in [tl_x, tl_y, br_x, br_y] format.
- gt_labels (list[Tensor]) – Class indices corresponding to each box
- gt_bboxes_ignore (None | list[Tensor]) – Specify which bounding boxes can be ignored when computing the loss.
Returns: A dictionary of loss components.
Return type: dict[str, Tensor]
-
onnx_export
(img, img_metas)[source]¶ Test function without test time augmentation.
Parameters: - img (torch.Tensor) – input images.
- img_metas (list[dict]) – List of image information.
Returns: - dets of shape [N, num_det, 5]
and class labels of shape [N, num_det].
Return type: tuple[Tensor, Tensor]
-
simple_test
(img, img_metas, rescale=False)[source]¶ Test function without test-time augmentation.
Parameters: - img (torch.Tensor) – Images with shape (N, C, H, W).
- img_metas (list[dict]) – List of image information.
- rescale (bool, optional) – Whether to rescale the results. Defaults to False.
Returns: - BBox results of each image and classes.
The outer list corresponds to each image. The inner list corresponds to each class.
Return type: list[list[np.ndarray]]
-
-
class
mmdet.models.detectors.
TwoStageDetector
(backbone, neck=None, rpn_head=None, roi_head=None, train_cfg=None, test_cfg=None, pretrained=None, init_cfg=None)[source]¶ Base class for two-stage detectors.
Two-stage detectors typically consisting of a region proposal network and a task-specific regression head.
-
async_simple_test
(img, img_meta, proposals=None, rescale=False)[source]¶ Async test without augmentation.
-
aug_test
(imgs, img_metas, rescale=False)[source]¶ Test with augmentations.
If rescale is False, then returned bboxes and masks will fit the scale of imgs[0].
-
forward_dummy
(img)[source]¶ Used for computing network flops.
See mmdetection/tools/analysis_tools/get_flops.py
-
forward_train
(img, img_metas, gt_bboxes, gt_labels, gt_bboxes_ignore=None, gt_masks=None, proposals=None, **kwargs)[source]¶ Parameters: - img (Tensor) – of shape (N, C, H, W) encoding input images. Typically these should be mean centered and std scaled.
- img_metas (list[dict]) – list of image info dict where each dict has: ‘img_shape’, ‘scale_factor’, ‘flip’, and may also contain ‘filename’, ‘ori_shape’, ‘pad_shape’, and ‘img_norm_cfg’. For details on the values of these keys see mmdet/datasets/pipelines/formatting.py:Collect.
- gt_bboxes (list[Tensor]) – Ground truth bboxes for each image with shape (num_gts, 4) in [tl_x, tl_y, br_x, br_y] format.
- gt_labels (list[Tensor]) – class indices corresponding to each box
- gt_bboxes_ignore (None | list[Tensor]) – specify which bounding boxes can be ignored when computing the loss.
- gt_masks (None | Tensor) – true segmentation masks for each box used if the architecture supports a segmentation task.
- proposals – override rpn proposals with custom proposals. Use when with_rpn is False.
Returns: a dictionary of loss components
Return type: dict[str, Tensor]
-
with_roi_head
¶ whether the detector has a RoI head
Type: bool
-
with_rpn
¶ whether the detector has RPN
Type: bool
-
-
class
mmdet.models.detectors.
RPN
(backbone, neck, rpn_head, train_cfg, test_cfg, pretrained=None, init_cfg=None)[source]¶ Implementation of Region Proposal Network.
-
aug_test
(imgs, img_metas, rescale=False)[source]¶ Test function with test time augmentation.
Parameters: - imgs (list[torch.Tensor]) – List of multiple images
- img_metas (list[dict]) – List of image information.
- rescale (bool, optional) – Whether to rescale the results. Defaults to False.
Returns: proposals
Return type: list[np.ndarray]
-
extract_feat
(img)[source]¶ Extract features.
Parameters: img (torch.Tensor) – Image tensor with shape (n, c, h ,w). Returns: - Multi-level features that may have
- different resolutions.
Return type: list[torch.Tensor]
-
forward_train
(img, img_metas, gt_bboxes=None, gt_bboxes_ignore=None)[source]¶ Parameters: - img (Tensor) – Input images of shape (N, C, H, W). Typically these should be mean centered and std scaled.
- img_metas (list[dict]) – A List of image info dict where each dict
has: ‘img_shape’, ‘scale_factor’, ‘flip’, and may also contain
‘filename’, ‘ori_shape’, ‘pad_shape’, and ‘img_norm_cfg’.
For details on the values of these keys see
mmdet.datasets.pipelines.Collect
. - gt_bboxes (list[Tensor]) – Each item are the truth boxes for each image in [tl_x, tl_y, br_x, br_y] format.
- gt_bboxes_ignore (None | list[Tensor]) – Specify which bounding boxes can be ignored when computing the loss.
Returns: A dictionary of loss components.
Return type: dict[str, Tensor]
-
show_result
(data, result, top_k=20, **kwargs)[source]¶ Show RPN proposals on the image.
Parameters: - data (str or np.ndarray) – Image filename or loaded image.
- result (Tensor or tuple) – The results to draw over img bbox_result or (bbox_result, segm_result).
- top_k (int) – Plot the first k bboxes only if set positive. Default: 20
Returns: The image with bboxes drawn on it.
Return type: np.ndarray
-
simple_test
(img, img_metas, rescale=False)[source]¶ Test function without test time augmentation.
Parameters: - imgs (list[torch.Tensor]) – List of multiple images
- img_metas (list[dict]) – List of image information.
- rescale (bool, optional) – Whether to rescale the results. Defaults to False.
Returns: proposals
Return type: list[np.ndarray]
-
-
class
mmdet.models.detectors.
KnowledgeDistillationSingleStageDetector
(backbone, neck, bbox_head, teacher_config, teacher_ckpt=None, eval_teacher=True, train_cfg=None, test_cfg=None, pretrained=None)[source]¶ Implementation of Distilling the Knowledge in a Neural Network..
Parameters: - teacher_config (str | dict) – Config file path or the config object of teacher model.
- teacher_ckpt (str, optional) – Checkpoint path of teacher model. If left as None, the model will not load any weights.
-
cuda
(device=None)[source]¶ Since teacher_model is registered as a plain object, it is necessary to put the teacher model to cuda when calling cuda function.
-
forward_train
(img, img_metas, gt_bboxes, gt_labels, gt_bboxes_ignore=None)[source]¶ Parameters: - img (Tensor) – Input images of shape (N, C, H, W). Typically these should be mean centered and std scaled.
- img_metas (list[dict]) – A List of image info dict where each dict
has: ‘img_shape’, ‘scale_factor’, ‘flip’, and may also contain
‘filename’, ‘ori_shape’, ‘pad_shape’, and ‘img_norm_cfg’.
For details on the values of these keys see
mmdet.datasets.pipelines.Collect
. - gt_bboxes (list[Tensor]) – Each item are the truth boxes for each image in [tl_x, tl_y, br_x, br_y] format.
- gt_labels (list[Tensor]) – Class indices corresponding to each box
- gt_bboxes_ignore (None | list[Tensor]) – Specify which bounding boxes can be ignored when computing the loss.
Returns: A dictionary of loss components.
Return type: dict[str, Tensor]
-
class
mmdet.models.detectors.
FastRCNN
(backbone, roi_head, train_cfg, test_cfg, neck=None, pretrained=None, init_cfg=None)[source]¶ Implementation of Fast R-CNN
-
forward_test
(imgs, img_metas, proposals, **kwargs)[source]¶ Parameters: - imgs (List[Tensor]) – the outer list indicates test-time augmentations and inner Tensor should have a shape NxCxHxW, which contains all images in the batch.
- img_metas (List[List[dict]]) – the outer list indicates test-time augs (multiscale, flip, etc.) and the inner list indicates images in a batch.
- proposals (List[List[Tensor]]) – the outer list indicates test-time augs (multiscale, flip, etc.) and the inner list indicates images in a batch. The Tensor should have a shape Px4, where P is the number of proposals.
-
-
class
mmdet.models.detectors.
FasterRCNN
(backbone, rpn_head, roi_head, train_cfg, test_cfg, neck=None, pretrained=None, init_cfg=None)[source]¶ Implementation of Faster R-CNN
-
class
mmdet.models.detectors.
MaskRCNN
(backbone, rpn_head, roi_head, train_cfg, test_cfg, neck=None, pretrained=None, init_cfg=None)[source]¶ Implementation of Mask R-CNN
-
class
mmdet.models.detectors.
CascadeRCNN
(backbone, neck=None, rpn_head=None, roi_head=None, train_cfg=None, test_cfg=None, pretrained=None, init_cfg=None)[source]¶ Implementation of Cascade R-CNN: Delving into High Quality Object Detection
-
show_result
(data, result, **kwargs)[source]¶ Show prediction results of the detector.
Parameters: - data (str or np.ndarray) – Image filename or loaded image.
- result (Tensor or tuple) – The results to draw over img bbox_result or (bbox_result, segm_result).
Returns: The image with bboxes drawn on it.
Return type: np.ndarray
-
-
class
mmdet.models.detectors.
HybridTaskCascade
(**kwargs)[source]¶ Implementation of HTC
-
with_semantic
¶ whether the detector has a semantic head
Type: bool
-
-
class
mmdet.models.detectors.
RetinaNet
(backbone, neck, bbox_head, train_cfg=None, test_cfg=None, pretrained=None, init_cfg=None)[source]¶ Implementation of RetinaNet
-
class
mmdet.models.detectors.
FCOS
(backbone, neck, bbox_head, train_cfg=None, test_cfg=None, pretrained=None, init_cfg=None)[source]¶ Implementation of FCOS
-
class
mmdet.models.detectors.
GridRCNN
(backbone, rpn_head, roi_head, train_cfg, test_cfg, neck=None, pretrained=None, init_cfg=None)[source]¶ Grid R-CNN.
This detector is the implementation of: - Grid R-CNN (https://arxiv.org/abs/1811.12030) - Grid R-CNN Plus: Faster and Better (https://arxiv.org/abs/1906.05688)
-
class
mmdet.models.detectors.
MaskScoringRCNN
(backbone, rpn_head, roi_head, train_cfg, test_cfg, neck=None, pretrained=None, init_cfg=None)[source]¶ Mask Scoring RCNN.
-
class
mmdet.models.detectors.
RepPointsDetector
(backbone, neck, bbox_head, train_cfg=None, test_cfg=None, pretrained=None, init_cfg=None)[source]¶ RepPoints: Point Set Representation for Object Detection.
This detector is the implementation of: - RepPoints detector (https://arxiv.org/pdf/1904.11490)
-
class
mmdet.models.detectors.
FOVEA
(backbone, neck, bbox_head, train_cfg=None, test_cfg=None, pretrained=None, init_cfg=None)[source]¶ Implementation of FoveaBox
-
class
mmdet.models.detectors.
FSAF
(backbone, neck, bbox_head, train_cfg=None, test_cfg=None, pretrained=None, init_cfg=None)[source]¶ Implementation of FSAF
-
class
mmdet.models.detectors.
NASFCOS
(backbone, neck, bbox_head, train_cfg=None, test_cfg=None, pretrained=None, init_cfg=None)[source]¶ NAS-FCOS: Fast Neural Architecture Search for Object Detection.
-
class
mmdet.models.detectors.
PointRend
(backbone, rpn_head, roi_head, train_cfg, test_cfg, neck=None, pretrained=None, init_cfg=None)[source]¶ PointRend: Image Segmentation as Rendering
This detector is the implementation of PointRend.
-
class
mmdet.models.detectors.
GFL
(backbone, neck, bbox_head, train_cfg=None, test_cfg=None, pretrained=None, init_cfg=None)[source]¶
-
class
mmdet.models.detectors.
CornerNet
(backbone, neck, bbox_head, train_cfg=None, test_cfg=None, pretrained=None, init_cfg=None)[source]¶ CornerNet.
This detector is the implementation of the paper CornerNet: Detecting Objects as Paired Keypoints .
-
aug_test
(imgs, img_metas, rescale=False)[source]¶ Augment testing of CornerNet.
Parameters: - imgs (list[Tensor]) – Augmented images.
- img_metas (list[list[dict]]) – Meta information of each image, e.g., image size, scaling factor, etc.
- rescale (bool) – If True, return boxes in original image space. Default: False.
Note
imgs
must including flipped image pairs.Returns: - BBox results of each image and classes.
- The outer list corresponds to each image. The inner list corresponds to each class.
Return type: list[list[np.ndarray]]
-
merge_aug_results
(aug_results, img_metas)[source]¶ Merge augmented detection bboxes and score.
Parameters: - aug_results (list[list[Tensor]]) – Det_bboxes and det_labels of each image.
- img_metas (list[list[dict]]) – Meta information of each image, e.g., image size, scaling factor, etc.
Returns: (bboxes, labels)
Return type: tuple
-
-
class
mmdet.models.detectors.
PAA
(backbone, neck, bbox_head, train_cfg=None, test_cfg=None, pretrained=None, init_cfg=None)[source]¶ Implementation of PAA.
-
class
mmdet.models.detectors.
YOLOV3
(backbone, neck, bbox_head, train_cfg=None, test_cfg=None, pretrained=None, init_cfg=None)[source]¶
-
class
mmdet.models.detectors.
YOLACT
(backbone, neck, bbox_head, segm_head, mask_head, train_cfg=None, test_cfg=None, pretrained=None, init_cfg=None)[source]¶ Implementation of YOLACT
-
forward_dummy
(img)[source]¶ Used for computing network flops.
See mmdetection/tools/analysis_tools/get_flops.py
-
forward_train
(img, img_metas, gt_bboxes, gt_labels, gt_bboxes_ignore=None, gt_masks=None)[source]¶ Parameters: - img (Tensor) – of shape (N, C, H, W) encoding input images. Typically these should be mean centered and std scaled.
- img_metas (list[dict]) – list of image info dict where each dict has: ‘img_shape’, ‘scale_factor’, ‘flip’, and may also contain ‘filename’, ‘ori_shape’, ‘pad_shape’, and ‘img_norm_cfg’. For details on the values of these keys see mmdet/datasets/pipelines/formatting.py:Collect.
- gt_bboxes (list[Tensor]) – Ground truth bboxes for each image with shape (num_gts, 4) in [tl_x, tl_y, br_x, br_y] format.
- gt_labels (list[Tensor]) – class indices corresponding to each box
- gt_bboxes_ignore (None | list[Tensor]) – specify which bounding boxes can be ignored when computing the loss.
- gt_masks (None | Tensor) – true segmentation masks for each box used if the architecture supports a segmentation task.
Returns: a dictionary of loss components
Return type: dict[str, Tensor]
-
-
class
mmdet.models.detectors.
VFNet
(backbone, neck, bbox_head, train_cfg=None, test_cfg=None, pretrained=None, init_cfg=None)[source]¶ Implementation of `VarifocalNet (VFNet).<https://arxiv.org/abs/2008.13367>`_
-
class
mmdet.models.detectors.
DETR
(backbone, bbox_head, train_cfg=None, test_cfg=None, pretrained=None, init_cfg=None)[source]¶ Implementation of DETR: End-to-End Object Detection with Transformers
-
forward_dummy
(img)[source]¶ Used for computing network flops.
See mmdetection/tools/analysis_tools/get_flops.py
-
onnx_export
(img, img_metas)[source]¶ Test function for exporting to ONNX, without test time augmentation.
Parameters: - img (torch.Tensor) – input images.
- img_metas (list[dict]) – List of image information.
Returns: - dets of shape [N, num_det, 5]
and class labels of shape [N, num_det].
Return type: tuple[Tensor, Tensor]
-
-
class
mmdet.models.detectors.
TridentFasterRCNN
(backbone, rpn_head, roi_head, train_cfg, test_cfg, neck=None, pretrained=None, init_cfg=None)[source]¶ Implementation of TridentNet
-
aug_test
(imgs, img_metas, rescale=False)[source]¶ Test with augmentations.
If rescale is False, then returned bboxes and masks will fit the scale of imgs[0].
-
-
class
mmdet.models.detectors.
SparseRCNN
(*args, **kwargs)[source]¶ Implementation of Sparse R-CNN: End-to-End Object Detection with Learnable Proposals
-
forward_dummy
(img)[source]¶ Used for computing network flops.
See mmdetection/tools/analysis_tools/get_flops.py
-
forward_train
(img, img_metas, gt_bboxes, gt_labels, gt_bboxes_ignore=None, gt_masks=None, proposals=None, **kwargs)[source]¶ Forward function of SparseR-CNN in train stage.
Parameters: - img (Tensor) – of shape (N, C, H, W) encoding input images. Typically these should be mean centered and std scaled.
- img_metas (list[dict]) – list of image info dict where each dict
has: ‘img_shape’, ‘scale_factor’, ‘flip’, and may also contain
‘filename’, ‘ori_shape’, ‘pad_shape’, and ‘img_norm_cfg’.
For details on the values of these keys see
mmdet.datasets.pipelines.Collect
. - gt_bboxes (list[Tensor]) – Ground truth bboxes for each image with shape (num_gts, 4) in [tl_x, tl_y, br_x, br_y] format.
- gt_labels (list[Tensor]) – class indices corresponding to each box
- gt_bboxes_ignore (None | list[Tensor) – specify which bounding boxes can be ignored when computing the loss.
- gt_masks (List[Tensor], optional) – Segmentation masks for each box. But we don’t support it in this architecture.
- proposals (List[Tensor], optional) – override rpn proposals with custom proposals. Use when with_rpn is False.
Returns: a dictionary of loss components
Return type: dict[str, Tensor]
-
simple_test
(img, img_metas, rescale=False)[source]¶ Test function without test time augmentation.
Parameters: - imgs (list[torch.Tensor]) – List of multiple images
- img_metas (list[dict]) – List of image information.
- rescale (bool) – Whether to rescale the results. Defaults to False.
Returns: - BBox results of each image and classes.
The outer list corresponds to each image. The inner list corresponds to each class.
Return type: list[list[np.ndarray]]
-
-
class
mmdet.models.detectors.
AutoAssign
(backbone, neck, bbox_head, train_cfg=None, test_cfg=None, pretrained=None)[source]¶ Implementation of AutoAssign: Differentiable Label Assignment for Dense Object Detection.
-
class
mmdet.models.detectors.
YOLOF
(backbone, neck, bbox_head, train_cfg=None, test_cfg=None, pretrained=None)[source]¶ Implementation of You Only Look One-level Feature
-
class
mmdet.models.detectors.
CenterNet
(backbone, neck, bbox_head, train_cfg=None, test_cfg=None, pretrained=None, init_cfg=None)[source]¶ Implementation of CenterNet(Objects as Points)
<https://arxiv.org/abs/1904.07850>.
-
aug_test
(imgs, img_metas, rescale=True)[source]¶ Augment testing of CenterNet. Aug test must have flipped image pair, and unlike CornerNet, it will perform an averaging operation on the feature map instead of detecting bbox.
Parameters: - imgs (list[Tensor]) – Augmented images.
- img_metas (list[list[dict]]) – Meta information of each image, e.g., image size, scaling factor, etc.
- rescale (bool) – If True, return boxes in original image space. Default: True.
Note
imgs
must including flipped image pairs.Returns: - BBox results of each image and classes.
- The outer list corresponds to each image. The inner list corresponds to each class.
Return type: list[list[np.ndarray]]
-
-
class
mmdet.models.detectors.
YOLOX
(backbone, neck, bbox_head, train_cfg=None, test_cfg=None, pretrained=None, init_cfg=None)[source]¶ Implementation of YOLOX: Exceeding YOLO Series in 2021
backbones¶
-
class
mmdet.models.backbones.
RegNet
(arch, in_channels=3, stem_channels=32, base_channels=32, strides=(2, 2, 2, 2), dilations=(1, 1, 1, 1), out_indices=(0, 1, 2, 3), style='pytorch', deep_stem=False, avg_down=False, frozen_stages=-1, conv_cfg=None, norm_cfg={'requires_grad': True, 'type': 'BN'}, norm_eval=True, dcn=None, stage_with_dcn=(False, False, False, False), plugins=None, with_cp=False, zero_init_residual=True, pretrained=None, init_cfg=None)[source]¶ RegNet backbone.
More details can be found in paper .
Parameters: - arch (dict) –
The parameter of RegNets.
- w0 (int): initial width
- wa (float): slope of width
- wm (float): quantization parameter to quantize the width
- depth (int): depth of the backbone
- group_w (int): width of group
- bot_mul (float): bottleneck ratio, i.e. expansion of bottleneck.
- strides (Sequence[int]) – Strides of the first block of each stage.
- base_channels (int) – Base channels after stem layer.
- in_channels (int) – Number of input image channels. Default: 3.
- dilations (Sequence[int]) – Dilation of each stage.
- out_indices (Sequence[int]) – Output from which stages.
- style (str) – pytorch or caffe. If set to “pytorch”, the stride-two layer is the 3x3 conv layer, otherwise the stride-two layer is the first 1x1 conv layer.
- frozen_stages (int) – Stages to be frozen (all param fixed). -1 means not freezing any parameters.
- norm_cfg (dict) – dictionary to construct and config norm layer.
- norm_eval (bool) – Whether to set norm layers to eval mode, namely, freeze running stats (mean and var). Note: Effect on Batch Norm and its variants only.
- with_cp (bool) – Use checkpoint or not. Using checkpoint will save some memory while slowing down the training speed.
- zero_init_residual (bool) – whether to use zero init for last norm layer in resblocks to let them behave as identity.
- pretrained (str, optional) – model pretrained path. Default: None
- init_cfg (dict or list[dict], optional) – Initialization config dict. Default: None
Example
>>> from mmdet.models import RegNet >>> import torch >>> self = RegNet( arch=dict( w0=88, wa=26.31, wm=2.25, group_w=48, depth=25, bot_mul=1.0)) >>> self.eval() >>> inputs = torch.rand(1, 3, 32, 32) >>> level_outputs = self.forward(inputs) >>> for level_out in level_outputs: ... print(tuple(level_out.shape)) (1, 96, 8, 8) (1, 192, 4, 4) (1, 432, 2, 2) (1, 1008, 1, 1)
-
adjust_width_group
(widths, bottleneck_ratio, groups)[source]¶ Adjusts the compatibility of widths and groups.
Parameters: - widths (list[int]) – Width of each stage.
- bottleneck_ratio (float) – Bottleneck ratio.
- groups (int) – number of groups in each stage
Returns: The adjusted widths and groups of each stage.
Return type: tuple(list)
-
generate_regnet
(initial_width, width_slope, width_parameter, depth, divisor=8)[source]¶ Generates per block width from RegNet parameters.
Parameters: - initial_width ([int]) – Initial width of the backbone
- width_slope ([float]) – Slope of the quantized linear function
- width_parameter ([int]) – Parameter used to quantize the width.
- depth ([int]) – Depth of the backbone.
- divisor (int, optional) – The divisor of channels. Defaults to 8.
Returns: return a list of widths of each stage and the number of stages
Return type: list, int
- arch (dict) –
-
class
mmdet.models.backbones.
ResNet
(depth, in_channels=3, stem_channels=None, base_channels=64, num_stages=4, strides=(1, 2, 2, 2), dilations=(1, 1, 1, 1), out_indices=(0, 1, 2, 3), style='pytorch', deep_stem=False, avg_down=False, frozen_stages=-1, conv_cfg=None, norm_cfg={'requires_grad': True, 'type': 'BN'}, norm_eval=True, dcn=None, stage_with_dcn=(False, False, False, False), plugins=None, with_cp=False, zero_init_residual=True, pretrained=None, init_cfg=None)[source]¶ ResNet backbone.
Parameters: - depth (int) – Depth of resnet, from {18, 34, 50, 101, 152}.
- stem_channels (int | None) – Number of stem channels. If not specified, it will be the same as base_channels. Default: None.
- base_channels (int) – Number of base channels of res layer. Default: 64.
- in_channels (int) – Number of input image channels. Default: 3.
- num_stages (int) – Resnet stages. Default: 4.
- strides (Sequence[int]) – Strides of the first block of each stage.
- dilations (Sequence[int]) – Dilation of each stage.
- out_indices (Sequence[int]) – Output from which stages.
- style (str) – pytorch or caffe. If set to “pytorch”, the stride-two layer is the 3x3 conv layer, otherwise the stride-two layer is the first 1x1 conv layer.
- deep_stem (bool) – Replace 7x7 conv in input stem with 3 3x3 conv
- avg_down (bool) – Use AvgPool instead of stride conv when downsampling in the bottleneck.
- frozen_stages (int) – Stages to be frozen (stop grad and set eval mode). -1 means not freezing any parameters.
- norm_cfg (dict) – Dictionary to construct and config norm layer.
- norm_eval (bool) – Whether to set norm layers to eval mode, namely, freeze running stats (mean and var). Note: Effect on Batch Norm and its variants only.
- plugins (list[dict]) –
List of plugins for stages, each dict contains:
- cfg (dict, required): Cfg dict to build plugin.
- position (str, required): Position inside block to insert plugin, options are ‘after_conv1’, ‘after_conv2’, ‘after_conv3’.
- stages (tuple[bool], optional): Stages to apply plugin, length should be same as ‘num_stages’.
- with_cp (bool) – Use checkpoint or not. Using checkpoint will save some memory while slowing down the training speed.
- zero_init_residual (bool) – Whether to use zero init for last norm layer in resblocks to let them behave as identity.
- pretrained (str, optional) – model pretrained path. Default: None
- init_cfg (dict or list[dict], optional) – Initialization config dict. Default: None
Example
>>> from mmdet.models import ResNet >>> import torch >>> self = ResNet(depth=18) >>> self.eval() >>> inputs = torch.rand(1, 3, 32, 32) >>> level_outputs = self.forward(inputs) >>> for level_out in level_outputs: ... print(tuple(level_out.shape)) (1, 64, 8, 8) (1, 128, 4, 4) (1, 256, 2, 2) (1, 512, 1, 1)
-
make_stage_plugins
(plugins, stage_idx)[source]¶ Make plugins for ResNet
stage_idx
th stage.Currently we support to insert
context_block
,empirical_attention_block
,nonlocal_block
into the backbone like ResNet/ResNeXt. They could be inserted after conv1/conv2/conv3 of Bottleneck.An example of plugins format could be:
Examples
>>> plugins=[ ... dict(cfg=dict(type='xxx', arg1='xxx'), ... stages=(False, True, True, True), ... position='after_conv2'), ... dict(cfg=dict(type='yyy'), ... stages=(True, True, True, True), ... position='after_conv3'), ... dict(cfg=dict(type='zzz', postfix='1'), ... stages=(True, True, True, True), ... position='after_conv3'), ... dict(cfg=dict(type='zzz', postfix='2'), ... stages=(True, True, True, True), ... position='after_conv3') ... ] >>> self = ResNet(depth=18) >>> stage_plugins = self.make_stage_plugins(plugins, 0) >>> assert len(stage_plugins) == 3
Suppose
stage_idx=0
, the structure of blocks in the stage would be:conv1-> conv2->conv3->yyy->zzz1->zzz2
Suppose ‘stage_idx=1’, the structure of blocks in the stage would be:
conv1-> conv2->xxx->conv3->yyy->zzz1->zzz2
If stages is missing, the plugin would be applied to all stages.
Parameters: - plugins (list[dict]) – List of plugins cfg to build. The postfix is required if multiple same type plugins are inserted.
- stage_idx (int) – Index of stage to build
Returns: Plugins for current stage
Return type: list[dict]
-
norm1
¶ the normalization layer named “norm1”
Type: nn.Module
-
class
mmdet.models.backbones.
ResNetV1d
(**kwargs)[source]¶ ResNetV1d variant described in Bag of Tricks.
Compared with default ResNet(ResNetV1b), ResNetV1d replaces the 7x7 conv in the input stem with three 3x3 convs. And in the downsampling block, a 2x2 avg_pool with stride 2 is added before conv, whose stride is changed to 1.
-
class
mmdet.models.backbones.
ResNeXt
(groups=1, base_width=4, **kwargs)[source]¶ ResNeXt backbone.
Parameters: - depth (int) – Depth of resnet, from {18, 34, 50, 101, 152}.
- in_channels (int) – Number of input image channels. Default: 3.
- num_stages (int) – Resnet stages. Default: 4.
- groups (int) – Group of resnext.
- base_width (int) – Base width of resnext.
- strides (Sequence[int]) – Strides of the first block of each stage.
- dilations (Sequence[int]) – Dilation of each stage.
- out_indices (Sequence[int]) – Output from which stages.
- style (str) – pytorch or caffe. If set to “pytorch”, the stride-two layer is the 3x3 conv layer, otherwise the stride-two layer is the first 1x1 conv layer.
- frozen_stages (int) – Stages to be frozen (all param fixed). -1 means not freezing any parameters.
- norm_cfg (dict) – dictionary to construct and config norm layer.
- norm_eval (bool) – Whether to set norm layers to eval mode, namely, freeze running stats (mean and var). Note: Effect on Batch Norm and its variants only.
- with_cp (bool) – Use checkpoint or not. Using checkpoint will save some memory while slowing down the training speed.
- zero_init_residual (bool) – whether to use zero init for last norm layer in resblocks to let them behave as identity.
-
class
mmdet.models.backbones.
SSDVGG
(depth, with_last_pool=False, ceil_mode=True, out_indices=(3, 4), out_feature_indices=(22, 34), pretrained=None, init_cfg=None, input_size=None, l2_norm_scale=None)[source]¶ VGG Backbone network for single-shot-detection.
Parameters: - depth (int) – Depth of vgg, from {11, 13, 16, 19}.
- with_last_pool (bool) – Whether to add a pooling layer at the last of the model
- ceil_mode (bool) – When True, will use ceil instead of floor to compute the output shape.
- out_indices (Sequence[int]) – Output from which stages.
- out_feature_indices (Sequence[int]) – Output from which feature map.
- pretrained (str, optional) – model pretrained path. Default: None
- init_cfg (dict or list[dict], optional) – Initialization config dict. Default: None
- input_size (int, optional) – Deprecated argumment. Width and height of input, from {300, 512}.
- l2_norm_scale (float, optional) – Deprecated argumment. L2 normalization layer init scale.
Example
>>> self = SSDVGG(input_size=300, depth=11) >>> self.eval() >>> inputs = torch.rand(1, 3, 300, 300) >>> level_outputs = self.forward(inputs) >>> for level_out in level_outputs: ... print(tuple(level_out.shape)) (1, 1024, 19, 19) (1, 512, 10, 10) (1, 256, 5, 5) (1, 256, 3, 3) (1, 256, 1, 1)
-
class
mmdet.models.backbones.
HRNet
(extra, in_channels=3, conv_cfg=None, norm_cfg={'type': 'BN'}, norm_eval=True, with_cp=False, zero_init_residual=False, pretrained=None, init_cfg=None)[source]¶ HRNet backbone.
High-Resolution Representations for Labeling Pixels and Regions arXiv: https://arxiv.org/abs/1904.04514
Parameters: - extra (dict) – detailed configuration for each stage of HRNet.
- in_channels (int) – Number of input image channels. Default: 3.
- conv_cfg (dict) – dictionary to construct and config conv layer.
- norm_cfg (dict) – dictionary to construct and config norm layer.
- norm_eval (bool) – Whether to set norm layers to eval mode, namely, freeze running stats (mean and var). Note: Effect on Batch Norm and its variants only.
- with_cp (bool) – Use checkpoint or not. Using checkpoint will save some memory while slowing down the training speed.
- zero_init_residual (bool) – whether to use zero init for last norm layer in resblocks to let them behave as identity.
- pretrained (str, optional) – model pretrained path. Default: None
- init_cfg (dict or list[dict], optional) – Initialization config dict. Default: None
Example
>>> from mmdet.models import HRNet >>> import torch >>> extra = dict( >>> stage1=dict( >>> num_modules=1, >>> num_branches=1, >>> block='BOTTLENECK', >>> num_blocks=(4, ), >>> num_channels=(64, )), >>> stage2=dict( >>> num_modules=1, >>> num_branches=2, >>> block='BASIC', >>> num_blocks=(4, 4), >>> num_channels=(32, 64)), >>> stage3=dict( >>> num_modules=4, >>> num_branches=3, >>> block='BASIC', >>> num_blocks=(4, 4, 4), >>> num_channels=(32, 64, 128)), >>> stage4=dict( >>> num_modules=3, >>> num_branches=4, >>> block='BASIC', >>> num_blocks=(4, 4, 4, 4), >>> num_channels=(32, 64, 128, 256))) >>> self = HRNet(extra, in_channels=1) >>> self.eval() >>> inputs = torch.rand(1, 1, 32, 32) >>> level_outputs = self.forward(inputs) >>> for level_out in level_outputs: ... print(tuple(level_out.shape)) (1, 32, 8, 8) (1, 64, 4, 4) (1, 128, 2, 2) (1, 256, 1, 1)
-
norm1
¶ the normalization layer named “norm1”
Type: nn.Module
-
norm2
¶ the normalization layer named “norm2”
Type: nn.Module
-
class
mmdet.models.backbones.
MobileNetV2
(widen_factor=1.0, out_indices=(1, 2, 4, 7), frozen_stages=-1, conv_cfg=None, norm_cfg={'type': 'BN'}, act_cfg={'type': 'ReLU6'}, norm_eval=False, with_cp=False, pretrained=None, init_cfg=None)[source]¶ MobileNetV2 backbone.
Parameters: - widen_factor (float) – Width multiplier, multiply number of channels in each layer by this amount. Default: 1.0.
- out_indices (Sequence[int], optional) – Output from which stages. Default: (1, 2, 4, 7).
- frozen_stages (int) – Stages to be frozen (all param fixed). Default: -1, which means not freezing any parameters.
- conv_cfg (dict, optional) – Config dict for convolution layer. Default: None, which means using conv2d.
- norm_cfg (dict) – Config dict for normalization layer. Default: dict(type=’BN’).
- act_cfg (dict) – Config dict for activation layer. Default: dict(type=’ReLU6’).
- norm_eval (bool) – Whether to set norm layers to eval mode, namely, freeze running stats (mean and var). Note: Effect on Batch Norm and its variants only. Default: False.
- with_cp (bool) – Use checkpoint or not. Using checkpoint will save some memory while slowing down the training speed. Default: False.
- pretrained (str, optional) – model pretrained path. Default: None
- init_cfg (dict or list[dict], optional) – Initialization config dict. Default: None
-
make_layer
(out_channels, num_blocks, stride, expand_ratio)[source]¶ Stack InvertedResidual blocks to build a layer for MobileNetV2.
Parameters: - out_channels (int) – out_channels of block.
- num_blocks (int) – number of blocks.
- stride (int) – stride of the first block. Default: 1
- expand_ratio (int) – Expand the number of channels of the hidden layer in InvertedResidual by this ratio. Default: 6.
-
class
mmdet.models.backbones.
Res2Net
(scales=4, base_width=26, style='pytorch', deep_stem=True, avg_down=True, pretrained=None, init_cfg=None, **kwargs)[source]¶ Res2Net backbone.
Parameters: - scales (int) – Scales used in Res2Net. Default: 4
- base_width (int) – Basic width of each scale. Default: 26
- depth (int) – Depth of res2net, from {50, 101, 152}.
- in_channels (int) – Number of input image channels. Default: 3.
- num_stages (int) – Res2net stages. Default: 4.
- strides (Sequence[int]) – Strides of the first block of each stage.
- dilations (Sequence[int]) – Dilation of each stage.
- out_indices (Sequence[int]) – Output from which stages.
- style (str) – pytorch or caffe. If set to “pytorch”, the stride-two layer is the 3x3 conv layer, otherwise the stride-two layer is the first 1x1 conv layer.
- deep_stem (bool) – Replace 7x7 conv in input stem with 3 3x3 conv
- avg_down (bool) – Use AvgPool instead of stride conv when downsampling in the bottle2neck.
- frozen_stages (int) – Stages to be frozen (stop grad and set eval mode). -1 means not freezing any parameters.
- norm_cfg (dict) – Dictionary to construct and config norm layer.
- norm_eval (bool) – Whether to set norm layers to eval mode, namely, freeze running stats (mean and var). Note: Effect on Batch Norm and its variants only.
- plugins (list[dict]) –
List of plugins for stages, each dict contains:
- cfg (dict, required): Cfg dict to build plugin.
- position (str, required): Position inside block to insert plugin, options are ‘after_conv1’, ‘after_conv2’, ‘after_conv3’.
- stages (tuple[bool], optional): Stages to apply plugin, length should be same as ‘num_stages’.
- with_cp (bool) – Use checkpoint or not. Using checkpoint will save some memory while slowing down the training speed.
- zero_init_residual (bool) – Whether to use zero init for last norm layer in resblocks to let them behave as identity.
- pretrained (str, optional) – model pretrained path. Default: None
- init_cfg (dict or list[dict], optional) – Initialization config dict. Default: None
Example
>>> from mmdet.models import Res2Net >>> import torch >>> self = Res2Net(depth=50, scales=4, base_width=26) >>> self.eval() >>> inputs = torch.rand(1, 3, 32, 32) >>> level_outputs = self.forward(inputs) >>> for level_out in level_outputs: ... print(tuple(level_out.shape)) (1, 256, 8, 8) (1, 512, 4, 4) (1, 1024, 2, 2) (1, 2048, 1, 1)
-
class
mmdet.models.backbones.
HourglassNet
(downsample_times=5, num_stacks=2, stage_channels=(256, 256, 384, 384, 384, 512), stage_blocks=(2, 2, 2, 2, 2, 4), feat_channel=256, norm_cfg={'requires_grad': True, 'type': 'BN'}, pretrained=None, init_cfg=None)[source]¶ HourglassNet backbone.
Stacked Hourglass Networks for Human Pose Estimation. More details can be found in the paper .
Parameters: - downsample_times (int) – Downsample times in a HourglassModule.
- num_stacks (int) – Number of HourglassModule modules stacked, 1 for Hourglass-52, 2 for Hourglass-104.
- stage_channels (list[int]) – Feature channel of each sub-module in a HourglassModule.
- stage_blocks (list[int]) – Number of sub-modules stacked in a HourglassModule.
- feat_channel (int) – Feature channel of conv after a HourglassModule.
- norm_cfg (dict) – Dictionary to construct and config norm layer.
- pretrained (str, optional) – model pretrained path. Default: None
- init_cfg (dict or list[dict], optional) – Initialization config dict. Default: None
Example
>>> from mmdet.models import HourglassNet >>> import torch >>> self = HourglassNet() >>> self.eval() >>> inputs = torch.rand(1, 3, 511, 511) >>> level_outputs = self.forward(inputs) >>> for level_output in level_outputs: ... print(tuple(level_output.shape)) (1, 256, 128, 128) (1, 256, 128, 128)
-
class
mmdet.models.backbones.
DetectoRS_ResNet
(sac=None, stage_with_sac=(False, False, False, False), rfp_inplanes=None, output_img=False, pretrained=None, init_cfg=None, **kwargs)[source]¶ ResNet backbone for DetectoRS.
Parameters: - sac (dict, optional) – Dictionary to construct SAC (Switchable Atrous Convolution). Default: None.
- stage_with_sac (list) – Which stage to use sac. Default: (False, False, False, False).
- rfp_inplanes (int, optional) – The number of channels from RFP.
Default: None. If specified, an additional conv layer will be
added for
rfp_feat
. Otherwise, the structure is the same as base class. - output_img (bool) – If
True
, the input image will be inserted into the starting position of output. Default: False.
-
class
mmdet.models.backbones.
DetectoRS_ResNeXt
(groups=1, base_width=4, **kwargs)[source]¶ ResNeXt backbone for DetectoRS.
Parameters: - groups (int) – The number of groups in ResNeXt.
- base_width (int) – The base width of ResNeXt.
-
class
mmdet.models.backbones.
Darknet
(depth=53, out_indices=(3, 4, 5), frozen_stages=-1, conv_cfg=None, norm_cfg={'requires_grad': True, 'type': 'BN'}, act_cfg={'negative_slope': 0.1, 'type': 'LeakyReLU'}, norm_eval=True, pretrained=None, init_cfg=None)[source]¶ Darknet backbone.
Parameters: - depth (int) – Depth of Darknet. Currently only support 53.
- out_indices (Sequence[int]) – Output from which stages.
- frozen_stages (int) – Stages to be frozen (stop grad and set eval mode). -1 means not freezing any parameters. Default: -1.
- conv_cfg (dict) – Config dict for convolution layer. Default: None.
- norm_cfg (dict) – Dictionary to construct and config norm layer. Default: dict(type=’BN’, requires_grad=True)
- act_cfg (dict) – Config dict for activation layer. Default: dict(type=’LeakyReLU’, negative_slope=0.1).
- norm_eval (bool) – Whether to set norm layers to eval mode, namely, freeze running stats (mean and var). Note: Effect on Batch Norm and its variants only.
- pretrained (str, optional) – model pretrained path. Default: None
- init_cfg (dict or list[dict], optional) – Initialization config dict. Default: None
Example
>>> from mmdet.models import Darknet >>> import torch >>> self = Darknet(depth=53) >>> self.eval() >>> inputs = torch.rand(1, 3, 416, 416) >>> level_outputs = self.forward(inputs) >>> for level_out in level_outputs: ... print(tuple(level_out.shape)) ... (1, 256, 52, 52) (1, 512, 26, 26) (1, 1024, 13, 13)
-
forward
(x)[source]¶ Defines the computation performed at every call.
Should be overridden by all subclasses.
Note
Although the recipe for forward pass needs to be defined within this function, one should call the
Module
instance afterwards instead of this since the former takes care of running the registered hooks while the latter silently ignores them.
-
static
make_conv_res_block
(in_channels, out_channels, res_repeat, conv_cfg=None, norm_cfg={'requires_grad': True, 'type': 'BN'}, act_cfg={'negative_slope': 0.1, 'type': 'LeakyReLU'})[source]¶ In Darknet backbone, ConvLayer is usually followed by ResBlock. This function will make that. The Conv layers always have 3x3 filters with stride=2. The number of the filters in Conv layer is the same as the out channels of the ResBlock.
Parameters: - in_channels (int) – The number of input channels.
- out_channels (int) – The number of output channels.
- res_repeat (int) – The number of ResBlocks.
- conv_cfg (dict) – Config dict for convolution layer. Default: None.
- norm_cfg (dict) – Dictionary to construct and config norm layer. Default: dict(type=’BN’, requires_grad=True)
- act_cfg (dict) – Config dict for activation layer. Default: dict(type=’LeakyReLU’, negative_slope=0.1).
-
train
(mode=True)[source]¶ Sets the module in training mode.
This has any effect only on certain modules. See documentations of particular modules for details of their behaviors in training/evaluation mode, if they are affected, e.g.
Dropout
,BatchNorm
, etc.Parameters: mode (bool) – whether to set training mode ( True
) or evaluation mode (False
). Default:True
.Returns: self Return type: Module
-
class
mmdet.models.backbones.
ResNeSt
(groups=1, base_width=4, radix=2, reduction_factor=4, avg_down_stride=True, **kwargs)[source]¶ ResNeSt backbone.
Parameters: - groups (int) – Number of groups of Bottleneck. Default: 1
- base_width (int) – Base width of Bottleneck. Default: 4
- radix (int) – Radix of SplitAttentionConv2d. Default: 2
- reduction_factor (int) – Reduction factor of inter_channels in SplitAttentionConv2d. Default: 4.
- avg_down_stride (bool) – Whether to use average pool for stride in Bottleneck. Default: True.
- kwargs (dict) – Keyword arguments for ResNet.
-
class
mmdet.models.backbones.
TridentResNet
(depth, num_branch, test_branch_idx, trident_dilations, **kwargs)[source]¶ The stem layer, stage 1 and stage 2 in Trident ResNet are identical to ResNet, while in stage 3, Trident BottleBlock is utilized to replace the normal BottleBlock to yield trident output. Different branch shares the convolution weight but uses different dilations to achieve multi-scale output.
/ stage3(b0) x - stem - stage1 - stage2 - stage3(b1) - output stage3(b2) /Parameters: - depth (int) – Depth of resnet, from {50, 101, 152}.
- num_branch (int) – Number of branches in TridentNet.
- test_branch_idx (int) – In inference, all 3 branches will be used if test_branch_idx==-1, otherwise only branch with index test_branch_idx will be used.
- trident_dilations (tuple[int]) – Dilations of different trident branch. len(trident_dilations) should be equal to num_branch.
-
class
mmdet.models.backbones.
CSPDarknet
(arch='P5', deepen_factor=1.0, widen_factor=1.0, out_indices=(2, 3, 4), frozen_stages=-1, use_depthwise=False, arch_ovewrite=None, spp_kernal_sizes=(5, 9, 13), conv_cfg=None, norm_cfg={'eps': 0.001, 'momentum': 0.03, 'type': 'BN'}, act_cfg={'type': 'Swish'}, norm_eval=False, init_cfg={'a': 2.23606797749979, 'distribution': 'uniform', 'layer': 'Conv2d', 'mode': 'fan_in', 'nonlinearity': 'leaky_relu', 'type': 'Kaiming'})[source]¶ CSP-Darknet backbone used in YOLOv5 and YOLOX.
Parameters: - arch (str) – Architechture of CSP-Darknet, from {P5, P6}. Default: P5.
- deepen_factor (float) – Depth multiplier, multiply number of channels in each layer by this amount. Default: 1.0.
- widen_factor (float) – Width multiplier, multiply number of blocks in CSP layer by this amount. Default: 1.0.
- out_indices (Sequence[int]) – Output from which stages. Default: (2, 3, 4).
- frozen_stages (int) – Stages to be frozen (stop grad and set eval mode). -1 means not freezing any parameters. Default: -1.
- use_depthwise (bool) – Whether to use depthwise separable convolution. Default: False.
- arch_ovewrite (list) – Overwrite default arch settings. Default: None.
- spp_kernal_sizes – (tuple[int]): Sequential of kernel sizes of SPP layers. Default: (5, 9, 13).
- conv_cfg (dict) – Config dict for convolution layer. Default: None.
- norm_cfg (dict) – Dictionary to construct and config norm layer. Default: dict(type=’BN’, requires_grad=True).
- act_cfg (dict) – Config dict for activation layer. Default: dict(type=’LeakyReLU’, negative_slope=0.1).
- norm_eval (bool) – Whether to set norm layers to eval mode, namely, freeze running stats (mean and var). Note: Effect on Batch Norm and its variants only.
- init_cfg (dict or list[dict], optional) – Initialization config dict. Default: None.
Example
>>> from mmdet.models import CSPDarknet >>> import torch >>> self = CSPDarknet(depth=53) >>> self.eval() >>> inputs = torch.rand(1, 3, 416, 416) >>> level_outputs = self.forward(inputs) >>> for level_out in level_outputs: ... print(tuple(level_out.shape)) ... (1, 256, 52, 52) (1, 512, 26, 26) (1, 1024, 13, 13)
-
forward
(x)[source]¶ Defines the computation performed at every call.
Should be overridden by all subclasses.
Note
Although the recipe for forward pass needs to be defined within this function, one should call the
Module
instance afterwards instead of this since the former takes care of running the registered hooks while the latter silently ignores them.
-
train
(mode=True)[source]¶ Sets the module in training mode.
This has any effect only on certain modules. See documentations of particular modules for details of their behaviors in training/evaluation mode, if they are affected, e.g.
Dropout
,BatchNorm
, etc.Parameters: mode (bool) – whether to set training mode ( True
) or evaluation mode (False
). Default:True
.Returns: self Return type: Module
necks¶
-
class
mmdet.models.necks.
FPN
(in_channels, out_channels, num_outs, start_level=0, end_level=-1, add_extra_convs=False, relu_before_extra_convs=False, no_norm_on_lateral=False, conv_cfg=None, norm_cfg=None, act_cfg=None, upsample_cfg={'mode': 'nearest'}, init_cfg={'distribution': 'uniform', 'layer': 'Conv2d', 'type': 'Xavier'})[source]¶ Feature Pyramid Network.
This is an implementation of paper Feature Pyramid Networks for Object Detection.
Parameters: - in_channels (List[int]) – Number of input channels per scale.
- out_channels (int) – Number of output channels (used at each scale)
- num_outs (int) – Number of output scales.
- start_level (int) – Index of the start input backbone level used to build the feature pyramid. Default: 0.
- end_level (int) – Index of the end input backbone level (exclusive) to build the feature pyramid. Default: -1, which means the last level.
- add_extra_convs (bool | str) –
If bool, it decides whether to add conv layers on top of the original feature maps. Default to False. If True, it is equivalent to add_extra_convs=’on_input’. If str, it specifies the source feature map of the extra convs. Only the following options are allowed
- ’on_input’: Last feat map of neck inputs (i.e. backbone feature).
- ’on_lateral’: Last feature map after lateral convs.
- ’on_output’: The last output feature map after fpn convs.
- relu_before_extra_convs (bool) – Whether to apply relu before the extra conv. Default: False.
- no_norm_on_lateral (bool) – Whether to apply norm on lateral. Default: False.
- conv_cfg (dict) – Config dict for convolution layer. Default: None.
- norm_cfg (dict) – Config dict for normalization layer. Default: None.
- act_cfg (str) – Config dict for activation layer in ConvModule. Default: None.
- upsample_cfg (dict) – Config dict for interpolate layer. Default: dict(mode=’nearest’)
- init_cfg (dict or list[dict], optional) – Initialization config dict.
Example
>>> import torch >>> in_channels = [2, 3, 5, 7] >>> scales = [340, 170, 84, 43] >>> inputs = [torch.rand(1, c, s, s) ... for c, s in zip(in_channels, scales)] >>> self = FPN(in_channels, 11, len(in_channels)).eval() >>> outputs = self.forward(inputs) >>> for i in range(len(outputs)): ... print(f'outputs[{i}].shape = {outputs[i].shape}') outputs[0].shape = torch.Size([1, 11, 340, 340]) outputs[1].shape = torch.Size([1, 11, 170, 170]) outputs[2].shape = torch.Size([1, 11, 84, 84]) outputs[3].shape = torch.Size([1, 11, 43, 43])
-
class
mmdet.models.necks.
BFP
(Balanced Feature Pyramids)[source]¶ BFP takes multi-level features as inputs and gather them into a single one, then refine the gathered feature and scatter the refined results to multi-level features. This module is used in Libra R-CNN (CVPR 2019), see the paper Libra R-CNN: Towards Balanced Learning for Object Detection for details.
Parameters: - in_channels (int) – Number of input channels (feature maps of all levels should have the same channels).
- num_levels (int) – Number of input feature levels.
- conv_cfg (dict) – The config dict for convolution layers.
- norm_cfg (dict) – The config dict for normalization layers.
- refine_level (int) – Index of integration and refine level of BSF in multi-level features from bottom to top.
- refine_type (str) – Type of the refine op, currently support [None, ‘conv’, ‘non_local’].
- init_cfg (dict or list[dict], optional) – Initialization config dict.
-
class
mmdet.models.necks.
ChannelMapper
(in_channels, out_channels, kernel_size=3, conv_cfg=None, norm_cfg=None, act_cfg={'type': 'ReLU'}, num_outs=None, init_cfg={'distribution': 'uniform', 'layer': 'Conv2d', 'type': 'Xavier'})[source]¶ Channel Mapper to reduce/increase channels of backbone features.
This is used to reduce/increase channels of backbone features.
Parameters: - in_channels (List[int]) – Number of input channels per scale.
- out_channels (int) – Number of output channels (used at each scale).
- kernel_size (int, optional) – kernel_size for reducing channels (used at each scale). Default: 3.
- conv_cfg (dict, optional) – Config dict for convolution layer. Default: None.
- norm_cfg (dict, optional) – Config dict for normalization layer. Default: None.
- act_cfg (dict, optional) – Config dict for activation layer in ConvModule. Default: dict(type=’ReLU’).
- num_outs (int, optional) – Number of output feature maps. There would be extra_convs when num_outs larger than the length of in_channels.
- init_cfg (dict or list[dict], optional) – Initialization config dict.
Example
>>> import torch >>> in_channels = [2, 3, 5, 7] >>> scales = [340, 170, 84, 43] >>> inputs = [torch.rand(1, c, s, s) ... for c, s in zip(in_channels, scales)] >>> self = ChannelMapper(in_channels, 11, 3).eval() >>> outputs = self.forward(inputs) >>> for i in range(len(outputs)): ... print(f'outputs[{i}].shape = {outputs[i].shape}') outputs[0].shape = torch.Size([1, 11, 340, 340]) outputs[1].shape = torch.Size([1, 11, 170, 170]) outputs[2].shape = torch.Size([1, 11, 84, 84]) outputs[3].shape = torch.Size([1, 11, 43, 43])
-
class
mmdet.models.necks.
HRFPN
(High Resolution Feature Pyramids)[source]¶ paper: High-Resolution Representations for Labeling Pixels and Regions.
Parameters: - in_channels (list) – number of channels for each branch.
- out_channels (int) – output channels of feature pyramids.
- num_outs (int) – number of output stages.
- pooling_type (str) – pooling for generating feature pyramids from {MAX, AVG}.
- conv_cfg (dict) – dictionary to construct and config conv layer.
- norm_cfg (dict) – dictionary to construct and config norm layer.
- with_cp (bool) – Use checkpoint or not. Using checkpoint will save some memory while slowing down the training speed.
- stride (int) – stride of 3x3 convolutional layers
- init_cfg (dict or list[dict], optional) – Initialization config dict.
-
class
mmdet.models.necks.
NASFPN
(in_channels, out_channels, num_outs, stack_times, start_level=0, end_level=-1, add_extra_convs=False, norm_cfg=None, init_cfg={'layer': 'Conv2d', 'type': 'Caffe2Xavier'})[source]¶ NAS-FPN.
Implementation of NAS-FPN: Learning Scalable Feature Pyramid Architecture for Object Detection
Parameters: - in_channels (List[int]) – Number of input channels per scale.
- out_channels (int) – Number of output channels (used at each scale)
- num_outs (int) – Number of output scales.
- stack_times (int) – The number of times the pyramid architecture will be stacked.
- start_level (int) – Index of the start input backbone level used to build the feature pyramid. Default: 0.
- end_level (int) – Index of the end input backbone level (exclusive) to build the feature pyramid. Default: -1, which means the last level.
- add_extra_convs (bool) – It decides whether to add conv layers on top of the original feature maps. Default to False. If True, its actual mode is specified by extra_convs_on_inputs.
- init_cfg (dict or list[dict], optional) – Initialization config dict.
-
class
mmdet.models.necks.
FPN_CARAFE
(in_channels, out_channels, num_outs, start_level=0, end_level=-1, norm_cfg=None, act_cfg=None, order=('conv', 'norm', 'act'), upsample_cfg={'encoder_dilation': 1, 'encoder_kernel': 3, 'type': 'carafe', 'up_group': 1, 'up_kernel': 5}, init_cfg=None)[source]¶ FPN_CARAFE is a more flexible implementation of FPN. It allows more choice for upsample methods during the top-down pathway.
It can reproduce the performance of ICCV 2019 paper CARAFE: Content-Aware ReAssembly of FEatures Please refer to https://arxiv.org/abs/1905.02188 for more details.
Parameters: - in_channels (list[int]) – Number of channels for each input feature map.
- out_channels (int) – Output channels of feature pyramids.
- num_outs (int) – Number of output stages.
- start_level (int) – Start level of feature pyramids. (Default: 0)
- end_level (int) – End level of feature pyramids. (Default: -1 indicates the last level).
- norm_cfg (dict) – Dictionary to construct and config norm layer.
- activate (str) – Type of activation function in ConvModule (Default: None indicates w/o activation).
- order (dict) – Order of components in ConvModule.
- upsample (str) – Type of upsample layer.
- upsample_cfg (dict) – Dictionary to construct and config upsample layer.
- init_cfg (dict or list[dict], optional) – Initialization config dict. Default: None
-
class
mmdet.models.necks.
PAFPN
(in_channels, out_channels, num_outs, start_level=0, end_level=-1, add_extra_convs=False, relu_before_extra_convs=False, no_norm_on_lateral=False, conv_cfg=None, norm_cfg=None, act_cfg=None, init_cfg={'distribution': 'uniform', 'layer': 'Conv2d', 'type': 'Xavier'})[source]¶ Path Aggregation Network for Instance Segmentation.
This is an implementation of the PAFPN in Path Aggregation Network.
Parameters: - in_channels (List[int]) – Number of input channels per scale.
- out_channels (int) – Number of output channels (used at each scale)
- num_outs (int) – Number of output scales.
- start_level (int) – Index of the start input backbone level used to build the feature pyramid. Default: 0.
- end_level (int) – Index of the end input backbone level (exclusive) to build the feature pyramid. Default: -1, which means the last level.
- add_extra_convs (bool | str) –
If bool, it decides whether to add conv layers on top of the original feature maps. Default to False. If True, it is equivalent to add_extra_convs=’on_input’. If str, it specifies the source feature map of the extra convs. Only the following options are allowed
- ’on_input’: Last feat map of neck inputs (i.e. backbone feature).
- ’on_lateral’: Last feature map after lateral convs.
- ’on_output’: The last output feature map after fpn convs.
- relu_before_extra_convs (bool) – Whether to apply relu before the extra conv. Default: False.
- no_norm_on_lateral (bool) – Whether to apply norm on lateral. Default: False.
- conv_cfg (dict) – Config dict for convolution layer. Default: None.
- norm_cfg (dict) – Config dict for normalization layer. Default: None.
- act_cfg (str) – Config dict for activation layer in ConvModule. Default: None.
- init_cfg (dict or list[dict], optional) – Initialization config dict.
-
class
mmdet.models.necks.
NASFCOS_FPN
(in_channels, out_channels, num_outs, start_level=1, end_level=-1, add_extra_convs=False, conv_cfg=None, norm_cfg=None, init_cfg=None)[source]¶ FPN structure in NASFPN.
Implementation of paper NAS-FCOS: Fast Neural Architecture Search for Object Detection
Parameters: - in_channels (List[int]) – Number of input channels per scale.
- out_channels (int) – Number of output channels (used at each scale)
- num_outs (int) – Number of output scales.
- start_level (int) – Index of the start input backbone level used to build the feature pyramid. Default: 0.
- end_level (int) – Index of the end input backbone level (exclusive) to build the feature pyramid. Default: -1, which means the last level.
- add_extra_convs (bool) – It decides whether to add conv layers on top of the original feature maps. Default to False. If True, its actual mode is specified by extra_convs_on_inputs.
- conv_cfg (dict) – dictionary to construct and config conv layer.
- norm_cfg (dict) – dictionary to construct and config norm layer.
- init_cfg (dict or list[dict], optional) – Initialization config dict. Default: None
-
class
mmdet.models.necks.
RFP
(Recursive Feature Pyramid)[source]¶ This is an implementation of RFP in DetectoRS. Different from standard FPN, the input of RFP should be multi level features along with origin input image of backbone.
Parameters: - rfp_steps (int) – Number of unrolled steps of RFP.
- rfp_backbone (dict) – Configuration of the backbone for RFP.
- aspp_out_channels (int) – Number of output channels of ASPP module.
- aspp_dilations (tuple[int]) – Dilation rates of four branches. Default: (1, 3, 6, 1)
- init_cfg (dict or list[dict], optional) – Initialization config dict. Default: None
-
class
mmdet.models.necks.
YOLOV3Neck
(num_scales, in_channels, out_channels, conv_cfg=None, norm_cfg={'requires_grad': True, 'type': 'BN'}, act_cfg={'negative_slope': 0.1, 'type': 'LeakyReLU'}, init_cfg=None)[source]¶ The neck of YOLOV3.
It can be treated as a simplified version of FPN. It will take the result from Darknet backbone and do some upsampling and concatenation. It will finally output the detection result.
Note
- The input feats should be from top to bottom.
- i.e., from high-lvl to low-lvl
- But YOLOV3Neck will process them in reversed order.
- i.e., from bottom (high-lvl) to top (low-lvl)
Parameters: - num_scales (int) – The number of scales / stages.
- in_channels (List[int]) – The number of input channels per scale.
- out_channels (List[int]) – The number of output channels per scale.
- conv_cfg (dict, optional) – Config dict for convolution layer. Default: None.
- norm_cfg (dict, optional) – Dictionary to construct and config norm layer. Default: dict(type=’BN’, requires_grad=True)
- act_cfg (dict, optional) – Config dict for activation layer. Default: dict(type=’LeakyReLU’, negative_slope=0.1).
- init_cfg (dict or list[dict], optional) – Initialization config dict. Default: None
-
forward
(feats)[source]¶ Defines the computation performed at every call.
Should be overridden by all subclasses.
Note
Although the recipe for forward pass needs to be defined within this function, one should call the
Module
instance afterwards instead of this since the former takes care of running the registered hooks while the latter silently ignores them.
-
class
mmdet.models.necks.
FPG
(in_channels, out_channels, num_outs, stack_times, paths, inter_channels=None, same_down_trans=None, same_up_trans={'kernel_size': 3, 'padding': 1, 'stride': 2, 'type': 'conv'}, across_lateral_trans={'kernel_size': 1, 'type': 'conv'}, across_down_trans={'kernel_size': 3, 'type': 'conv'}, across_up_trans=None, across_skip_trans={'type': 'identity'}, output_trans={'kernel_size': 3, 'type': 'last_conv'}, start_level=0, end_level=-1, add_extra_convs=False, norm_cfg=None, skip_inds=None, init_cfg=[{'type': 'Caffe2Xavier', 'layer': 'Conv2d'}, {'type': 'Constant', 'layer': ['_BatchNorm', '_InstanceNorm', 'GroupNorm', 'LayerNorm'], 'val': 1.0}])[source]¶ FPG.
Implementation of Feature Pyramid Grids (FPG). This implementation only gives the basic structure stated in the paper. But users can implement different type of transitions to fully explore the the potential power of the structure of FPG.
Parameters: - in_channels (int) – Number of input channels (feature maps of all levels should have the same channels).
- out_channels (int) – Number of output channels (used at each scale)
- num_outs (int) – Number of output scales.
- stack_times (int) – The number of times the pyramid architecture will be stacked.
- paths (list[str]) – Specify the path order of each stack level. Each element in the list should be either ‘bu’ (bottom-up) or ‘td’ (top-down).
- inter_channels (int) – Number of inter channels.
- same_up_trans (dict) – Transition that goes down at the same stage.
- same_down_trans (dict) – Transition that goes up at the same stage.
- across_lateral_trans (dict) – Across-pathway same-stage
- across_down_trans (dict) – Across-pathway bottom-up connection.
- across_up_trans (dict) – Across-pathway top-down connection.
- across_skip_trans (dict) – Across-pathway skip connection.
- output_trans (dict) – Transition that trans the output of the last stage.
- start_level (int) – Index of the start input backbone level used to build the feature pyramid. Default: 0.
- end_level (int) – Index of the end input backbone level (exclusive) to build the feature pyramid. Default: -1, which means the last level.
- add_extra_convs (bool) – It decides whether to add conv layers on top of the original feature maps. Default to False. If True, its actual mode is specified by extra_convs_on_inputs.
- norm_cfg (dict) – Config dict for normalization layer. Default: None.
- init_cfg (dict or list[dict], optional) – Initialization config dict.
-
forward
(inputs)[source]¶ Defines the computation performed at every call.
Should be overridden by all subclasses.
Note
Although the recipe for forward pass needs to be defined within this function, one should call the
Module
instance afterwards instead of this since the former takes care of running the registered hooks while the latter silently ignores them.
-
class
mmdet.models.necks.
DilatedEncoder
(in_channels, out_channels, block_mid_channels, num_residual_blocks)[source]¶ Dilated Encoder for YOLOF <https://arxiv.org/abs/2103.09460>`.
- This module contains two types of components:
- the original FPN lateral convolution layer and fpn convolution layer,
- which are 1x1 conv + 3x3 conv
- the dilated residual block
Parameters: - in_channels (int) – The number of input channels.
- out_channels (int) – The number of output channels.
- block_mid_channels (int) – The number of middle block output channels
- num_residual_blocks (int) – The number of residual blocks.
-
forward
(feature)[source]¶ Defines the computation performed at every call.
Should be overridden by all subclasses.
Note
Although the recipe for forward pass needs to be defined within this function, one should call the
Module
instance afterwards instead of this since the former takes care of running the registered hooks while the latter silently ignores them.
-
class
mmdet.models.necks.
CTResNetNeck
(in_channel, num_deconv_filters, num_deconv_kernels, use_dcn=True, init_cfg=None)[source]¶ The neck used in CenterNet for object classification and box regression.
Parameters: - in_channel (int) – Number of input channels.
- num_deconv_filters (tuple[int]) – Number of filters per stage.
- num_deconv_kernels (tuple[int]) – Number of kernels per stage.
- use_dcn (bool) – If True, use DCNv2. Default: True.
- init_cfg (dict or list[dict], optional) – Initialization config dict.
-
forward
(inputs)[source]¶ Defines the computation performed at every call.
Should be overridden by all subclasses.
Note
Although the recipe for forward pass needs to be defined within this function, one should call the
Module
instance afterwards instead of this since the former takes care of running the registered hooks while the latter silently ignores them.
-
class
mmdet.models.necks.
SSDNeck
(in_channels, out_channels, level_strides, level_paddings, l2_norm_scale=20.0, last_kernel_size=3, use_depthwise=False, conv_cfg=None, norm_cfg=None, act_cfg={'type': 'ReLU'}, init_cfg=[{'type': 'Xavier', 'distribution': 'uniform', 'layer': 'Conv2d'}, {'type': 'Constant', 'val': 1, 'layer': 'BatchNorm2d'}])[source]¶ Extra layers of SSD backbone to generate multi-scale feature maps.
Parameters: - in_channels (Sequence[int]) – Number of input channels per scale.
- out_channels (Sequence[int]) – Number of output channels per scale.
- level_strides (Sequence[int]) – Stride of 3x3 conv per level.
- level_paddings (Sequence[int]) – Padding size of 3x3 conv per level.
- l2_norm_scale (float|None) – L2 normalization layer init scale. If None, not use L2 normalization on the first input feature.
- last_kernel_size (int) – Kernel size of the last conv layer. Default: 3.
- use_depthwise (bool) – Whether to use DepthwiseSeparableConv. Default: False.
- conv_cfg (dict) – Config dict for convolution layer. Default: None.
- norm_cfg (dict) – Dictionary to construct and config norm layer. Default: None.
- act_cfg (dict) – Config dict for activation layer. Default: dict(type=’ReLU’).
- init_cfg (dict or list[dict], optional) – Initialization config dict.
-
class
mmdet.models.necks.
YOLOXPAFPN
(in_channels, out_channels, num_csp_blocks=3, use_depthwise=False, upsample_cfg={'mode': 'nearest', 'scale_factor': 2}, conv_cfg=None, norm_cfg={'eps': 0.001, 'momentum': 0.03, 'type': 'BN'}, act_cfg={'type': 'Swish'}, init_cfg={'a': 2.23606797749979, 'distribution': 'uniform', 'layer': 'Conv2d', 'mode': 'fan_in', 'nonlinearity': 'leaky_relu', 'type': 'Kaiming'})[source]¶ Path Aggregation Network used in YOLOX.
Parameters: - in_channels (List[int]) – Number of input channels per scale.
- out_channels (int) – Number of output channels (used at each scale)
- num_csp_blocks (int) – Number of bottlenecks in CSPLayer. Default: 3
- use_depthwise (bool) – Whether to depthwise separable convolution in blocks. Default: False
- upsample_cfg (dict) – Config dict for interpolate layer. Default: dict(scale_factor=2, mode=’nearest’)
- conv_cfg (dict, optional) – Config dict for convolution layer. Default: None, which means using conv2d.
- norm_cfg (dict) – Config dict for normalization layer. Default: dict(type=’BN’)
- act_cfg (dict) – Config dict for activation layer. Default: dict(type=’Swish’)
- init_cfg (dict or list[dict], optional) – Initialization config dict. Default: None.
dense_heads¶
-
class
mmdet.models.dense_heads.
AnchorFreeHead
(num_classes, in_channels, feat_channels=256, stacked_convs=4, strides=(4, 8, 16, 32, 64), dcn_on_last_conv=False, conv_bias='auto', loss_cls={'alpha': 0.25, 'gamma': 2.0, 'loss_weight': 1.0, 'type': 'FocalLoss', 'use_sigmoid': True}, loss_bbox={'loss_weight': 1.0, 'type': 'IoULoss'}, conv_cfg=None, norm_cfg=None, train_cfg=None, test_cfg=None, init_cfg={'layer': 'Conv2d', 'override': {'bias_prob': 0.01, 'name': 'conv_cls', 'std': 0.01, 'type': 'Normal'}, 'std': 0.01, 'type': 'Normal'})[source]¶ Anchor-free head (FCOS, Fovea, RepPoints, etc.).
Parameters: - num_classes (int) – Number of categories excluding the background category.
- in_channels (int) – Number of channels in the input feature map.
- feat_channels (int) – Number of hidden channels. Used in child classes.
- stacked_convs (int) – Number of stacking convs of the head.
- strides (tuple) – Downsample factor of each feature map.
- dcn_on_last_conv (bool) – If true, use dcn in the last layer of towers. Default: False.
- conv_bias (bool | str) – If specified as auto, it will be decided by the norm_cfg. Bias of conv will be set as True if norm_cfg is None, otherwise False. Default: “auto”.
- loss_cls (dict) – Config of classification loss.
- loss_bbox (dict) – Config of localization loss.
- conv_cfg (dict) – Config dict for convolution layer. Default: None.
- norm_cfg (dict) – Config dict for normalization layer. Default: None.
- train_cfg (dict) – Training config of anchor head.
- test_cfg (dict) – Testing config of anchor head.
- init_cfg (dict or list[dict], optional) – Initialization config dict.
-
aug_test
(feats, img_metas, rescale=False)[source]¶ Test function with test time augmentation.
Parameters: - feats (list[Tensor]) – the outer list indicates test-time augmentations and inner Tensor should have a shape NxCxHxW, which contains features for all images in the batch.
- img_metas (list[list[dict]]) – the outer list indicates test-time augs (multiscale, flip, etc.) and the inner list indicates images in a batch. each dict has image information.
- rescale (bool, optional) – Whether to rescale the results. Defaults to False.
Returns: bbox results of each class
Return type: list[ndarray]
-
forward
(feats)[source]¶ Forward features from the upstream network.
Parameters: feats (tuple[Tensor]) – Features from the upstream network, each is a 4D-tensor. Returns: - Usually contain classification scores and bbox predictions.
- cls_scores (list[Tensor]): Box scores for each scale level,
- each is a 4D-tensor, the channel number is num_points * num_classes.
- bbox_preds (list[Tensor]): Box energies / deltas for each scale
- level, each is a 4D-tensor, the channel number is num_points * 4.
Return type: tuple
-
forward_single
(x)[source]¶ Forward features of a single scale level.
Parameters: x (Tensor) – FPN feature maps of the specified stride. Returns: - Scores for each class, bbox predictions, features
- after classification and regression conv layers, some models needs these features like FCOS.
Return type: tuple
-
get_bboxes
(cls_scores, bbox_preds, img_metas, cfg=None, rescale=None)[source]¶ Transform network output for a batch into bbox predictions.
Parameters: - cls_scores (list[Tensor]) – Box scores for each scale level Has shape (N, num_points * num_classes, H, W)
- bbox_preds (list[Tensor]) – Box energies / deltas for each scale level with shape (N, num_points * 4, H, W)
- img_metas (list[dict]) – Meta information of each image, e.g., image size, scaling factor, etc.
- cfg (mmcv.Config) – Test / postprocessing configuration, if None, test_cfg would be used
- rescale (bool) – If True, return boxes in original image space
-
get_points
(featmap_sizes, dtype, device, flatten=False)[source]¶ Get points according to feature map sizes.
Parameters: - featmap_sizes (list[tuple]) – Multi-level feature map sizes.
- dtype (torch.dtype) – Type of points.
- device (torch.device) – Device of points.
Returns: points of each image.
Return type: tuple
-
get_targets
(points, gt_bboxes_list, gt_labels_list)[source]¶ Compute regression, classification and centerness targets for points in multiple images.
Parameters: - points (list[Tensor]) – Points of each fpn level, each has shape (num_points, 2).
- gt_bboxes_list (list[Tensor]) – Ground truth bboxes of each image, each has shape (num_gt, 4).
- gt_labels_list (list[Tensor]) – Ground truth labels of each box, each has shape (num_gt,).
-
loss
(cls_scores, bbox_preds, gt_bboxes, gt_labels, img_metas, gt_bboxes_ignore=None)[source]¶ Compute loss of the head.
Parameters: - cls_scores (list[Tensor]) – Box scores for each scale level, each is a 4D-tensor, the channel number is num_points * num_classes.
- bbox_preds (list[Tensor]) – Box energies / deltas for each scale level, each is a 4D-tensor, the channel number is num_points * 4.
- gt_bboxes (list[Tensor]) – Ground truth bboxes for each image with shape (num_gts, 4) in [tl_x, tl_y, br_x, br_y] format.
- gt_labels (list[Tensor]) – class indices corresponding to each box
- img_metas (list[dict]) – Meta information of each image, e.g., image size, scaling factor, etc.
- gt_bboxes_ignore (None | list[Tensor]) – specify which bounding boxes can be ignored when computing the loss.
-
class
mmdet.models.dense_heads.
AnchorHead
(num_classes, in_channels, feat_channels=256, anchor_generator={'ratios': [0.5, 1.0, 2.0], 'scales': [8, 16, 32], 'strides': [4, 8, 16, 32, 64], 'type': 'AnchorGenerator'}, bbox_coder={'clip_border': True, 'target_means': (0.0, 0.0, 0.0, 0.0), 'target_stds': (1.0, 1.0, 1.0, 1.0), 'type': 'DeltaXYWHBBoxCoder'}, reg_decoded_bbox=False, loss_cls={'loss_weight': 1.0, 'type': 'CrossEntropyLoss', 'use_sigmoid': True}, loss_bbox={'beta': 0.1111111111111111, 'loss_weight': 1.0, 'type': 'SmoothL1Loss'}, train_cfg=None, test_cfg=None, init_cfg={'layer': 'Conv2d', 'std': 0.01, 'type': 'Normal'})[source]¶ Anchor-based head (RPN, RetinaNet, SSD, etc.).
Parameters: - num_classes (int) – Number of categories excluding the background category.
- in_channels (int) – Number of channels in the input feature map.
- feat_channels (int) – Number of hidden channels. Used in child classes.
- anchor_generator (dict) – Config dict for anchor generator
- bbox_coder (dict) – Config of bounding box coder.
- reg_decoded_bbox (bool) – If true, the regression loss would be applied directly on decoded bounding boxes, converting both the predicted boxes and regression targets to absolute coordinates format. Default False. It should be True when using IoULoss, GIoULoss, or DIoULoss in the bbox head.
- loss_cls (dict) – Config of classification loss.
- loss_bbox (dict) – Config of localization loss.
- train_cfg (dict) – Training config of anchor head.
- test_cfg (dict) – Testing config of anchor head.
- init_cfg (dict or list[dict], optional) – Initialization config dict.
-
aug_test
(feats, img_metas, rescale=False)[source]¶ Test function with test time augmentation.
Parameters: - feats (list[Tensor]) – the outer list indicates test-time augmentations and inner Tensor should have a shape NxCxHxW, which contains features for all images in the batch.
- img_metas (list[list[dict]]) – the outer list indicates test-time augs (multiscale, flip, etc.) and the inner list indicates images in a batch. each dict has image information.
- rescale (bool, optional) – Whether to rescale the results. Defaults to False.
Returns: - Each item in result_list is 2-tuple.
The first item is
bboxes
with shape (n, 5), where 5 represent (tl_x, tl_y, br_x, br_y, score). The shape of the second tensor in the tuple islabels
with shape (n,), The length of list should always be 1.
Return type: list[tuple[Tensor, Tensor]]
-
forward
(feats)[source]¶ Forward features from the upstream network.
Parameters: feats (tuple[Tensor]) – Features from the upstream network, each is a 4D-tensor. Returns: A tuple of classification scores and bbox prediction. - cls_scores (list[Tensor]): Classification scores for all scale levels, each is a 4D-tensor, the channels number is num_anchors * num_classes.
- bbox_preds (list[Tensor]): Box energies / deltas for all scale levels, each is a 4D-tensor, the channels number is num_anchors * 4.
Return type: tuple
-
forward_single
(x)[source]¶ Forward feature of a single scale level.
Parameters: x (Tensor) – Features of a single scale level. Returns: cls_score (Tensor): Cls scores for a single scale level the channels number is num_anchors * num_classes. bbox_pred (Tensor): Box energies / deltas for a single scale level, the channels number is num_anchors * 4. Return type: tuple
-
get_anchors
(featmap_sizes, img_metas, device='cuda')[source]¶ Get anchors according to feature map sizes.
Parameters: - featmap_sizes (list[tuple]) – Multi-level feature map sizes.
- img_metas (list[dict]) – Image meta info.
- device (torch.device | str) – Device for returned tensors
Returns: anchor_list (list[Tensor]): Anchors of each image. valid_flag_list (list[Tensor]): Valid flags of each image.
Return type: tuple
-
get_bboxes
(cls_scores, bbox_preds, img_metas, cfg=None, rescale=False, with_nms=True)[source]¶ Transform network output for a batch into bbox predictions.
Parameters: - cls_scores (list[Tensor]) – Box scores for each level in the feature pyramid, has shape (N, num_anchors * num_classes, H, W).
- bbox_preds (list[Tensor]) – Box energies / deltas for each level in the feature pyramid, has shape (N, num_anchors * 4, H, W).
- img_metas (list[dict]) – Meta information of each image, e.g., image size, scaling factor, etc.
- cfg (mmcv.Config | None) – Test / postprocessing configuration, if None, test_cfg would be used
- rescale (bool) – If True, return boxes in original image space. Default: False.
- with_nms (bool) – If True, do nms before return boxes. Default: True.
Returns: - Each item in result_list is 2-tuple.
The first item is an (n, 5) tensor, where 5 represent (tl_x, tl_y, br_x, br_y, score) and the score between 0 and 1. The shape of the second tensor in the tuple is (n,), and each element represents the class label of the corresponding box.
Return type: list[tuple[Tensor, Tensor]]
Example
>>> import mmcv >>> self = AnchorHead( >>> num_classes=9, >>> in_channels=1, >>> anchor_generator=dict( >>> type='AnchorGenerator', >>> scales=[8], >>> ratios=[0.5, 1.0, 2.0], >>> strides=[4,])) >>> img_metas = [{'img_shape': (32, 32, 3), 'scale_factor': 1}] >>> cfg = mmcv.Config(dict( >>> score_thr=0.00, >>> nms=dict(type='nms', iou_thr=1.0), >>> max_per_img=10)) >>> feat = torch.rand(1, 1, 3, 3) >>> cls_score, bbox_pred = self.forward_single(feat) >>> # note the input lists are over different levels, not images >>> cls_scores, bbox_preds = [cls_score], [bbox_pred] >>> result_list = self.get_bboxes(cls_scores, bbox_preds, >>> img_metas, cfg) >>> det_bboxes, det_labels = result_list[0] >>> assert len(result_list) == 1 >>> assert det_bboxes.shape[1] == 5 >>> assert len(det_bboxes) == len(det_labels) == cfg.max_per_img
-
get_targets
(anchor_list, valid_flag_list, gt_bboxes_list, img_metas, gt_bboxes_ignore_list=None, gt_labels_list=None, label_channels=1, unmap_outputs=True, return_sampling_results=False)[source]¶ Compute regression and classification targets for anchors in multiple images.
Parameters: - anchor_list (list[list[Tensor]]) – Multi level anchors of each image. The outer list indicates images, and the inner list corresponds to feature levels of the image. Each element of the inner list is a tensor of shape (num_anchors, 4).
- valid_flag_list (list[list[Tensor]]) – Multi level valid flags of each image. The outer list indicates images, and the inner list corresponds to feature levels of the image. Each element of the inner list is a tensor of shape (num_anchors, )
- gt_bboxes_list (list[Tensor]) – Ground truth bboxes of each image.
- img_metas (list[dict]) – Meta info of each image.
- gt_bboxes_ignore_list (list[Tensor]) – Ground truth bboxes to be ignored.
- gt_labels_list (list[Tensor]) – Ground truth labels of each box.
- label_channels (int) – Channel of label.
- unmap_outputs (bool) – Whether to map outputs back to the original set of anchors.
Returns: Usually returns a tuple containing learning targets.
- labels_list (list[Tensor]): Labels of each level.
- label_weights_list (list[Tensor]): Label weights of each level.
- bbox_targets_list (list[Tensor]): BBox targets of each level.
- bbox_weights_list (list[Tensor]): BBox weights of each level.
- num_total_pos (int): Number of positive samples in all images.
- num_total_neg (int): Number of negative samples in all images.
- additional_returns: This function enables user-defined returns from
self._get_targets_single. These returns are currently refined to properties at each feature map (i.e. having HxW dimension). The results will be concatenated after the end
Return type: tuple
-
loss
(cls_scores, bbox_preds, gt_bboxes, gt_labels, img_metas, gt_bboxes_ignore=None)[source]¶ Compute losses of the head.
Parameters: - cls_scores (list[Tensor]) – Box scores for each scale level Has shape (N, num_anchors * num_classes, H, W)
- bbox_preds (list[Tensor]) – Box energies / deltas for each scale level with shape (N, num_anchors * 4, H, W)
- gt_bboxes (list[Tensor]) – Ground truth bboxes for each image with shape (num_gts, 4) in [tl_x, tl_y, br_x, br_y] format.
- gt_labels (list[Tensor]) – class indices corresponding to each box
- img_metas (list[dict]) – Meta information of each image, e.g., image size, scaling factor, etc.
- gt_bboxes_ignore (None | list[Tensor]) – specify which bounding boxes can be ignored when computing the loss. Default: None
Returns: A dictionary of loss components.
Return type: dict[str, Tensor]
-
loss_single
(cls_score, bbox_pred, anchors, labels, label_weights, bbox_targets, bbox_weights, num_total_samples)[source]¶ Compute loss of a single scale level.
Parameters: - cls_score (Tensor) – Box scores for each scale level Has shape (N, num_anchors * num_classes, H, W).
- bbox_pred (Tensor) – Box energies / deltas for each scale level with shape (N, num_anchors * 4, H, W).
- anchors (Tensor) – Box reference for each scale level with shape (N, num_total_anchors, 4).
- labels (Tensor) – Labels of each anchors with shape (N, num_total_anchors).
- label_weights (Tensor) – Label weights of each anchor with shape (N, num_total_anchors)
- bbox_targets (Tensor) – BBox regression targets of each anchor wight shape (N, num_total_anchors, 4).
- bbox_weights (Tensor) – BBox regression loss weights of each anchor with shape (N, num_total_anchors, 4).
- num_total_samples (int) – If sampling, num total samples equal to the number of total anchors; Otherwise, it is the number of positive anchors.
Returns: A dictionary of loss components.
Return type: dict[str, Tensor]
-
class
mmdet.models.dense_heads.
GuidedAnchorHead
(num_classes, in_channels, feat_channels=256, approx_anchor_generator={'octave_base_scale': 8, 'ratios': [0.5, 1.0, 2.0], 'scales_per_octave': 3, 'strides': [4, 8, 16, 32, 64], 'type': 'AnchorGenerator'}, square_anchor_generator={'ratios': [1.0], 'scales': [8], 'strides': [4, 8, 16, 32, 64], 'type': 'AnchorGenerator'}, anchor_coder={'target_means': [0.0, 0.0, 0.0, 0.0], 'target_stds': [1.0, 1.0, 1.0, 1.0], 'type': 'DeltaXYWHBBoxCoder'}, bbox_coder={'target_means': [0.0, 0.0, 0.0, 0.0], 'target_stds': [1.0, 1.0, 1.0, 1.0], 'type': 'DeltaXYWHBBoxCoder'}, reg_decoded_bbox=False, deform_groups=4, loc_filter_thr=0.01, train_cfg=None, test_cfg=None, loss_loc={'alpha': 0.25, 'gamma': 2.0, 'loss_weight': 1.0, 'type': 'FocalLoss', 'use_sigmoid': True}, loss_shape={'beta': 0.2, 'loss_weight': 1.0, 'type': 'BoundedIoULoss'}, loss_cls={'loss_weight': 1.0, 'type': 'CrossEntropyLoss', 'use_sigmoid': True}, loss_bbox={'beta': 1.0, 'loss_weight': 1.0, 'type': 'SmoothL1Loss'}, init_cfg={'layer': 'Conv2d', 'override': {'bias_prob': 0.01, 'name': 'conv_loc', 'std': 0.01, 'type': 'Normal'}, 'std': 0.01, 'type': 'Normal'})[source]¶ Guided-Anchor-based head (GA-RPN, GA-RetinaNet, etc.).
This GuidedAnchorHead will predict high-quality feature guided anchors and locations where anchors will be kept in inference. There are mainly 3 categories of bounding-boxes.
- Sampled 9 pairs for target assignment. (approxes)
- The square boxes where the predicted anchors are based on. (squares)
- Guided anchors.
Please refer to https://arxiv.org/abs/1901.03278 for more details.
Parameters: - num_classes (int) – Number of classes.
- in_channels (int) – Number of channels in the input feature map.
- feat_channels (int) – Number of hidden channels.
- approx_anchor_generator (dict) – Config dict for approx generator
- square_anchor_generator (dict) – Config dict for square generator
- anchor_coder (dict) – Config dict for anchor coder
- bbox_coder (dict) – Config dict for bbox coder
- reg_decoded_bbox (bool) – If true, the regression loss would be applied directly on decoded bounding boxes, converting both the predicted boxes and regression targets to absolute coordinates format. Default False. It should be True when using IoULoss, GIoULoss, or DIoULoss in the bbox head.
- deform_groups – (int): Group number of DCN in FeatureAdaption module.
- loc_filter_thr (float) – Threshold to filter out unconcerned regions.
- loss_loc (dict) – Config of location loss.
- loss_shape (dict) – Config of anchor shape loss.
- loss_cls (dict) – Config of classification loss.
- loss_bbox (dict) – Config of bbox regression loss.
- init_cfg (dict or list[dict], optional) – Initialization config dict.
-
forward
(feats)[source]¶ Forward features from the upstream network.
Parameters: feats (tuple[Tensor]) – Features from the upstream network, each is a 4D-tensor. Returns: A tuple of classification scores and bbox prediction. - cls_scores (list[Tensor]): Classification scores for all scale levels, each is a 4D-tensor, the channels number is num_anchors * num_classes.
- bbox_preds (list[Tensor]): Box energies / deltas for all scale levels, each is a 4D-tensor, the channels number is num_anchors * 4.
Return type: tuple
-
forward_single
(x)[source]¶ Forward feature of a single scale level.
Parameters: x (Tensor) – Features of a single scale level. Returns: cls_score (Tensor): Cls scores for a single scale level the channels number is num_anchors * num_classes. bbox_pred (Tensor): Box energies / deltas for a single scale level, the channels number is num_anchors * 4. Return type: tuple
-
ga_loc_targets
(gt_bboxes_list, featmap_sizes)[source]¶ Compute location targets for guided anchoring.
Each feature map is divided into positive, negative and ignore regions. - positive regions: target 1, weight 1 - ignore regions: target 0, weight 0 - negative regions: target 0, weight 0.1
Parameters: - gt_bboxes_list (list[Tensor]) – Gt bboxes of each image.
- featmap_sizes (list[tuple]) – Multi level sizes of each feature maps.
Returns: tuple
-
ga_shape_targets
(approx_list, inside_flag_list, square_list, gt_bboxes_list, img_metas, gt_bboxes_ignore_list=None, unmap_outputs=True)[source]¶ Compute guided anchoring targets.
Parameters: - approx_list (list[list]) – Multi level approxs of each image.
- inside_flag_list (list[list]) – Multi level inside flags of each image.
- square_list (list[list]) – Multi level squares of each image.
- gt_bboxes_list (list[Tensor]) – Ground truth bboxes of each image.
- img_metas (list[dict]) – Meta info of each image.
- gt_bboxes_ignore_list (list[Tensor]) – ignore list of gt bboxes.
- unmap_outputs (bool) – unmap outputs or not.
Returns: tuple
-
get_anchors
(featmap_sizes, shape_preds, loc_preds, img_metas, use_loc_filter=False, device='cuda')[source]¶ Get squares according to feature map sizes and guided anchors.
Parameters: - featmap_sizes (list[tuple]) – Multi-level feature map sizes.
- shape_preds (list[tensor]) – Multi-level shape predictions.
- loc_preds (list[tensor]) – Multi-level location predictions.
- img_metas (list[dict]) – Image meta info.
- use_loc_filter (bool) – Use loc filter or not.
- device (torch.device | str) – device for returned tensors
Returns: - square approxs of each image, guided anchors of each image,
loc masks of each image
Return type: tuple
-
get_bboxes
(cls_scores, bbox_preds, shape_preds, loc_preds, img_metas, cfg=None, rescale=False)[source]¶ Transform network output for a batch into bbox predictions.
Parameters: - cls_scores (list[Tensor]) – Box scores for each level in the feature pyramid, has shape (N, num_anchors * num_classes, H, W).
- bbox_preds (list[Tensor]) – Box energies / deltas for each level in the feature pyramid, has shape (N, num_anchors * 4, H, W).
- img_metas (list[dict]) – Meta information of each image, e.g., image size, scaling factor, etc.
- cfg (mmcv.Config | None) – Test / postprocessing configuration, if None, test_cfg would be used
- rescale (bool) – If True, return boxes in original image space. Default: False.
- with_nms (bool) – If True, do nms before return boxes. Default: True.
Returns: - Each item in result_list is 2-tuple.
The first item is an (n, 5) tensor, where 5 represent (tl_x, tl_y, br_x, br_y, score) and the score between 0 and 1. The shape of the second tensor in the tuple is (n,), and each element represents the class label of the corresponding box.
Return type: list[tuple[Tensor, Tensor]]
Example
>>> import mmcv >>> self = AnchorHead( >>> num_classes=9, >>> in_channels=1, >>> anchor_generator=dict( >>> type='AnchorGenerator', >>> scales=[8], >>> ratios=[0.5, 1.0, 2.0], >>> strides=[4,])) >>> img_metas = [{'img_shape': (32, 32, 3), 'scale_factor': 1}] >>> cfg = mmcv.Config(dict( >>> score_thr=0.00, >>> nms=dict(type='nms', iou_thr=1.0), >>> max_per_img=10)) >>> feat = torch.rand(1, 1, 3, 3) >>> cls_score, bbox_pred = self.forward_single(feat) >>> # note the input lists are over different levels, not images >>> cls_scores, bbox_preds = [cls_score], [bbox_pred] >>> result_list = self.get_bboxes(cls_scores, bbox_preds, >>> img_metas, cfg) >>> det_bboxes, det_labels = result_list[0] >>> assert len(result_list) == 1 >>> assert det_bboxes.shape[1] == 5 >>> assert len(det_bboxes) == len(det_labels) == cfg.max_per_img
-
get_sampled_approxs
(featmap_sizes, img_metas, device='cuda')[source]¶ Get sampled approxs and inside flags according to feature map sizes.
Parameters: - featmap_sizes (list[tuple]) – Multi-level feature map sizes.
- img_metas (list[dict]) – Image meta info.
- device (torch.device | str) – device for returned tensors
Returns: approxes of each image, inside flags of each image
Return type: tuple
-
loss
(cls_scores, bbox_preds, shape_preds, loc_preds, gt_bboxes, gt_labels, img_metas, gt_bboxes_ignore=None)[source]¶ Compute losses of the head.
Parameters: - cls_scores (list[Tensor]) – Box scores for each scale level Has shape (N, num_anchors * num_classes, H, W)
- bbox_preds (list[Tensor]) – Box energies / deltas for each scale level with shape (N, num_anchors * 4, H, W)
- gt_bboxes (list[Tensor]) – Ground truth bboxes for each image with shape (num_gts, 4) in [tl_x, tl_y, br_x, br_y] format.
- gt_labels (list[Tensor]) – class indices corresponding to each box
- img_metas (list[dict]) – Meta information of each image, e.g., image size, scaling factor, etc.
- gt_bboxes_ignore (None | list[Tensor]) – specify which bounding boxes can be ignored when computing the loss. Default: None
Returns: A dictionary of loss components.
Return type: dict[str, Tensor]
-
class
mmdet.models.dense_heads.
FeatureAdaption
(in_channels, out_channels, kernel_size=3, deform_groups=4, init_cfg={'layer': 'Conv2d', 'override': {'name': 'conv_adaption', 'std': 0.01, 'type': 'Normal'}, 'std': 0.1, 'type': 'Normal'})[source]¶ Feature Adaption Module.
Feature Adaption Module is implemented based on DCN v1. It uses anchor shape prediction rather than feature map to predict offsets of deform conv layer.
Parameters: - in_channels (int) – Number of channels in the input feature map.
- out_channels (int) – Number of channels in the output feature map.
- kernel_size (int) – Deformable conv kernel size.
- deform_groups (int) – Deformable conv group size.
- init_cfg (dict or list[dict], optional) – Initialization config dict.
-
forward
(x, shape)[source]¶ Defines the computation performed at every call.
Should be overridden by all subclasses.
Note
Although the recipe for forward pass needs to be defined within this function, one should call the
Module
instance afterwards instead of this since the former takes care of running the registered hooks while the latter silently ignores them.
-
class
mmdet.models.dense_heads.
RPNHead
(in_channels, init_cfg={'layer': 'Conv2d', 'std': 0.01, 'type': 'Normal'}, **kwargs)[source]¶ RPN head.
Parameters: - in_channels (int) – Number of channels in the input feature map.
- init_cfg (dict or list[dict], optional) – Initialization config dict.
-
get_bboxes
(cls_scores, bbox_preds, img_metas, cfg=None, rescale=False, with_nms=True)[source]¶ Transform network output for a batch into bbox predictions.
Parameters: - cls_scores (list[Tensor]) – Box scores for each scale level Has shape (N, num_anchors * num_classes, H, W)
- bbox_preds (list[Tensor]) – Box energies / deltas for each scale level with shape (N, num_anchors * 4, H, W)
- img_metas (list[dict]) – Meta information of each image, e.g., image size, scaling factor, etc.
- cfg (mmcv.Config | None) – Test / postprocessing configuration, if None, test_cfg would be used
- rescale (bool) – If True, return boxes in original image space. Default: False.
- with_nms (bool) – If True, do nms before return boxes. Default: True.
Returns: - Each item in result_list is 2-tuple.
The first item is an (n, 5) tensor, where the first 4 columns are bounding box positions (tl_x, tl_y, br_x, br_y) and the 5-th column is a score between 0 and 1. The second item is a (n,) tensor where each item is the predicted class label of the corresponding box.
Return type: list[tuple[Tensor, Tensor]]
-
loss
(cls_scores, bbox_preds, gt_bboxes, img_metas, gt_bboxes_ignore=None)[source]¶ Compute losses of the head.
Parameters: - cls_scores (list[Tensor]) – Box scores for each scale level Has shape (N, num_anchors * num_classes, H, W)
- bbox_preds (list[Tensor]) – Box energies / deltas for each scale level with shape (N, num_anchors * 4, H, W)
- gt_bboxes (list[Tensor]) – Ground truth bboxes for each image with shape (num_gts, 4) in [tl_x, tl_y, br_x, br_y] format.
- img_metas (list[dict]) – Meta information of each image, e.g., image size, scaling factor, etc.
- gt_bboxes_ignore (None | list[Tensor]) – specify which bounding boxes can be ignored when computing the loss.
Returns: A dictionary of loss components.
Return type: dict[str, Tensor]
-
onnx_export
(x, img_metas)[source]¶ Test without augmentation.
Parameters: - x (tuple[Tensor]) – Features from the upstream network, each is a 4D-tensor.
- img_metas (list[dict]) – Meta info of each image.
Returns: - dets of shape [N, num_det, 5]
and class labels of shape [N, num_det].
Return type: tuple[Tensor, Tensor]
-
class
mmdet.models.dense_heads.
GARPNHead
(in_channels, init_cfg={'layer': 'Conv2d', 'override': {'bias_prob': 0.01, 'name': 'conv_loc', 'std': 0.01, 'type': 'Normal'}, 'std': 0.01, 'type': 'Normal'}, **kwargs)[source]¶ Guided-Anchor-based RPN head.
-
loss
(cls_scores, bbox_preds, shape_preds, loc_preds, gt_bboxes, img_metas, gt_bboxes_ignore=None)[source]¶ Compute losses of the head.
Parameters: - cls_scores (list[Tensor]) – Box scores for each scale level Has shape (N, num_anchors * num_classes, H, W)
- bbox_preds (list[Tensor]) – Box energies / deltas for each scale level with shape (N, num_anchors * 4, H, W)
- gt_bboxes (list[Tensor]) – Ground truth bboxes for each image with shape (num_gts, 4) in [tl_x, tl_y, br_x, br_y] format.
- gt_labels (list[Tensor]) – class indices corresponding to each box
- img_metas (list[dict]) – Meta information of each image, e.g., image size, scaling factor, etc.
- gt_bboxes_ignore (None | list[Tensor]) – specify which bounding boxes can be ignored when computing the loss. Default: None
Returns: A dictionary of loss components.
Return type: dict[str, Tensor]
-
-
class
mmdet.models.dense_heads.
RetinaHead
(num_classes, in_channels, stacked_convs=4, conv_cfg=None, norm_cfg=None, anchor_generator={'octave_base_scale': 4, 'ratios': [0.5, 1.0, 2.0], 'scales_per_octave': 3, 'strides': [8, 16, 32, 64, 128], 'type': 'AnchorGenerator'}, init_cfg={'layer': 'Conv2d', 'override': {'bias_prob': 0.01, 'name': 'retina_cls', 'std': 0.01, 'type': 'Normal'}, 'std': 0.01, 'type': 'Normal'}, **kwargs)[source]¶ An anchor-based head used in RetinaNet.
The head contains two subnetworks. The first classifies anchor boxes and the second regresses deltas for the anchors.
Example
>>> import torch >>> self = RetinaHead(11, 7) >>> x = torch.rand(1, 7, 32, 32) >>> cls_score, bbox_pred = self.forward_single(x) >>> # Each anchor predicts a score for each class except background >>> cls_per_anchor = cls_score.shape[1] / self.num_anchors >>> box_per_anchor = bbox_pred.shape[1] / self.num_anchors >>> assert cls_per_anchor == (self.num_classes) >>> assert box_per_anchor == 4
-
forward_single
(x)[source]¶ Forward feature of a single scale level.
Parameters: x (Tensor) – Features of a single scale level. Returns: - cls_score (Tensor): Cls scores for a single scale level
- the channels number is num_anchors * num_classes.
- bbox_pred (Tensor): Box energies / deltas for a single scale
- level, the channels number is num_anchors * 4.
Return type: tuple
-
-
class
mmdet.models.dense_heads.
RetinaSepBNHead
(num_classes, num_ins, in_channels, stacked_convs=4, conv_cfg=None, norm_cfg=None, init_cfg=None, **kwargs)[source]¶ “RetinaHead with separate BN.
In RetinaHead, conv/norm layers are shared across different FPN levels, while in RetinaSepBNHead, conv layers are shared across different FPN levels, but BN layers are separated.
-
forward
(feats)[source]¶ Forward features from the upstream network.
Parameters: feats (tuple[Tensor]) – Features from the upstream network, each is a 4D-tensor. Returns: - Usually a tuple of classification scores and bbox prediction
- cls_scores (list[Tensor]): Classification scores for all scale
- levels, each is a 4D-tensor, the channels number is num_anchors * num_classes.
- bbox_preds (list[Tensor]): Box energies / deltas for all scale
- levels, each is a 4D-tensor, the channels number is num_anchors * 4.
Return type: tuple
-
-
class
mmdet.models.dense_heads.
GARetinaHead
(num_classes, in_channels, stacked_convs=4, conv_cfg=None, norm_cfg=None, init_cfg=None, **kwargs)[source]¶ Guided-Anchor-based RetinaNet head.
-
class
mmdet.models.dense_heads.
SSDHead
(num_classes=80, in_channels=(512, 1024, 512, 256, 256, 256), stacked_convs=0, feat_channels=256, use_depthwise=False, conv_cfg=None, norm_cfg=None, act_cfg=None, anchor_generator={'basesize_ratio_range': (0.1, 0.9), 'input_size': 300, 'ratios': ([2], [2, 3], [2, 3], [2, 3], [2], [2]), 'scale_major': False, 'strides': [8, 16, 32, 64, 100, 300], 'type': 'SSDAnchorGenerator'}, bbox_coder={'clip_border': True, 'target_means': [0.0, 0.0, 0.0, 0.0], 'target_stds': [1.0, 1.0, 1.0, 1.0], 'type': 'DeltaXYWHBBoxCoder'}, reg_decoded_bbox=False, train_cfg=None, test_cfg=None, init_cfg={'bias': 0, 'distribution': 'uniform', 'layer': 'Conv2d', 'type': 'Xavier'})[source]¶ SSD head used in https://arxiv.org/abs/1512.02325.
Parameters: - num_classes (int) – Number of categories excluding the background category.
- in_channels (int) – Number of channels in the input feature map.
- stacked_convs (int) – Number of conv layers in cls and reg tower. Default: 0.
- feat_channels (int) – Number of hidden channels when stacked_convs > 0. Default: 256.
- use_depthwise (bool) – Whether to use DepthwiseSeparableConv. Default: False.
- conv_cfg (dict) – Dictionary to construct and config conv layer. Default: None.
- norm_cfg (dict) – Dictionary to construct and config norm layer. Default: None.
- act_cfg (dict) – Dictionary to construct and config activation layer. Default: None.
- anchor_generator (dict) – Config dict for anchor generator
- bbox_coder (dict) – Config of bounding box coder.
- reg_decoded_bbox (bool) – If true, the regression loss would be applied directly on decoded bounding boxes, converting both the predicted boxes and regression targets to absolute coordinates format. Default False. It should be True when using IoULoss, GIoULoss, or DIoULoss in the bbox head.
- train_cfg (dict) – Training config of anchor head.
- test_cfg (dict) – Testing config of anchor head.
- init_cfg (dict or list[dict], optional) – Initialization config dict.
-
forward
(feats)[source]¶ Forward features from the upstream network.
Parameters: feats (tuple[Tensor]) – Features from the upstream network, each is a 4D-tensor. Returns: - cls_scores (list[Tensor]): Classification scores for all scale
- levels, each is a 4D-tensor, the channels number is num_anchors * num_classes.
- bbox_preds (list[Tensor]): Box energies / deltas for all scale
- levels, each is a 4D-tensor, the channels number is num_anchors * 4.
Return type: tuple
-
loss
(cls_scores, bbox_preds, gt_bboxes, gt_labels, img_metas, gt_bboxes_ignore=None)[source]¶ Compute losses of the head.
Parameters: - cls_scores (list[Tensor]) – Box scores for each scale level Has shape (N, num_anchors * num_classes, H, W)
- bbox_preds (list[Tensor]) – Box energies / deltas for each scale level with shape (N, num_anchors * 4, H, W)
- gt_bboxes (list[Tensor]) – each item are the truth boxes for each image in [tl_x, tl_y, br_x, br_y] format.
- gt_labels (list[Tensor]) – class indices corresponding to each box
- img_metas (list[dict]) – Meta information of each image, e.g., image size, scaling factor, etc.
- gt_bboxes_ignore (None | list[Tensor]) – specify which bounding boxes can be ignored when computing the loss.
Returns: A dictionary of loss components.
Return type: dict[str, Tensor]
-
loss_single
(cls_score, bbox_pred, anchor, labels, label_weights, bbox_targets, bbox_weights, num_total_samples)[source]¶ Compute loss of a single image.
Parameters: - cls_score (Tensor) – Box scores for eachimage Has shape (num_total_anchors, num_classes).
- bbox_pred (Tensor) – Box energies / deltas for each image level with shape (num_total_anchors, 4).
- anchors (Tensor) – Box reference for each scale level with shape (num_total_anchors, 4).
- labels (Tensor) – Labels of each anchors with shape (num_total_anchors,).
- label_weights (Tensor) – Label weights of each anchor with shape (num_total_anchors,)
- bbox_targets (Tensor) – BBox regression targets of each anchor wight shape (num_total_anchors, 4).
- bbox_weights (Tensor) – BBox regression loss weights of each anchor with shape (num_total_anchors, 4).
- num_total_samples (int) – If sampling, num total samples equal to the number of total anchors; Otherwise, it is the number of positive anchors.
Returns: A dictionary of loss components.
Return type: dict[str, Tensor]
-
class
mmdet.models.dense_heads.
FCOSHead
(num_classes, in_channels, regress_ranges=((-1, 64), (64, 128), (128, 256), (256, 512), (512, 100000000.0)), center_sampling=False, center_sample_radius=1.5, norm_on_bbox=False, centerness_on_reg=False, loss_cls={'alpha': 0.25, 'gamma': 2.0, 'loss_weight': 1.0, 'type': 'FocalLoss', 'use_sigmoid': True}, loss_bbox={'loss_weight': 1.0, 'type': 'IoULoss'}, loss_centerness={'loss_weight': 1.0, 'type': 'CrossEntropyLoss', 'use_sigmoid': True}, norm_cfg={'num_groups': 32, 'requires_grad': True, 'type': 'GN'}, init_cfg={'layer': 'Conv2d', 'override': {'bias_prob': 0.01, 'name': 'conv_cls', 'std': 0.01, 'type': 'Normal'}, 'std': 0.01, 'type': 'Normal'}, **kwargs)[source]¶ Anchor-free head used in FCOS.
The FCOS head does not use anchor boxes. Instead bounding boxes are predicted at each pixel and a centerness measure is used to suppress low-quality predictions. Here norm_on_bbox, centerness_on_reg, dcn_on_last_conv are training tricks used in official repo, which will bring remarkable mAP gains of up to 4.9. Please see https://github.com/tianzhi0549/FCOS for more detail.
Parameters: - num_classes (int) – Number of categories excluding the background category.
- in_channels (int) – Number of channels in the input feature map.
- strides (list[int] | list[tuple[int, int]]) – Strides of points in multiple feature levels. Default: (4, 8, 16, 32, 64).
- regress_ranges (tuple[tuple[int, int]]) – Regress range of multiple level points.
- center_sampling (bool) – If true, use center sampling. Default: False.
- center_sample_radius (float) – Radius of center sampling. Default: 1.5.
- norm_on_bbox (bool) – If true, normalize the regression targets with FPN strides. Default: False.
- centerness_on_reg (bool) – If true, position centerness on the regress branch. Please refer to https://github.com/tianzhi0549/FCOS/issues/89#issuecomment-516877042. Default: False.
- conv_bias (bool | str) – If specified as auto, it will be decided by the norm_cfg. Bias of conv will be set as True if norm_cfg is None, otherwise False. Default: “auto”.
- loss_cls (dict) – Config of classification loss.
- loss_bbox (dict) – Config of localization loss.
- loss_centerness (dict) – Config of centerness loss.
- norm_cfg (dict) – dictionary to construct and config norm layer. Default: norm_cfg=dict(type=’GN’, num_groups=32, requires_grad=True).
- init_cfg (dict or list[dict], optional) – Initialization config dict.
Example
>>> self = FCOSHead(11, 7) >>> feats = [torch.rand(1, 7, s, s) for s in [4, 8, 16, 32, 64]] >>> cls_score, bbox_pred, centerness = self.forward(feats) >>> assert len(cls_score) == len(self.scales)
-
centerness_target
(pos_bbox_targets)[source]¶ Compute centerness targets.
Parameters: pos_bbox_targets (Tensor) – BBox targets of positive bboxes in shape (num_pos, 4) Returns: Centerness target. Return type: Tensor
-
forward
(feats)[source]¶ Forward features from the upstream network.
Parameters: feats (tuple[Tensor]) – Features from the upstream network, each is a 4D-tensor. Returns: cls_scores (list[Tensor]): Box scores for each scale level, each is a 4D-tensor, the channel number is num_points * num_classes. bbox_preds (list[Tensor]): Box energies / deltas for each scale level, each is a 4D-tensor, the channel number is num_points * 4. centernesses (list[Tensor]): centerness for each scale level, each is a 4D-tensor, the channel number is num_points * 1. Return type: tuple
-
forward_single
(x, scale, stride)[source]¶ Forward features of a single scale level.
Parameters: - x (Tensor) – FPN feature maps of the specified stride.
- ( (scale) – obj: mmcv.cnn.Scale): Learnable scale module to resize the bbox prediction.
- stride (int) – The corresponding stride for feature maps, only used to normalize the bbox prediction when self.norm_on_bbox is True.
Returns: scores for each class, bbox predictions and centerness predictions of input feature maps.
Return type: tuple
-
get_bboxes
(cls_scores, bbox_preds, centernesses, img_metas, cfg=None, rescale=False, with_nms=True)[source]¶ Transform network output for a batch into bbox predictions.
Parameters: - cls_scores (list[Tensor]) – Box scores for each scale level with shape (N, num_points * num_classes, H, W).
- bbox_preds (list[Tensor]) – Box energies / deltas for each scale level with shape (N, num_points * 4, H, W).
- centernesses (list[Tensor]) – Centerness for each scale level with shape (N, num_points * 1, H, W).
- img_metas (list[dict]) – Meta information of each image, e.g., image size, scaling factor, etc.
- cfg (mmcv.Config | None) – Test / postprocessing configuration, if None, test_cfg would be used. Default: None.
- rescale (bool) – If True, return boxes in original image space. Default: False.
- with_nms (bool) – If True, do nms before return boxes. Default: True.
Returns: - Each item in result_list is 2-tuple.
The first item is an (n, 5) tensor, where 5 represent (tl_x, tl_y, br_x, br_y, score) and the score between 0 and 1. The shape of the second tensor in the tuple is (n,), and each element represents the class label of the corresponding box.
Return type: list[tuple[Tensor, Tensor]]
-
get_targets
(points, gt_bboxes_list, gt_labels_list)[source]¶ Compute regression, classification and centerness targets for points in multiple images.
Parameters: - points (list[Tensor]) – Points of each fpn level, each has shape (num_points, 2).
- gt_bboxes_list (list[Tensor]) – Ground truth bboxes of each image, each has shape (num_gt, 4).
- gt_labels_list (list[Tensor]) – Ground truth labels of each box, each has shape (num_gt,).
Returns: concat_lvl_labels (list[Tensor]): Labels of each level. concat_lvl_bbox_targets (list[Tensor]): BBox targets of each level.
Return type: tuple
-
loss
(cls_scores, bbox_preds, centernesses, gt_bboxes, gt_labels, img_metas, gt_bboxes_ignore=None)[source]¶ Compute loss of the head.
Parameters: - cls_scores (list[Tensor]) – Box scores for each scale level, each is a 4D-tensor, the channel number is num_points * num_classes.
- bbox_preds (list[Tensor]) – Box energies / deltas for each scale level, each is a 4D-tensor, the channel number is num_points * 4.
- centernesses (list[Tensor]) – centerness for each scale level, each is a 4D-tensor, the channel number is num_points * 1.
- gt_bboxes (list[Tensor]) – Ground truth bboxes for each image with shape (num_gts, 4) in [tl_x, tl_y, br_x, br_y] format.
- gt_labels (list[Tensor]) – class indices corresponding to each box
- img_metas (list[dict]) – Meta information of each image, e.g., image size, scaling factor, etc.
- gt_bboxes_ignore (None | list[Tensor]) – specify which bounding boxes can be ignored when computing the loss.
Returns: A dictionary of loss components.
Return type: dict[str, Tensor]
-
class
mmdet.models.dense_heads.
RepPointsHead
(num_classes, in_channels, point_feat_channels=256, num_points=9, gradient_mul=0.1, point_strides=[8, 16, 32, 64, 128], point_base_scale=4, loss_cls={'alpha': 0.25, 'gamma': 2.0, 'loss_weight': 1.0, 'type': 'FocalLoss', 'use_sigmoid': True}, loss_bbox_init={'beta': 0.1111111111111111, 'loss_weight': 0.5, 'type': 'SmoothL1Loss'}, loss_bbox_refine={'beta': 0.1111111111111111, 'loss_weight': 1.0, 'type': 'SmoothL1Loss'}, use_grid_points=False, center_init=True, transform_method='moment', moment_mul=0.01, init_cfg={'layer': 'Conv2d', 'override': {'bias_prob': 0.01, 'name': 'reppoints_cls_out', 'std': 0.01, 'type': 'Normal'}, 'std': 0.01, 'type': 'Normal'}, **kwargs)[source]¶ RepPoint head.
Parameters: - point_feat_channels (int) – Number of channels of points features.
- gradient_mul (float) – The multiplier to gradients from points refinement and recognition.
- point_strides (Iterable) – points strides.
- point_base_scale (int) – bbox scale for assigning labels.
- loss_cls (dict) – Config of classification loss.
- loss_bbox_init (dict) – Config of initial points loss.
- loss_bbox_refine (dict) – Config of points loss in refinement.
- use_grid_points (bool) – If we use bounding box representation, the
- is represented as grid points on the bounding box. (reppoints) –
- center_init (bool) – Whether to use center point assignment.
- transform_method (str) – The methods to transform RepPoints to bbox.
- init_cfg (dict or list[dict], optional) – Initialization config dict.
-
centers_to_bboxes
(point_list)[source]¶ Get bboxes according to center points.
Only used in
MaxIoUAssigner
.
-
forward
(feats)[source]¶ Forward features from the upstream network.
Parameters: feats (tuple[Tensor]) – Features from the upstream network, each is a 4D-tensor. Returns: - Usually contain classification scores and bbox predictions.
- cls_scores (list[Tensor]): Box scores for each scale level,
- each is a 4D-tensor, the channel number is num_points * num_classes.
- bbox_preds (list[Tensor]): Box energies / deltas for each scale
- level, each is a 4D-tensor, the channel number is num_points * 4.
Return type: tuple
-
gen_grid_from_reg
(reg, previous_boxes)[source]¶ Base on the previous bboxes and regression values, we compute the regressed bboxes and generate the grids on the bboxes.
Parameters: - reg – the regression value to previous bboxes.
- previous_boxes – previous bboxes.
Returns: generate grids on the regressed bboxes.
-
get_bboxes
(cls_scores, pts_preds_init, pts_preds_refine, img_metas, cfg=None, rescale=False, with_nms=True)[source]¶ Transform network output for a batch into bbox predictions.
Parameters: - cls_scores (list[Tensor]) – Box scores for each scale level Has shape (N, num_points * num_classes, H, W)
- bbox_preds (list[Tensor]) – Box energies / deltas for each scale level with shape (N, num_points * 4, H, W)
- img_metas (list[dict]) – Meta information of each image, e.g., image size, scaling factor, etc.
- cfg (mmcv.Config) – Test / postprocessing configuration, if None, test_cfg would be used
- rescale (bool) – If True, return boxes in original image space
-
get_points
(featmap_sizes, img_metas, device)[source]¶ Get points according to feature map sizes.
Parameters: - featmap_sizes (list[tuple]) – Multi-level feature map sizes.
- img_metas (list[dict]) – Image meta info.
Returns: points of each image, valid flags of each image
Return type: tuple
-
get_targets
(proposals_list, valid_flag_list, gt_bboxes_list, img_metas, gt_bboxes_ignore_list=None, gt_labels_list=None, stage='init', label_channels=1, unmap_outputs=True)[source]¶ Compute corresponding GT box and classification targets for proposals.
Parameters: - proposals_list (list[list]) – Multi level points/bboxes of each image.
- valid_flag_list (list[list]) – Multi level valid flags of each image.
- gt_bboxes_list (list[Tensor]) – Ground truth bboxes of each image.
- img_metas (list[dict]) – Meta info of each image.
- gt_bboxes_ignore_list (list[Tensor]) – Ground truth bboxes to be ignored.
- gt_bboxes_list – Ground truth labels of each box.
- stage (str) – init or refine. Generate target for init stage or refine stage
- label_channels (int) – Channel of label.
- unmap_outputs (bool) – Whether to map outputs back to the original set of anchors.
Returns: - labels_list (list[Tensor]): Labels of each level.
- label_weights_list (list[Tensor]): Label weights of each level. # noqa: E501
- bbox_gt_list (list[Tensor]): Ground truth bbox of each level.
- proposal_list (list[Tensor]): Proposals(points/bboxes) of each level. # noqa: E501
- proposal_weights_list (list[Tensor]): Proposal weights of each level. # noqa: E501
- num_total_pos (int): Number of positive samples in all images. # noqa: E501
- num_total_neg (int): Number of negative samples in all images. # noqa: E501
Return type: tuple
-
loss
(cls_scores, pts_preds_init, pts_preds_refine, gt_bboxes, gt_labels, img_metas, gt_bboxes_ignore=None)[source]¶ Compute loss of the head.
Parameters: - cls_scores (list[Tensor]) – Box scores for each scale level, each is a 4D-tensor, the channel number is num_points * num_classes.
- bbox_preds (list[Tensor]) – Box energies / deltas for each scale level, each is a 4D-tensor, the channel number is num_points * 4.
- gt_bboxes (list[Tensor]) – Ground truth bboxes for each image with shape (num_gts, 4) in [tl_x, tl_y, br_x, br_y] format.
- gt_labels (list[Tensor]) – class indices corresponding to each box
- img_metas (list[dict]) – Meta information of each image, e.g., image size, scaling factor, etc.
- gt_bboxes_ignore (None | list[Tensor]) – specify which bounding boxes can be ignored when computing the loss.
-
points2bbox
(pts, y_first=True)[source]¶ Converting the points set into bounding box.
Parameters: - pts – the input points sets (fields), each points set (fields) is represented as 2n scalar.
- y_first – if y_first=True, the point set is represented as [y1, x1, y2, x2 … yn, xn], otherwise the point set is represented as [x1, y1, x2, y2 … xn, yn].
Returns: each points set is converting to a bbox [x1, y1, x2, y2].
-
class
mmdet.models.dense_heads.
FoveaHead
(num_classes, in_channels, base_edge_list=(16, 32, 64, 128, 256), scale_ranges=((8, 32), (16, 64), (32, 128), (64, 256), (128, 512)), sigma=0.4, with_deform=False, deform_groups=4, init_cfg={'layer': 'Conv2d', 'override': {'bias_prob': 0.01, 'name': 'conv_cls', 'std': 0.01, 'type': 'Normal'}, 'std': 0.01, 'type': 'Normal'}, **kwargs)[source]¶ FoveaBox: Beyond Anchor-based Object Detector https://arxiv.org/abs/1904.03797
-
forward_single
(x)[source]¶ Forward features of a single scale level.
Parameters: x (Tensor) – FPN feature maps of the specified stride. Returns: - Scores for each class, bbox predictions, features
- after classification and regression conv layers, some models needs these features like FCOS.
Return type: tuple
-
get_bboxes
(cls_scores, bbox_preds, img_metas, cfg=None, rescale=None)[source]¶ Transform network output for a batch into bbox predictions.
Parameters: - cls_scores (list[Tensor]) – Box scores for each scale level Has shape (N, num_points * num_classes, H, W)
- bbox_preds (list[Tensor]) – Box energies / deltas for each scale level with shape (N, num_points * 4, H, W)
- img_metas (list[dict]) – Meta information of each image, e.g., image size, scaling factor, etc.
- cfg (mmcv.Config) – Test / postprocessing configuration, if None, test_cfg would be used
- rescale (bool) – If True, return boxes in original image space
-
get_targets
(gt_bbox_list, gt_label_list, featmap_sizes, points)[source]¶ Compute regression, classification and centerness targets for points in multiple images.
Parameters: - points (list[Tensor]) – Points of each fpn level, each has shape (num_points, 2).
- gt_bboxes_list (list[Tensor]) – Ground truth bboxes of each image, each has shape (num_gt, 4).
- gt_labels_list (list[Tensor]) – Ground truth labels of each box, each has shape (num_gt,).
-
loss
(cls_scores, bbox_preds, gt_bbox_list, gt_label_list, img_metas, gt_bboxes_ignore=None)[source]¶ Compute loss of the head.
Parameters: - cls_scores (list[Tensor]) – Box scores for each scale level, each is a 4D-tensor, the channel number is num_points * num_classes.
- bbox_preds (list[Tensor]) – Box energies / deltas for each scale level, each is a 4D-tensor, the channel number is num_points * 4.
- gt_bboxes (list[Tensor]) – Ground truth bboxes for each image with shape (num_gts, 4) in [tl_x, tl_y, br_x, br_y] format.
- gt_labels (list[Tensor]) – class indices corresponding to each box
- img_metas (list[dict]) – Meta information of each image, e.g., image size, scaling factor, etc.
- gt_bboxes_ignore (None | list[Tensor]) – specify which bounding boxes can be ignored when computing the loss.
-
-
class
mmdet.models.dense_heads.
FreeAnchorRetinaHead
(num_classes, in_channels, stacked_convs=4, conv_cfg=None, norm_cfg=None, pre_anchor_topk=50, bbox_thr=0.6, gamma=2.0, alpha=0.5, **kwargs)[source]¶ FreeAnchor RetinaHead used in https://arxiv.org/abs/1909.02466.
Parameters: - num_classes (int) – Number of categories excluding the background category.
- in_channels (int) – Number of channels in the input feature map.
- stacked_convs (int) – Number of conv layers in cls and reg tower. Default: 4.
- conv_cfg (dict) – dictionary to construct and config conv layer. Default: None.
- norm_cfg (dict) – dictionary to construct and config norm layer. Default: norm_cfg=dict(type=’GN’, num_groups=32, requires_grad=True).
- pre_anchor_topk (int) – Number of boxes that be token in each bag.
- bbox_thr (float) – The threshold of the saturated linear function. It is usually the same with the IoU threshold used in NMS.
- gamma (float) – Gamma parameter in focal loss.
- alpha (float) – Alpha parameter in focal loss.
-
loss
(cls_scores, bbox_preds, gt_bboxes, gt_labels, img_metas, gt_bboxes_ignore=None)[source]¶ Compute losses of the head.
Parameters: - cls_scores (list[Tensor]) – Box scores for each scale level Has shape (N, num_anchors * num_classes, H, W)
- bbox_preds (list[Tensor]) – Box energies / deltas for each scale level with shape (N, num_anchors * 4, H, W)
- gt_bboxes (list[Tensor]) – each item are the truth boxes for each image in [tl_x, tl_y, br_x, br_y] format.
- gt_labels (list[Tensor]) – class indices corresponding to each box
- img_metas (list[dict]) – Meta information of each image, e.g., image size, scaling factor, etc.
- gt_bboxes_ignore (None | list[Tensor]) – specify which bounding boxes can be ignored when computing the loss.
Returns: A dictionary of loss components.
Return type: dict[str, Tensor]
-
negative_bag_loss
(cls_prob, box_prob)[source]¶ Compute negative bag loss.
\(FL((1 - P_{a_{j} \in A_{+}}) * (1 - P_{j}^{bg}))\).
\(P_{a_{j} \in A_{+}}\): Box_probability of matched samples.
\(P_{j}^{bg}\): Classification probability of negative samples.
Parameters: - cls_prob (Tensor) – Classification probability, in shape (num_img, num_anchors, num_classes).
- box_prob (Tensor) – Box probability, in shape (num_img, num_anchors, num_classes).
Returns: Negative bag loss in shape (num_img, num_anchors, num_classes).
Return type: Tensor
-
positive_bag_loss
(matched_cls_prob, matched_box_prob)[source]¶ Compute positive bag loss.
\(-log( Mean-max(P_{ij}^{cls} * P_{ij}^{loc}) )\).
\(P_{ij}^{cls}\): matched_cls_prob, classification probability of matched samples.
\(P_{ij}^{loc}\): matched_box_prob, box probability of matched samples.
Parameters: - matched_cls_prob (Tensor) – Classification probability of matched samples in shape (num_gt, pre_anchor_topk).
- matched_box_prob (Tensor) – BBox probability of matched samples, in shape (num_gt, pre_anchor_topk).
Returns: Positive bag loss in shape (num_gt,).
Return type: Tensor
-
class
mmdet.models.dense_heads.
ATSSHead
(num_classes, in_channels, stacked_convs=4, conv_cfg=None, norm_cfg={'num_groups': 32, 'requires_grad': True, 'type': 'GN'}, loss_centerness={'loss_weight': 1.0, 'type': 'CrossEntropyLoss', 'use_sigmoid': True}, init_cfg={'layer': 'Conv2d', 'override': {'bias_prob': 0.01, 'name': 'atss_cls', 'std': 0.01, 'type': 'Normal'}, 'std': 0.01, 'type': 'Normal'}, **kwargs)[source]¶ Bridging the Gap Between Anchor-based and Anchor-free Detection via Adaptive Training Sample Selection.
ATSS head structure is similar with FCOS, however ATSS use anchor boxes and assign label by Adaptive Training Sample Selection instead max-iou.
https://arxiv.org/abs/1912.02424
-
forward
(feats)[source]¶ Forward features from the upstream network.
Parameters: feats (tuple[Tensor]) – Features from the upstream network, each is a 4D-tensor. Returns: - Usually a tuple of classification scores and bbox prediction
- cls_scores (list[Tensor]): Classification scores for all scale
- levels, each is a 4D-tensor, the channels number is num_anchors * num_classes.
- bbox_preds (list[Tensor]): Box energies / deltas for all scale
- levels, each is a 4D-tensor, the channels number is num_anchors * 4.
Return type: tuple
-
forward_single
(x, scale)[source]¶ Forward feature of a single scale level.
Parameters: - x (Tensor) – Features of a single scale level.
- ( (scale) – obj: mmcv.cnn.Scale): Learnable scale module to resize the bbox prediction.
Returns: - cls_score (Tensor): Cls scores for a single scale level
the channels number is num_anchors * num_classes.
- bbox_pred (Tensor): Box energies / deltas for a single scale
level, the channels number is num_anchors * 4.
- centerness (Tensor): Centerness for a single scale level, the
channel number is (N, num_anchors * 1, H, W).
Return type: tuple
-
get_bboxes
(cls_scores, bbox_preds, centernesses, img_metas, cfg=None, rescale=False, with_nms=True)[source]¶ Transform network output for a batch into bbox predictions.
Parameters: - cls_scores (list[Tensor]) – Box scores for each scale level with shape (N, num_anchors * num_classes, H, W).
- bbox_preds (list[Tensor]) – Box energies / deltas for each scale level with shape (N, num_anchors * 4, H, W).
- centernesses (list[Tensor]) – Centerness for each scale level with shape (N, num_anchors * 1, H, W).
- img_metas (list[dict]) – Meta information of each image, e.g., image size, scaling factor, etc.
- cfg (mmcv.Config | None) – Test / postprocessing configuration, if None, test_cfg would be used. Default: None.
- rescale (bool) – If True, return boxes in original image space. Default: False.
- with_nms (bool) – If True, do nms before return boxes. Default: True.
Returns: - Each item in result_list is 2-tuple.
The first item is an (n, 5) tensor, where 5 represent (tl_x, tl_y, br_x, br_y, score) and the score between 0 and 1. The shape of the second tensor in the tuple is (n,), and each element represents the class label of the corresponding box.
Return type: list[tuple[Tensor, Tensor]]
-
get_targets
(anchor_list, valid_flag_list, gt_bboxes_list, img_metas, gt_bboxes_ignore_list=None, gt_labels_list=None, label_channels=1, unmap_outputs=True)[source]¶ Get targets for ATSS head.
This method is almost the same as AnchorHead.get_targets(). Besides returning the targets as the parent method does, it also returns the anchors as the first element of the returned tuple.
-
loss
(cls_scores, bbox_preds, centernesses, gt_bboxes, gt_labels, img_metas, gt_bboxes_ignore=None)[source]¶ Compute losses of the head.
Parameters: - cls_scores (list[Tensor]) – Box scores for each scale level Has shape (N, num_anchors * num_classes, H, W)
- bbox_preds (list[Tensor]) – Box energies / deltas for each scale level with shape (N, num_anchors * 4, H, W)
- centernesses (list[Tensor]) – Centerness for each scale level with shape (N, num_anchors * 1, H, W)
- gt_bboxes (list[Tensor]) – Ground truth bboxes for each image with shape (num_gts, 4) in [tl_x, tl_y, br_x, br_y] format.
- gt_labels (list[Tensor]) – class indices corresponding to each box
- img_metas (list[dict]) – Meta information of each image, e.g., image size, scaling factor, etc.
- gt_bboxes_ignore (list[Tensor] | None) – specify which bounding boxes can be ignored when computing the loss.
Returns: A dictionary of loss components.
Return type: dict[str, Tensor]
-
loss_single
(anchors, cls_score, bbox_pred, centerness, labels, label_weights, bbox_targets, num_total_samples)[source]¶ Compute loss of a single scale level.
Parameters: - cls_score (Tensor) – Box scores for each scale level Has shape (N, num_anchors * num_classes, H, W).
- bbox_pred (Tensor) – Box energies / deltas for each scale level with shape (N, num_anchors * 4, H, W).
- anchors (Tensor) – Box reference for each scale level with shape (N, num_total_anchors, 4).
- labels (Tensor) – Labels of each anchors with shape (N, num_total_anchors).
- label_weights (Tensor) – Label weights of each anchor with shape (N, num_total_anchors)
- bbox_targets (Tensor) – BBox regression targets of each anchor wight shape (N, num_total_anchors, 4).
- num_total_samples (int) – Number os positive samples that is reduced over all GPUs.
Returns: A dictionary of loss components.
Return type: dict[str, Tensor]
-
-
class
mmdet.models.dense_heads.
FSAFHead
(*args, score_threshold=None, init_cfg=None, **kwargs)[source]¶ Anchor-free head used in FSAF.
The head contains two subnetworks. The first classifies anchor boxes and the second regresses deltas for the anchors (num_anchors is 1 for anchor- free methods)
Parameters: - *args – Same as its base class in
RetinaHead
- score_threshold (float, optional) – The score_threshold to calculate positive recall. If given, prediction scores lower than this value is counted as incorrect prediction. Default to None.
- init_cfg (dict or list[dict], optional) – Initialization config dict. Default: None
- **kwargs – Same as its base class in
RetinaHead
Example
>>> import torch >>> self = FSAFHead(11, 7) >>> x = torch.rand(1, 7, 32, 32) >>> cls_score, bbox_pred = self.forward_single(x) >>> # Each anchor predicts a score for each class except background >>> cls_per_anchor = cls_score.shape[1] / self.num_anchors >>> box_per_anchor = bbox_pred.shape[1] / self.num_anchors >>> assert cls_per_anchor == self.num_classes >>> assert box_per_anchor == 4
-
calculate_pos_recall
(cls_scores, labels_list, pos_inds)[source]¶ Calculate positive recall with score threshold.
Parameters: - cls_scores (list[Tensor]) – Classification scores at all fpn levels. Each tensor is in shape (N, num_classes * num_anchors, H, W)
- labels_list (list[Tensor]) – The label that each anchor is assigned to. Shape (N * H * W * num_anchors, )
- pos_inds (list[Tensor]) – List of bool tensors indicating whether the anchor is assigned to a positive label. Shape (N * H * W * num_anchors, )
Returns: A single float number indicating the positive recall.
Return type: Tensor
-
collect_loss_level_single
(cls_loss, reg_loss, assigned_gt_inds, labels_seq)[source]¶ Get the average loss in each FPN level w.r.t. each gt label.
Parameters: - cls_loss (Tensor) – Classification loss of each feature map pixel, shape (num_anchor, num_class)
- reg_loss (Tensor) – Regression loss of each feature map pixel, shape (num_anchor, 4)
- assigned_gt_inds (Tensor) – It indicates which gt the prior is assigned to (0-based, -1: no assignment). shape (num_anchor),
- labels_seq – The rank of labels. shape (num_gt)
Returns: (num_gt), average loss of each gt in this level
Return type: shape
-
forward_single
(x)[source]¶ Forward feature map of a single scale level.
Parameters: x (Tensor) – Feature map of a single scale level. Returns: - cls_score (Tensor): Box scores for each scale level
- Has shape (N, num_points * num_classes, H, W).
- bbox_pred (Tensor): Box energies / deltas for each scale
- level with shape (N, num_points * 4, H, W).
Return type: tuple (Tensor)
-
loss
(cls_scores, bbox_preds, gt_bboxes, gt_labels, img_metas, gt_bboxes_ignore=None)[source]¶ Compute loss of the head.
Parameters: - cls_scores (list[Tensor]) – Box scores for each scale level Has shape (N, num_points * num_classes, H, W).
- bbox_preds (list[Tensor]) – Box energies / deltas for each scale level with shape (N, num_points * 4, H, W).
- gt_bboxes (list[Tensor]) – each item are the truth boxes for each image in [tl_x, tl_y, br_x, br_y] format.
- gt_labels (list[Tensor]) – class indices corresponding to each box
- img_metas (list[dict]) – Meta information of each image, e.g., image size, scaling factor, etc.
- gt_bboxes_ignore (None | list[Tensor]) – specify which bounding boxes can be ignored when computing the loss.
Returns: A dictionary of loss components.
Return type: dict[str, Tensor]
-
reweight_loss_single
(cls_loss, reg_loss, assigned_gt_inds, labels, level, min_levels)[source]¶ Reweight loss values at each level.
Reassign loss values at each level by masking those where the pre-calculated loss is too large. Then return the reduced losses.
Parameters: - cls_loss (Tensor) – Element-wise classification loss. Shape: (num_anchors, num_classes)
- reg_loss (Tensor) – Element-wise regression loss. Shape: (num_anchors, 4)
- assigned_gt_inds (Tensor) – The gt indices that each anchor bbox is assigned to. -1 denotes a negative anchor, otherwise it is the gt index (0-based). Shape: (num_anchors, ),
- labels (Tensor) – Label assigned to anchors. Shape: (num_anchors, ).
- level (int) – The current level index in the pyramid (0-4 for RetinaNet)
- min_levels (Tensor) – The best-matching level for each gt. Shape: (num_gts, ),
Returns: - cls_loss: Reduced corrected classification loss. Scalar.
- reg_loss: Reduced corrected regression loss. Scalar.
- pos_flags (Tensor): Corrected bool tensor indicating the final positive anchors. Shape: (num_anchors, ).
Return type: tuple
- *args – Same as its base class in
-
class
mmdet.models.dense_heads.
NASFCOSHead
(*args, init_cfg=None, **kwargs)[source]¶ Anchor-free head used in NASFCOS.
It is quite similar with FCOS head, except for the searched structure of classification branch and bbox regression branch, where a structure of “dconv3x3, conv3x3, dconv3x3, conv1x1” is utilized instead.
-
class
mmdet.models.dense_heads.
PISARetinaHead
(num_classes, in_channels, stacked_convs=4, conv_cfg=None, norm_cfg=None, anchor_generator={'octave_base_scale': 4, 'ratios': [0.5, 1.0, 2.0], 'scales_per_octave': 3, 'strides': [8, 16, 32, 64, 128], 'type': 'AnchorGenerator'}, init_cfg={'layer': 'Conv2d', 'override': {'bias_prob': 0.01, 'name': 'retina_cls', 'std': 0.01, 'type': 'Normal'}, 'std': 0.01, 'type': 'Normal'}, **kwargs)[source]¶ PISA Retinanet Head.
- The head owns the same structure with Retinanet Head, but differs in two
aspects: 1. Importance-based Sample Reweighting Positive (ISR-P) is applied to
change the positive loss weights.- Classification-aware regression loss is adopted as a third loss.
-
loss
(cls_scores, bbox_preds, gt_bboxes, gt_labels, img_metas, gt_bboxes_ignore=None)[source]¶ Compute losses of the head.
Parameters: - cls_scores (list[Tensor]) – Box scores for each scale level Has shape (N, num_anchors * num_classes, H, W)
- bbox_preds (list[Tensor]) – Box energies / deltas for each scale level with shape (N, num_anchors * 4, H, W)
- gt_bboxes (list[Tensor]) – Ground truth bboxes of each image with shape (num_obj, 4).
- gt_labels (list[Tensor]) – Ground truth labels of each image with shape (num_obj, 4).
- img_metas (list[dict]) – Meta information of each image, e.g., image size, scaling factor, etc.
- gt_bboxes_ignore (list[Tensor]) – Ignored gt bboxes of each image. Default: None.
Returns: - Loss dict, comprise classification loss, regression loss and
carl loss.
Return type: dict
-
class
mmdet.models.dense_heads.
PISASSDHead
(num_classes=80, in_channels=(512, 1024, 512, 256, 256, 256), stacked_convs=0, feat_channels=256, use_depthwise=False, conv_cfg=None, norm_cfg=None, act_cfg=None, anchor_generator={'basesize_ratio_range': (0.1, 0.9), 'input_size': 300, 'ratios': ([2], [2, 3], [2, 3], [2, 3], [2], [2]), 'scale_major': False, 'strides': [8, 16, 32, 64, 100, 300], 'type': 'SSDAnchorGenerator'}, bbox_coder={'clip_border': True, 'target_means': [0.0, 0.0, 0.0, 0.0], 'target_stds': [1.0, 1.0, 1.0, 1.0], 'type': 'DeltaXYWHBBoxCoder'}, reg_decoded_bbox=False, train_cfg=None, test_cfg=None, init_cfg={'bias': 0, 'distribution': 'uniform', 'layer': 'Conv2d', 'type': 'Xavier'})[source]¶ -
loss
(cls_scores, bbox_preds, gt_bboxes, gt_labels, img_metas, gt_bboxes_ignore=None)[source]¶ Compute losses of the head.
Parameters: - cls_scores (list[Tensor]) – Box scores for each scale level Has shape (N, num_anchors * num_classes, H, W)
- bbox_preds (list[Tensor]) – Box energies / deltas for each scale level with shape (N, num_anchors * 4, H, W)
- gt_bboxes (list[Tensor]) – Ground truth bboxes of each image with shape (num_obj, 4).
- gt_labels (list[Tensor]) – Ground truth labels of each image with shape (num_obj, 4).
- img_metas (list[dict]) – Meta information of each image, e.g., image size, scaling factor, etc.
- gt_bboxes_ignore (list[Tensor]) – Ignored gt bboxes of each image. Default: None.
Returns: - Loss dict, comprise classification loss regression loss and
carl loss.
Return type: dict
-
-
class
mmdet.models.dense_heads.
GFLHead
(num_classes, in_channels, stacked_convs=4, conv_cfg=None, norm_cfg={'num_groups': 32, 'requires_grad': True, 'type': 'GN'}, loss_dfl={'loss_weight': 0.25, 'type': 'DistributionFocalLoss'}, reg_max=16, init_cfg={'layer': 'Conv2d', 'override': {'bias_prob': 0.01, 'name': 'gfl_cls', 'std': 0.01, 'type': 'Normal'}, 'std': 0.01, 'type': 'Normal'}, **kwargs)[source]¶ Generalized Focal Loss: Learning Qualified and Distributed Bounding Boxes for Dense Object Detection.
GFL head structure is similar with ATSS, however GFL uses 1) joint representation for classification and localization quality, and 2) flexible General distribution for bounding box locations, which are supervised by Quality Focal Loss (QFL) and Distribution Focal Loss (DFL), respectively
https://arxiv.org/abs/2006.04388
Parameters: - num_classes (int) – Number of categories excluding the background category.
- in_channels (int) – Number of channels in the input feature map.
- stacked_convs (int) – Number of conv layers in cls and reg tower. Default: 4.
- conv_cfg (dict) – dictionary to construct and config conv layer. Default: None.
- norm_cfg (dict) – dictionary to construct and config norm layer. Default: dict(type=’GN’, num_groups=32, requires_grad=True).
- loss_qfl (dict) – Config of Quality Focal Loss (QFL).
- reg_max (int) – Max value of integral set :math: {0, …, reg_max} in QFL setting. Default: 16.
- init_cfg (dict or list[dict], optional) – Initialization config dict.
Example
>>> self = GFLHead(11, 7) >>> feats = [torch.rand(1, 7, s, s) for s in [4, 8, 16, 32, 64]] >>> cls_quality_score, bbox_pred = self.forward(feats) >>> assert len(cls_quality_score) == len(self.scales)
-
anchor_center
(anchors)[source]¶ Get anchor centers from anchors.
Parameters: anchors (Tensor) – Anchor list with shape (N, 4), “xyxy” format. Returns: Anchor centers with shape (N, 2), “xy” format. Return type: Tensor
-
forward
(feats)[source]¶ Forward features from the upstream network.
Parameters: feats (tuple[Tensor]) – Features from the upstream network, each is a 4D-tensor. Returns: - Usually a tuple of classification scores and bbox prediction
- cls_scores (list[Tensor]): Classification and quality (IoU)
- joint scores for all scale levels, each is a 4D-tensor, the channel number is num_classes.
- bbox_preds (list[Tensor]): Box distribution logits for all
- scale levels, each is a 4D-tensor, the channel number is 4*(n+1), n is max value of integral set.
Return type: tuple
-
forward_single
(x, scale)[source]¶ Forward feature of a single scale level.
Parameters: - x (Tensor) – Features of a single scale level.
- ( (scale) – obj: mmcv.cnn.Scale): Learnable scale module to resize the bbox prediction.
Returns: - cls_score (Tensor): Cls and quality joint scores for a single
scale level the channel number is num_classes.
- bbox_pred (Tensor): Box distribution logits for a single scale
level, the channel number is 4*(n+1), n is max value of integral set.
Return type: tuple
-
get_targets
(anchor_list, valid_flag_list, gt_bboxes_list, img_metas, gt_bboxes_ignore_list=None, gt_labels_list=None, label_channels=1, unmap_outputs=True)[source]¶ Get targets for GFL head.
This method is almost the same as AnchorHead.get_targets(). Besides returning the targets as the parent method does, it also returns the anchors as the first element of the returned tuple.
-
loss
(cls_scores, bbox_preds, gt_bboxes, gt_labels, img_metas, gt_bboxes_ignore=None)[source]¶ Compute losses of the head.
Parameters: - cls_scores (list[Tensor]) – Cls and quality scores for each scale level has shape (N, num_classes, H, W).
- bbox_preds (list[Tensor]) – Box distribution logits for each scale level with shape (N, 4*(n+1), H, W), n is max value of integral set.
- gt_bboxes (list[Tensor]) – Ground truth bboxes for each image with shape (num_gts, 4) in [tl_x, tl_y, br_x, br_y] format.
- gt_labels (list[Tensor]) – class indices corresponding to each box
- img_metas (list[dict]) – Meta information of each image, e.g., image size, scaling factor, etc.
- gt_bboxes_ignore (list[Tensor] | None) – specify which bounding boxes can be ignored when computing the loss.
Returns: A dictionary of loss components.
Return type: dict[str, Tensor]
-
loss_single
(anchors, cls_score, bbox_pred, labels, label_weights, bbox_targets, stride, num_total_samples)[source]¶ Compute loss of a single scale level.
Parameters: - anchors (Tensor) – Box reference for each scale level with shape (N, num_total_anchors, 4).
- cls_score (Tensor) – Cls and quality joint scores for each scale level has shape (N, num_classes, H, W).
- bbox_pred (Tensor) – Box distribution logits for each scale level with shape (N, 4*(n+1), H, W), n is max value of integral set.
- labels (Tensor) – Labels of each anchors with shape (N, num_total_anchors).
- label_weights (Tensor) – Label weights of each anchor with shape (N, num_total_anchors)
- bbox_targets (Tensor) – BBox regression targets of each anchor wight shape (N, num_total_anchors, 4).
- stride (tuple) – Stride in this scale level.
- num_total_samples (int) – Number of positive samples that is reduced over all GPUs.
Returns: A dictionary of loss components.
Return type: dict[str, Tensor]
-
class
mmdet.models.dense_heads.
CornerHead
(num_classes, in_channels, num_feat_levels=2, corner_emb_channels=1, train_cfg=None, test_cfg=None, loss_heatmap={'alpha': 2.0, 'gamma': 4.0, 'loss_weight': 1, 'type': 'GaussianFocalLoss'}, loss_embedding={'pull_weight': 0.25, 'push_weight': 0.25, 'type': 'AssociativeEmbeddingLoss'}, loss_offset={'beta': 1.0, 'loss_weight': 1, 'type': 'SmoothL1Loss'}, init_cfg=None)[source]¶ Head of CornerNet: Detecting Objects as Paired Keypoints.
Code is modified from the official github repo .
More details can be found in the paper .
Parameters: - num_classes (int) – Number of categories excluding the background category.
- in_channels (int) – Number of channels in the input feature map.
- num_feat_levels (int) – Levels of feature from the previous module. 2 for HourglassNet-104 and 1 for HourglassNet-52. Because HourglassNet-104 outputs the final feature and intermediate supervision feature and HourglassNet-52 only outputs the final feature. Default: 2.
- corner_emb_channels (int) – Channel of embedding vector. Default: 1.
- train_cfg (dict | None) – Training config. Useless in CornerHead, but we keep this variable for SingleStageDetector. Default: None.
- test_cfg (dict | None) – Testing config of CornerHead. Default: None.
- loss_heatmap (dict | None) – Config of corner heatmap loss. Default: GaussianFocalLoss.
- loss_embedding (dict | None) – Config of corner embedding loss. Default: AssociativeEmbeddingLoss.
- loss_offset (dict | None) – Config of corner offset loss. Default: SmoothL1Loss.
- init_cfg (dict or list[dict], optional) – Initialization config dict. Default: None
-
decode_heatmap
(tl_heat, br_heat, tl_off, br_off, tl_emb=None, br_emb=None, tl_centripetal_shift=None, br_centripetal_shift=None, img_meta=None, k=100, kernel=3, distance_threshold=0.5, num_dets=1000)[source]¶ Transform outputs for a single batch item into raw bbox predictions.
Parameters: - tl_heat (Tensor) – Top-left corner heatmap for current level with shape (N, num_classes, H, W).
- br_heat (Tensor) – Bottom-right corner heatmap for current level with shape (N, num_classes, H, W).
- tl_off (Tensor) – Top-left corner offset for current level with shape (N, corner_offset_channels, H, W).
- br_off (Tensor) – Bottom-right corner offset for current level with shape (N, corner_offset_channels, H, W).
- tl_emb (Tensor | None) – Top-left corner embedding for current level with shape (N, corner_emb_channels, H, W).
- br_emb (Tensor | None) – Bottom-right corner embedding for current level with shape (N, corner_emb_channels, H, W).
- tl_centripetal_shift (Tensor | None) – Top-left centripetal shift for current level with shape (N, 2, H, W).
- br_centripetal_shift (Tensor | None) – Bottom-right centripetal shift for current level with shape (N, 2, H, W).
- img_meta (dict) – Meta information of current image, e.g., image size, scaling factor, etc.
- k (int) – Get top k corner keypoints from heatmap.
- kernel (int) – Max pooling kernel for extract local maximum pixels.
- distance_threshold (float) – Distance threshold. Top-left and bottom-right corner keypoints with feature distance less than the threshold will be regarded as keypoints from same object.
- num_dets (int) – Num of raw boxes before doing nms.
Returns: Decoded output of CornerHead, containing the following Tensors:
- bboxes (Tensor): Coords of each box.
- scores (Tensor): Scores of each box.
- clses (Tensor): Categories of each box.
Return type: tuple[torch.Tensor]
-
forward
(feats)[source]¶ Forward features from the upstream network.
Parameters: feats (tuple[Tensor]) – Features from the upstream network, each is a 4D-tensor. Returns: Usually a tuple of corner heatmaps, offset heatmaps and embedding heatmaps. - tl_heats (list[Tensor]): Top-left corner heatmaps for all levels, each is a 4D-tensor, the channels number is num_classes.
- br_heats (list[Tensor]): Bottom-right corner heatmaps for all levels, each is a 4D-tensor, the channels number is num_classes.
- tl_embs (list[Tensor] | list[None]): Top-left embedding heatmaps for all levels, each is a 4D-tensor or None. If not None, the channels number is corner_emb_channels.
- br_embs (list[Tensor] | list[None]): Bottom-right embedding heatmaps for all levels, each is a 4D-tensor or None. If not None, the channels number is corner_emb_channels.
- tl_offs (list[Tensor]): Top-left offset heatmaps for all levels, each is a 4D-tensor. The channels number is corner_offset_channels.
- br_offs (list[Tensor]): Bottom-right offset heatmaps for all levels, each is a 4D-tensor. The channels number is corner_offset_channels.
Return type: tuple
-
forward_single
(x, lvl_ind, return_pool=False)[source]¶ Forward feature of a single level.
Parameters: - x (Tensor) – Feature of a single level.
- lvl_ind (int) – Level index of current feature.
- return_pool (bool) – Return corner pool feature or not.
Returns: A tuple of CornerHead’s output for current feature level. Containing the following Tensors:
- tl_heat (Tensor): Predicted top-left corner heatmap.
- br_heat (Tensor): Predicted bottom-right corner heatmap.
- tl_emb (Tensor | None): Predicted top-left embedding heatmap. None for self.with_corner_emb == False.
- br_emb (Tensor | None): Predicted bottom-right embedding heatmap. None for self.with_corner_emb == False.
- tl_off (Tensor): Predicted top-left offset heatmap.
- br_off (Tensor): Predicted bottom-right offset heatmap.
- tl_pool (Tensor): Top-left corner pool feature. Not must have.
- br_pool (Tensor): Bottom-right corner pool feature. Not must have.
Return type: tuple[Tensor]
-
get_bboxes
(tl_heats, br_heats, tl_embs, br_embs, tl_offs, br_offs, img_metas, rescale=False, with_nms=True)[source]¶ Transform network output for a batch into bbox predictions.
Parameters: - tl_heats (list[Tensor]) – Top-left corner heatmaps for each level with shape (N, num_classes, H, W).
- br_heats (list[Tensor]) – Bottom-right corner heatmaps for each level with shape (N, num_classes, H, W).
- tl_embs (list[Tensor]) – Top-left corner embeddings for each level with shape (N, corner_emb_channels, H, W).
- br_embs (list[Tensor]) – Bottom-right corner embeddings for each level with shape (N, corner_emb_channels, H, W).
- tl_offs (list[Tensor]) – Top-left corner offsets for each level with shape (N, corner_offset_channels, H, W).
- br_offs (list[Tensor]) – Bottom-right corner offsets for each level with shape (N, corner_offset_channels, H, W).
- img_metas (list[dict]) – Meta information of each image, e.g., image size, scaling factor, etc.
- rescale (bool) – If True, return boxes in original image space. Default: False.
- with_nms (bool) – If True, do nms before return boxes. Default: True.
-
get_targets
(gt_bboxes, gt_labels, feat_shape, img_shape, with_corner_emb=False, with_guiding_shift=False, with_centripetal_shift=False)[source]¶ Generate corner targets.
Including corner heatmap, corner offset.
Optional: corner embedding, corner guiding shift, centripetal shift.
For CornerNet, we generate corner heatmap, corner offset and corner embedding from this function.
For CentripetalNet, we generate corner heatmap, corner offset, guiding shift and centripetal shift from this function.
Parameters: - gt_bboxes (list[Tensor]) – Ground truth bboxes of each image, each has shape (num_gt, 4).
- gt_labels (list[Tensor]) – Ground truth labels of each box, each has shape (num_gt,).
- feat_shape (list[int]) – Shape of output feature, [batch, channel, height, width].
- img_shape (list[int]) – Shape of input image, [height, width, channel].
- with_corner_emb (bool) – Generate corner embedding target or not. Default: False.
- with_guiding_shift (bool) – Generate guiding shift target or not. Default: False.
- with_centripetal_shift (bool) – Generate centripetal shift target or not. Default: False.
Returns: Ground truth of corner heatmap, corner offset, corner embedding, guiding shift and centripetal shift. Containing the following keys:
- topleft_heatmap (Tensor): Ground truth top-left corner heatmap.
- bottomright_heatmap (Tensor): Ground truth bottom-right corner heatmap.
- topleft_offset (Tensor): Ground truth top-left corner offset.
- bottomright_offset (Tensor): Ground truth bottom-right corner offset.
- corner_embedding (list[list[list[int]]]): Ground truth corner embedding. Not must have.
- topleft_guiding_shift (Tensor): Ground truth top-left corner guiding shift. Not must have.
- bottomright_guiding_shift (Tensor): Ground truth bottom-right corner guiding shift. Not must have.
- topleft_centripetal_shift (Tensor): Ground truth top-left corner centripetal shift. Not must have.
- bottomright_centripetal_shift (Tensor): Ground truth bottom-right corner centripetal shift. Not must have.
Return type: dict
-
loss
(tl_heats, br_heats, tl_embs, br_embs, tl_offs, br_offs, gt_bboxes, gt_labels, img_metas, gt_bboxes_ignore=None)[source]¶ Compute losses of the head.
Parameters: - tl_heats (list[Tensor]) – Top-left corner heatmaps for each level with shape (N, num_classes, H, W).
- br_heats (list[Tensor]) – Bottom-right corner heatmaps for each level with shape (N, num_classes, H, W).
- tl_embs (list[Tensor]) – Top-left corner embeddings for each level with shape (N, corner_emb_channels, H, W).
- br_embs (list[Tensor]) – Bottom-right corner embeddings for each level with shape (N, corner_emb_channels, H, W).
- tl_offs (list[Tensor]) – Top-left corner offsets for each level with shape (N, corner_offset_channels, H, W).
- br_offs (list[Tensor]) – Bottom-right corner offsets for each level with shape (N, corner_offset_channels, H, W).
- gt_bboxes (list[Tensor]) – Ground truth bboxes for each image with shape (num_gts, 4) in [left, top, right, bottom] format.
- gt_labels (list[Tensor]) – Class indices corresponding to each box.
- img_metas (list[dict]) – Meta information of each image, e.g., image size, scaling factor, etc.
- gt_bboxes_ignore (list[Tensor] | None) – Specify which bounding boxes can be ignored when computing the loss.
Returns: A dictionary of loss components. Containing the following losses:
- det_loss (list[Tensor]): Corner keypoint losses of all feature levels.
- pull_loss (list[Tensor]): Part one of AssociativeEmbedding losses of all feature levels.
- push_loss (list[Tensor]): Part two of AssociativeEmbedding losses of all feature levels.
- off_loss (list[Tensor]): Corner offset losses of all feature levels.
Return type: dict[str, Tensor]
-
loss_single
(tl_hmp, br_hmp, tl_emb, br_emb, tl_off, br_off, targets)[source]¶ Compute losses for single level.
Parameters: - tl_hmp (Tensor) – Top-left corner heatmap for current level with shape (N, num_classes, H, W).
- br_hmp (Tensor) – Bottom-right corner heatmap for current level with shape (N, num_classes, H, W).
- tl_emb (Tensor) – Top-left corner embedding for current level with shape (N, corner_emb_channels, H, W).
- br_emb (Tensor) – Bottom-right corner embedding for current level with shape (N, corner_emb_channels, H, W).
- tl_off (Tensor) – Top-left corner offset for current level with shape (N, corner_offset_channels, H, W).
- br_off (Tensor) – Bottom-right corner offset for current level with shape (N, corner_offset_channels, H, W).
- targets (dict) – Corner target generated by get_targets.
Returns: Losses of the head’s differnet branches containing the following losses:
- det_loss (Tensor): Corner keypoint loss.
- pull_loss (Tensor): Part one of AssociativeEmbedding loss.
- push_loss (Tensor): Part two of AssociativeEmbedding loss.
- off_loss (Tensor): Corner offset loss.
Return type: tuple[torch.Tensor]
-
class
mmdet.models.dense_heads.
YOLACTHead
(num_classes, in_channels, anchor_generator={'octave_base_scale': 3, 'ratios': [0.5, 1.0, 2.0], 'scales_per_octave': 1, 'strides': [8, 16, 32, 64, 128], 'type': 'AnchorGenerator'}, loss_cls={'loss_weight': 1.0, 'reduction': 'none', 'type': 'CrossEntropyLoss', 'use_sigmoid': False}, loss_bbox={'beta': 1.0, 'loss_weight': 1.5, 'type': 'SmoothL1Loss'}, num_head_convs=1, num_protos=32, use_ohem=True, conv_cfg=None, norm_cfg=None, init_cfg={'bias': 0, 'distribution': 'uniform', 'layer': 'Conv2d', 'type': 'Xavier'}, **kwargs)[source]¶ YOLACT box head used in https://arxiv.org/abs/1904.02689.
Note that YOLACT head is a light version of RetinaNet head. Four differences are described as follows:
- YOLACT box head has three-times fewer anchors.
- YOLACT box head shares the convs for box and cls branches.
- YOLACT box head uses OHEM instead of Focal loss.
- YOLACT box head predicts a set of mask coefficients for each box.
Parameters: - num_classes (int) – Number of categories excluding the background category.
- in_channels (int) – Number of channels in the input feature map.
- anchor_generator (dict) – Config dict for anchor generator
- loss_cls (dict) – Config of classification loss.
- loss_bbox (dict) – Config of localization loss.
- num_head_convs (int) – Number of the conv layers shared by box and cls branches.
- num_protos (int) – Number of the mask coefficients.
- use_ohem (bool) – If true,
loss_single_OHEM
will be used for cls loss calculation. If false,loss_single
will be used. - conv_cfg (dict) – Dictionary to construct and config conv layer.
- norm_cfg (dict) – Dictionary to construct and config norm layer.
- init_cfg (dict or list[dict], optional) – Initialization config dict.
-
forward_single
(x)[source]¶ Forward feature of a single scale level.
Parameters: x (Tensor) – Features of a single scale level. Returns: cls_score (Tensor): Cls scores for a single scale level the channels number is num_anchors * num_classes. bbox_pred (Tensor): Box energies / deltas for a single scale level, the channels number is num_anchors * 4. coeff_pred (Tensor): Mask coefficients for a single scale level, the channels number is num_anchors * num_protos. Return type: tuple
-
get_bboxes
(cls_scores, bbox_preds, coeff_preds, img_metas, cfg=None, rescale=False)[source]¶ “Similiar to func:
AnchorHead.get_bboxes
, but additionally processes coeff_preds.Parameters: - cls_scores (list[Tensor]) – Box scores for each scale level with shape (N, num_anchors * num_classes, H, W)
- bbox_preds (list[Tensor]) – Box energies / deltas for each scale level with shape (N, num_anchors * 4, H, W)
- coeff_preds (list[Tensor]) – Mask coefficients for each scale level with shape (N, num_anchors * num_protos, H, W)
- img_metas (list[dict]) – Meta information of each image, e.g., image size, scaling factor, etc.
- cfg (mmcv.Config | None) – Test / postprocessing configuration, if None, test_cfg would be used
- rescale (bool) – If True, return boxes in original image space. Default: False.
Returns: - Each item in result_list is
a 3-tuple. The first item is an (n, 5) tensor, where the first 4 columns are bounding box positions (tl_x, tl_y, br_x, br_y) and the 5-th column is a score between 0 and 1. The second item is an (n,) tensor where each item is the predicted class label of the corresponding box. The third item is an (n, num_protos) tensor where each item is the predicted mask coefficients of instance inside the corresponding box.
Return type: list[tuple[Tensor, Tensor, Tensor]]
-
loss
(cls_scores, bbox_preds, gt_bboxes, gt_labels, img_metas, gt_bboxes_ignore=None)[source]¶ A combination of the func:
AnchorHead.loss
and func:SSDHead.loss
.When
self.use_ohem == True
, it functions likeSSDHead.loss
, otherwise, it followsAnchorHead.loss
. Besides, it additionally returnssampling_results
.Parameters: - cls_scores (list[Tensor]) – Box scores for each scale level Has shape (N, num_anchors * num_classes, H, W)
- bbox_preds (list[Tensor]) – Box energies / deltas for each scale level with shape (N, num_anchors * 4, H, W)
- gt_bboxes (list[Tensor]) – Ground truth bboxes for each image with shape (num_gts, 4) in [tl_x, tl_y, br_x, br_y] format.
- gt_labels (list[Tensor]) – Class indices corresponding to each box
- img_metas (list[dict]) – Meta information of each image, e.g., image size, scaling factor, etc.
- gt_bboxes_ignore (None | list[Tensor]) – Specify which bounding boxes can be ignored when computing the loss. Default: None
Returns: dict[str, Tensor]: A dictionary of loss components. List[:obj:
SamplingResult
]: Sampler results for each image.Return type: tuple
-
class
mmdet.models.dense_heads.
YOLACTSegmHead
(num_classes, in_channels=256, loss_segm={'loss_weight': 1.0, 'type': 'CrossEntropyLoss', 'use_sigmoid': True}, init_cfg={'distribution': 'uniform', 'override': {'name': 'segm_conv'}, 'type': 'Xavier'})[source]¶ YOLACT segmentation head used in https://arxiv.org/abs/1904.02689.
Apply a semantic segmentation loss on feature space using layers that are only evaluated during training to increase performance with no speed penalty.
Parameters: - in_channels (int) – Number of channels in the input feature map.
- num_classes (int) – Number of categories excluding the background category.
- loss_segm (dict) – Config of semantic segmentation loss.
- init_cfg (dict or list[dict], optional) – Initialization config dict.
-
forward
(x)[source]¶ Forward feature from the upstream network.
Parameters: x (Tensor) – Feature from the upstream network, which is a 4D-tensor. Returns: - Predicted semantic segmentation map with shape
- (N, num_classes, H, W).
Return type: Tensor
-
get_targets
(segm_pred, gt_masks, gt_labels)[source]¶ Compute semantic segmentation targets for each image.
Parameters: - segm_pred (Tensor) – Predicted semantic segmentation map with shape (num_classes, H, W).
- gt_masks (Tensor) – Ground truth masks for each image with the same shape of the input image.
- gt_labels (Tensor) – Class indices corresponding to each box.
Returns: - Semantic segmentation targets with shape
(num_classes, H, W).
Return type: Tensor
-
loss
(segm_pred, gt_masks, gt_labels)[source]¶ Compute loss of the head.
Parameters: - segm_pred (list[Tensor]) – Predicted semantic segmentation map with shape (N, num_classes, H, W).
- gt_masks (list[Tensor]) – Ground truth masks for each image with the same shape of the input image.
- gt_labels (list[Tensor]) – Class indices corresponding to each box.
Returns: A dictionary of loss components.
Return type: dict[str, Tensor]
-
class
mmdet.models.dense_heads.
YOLACTProtonet
(num_classes, in_channels=256, proto_channels=(256, 256, 256, None, 256, 32), proto_kernel_sizes=(3, 3, 3, -2, 3, 1), include_last_relu=True, num_protos=32, loss_mask_weight=1.0, max_masks_to_train=100, init_cfg={'distribution': 'uniform', 'override': {'name': 'protonet'}, 'type': 'Xavier'})[source]¶ YOLACT mask head used in https://arxiv.org/abs/1904.02689.
This head outputs the mask prototypes for YOLACT.
Parameters: - in_channels (int) – Number of channels in the input feature map.
- proto_channels (tuple[int]) – Output channels of protonet convs.
- proto_kernel_sizes (tuple[int]) – Kernel sizes of protonet convs.
- include_last_relu (Bool) – If keep the last relu of protonet.
- num_protos (int) – Number of prototypes.
- num_classes (int) – Number of categories excluding the background category.
- loss_mask_weight (float) – Reweight the mask loss by this factor.
- max_masks_to_train (int) – Maximum number of masks to train for each image.
- init_cfg (dict or list[dict], optional) – Initialization config dict.
-
crop
(masks, boxes, padding=1)[source]¶ Crop predicted masks by zeroing out everything not in the predicted bbox.
Parameters: - masks (Tensor) – shape [H, W, N].
- boxes (Tensor) – bbox coords in relative point form with shape [N, 4].
Returns: The cropped masks.
Return type: Tensor
-
forward
(x, coeff_pred, bboxes, img_meta, sampling_results=None)[source]¶ Forward feature from the upstream network to get prototypes and linearly combine the prototypes, using masks coefficients, into instance masks. Finally, crop the instance masks with given bboxes.
Parameters: - x (Tensor) – Feature from the upstream network, which is a 4D-tensor.
- coeff_pred (list[Tensor]) – Mask coefficients for each scale level with shape (N, num_anchors * num_protos, H, W).
- bboxes (list[Tensor]) – Box used for cropping with shape (N, num_anchors * 4, H, W). During training, they are ground truth boxes. During testing, they are predicted boxes.
- img_meta (list[dict]) – Meta information of each image, e.g., image size, scaling factor, etc.
- sampling_results (List[:obj:
SamplingResult
]) – Sampler results for each image.
Returns: Predicted instance segmentation masks.
Return type: list[Tensor]
-
get_seg_masks
(mask_pred, label_pred, img_meta, rescale)[source]¶ Resize, binarize, and format the instance mask predictions.
Parameters: - mask_pred (Tensor) – shape (N, H, W).
- label_pred (Tensor) – shape (N, ).
- img_meta (dict) – Meta information of each image, e.g., image size, scaling factor, etc.
- rescale (bool) – If rescale is False, then returned masks will fit the scale of imgs[0].
Returns: Mask predictions grouped by their predicted classes.
Return type: list[ndarray]
-
get_targets
(mask_pred, gt_masks, pos_assigned_gt_inds)[source]¶ Compute instance segmentation targets for each image.
Parameters: - mask_pred (Tensor) – Predicted prototypes with shape (num_classes, H, W).
- gt_masks (Tensor) – Ground truth masks for each image with the same shape of the input image.
- pos_assigned_gt_inds (Tensor) – GT indices of the corresponding positive samples.
Returns: - Instance segmentation targets with shape
(num_instances, H, W).
Return type: Tensor
-
loss
(mask_pred, gt_masks, gt_bboxes, img_meta, sampling_results)[source]¶ Compute loss of the head.
Parameters: - mask_pred (list[Tensor]) – Predicted prototypes with shape (num_classes, H, W).
- gt_masks (list[Tensor]) – Ground truth masks for each image with the same shape of the input image.
- gt_bboxes (list[Tensor]) – Ground truth bboxes for each image with shape (num_gts, 4) in [tl_x, tl_y, br_x, br_y] format.
- img_meta (list[dict]) – Meta information of each image, e.g., image size, scaling factor, etc.
- sampling_results (List[:obj:
SamplingResult
]) – Sampler results for each image.
Returns: A dictionary of loss components.
Return type: dict[str, Tensor]
-
sanitize_coordinates
(x1, x2, img_size, padding=0, cast=True)[source]¶ Sanitizes the input coordinates so that x1 < x2, x1 != x2, x1 >= 0, and x2 <= image_size. Also converts from relative to absolute coordinates and casts the results to long tensors.
Warning: this does things in-place behind the scenes so copy if necessary.
Parameters: - _x1 (Tensor) – shape (N, ).
- _x2 (Tensor) – shape (N, ).
- img_size (int) – Size of the input image.
- padding (int) – x1 >= padding, x2 <= image_size-padding.
- cast (bool) – If cast is false, the result won’t be cast to longs.
Returns: x1 (Tensor): Sanitized _x1. x2 (Tensor): Sanitized _x2.
Return type: tuple
-
simple_test
(feats, det_bboxes, det_labels, det_coeffs, img_metas, rescale=False)[source]¶ Test function without test-time augmentation.
Parameters: - feats (tuple[torch.Tensor]) – Multi-level features from the upstream network, each is a 4D-tensor.
- det_bboxes (list[Tensor]) – BBox results of each image. each element is (n, 5) tensor, where 5 represent (tl_x, tl_y, br_x, br_y, score) and the score between 0 and 1.
- det_labels (list[Tensor]) – BBox results of each image. each element is (n, ) tensor, each element represents the class label of the corresponding box.
- det_coeffs (list[Tensor]) – BBox coefficient of each image. each element is (n, m) tensor, m is vector length.
- img_metas (list[dict]) – Meta information of each image, e.g., image size, scaling factor, etc.
- rescale (bool, optional) – Whether to rescale the results. Defaults to False.
Returns: - encoded masks. The c-th item in the outer list
corresponds to the c-th class. Given the c-th outer list, the i-th item in that inner list is the mask for the i-th box with class label c.
Return type: list[list]
-
class
mmdet.models.dense_heads.
YOLOV3Head
(num_classes, in_channels, out_channels=(1024, 512, 256), anchor_generator={'base_sizes': [[(116, 90), (156, 198), (373, 326)], [(30, 61), (62, 45), (59, 119)], [(10, 13), (16, 30), (33, 23)]], 'strides': [32, 16, 8], 'type': 'YOLOAnchorGenerator'}, bbox_coder={'type': 'YOLOBBoxCoder'}, featmap_strides=[32, 16, 8], one_hot_smoother=0.0, conv_cfg=None, norm_cfg={'requires_grad': True, 'type': 'BN'}, act_cfg={'negative_slope': 0.1, 'type': 'LeakyReLU'}, loss_cls={'loss_weight': 1.0, 'type': 'CrossEntropyLoss', 'use_sigmoid': True}, loss_conf={'loss_weight': 1.0, 'type': 'CrossEntropyLoss', 'use_sigmoid': True}, loss_xy={'loss_weight': 1.0, 'type': 'CrossEntropyLoss', 'use_sigmoid': True}, loss_wh={'loss_weight': 1.0, 'type': 'MSELoss'}, train_cfg=None, test_cfg=None, init_cfg={'override': {'name': 'convs_pred'}, 'std': 0.01, 'type': 'Normal'})[source]¶ YOLOV3Head Paper link: https://arxiv.org/abs/1804.02767.
Parameters: - num_classes (int) – The number of object classes (w/o background)
- in_channels (List[int]) – Number of input channels per scale.
- out_channels (List[int]) – The number of output channels per scale before the final 1x1 layer. Default: (1024, 512, 256).
- anchor_generator (dict) – Config dict for anchor generator
- bbox_coder (dict) – Config of bounding box coder.
- featmap_strides (List[int]) – The stride of each scale. Should be in descending order. Default: (32, 16, 8).
- one_hot_smoother (float) – Set a non-zero value to enable label-smooth Default: 0.
- conv_cfg (dict) – Config dict for convolution layer. Default: None.
- norm_cfg (dict) – Dictionary to construct and config norm layer. Default: dict(type=’BN’, requires_grad=True)
- act_cfg (dict) – Config dict for activation layer. Default: dict(type=’LeakyReLU’, negative_slope=0.1).
- loss_cls (dict) – Config of classification loss.
- loss_conf (dict) – Config of confidence loss.
- loss_xy (dict) – Config of xy coordinate loss.
- loss_wh (dict) – Config of wh coordinate loss.
- train_cfg (dict) – Training config of YOLOV3 head. Default: None.
- test_cfg (dict) – Testing config of YOLOV3 head. Default: None.
- init_cfg (dict or list[dict], optional) – Initialization config dict.
-
aug_test
(feats, img_metas, rescale=False)[source]¶ Test function with test time augmentation.
Parameters: - feats (list[Tensor]) – the outer list indicates test-time augmentations and inner Tensor should have a shape NxCxHxW, which contains features for all images in the batch.
- img_metas (list[list[dict]]) – the outer list indicates test-time augs (multiscale, flip, etc.) and the inner list indicates images in a batch. each dict has image information.
- rescale (bool, optional) – Whether to rescale the results. Defaults to False.
Returns: bbox results of each class
Return type: list[ndarray]
-
forward
(feats)[source]¶ Forward features from the upstream network.
Parameters: feats (tuple[Tensor]) – Features from the upstream network, each is a 4D-tensor. Returns: - A tuple of multi-level predication map, each is a
- 4D-tensor of shape (batch_size, 5+num_classes, height, width).
Return type: tuple[Tensor]
-
get_bboxes
(pred_maps, img_metas, cfg=None, rescale=False, with_nms=True)[source]¶ Transform network output for a batch into bbox predictions.
Parameters: - pred_maps (list[Tensor]) – Raw predictions for a batch of images.
- img_metas (list[dict]) – Meta information of each image, e.g., image size, scaling factor, etc.
- cfg (mmcv.Config | None) – Test / postprocessing configuration, if None, test_cfg would be used. Default: None.
- rescale (bool) – If True, return boxes in original image space. Default: False.
- with_nms (bool) – If True, do nms before return boxes. Default: True.
Returns: - Each item in result_list is 2-tuple.
The first item is an (n, 5) tensor, where 5 represent (tl_x, tl_y, br_x, br_y, score) and the score between 0 and 1. The shape of the second tensor in the tuple is (n,), and each element represents the class label of the corresponding box.
Return type: list[tuple[Tensor, Tensor]]
-
get_targets
(anchor_list, responsible_flag_list, gt_bboxes_list, gt_labels_list)[source]¶ Compute target maps for anchors in multiple images.
Parameters: - anchor_list (list[list[Tensor]]) – Multi level anchors of each image. The outer list indicates images, and the inner list corresponds to feature levels of the image. Each element of the inner list is a tensor of shape (num_total_anchors, 4).
- responsible_flag_list (list[list[Tensor]]) – Multi level responsible flags of each image. Each element is a tensor of shape (num_total_anchors, )
- gt_bboxes_list (list[Tensor]) – Ground truth bboxes of each image.
- gt_labels_list (list[Tensor]) – Ground truth labels of each box.
Returns: - Usually returns a tuple containing learning targets.
- target_map_list (list[Tensor]): Target map of each level.
- neg_map_list (list[Tensor]): Negative map of each level.
Return type: tuple
-
loss
(pred_maps, gt_bboxes, gt_labels, img_metas, gt_bboxes_ignore=None)[source]¶ Compute loss of the head.
Parameters: - pred_maps (list[Tensor]) – Prediction map for each scale level, shape (N, num_anchors * num_attrib, H, W)
- gt_bboxes (list[Tensor]) – Ground truth bboxes for each image with shape (num_gts, 4) in [tl_x, tl_y, br_x, br_y] format.
- gt_labels (list[Tensor]) – class indices corresponding to each box
- img_metas (list[dict]) – Meta information of each image, e.g., image size, scaling factor, etc.
- gt_bboxes_ignore (None | list[Tensor]) – specify which bounding boxes can be ignored when computing the loss.
Returns: A dictionary of loss components.
Return type: dict[str, Tensor]
-
loss_single
(pred_map, target_map, neg_map)[source]¶ Compute loss of a single image from a batch.
Parameters: - pred_map (Tensor) – Raw predictions for a single level.
- target_map (Tensor) – The Ground-Truth target for a single level.
- neg_map (Tensor) – The negative masks for a single level.
Returns: loss_cls (Tensor): Classification loss. loss_conf (Tensor): Confidence loss. loss_xy (Tensor): Regression loss of x, y coordinate. loss_wh (Tensor): Regression loss of w, h coordinate.
Return type: tuple
-
num_attrib
¶ number of attributes in pred_map, bboxes (4) + objectness (1) + num_classes
Type: int
-
class
mmdet.models.dense_heads.
PAAHead
(*args, topk=9, score_voting=True, covariance_type='diag', **kwargs)[source]¶ Head of PAAAssignment: Probabilistic Anchor Assignment with IoU Prediction for Object Detection.
Code is modified from the official github repo.
More details can be found in the paper .
Parameters: - topk (int) – Select topk samples with smallest loss in each level.
- score_voting (bool) – Whether to use score voting in post-process.
- covariance_type –
String describing the type of covariance parameters to be used in
sklearn.mixture.GaussianMixture
. It must be one of:- ’full’: each component has its own general covariance matrix
- ’tied’: all components share the same general covariance matrix
- ’diag’: each component has its own diagonal covariance matrix
- ’spherical’: each component has its own single variance
Default: ‘diag’. From ‘full’ to ‘spherical’, the gmm fitting process is faster yet the performance could be influenced. For most cases, ‘diag’ should be a good choice.
-
get_pos_loss
(anchors, cls_score, bbox_pred, label, label_weight, bbox_target, bbox_weight, pos_inds)[source]¶ Calculate loss of all potential positive samples obtained from first match process.
Parameters: - anchors (list[Tensor]) – Anchors of each scale.
- cls_score (Tensor) – Box scores of single image with shape (num_anchors, num_classes)
- bbox_pred (Tensor) – Box energies / deltas of single image with shape (num_anchors, 4)
- label (Tensor) – classification target of each anchor with shape (num_anchors,)
- label_weight (Tensor) – Classification loss weight of each anchor with shape (num_anchors).
- bbox_target (dict) – Regression target of each anchor with shape (num_anchors, 4).
- bbox_weight (Tensor) – Bbox weight of each anchor with shape (num_anchors, 4).
- pos_inds (Tensor) – Index of all positive samples got from first assign process.
Returns: Losses of all positive samples in single image.
Return type: Tensor
-
get_targets
(anchor_list, valid_flag_list, gt_bboxes_list, img_metas, gt_bboxes_ignore_list=None, gt_labels_list=None, label_channels=1, unmap_outputs=True)[source]¶ Get targets for PAA head.
This method is almost the same as AnchorHead.get_targets(). We direct return the results from _get_targets_single instead map it to levels by images_to_levels function.
Parameters: - anchor_list (list[list[Tensor]]) – Multi level anchors of each image. The outer list indicates images, and the inner list corresponds to feature levels of the image. Each element of the inner list is a tensor of shape (num_anchors, 4).
- valid_flag_list (list[list[Tensor]]) – Multi level valid flags of each image. The outer list indicates images, and the inner list corresponds to feature levels of the image. Each element of the inner list is a tensor of shape (num_anchors, )
- gt_bboxes_list (list[Tensor]) – Ground truth bboxes of each image.
- img_metas (list[dict]) – Meta info of each image.
- gt_bboxes_ignore_list (list[Tensor]) – Ground truth bboxes to be ignored.
- gt_labels_list (list[Tensor]) – Ground truth labels of each box.
- label_channels (int) – Channel of label.
- unmap_outputs (bool) – Whether to map outputs back to the original set of anchors.
Returns: Usually returns a tuple containing learning targets.
- labels (list[Tensor]): Labels of all anchors, each with
- shape (num_anchors,).
- label_weights (list[Tensor]): Label weights of all anchor.
- each with shape (num_anchors,).
- bbox_targets (list[Tensor]): BBox targets of all anchors.
- each with shape (num_anchors, 4).
- bbox_weights (list[Tensor]): BBox weights of all anchors.
- each with shape (num_anchors, 4).
- pos_inds (list[Tensor]): Contains all index of positive
- sample in all anchor.
- gt_inds (list[Tensor]): Contains all gt_index of positive
- sample in all anchor.
Return type: tuple
-
gmm_separation_scheme
(gmm_assignment, scores, pos_inds_gmm)[source]¶ A general separation scheme for gmm model.
It separates a GMM distribution of candidate samples into three parts, 0 1 and uncertain areas, and you can implement other separation schemes by rewriting this function.
Parameters: - gmm_assignment (Tensor) – The prediction of GMM which is of shape (num_samples,). The 0/1 value indicates the distribution that each sample comes from.
- scores (Tensor) – The probability of sample coming from the fit GMM distribution. The tensor is of shape (num_samples,).
- pos_inds_gmm (Tensor) – All the indexes of samples which are used to fit GMM model. The tensor is of shape (num_samples,)
Returns: The indices of positive and ignored samples.
- pos_inds_temp (Tensor): Indices of positive samples.
- ignore_inds_temp (Tensor): Indices of ignore samples.
Return type: tuple[Tensor]
-
loss
(cls_scores, bbox_preds, iou_preds, gt_bboxes, gt_labels, img_metas, gt_bboxes_ignore=None)[source]¶ Compute losses of the head.
Parameters: - cls_scores (list[Tensor]) – Box scores for each scale level Has shape (N, num_anchors * num_classes, H, W)
- bbox_preds (list[Tensor]) – Box energies / deltas for each scale level with shape (N, num_anchors * 4, H, W)
- iou_preds (list[Tensor]) – iou_preds for each scale level with shape (N, num_anchors * 1, H, W)
- gt_bboxes (list[Tensor]) – Ground truth bboxes for each image with shape (num_gts, 4) in [tl_x, tl_y, br_x, br_y] format.
- gt_labels (list[Tensor]) – class indices corresponding to each box
- img_metas (list[dict]) – Meta information of each image, e.g., image size, scaling factor, etc.
- gt_bboxes_ignore (list[Tensor] | None) – Specify which bounding boxes can be ignored when are computing the loss.
Returns: A dictionary of loss gmm_assignment.
Return type: dict[str, Tensor]
-
paa_reassign
(pos_losses, label, label_weight, bbox_weight, pos_inds, pos_gt_inds, anchors)[source]¶ Fit loss to GMM distribution and separate positive, ignore, negative samples again with GMM model.
Parameters: - pos_losses (Tensor) – Losses of all positive samples in single image.
- label (Tensor) – classification target of each anchor with shape (num_anchors,)
- label_weight (Tensor) – Classification loss weight of each anchor with shape (num_anchors).
- bbox_weight (Tensor) – Bbox weight of each anchor with shape (num_anchors, 4).
- pos_inds (Tensor) – Index of all positive samples got from first assign process.
- pos_gt_inds (Tensor) – Gt_index of all positive samples got from first assign process.
- anchors (list[Tensor]) – Anchors of each scale.
Returns: Usually returns a tuple containing learning targets.
- label (Tensor): classification target of each anchor after paa assign, with shape (num_anchors,)
- label_weight (Tensor): Classification loss weight of each anchor after paa assign, with shape (num_anchors).
- bbox_weight (Tensor): Bbox weight of each anchor with shape (num_anchors, 4).
- num_pos (int): The number of positive samples after paa assign.
Return type: tuple
-
score_voting
(det_bboxes, det_labels, mlvl_bboxes, mlvl_nms_scores, score_thr)[source]¶ Implementation of score voting method works on each remaining boxes after NMS procedure.
Parameters: - det_bboxes (Tensor) – Remaining boxes after NMS procedure, with shape (k, 5), each dimension means (x1, y1, x2, y2, score).
- det_labels (Tensor) – The label of remaining boxes, with shape (k, 1),Labels are 0-based.
- mlvl_bboxes (Tensor) – All boxes before the NMS procedure, with shape (num_anchors,4).
- mlvl_nms_scores (Tensor) – The scores of all boxes which is used in the NMS procedure, with shape (num_anchors, num_class)
- mlvl_iou_preds (Tensor) – The predictions of IOU of all boxes before the NMS procedure, with shape (num_anchors, 1)
- score_thr (float) – The score threshold of bboxes.
Returns: Usually returns a tuple containing voting results.
- det_bboxes_voted (Tensor): Remaining boxes after
- score voting procedure, with shape (k, 5), each dimension means (x1, y1, x2, y2, score).
- det_labels_voted (Tensor): Label of remaining bboxes
- after voting, with shape (num_anchors,).
Return type: tuple
-
class
mmdet.models.dense_heads.
SABLRetinaHead
(num_classes, in_channels, stacked_convs=4, feat_channels=256, approx_anchor_generator={'octave_base_scale': 4, 'ratios': [0.5, 1.0, 2.0], 'scales_per_octave': 3, 'strides': [8, 16, 32, 64, 128], 'type': 'AnchorGenerator'}, square_anchor_generator={'ratios': [1.0], 'scales': [4], 'strides': [8, 16, 32, 64, 128], 'type': 'AnchorGenerator'}, conv_cfg=None, norm_cfg=None, bbox_coder={'num_buckets': 14, 'scale_factor': 3.0, 'type': 'BucketingBBoxCoder'}, reg_decoded_bbox=False, train_cfg=None, test_cfg=None, loss_cls={'alpha': 0.25, 'gamma': 2.0, 'loss_weight': 1.0, 'type': 'FocalLoss', 'use_sigmoid': True}, loss_bbox_cls={'loss_weight': 1.5, 'type': 'CrossEntropyLoss', 'use_sigmoid': True}, loss_bbox_reg={'beta': 0.1111111111111111, 'loss_weight': 1.5, 'type': 'SmoothL1Loss'}, init_cfg={'layer': 'Conv2d', 'override': {'bias_prob': 0.01, 'name': 'retina_cls', 'std': 0.01, 'type': 'Normal'}, 'std': 0.01, 'type': 'Normal'})[source]¶ Side-Aware Boundary Localization (SABL) for RetinaNet.
The anchor generation, assigning and sampling in SABLRetinaHead are the same as GuidedAnchorHead for guided anchoring.
Please refer to https://arxiv.org/abs/1912.04260 for more details.
Parameters: - num_classes (int) – Number of classes.
- in_channels (int) – Number of channels in the input feature map.
- stacked_convs (int) – Number of Convs for classification and regression branches. Defaults to 4.
- feat_channels (int) – Number of hidden channels. Defaults to 256.
- approx_anchor_generator (dict) – Config dict for approx generator.
- square_anchor_generator (dict) – Config dict for square generator.
- conv_cfg (dict) – Config dict for ConvModule. Defaults to None.
- norm_cfg (dict) – Config dict for Norm Layer. Defaults to None.
- bbox_coder (dict) – Config dict for bbox coder.
- reg_decoded_bbox (bool) – If true, the regression loss would be applied directly on decoded bounding boxes, converting both the predicted boxes and regression targets to absolute coordinates format. Default False. It should be True when using IoULoss, GIoULoss, or DIoULoss in the bbox head.
- train_cfg (dict) – Training config of SABLRetinaHead.
- test_cfg (dict) – Testing config of SABLRetinaHead.
- loss_cls (dict) – Config of classification loss.
- loss_bbox_cls (dict) – Config of classification loss for bbox branch.
- loss_bbox_reg (dict) – Config of regression loss for bbox branch.
- init_cfg (dict or list[dict], optional) – Initialization config dict.
-
forward
(feats)[source]¶ Defines the computation performed at every call.
Should be overridden by all subclasses.
Note
Although the recipe for forward pass needs to be defined within this function, one should call the
Module
instance afterwards instead of this since the former takes care of running the registered hooks while the latter silently ignores them.
-
get_anchors
(featmap_sizes, img_metas, device='cuda')[source]¶ Get squares according to feature map sizes and guided anchors.
Parameters: - featmap_sizes (list[tuple]) – Multi-level feature map sizes.
- img_metas (list[dict]) – Image meta info.
- device (torch.device | str) – device for returned tensors
Returns: square approxs of each image
Return type: tuple
-
get_bboxes
(cls_scores, bbox_preds, img_metas, cfg=None, rescale=False)[source]¶ Transform network output for a batch into bbox predictions.
-
get_target
(approx_list, inside_flag_list, square_list, gt_bboxes_list, img_metas, gt_bboxes_ignore_list=None, gt_labels_list=None, label_channels=None, sampling=True, unmap_outputs=True)[source]¶ Compute bucketing targets. :param approx_list: Multi level approxs of each image. :type approx_list: list[list] :param inside_flag_list: Multi level inside flags of each
image.Parameters: - square_list (list[list]) – Multi level squares of each image.
- gt_bboxes_list (list[Tensor]) – Ground truth bboxes of each image.
- img_metas (list[dict]) – Meta info of each image.
- gt_bboxes_ignore_list (list[Tensor]) – ignore list of gt bboxes.
- gt_bboxes_list – Gt bboxes of each image.
- label_channels (int) – Channel of label.
- sampling (bool) – Sample Anchors or not.
- unmap_outputs (bool) – unmap outputs or not.
Returns: Returns a tuple containing learning targets.
- labels_list (list[Tensor]): Labels of each level.
- label_weights_list (list[Tensor]): Label weights of each level.
- bbox_cls_targets_list (list[Tensor]): BBox cls targets of each level.
- bbox_cls_weights_list (list[Tensor]): BBox cls weights of each level.
- bbox_reg_targets_list (list[Tensor]): BBox reg targets of each level.
- bbox_reg_weights_list (list[Tensor]): BBox reg weights of each level.
- num_total_pos (int): Number of positive samples in all images.
- num_total_neg (int): Number of negative samples in all images.
Return type: tuple
-
class
mmdet.models.dense_heads.
CentripetalHead
(*args, centripetal_shift_channels=2, guiding_shift_channels=2, feat_adaption_conv_kernel=3, loss_guiding_shift={'beta': 1.0, 'loss_weight': 0.05, 'type': 'SmoothL1Loss'}, loss_centripetal_shift={'beta': 1.0, 'loss_weight': 1, 'type': 'SmoothL1Loss'}, init_cfg=None, **kwargs)[source]¶ Head of CentripetalNet: Pursuing High-quality Keypoint Pairs for Object Detection.
CentripetalHead inherits from
CornerHead
. It removes the embedding branch and adds guiding shift and centripetal shift branches. More details can be found in the paper .Parameters: - num_classes (int) – Number of categories excluding the background category.
- in_channels (int) – Number of channels in the input feature map.
- num_feat_levels (int) – Levels of feature from the previous module. 2 for HourglassNet-104 and 1 for HourglassNet-52. HourglassNet-104 outputs the final feature and intermediate supervision feature and HourglassNet-52 only outputs the final feature. Default: 2.
- corner_emb_channels (int) – Channel of embedding vector. Default: 1.
- train_cfg (dict | None) – Training config. Useless in CornerHead, but we keep this variable for SingleStageDetector. Default: None.
- test_cfg (dict | None) – Testing config of CornerHead. Default: None.
- loss_heatmap (dict | None) – Config of corner heatmap loss. Default: GaussianFocalLoss.
- loss_embedding (dict | None) – Config of corner embedding loss. Default: AssociativeEmbeddingLoss.
- loss_offset (dict | None) – Config of corner offset loss. Default: SmoothL1Loss.
- loss_guiding_shift (dict) – Config of guiding shift loss. Default: SmoothL1Loss.
- loss_centripetal_shift (dict) – Config of centripetal shift loss. Default: SmoothL1Loss.
- init_cfg (dict or list[dict], optional) – Initialization config dict. Default: None
-
forward_single
(x, lvl_ind)[source]¶ Forward feature of a single level.
Parameters: - x (Tensor) – Feature of a single level.
- lvl_ind (int) – Level index of current feature.
Returns: A tuple of CentripetalHead’s output for current feature level. Containing the following Tensors:
- tl_heat (Tensor): Predicted top-left corner heatmap.
- br_heat (Tensor): Predicted bottom-right corner heatmap.
- tl_off (Tensor): Predicted top-left offset heatmap.
- br_off (Tensor): Predicted bottom-right offset heatmap.
- tl_guiding_shift (Tensor): Predicted top-left guiding shift heatmap.
- br_guiding_shift (Tensor): Predicted bottom-right guiding shift heatmap.
- tl_centripetal_shift (Tensor): Predicted top-left centripetal shift heatmap.
- br_centripetal_shift (Tensor): Predicted bottom-right centripetal shift heatmap.
Return type: tuple[Tensor]
-
get_bboxes
(tl_heats, br_heats, tl_offs, br_offs, tl_guiding_shifts, br_guiding_shifts, tl_centripetal_shifts, br_centripetal_shifts, img_metas, rescale=False, with_nms=True)[source]¶ Transform network output for a batch into bbox predictions.
Parameters: - tl_heats (list[Tensor]) – Top-left corner heatmaps for each level with shape (N, num_classes, H, W).
- br_heats (list[Tensor]) – Bottom-right corner heatmaps for each level with shape (N, num_classes, H, W).
- tl_offs (list[Tensor]) – Top-left corner offsets for each level with shape (N, corner_offset_channels, H, W).
- br_offs (list[Tensor]) – Bottom-right corner offsets for each level with shape (N, corner_offset_channels, H, W).
- tl_guiding_shifts (list[Tensor]) – Top-left guiding shifts for each level with shape (N, guiding_shift_channels, H, W). Useless in this function, we keep this arg because it’s the raw output from CentripetalHead.
- br_guiding_shifts (list[Tensor]) – Bottom-right guiding shifts for each level with shape (N, guiding_shift_channels, H, W). Useless in this function, we keep this arg because it’s the raw output from CentripetalHead.
- tl_centripetal_shifts (list[Tensor]) – Top-left centripetal shifts for each level with shape (N, centripetal_shift_channels, H, W).
- br_centripetal_shifts (list[Tensor]) – Bottom-right centripetal shifts for each level with shape (N, centripetal_shift_channels, H, W).
- img_metas (list[dict]) – Meta information of each image, e.g., image size, scaling factor, etc.
- rescale (bool) – If True, return boxes in original image space. Default: False.
- with_nms (bool) – If True, do nms before return boxes. Default: True.
-
loss
(tl_heats, br_heats, tl_offs, br_offs, tl_guiding_shifts, br_guiding_shifts, tl_centripetal_shifts, br_centripetal_shifts, gt_bboxes, gt_labels, img_metas, gt_bboxes_ignore=None)[source]¶ Compute losses of the head.
Parameters: - tl_heats (list[Tensor]) – Top-left corner heatmaps for each level with shape (N, num_classes, H, W).
- br_heats (list[Tensor]) – Bottom-right corner heatmaps for each level with shape (N, num_classes, H, W).
- tl_offs (list[Tensor]) – Top-left corner offsets for each level with shape (N, corner_offset_channels, H, W).
- br_offs (list[Tensor]) – Bottom-right corner offsets for each level with shape (N, corner_offset_channels, H, W).
- tl_guiding_shifts (list[Tensor]) – Top-left guiding shifts for each level with shape (N, guiding_shift_channels, H, W).
- br_guiding_shifts (list[Tensor]) – Bottom-right guiding shifts for each level with shape (N, guiding_shift_channels, H, W).
- tl_centripetal_shifts (list[Tensor]) – Top-left centripetal shifts for each level with shape (N, centripetal_shift_channels, H, W).
- br_centripetal_shifts (list[Tensor]) – Bottom-right centripetal shifts for each level with shape (N, centripetal_shift_channels, H, W).
- gt_bboxes (list[Tensor]) – Ground truth bboxes for each image with shape (num_gts, 4) in [left, top, right, bottom] format.
- gt_labels (list[Tensor]) – Class indices corresponding to each box.
- img_metas (list[dict]) – Meta information of each image, e.g., image size, scaling factor, etc.
- gt_bboxes_ignore (list[Tensor] | None) – Specify which bounding boxes can be ignored when computing the loss.
Returns: A dictionary of loss components. Containing the following losses:
- det_loss (list[Tensor]): Corner keypoint losses of all feature levels.
- off_loss (list[Tensor]): Corner offset losses of all feature levels.
- guiding_loss (list[Tensor]): Guiding shift losses of all feature levels.
- centripetal_loss (list[Tensor]): Centripetal shift losses of all feature levels.
Return type: dict[str, Tensor]
-
loss_single
(tl_hmp, br_hmp, tl_off, br_off, tl_guiding_shift, br_guiding_shift, tl_centripetal_shift, br_centripetal_shift, targets)[source]¶ Compute losses for single level.
Parameters: - tl_hmp (Tensor) – Top-left corner heatmap for current level with shape (N, num_classes, H, W).
- br_hmp (Tensor) – Bottom-right corner heatmap for current level with shape (N, num_classes, H, W).
- tl_off (Tensor) – Top-left corner offset for current level with shape (N, corner_offset_channels, H, W).
- br_off (Tensor) – Bottom-right corner offset for current level with shape (N, corner_offset_channels, H, W).
- tl_guiding_shift (Tensor) – Top-left guiding shift for current level with shape (N, guiding_shift_channels, H, W).
- br_guiding_shift (Tensor) – Bottom-right guiding shift for current level with shape (N, guiding_shift_channels, H, W).
- tl_centripetal_shift (Tensor) – Top-left centripetal shift for current level with shape (N, centripetal_shift_channels, H, W).
- br_centripetal_shift (Tensor) – Bottom-right centripetal shift for current level with shape (N, centripetal_shift_channels, H, W).
- targets (dict) – Corner target generated by get_targets.
Returns: Losses of the head’s differnet branches containing the following losses:
- det_loss (Tensor): Corner keypoint loss.
- off_loss (Tensor): Corner offset loss.
- guiding_loss (Tensor): Guiding shift loss.
- centripetal_loss (Tensor): Centripetal shift loss.
Return type: tuple[torch.Tensor]
-
class
mmdet.models.dense_heads.
VFNetHead
(num_classes, in_channels, regress_ranges=((-1, 64), (64, 128), (128, 256), (256, 512), (512, 100000000.0)), center_sampling=False, center_sample_radius=1.5, sync_num_pos=True, gradient_mul=0.1, bbox_norm_type='reg_denom', loss_cls_fl={'alpha': 0.25, 'gamma': 2.0, 'loss_weight': 1.0, 'type': 'FocalLoss', 'use_sigmoid': True}, use_vfl=True, loss_cls={'alpha': 0.75, 'gamma': 2.0, 'iou_weighted': True, 'loss_weight': 1.0, 'type': 'VarifocalLoss', 'use_sigmoid': True}, loss_bbox={'loss_weight': 1.5, 'type': 'GIoULoss'}, loss_bbox_refine={'loss_weight': 2.0, 'type': 'GIoULoss'}, norm_cfg={'num_groups': 32, 'requires_grad': True, 'type': 'GN'}, use_atss=True, anchor_generator={'center_offset': 0.0, 'octave_base_scale': 8, 'ratios': [1.0], 'scales_per_octave': 1, 'strides': [8, 16, 32, 64, 128], 'type': 'AnchorGenerator'}, init_cfg={'layer': 'Conv2d', 'override': {'bias_prob': 0.01, 'name': 'vfnet_cls', 'std': 0.01, 'type': 'Normal'}, 'std': 0.01, 'type': 'Normal'}, **kwargs)[source]¶ Head of `VarifocalNet (VFNet): An IoU-aware Dense Object Detector.<https://arxiv.org/abs/2008.13367>`_.
The VFNet predicts IoU-aware classification scores which mix the object presence confidence and object localization accuracy as the detection score. It is built on the FCOS architecture and uses ATSS for defining positive/negative training examples. The VFNet is trained with Varifocal Loss and empolys star-shaped deformable convolution to extract features for a bbox.
Parameters: - num_classes (int) – Number of categories excluding the background category.
- in_channels (int) – Number of channels in the input feature map.
- regress_ranges (tuple[tuple[int, int]]) – Regress range of multiple level points.
- center_sampling (bool) – If true, use center sampling. Default: False.
- center_sample_radius (float) – Radius of center sampling. Default: 1.5.
- sync_num_pos (bool) – If true, synchronize the number of positive examples across GPUs. Default: True
- gradient_mul (float) – The multiplier to gradients from bbox refinement and recognition. Default: 0.1.
- bbox_norm_type (str) – The bbox normalization type, ‘reg_denom’ or ‘stride’. Default: reg_denom
- loss_cls_fl (dict) – Config of focal loss.
- use_vfl (bool) – If true, use varifocal loss for training. Default: True.
- loss_cls (dict) – Config of varifocal loss.
- loss_bbox (dict) – Config of localization loss, GIoU Loss.
- loss_bbox – Config of localization refinement loss, GIoU Loss.
- norm_cfg (dict) – dictionary to construct and config norm layer. Default: norm_cfg=dict(type=’GN’, num_groups=32, requires_grad=True).
- use_atss (bool) – If true, use ATSS to define positive/negative examples. Default: True.
- anchor_generator (dict) – Config of anchor generator for ATSS.
- init_cfg (dict or list[dict], optional) – Initialization config dict.
Example
>>> self = VFNetHead(11, 7) >>> feats = [torch.rand(1, 7, s, s) for s in [4, 8, 16, 32, 64]] >>> cls_score, bbox_pred, bbox_pred_refine= self.forward(feats) >>> assert len(cls_score) == len(self.scales)
-
forward
(feats)[source]¶ Forward features from the upstream network.
Parameters: feats (tuple[Tensor]) – Features from the upstream network, each is a 4D-tensor. Returns: - cls_scores (list[Tensor]): Box iou-aware scores for each scale
- level, each is a 4D-tensor, the channel number is num_points * num_classes.
- bbox_preds (list[Tensor]): Box offsets for each
- scale level, each is a 4D-tensor, the channel number is num_points * 4.
- bbox_preds_refine (list[Tensor]): Refined Box offsets for
- each scale level, each is a 4D-tensor, the channel number is num_points * 4.
Return type: tuple
-
forward_single
(x, scale, scale_refine, stride, reg_denom)[source]¶ Forward features of a single scale level.
Parameters: - x (Tensor) – FPN feature maps of the specified stride.
- ( (scale_refine) – obj: mmcv.cnn.Scale): Learnable scale module to resize the bbox prediction.
- ( – obj: mmcv.cnn.Scale): Learnable scale module to resize the refined bbox prediction.
- stride (int) – The corresponding stride for feature maps, used to normalize the bbox prediction when bbox_norm_type = ‘stride’.
- reg_denom (int) – The corresponding regression range for feature maps, only used to normalize the bbox prediction when bbox_norm_type = ‘reg_denom’.
Returns: - iou-aware cls scores for each box, bbox predictions and
refined bbox predictions of input feature maps.
Return type: tuple
-
get_atss_targets
(cls_scores, mlvl_points, gt_bboxes, gt_labels, img_metas, gt_bboxes_ignore=None)[source]¶ A wrapper for computing ATSS targets for points in multiple images.
Parameters: - cls_scores (list[Tensor]) – Box iou-aware scores for each scale level with shape (N, num_points * num_classes, H, W).
- mlvl_points (list[Tensor]) – Points of each fpn level, each has shape (num_points, 2).
- gt_bboxes (list[Tensor]) – Ground truth bboxes of each image, each has shape (num_gt, 4).
- gt_labels (list[Tensor]) – Ground truth labels of each box, each has shape (num_gt,).
- img_metas (list[dict]) – Meta information of each image, e.g., image size, scaling factor, etc.
- gt_bboxes_ignore (None | Tensor) – Ground truth bboxes to be ignored, shape (num_ignored_gts, 4). Default: None.
Returns: labels_list (list[Tensor]): Labels of each level. label_weights (Tensor): Label weights of all levels. bbox_targets_list (list[Tensor]): Regression targets of each
level, (l, t, r, b).
bbox_weights (Tensor): Bbox weights of all levels.
Return type: tuple
-
get_bboxes
(cls_scores, bbox_preds, bbox_preds_refine, img_metas, cfg=None, rescale=None, with_nms=True)[source]¶ Transform network outputs for a batch into bbox predictions.
Parameters: - cls_scores (list[Tensor]) – Box iou-aware scores for each scale level with shape (N, num_points * num_classes, H, W).
- bbox_preds (list[Tensor]) – Box offsets for each scale level with shape (N, num_points * 4, H, W).
- bbox_preds_refine (list[Tensor]) – Refined Box offsets for each scale level with shape (N, num_points * 4, H, W).
- img_metas (list[dict]) – Meta information of each image, e.g., image size, scaling factor, etc.
- cfg (mmcv.Config) – Test / postprocessing configuration, if None, test_cfg would be used. Default: None.
- rescale (bool) – If True, return boxes in original image space. Default: False.
- with_nms (bool) – If True, do nms before returning boxes. Default: True.
Returns: - Each item in result_list is 2-tuple.
The first item is an (n, 5) tensor, where the first 4 columns are bounding box positions (tl_x, tl_y, br_x, br_y) and the 5-th column is a score between 0 and 1. The second item is a (n,) tensor where each item is the predicted class label of the corresponding box.
Return type: list[tuple[Tensor, Tensor]]
-
get_fcos_targets
(points, gt_bboxes_list, gt_labels_list)[source]¶ Compute FCOS regression and classification targets for points in multiple images.
Parameters: - points (list[Tensor]) – Points of each fpn level, each has shape (num_points, 2).
- gt_bboxes_list (list[Tensor]) – Ground truth bboxes of each image, each has shape (num_gt, 4).
- gt_labels_list (list[Tensor]) – Ground truth labels of each box, each has shape (num_gt,).
Returns: labels (list[Tensor]): Labels of each level. label_weights: None, to be compatible with ATSS targets. bbox_targets (list[Tensor]): BBox targets of each level. bbox_weights: None, to be compatible with ATSS targets.
Return type: tuple
-
get_targets
(cls_scores, mlvl_points, gt_bboxes, gt_labels, img_metas, gt_bboxes_ignore)[source]¶ A wrapper for computing ATSS and FCOS targets for points in multiple images.
Parameters: - cls_scores (list[Tensor]) – Box iou-aware scores for each scale level with shape (N, num_points * num_classes, H, W).
- mlvl_points (list[Tensor]) – Points of each fpn level, each has shape (num_points, 2).
- gt_bboxes (list[Tensor]) – Ground truth bboxes of each image, each has shape (num_gt, 4).
- gt_labels (list[Tensor]) – Ground truth labels of each box, each has shape (num_gt,).
- img_metas (list[dict]) – Meta information of each image, e.g., image size, scaling factor, etc.
- gt_bboxes_ignore (None | Tensor) – Ground truth bboxes to be ignored, shape (num_ignored_gts, 4).
Returns: labels_list (list[Tensor]): Labels of each level. label_weights (Tensor/None): Label weights of all levels. bbox_targets_list (list[Tensor]): Regression targets of each
level, (l, t, r, b).
bbox_weights (Tensor/None): Bbox weights of all levels.
Return type: tuple
-
loss
(cls_scores, bbox_preds, bbox_preds_refine, gt_bboxes, gt_labels, img_metas, gt_bboxes_ignore=None)[source]¶ Compute loss of the head.
Parameters: - cls_scores (list[Tensor]) – Box iou-aware scores for each scale level, each is a 4D-tensor, the channel number is num_points * num_classes.
- bbox_preds (list[Tensor]) – Box offsets for each scale level, each is a 4D-tensor, the channel number is num_points * 4.
- bbox_preds_refine (list[Tensor]) – Refined Box offsets for each scale level, each is a 4D-tensor, the channel number is num_points * 4.
- gt_bboxes (list[Tensor]) – Ground truth bboxes for each image with shape (num_gts, 4) in [tl_x, tl_y, br_x, br_y] format.
- gt_labels (list[Tensor]) – class indices corresponding to each box
- img_metas (list[dict]) – Meta information of each image, e.g., image size, scaling factor, etc.
- gt_bboxes_ignore (None | list[Tensor]) – specify which bounding boxes can be ignored when computing the loss. Default: None.
Returns: A dictionary of loss components.
Return type: dict[str, Tensor]
-
star_dcn_offset
(bbox_pred, gradient_mul, stride)[source]¶ Compute the star deformable conv offsets.
Parameters: - bbox_pred (Tensor) – Predicted bbox distance offsets (l, r, t, b).
- gradient_mul (float) – Gradient multiplier.
- stride (int) – The corresponding stride for feature maps, used to project the bbox onto the feature map.
Returns: The offsets for deformable convolution.
Return type: dcn_offsets (Tensor)
-
transform_bbox_targets
(decoded_bboxes, mlvl_points, num_imgs)[source]¶ Transform bbox_targets (x1, y1, x2, y2) into (l, t, r, b) format.
Parameters: - decoded_bboxes (list[Tensor]) – Regression targets of each level, in the form of (x1, y1, x2, y2).
- mlvl_points (list[Tensor]) – Points of each fpn level, each has shape (num_points, 2).
- num_imgs (int) – the number of images in a batch.
Returns: - Regression targets of each level in
the form of (l, t, r, b).
Return type: bbox_targets (list[Tensor])
-
class
mmdet.models.dense_heads.
StageCascadeRPNHead
(in_channels, anchor_generator={'ratios': [1.0], 'scales': [8], 'strides': [4, 8, 16, 32, 64], 'type': 'AnchorGenerator'}, adapt_cfg={'dilation': 3, 'type': 'dilation'}, bridged_feature=False, with_cls=True, sampling=True, init_cfg=None, **kwargs)[source]¶ Stage of CascadeRPNHead.
Parameters: - in_channels (int) – Number of channels in the input feature map.
- anchor_generator (dict) – anchor generator config.
- adapt_cfg (dict) – adaptation config.
- bridged_feature (bool, optional) – whether update rpn feature. Default: False.
- with_cls (bool, optional) – wheather use classification branch. Default: True.
- sampling (bool, optional) – wheather use sampling. Default: True.
- init_cfg (dict or list[dict], optional) – Initialization config dict. Default: None
-
anchor_offset
(anchor_list, anchor_strides, featmap_sizes)[source]¶ Get offest for deformable conv based on anchor shape NOTE: currently support deformable kernel_size=3 and dilation=1
Parameters: - anchor_list (list[list[tensor])) – [NI, NLVL, NA, 4] list of multi-level anchors
- anchor_strides (list[int]) – anchor stride of each level
Returns: - [NLVL, NA, 2, 18]: offset of DeformConv
kernel.
Return type: offset_list (list[tensor])
-
get_bboxes
(anchor_list, cls_scores, bbox_preds, img_metas, cfg, rescale=False)[source]¶ Get proposal predict.
-
get_targets
(anchor_list, valid_flag_list, gt_bboxes, img_metas, featmap_sizes, gt_bboxes_ignore=None, label_channels=1)[source]¶ Compute regression and classification targets for anchors.
Parameters: - anchor_list (list[list]) – Multi level anchors of each image.
- valid_flag_list (list[list]) – Multi level valid flags of each image.
- gt_bboxes (list[Tensor]) – Ground truth bboxes of each image.
- img_metas (list[dict]) – Meta info of each image.
- featmap_sizes (list[Tensor]) – Feature mapsize each level
- gt_bboxes_ignore (list[Tensor]) – Ignore bboxes of each images
- label_channels (int) – Channel of label.
Returns: cls_reg_targets (tuple)
-
loss
(anchor_list, valid_flag_list, cls_scores, bbox_preds, gt_bboxes, img_metas, gt_bboxes_ignore=None)[source]¶ Compute losses of the head.
Parameters: - anchor_list (list[list]) – Multi level anchors of each image.
- cls_scores (list[Tensor]) – Box scores for each scale level Has shape (N, num_anchors * num_classes, H, W)
- bbox_preds (list[Tensor]) – Box energies / deltas for each scale level with shape (N, num_anchors * 4, H, W)
- gt_bboxes (list[Tensor]) – Ground truth bboxes for each image with shape (num_gts, 4) in [tl_x, tl_y, br_x, br_y] format.
- img_metas (list[dict]) – Meta information of each image, e.g., image size, scaling factor, etc.
- gt_bboxes_ignore (None | list[Tensor]) – specify which bounding boxes can be ignored when computing the loss. Default: None
Returns: A dictionary of loss components.
Return type: dict[str, Tensor]
-
loss_single
(cls_score, bbox_pred, anchors, labels, label_weights, bbox_targets, bbox_weights, num_total_samples)[source]¶ Loss function on single scale.
-
class
mmdet.models.dense_heads.
CascadeRPNHead
(num_stages, stages, train_cfg, test_cfg, init_cfg=None)[source]¶ The CascadeRPNHead will predict more accurate region proposals, which is required for two-stage detectors (such as Fast/Faster R-CNN). CascadeRPN consists of a sequence of RPNStage to progressively improve the accuracy of the detected proposals.
More details can be found in
https://arxiv.org/abs/1909.06720
.Parameters: - num_stages (int) – number of CascadeRPN stages.
- stages (list[dict]) – list of configs to build the stages.
- train_cfg (list[dict]) – list of configs at training time each stage.
- test_cfg (dict) – config at testing time.
-
class
mmdet.models.dense_heads.
EmbeddingRPNHead
(num_proposals=100, proposal_feature_channel=256, init_cfg=None, **kwargs)[source]¶ RPNHead in the Sparse R-CNN .
Unlike traditional RPNHead, this module does not need FPN input, but just decode init_proposal_bboxes and expand the first dimension of init_proposal_bboxes and init_proposal_features to the batch_size.
Parameters: - num_proposals (int) – Number of init_proposals. Default 100.
- proposal_feature_channel (int) – Channel number of init_proposal_feature. Defaults to 256.
- init_cfg (dict or list[dict], optional) – Initialization config dict. Default: None
-
class
mmdet.models.dense_heads.
LDHead
(num_classes, in_channels, loss_ld={'T': 10, 'loss_weight': 0.25, 'type': 'LocalizationDistillationLoss'}, **kwargs)[source]¶ Localization distillation Head. (Short description)
It utilizes the learned bbox distributions to transfer the localization dark knowledge from teacher to student. Original paper: Localization Distillation for Object Detection.
Parameters: - num_classes (int) – Number of categories excluding the background category.
- in_channels (int) – Number of channels in the input feature map.
- loss_ld (dict) – Config of Localization Distillation Loss (LD), T is the temperature for distillation.
-
forward_train
(x, out_teacher, img_metas, gt_bboxes, gt_labels=None, gt_bboxes_ignore=None, proposal_cfg=None, **kwargs)[source]¶ Parameters: - x (list[Tensor]) – Features from FPN.
- img_metas (list[dict]) – Meta information of each image, e.g., image size, scaling factor, etc.
- gt_bboxes (Tensor) – Ground truth bboxes of the image, shape (num_gts, 4).
- gt_labels (Tensor) – Ground truth labels of each box, shape (num_gts,).
- gt_bboxes_ignore (Tensor) – Ground truth bboxes to be ignored, shape (num_ignored_gts, 4).
- proposal_cfg (mmcv.Config) – Test / postprocessing configuration, if None, test_cfg would be used
Returns: The loss components and proposals of each image.
- losses (dict[str, Tensor]): A dictionary of loss components.
- proposal_list (list[Tensor]): Proposals of each image.
Return type: tuple[dict, list]
-
loss
(cls_scores, bbox_preds, gt_bboxes, gt_labels, soft_target, img_metas, gt_bboxes_ignore=None)[source]¶ Compute losses of the head.
Parameters: - cls_scores (list[Tensor]) – Cls and quality scores for each scale level has shape (N, num_classes, H, W).
- bbox_preds (list[Tensor]) – Box distribution logits for each scale level with shape (N, 4*(n+1), H, W), n is max value of integral set.
- gt_bboxes (list[Tensor]) – Ground truth bboxes for each image with shape (num_gts, 4) in [tl_x, tl_y, br_x, br_y] format.
- gt_labels (list[Tensor]) – class indices corresponding to each box
- img_metas (list[dict]) – Meta information of each image, e.g., image size, scaling factor, etc.
- gt_bboxes_ignore (list[Tensor] | None) – specify which bounding boxes can be ignored when computing the loss.
Returns: A dictionary of loss components.
Return type: dict[str, Tensor]
-
loss_single
(anchors, cls_score, bbox_pred, labels, label_weights, bbox_targets, stride, soft_targets, num_total_samples)[source]¶ Compute loss of a single scale level.
Parameters: - anchors (Tensor) – Box reference for each scale level with shape (N, num_total_anchors, 4).
- cls_score (Tensor) – Cls and quality joint scores for each scale level has shape (N, num_classes, H, W).
- bbox_pred (Tensor) – Box distribution logits for each scale level with shape (N, 4*(n+1), H, W), n is max value of integral set.
- labels (Tensor) – Labels of each anchors with shape (N, num_total_anchors).
- label_weights (Tensor) – Label weights of each anchor with shape (N, num_total_anchors)
- bbox_targets (Tensor) – BBox regression targets of each anchor wight shape (N, num_total_anchors, 4).
- stride (tuple) – Stride in this scale level.
- num_total_samples (int) – Number of positive samples that is reduced over all GPUs.
Returns: Loss components and weight targets.
Return type: dict[tuple, Tensor]
-
class
mmdet.models.dense_heads.
CascadeRPNHead
(num_stages, stages, train_cfg, test_cfg, init_cfg=None)[source] The CascadeRPNHead will predict more accurate region proposals, which is required for two-stage detectors (such as Fast/Faster R-CNN). CascadeRPN consists of a sequence of RPNStage to progressively improve the accuracy of the detected proposals.
More details can be found in
https://arxiv.org/abs/1909.06720
.Parameters: - num_stages (int) – number of CascadeRPN stages.
- stages (list[dict]) – list of configs to build the stages.
- train_cfg (list[dict]) – list of configs at training time each stage.
- test_cfg (dict) – config at testing time.
-
aug_test_rpn
(x, img_metas)[source] Augmented forward test function.
-
forward_train
(x, img_metas, gt_bboxes, gt_labels=None, gt_bboxes_ignore=None, proposal_cfg=None)[source] Forward train function.
-
get_bboxes
()[source] get_bboxes() is implemented in StageCascadeRPNHead.
-
loss
()[source] loss() is implemented in StageCascadeRPNHead.
-
simple_test_rpn
(x, img_metas)[source] Simple forward test function.
-
class
mmdet.models.dense_heads.
AutoAssignHead
(*args, force_topk=False, topk=9, pos_loss_weight=0.25, neg_loss_weight=0.75, center_loss_weight=0.75, **kwargs)[source]¶ AutoAssignHead head used in AutoAssign.
More details can be found in the paper .
Parameters: - force_topk (bool) – Used in center prior initialization to handle extremely small gt. Default is False.
- topk (int) – The number of points used to calculate the center prior when no point falls in gt_bbox. Only work when force_topk if True. Defaults to 9.
- pos_loss_weight (float) – The loss weight of positive loss and with default value 0.25.
- neg_loss_weight (float) – The loss weight of negative loss and with default value 0.75.
- center_loss_weight (float) – The loss weight of center prior loss and with default value 0.75.
-
forward_single
(x, scale, stride)[source]¶ Forward features of a single scale level.
Parameters: - x (Tensor) – FPN feature maps of the specified stride.
- ( (scale) – obj: mmcv.cnn.Scale): Learnable scale module to resize the bbox prediction.
- stride (int) – The corresponding stride for feature maps, only used to normalize the bbox prediction when self.norm_on_bbox is True.
Returns: scores for each class, bbox predictions and centerness predictions of input feature maps.
Return type: tuple
-
get_neg_loss_single
(cls_score, objectness, gt_labels, ious, inside_gt_bbox_mask)[source]¶ Calculate the negative loss of all points in feature map.
Parameters: - cls_score (Tensor) – All category scores for each point on the feature map. The shape is (num_points, num_class).
- objectness (Tensor) – Foreground probability of all points and is shape of (num_points, 1).
- gt_labels (Tensor) – The zeros based label of all gt with shape of (num_gt).
- ious (Tensor) – Float tensor with shape of (num_points, num_gt). Each value represent the iou of pred_bbox and gt_bboxes.
- inside_gt_bbox_mask (Tensor) – Tensor of bool type, with shape of (num_points, num_gt), each value is used to mark whether this point falls within a certain gt.
Returns: - neg_loss (Tensor): The negative loss of all points in the feature map.
Return type: tuple[Tensor]
-
get_pos_loss_single
(cls_score, objectness, reg_loss, gt_labels, center_prior_weights)[source]¶ Calculate the positive loss of all points in gt_bboxes.
Parameters: - cls_score (Tensor) – All category scores for each point on the feature map. The shape is (num_points, num_class).
- objectness (Tensor) – Foreground probability of all points, has shape (num_points, 1).
- reg_loss (Tensor) – The regression loss of each gt_bbox and each prediction box, has shape of (num_points, num_gt).
- gt_labels (Tensor) – The zeros based gt_labels of all gt with shape of (num_gt,).
- center_prior_weights (Tensor) – Float tensor with shape of (num_points, num_gt). Each value represents the center weighting coefficient.
Returns: - pos_loss (Tensor): The positive loss of all points in the gt_bboxes.
Return type: tuple[Tensor]
-
get_targets
(points, gt_bboxes_list)[source]¶ Compute regression targets and each point inside or outside gt_bbox in multiple images.
Parameters: - points (list[Tensor]) – Points of all fpn level, each has shape (num_points, 2).
- gt_bboxes_list (list[Tensor]) – Ground truth bboxes of each image, each has shape (num_gt, 4).
Returns: - inside_gt_bbox_mask_list (list[Tensor]): Each Tensor is with bool type and shape of (num_points, num_gt), each value is used to mark whether this point falls within a certain gt.
- concat_lvl_bbox_targets (list[Tensor]): BBox targets of each level. Each tensor has shape (num_points, num_gt, 4).
Return type: tuple(list[Tensor])
-
init_weights
()[source]¶ Initialize weights of the head.
In particular, we have special initialization for classified conv’s and regression conv’s bias
-
loss
(cls_scores, bbox_preds, objectnesses, gt_bboxes, gt_labels, img_metas, gt_bboxes_ignore=None)[source]¶ Compute loss of the head.
Parameters: - cls_scores (list[Tensor]) – Box scores for each scale level, each is a 4D-tensor, the channel number is num_points * num_classes.
- bbox_preds (list[Tensor]) – Box energies / deltas for each scale level, each is a 4D-tensor, the channel number is num_points * 4.
- objectnesses (list[Tensor]) – objectness for each scale level, each is a 4D-tensor, the channel number is num_points * 1.
- gt_bboxes (list[Tensor]) – Ground truth bboxes for each image with shape (num_gts, 4) in [tl_x, tl_y, br_x, br_y] format.
- gt_labels (list[Tensor]) – class indices corresponding to each box
- img_metas (list[dict]) – Meta information of each image, e.g., image size, scaling factor, etc.
- gt_bboxes_ignore (None | list[Tensor]) – specify which bounding boxes can be ignored when computing the loss.
Returns: A dictionary of loss components.
Return type: dict[str, Tensor]
-
class
mmdet.models.dense_heads.
DETRHead
(num_classes, in_channels, num_query=100, num_reg_fcs=2, transformer=None, sync_cls_avg_factor=False, positional_encoding={'normalize': True, 'num_feats': 128, 'type': 'SinePositionalEncoding'}, loss_cls={'bg_cls_weight': 0.1, 'class_weight': 1.0, 'loss_weight': 1.0, 'type': 'CrossEntropyLoss', 'use_sigmoid': False}, loss_bbox={'loss_weight': 5.0, 'type': 'L1Loss'}, loss_iou={'loss_weight': 2.0, 'type': 'GIoULoss'}, train_cfg={'assigner': {'cls_cost': {'type': 'ClassificationCost', 'weight': 1.0}, 'iou_cost': {'iou_mode': 'giou', 'type': 'IoUCost', 'weight': 2.0}, 'reg_cost': {'type': 'BBoxL1Cost', 'weight': 5.0}, 'type': 'HungarianAssigner'}}, test_cfg={'max_per_img': 100}, init_cfg=None, **kwargs)[source]¶ Implements the DETR transformer head.
See paper: End-to-End Object Detection with Transformers for details.
Parameters: - num_classes (int) – Number of categories excluding the background.
- in_channels (int) – Number of channels in the input feature map.
- num_query (int) – Number of query in Transformer.
- num_reg_fcs (int, optional) – Number of fully-connected layers used in FFN, which is then used for the regression head. Default 2.
- (obj (test_cfg) – `mmcv.ConfigDict`|dict): Config for transformer. Default: None.
- sync_cls_avg_factor (bool) – Whether to sync the avg_factor of all ranks. Default to False.
- (obj – `mmcv.ConfigDict`|dict): Config for position encoding.
- (obj – mmcv.ConfigDict`|dict): Config of the classification loss. Default `CrossEntropyLoss.
- (obj – mmcv.ConfigDict`|dict): Config of the regression loss. Default `L1Loss.
- (obj – mmcv.ConfigDict`|dict): Config of the regression iou loss. Default `GIoULoss.
- (obj – `mmcv.ConfigDict`|dict): Training config of transformer head.
- (obj – `mmcv.ConfigDict`|dict): Testing config of transformer head.
- init_cfg (dict or list[dict], optional) – Initialization config dict. Default: None
-
forward
(feats, img_metas)[source]¶ Forward function.
Parameters: - feats (tuple[Tensor]) – Features from the upstream network, each is a 4D-tensor.
- img_metas (list[dict]) – List of image information.
Returns: Outputs for all scale levels.
- all_cls_scores_list (list[Tensor]): Classification scores for each scale level. Each is a 4D-tensor with shape [nb_dec, bs, num_query, cls_out_channels]. Note cls_out_channels should includes background.
- all_bbox_preds_list (list[Tensor]): Sigmoid regression outputs for each scale level. Each is a 4D-tensor with normalized coordinate format (cx, cy, w, h) and shape [nb_dec, bs, num_query, 4].
Return type: tuple[list[Tensor], list[Tensor]]
-
forward_onnx
(feats, img_metas)[source]¶ Forward function for exporting to ONNX.
Over-write forward because: masks is directly created with zero (valid position tag) and has the same spatial size as x. Thus the construction of masks is different from that in forward.
Parameters: - feats (tuple[Tensor]) – Features from the upstream network, each is a 4D-tensor.
- img_metas (list[dict]) – List of image information.
Returns: Outputs for all scale levels.
- all_cls_scores_list (list[Tensor]): Classification scores for each scale level. Each is a 4D-tensor with shape [nb_dec, bs, num_query, cls_out_channels]. Note cls_out_channels should includes background.
- all_bbox_preds_list (list[Tensor]): Sigmoid regression outputs for each scale level. Each is a 4D-tensor with normalized coordinate format (cx, cy, w, h) and shape [nb_dec, bs, num_query, 4].
Return type: tuple[list[Tensor], list[Tensor]]
-
forward_single
(x, img_metas)[source]¶ “Forward function for a single feature level.
Parameters: - x (Tensor) – Input feature from backbone’s single stage, shape [bs, c, h, w].
- img_metas (list[dict]) – List of image information.
Returns: - Outputs from the classification head,
shape [nb_dec, bs, num_query, cls_out_channels]. Note cls_out_channels should includes background.
- all_bbox_preds (Tensor): Sigmoid outputs from the regression
head with normalized coordinate format (cx, cy, w, h). Shape [nb_dec, bs, num_query, 4].
Return type: all_cls_scores (Tensor)
-
forward_single_onnx
(x, img_metas)[source]¶ “Forward function for a single feature level with ONNX exportation.
Parameters: - x (Tensor) – Input feature from backbone’s single stage, shape [bs, c, h, w].
- img_metas (list[dict]) – List of image information.
Returns: - Outputs from the classification head,
shape [nb_dec, bs, num_query, cls_out_channels]. Note cls_out_channels should includes background.
- all_bbox_preds (Tensor): Sigmoid outputs from the regression
head with normalized coordinate format (cx, cy, w, h). Shape [nb_dec, bs, num_query, 4].
Return type: all_cls_scores (Tensor)
-
forward_train
(x, img_metas, gt_bboxes, gt_labels=None, gt_bboxes_ignore=None, proposal_cfg=None, **kwargs)[source]¶ Forward function for training mode.
Parameters: - x (list[Tensor]) – Features from backbone.
- img_metas (list[dict]) – Meta information of each image, e.g., image size, scaling factor, etc.
- gt_bboxes (Tensor) – Ground truth bboxes of the image, shape (num_gts, 4).
- gt_labels (Tensor) – Ground truth labels of each box, shape (num_gts,).
- gt_bboxes_ignore (Tensor) – Ground truth bboxes to be ignored, shape (num_ignored_gts, 4).
- proposal_cfg (mmcv.Config) – Test / postprocessing configuration, if None, test_cfg would be used.
Returns: A dictionary of loss components.
Return type: dict[str, Tensor]
-
get_bboxes
(all_cls_scores_list, all_bbox_preds_list, img_metas, rescale=False)[source]¶ Transform network outputs for a batch into bbox predictions.
Parameters: - all_cls_scores_list (list[Tensor]) – Classification outputs for each feature level. Each is a 4D-tensor with shape [nb_dec, bs, num_query, cls_out_channels].
- all_bbox_preds_list (list[Tensor]) – Sigmoid regression outputs for each feature level. Each is a 4D-tensor with normalized coordinate format (cx, cy, w, h) and shape [nb_dec, bs, num_query, 4].
- img_metas (list[dict]) – Meta information of each image.
- rescale (bool, optional) – If True, return boxes in original image space. Default False.
Returns: Each item in result_list is 2-tuple. The first item is an (n, 5) tensor, where the first 4 columns are bounding box positions (tl_x, tl_y, br_x, br_y) and the 5-th column is a score between 0 and 1. The second item is a (n,) tensor where each item is the predicted class label of the corresponding box.
Return type: list[list[Tensor, Tensor]]
-
get_targets
(cls_scores_list, bbox_preds_list, gt_bboxes_list, gt_labels_list, img_metas, gt_bboxes_ignore_list=None)[source]¶ “Compute regression and classification targets for a batch image.
Outputs from a single decoder layer of a single feature level are used.
Parameters: - cls_scores_list (list[Tensor]) – Box score logits from a single decoder layer for each image with shape [num_query, cls_out_channels].
- bbox_preds_list (list[Tensor]) – Sigmoid outputs from a single decoder layer for each image, with normalized coordinate (cx, cy, w, h) and shape [num_query, 4].
- gt_bboxes_list (list[Tensor]) – Ground truth bboxes for each image with shape (num_gts, 4) in [tl_x, tl_y, br_x, br_y] format.
- gt_labels_list (list[Tensor]) – Ground truth class indices for each image with shape (num_gts, ).
- img_metas (list[dict]) – List of image meta information.
- gt_bboxes_ignore_list (list[Tensor], optional) – Bounding boxes which can be ignored for each image. Default None.
Returns: a tuple containing the following targets.
- labels_list (list[Tensor]): Labels for all images.
- label_weights_list (list[Tensor]): Label weights for all images.
- bbox_targets_list (list[Tensor]): BBox targets for all images.
- bbox_weights_list (list[Tensor]): BBox weights for all images.
- num_total_pos (int): Number of positive samples in all images.
- num_total_neg (int): Number of negative samples in all images.
Return type: tuple
-
loss
(all_cls_scores_list, all_bbox_preds_list, gt_bboxes_list, gt_labels_list, img_metas, gt_bboxes_ignore=None)[source]¶ “Loss function.
Only outputs from the last feature level are used for computing losses by default.
Parameters: - all_cls_scores_list (list[Tensor]) – Classification outputs for each feature level. Each is a 4D-tensor with shape [nb_dec, bs, num_query, cls_out_channels].
- all_bbox_preds_list (list[Tensor]) – Sigmoid regression outputs for each feature level. Each is a 4D-tensor with normalized coordinate format (cx, cy, w, h) and shape [nb_dec, bs, num_query, 4].
- gt_bboxes_list (list[Tensor]) – Ground truth bboxes for each image with shape (num_gts, 4) in [tl_x, tl_y, br_x, br_y] format.
- gt_labels_list (list[Tensor]) – Ground truth class indices for each image with shape (num_gts, ).
- img_metas (list[dict]) – List of image meta information.
- gt_bboxes_ignore (list[Tensor], optional) – Bounding boxes which can be ignored for each image. Default None.
Returns: A dictionary of loss components.
Return type: dict[str, Tensor]
-
loss_single
(cls_scores, bbox_preds, gt_bboxes_list, gt_labels_list, img_metas, gt_bboxes_ignore_list=None)[source]¶ “Loss function for outputs from a single decoder layer of a single feature level.
Parameters: - cls_scores (Tensor) – Box score logits from a single decoder layer for all images. Shape [bs, num_query, cls_out_channels].
- bbox_preds (Tensor) – Sigmoid outputs from a single decoder layer for all images, with normalized coordinate (cx, cy, w, h) and shape [bs, num_query, 4].
- gt_bboxes_list (list[Tensor]) – Ground truth bboxes for each image with shape (num_gts, 4) in [tl_x, tl_y, br_x, br_y] format.
- gt_labels_list (list[Tensor]) – Ground truth class indices for each image with shape (num_gts, ).
- img_metas (list[dict]) – List of image meta information.
- gt_bboxes_ignore_list (list[Tensor], optional) – Bounding boxes which can be ignored for each image. Default None.
Returns: - A dictionary of loss components for outputs from
a single decoder layer.
Return type: dict[str, Tensor]
-
onnx_export
(all_cls_scores_list, all_bbox_preds_list, img_metas)[source]¶ Transform network outputs into bbox predictions, with ONNX exportation.
Parameters: - all_cls_scores_list (list[Tensor]) – Classification outputs for each feature level. Each is a 4D-tensor with shape [nb_dec, bs, num_query, cls_out_channels].
- all_bbox_preds_list (list[Tensor]) – Sigmoid regression outputs for each feature level. Each is a 4D-tensor with normalized coordinate format (cx, cy, w, h) and shape [nb_dec, bs, num_query, 4].
- img_metas (list[dict]) – Meta information of each image.
Returns: - dets of shape [N, num_det, 5]
and class labels of shape [N, num_det].
Return type: tuple[Tensor, Tensor]
-
simple_test_bboxes
(feats, img_metas, rescale=False)[source]¶ Test det bboxes without test-time augmentation.
Parameters: - feats (tuple[torch.Tensor]) – Multi-level features from the upstream network, each is a 4D-tensor.
- img_metas (list[dict]) – List of image information.
- rescale (bool, optional) – Whether to rescale the results. Defaults to False.
Returns: - Each item in result_list is 2-tuple.
The first item is
bboxes
with shape (n, 5), where 5 represent (tl_x, tl_y, br_x, br_y, score). The shape of the second tensor in the tuple islabels
with shape (n,)
Return type: list[tuple[Tensor, Tensor]]
-
class
mmdet.models.dense_heads.
YOLOFHead
(num_classes, in_channels, num_cls_convs=2, num_reg_convs=4, norm_cfg={'requires_grad': True, 'type': 'BN'}, **kwargs)[source]¶ YOLOFHead Paper link: https://arxiv.org/abs/2103.09460.
Parameters: - num_classes (int) – The number of object classes (w/o background)
- in_channels (List[int]) – The number of input channels per scale.
- cls_num_convs (int) – The number of convolutions of cls branch. Default 2.
- reg_num_convs (int) – The number of convolutions of reg branch. Default 4.
- norm_cfg (dict) – Dictionary to construct and config norm layer.
-
forward_single
(feature)[source]¶ Forward feature of a single scale level.
Parameters: x (Tensor) – Features of a single scale level. Returns: cls_score (Tensor): Cls scores for a single scale level the channels number is num_anchors * num_classes. bbox_pred (Tensor): Box energies / deltas for a single scale level, the channels number is num_anchors * 4. Return type: tuple
-
get_targets
(cls_scores_list, bbox_preds_list, anchor_list, valid_flag_list, gt_bboxes_list, img_metas, gt_bboxes_ignore_list=None, gt_labels_list=None, label_channels=1, unmap_outputs=True)[source]¶ Compute regression and classification targets for anchors in multiple images.
Parameters: - cls_scores_list (list[Tensor]) – each image. each is a 4D-tensor, the shape is (h * w, num_anchors * num_classes).
- bbox_preds_list (list[Tensor]) – each is a 4D-tensor, the shape is (h * w, num_anchors * 4).
- anchor_list (list[Tensor]) – Anchors of each image. Each element of is a tensor of shape (h * w * num_anchors, 4).
- valid_flag_list (list[Tensor]) – Valid flags of each image. Each element of is a tensor of shape (h * w * num_anchors, )
- gt_bboxes_list (list[Tensor]) – Ground truth bboxes of each image.
- img_metas (list[dict]) – Meta info of each image.
- gt_bboxes_ignore_list (list[Tensor]) – Ground truth bboxes to be ignored.
- gt_labels_list (list[Tensor]) – Ground truth labels of each box.
- label_channels (int) – Channel of label.
- unmap_outputs (bool) – Whether to map outputs back to the original set of anchors.
Returns: Usually returns a tuple containing learning targets.
- batch_labels (Tensor): Label of all images. Each element of is a tensor of shape (batch, h * w * num_anchors)
- batch_label_weights (Tensor): Label weights of all images of is a tensor of shape (batch, h * w * num_anchors)
- num_total_pos (int): Number of positive samples in all images.
- num_total_neg (int): Number of negative samples in all images.
- additional_returns: This function enables user-defined returns from
self._get_targets_single. These returns are currently refined to properties at each feature map (i.e. having HxW dimension). The results will be concatenated after the end
Return type: tuple
-
loss
(cls_scores, bbox_preds, gt_bboxes, gt_labels, img_metas, gt_bboxes_ignore=None)[source]¶ Compute losses of the head.
Parameters: - cls_scores (list[Tensor]) – Box scores for each scale level Has shape (batch, num_anchors * num_classes, h, w)
- bbox_preds (list[Tensor]) – Box energies / deltas for each scale level with shape (batch, num_anchors * 4, h, w)
- gt_bboxes (list[Tensor]) – Ground truth bboxes for each image with shape (num_gts, 4) in [tl_x, tl_y, br_x, br_y] format.
- gt_labels (list[Tensor]) – class indices corresponding to each box
- img_metas (list[dict]) – Meta information of each image, e.g., image size, scaling factor, etc.
- gt_bboxes_ignore (None | list[Tensor]) – specify which bounding boxes can be ignored when computing the loss. Default: None
Returns: A dictionary of loss components.
Return type: dict[str, Tensor]
-
class
mmdet.models.dense_heads.
DeformableDETRHead
(*args, with_box_refine=False, as_two_stage=False, transformer=None, **kwargs)[source]¶ Head of DeformDETR: Deformable DETR: Deformable Transformers for End-to- End Object Detection.
Code is modified from the official github repo.
More details can be found in the paper .
Parameters: - with_box_refine (bool) – Whether to refine the reference points in the decoder. Defaults to False.
- as_two_stage (bool) – Whether to generate the proposal from the outputs of encoder.
- (obj (transformer) – ConfigDict): ConfigDict is used for building the Encoder and Decoder.
-
forward
(mlvl_feats, img_metas)[source]¶ Forward function.
Parameters: - mlvl_feats (tuple[Tensor]) – Features from the upstream network, each is a 4D-tensor with shape (N, C, H, W).
- img_metas (list[dict]) – List of image information.
Returns: Outputs from the classification head, shape [nb_dec, bs, num_query, cls_out_channels]. Note cls_out_channels should includes background. all_bbox_preds (Tensor): Sigmoid outputs from the regression head with normalized coordinate format (cx, cy, w, h). Shape [nb_dec, bs, num_query, 4]. enc_outputs_class (Tensor): The score of each point on encode feature map, has shape (N, h*w, num_class). Only when as_two_stage is True it would be returned, otherwise None would be returned. enc_outputs_coord (Tensor): The proposal generate from the encode feature map, has shape (N, h*w, 4). Only when as_two_stage is True it would be returned, otherwise None would be returned.
Return type: all_cls_scores (Tensor)
-
get_bboxes
(all_cls_scores, all_bbox_preds, enc_cls_scores, enc_bbox_preds, img_metas, rescale=False)[source]¶ Transform network outputs for a batch into bbox predictions.
Parameters: - all_cls_scores (Tensor) – Classification score of all decoder layers, has shape [nb_dec, bs, num_query, cls_out_channels].
- all_bbox_preds (Tensor) – Sigmoid regression outputs of all decode layers. Each is a 4D-tensor with normalized coordinate format (cx, cy, w, h) and shape [nb_dec, bs, num_query, 4].
- enc_cls_scores (Tensor) – Classification scores of points on encode feature map , has shape (N, h*w, num_classes). Only be passed when as_two_stage is True, otherwise is None.
- enc_bbox_preds (Tensor) – Regression results of each points on the encode feature map, has shape (N, h*w, 4). Only be passed when as_two_stage is True, otherwise is None.
- img_metas (list[dict]) – Meta information of each image.
- rescale (bool, optional) – If True, return boxes in original image space. Default False.
Returns: Each item in result_list is 2-tuple. The first item is an (n, 5) tensor, where the first 4 columns are bounding box positions (tl_x, tl_y, br_x, br_y) and the 5-th column is a score between 0 and 1. The second item is a (n,) tensor where each item is the predicted class label of the corresponding box.
Return type: list[list[Tensor, Tensor]]
-
loss
(all_cls_scores, all_bbox_preds, enc_cls_scores, enc_bbox_preds, gt_bboxes_list, gt_labels_list, img_metas, gt_bboxes_ignore=None)[source]¶ “Loss function.
Parameters: - all_cls_scores (Tensor) – Classification score of all decoder layers, has shape [nb_dec, bs, num_query, cls_out_channels].
- all_bbox_preds (Tensor) – Sigmoid regression outputs of all decode layers. Each is a 4D-tensor with normalized coordinate format (cx, cy, w, h) and shape [nb_dec, bs, num_query, 4].
- enc_cls_scores (Tensor) – Classification scores of points on encode feature map , has shape (N, h*w, num_classes). Only be passed when as_two_stage is True, otherwise is None.
- enc_bbox_preds (Tensor) – Regression results of each points on the encode feature map, has shape (N, h*w, 4). Only be passed when as_two_stage is True, otherwise is None.
- gt_bboxes_list (list[Tensor]) – Ground truth bboxes for each image with shape (num_gts, 4) in [tl_x, tl_y, br_x, br_y] format.
- gt_labels_list (list[Tensor]) – Ground truth class indices for each image with shape (num_gts, ).
- img_metas (list[dict]) – List of image meta information.
- gt_bboxes_ignore (list[Tensor], optional) – Bounding boxes which can be ignored for each image. Default None.
Returns: A dictionary of loss components.
Return type: dict[str, Tensor]
-
class
mmdet.models.dense_heads.
CenterNetHead
(in_channel, feat_channel, num_classes, loss_center_heatmap={'loss_weight': 1.0, 'type': 'GaussianFocalLoss'}, loss_wh={'loss_weight': 0.1, 'type': 'L1Loss'}, loss_offset={'loss_weight': 1.0, 'type': 'L1Loss'}, train_cfg=None, test_cfg=None, init_cfg=None)[source]¶ Objects as Points Head. CenterHead use center_point to indicate object’s position. Paper link <https://arxiv.org/abs/1904.07850>
Parameters: - in_channel (int) – Number of channel in the input feature map.
- feat_channel (int) – Number of channel in the intermediate feature map.
- num_classes (int) – Number of categories excluding the background category.
- loss_center_heatmap (dict | None) – Config of center heatmap loss. Default: GaussianFocalLoss.
- loss_wh (dict | None) – Config of wh loss. Default: L1Loss.
- loss_offset (dict | None) – Config of offset loss. Default: L1Loss.
- train_cfg (dict | None) – Training config. Useless in CenterNet, but we keep this variable for SingleStageDetector. Default: None.
- test_cfg (dict | None) – Testing config of CenterNet. Default: None.
- init_cfg (dict or list[dict], optional) – Initialization config dict. Default: None
-
decode_heatmap
(center_heatmap_pred, wh_pred, offset_pred, img_shape, k=100, kernel=3)[source]¶ Transform outputs into detections raw bbox prediction.
Parameters: - center_heatmap_pred (Tensor) – center predict heatmap, shape (B, num_classes, H, W).
- wh_pred (Tensor) – wh predict, shape (B, 2, H, W).
- offset_pred (Tensor) – offset predict, shape (B, 2, H, W).
- img_shape (list[int]) – image shape in [h, w] format.
- k (int) – Get top k center keypoints from heatmap. Default 100.
- kernel (int) – Max pooling kernel for extract local maximum pixels. Default 3.
Returns: - Decoded output of CenterNetHead, containing
the following Tensors:
- batch_bboxes (Tensor): Coords of each box with shape (B, k, 5)
- batch_topk_labels (Tensor): Categories of each box with shape (B, k)
Return type: tuple[torch.Tensor]
-
forward
(feats)[source]¶ Forward features. Notice CenterNet head does not use FPN.
Parameters: feats (tuple[Tensor]) – Features from the upstream network, each is a 4D-tensor. Returns: - center predict heatmaps for
- all levels, the channels number is num_classes.
- wh_preds (List[Tensor]): wh predicts for all levels, the channels
- number is 2.
- offset_preds (List[Tensor]): offset predicts for all levels, the
- channels number is 2.
Return type: center_heatmap_preds (List[Tensor])
-
forward_single
(feat)[source]¶ Forward feature of a single level.
Parameters: feat (Tensor) – Feature of a single level. Returns: - center predict heatmaps, the
- channels number is num_classes.
wh_pred (Tensor): wh predicts, the channels number is 2. offset_pred (Tensor): offset predicts, the channels number is 2.
Return type: center_heatmap_pred (Tensor)
-
get_bboxes
(center_heatmap_preds, wh_preds, offset_preds, img_metas, rescale=True, with_nms=False)[source]¶ Transform network output for a batch into bbox predictions.
Parameters: - center_heatmap_preds (list[Tensor]) – center predict heatmaps for all levels with shape (B, num_classes, H, W).
- wh_preds (list[Tensor]) – wh predicts for all levels with shape (B, 2, H, W).
- offset_preds (list[Tensor]) – offset predicts for all levels with shape (B, 2, H, W).
- img_metas (list[dict]) – Meta information of each image, e.g., image size, scaling factor, etc.
- rescale (bool) – If True, return boxes in original image space. Default: True.
- with_nms (bool) – If True, do nms before return boxes. Default: False.
Returns: - Each item in result_list is 2-tuple.
The first item is an (n, 5) tensor, where 5 represent (tl_x, tl_y, br_x, br_y, score) and the score between 0 and 1. The shape of the second tensor in the tuple is (n,), and each element represents the class label of the corresponding box.
Return type: list[tuple[Tensor, Tensor]]
-
get_targets
(gt_bboxes, gt_labels, feat_shape, img_shape)[source]¶ Compute regression and classification targets in multiple images.
Parameters: - gt_bboxes (list[Tensor]) – Ground truth bboxes for each image with shape (num_gts, 4) in [tl_x, tl_y, br_x, br_y] format.
- gt_labels (list[Tensor]) – class indices corresponding to each box.
- feat_shape (list[int]) – feature map shape with value [B, _, H, W]
- img_shape (list[int]) – image shape in [h, w] format.
Returns: - The float value is mean avg_factor, the dict has
components below: - center_heatmap_target (Tensor): targets of center heatmap, shape (B, num_classes, H, W). - wh_target (Tensor): targets of wh predict, shape (B, 2, H, W). - offset_target (Tensor): targets of offset predict, shape (B, 2, H, W). - wh_offset_target_weight (Tensor): weights of wh and offset predict, shape (B, 2, H, W).
Return type: tuple[dict,float]
-
loss
(center_heatmap_preds, wh_preds, offset_preds, gt_bboxes, gt_labels, img_metas, gt_bboxes_ignore=None)[source]¶ Compute losses of the head.
Parameters: - center_heatmap_preds (list[Tensor]) – center predict heatmaps for all levels with shape (B, num_classes, H, W).
- wh_preds (list[Tensor]) – wh predicts for all levels with shape (B, 2, H, W).
- offset_preds (list[Tensor]) – offset predicts for all levels with shape (B, 2, H, W).
- gt_bboxes (list[Tensor]) – Ground truth bboxes for each image with shape (num_gts, 4) in [tl_x, tl_y, br_x, br_y] format.
- gt_labels (list[Tensor]) – class indices corresponding to each box.
- img_metas (list[dict]) – Meta information of each image, e.g., image size, scaling factor, etc.
- gt_bboxes_ignore (None | list[Tensor]) – specify which bounding boxes can be ignored when computing the loss. Default: None
Returns: - which has components below:
- loss_center_heatmap (Tensor): loss of center heatmap.
- loss_wh (Tensor): loss of hw heatmap
- loss_offset (Tensor): loss of offset heatmap.
Return type: dict[str, Tensor]
-
class
mmdet.models.dense_heads.
YOLOXHead
(num_classes, in_channels, feat_channels=256, stacked_convs=2, strides=[8, 16, 32], use_depthwise=False, dcn_on_last_conv=False, conv_bias='auto', conv_cfg=None, norm_cfg={'eps': 0.001, 'momentum': 0.03, 'type': 'BN'}, act_cfg={'type': 'Swish'}, loss_cls={'loss_weight': 1.0, 'reduction': 'sum', 'type': 'CrossEntropyLoss', 'use_sigmoid': True}, loss_bbox={'eps': 1e-16, 'loss_weight': 5.0, 'mode': 'square', 'reduction': 'sum', 'type': 'IoULoss'}, loss_obj={'loss_weight': 1.0, 'reduction': 'sum', 'type': 'CrossEntropyLoss', 'use_sigmoid': True}, loss_l1={'loss_weight': 1.0, 'reduction': 'sum', 'type': 'L1Loss'}, train_cfg=None, test_cfg=None, init_cfg={'a': 2.23606797749979, 'distribution': 'uniform', 'layer': 'Conv2d', 'mode': 'fan_in', 'nonlinearity': 'leaky_relu', 'type': 'Kaiming'})[source]¶ YOLOXHead head used in YOLOX.
Parameters: - num_classes (int) – Number of categories excluding the background category.
- in_channels (int) – Number of channels in the input feature map.
- feat_channels (int) – Number of hidden channels in stacking convs. Default: 256
- stacked_convs (int) – Number of stacking convs of the head. Default: 2.
- strides (tuple) – Downsample factor of each feature map.
- use_depthwise (bool) – Whether to depthwise separable convolution in blocks. Default: False
- dcn_on_last_conv (bool) – If true, use dcn in the last layer of towers. Default: False.
- conv_bias (bool | str) – If specified as auto, it will be decided by the norm_cfg. Bias of conv will be set as True if norm_cfg is None, otherwise False. Default: “auto”.
- conv_cfg (dict) – Config dict for convolution layer. Default: None.
- norm_cfg (dict) – Config dict for normalization layer. Default: None.
- act_cfg (dict) – Config dict for activation layer. Default: None.
- loss_cls (dict) – Config of classification loss.
- loss_bbox (dict) – Config of localization loss.
- loss_obj (dict) – Config of objectness loss.
- loss_l1 (dict) – Config of L1 loss.
- train_cfg (dict) – Training config of anchor head.
- test_cfg (dict) – Testing config of anchor head.
- init_cfg (dict or list[dict], optional) – Initialization config dict.
-
forward
(feats)[source]¶ Forward features from the upstream network.
Parameters: feats (tuple[Tensor]) – Features from the upstream network, each is a 4D-tensor. Returns: - A tuple of multi-level predication map, each is a
- 4D-tensor of shape (batch_size, 5+num_classes, height, width).
Return type: tuple[Tensor]
-
forward_single
(x, cls_convs, reg_convs, conv_cls, conv_reg, conv_obj)[source]¶ Forward feature of a single scale level.
-
get_bboxes
(cls_scores, bbox_preds, objectnesses, img_metas=None, cfg=None, rescale=False, with_nms=True)[source]¶ Transform network outputs of a batch into bbox results. :param cls_scores: Classification scores for all
scale levels, each is a 4D-tensor, has shape (batch_size, num_priors * num_classes, H, W).Parameters: - bbox_preds (list[Tensor]) – Box energies / deltas for all scale levels, each is a 4D-tensor, has shape (batch_size, num_priors * 4, H, W).
- objectnesses (list[Tensor], Optional) – Score factor for all scale level, each is a 4D-tensor, has shape (batch_size, 1, H, W).
- img_metas (list[dict], Optional) – Image meta info. Default None.
- cfg (mmcv.Config, Optional) – Test / postprocessing configuration, if None, test_cfg would be used. Default None.
- rescale (bool) – If True, return boxes in original image space. Default False.
- with_nms (bool) – If True, do nms before return boxes. Default True.
Returns: - Each item in result_list is 2-tuple.
The first item is an (n, 5) tensor, where the first 4 columns are bounding box positions (tl_x, tl_y, br_x, br_y) and the 5-th column is a score between 0 and 1. The second item is a (n,) tensor where each item is the predicted class label of the corresponding box.
Return type: list[list[Tensor, Tensor]]
-
loss
(cls_scores, bbox_preds, objectnesses, gt_bboxes, gt_labels, img_metas, gt_bboxes_ignore=None)[source]¶ Compute loss of the head. :param cls_scores: Box scores for each scale level,
each is a 4D-tensor, the channel number is num_priors * num_classes.Parameters: - bbox_preds (list[Tensor]) – Box energies / deltas for each scale level, each is a 4D-tensor, the channel number is num_priors * 4.
- objectnesses (list[Tensor], Optional) – Score factor for all scale level, each is a 4D-tensor, has shape (batch_size, 1, H, W).
- gt_bboxes (list[Tensor]) – Ground truth bboxes for each image with shape (num_gts, 4) in [tl_x, tl_y, br_x, br_y] format.
- gt_labels (list[Tensor]) – class indices corresponding to each box
- img_metas (list[dict]) – Meta information of each image, e.g., image size, scaling factor, etc.
- gt_bboxes_ignore (None | list[Tensor]) – specify which bounding boxes can be ignored when computing the loss.
roi_heads¶
-
class
mmdet.models.roi_heads.
BaseRoIHead
(bbox_roi_extractor=None, bbox_head=None, mask_roi_extractor=None, mask_head=None, shared_head=None, train_cfg=None, test_cfg=None, pretrained=None, init_cfg=None)[source]¶ Base class for RoIHeads.
-
async_simple_test
(x, proposal_list, img_metas, proposals=None, rescale=False, **kwargs)[source]¶ Asynchronized test function.
-
aug_test
(x, proposal_list, img_metas, rescale=False, **kwargs)[source]¶ Test with augmentations.
If rescale is False, then returned bboxes and masks will fit the scale of imgs[0].
-
forward_train
(x, img_meta, proposal_list, gt_bboxes, gt_labels, gt_bboxes_ignore=None, gt_masks=None, **kwargs)[source]¶ Forward function during training.
-
simple_test
(x, proposal_list, img_meta, proposals=None, rescale=False, **kwargs)[source]¶ Test without augmentation.
-
with_bbox
¶ whether the RoI head contains a bbox_head
Type: bool
-
with_mask
¶ whether the RoI head contains a mask_head
Type: bool
whether the RoI head contains a shared_head
Type: bool
-
-
class
mmdet.models.roi_heads.
CascadeRoIHead
(num_stages, stage_loss_weights, bbox_roi_extractor=None, bbox_head=None, mask_roi_extractor=None, mask_head=None, shared_head=None, train_cfg=None, test_cfg=None, pretrained=None, init_cfg=None)[source]¶ Cascade roi head including one bbox head and one mask head.
https://arxiv.org/abs/1712.00726
-
aug_test
(features, proposal_list, img_metas, rescale=False)[source]¶ Test with augmentations.
If rescale is False, then returned bboxes and masks will fit the scale of imgs[0].
-
forward_train
(x, img_metas, proposal_list, gt_bboxes, gt_labels, gt_bboxes_ignore=None, gt_masks=None)[source]¶ Parameters: - x (list[Tensor]) – list of multi-level img features.
- img_metas (list[dict]) – list of image info dict where each dict has: ‘img_shape’, ‘scale_factor’, ‘flip’, and may also contain ‘filename’, ‘ori_shape’, ‘pad_shape’, and ‘img_norm_cfg’. For details on the values of these keys see mmdet/datasets/pipelines/formatting.py:Collect.
- proposals (list[Tensors]) – list of region proposals.
- gt_bboxes (list[Tensor]) – Ground truth bboxes for each image with shape (num_gts, 4) in [tl_x, tl_y, br_x, br_y] format.
- gt_labels (list[Tensor]) – class indices corresponding to each box
- gt_bboxes_ignore (None | list[Tensor]) – specify which bounding boxes can be ignored when computing the loss.
- gt_masks (None | Tensor) – true segmentation masks for each box used if the architecture supports a segmentation task.
Returns: a dictionary of loss components
Return type: dict[str, Tensor]
-
init_bbox_head
(bbox_roi_extractor, bbox_head)[source]¶ Initialize box head and box roi extractor.
Parameters: - bbox_roi_extractor (dict) – Config of box roi extractor.
- bbox_head (dict) – Config of box in box head.
-
-
class
mmdet.models.roi_heads.
DoubleHeadRoIHead
(reg_roi_scale_factor, **kwargs)[source]¶ RoI head for Double Head RCNN.
-
class
mmdet.models.roi_heads.
MaskScoringRoIHead
(mask_iou_head, **kwargs)[source]¶ Mask Scoring RoIHead for Mask Scoring RCNN.
-
class
mmdet.models.roi_heads.
HybridTaskCascadeRoIHead
(num_stages, stage_loss_weights, semantic_roi_extractor=None, semantic_head=None, semantic_fusion=('bbox', 'mask'), interleaved=True, mask_info_flow=True, **kwargs)[source]¶ Hybrid task cascade roi head including one bbox head and one mask head.
https://arxiv.org/abs/1901.07518
-
aug_test
(img_feats, proposal_list, img_metas, rescale=False)[source]¶ Test with augmentations.
If rescale is False, then returned bboxes and masks will fit the scale of imgs[0].
-
forward_train
(x, img_metas, proposal_list, gt_bboxes, gt_labels, gt_bboxes_ignore=None, gt_masks=None, gt_semantic_seg=None)[source]¶ Parameters: - x (list[Tensor]) – list of multi-level img features.
- img_metas (list[dict]) – list of image info dict where each dict has: ‘img_shape’, ‘scale_factor’, ‘flip’, and may also contain ‘filename’, ‘ori_shape’, ‘pad_shape’, and ‘img_norm_cfg’. For details on the values of these keys see mmdet/datasets/pipelines/formatting.py:Collect.
- proposal_list (list[Tensors]) – list of region proposals.
- gt_bboxes (list[Tensor]) – Ground truth bboxes for each image with shape (num_gts, 4) in [tl_x, tl_y, br_x, br_y] format.
- gt_labels (list[Tensor]) – class indices corresponding to each box
- gt_bboxes_ignore (None, list[Tensor]) – specify which bounding boxes can be ignored when computing the loss.
- gt_masks (None, Tensor) – true segmentation masks for each box used if the architecture supports a segmentation task.
- gt_semantic_seg (None, list[Tensor]) – semantic segmentation masks used if the architecture supports semantic segmentation task.
Returns: a dictionary of loss components
Return type: dict[str, Tensor]
-
with_semantic
¶ whether the head has semantic head
Type: bool
-
-
class
mmdet.models.roi_heads.
GridRoIHead
(grid_roi_extractor, grid_head, **kwargs)[source]¶ Grid roi head for Grid R-CNN.
-
class
mmdet.models.roi_heads.
ResLayer
(depth, stage=3, stride=2, dilation=1, style='pytorch', norm_cfg={'requires_grad': True, 'type': 'BN'}, norm_eval=True, with_cp=False, dcn=None, pretrained=None, init_cfg=None)[source]¶ -
forward
(x)[source]¶ Defines the computation performed at every call.
Should be overridden by all subclasses.
Note
Although the recipe for forward pass needs to be defined within this function, one should call the
Module
instance afterwards instead of this since the former takes care of running the registered hooks while the latter silently ignores them.
-
train
(mode=True)[source]¶ Sets the module in training mode.
This has any effect only on certain modules. See documentations of particular modules for details of their behaviors in training/evaluation mode, if they are affected, e.g.
Dropout
,BatchNorm
, etc.Parameters: mode (bool) – whether to set training mode ( True
) or evaluation mode (False
). Default:True
.Returns: self Return type: Module
-
-
class
mmdet.models.roi_heads.
BBoxHead
(with_avg_pool=False, with_cls=True, with_reg=True, roi_feat_size=7, in_channels=256, num_classes=80, bbox_coder={'clip_border': True, 'target_means': [0.0, 0.0, 0.0, 0.0], 'target_stds': [0.1, 0.1, 0.2, 0.2], 'type': 'DeltaXYWHBBoxCoder'}, reg_class_agnostic=False, reg_decoded_bbox=False, reg_predictor_cfg={'type': 'Linear'}, cls_predictor_cfg={'type': 'Linear'}, loss_cls={'loss_weight': 1.0, 'type': 'CrossEntropyLoss', 'use_sigmoid': False}, loss_bbox={'beta': 1.0, 'loss_weight': 1.0, 'type': 'SmoothL1Loss'}, init_cfg=None)[source]¶ Simplest RoI head, with only two fc layers for classification and regression respectively.
-
forward
(x)[source]¶ Defines the computation performed at every call.
Should be overridden by all subclasses.
Note
Although the recipe for forward pass needs to be defined within this function, one should call the
Module
instance afterwards instead of this since the former takes care of running the registered hooks while the latter silently ignores them.
-
get_bboxes
(rois, cls_score, bbox_pred, img_shape, scale_factor, rescale=False, cfg=None)[source]¶ Transform network output for a batch into bbox predictions.
Parameters: - rois (Tensor) – Boxes to be transformed. Has shape (num_boxes, 5). last dimension 5 arrange as (batch_index, x1, y1, x2, y2).
- cls_score (Tensor) – Box scores, has shape (num_boxes, num_classes + 1).
- bbox_pred (Tensor, optional) – Box energies / deltas. has shape (num_boxes, num_classes * 4).
- img_shape (Sequence[int], optional) – Maximum bounds for boxes, specifies (H, W, C) or (H, W).
- scale_factor (ndarray) – Scale factor of the image arrange as (w_scale, h_scale, w_scale, h_scale).
- rescale (bool) – If True, return boxes in original image space. Default: False.
- (obj (cfg) – ConfigDict): test_cfg of Bbox Head. Default: None
Returns: Fisrt tensor is det_bboxes, has the shape (num_boxes, 5) and last dimension 5 represent (tl_x, tl_y, br_x, br_y, score). Second tensor is the labels with shape (num_boxes, ).
Return type: tuple[Tensor, Tensor]
-
get_targets
(sampling_results, gt_bboxes, gt_labels, rcnn_train_cfg, concat=True)[source]¶ Calculate the ground truth for all samples in a batch according to the sampling_results.
Almost the same as the implementation in bbox_head, we passed additional parameters pos_inds_list and neg_inds_list to _get_target_single function.
Parameters: - (List[obj (sampling_results) – SamplingResults]): Assign results of all images in a batch after sampling.
- gt_bboxes (list[Tensor]) – Gt_bboxes of all images in a batch, each tensor has shape (num_gt, 4), the last dimension 4 represents [tl_x, tl_y, br_x, br_y].
- gt_labels (list[Tensor]) – Gt_labels of all images in a batch, each tensor has shape (num_gt,).
- (obj (rcnn_train_cfg) – ConfigDict): train_cfg of RCNN.
- concat (bool) – Whether to concatenate the results of all the images in a single batch.
Returns: Ground truth for proposals in a single image. Containing the following list of Tensors:
- labels (list[Tensor],Tensor): Gt_labels for all proposals in a batch, each tensor in list has shape (num_proposals,) when concat=False, otherwise just a single tensor has shape (num_all_proposals,).
- label_weights (list[Tensor]): Labels_weights for all proposals in a batch, each tensor in list has shape (num_proposals,) when concat=False, otherwise just a single tensor has shape (num_all_proposals,).
- bbox_targets (list[Tensor],Tensor): Regression target for all proposals in a batch, each tensor in list has shape (num_proposals, 4) when concat=False, otherwise just a single tensor has shape (num_all_proposals, 4), the last dimension 4 represents [tl_x, tl_y, br_x, br_y].
- bbox_weights (list[tensor],Tensor): Regression weights for all proposals in a batch, each tensor in list has shape (num_proposals, 4) when concat=False, otherwise just a single tensor has shape (num_all_proposals, 4).
Return type: Tuple[Tensor]
-
onnx_export
(rois, cls_score, bbox_pred, img_shape, cfg=None, **kwargs)[source]¶ Transform network output for a batch into bbox predictions.
Parameters: - rois (Tensor) – Boxes to be transformed. Has shape (B, num_boxes, 5)
- cls_score (Tensor) – Box scores. has shape (B, num_boxes, num_classes + 1), 1 represent the background.
- bbox_pred (Tensor, optional) – Box energies / deltas for, has shape (B, num_boxes, num_classes * 4) when.
- img_shape (torch.Tensor) – Shape of image.
- (obj (cfg) – ConfigDict): test_cfg of Bbox Head. Default: None
Returns: - dets of shape [N, num_det, 5]
and class labels of shape [N, num_det].
Return type: tuple[Tensor, Tensor]
-
refine_bboxes
(rois, labels, bbox_preds, pos_is_gts, img_metas)[source]¶ Refine bboxes during training.
Parameters: - rois (Tensor) – Shape (n*bs, 5), where n is image number per GPU, and bs is the sampled RoIs per image. The first column is the image id and the next 4 columns are x1, y1, x2, y2.
- labels (Tensor) – Shape (n*bs, ).
- bbox_preds (Tensor) – Shape (n*bs, 4) or (n*bs, 4*#class).
- pos_is_gts (list[Tensor]) – Flags indicating if each positive bbox is a gt bbox.
- img_metas (list[dict]) – Meta info of each image.
Returns: Refined bboxes of each image in a mini-batch.
Return type: list[Tensor]
Example
>>> # xdoctest: +REQUIRES(module:kwarray) >>> import kwarray >>> import numpy as np >>> from mmdet.core.bbox.demodata import random_boxes >>> self = BBoxHead(reg_class_agnostic=True) >>> n_roi = 2 >>> n_img = 4 >>> scale = 512 >>> rng = np.random.RandomState(0) >>> img_metas = [{'img_shape': (scale, scale)} ... for _ in range(n_img)] >>> # Create rois in the expected format >>> roi_boxes = random_boxes(n_roi, scale=scale, rng=rng) >>> img_ids = torch.randint(0, n_img, (n_roi,)) >>> img_ids = img_ids.float() >>> rois = torch.cat([img_ids[:, None], roi_boxes], dim=1) >>> # Create other args >>> labels = torch.randint(0, 2, (n_roi,)).long() >>> bbox_preds = random_boxes(n_roi, scale=scale, rng=rng) >>> # For each image, pretend random positive boxes are gts >>> is_label_pos = (labels.numpy() > 0).astype(np.int) >>> lbl_per_img = kwarray.group_items(is_label_pos, ... img_ids.numpy()) >>> pos_per_img = [sum(lbl_per_img.get(gid, [])) ... for gid in range(n_img)] >>> pos_is_gts = [ >>> torch.randint(0, 2, (npos,)).byte().sort( >>> descending=True)[0] >>> for npos in pos_per_img >>> ] >>> bboxes_list = self.refine_bboxes(rois, labels, bbox_preds, >>> pos_is_gts, img_metas) >>> print(bboxes_list)
-
regress_by_class
(rois, label, bbox_pred, img_meta)[source]¶ Regress the bbox for the predicted class. Used in Cascade R-CNN.
Parameters: - rois (Tensor) – shape (n, 4) or (n, 5)
- label (Tensor) – shape (n, )
- bbox_pred (Tensor) – shape (n, 4*(#class)) or (n, 4)
- img_meta (dict) – Image meta info.
Returns: Regressed bboxes, the same shape as input rois.
Return type: Tensor
-
-
class
mmdet.models.roi_heads.
ConvFCBBoxHead
(num_shared_convs=0, num_shared_fcs=0, num_cls_convs=0, num_cls_fcs=0, num_reg_convs=0, num_reg_fcs=0, conv_out_channels=256, fc_out_channels=1024, conv_cfg=None, norm_cfg=None, init_cfg=None, *args, **kwargs)[source]¶ More general bbox head, with shared conv and fc layers and two optional separated branches.
/-> cls convs -> cls fcs -> cls shared convs -> shared fcs \-> reg convs -> reg fcs -> reg
-
forward
(x)[source]¶ Defines the computation performed at every call.
Should be overridden by all subclasses.
Note
Although the recipe for forward pass needs to be defined within this function, one should call the
Module
instance afterwards instead of this since the former takes care of running the registered hooks while the latter silently ignores them.
-
-
class
mmdet.models.roi_heads.
DIIHead
(num_classes=80, num_ffn_fcs=2, num_heads=8, num_cls_fcs=1, num_reg_fcs=3, feedforward_channels=2048, in_channels=256, dropout=0.0, ffn_act_cfg={'inplace': True, 'type': 'ReLU'}, dynamic_conv_cfg={'act_cfg': {'inplace': True, 'type': 'ReLU'}, 'feat_channels': 64, 'in_channels': 256, 'input_feat_shape': 7, 'norm_cfg': {'type': 'LN'}, 'out_channels': 256, 'type': 'DynamicConv'}, loss_iou={'loss_weight': 2.0, 'type': 'GIoULoss'}, init_cfg=None, **kwargs)[source]¶ Dynamic Instance Interactive Head for Sparse R-CNN: End-to-End Object Detection with Learnable Proposals
Parameters: - num_classes (int) – Number of class in dataset. Defaults to 80.
- num_ffn_fcs (int) – The number of fully-connected layers in FFNs. Defaults to 2.
- num_heads (int) – The hidden dimension of FFNs. Defaults to 8.
- num_cls_fcs (int) – The number of fully-connected layers in classification subnet. Defaults to 1.
- num_reg_fcs (int) – The number of fully-connected layers in regression subnet. Defaults to 3.
- feedforward_channels (int) – The hidden dimension of FFNs. Defaults to 2048
- in_channels (int) – Hidden_channels of MultiheadAttention. Defaults to 256.
- dropout (float) – Probability of drop the channel. Defaults to 0.0
- ffn_act_cfg (dict) – The activation config for FFNs.
- dynamic_conv_cfg (dict) – The convolution config for DynamicConv.
- loss_iou (dict) – The config for iou or giou loss.
-
forward
(roi_feat, proposal_feat)[source]¶ Forward function of Dynamic Instance Interactive Head.
Parameters: - roi_feat (Tensor) – Roi-pooling features with shape (batch_size*num_proposals, feature_dimensions, pooling_h , pooling_w).
- proposal_feat – Intermediate feature get from diihead in last stage, has shape (batch_size, num_proposals, feature_dimensions)
-
get_targets
(sampling_results, gt_bboxes, gt_labels, rcnn_train_cfg, concat=True)[source]¶ Calculate the ground truth for all samples in a batch according to the sampling_results.
Almost the same as the implementation in bbox_head, we passed additional parameters pos_inds_list and neg_inds_list to _get_target_single function.
Parameters: - (List[obj (sampling_results) – SamplingResults]): Assign results of all images in a batch after sampling.
- gt_bboxes (list[Tensor]) – Gt_bboxes of all images in a batch, each tensor has shape (num_gt, 4), the last dimension 4 represents [tl_x, tl_y, br_x, br_y].
- gt_labels (list[Tensor]) – Gt_labels of all images in a batch, each tensor has shape (num_gt,).
- (obj (rcnn_train_cfg) – ConfigDict): train_cfg of RCNN.
- concat (bool) – Whether to concatenate the results of all the images in a single batch.
Returns: Ground truth for proposals in a single image. Containing the following list of Tensors:
- labels (list[Tensor],Tensor): Gt_labels for all proposals in a batch, each tensor in list has shape (num_proposals,) when concat=False, otherwise just a single tensor has shape (num_all_proposals,).
- label_weights (list[Tensor]): Labels_weights for all proposals in a batch, each tensor in list has shape (num_proposals,) when concat=False, otherwise just a single tensor has shape (num_all_proposals,).
- bbox_targets (list[Tensor],Tensor): Regression target for all proposals in a batch, each tensor in list has shape (num_proposals, 4) when concat=False, otherwise just a single tensor has shape (num_all_proposals, 4), the last dimension 4 represents [tl_x, tl_y, br_x, br_y].
- bbox_weights (list[tensor],Tensor): Regression weights for all proposals in a batch, each tensor in list has shape (num_proposals, 4) when concat=False, otherwise just a single tensor has shape (num_all_proposals, 4).
Return type: Tuple[Tensor]
-
init_weights
()[source]¶ Use xavier initialization for all weight parameter and set classification head bias as a specific value when use focal loss.
-
loss
(cls_score, bbox_pred, labels, label_weights, bbox_targets, bbox_weights, imgs_whwh=None, reduction_override=None, **kwargs)[source]¶ “Loss function of DIIHead, get loss of all images.
Parameters: - cls_score (Tensor) – Classification prediction results of all class, has shape (batch_size * num_proposals_single_image, num_classes)
- bbox_pred (Tensor) – Regression prediction results, has shape (batch_size * num_proposals_single_image, 4), the last dimension 4 represents [tl_x, tl_y, br_x, br_y].
- labels (Tensor) – Label of each proposals, has shape (batch_size * num_proposals_single_image
- label_weights (Tensor) – Classification loss weight of each proposals, has shape (batch_size * num_proposals_single_image
- bbox_targets (Tensor) – Regression targets of each proposals, has shape (batch_size * num_proposals_single_image, 4), the last dimension 4 represents [tl_x, tl_y, br_x, br_y].
- bbox_weights (Tensor) – Regression loss weight of each proposals’s coordinate, has shape (batch_size * num_proposals_single_image, 4),
- imgs_whwh (Tensor) – imgs_whwh (Tensor): Tensor with shape (batch_size, num_proposals, 4), the last dimension means [img_width,img_height, img_width, img_height].
- reduction_override (str, optional) – The reduction method used to override the original reduction method of the loss. Options are “none”, “mean” and “sum”. Defaults to None,
- Returns – dict[str, Tensor]: Dictionary of loss components
-
class
mmdet.models.roi_heads.
SABLHead
(num_classes, cls_in_channels=256, reg_in_channels=256, roi_feat_size=7, reg_feat_up_ratio=2, reg_pre_kernel=3, reg_post_kernel=3, reg_pre_num=2, reg_post_num=1, cls_out_channels=1024, reg_offset_out_channels=256, reg_cls_out_channels=256, num_cls_fcs=1, num_reg_fcs=0, reg_class_agnostic=True, norm_cfg=None, bbox_coder={'num_buckets': 14, 'scale_factor': 1.7, 'type': 'BucketingBBoxCoder'}, loss_cls={'loss_weight': 1.0, 'type': 'CrossEntropyLoss', 'use_sigmoid': False}, loss_bbox_cls={'loss_weight': 1.0, 'type': 'CrossEntropyLoss', 'use_sigmoid': True}, loss_bbox_reg={'beta': 0.1, 'loss_weight': 1.0, 'type': 'SmoothL1Loss'}, init_cfg=None)[source]¶ Side-Aware Boundary Localization (SABL) for RoI-Head.
Side-Aware features are extracted by conv layers with an attention mechanism. Boundary Localization with Bucketing and Bucketing Guided Rescoring are implemented in BucketingBBoxCoder.
Please refer to https://arxiv.org/abs/1912.04260 for more details.
Parameters: - cls_in_channels (int) – Input channels of cls RoI feature. Defaults to 256.
- reg_in_channels (int) – Input channels of reg RoI feature. Defaults to 256.
- roi_feat_size (int) – Size of RoI features. Defaults to 7.
- reg_feat_up_ratio (int) – Upsample ratio of reg features. Defaults to 2.
- reg_pre_kernel (int) – Kernel of 2D conv layers before attention pooling. Defaults to 3.
- reg_post_kernel (int) – Kernel of 1D conv layers after attention pooling. Defaults to 3.
- reg_pre_num (int) – Number of pre convs. Defaults to 2.
- reg_post_num (int) – Number of post convs. Defaults to 1.
- num_classes (int) – Number of classes in dataset. Defaults to 80.
- cls_out_channels (int) – Hidden channels in cls fcs. Defaults to 1024.
- reg_offset_out_channels (int) – Hidden and output channel of reg offset branch. Defaults to 256.
- reg_cls_out_channels (int) – Hidden and output channel of reg cls branch. Defaults to 256.
- num_cls_fcs (int) – Number of fcs for cls branch. Defaults to 1.
- num_reg_fcs (int) – Number of fcs for reg branch.. Defaults to 0.
- reg_class_agnostic (bool) – Class agnostic regresion or not. Defaults to True.
- norm_cfg (dict) – Config of norm layers. Defaults to None.
- bbox_coder (dict) – Config of bbox coder. Defaults ‘BucketingBBoxCoder’.
- loss_cls (dict) – Config of classification loss.
- loss_bbox_cls (dict) – Config of classification loss for bbox branch.
- loss_bbox_reg (dict) – Config of regression loss for bbox branch.
- init_cfg (dict or list[dict], optional) – Initialization config dict. Default: None
-
attention_pool
(reg_x)[source]¶ Extract direction-specific features fx and fy with attention methanism.
-
bbox_pred_split
(bbox_pred, num_proposals_per_img)[source]¶ Split batch bbox prediction back to each image.
-
forward
(x)[source]¶ Defines the computation performed at every call.
Should be overridden by all subclasses.
Note
Although the recipe for forward pass needs to be defined within this function, one should call the
Module
instance afterwards instead of this since the former takes care of running the registered hooks while the latter silently ignores them.
-
refine_bboxes
(rois, labels, bbox_preds, pos_is_gts, img_metas)[source]¶ Refine bboxes during training.
Parameters: - rois (Tensor) – Shape (n*bs, 5), where n is image number per GPU, and bs is the sampled RoIs per image.
- labels (Tensor) – Shape (n*bs, ).
- bbox_preds (list[Tensor]) – Shape [(n*bs, num_buckets*2), (n*bs, num_buckets*2)].
- pos_is_gts (list[Tensor]) – Flags indicating if each positive bbox is a gt bbox.
- img_metas (list[dict]) – Meta info of each image.
Returns: Refined bboxes of each image in a mini-batch.
Return type: list[Tensor]
-
reg_pred
(x, offset_fcs, cls_fcs)[source]¶ Predict bucketing estimation (cls_pred) and fine regression (offset pred) with side-aware features.
-
regress_by_class
(rois, label, bbox_pred, img_meta)[source]¶ Regress the bbox for the predicted class. Used in Cascade R-CNN.
Parameters: Returns: Regressed bboxes, the same shape as input rois.
Return type: Tensor
-
class
mmdet.models.roi_heads.
StandardRoIHead
(bbox_roi_extractor=None, bbox_head=None, mask_roi_extractor=None, mask_head=None, shared_head=None, train_cfg=None, test_cfg=None, pretrained=None, init_cfg=None)[source]¶ Simplest base roi head including one bbox head and one mask head.
-
async_simple_test
(x, proposal_list, img_metas, proposals=None, rescale=False)[source]¶ Async test without augmentation.
-
aug_test
(x, proposal_list, img_metas, rescale=False)[source]¶ Test with augmentations.
If rescale is False, then returned bboxes and masks will fit the scale of imgs[0].
-
bbox_onnx_export
(x, img_metas, proposals, rcnn_test_cfg, **kwargs)[source]¶ Export bbox branch to onnx which supports batch inference.
Parameters: - x (tuple[Tensor]) – Feature maps of all scale level.
- img_metas (list[dict]) – Image meta info.
- proposals (Tensor) – Region proposals with batch dimension, has shape [N, num_bboxes, 5].
- (obj (rcnn_test_cfg) – ConfigDict): test_cfg of R-CNN.
Returns: - bboxes of shape [N, num_bboxes, 5]
and class labels of shape [N, num_bboxes].
Return type: tuple[Tensor, Tensor]
-
forward_train
(x, img_metas, proposal_list, gt_bboxes, gt_labels, gt_bboxes_ignore=None, gt_masks=None)[source]¶ Parameters: - x (list[Tensor]) – list of multi-level img features.
- img_metas (list[dict]) – list of image info dict where each dict has: ‘img_shape’, ‘scale_factor’, ‘flip’, and may also contain ‘filename’, ‘ori_shape’, ‘pad_shape’, and ‘img_norm_cfg’. For details on the values of these keys see mmdet/datasets/pipelines/formatting.py:Collect.
- proposals (list[Tensors]) – list of region proposals.
- gt_bboxes (list[Tensor]) – Ground truth bboxes for each image with shape (num_gts, 4) in [tl_x, tl_y, br_x, br_y] format.
- gt_labels (list[Tensor]) – class indices corresponding to each box
- gt_bboxes_ignore (None | list[Tensor]) – specify which bounding boxes can be ignored when computing the loss.
- gt_masks (None | Tensor) – true segmentation masks for each box used if the architecture supports a segmentation task.
Returns: a dictionary of loss components
Return type: dict[str, Tensor]
-
mask_onnx_export
(x, img_metas, det_bboxes, det_labels, **kwargs)[source]¶ Export mask branch to onnx which supports batch inference.
Parameters: - x (tuple[Tensor]) – Feature maps of all scale level.
- img_metas (list[dict]) – Image meta info.
- det_bboxes (Tensor) – Bboxes and corresponding scores. has shape [N, num_bboxes, 5].
- det_labels (Tensor) – class labels of shape [N, num_bboxes].
Returns: - The segmentation results of shape [N, num_bboxes,
image_height, image_width].
Return type: Tensor
-
-
class
mmdet.models.roi_heads.
DoubleConvFCBBoxHead
(num_convs=0, num_fcs=0, conv_out_channels=1024, fc_out_channels=1024, conv_cfg=None, norm_cfg={'type': 'BN'}, init_cfg={'override': [{'type': 'Normal', 'name': 'fc_cls', 'std': 0.01}, {'type': 'Normal', 'name': 'fc_reg', 'std': 0.001}, {'type': 'Xavier', 'name': 'fc_branch', 'distribution': 'uniform'}], 'type': 'Normal'}, **kwargs)[source]¶ Bbox head used in Double-Head R-CNN
/-> cls /-> shared convs -> \-> reg roi features /-> cls \-> shared fc -> \-> reg
-
forward
(x_cls, x_reg)[source]¶ Defines the computation performed at every call.
Should be overridden by all subclasses.
Note
Although the recipe for forward pass needs to be defined within this function, one should call the
Module
instance afterwards instead of this since the former takes care of running the registered hooks while the latter silently ignores them.
-
-
class
mmdet.models.roi_heads.
FCNMaskHead
(num_convs=4, roi_feat_size=14, in_channels=256, conv_kernel_size=3, conv_out_channels=256, num_classes=80, class_agnostic=False, upsample_cfg={'scale_factor': 2, 'type': 'deconv'}, conv_cfg=None, norm_cfg=None, predictor_cfg={'type': 'Conv'}, loss_mask={'loss_weight': 1.0, 'type': 'CrossEntropyLoss', 'use_mask': True}, init_cfg=None)[source]¶ -
forward
(x)[source]¶ Defines the computation performed at every call.
Should be overridden by all subclasses.
Note
Although the recipe for forward pass needs to be defined within this function, one should call the
Module
instance afterwards instead of this since the former takes care of running the registered hooks while the latter silently ignores them.
-
get_seg_masks
(mask_pred, det_bboxes, det_labels, rcnn_test_cfg, ori_shape, scale_factor, rescale)[source]¶ Get segmentation masks from mask_pred and bboxes.
Parameters: - mask_pred (Tensor or ndarray) – shape (n, #class, h, w). For single-scale testing, mask_pred is the direct output of model, whose type is Tensor, while for multi-scale testing, it will be converted to numpy array outside of this method.
- det_bboxes (Tensor) – shape (n, 4/5)
- det_labels (Tensor) – shape (n, )
- rcnn_test_cfg (dict) – rcnn testing config
- ori_shape (Tuple) – original image height and width, shape (2,)
- scale_factor (ndarray | Tensor) – If
rescale is True
, box coordinates are divided by this scale factor to fitori_shape
. - rescale (bool) – If True, the resulting masks will be rescaled to
ori_shape
.
Returns: - encoded masks. The c-th item in the outer list
corresponds to the c-th class. Given the c-th outer list, the i-th item in that inner list is the mask for the i-th box with class label c.
Return type: list[list]
Example
>>> import mmcv >>> from mmdet.models.roi_heads.mask_heads.fcn_mask_head import * # NOQA >>> N = 7 # N = number of extracted ROIs >>> C, H, W = 11, 32, 32 >>> # Create example instance of FCN Mask Head. >>> self = FCNMaskHead(num_classes=C, num_convs=0) >>> inputs = torch.rand(N, self.in_channels, H, W) >>> mask_pred = self.forward(inputs) >>> # Each input is associated with some bounding box >>> det_bboxes = torch.Tensor([[1, 1, 42, 42 ]] * N) >>> det_labels = torch.randint(0, C, size=(N,)) >>> rcnn_test_cfg = mmcv.Config({'mask_thr_binary': 0, }) >>> ori_shape = (H * 4, W * 4) >>> scale_factor = torch.FloatTensor((1, 1)) >>> rescale = False >>> # Encoded masks are a list for each category. >>> encoded_masks = self.get_seg_masks( >>> mask_pred, det_bboxes, det_labels, rcnn_test_cfg, ori_shape, >>> scale_factor, rescale >>> ) >>> assert len(encoded_masks) == C >>> assert sum(list(map(len, encoded_masks))) == N
-
loss
(mask_pred, mask_targets, labels)[source]¶ Example
>>> from mmdet.models.roi_heads.mask_heads.fcn_mask_head import * # NOQA >>> N = 7 # N = number of extracted ROIs >>> C, H, W = 11, 32, 32 >>> # Create example instance of FCN Mask Head. >>> # There are lots of variations depending on the configuration >>> self = FCNMaskHead(num_classes=C, num_convs=1) >>> inputs = torch.rand(N, self.in_channels, H, W) >>> mask_pred = self.forward(inputs) >>> sf = self.scale_factor >>> labels = torch.randint(0, C, size=(N,)) >>> # With the default properties the mask targets should indicate >>> # a (potentially soft) single-class label >>> mask_targets = torch.rand(N, H * sf, W * sf) >>> loss = self.loss(mask_pred, mask_targets, labels) >>> print('loss = {!r}'.format(loss))
-
onnx_export
(mask_pred, det_bboxes, det_labels, rcnn_test_cfg, ori_shape, **kwargs)[source]¶ Get segmentation masks from mask_pred and bboxes.
Parameters: - mask_pred (Tensor) – shape (n, #class, h, w).
- det_bboxes (Tensor) – shape (n, 4/5)
- det_labels (Tensor) – shape (n, )
- rcnn_test_cfg (dict) – rcnn testing config
- ori_shape (Tuple) – original image height and width, shape (2,)
Returns: a mask of shape (N, img_h, img_w).
Return type: Tensor
-
-
class
mmdet.models.roi_heads.
HTCMaskHead
(with_conv_res=True, *args, **kwargs)[source]¶ -
forward
(x, res_feat=None, return_logits=True, return_feat=True)[source]¶ Defines the computation performed at every call.
Should be overridden by all subclasses.
Note
Although the recipe for forward pass needs to be defined within this function, one should call the
Module
instance afterwards instead of this since the former takes care of running the registered hooks while the latter silently ignores them.
-
-
class
mmdet.models.roi_heads.
FusedSemanticHead
(num_ins, fusion_level, num_convs=4, in_channels=256, conv_out_channels=256, num_classes=183, ignore_label=255, loss_weight=0.2, conv_cfg=None, norm_cfg=None, init_cfg={'override': {'name': 'conv_logits'}, 'type': 'Kaiming'})[source]¶ Multi-level fused semantic segmentation head.
in_1 -> 1x1 conv --- | in_2 -> 1x1 conv -- | || in_3 -> 1x1 conv - || ||| /-> 1x1 conv (mask prediction) in_4 -> 1x1 conv -----> 3x3 convs (*4) | \-> 1x1 conv (feature) in_5 -> 1x1 conv ---
-
forward
(feats)[source]¶ Defines the computation performed at every call.
Should be overridden by all subclasses.
Note
Although the recipe for forward pass needs to be defined within this function, one should call the
Module
instance afterwards instead of this since the former takes care of running the registered hooks while the latter silently ignores them.
-
-
class
mmdet.models.roi_heads.
GridHead
(grid_points=9, num_convs=8, roi_feat_size=14, in_channels=256, conv_kernel_size=3, point_feat_channels=64, deconv_kernel_size=4, class_agnostic=False, loss_grid={'loss_weight': 15, 'type': 'CrossEntropyLoss', 'use_sigmoid': True}, conv_cfg=None, norm_cfg={'num_groups': 36, 'type': 'GN'}, init_cfg=[{'type': 'Kaiming', 'layer': ['Conv2d', 'Linear']}, {'type': 'Normal', 'layer': 'ConvTranspose2d', 'std': 0.001, 'override': {'type': 'Normal', 'name': 'deconv2', 'std': 0.001, 'bias': -4.59511985013459}}])[source]¶ -
calc_sub_regions
()[source]¶ Compute point specific representation regions.
See Grid R-CNN Plus (https://arxiv.org/abs/1906.05688) for details.
-
forward
(x)[source]¶ Defines the computation performed at every call.
Should be overridden by all subclasses.
Note
Although the recipe for forward pass needs to be defined within this function, one should call the
Module
instance afterwards instead of this since the former takes care of running the registered hooks while the latter silently ignores them.
-
-
class
mmdet.models.roi_heads.
MaskIoUHead
(num_convs=4, num_fcs=2, roi_feat_size=14, in_channels=256, conv_out_channels=256, fc_out_channels=1024, num_classes=80, loss_iou={'loss_weight': 0.5, 'type': 'MSELoss'}, init_cfg=[{'type': 'Kaiming', 'override': {'name': 'convs'}}, {'type': 'Caffe2Xavier', 'override': {'name': 'fcs'}}, {'type': 'Normal', 'std': 0.01, 'override': {'name': 'fc_mask_iou'}}])[source]¶ Mask IoU Head.
This head predicts the IoU of predicted masks and corresponding gt masks.
-
forward
(mask_feat, mask_pred)[source]¶ Defines the computation performed at every call.
Should be overridden by all subclasses.
Note
Although the recipe for forward pass needs to be defined within this function, one should call the
Module
instance afterwards instead of this since the former takes care of running the registered hooks while the latter silently ignores them.
-
get_mask_scores
(mask_iou_pred, det_bboxes, det_labels)[source]¶ Get the mask scores.
mask_score = bbox_score * mask_iou
-
get_targets
(sampling_results, gt_masks, mask_pred, mask_targets, rcnn_train_cfg)[source]¶ Compute target of mask IoU.
Mask IoU target is the IoU of the predicted mask (inside a bbox) and the gt mask of corresponding gt mask (the whole instance). The intersection area is computed inside the bbox, and the gt mask area is computed with two steps, firstly we compute the gt area inside the bbox, then divide it by the area ratio of gt area inside the bbox and the gt area of the whole instance.
Parameters: - sampling_results (list[
SamplingResult
]) – sampling results. - gt_masks (BitmapMask | PolygonMask) – Gt masks (the whole instance) of each image, with the same shape of the input image.
- mask_pred (Tensor) – Predicted masks of each positive proposal, shape (num_pos, h, w).
- mask_targets (Tensor) – Gt mask of each positive proposal, binary map of the shape (num_pos, h, w).
- rcnn_train_cfg (dict) – Training config for R-CNN part.
Returns: mask iou target (length == num positive).
Return type: Tensor
- sampling_results (list[
-
-
class
mmdet.models.roi_heads.
BaseRoIExtractor
(roi_layer, out_channels, featmap_strides, init_cfg=None)[source]¶ Base class for RoI extractor.
Parameters: - roi_layer (dict) – Specify RoI layer type and arguments.
- out_channels (int) – Output channels of RoI layers.
- featmap_strides (int) – Strides of input feature maps.
- init_cfg (dict or list[dict], optional) – Initialization config dict. Default: None
-
build_roi_layers
(layer_cfg, featmap_strides)[source]¶ Build RoI operator to extract feature from each level feature map.
Parameters: - layer_cfg (dict) – Dictionary to construct and config RoI layer
operation. Options are modules under
mmcv/ops
such asRoIAlign
. - featmap_strides (List[int]) – The stride of input feature map w.r.t to the original image size, which would be used to scale RoI coordinate (original image coordinate system) to feature coordinate system.
Returns: - The RoI extractor modules for each level feature
map.
Return type: nn.ModuleList
- layer_cfg (dict) – Dictionary to construct and config RoI layer
operation. Options are modules under
-
forward
(feats, rois, roi_scale_factor=None)[source]¶ Defines the computation performed at every call.
Should be overridden by all subclasses.
Note
Although the recipe for forward pass needs to be defined within this function, one should call the
Module
instance afterwards instead of this since the former takes care of running the registered hooks while the latter silently ignores them.
-
num_inputs
¶ Number of input feature maps.
Type: int
-
class
mmdet.models.roi_heads.
GenericRoIExtractor
(aggregation='sum', pre_cfg=None, post_cfg=None, **kwargs)[source]¶ Extract RoI features from all level feature maps levels.
This is the implementation of A novel Region of Interest Extraction Layer for Instance Segmentation.
Parameters: - aggregation (str) – The method to aggregate multiple feature maps. Options are ‘sum’, ‘concat’. Default: ‘sum’.
- pre_cfg (dict | None) – Specify pre-processing modules. Default: None.
- post_cfg (dict | None) – Specify post-processing modules. Default: None.
- kwargs (keyword arguments) – Arguments that are the same
as
BaseRoIExtractor
.
-
class
mmdet.models.roi_heads.
SingleRoIExtractor
(roi_layer, out_channels, featmap_strides, finest_scale=56, init_cfg=None)[source]¶ Extract RoI features from a single level feature map.
If there are multiple input feature levels, each RoI is mapped to a level according to its scale. The mapping rule is proposed in FPN.
Parameters: - roi_layer (dict) – Specify RoI layer type and arguments.
- out_channels (int) – Output channels of RoI layers.
- featmap_strides (List[int]) – Strides of input feature maps.
- finest_scale (int) – Scale threshold of mapping to level 0. Default: 56.
- init_cfg (dict or list[dict], optional) – Initialization config dict. Default: None
-
map_roi_levels
(rois, num_levels)[source]¶ Map rois to corresponding feature levels by scales.
- scale < finest_scale * 2: level 0
- finest_scale * 2 <= scale < finest_scale * 4: level 1
- finest_scale * 4 <= scale < finest_scale * 8: level 2
- scale >= finest_scale * 8: level 3
Parameters: - rois (Tensor) – Input RoIs, shape (k, 5).
- num_levels (int) – Total level number.
Returns: Level index (0-based) of each RoI, shape (k, )
Return type: Tensor
-
class
mmdet.models.roi_heads.
PISARoIHead
(bbox_roi_extractor=None, bbox_head=None, mask_roi_extractor=None, mask_head=None, shared_head=None, train_cfg=None, test_cfg=None, pretrained=None, init_cfg=None)[source]¶ The RoI head for Prime Sample Attention in Object Detection.
-
forward_train
(x, img_metas, proposal_list, gt_bboxes, gt_labels, gt_bboxes_ignore=None, gt_masks=None)[source]¶ Forward function for training.
Parameters: - x (list[Tensor]) – List of multi-level img features.
- img_metas (list[dict]) – List of image info dict where each dict has: ‘img_shape’, ‘scale_factor’, ‘flip’, and may also contain ‘filename’, ‘ori_shape’, ‘pad_shape’, and ‘img_norm_cfg’. For details on the values of these keys see mmdet/datasets/pipelines/formatting.py:Collect.
- proposals (list[Tensors]) – List of region proposals.
- gt_bboxes (list[Tensor]) – Each item are the truth boxes for each image in [tl_x, tl_y, br_x, br_y] format.
- gt_labels (list[Tensor]) – Class indices corresponding to each box
- gt_bboxes_ignore (list[Tensor], optional) – Specify which bounding boxes can be ignored when computing the loss.
- gt_masks (None | Tensor) – True segmentation masks for each box used if the architecture supports a segmentation task.
Returns: a dictionary of loss components
Return type: dict[str, Tensor]
-
-
class
mmdet.models.roi_heads.
PointRendRoIHead
(point_head, *args, **kwargs)[source]¶ -
-
aug_test_mask
(feats, img_metas, det_bboxes, det_labels)[source]¶ Test for mask head with test time augmentation.
-
mask_onnx_export
(x, img_metas, det_bboxes, det_labels, **kwargs)[source]¶ Export mask branch to onnx which supports batch inference.
Parameters: - x (tuple[Tensor]) – Feature maps of all scale level.
- img_metas (list[dict]) – Image meta info.
- det_bboxes (Tensor) – Bboxes and corresponding scores. has shape [N, num_bboxes, 5].
- det_labels (Tensor) – class labels of shape [N, num_bboxes].
Returns: - The segmentation results of shape [N, num_bboxes,
image_height, image_width].
Return type: Tensor
-
-
class
mmdet.models.roi_heads.
MaskPointHead
(num_classes, num_fcs=3, in_channels=256, fc_channels=256, class_agnostic=False, coarse_pred_each_layer=True, conv_cfg={'type': 'Conv1d'}, norm_cfg=None, act_cfg={'type': 'ReLU'}, loss_point={'loss_weight': 1.0, 'type': 'CrossEntropyLoss', 'use_mask': True}, init_cfg={'override': {'name': 'fc_logits'}, 'std': 0.001, 'type': 'Normal'})[source]¶ A mask point head use in PointRend.
MaskPointHead
use shared multi-layer perceptron (equivalent to nn.Conv1d) to predict the logit of input points. The fine-grained feature and coarse feature will be concatenate together for predication.Parameters: - num_fcs (int) – Number of fc layers in the head. Default: 3.
- in_channels (int) – Number of input channels. Default: 256.
- fc_channels (int) – Number of fc channels. Default: 256.
- num_classes (int) – Number of classes for logits. Default: 80.
- class_agnostic (bool) – Whether use class agnostic classification. If so, the output channels of logits will be 1. Default: False.
- coarse_pred_each_layer (bool) – Whether concatenate coarse feature with the output of each fc layer. Default: True.
- conv_cfg (dict | None) – Dictionary to construct and config conv layer. Default: dict(type=’Conv1d’))
- norm_cfg (dict | None) – Dictionary to construct and config norm layer. Default: None.
- loss_point (dict) – Dictionary to construct and config loss layer of point head. Default: dict(type=’CrossEntropyLoss’, use_mask=True, loss_weight=1.0).
- init_cfg (dict or list[dict], optional) – Initialization config dict.
-
forward
(fine_grained_feats, coarse_feats)[source]¶ Classify each point base on fine grained and coarse feats.
Parameters: - fine_grained_feats (Tensor) – Fine grained feature sampled from FPN, shape (num_rois, in_channels, num_points).
- coarse_feats (Tensor) – Coarse feature sampled from CoarseMaskHead, shape (num_rois, num_classes, num_points).
Returns: - Point classification results,
shape (num_rois, num_class, num_points).
Return type: Tensor
-
get_roi_rel_points_test
(mask_pred, pred_label, cfg)[source]¶ Get
num_points
most uncertain points during test.Parameters: - mask_pred (Tensor) – A tensor of shape (num_rois, num_classes, mask_height, mask_width) for class-specific or class-agnostic prediction.
- pred_label (list) – The predication class for each instance.
- cfg (dict) – Testing config of point head.
Returns: - A tensor of shape (num_rois, num_points)
that contains indices from [0, mask_height x mask_width) of the most uncertain points.
- point_coords (Tensor): A tensor of shape (num_rois, num_points, 2)
that contains [0, 1] x [0, 1] normalized coordinates of the most uncertain points from the [mask_height, mask_width] grid .
Return type: point_indices (Tensor)
-
get_roi_rel_points_train
(mask_pred, labels, cfg)[source]¶ Get
num_points
most uncertain points with random points during train.Sample points in [0, 1] x [0, 1] coordinate space based on their uncertainty. The uncertainties are calculated for each point using ‘_get_uncertainty()’ function that takes point’s logit prediction as input.
Parameters: - mask_pred (Tensor) – A tensor of shape (num_rois, num_classes, mask_height, mask_width) for class-specific or class-agnostic prediction.
- labels (list) – The ground truth class for each instance.
- cfg (dict) – Training config of point head.
Returns: - A tensor of shape (num_rois, num_points, 2)
that contains the coordinates sampled points.
Return type: point_coords (Tensor)
-
get_targets
(rois, rel_roi_points, sampling_results, gt_masks, cfg)[source]¶ Get training targets of MaskPointHead for all images.
Parameters: - rois (Tensor) – Region of Interest, shape (num_rois, 5).
- rel_roi_points – Points coordinates relative to RoI, shape (num_rois, num_points, 2).
- sampling_results (
SamplingResult
) – Sampling result after sampling and assignment. - gt_masks (Tensor) – Ground truth segmentation masks of corresponding boxes, shape (num_rois, height, width).
- cfg (dict) – Training cfg.
Returns: Point target, shape (num_rois, num_points).
Return type: Tensor
-
loss
(point_pred, point_targets, labels)[source]¶ Calculate loss for MaskPointHead.
Parameters: - point_pred (Tensor) – Point predication result, shape (num_rois, num_classes, num_points).
- point_targets (Tensor) – Point targets, shape (num_roi, num_points).
- labels (Tensor) – Class label of corresponding boxes, shape (num_rois, )
Returns: a dictionary of point loss components
Return type: dict[str, Tensor]
-
class
mmdet.models.roi_heads.
CoarseMaskHead
(num_convs=0, num_fcs=2, fc_out_channels=1024, downsample_factor=2, init_cfg={'override': [{'name': 'fcs'}, {'type': 'Constant', 'val': 0.001, 'name': 'fc_logits'}], 'type': 'Xavier'}, *arg, **kwarg)[source]¶ Coarse mask head used in PointRend.
Compared with standard
FCNMaskHead
,CoarseMaskHead
will downsample the input feature map instead of upsample it.Parameters: - num_convs (int) – Number of conv layers in the head. Default: 0.
- num_fcs (int) – Number of fc layers in the head. Default: 2.
- fc_out_channels (int) – Number of output channels of fc layer. Default: 1024.
- downsample_factor (int) – The factor that feature map is downsampled by. Default: 2.
- init_cfg (dict or list[dict], optional) – Initialization config dict.
-
forward
(x)[source]¶ Defines the computation performed at every call.
Should be overridden by all subclasses.
Note
Although the recipe for forward pass needs to be defined within this function, one should call the
Module
instance afterwards instead of this since the former takes care of running the registered hooks while the latter silently ignores them.
-
class
mmdet.models.roi_heads.
DynamicRoIHead
(**kwargs)[source]¶ RoI head for Dynamic R-CNN.
-
forward_train
(x, img_metas, proposal_list, gt_bboxes, gt_labels, gt_bboxes_ignore=None, gt_masks=None)[source]¶ Forward function for training.
Parameters: - x (list[Tensor]) – list of multi-level img features.
- img_metas (list[dict]) – list of image info dict where each dict has: ‘img_shape’, ‘scale_factor’, ‘flip’, and may also contain ‘filename’, ‘ori_shape’, ‘pad_shape’, and ‘img_norm_cfg’. For details on the values of these keys see mmdet/datasets/pipelines/formatting.py:Collect.
- proposals (list[Tensors]) – list of region proposals.
- gt_bboxes (list[Tensor]) – each item are the truth boxes for each image in [tl_x, tl_y, br_x, br_y] format.
- gt_labels (list[Tensor]) – class indices corresponding to each box
- gt_bboxes_ignore (None | list[Tensor]) – specify which bounding boxes can be ignored when computing the loss.
- gt_masks (None | Tensor) – true segmentation masks for each box used if the architecture supports a segmentation task.
Returns: a dictionary of loss components
Return type: dict[str, Tensor]
-
-
class
mmdet.models.roi_heads.
SparseRoIHead
(num_stages=6, stage_loss_weights=(1, 1, 1, 1, 1, 1), proposal_feature_channel=256, bbox_roi_extractor={'featmap_strides': [4, 8, 16, 32], 'out_channels': 256, 'roi_layer': {'output_size': 7, 'sampling_ratio': 2, 'type': 'RoIAlign'}, 'type': 'SingleRoIExtractor'}, bbox_head={'dropout': 0.0, 'feedforward_channels': 2048, 'ffn_act_cfg': {'inplace': True, 'type': 'ReLU'}, 'hidden_channels': 256, 'num_classes': 80, 'num_cls_fcs': 1, 'num_fcs': 2, 'num_heads': 8, 'num_reg_fcs': 3, 'roi_feat_size': 7, 'type': 'DIIHead'}, train_cfg=None, test_cfg=None, pretrained=None, init_cfg=None)[source]¶ The RoIHead for Sparse R-CNN: End-to-End Object Detection with Learnable Proposals
Parameters: - num_stages (int) – Number of stage whole iterative process. Defaults to 6.
- stage_loss_weights (Tuple[float]) – The loss weight of each stage. By default all stages have the same weight 1.
- bbox_roi_extractor (dict) – Config of box roi extractor.
- bbox_head (dict) – Config of box head.
- train_cfg (dict, optional) – Configuration information in train stage. Defaults to None.
- test_cfg (dict, optional) – Configuration information in test stage. Defaults to None.
- pretrained (str, optional) – model pretrained path. Default: None
- init_cfg (dict or list[dict], optional) – Initialization config dict. Default: None
-
aug_test
(features, proposal_list, img_metas, rescale=False)[source]¶ Test with augmentations.
If rescale is False, then returned bboxes and masks will fit the scale of imgs[0].
-
forward_dummy
(x, proposal_boxes, proposal_features, img_metas)[source]¶ Dummy forward function when do the flops computing.
-
forward_train
(x, proposal_boxes, proposal_features, img_metas, gt_bboxes, gt_labels, gt_bboxes_ignore=None, imgs_whwh=None, gt_masks=None)[source]¶ Forward function in training stage.
Parameters: - x (list[Tensor]) – list of multi-level img features.
- proposals (Tensor) – Decoded proposal bboxes, has shape (batch_size, num_proposals, 4)
- proposal_features (Tensor) – Expanded proposal features, has shape (batch_size, num_proposals, proposal_feature_channel)
- img_metas (list[dict]) – list of image info dict where each dict has: ‘img_shape’, ‘scale_factor’, ‘flip’, and may also contain ‘filename’, ‘ori_shape’, ‘pad_shape’, and ‘img_norm_cfg’. For details on the values of these keys see mmdet/datasets/pipelines/formatting.py:Collect.
- gt_bboxes (list[Tensor]) – Ground truth bboxes for each image with shape (num_gts, 4) in [tl_x, tl_y, br_x, br_y] format.
- gt_labels (list[Tensor]) – class indices corresponding to each box
- gt_bboxes_ignore (None | list[Tensor]) – specify which bounding boxes can be ignored when computing the loss.
- imgs_whwh (Tensor) – Tensor with shape (batch_size, 4), the dimension means [img_width,img_height, img_width, img_height].
- gt_masks (None | Tensor) – true segmentation masks for each box used if the architecture supports a segmentation task.
Returns: a dictionary of loss components of all stage.
Return type: dict[str, Tensor]
-
simple_test
(x, proposal_boxes, proposal_features, img_metas, imgs_whwh, rescale=False)[source]¶ Test without augmentation.
Parameters: - x (list[Tensor]) – list of multi-level img features.
- proposal_boxes (Tensor) – Decoded proposal bboxes, has shape (batch_size, num_proposals, 4)
- proposal_features (Tensor) – Expanded proposal features, has shape (batch_size, num_proposals, proposal_feature_channel)
- img_metas (dict) – meta information of images.
- imgs_whwh (Tensor) – Tensor with shape (batch_size, 4), the dimension means [img_width,img_height, img_width, img_height].
- rescale (bool) – If True, return boxes in original image space. Defaults to False.
Returns: [[cls1_det, cls2_det, …], …]. The outer list indicates images, and the inner list indicates per-class detected bboxes. The np.ndarray has shape (num_det, 5) and the last dimension 5 represents (x1, y1, x2, y2, score).
Return type: bbox_results (list[tuple[np.ndarray]])
-
class
mmdet.models.roi_heads.
TridentRoIHead
(num_branch, test_branch_idx, **kwargs)[source]¶ Trident roi head.
Parameters: - num_branch (int) – Number of branches in TridentNet.
- test_branch_idx (int) – In inference, all 3 branches will be used if test_branch_idx==-1, otherwise only branch with index test_branch_idx will be used.
-
aug_test_bboxes
(feats, img_metas, proposal_list, rcnn_test_cfg)[source]¶ Test det bboxes with test time augmentation.
-
class
mmdet.models.roi_heads.
SCNetRoIHead
(num_stages, stage_loss_weights, semantic_roi_extractor=None, semantic_head=None, feat_relay_head=None, glbctx_head=None, **kwargs)[source]¶ RoIHead for SCNet.
Parameters: - num_stages (int) – number of cascade stages.
- stage_loss_weights (list) – loss weight of cascade stages.
- semantic_roi_extractor (dict) – config to init semantic roi extractor.
- semantic_head (dict) – config to init semantic head.
- feat_relay_head (dict) – config to init feature_relay_head.
- glbctx_head (dict) – config to init global context head.
-
aug_test
(img_feats, proposal_list, img_metas, rescale=False)[source]¶ Test with augmentations.
If rescale is False, then returned bboxes and masks will fit the scale of imgs[0].
-
forward_train
(x, img_metas, proposal_list, gt_bboxes, gt_labels, gt_bboxes_ignore=None, gt_masks=None, gt_semantic_seg=None)[source]¶ Parameters: - x (list[Tensor]) – list of multi-level img features.
- img_metas (list[dict]) – list of image info dict where each dict has: ‘img_shape’, ‘scale_factor’, ‘flip’, and may also contain ‘filename’, ‘ori_shape’, ‘pad_shape’, and ‘img_norm_cfg’. For details on the values of these keys see mmdet/datasets/pipelines/formatting.py:Collect.
- proposal_list (list[Tensors]) – list of region proposals.
- gt_bboxes (list[Tensor]) – Ground truth bboxes for each image with shape (num_gts, 4) in [tl_x, tl_y, br_x, br_y] format.
- gt_labels (list[Tensor]) – class indices corresponding to each box
- gt_bboxes_ignore (None, list[Tensor]) – specify which bounding boxes can be ignored when computing the loss.
- gt_masks (None, Tensor) – true segmentation masks for each box used if the architecture supports a segmentation task.
- gt_semantic_seg (None, list[Tensor]) – semantic segmentation masks used if the architecture supports semantic segmentation task.
Returns: a dictionary of loss components
Return type: dict[str, Tensor]
-
with_feat_relay
¶ whether the head has feature relay head
Type: bool
-
with_glbctx
¶ whether the head has global context head
Type: bool
-
with_semantic
¶ whether the head has semantic head
Type: bool
-
class
mmdet.models.roi_heads.
SCNetMaskHead
(conv_to_res=True, **kwargs)[source]¶ Mask head for SCNet.
Parameters: conv_to_res (bool, optional) – if True, change the conv layers to SimplifiedBasicBlock
.
-
class
mmdet.models.roi_heads.
SCNetSemanticHead
(conv_to_res=True, **kwargs)[source]¶ Mask head for SCNet.
Parameters: conv_to_res (bool, optional) – if True, change the conv layers to SimplifiedBasicBlock
.
-
class
mmdet.models.roi_heads.
SCNetBBoxHead
(num_shared_convs=0, num_shared_fcs=0, num_cls_convs=0, num_cls_fcs=0, num_reg_convs=0, num_reg_fcs=0, conv_out_channels=256, fc_out_channels=1024, conv_cfg=None, norm_cfg=None, init_cfg=None, *args, **kwargs)[source]¶ BBox head for SCNet.
This inherits
ConvFCBBoxHead
with modified forward() function, allow us to get intermediate shared feature.-
forward
(x, return_shared_feat=False)[source]¶ Forward function.
Parameters: - x (Tensor) – input features
- return_shared_feat (bool) – If True, return cls-reg-shared feature.
Returns: - contain
cls_score
andbbox_pred
, if
return_shared_feat
is True, appendx_shared
to the returned tuple.
Return type: out (tuple[Tensor])
-
-
class
mmdet.models.roi_heads.
FeatureRelayHead
(in_channels=1024, out_conv_channels=256, roi_feat_size=7, scale_factor=2, init_cfg={'layer': 'Linear', 'type': 'Kaiming'})[source]¶ Feature Relay Head used in SCNet.
Parameters: - in_channels (int, optional) – number of input channels. Default: 256.
- conv_out_channels (int, optional) – number of output channels before classification layer. Default: 256.
- roi_feat_size (int, optional) – roi feat size at box head. Default: 7.
- scale_factor (int, optional) – scale factor to match roi feat size at mask head. Default: 2.
- init_cfg (dict or list[dict], optional) – Initialization config dict.
-
class
mmdet.models.roi_heads.
GlobalContextHead
(num_convs=4, in_channels=256, conv_out_channels=256, num_classes=80, loss_weight=1.0, conv_cfg=None, norm_cfg=None, conv_to_res=False, init_cfg={'override': {'name': 'fc'}, 'std': 0.01, 'type': 'Normal'})[source]¶ Global context head used in SCNet.
Parameters: - num_convs (int, optional) – number of convolutional layer in GlbCtxHead. Default: 4.
- in_channels (int, optional) – number of input channels. Default: 256.
- conv_out_channels (int, optional) – number of output channels before classification layer. Default: 256.
- num_classes (int, optional) – number of classes. Default: 80.
- loss_weight (float, optional) – global context loss weight. Default: 1.
- conv_cfg (dict, optional) – config to init conv layer. Default: None.
- norm_cfg (dict, optional) – config to init norm layer. Default: None.
- conv_to_res (bool, optional) – if True, 2 convs will be grouped into 1 SimplifiedBasicBlock using a skip connection. Default: False.
- init_cfg (dict or list[dict], optional) – Initialization config dict.
losses¶
-
mmdet.models.losses.
cross_entropy
(pred, label, weight=None, reduction='mean', avg_factor=None, class_weight=None, ignore_index=-100)[source]¶ Calculate the CrossEntropy loss.
Parameters: - pred (torch.Tensor) – The prediction with shape (N, C), C is the number of classes.
- label (torch.Tensor) – The learning label of the prediction.
- weight (torch.Tensor, optional) – Sample-wise loss weight.
- reduction (str, optional) – The method used to reduce the loss.
- avg_factor (int, optional) – Average factor that is used to average the loss. Defaults to None.
- class_weight (list[float], optional) – The weight for each class.
- ignore_index (int | None) – The label index to be ignored. If None, it will be set to default value. Default: -100.
Returns: The calculated loss
Return type: torch.Tensor
-
mmdet.models.losses.
binary_cross_entropy
(pred, label, weight=None, reduction='mean', avg_factor=None, class_weight=None, ignore_index=-100)[source]¶ Calculate the binary CrossEntropy loss.
Parameters: - pred (torch.Tensor) – The prediction with shape (N, 1).
- label (torch.Tensor) – The learning label of the prediction.
- weight (torch.Tensor, optional) – Sample-wise loss weight.
- reduction (str, optional) – The method used to reduce the loss. Options are “none”, “mean” and “sum”.
- avg_factor (int, optional) – Average factor that is used to average the loss. Defaults to None.
- class_weight (list[float], optional) – The weight for each class.
- ignore_index (int | None) – The label index to be ignored. If None, it will be set to default value. Default: -100.
Returns: The calculated loss.
Return type: torch.Tensor
-
mmdet.models.losses.
mask_cross_entropy
(pred, target, label, reduction='mean', avg_factor=None, class_weight=None, ignore_index=None)[source]¶ Calculate the CrossEntropy loss for masks.
Parameters: - pred (torch.Tensor) – The prediction with shape (N, C, *), C is the number of classes. The trailing * indicates arbitrary shape.
- target (torch.Tensor) – The learning label of the prediction.
- label (torch.Tensor) –
label
indicates the class label of the mask corresponding object. This will be used to select the mask in the of the class which the object belongs to when the mask prediction if not class-agnostic. - reduction (str, optional) – The method used to reduce the loss. Options are “none”, “mean” and “sum”.
- avg_factor (int, optional) – Average factor that is used to average the loss. Defaults to None.
- class_weight (list[float], optional) – The weight for each class.
- ignore_index (None) – Placeholder, to be consistent with other loss. Default: None.
Returns: The calculated loss
Return type: torch.Tensor
Example
>>> N, C = 3, 11 >>> H, W = 2, 2 >>> pred = torch.randn(N, C, H, W) * 1000 >>> target = torch.rand(N, H, W) >>> label = torch.randint(0, C, size=(N,)) >>> reduction = 'mean' >>> avg_factor = None >>> class_weights = None >>> loss = mask_cross_entropy(pred, target, label, reduction, >>> avg_factor, class_weights) >>> assert loss.shape == (1,)
-
class
mmdet.models.losses.
CrossEntropyLoss
(use_sigmoid=False, use_mask=False, reduction='mean', class_weight=None, ignore_index=None, loss_weight=1.0)[source]¶ -
forward
(cls_score, label, weight=None, avg_factor=None, reduction_override=None, ignore_index=None, **kwargs)[source]¶ Forward function.
Parameters: - cls_score (torch.Tensor) – The prediction.
- label (torch.Tensor) – The learning label of the prediction.
- weight (torch.Tensor, optional) – Sample-wise loss weight.
- avg_factor (int, optional) – Average factor that is used to average the loss. Defaults to None.
- reduction_override (str, optional) – The method used to reduce the loss. Options are “none”, “mean” and “sum”.
- ignore_index (int | None) – The label index to be ignored. If not None, it will override the default value. Default: None.
Returns: The calculated loss.
Return type: torch.Tensor
-
-
mmdet.models.losses.
sigmoid_focal_loss
(pred, target, weight=None, gamma=2.0, alpha=0.25, reduction='mean', avg_factor=None)[source]¶ A warpper of cuda version Focal Loss.
Parameters: - pred (torch.Tensor) – The prediction with shape (N, C), C is the number of classes.
- target (torch.Tensor) – The learning label of the prediction.
- weight (torch.Tensor, optional) – Sample-wise loss weight.
- gamma (float, optional) – The gamma for calculating the modulating factor. Defaults to 2.0.
- alpha (float, optional) – A balanced form for Focal Loss. Defaults to 0.25.
- reduction (str, optional) – The method used to reduce the loss into a scalar. Defaults to ‘mean’. Options are “none”, “mean” and “sum”.
- avg_factor (int, optional) – Average factor that is used to average the loss. Defaults to None.
-
class
mmdet.models.losses.
FocalLoss
(use_sigmoid=True, gamma=2.0, alpha=0.25, reduction='mean', loss_weight=1.0)[source]¶ -
forward
(pred, target, weight=None, avg_factor=None, reduction_override=None)[source]¶ Forward function.
Parameters: - pred (torch.Tensor) – The prediction.
- target (torch.Tensor) – The learning label of the prediction.
- weight (torch.Tensor, optional) – The weight of loss for each prediction. Defaults to None.
- avg_factor (int, optional) – Average factor that is used to average the loss. Defaults to None.
- reduction_override (str, optional) – The reduction method used to override the original reduction method of the loss. Options are “none”, “mean” and “sum”.
Returns: The calculated loss
Return type: torch.Tensor
-
-
class
mmdet.models.losses.
SmoothL1Loss
(beta=1.0, reduction='mean', loss_weight=1.0)[source]¶ Smooth L1 loss.
Parameters: - beta (float, optional) – The threshold in the piecewise function. Defaults to 1.0.
- reduction (str, optional) – The method to reduce the loss. Options are “none”, “mean” and “sum”. Defaults to “mean”.
- loss_weight (float, optional) – The weight of loss.
-
forward
(pred, target, weight=None, avg_factor=None, reduction_override=None, **kwargs)[source]¶ Forward function.
Parameters: - pred (torch.Tensor) – The prediction.
- target (torch.Tensor) – The learning target of the prediction.
- weight (torch.Tensor, optional) – The weight of loss for each prediction. Defaults to None.
- avg_factor (int, optional) – Average factor that is used to average the loss. Defaults to None.
- reduction_override (str, optional) – The reduction method used to override the original reduction method of the loss. Defaults to None.
-
class
mmdet.models.losses.
BalancedL1Loss
(alpha=0.5, gamma=1.5, beta=1.0, reduction='mean', loss_weight=1.0)[source]¶ Balanced L1 Loss.
arXiv: https://arxiv.org/pdf/1904.02701.pdf (CVPR 2019)
Parameters: - alpha (float) – The denominator
alpha
in the balanced L1 loss. Defaults to 0.5. - gamma (float) – The
gamma
in the balanced L1 loss. Defaults to 1.5. - beta (float, optional) – The loss is a piecewise function of prediction
and target.
beta
serves as a threshold for the difference between the prediction and target. Defaults to 1.0. - reduction (str, optional) – The method that reduces the loss to a scalar. Options are “none”, “mean” and “sum”.
- loss_weight (float, optional) – The weight of the loss. Defaults to 1.0
-
forward
(pred, target, weight=None, avg_factor=None, reduction_override=None, **kwargs)[source]¶ Forward function of loss.
Parameters: - pred (torch.Tensor) – The prediction with shape (N, 4).
- target (torch.Tensor) – The learning target of the prediction with shape (N, 4).
- weight (torch.Tensor, optional) – Sample-wise loss weight with shape (N, ).
- avg_factor (int, optional) – Average factor that is used to average the loss. Defaults to None.
- reduction_override (str, optional) – The reduction method used to override the original reduction method of the loss. Options are “none”, “mean” and “sum”.
Returns: The calculated loss
Return type: torch.Tensor
- alpha (float) – The denominator
-
class
mmdet.models.losses.
MSELoss
(reduction='mean', loss_weight=1.0)[source]¶ MSELoss.
Parameters: - reduction (str, optional) – The method that reduces the loss to a scalar. Options are “none”, “mean” and “sum”.
- loss_weight (float, optional) – The weight of the loss. Defaults to 1.0
-
forward
(pred, target, weight=None, avg_factor=None, reduction_override=None)[source]¶ Forward function of loss.
Parameters: - pred (torch.Tensor) – The prediction.
- target (torch.Tensor) – The learning target of the prediction.
- weight (torch.Tensor, optional) – Weight of the loss for each prediction. Defaults to None.
- avg_factor (int, optional) – Average factor that is used to average the loss. Defaults to None.
- reduction_override (str, optional) – The reduction method used to override the original reduction method of the loss. Defaults to None.
Returns: The calculated loss
Return type: torch.Tensor
-
class
mmdet.models.losses.
IoULoss
(linear=False, eps=1e-06, reduction='mean', loss_weight=1.0, mode='log')[source]¶ IoULoss.
Computing the IoU loss between a set of predicted bboxes and target bboxes.
Parameters: - linear (bool) – If True, use linear scale of loss else determined by mode. Default: False.
- eps (float) – Eps to avoid log(0).
- reduction (str) – Options are “none”, “mean” and “sum”.
- loss_weight (float) – Weight of loss.
- mode (str) – Loss scaling mode, including “linear”, “square”, and “log”. Default: ‘log’
-
forward
(pred, target, weight=None, avg_factor=None, reduction_override=None, **kwargs)[source]¶ Forward function.
Parameters: - pred (torch.Tensor) – The prediction.
- target (torch.Tensor) – The learning target of the prediction.
- weight (torch.Tensor, optional) – The weight of loss for each prediction. Defaults to None.
- avg_factor (int, optional) – Average factor that is used to average the loss. Defaults to None.
- reduction_override (str, optional) – The reduction method used to override the original reduction method of the loss. Defaults to None. Options are “none”, “mean” and “sum”.
-
class
mmdet.models.losses.
BoundedIoULoss
(beta=0.2, eps=0.001, reduction='mean', loss_weight=1.0)[source]¶ -
forward
(pred, target, weight=None, avg_factor=None, reduction_override=None, **kwargs)[source]¶ Defines the computation performed at every call.
Should be overridden by all subclasses.
Note
Although the recipe for forward pass needs to be defined within this function, one should call the
Module
instance afterwards instead of this since the former takes care of running the registered hooks while the latter silently ignores them.
-
-
class
mmdet.models.losses.
GIoULoss
(eps=1e-06, reduction='mean', loss_weight=1.0)[source]¶ -
forward
(pred, target, weight=None, avg_factor=None, reduction_override=None, **kwargs)[source]¶ Defines the computation performed at every call.
Should be overridden by all subclasses.
Note
Although the recipe for forward pass needs to be defined within this function, one should call the
Module
instance afterwards instead of this since the former takes care of running the registered hooks while the latter silently ignores them.
-
-
class
mmdet.models.losses.
DIoULoss
(eps=1e-06, reduction='mean', loss_weight=1.0)[source]¶ -
forward
(pred, target, weight=None, avg_factor=None, reduction_override=None, **kwargs)[source]¶ Defines the computation performed at every call.
Should be overridden by all subclasses.
Note
Although the recipe for forward pass needs to be defined within this function, one should call the
Module
instance afterwards instead of this since the former takes care of running the registered hooks while the latter silently ignores them.
-
-
class
mmdet.models.losses.
CIoULoss
(eps=1e-06, reduction='mean', loss_weight=1.0)[source]¶ -
forward
(pred, target, weight=None, avg_factor=None, reduction_override=None, **kwargs)[source]¶ Defines the computation performed at every call.
Should be overridden by all subclasses.
Note
Although the recipe for forward pass needs to be defined within this function, one should call the
Module
instance afterwards instead of this since the former takes care of running the registered hooks while the latter silently ignores them.
-
-
class
mmdet.models.losses.
GHMC
(bins=10, momentum=0, use_sigmoid=True, loss_weight=1.0, reduction='mean')[source]¶ GHM Classification Loss.
Details of the theorem can be viewed in the paper Gradient Harmonized Single-stage Detector.
Parameters: - bins (int) – Number of the unit regions for distribution calculation.
- momentum (float) – The parameter for moving average.
- use_sigmoid (bool) – Can only be true for BCE based loss now.
- loss_weight (float) – The weight of the total GHM-C loss.
- reduction (str) – Options are “none”, “mean” and “sum”. Defaults to “mean”
-
forward
(pred, target, label_weight, reduction_override=None, **kwargs)[source]¶ Calculate the GHM-C loss.
Parameters: - pred (float tensor of size [batch_num, class_num]) – The direct prediction of classification fc layer.
- target (float tensor of size [batch_num, class_num]) – Binary class target for each sample.
- label_weight (float tensor of size [batch_num, class_num]) – the value is 1 if the sample is valid and 0 if ignored.
- reduction_override (str, optional) – The reduction method used to override the original reduction method of the loss. Defaults to None.
Returns: The gradient harmonized loss.
-
class
mmdet.models.losses.
GHMR
(mu=0.02, bins=10, momentum=0, loss_weight=1.0, reduction='mean')[source]¶ GHM Regression Loss.
Details of the theorem can be viewed in the paper Gradient Harmonized Single-stage Detector.
Parameters: - mu (float) – The parameter for the Authentic Smooth L1 loss.
- bins (int) – Number of the unit regions for distribution calculation.
- momentum (float) – The parameter for moving average.
- loss_weight (float) – The weight of the total GHM-R loss.
- reduction (str) – Options are “none”, “mean” and “sum”. Defaults to “mean”
-
forward
(pred, target, label_weight, avg_factor=None, reduction_override=None)[source]¶ Calculate the GHM-R loss.
Parameters: - pred (float tensor of size [batch_num, 4 (* class_num)]) – The prediction of box regression layer. Channel number can be 4 or 4 * class_num depending on whether it is class-agnostic.
- target (float tensor of size [batch_num, 4 (* class_num)]) – The target regression values with the same size of pred.
- label_weight (float tensor of size [batch_num, 4 (* class_num)]) – The weight of each sample, 0 if ignored.
- reduction_override (str, optional) – The reduction method used to override the original reduction method of the loss. Defaults to None.
Returns: The gradient harmonized loss.
-
mmdet.models.losses.
reduce_loss
(loss, reduction)[source]¶ Reduce loss as specified.
Parameters: - loss (Tensor) – Elementwise loss tensor.
- reduction (str) – Options are “none”, “mean” and “sum”.
Returns: Reduced loss tensor.
Return type: Tensor
-
mmdet.models.losses.
weighted_loss
(loss_func)[source]¶ Create a weighted version of a given loss function.
To use this decorator, the loss function must have the signature like loss_func(pred, target, **kwargs). The function only needs to compute element-wise loss without any reduction. This decorator will add weight and reduction arguments to the function. The decorated function will have the signature like loss_func(pred, target, weight=None, reduction=’mean’, avg_factor=None, **kwargs).
Example: >>> import torch >>> @weighted_loss >>> def l1_loss(pred, target): >>> return (pred - target).abs()
>>> pred = torch.Tensor([0, 2, 3]) >>> target = torch.Tensor([1, 1, 1]) >>> weight = torch.Tensor([1, 0, 1])
>>> l1_loss(pred, target) tensor(1.3333) >>> l1_loss(pred, target, weight) tensor(1.) >>> l1_loss(pred, target, reduction='none') tensor([1., 1., 2.]) >>> l1_loss(pred, target, weight, avg_factor=2) tensor(1.5000)
-
class
mmdet.models.losses.
L1Loss
(reduction='mean', loss_weight=1.0)[source]¶ L1 loss.
Parameters: - reduction (str, optional) – The method to reduce the loss. Options are “none”, “mean” and “sum”.
- loss_weight (float, optional) – The weight of loss.
-
forward
(pred, target, weight=None, avg_factor=None, reduction_override=None)[source]¶ Forward function.
Parameters: - pred (torch.Tensor) – The prediction.
- target (torch.Tensor) – The learning target of the prediction.
- weight (torch.Tensor, optional) – The weight of loss for each prediction. Defaults to None.
- avg_factor (int, optional) – Average factor that is used to average the loss. Defaults to None.
- reduction_override (str, optional) – The reduction method used to override the original reduction method of the loss. Defaults to None.
-
class
mmdet.models.losses.
AssociativeEmbeddingLoss
(pull_weight=0.25, push_weight=0.25)[source]¶ Associative Embedding Loss.
More details can be found in Associative Embedding and CornerNet . Code is modified from kp_utils.py # noqa: E501
Parameters: - pull_weight (float) – Loss weight for corners from same object.
- push_weight (float) – Loss weight for corners from different object.
-
class
mmdet.models.losses.
GaussianFocalLoss
(alpha=2.0, gamma=4.0, reduction='mean', loss_weight=1.0)[source]¶ GaussianFocalLoss is a variant of focal loss.
More details can be found in the paper Code is modified from kp_utils.py # noqa: E501 Please notice that the target in GaussianFocalLoss is a gaussian heatmap, not 0/1 binary target.
Parameters: - alpha (float) – Power of prediction.
- gamma (float) – Power of target for negative samples.
- reduction (str) – Options are “none”, “mean” and “sum”.
- loss_weight (float) – Loss weight of current loss.
-
forward
(pred, target, weight=None, avg_factor=None, reduction_override=None)[source]¶ Forward function.
Parameters: - pred (torch.Tensor) – The prediction.
- target (torch.Tensor) – The learning target of the prediction in gaussian distribution.
- weight (torch.Tensor, optional) – The weight of loss for each prediction. Defaults to None.
- avg_factor (int, optional) – Average factor that is used to average the loss. Defaults to None.
- reduction_override (str, optional) – The reduction method used to override the original reduction method of the loss. Defaults to None.
-
class
mmdet.models.losses.
QualityFocalLoss
(use_sigmoid=True, beta=2.0, reduction='mean', loss_weight=1.0)[source]¶ Quality Focal Loss (QFL) is a variant of Generalized Focal Loss: Learning Qualified and Distributed Bounding Boxes for Dense Object Detection.
Parameters: - use_sigmoid (bool) – Whether sigmoid operation is conducted in QFL. Defaults to True.
- beta (float) – The beta parameter for calculating the modulating factor. Defaults to 2.0.
- reduction (str) – Options are “none”, “mean” and “sum”.
- loss_weight (float) – Loss weight of current loss.
-
forward
(pred, target, weight=None, avg_factor=None, reduction_override=None)[source]¶ Forward function.
Parameters: - pred (torch.Tensor) – Predicted joint representation of classification and quality (IoU) estimation with shape (N, C), C is the number of classes.
- target (tuple([torch.Tensor])) – Target category label with shape (N,) and target quality label with shape (N,).
- weight (torch.Tensor, optional) – The weight of loss for each prediction. Defaults to None.
- avg_factor (int, optional) – Average factor that is used to average the loss. Defaults to None.
- reduction_override (str, optional) – The reduction method used to override the original reduction method of the loss. Defaults to None.
-
class
mmdet.models.losses.
DistributionFocalLoss
(reduction='mean', loss_weight=1.0)[source]¶ Distribution Focal Loss (DFL) is a variant of Generalized Focal Loss: Learning Qualified and Distributed Bounding Boxes for Dense Object Detection.
Parameters: - reduction (str) – Options are ‘none’, ‘mean’ and ‘sum’.
- loss_weight (float) – Loss weight of current loss.
-
forward
(pred, target, weight=None, avg_factor=None, reduction_override=None)[source]¶ Forward function.
Parameters: - pred (torch.Tensor) – Predicted general distribution of bounding boxes (before softmax) with shape (N, n+1), n is the max value of the integral set {0, …, n} in paper.
- target (torch.Tensor) – Target distance label for bounding boxes with shape (N,).
- weight (torch.Tensor, optional) – The weight of loss for each prediction. Defaults to None.
- avg_factor (int, optional) – Average factor that is used to average the loss. Defaults to None.
- reduction_override (str, optional) – The reduction method used to override the original reduction method of the loss. Defaults to None.
-
class
mmdet.models.losses.
VarifocalLoss
(use_sigmoid=True, alpha=0.75, gamma=2.0, iou_weighted=True, reduction='mean', loss_weight=1.0)[source]¶ -
forward
(pred, target, weight=None, avg_factor=None, reduction_override=None)[source]¶ Forward function.
Parameters: - pred (torch.Tensor) – The prediction.
- target (torch.Tensor) – The learning target of the prediction.
- weight (torch.Tensor, optional) – The weight of loss for each prediction. Defaults to None.
- avg_factor (int, optional) – Average factor that is used to average the loss. Defaults to None.
- reduction_override (str, optional) – The reduction method used to override the original reduction method of the loss. Options are “none”, “mean” and “sum”.
Returns: The calculated loss
Return type: torch.Tensor
-
-
class
mmdet.models.losses.
KnowledgeDistillationKLDivLoss
(reduction='mean', loss_weight=1.0, T=10)[source]¶ Loss function for knowledge distilling using KL divergence.
Parameters: - reduction (str) – Options are ‘none’, ‘mean’ and ‘sum’.
- loss_weight (float) – Loss weight of current loss.
- T (int) – Temperature for distillation.
-
forward
(pred, soft_label, weight=None, avg_factor=None, reduction_override=None)[source]¶ Forward function.
Parameters: - pred (Tensor) – Predicted logits with shape (N, n + 1).
- soft_label (Tensor) – Target logits with shape (N, N + 1).
- weight (torch.Tensor, optional) – The weight of loss for each prediction. Defaults to None.
- avg_factor (int, optional) – Average factor that is used to average the loss. Defaults to None.
- reduction_override (str, optional) – The reduction method used to override the original reduction method of the loss. Defaults to None.
-
class
mmdet.models.losses.
SeesawLoss
(use_sigmoid=False, p=0.8, q=2.0, num_classes=1203, eps=0.01, reduction='mean', loss_weight=1.0, return_dict=True)[source]¶ Seesaw Loss for Long-Tailed Instance Segmentation (CVPR 2021) arXiv: https://arxiv.org/abs/2008.10032
Parameters: - use_sigmoid (bool, optional) – Whether the prediction uses sigmoid of softmax. Only False is supported.
- p (float, optional) – The
p
in the mitigation factor. Defaults to 0.8. - q (float, optional) – The
q
in the compenstation factor. Defaults to 2.0. - num_classes (int, optional) – The number of classes. Default to 1203 for LVIS v1 dataset.
- eps (float, optional) – The minimal value of divisor to smooth the computation of compensation factor
- reduction (str, optional) – The method that reduces the loss to a scalar. Options are “none”, “mean” and “sum”.
- loss_weight (float, optional) – The weight of the loss. Defaults to 1.0
- return_dict (bool, optional) – Whether return the losses as a dict. Default to True.
-
forward
(cls_score, labels, label_weights=None, avg_factor=None, reduction_override=None)[source]¶ Forward function.
Parameters: - cls_score (torch.Tensor) – The prediction with shape (N, C + 2).
- labels (torch.Tensor) – The learning label of the prediction.
- label_weights (torch.Tensor, optional) – Sample-wise loss weight.
- avg_factor (int, optional) – Average factor that is used to average the loss. Defaults to None.
- reduction (str, optional) – The method used to reduce the loss. Options are “none”, “mean” and “sum”.
Returns: if return_dict == False: The calculated loss | if return_dict == True: The dict of calculated losses for objectness and classes, respectively.
Return type: torch.Tensor | Dict [str, torch.Tensor]
-
get_accuracy
(cls_score, labels)[source]¶ Get custom accuracy w.r.t. cls_score and labels.
Parameters: - cls_score (torch.Tensor) – The prediction with shape (N, C + 2).
- labels (torch.Tensor) – The learning label of the prediction.
Returns: - The accuracy for objectness and classes,
respectively.
Return type: Dict [str, torch.Tensor]
utils¶
-
class
mmdet.models.utils.
ResLayer
(block, inplanes, planes, num_blocks, stride=1, avg_down=False, conv_cfg=None, norm_cfg={'type': 'BN'}, downsample_first=True, **kwargs)[source]¶ ResLayer to build ResNet style backbone.
Parameters: - block (nn.Module) – block used to build ResLayer.
- inplanes (int) – inplanes of block.
- planes (int) – planes of block.
- num_blocks (int) – number of blocks.
- stride (int) – stride of the first block. Default: 1
- avg_down (bool) – Use AvgPool instead of stride conv when downsampling in the bottleneck. Default: False
- conv_cfg (dict) – dictionary to construct and config conv layer. Default: None
- norm_cfg (dict) – dictionary to construct and config norm layer. Default: dict(type=’BN’)
- downsample_first (bool) – Downsample at the first block or last block. False for Hourglass, True for ResNet. Default: True
-
mmdet.models.utils.
gaussian_radius
(det_size, min_overlap)[source]¶ Generate 2D gaussian radius.
This function is modified from the official github repo.
Given
min_overlap
, radius could computed by a quadratic equation according to Vieta’s formulas.There are 3 cases for computing gaussian radius, details are following:
- Explanation of figure:
lt
andbr
indicates the left-top and bottom-right corner of ground truth box.x
indicates the generated corner at the limited position whenradius=r
. - Case1: one corner is inside the gt box and the other is outside.
|< width >| lt-+----------+ - | | | ^ +--x----------+--+ | | | | | | | | height | | overlap | | | | | | | | | | v +--+---------br--+ - | | | +----------+--x
To ensure IoU of generated box and gt box is larger than
min_overlap
:\[\begin{split}\cfrac{(w-r)*(h-r)}{w*h+(w+h)r-r^2} \ge {iou} \quad\Rightarrow\quad {r^2-(w+h)r+\cfrac{1-iou}{1+iou}*w*h} \ge 0 \\ {a} = 1,\quad{b} = {-(w+h)},\quad{c} = {\cfrac{1-iou}{1+iou}*w*h} {r} \le \cfrac{-b-\sqrt{b^2-4*a*c}}{2*a}\end{split}\]- Case2: both two corners are inside the gt box.
|< width >| lt-+----------+ - | | | ^ +--x-------+ | | | | | | |overlap| | height | | | | | +-------x--+ | | | v +----------+-br -
To ensure IoU of generated box and gt box is larger than
min_overlap
:\[\begin{split}\cfrac{(w-2*r)*(h-2*r)}{w*h} \ge {iou} \quad\Rightarrow\quad {4r^2-2(w+h)r+(1-iou)*w*h} \ge 0 \\ {a} = 4,\quad {b} = {-2(w+h)},\quad {c} = {(1-iou)*w*h} {r} \le \cfrac{-b-\sqrt{b^2-4*a*c}}{2*a}\end{split}\]- Case3: both two corners are outside the gt box.
|< width >| x--+----------------+ | | | +-lt-------------+ | - | | | | ^ | | | | | | overlap | | height | | | | | | | | v | +------------br--+ - | | | +----------------+--x
To ensure IoU of generated box and gt box is larger than
min_overlap
:\[\begin{split}\cfrac{w*h}{(w+2*r)*(h+2*r)} \ge {iou} \quad\Rightarrow\quad {4*iou*r^2+2*iou*(w+h)r+(iou-1)*w*h} \le 0 \\ {a} = {4*iou},\quad {b} = {2*iou*(w+h)},\quad {c} = {(iou-1)*w*h} \\ {r} \le \cfrac{-b+\sqrt{b^2-4*a*c}}{2*a}\end{split}\]Parameters: - det_size (list[int]) – Shape of object.
- min_overlap (float) – Min IoU with ground truth for boxes generated by keypoints inside the gaussian kernel.
Returns: Radius of gaussian kernel.
Return type: radius (int)
- Explanation of figure:
-
mmdet.models.utils.
gen_gaussian_target
(heatmap, center, radius, k=1)[source]¶ Generate 2D gaussian heatmap.
Parameters: - heatmap (Tensor) – Input heatmap, the gaussian kernel will cover on it and maintain the max value.
- center (list[int]) – Coord of gaussian kernel’s center.
- radius (int) – Radius of gaussian kernel.
- k (int) – Coefficient of gaussian kernel. Default: 1.
Returns: Updated heatmap covered by gaussian kernel.
Return type: out_heatmap (Tensor)
-
class
mmdet.models.utils.
DetrTransformerDecoderLayer
(attn_cfgs, feedforward_channels, ffn_dropout=0.0, operation_order=None, act_cfg={'inplace': True, 'type': 'ReLU'}, norm_cfg={'type': 'LN'}, ffn_num_fcs=2, **kwargs)[source]¶ Implements decoder layer in DETR transformer.
Parameters: - attn_cfgs (list[mmcv.ConfigDict] | list[dict] | dict )) – Configs for self_attention or cross_attention, the order should be consistent with it in operation_order. If it is a dict, it would be expand to the number of attention in operation_order.
- feedforward_channels (int) – The hidden dimension for FFNs.
- ffn_dropout (float) – Probability of an element to be zeroed in ffn. Default 0.0.
- operation_order (tuple[str]) – The execution order of operation in transformer. Such as (‘self_attn’, ‘norm’, ‘ffn’, ‘norm’). Default:None
- act_cfg (dict) – The activation config for FFNs. Default: LN
- norm_cfg (dict) – Config dict for normalization layer. Default: LN.
- ffn_num_fcs (int) – The number of fully-connected layers in FFNs. Default:2.
-
class
mmdet.models.utils.
DetrTransformerDecoder
(*args, post_norm_cfg={'type': 'LN'}, return_intermediate=False, **kwargs)[source]¶ Implements the decoder in DETR transformer.
Parameters: - return_intermediate (bool) – Whether to return intermediate outputs.
- post_norm_cfg (dict) – Config of last normalization layer. Default: LN.
-
forward
(query, *args, **kwargs)[source]¶ Forward function for TransformerDecoder.
Parameters: query (Tensor) – Input query with shape (num_query, bs, embed_dims). Returns: - Results with shape [1, num_query, bs, embed_dims] when
- return_intermediate is False, otherwise it has shape [num_layers, num_query, bs, embed_dims].
Return type: Tensor
-
class
mmdet.models.utils.
Transformer
(encoder=None, decoder=None, init_cfg=None)[source]¶ Implements the DETR transformer.
Following the official DETR implementation, this module copy-paste from torch.nn.Transformer with modifications:
- positional encodings are passed in MultiheadAttention
- extra LN at the end of encoder is removed
- decoder returns a stack of activations from all decoding layers
See paper: End-to-End Object Detection with Transformers for details.
Parameters: - encoder (mmcv.ConfigDict | Dict) – Config of TransformerEncoder. Defaults to None.
- decoder ((mmcv.ConfigDict | Dict)) – Config of TransformerDecoder. Defaults to None
- (obj (init_cfg) – mmcv.ConfigDict): The Config for initialization. Defaults to None.
-
forward
(x, mask, query_embed, pos_embed)[source]¶ Forward function for Transformer.
Parameters: - x (Tensor) – Input query with shape [bs, c, h, w] where c = embed_dims.
- mask (Tensor) – The key_padding_mask used for encoder and decoder, with shape [bs, h, w].
- query_embed (Tensor) – The query embedding for decoder, with shape [num_query, c].
- pos_embed (Tensor) – The positional encoding for encoder and decoder, with the same shape as x.
Returns: results of decoder containing the following tensor.
- out_dec: Output from decoder. If return_intermediate_dec is True output has shape [num_dec_layers, bs,
- num_query, embed_dims], else has shape [1, bs, num_query, embed_dims].
- memory: Output results from encoder, with shape [bs, embed_dims, h, w].
Return type: tuple[Tensor]
-
mmdet.models.utils.
build_linear_layer
(cfg, *args, **kwargs)[source]¶ Build linear layer. :param cfg: The linear layer config, which should contain:
- type (str): Layer type.
- layer args: Args needed to instantiate an linear layer.
Parameters: - args (argument list) – Arguments passed to the __init__ method of the corresponding linear layer.
- kwargs (keyword arguments) – Keyword arguments passed to the __init__ method of the corresponding linear layer.
Returns: Created linear layer.
Return type: nn.Module
-
class
mmdet.models.utils.
SinePositionalEncoding
(num_feats, temperature=10000, normalize=False, scale=6.283185307179586, eps=1e-06, offset=0.0, init_cfg=None)[source]¶ Position encoding with sine and cosine functions.
See End-to-End Object Detection with Transformers for details.
Parameters: - num_feats (int) – The feature dimension for each position along x-axis or y-axis. Note the final returned dimension for each position is 2 times of this value.
- temperature (int, optional) – The temperature used for scaling the position embedding. Defaults to 10000.
- normalize (bool, optional) – Whether to normalize the position embedding. Defaults to False.
- scale (float, optional) – A scale factor that scales the position embedding. The scale will be used only when normalize is True. Defaults to 2*pi.
- eps (float, optional) – A value added to the denominator for numerical stability. Defaults to 1e-6.
- offset (float) – offset add to embed when do the normalization. Defaults to 0.
- init_cfg (dict or list[dict], optional) – Initialization config dict. Default: None
-
forward
(mask)[source]¶ Forward function for SinePositionalEncoding.
Parameters: mask (Tensor) – ByteTensor mask. Non-zero values representing ignored positions, while zero values means valid positions for this image. Shape [bs, h, w]. Returns: - Returned position embedding with shape
- [bs, num_feats*2, h, w].
Return type: pos (Tensor)
-
class
mmdet.models.utils.
LearnedPositionalEncoding
(num_feats, row_num_embed=50, col_num_embed=50, init_cfg={'layer': 'Embedding', 'type': 'Uniform'})[source]¶ Position embedding with learnable embedding weights.
Parameters: - num_feats (int) – The feature dimension for each position along x-axis or y-axis. The final returned dimension for each position is 2 times of this value.
- row_num_embed (int, optional) – The dictionary size of row embeddings. Default 50.
- col_num_embed (int, optional) – The dictionary size of col embeddings. Default 50.
- init_cfg (dict or list[dict], optional) – Initialization config dict.
-
forward
(mask)[source]¶ Forward function for LearnedPositionalEncoding.
Parameters: mask (Tensor) – ByteTensor mask. Non-zero values representing ignored positions, while zero values means valid positions for this image. Shape [bs, h, w]. Returns: - Returned position embedding with shape
- [bs, num_feats*2, h, w].
Return type: pos (Tensor)
-
class
mmdet.models.utils.
DynamicConv
(in_channels=256, feat_channels=64, out_channels=None, input_feat_shape=7, act_cfg={'inplace': True, 'type': 'ReLU'}, norm_cfg={'type': 'LN'}, init_cfg=None)[source]¶ Implements Dynamic Convolution.
This module generate parameters for each sample and use bmm to implement 1*1 convolution. Code is modified from the official github repo .
Parameters: - in_channels (int) – The input feature channel. Defaults to 256.
- feat_channels (int) – The inner feature channel. Defaults to 64.
- out_channels (int, optional) – The output feature channel. When not specified, it will be set to in_channels by default
- input_feat_shape (int) – The shape of input feature. Defaults to 7.
- act_cfg (dict) – The activation config for DynamicConv.
- norm_cfg (dict) – Config dict for normalization layer. Default layer normalization.
- (obj (init_cfg) – mmcv.ConfigDict): The Config for initialization. Default: None.
-
forward
(param_feature, input_feature)[source]¶ Forward function for DynamicConv.
Parameters: - param_feature (Tensor) – The feature can be used to generate the parameter, has shape (num_all_proposals, in_channels).
- input_feature (Tensor) – Feature that interact with parameters, has shape (num_all_proposals, in_channels, H, W).
Returns: The output feature has shape (num_all_proposals, out_channels).
Return type: Tensor
-
class
mmdet.models.utils.
SimplifiedBasicBlock
(inplanes, planes, stride=1, dilation=1, downsample=None, style='pytorch', with_cp=False, conv_cfg=None, norm_cfg={'type': 'BN'}, dcn=None, plugins=None, init_fg=None)[source]¶ Simplified version of original basic residual block. This is used in SCNet.
- Norm layer is now optional
- Last ReLU in forward function is removed
-
norm1
¶ normalization layer after the first convolution layer
Type: nn.Module
-
norm2
¶ normalization layer after the second convolution layer
Type: nn.Module
-
class
mmdet.models.utils.
NormedLinear
(*args, tempearture=20, power=1.0, eps=1e-06, **kwargs)[source]¶ Normalized Linear Layer.
Parameters: - tempeature (float, optional) – Tempeature term. Default to 20.
- power (int, optional) – Power term. Default to 1.0.
- eps (float, optional) – The minimal value of divisor to keep numerical stability. Default to 1e-6.
-
forward
(x)[source]¶ Defines the computation performed at every call.
Should be overridden by all subclasses.
Note
Although the recipe for forward pass needs to be defined within this function, one should call the
Module
instance afterwards instead of this since the former takes care of running the registered hooks while the latter silently ignores them.
-
class
mmdet.models.utils.
NormedConv2d
(*args, tempearture=20, power=1.0, eps=1e-06, norm_over_kernel=False, **kwargs)[source]¶ Normalized Conv2d Layer.
Parameters: - tempeature (float, optional) – Tempeature term. Default to 20.
- power (int, optional) – Power term. Default to 1.0.
- eps (float, optional) – The minimal value of divisor to keep numerical stability. Default to 1e-6.
- norm_over_kernel (bool, optional) – Normalize over kernel. Default to False.
-
forward
(x)[source]¶ Defines the computation performed at every call.
Should be overridden by all subclasses.
Note
Although the recipe for forward pass needs to be defined within this function, one should call the
Module
instance afterwards instead of this since the former takes care of running the registered hooks while the latter silently ignores them.
-
mmdet.models.utils.
make_divisible
(value, divisor, min_value=None, min_ratio=0.9)[source]¶ Make divisible function.
This function rounds the channel number to the nearest value that can be divisible by the divisor. It is taken from the original tf repo. It ensures that all layers have a channel number that is divisible by divisor. It can be seen here: https://github.com/tensorflow/models/blob/master/research/slim/nets/mobilenet/mobilenet.py # noqa
Parameters: - value (int) – The original channel number.
- divisor (int) – The divisor to fully divide the channel number.
- min_value (int) – The minimum value of the output channel. Default: None, means that the minimum value equal to the divisor.
- min_ratio (float) – The minimum ratio of the rounded channel number to the original channel number. Default: 0.9.
Returns: The modified output channel number.
Return type: int
-
class
mmdet.models.utils.
InvertedResidual
(in_channels, out_channels, mid_channels, kernel_size=3, stride=1, se_cfg=None, with_expand_conv=True, conv_cfg=None, norm_cfg={'type': 'BN'}, act_cfg={'type': 'ReLU'}, with_cp=False, init_cfg=None)[source]¶ Inverted Residual Block.
Parameters: - in_channels (int) – The input channels of this Module.
- out_channels (int) – The output channels of this Module.
- mid_channels (int) – The input channels of the depthwise convolution.
- kernel_size (int) – The kernal size of the depthwise convolution. Default: 3.
- stride (int) – The stride of the depthwise convolution. Default: 1.
- se_cfg (dict) – Config dict for se layer. Defaul: None, which means no se layer.
- with_expand_conv (bool) – Use expand conv or not. If set False, mid_channels must be the same with in_channels. Default: True.
- conv_cfg (dict) – Config dict for convolution layer. Default: None, which means using conv2d.
- norm_cfg (dict) – Config dict for normalization layer. Default: dict(type=’BN’).
- act_cfg (dict) – Config dict for activation layer. Default: dict(type=’ReLU’).
- with_cp (bool) – Use checkpoint or not. Using checkpoint will save some memory while slowing down the training speed. Default: False.
- init_cfg (dict or list[dict], optional) – Initialization config dict. Default: None
Returns: The output tensor.
Return type: Tensor
-
forward
(x)[source]¶ Defines the computation performed at every call.
Should be overridden by all subclasses.
Note
Although the recipe for forward pass needs to be defined within this function, one should call the
Module
instance afterwards instead of this since the former takes care of running the registered hooks while the latter silently ignores them.
-
class
mmdet.models.utils.
SELayer
(channels, ratio=16, conv_cfg=None, act_cfg=({'type': 'ReLU'}, {'type': 'Sigmoid'}), init_cfg=None)[source]¶ Squeeze-and-Excitation Module.
Parameters: - channels (int) – The input (and output) channels of the SE layer.
- ratio (int) – Squeeze ratio in SELayer, the intermediate channel will be
int(channels/ratio)
. Default: 16. - conv_cfg (None or dict) – Config dict for convolution layer. Default: None, which means using conv2d.
- act_cfg (dict or Sequence[dict]) – Config dict for activation layer. If act_cfg is a dict, two activation layers will be configurated by this dict. If act_cfg is a sequence of dicts, the first activation layer will be configurated by the first dict and the second activation layer will be configurated by the second dict. Default: (dict(type=’ReLU’), dict(type=’Sigmoid’))
- init_cfg (dict or list[dict], optional) – Initialization config dict. Default: None
-
forward
(x)[source]¶ Defines the computation performed at every call.
Should be overridden by all subclasses.
Note
Although the recipe for forward pass needs to be defined within this function, one should call the
Module
instance afterwards instead of this since the former takes care of running the registered hooks while the latter silently ignores them.
-
mmdet.models.utils.
interpolate_as
(source, target, mode='bilinear', align_corners=False)[source]¶ Interpolate the source to the shape of the target.
Interpolate the source to the shape of target. The input must be a Tensor, but the target can be a Tensor or a np.ndarray with the shape (…, target_h, target_w).
Parameters: - source (Tensor) – A 3D/4D Tensor with the shape (N, H, W) or (N, C, H, W).
- target (Tensor | np.ndarray) – The interpolation target with the shape (…, target_h, target_w).
- mode (str) – Algorithm used for interpolation. The options are the
same as those in F.interpolate(). Default:
'bilinear'
. - align_corners (bool) – The same as the argument in F.interpolate().
Returns: The interpolated source Tensor.
Return type: Tensor
-
class
mmdet.models.utils.
ConvUpsample
(in_channels, inner_channels, num_layers=1, num_upsample=None, conv_cfg=None, norm_cfg=None, init_cfg=None, **kwargs)[source]¶ ConvUpsample performs 2x upsampling after Conv.
There are several ConvModule layers. In the first few layers, upsampling will be applied after each layer of convolution. The number of upsampling must be no more than the number of ConvModule layers.
Parameters: - in_channels (int) – Number of channels in the input feature map.
- inner_channels (int) – Number of channels produced by the convolution.
- num_layers (int) – Number of convolution layers.
- num_upsample (int | optional) – Number of upsampling layer. Must be no
more than num_layers. Upsampling will be applied after the first
num_upsample
layers of convolution. Default:num_layers
. - conv_cfg (dict) – Config dict for convolution layer. Default: None, which means using conv2d.
- norm_cfg (dict) – Config dict for normalization layer. Default: None.
- init_cfg (dict) – Config dict for initialization. Default: None.
- kwargs (key word augments) – Other augments used in ConvModule.
-
forward
(x)[source]¶ Defines the computation performed at every call.
Should be overridden by all subclasses.
Note
Although the recipe for forward pass needs to be defined within this function, one should call the
Module
instance afterwards instead of this since the former takes care of running the registered hooks while the latter silently ignores them.
-
class
mmdet.models.utils.
CSPLayer
(in_channels, out_channels, expand_ratio=0.5, num_blocks=1, add_identity=True, use_depthwise=False, conv_cfg=None, norm_cfg={'eps': 0.001, 'momentum': 0.03, 'type': 'BN'}, act_cfg={'type': 'Swish'}, init_cfg=None)[source]¶ Cross Stage Partial Layer.
Parameters: - in_channels (int) – The input channels of the CSP layer.
- out_channels (int) – The output channels of the CSP layer.
- expand_ratio (float) – Ratio to adjust the number of channels of the hidden layer. Default: 0.5
- num_blocks (int) – Number of blocks. Default: 1
- add_identity (bool) – Whether to add identity in blocks. Default: True
- use_depthwise (bool) – Whether to depthwise separable convolution in blocks. Default: False
- conv_cfg (dict, optional) – Config dict for convolution layer. Default: None, which means using conv2d.
- norm_cfg (dict) – Config dict for normalization layer. Default: dict(type=’BN’)
- act_cfg (dict) – Config dict for activation layer. Default: dict(type=’Swish’)
-
forward
(x)[source]¶ Defines the computation performed at every call.
Should be overridden by all subclasses.
Note
Although the recipe for forward pass needs to be defined within this function, one should call the
Module
instance afterwards instead of this since the former takes care of running the registered hooks while the latter silently ignores them.