You signed in with another tab or window. Reload to refresh your session.You signed out in another tab or window. Reload to refresh your session.You switched accounts on another tab or window. Reload to refresh your session.Dismiss alert
{{ message }}
Repository navigation
ConfusionMatrixMetric: ignore_index excludes a channel, not the ignored voxels #9206
get_confusion_matrix() (the backbone of ConfusionMatrixMetric) handles a valid ignore_index by zeroing the ignored channel in y_pred and y. Zeroing a channel does not remove those voxels from the other channels' counts, so a voxel of the ignored class that the model assigned to another class still counts as a false positive for that class.
DiceMetric and MeanIoU instead build a spatial mask with create_ignore_mask() (#9134 fixed compute_iou() for exactly this inconsistency), so ConfusionMatrixMetric disagrees with them for the same input and the same setting.
Two related facets of the same root cause:
With a single-channel binary target and ignore_index=0, include_background=True additionally zeroes the only channel, hiding a valid false positive at a non-ignored location (specificity 1.0 instead of 0.0).
importtorchfrommonai.metricsimportget_confusion_matrix# 4 voxels, 3 one-hot classes; voxel 1 belongs to the ignored class 1y=torch.tensor([[[1., 0, 0, 1], [0, 1, 0, 0], [0, 0, 1, 0]]])
# a perfect prediction except the ignored voxel is called class 0y_pred=torch.tensor([[[1., 1, 0, 1], [0, 0, 0, 0], [0, 0, 1, 0]]])
counts=get_confusion_matrix(y_pred, y, include_background=True, ignore_index=1)
counts[0, 0].tolist() # [2.0, 1.0, 1.0, 0.0] — TP, FP, TN, FN for class 0
Single-channel case:
importtorchfrommonai.metricsimportConfusionMatrixMetricy=torch.tensor([[[1., 1., 0.]]]) # single-channel binary targetmetric=ConfusionMatrixMetric(metric_name="specificity", reduction="none", ignore_index=0)
metric(y_pred=torch.ones_like(y), y=y)
metric.aggregate() # specificity 1.0 — the false positive at the only valid voxel is hidden
Expected behavior
counts[0, 0].tolist() # [2.0, 0.0, 1.0, 0.0] — the ignored voxel is excluded from every class's counts
Class-0 precision on the valid voxels is 1.0, matching DiceMetric/MeanIoU for the same input. In the single-channel case, specificity is 0.0: the false positive at the only valid voxel is retained regardless of include_background.
Environment
Validated on dev @ bbebca6f17cabf98ea5fb8ec5776eeacd10aa60d, Windows, Python 3.14.1, PyTorch 2.12.1+cpu, NumPy 2.3.5.
Additional context
PR #9195 fixes this by excluding the ignored voxels spatially (reusing create_ignore_mask() semantics), while preserving the existing channel exclusion for multi-channel targets, retaining valid single-channel errors regardless of include_background, and supporting boolean targets. Related to #9134, which fixed the same inconsistency for MeanIoU.
Activity