Skip to content

Enhancement: Add explicit UserWarning for NaN classes in macro-averaged metrics #558

New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

Sign up for GitHub

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Open
karttikjangid wants to merge 4 commits into
base: master
Choose a base branch
from
Open

Enhancement: Add explicit UserWarning for NaN classes in macro-averaged metrics #558

Changes from all commits
Commits
Show all changes
4 commits
Select commit Hold shift + click to select a range
36f27ba
Enhancement: Add explicit UserWarning for NaN classes in macro-averag…
karttikjangid Apr 19, 2026
47fbe09
chore: Update poetry.lock to resolve CI dependency sync
karttikjangid Apr 19, 2026
1e31226
fix: resolve CI linter issues and sync torch dependencies
karttikjangid Apr 19, 2026
af10d67
Reinstating original poetry from master branch for reference
karttikjangid Apr 19, 2026
File filter

Filter by extension

Filter by extension
Conversations
Failed to load comments.
Loading
Jump to
Jump to file
Failed to load files.
Loading
Diff view
Diff view
12 changes: 12 additions & 0 deletions perceptionmetrics/utils/segmentation_metrics.py
View file
Open in desktop
Original file line number Diff line number Diff line change
Expand Up @@ -255,6 +255,18 @@ def get_averaged_metric(
"""
metric = getattr(self, f"get_{metric_name}")
if method == "macro":
per_class_values = metric(per_class=True)
nan_mask = np.isnan(per_class_values)
missing_count = int(np.sum(nan_mask))

if missing_count > 0:
import warnings

@dpascualhe dpascualhe Jul 10, 2026

Copy link
Copy Markdown
Collaborator

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

raise the warning as part of logging (you can check out logging.warning in line 60 of rellis3d.py). At some point we will do a project-wide effort to actually settle down on a proper logging system

# Use implicit string concatenation to keep lines short
msg = (
f"Warning: {missing_count} class(es) were missing from the confusion matrix. "
f"Their {metric_name.upper()} evaluated to NaN and will be ignored in the macro-average."
)
warnings.warn(msg, UserWarning)
return float(np.nanmean(metric(per_class=True)))

@dpascualhe dpascualhe Jul 10, 2026

Copy link
Copy Markdown
Collaborator

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

reuse here the per_class_values variable to avoid computing the metric twice

if method == "micro":
return float(metric(per_class=False))
Expand Down
Loading