Skip to content

[Labelling Health] Labelling Health Report — 2026-04-10 #132

@mnkiefer

Description

@mnkiefer

Summary

Overall status: Mixed

Auto-labelling via the Label Discussions workflow ramped up significantly this week (10 runs in 7 days vs. 3 in the previous window, all successful). However, the correction feedback loop is completely inactive—all 30 Labelling Correction Feedback runs in the last 7 days concluded as skipped—meaning the 79 open correction signals are accumulating with zero resolution. The system is processing discussions actively but cannot yet act on the trusted correction signals it is collecting.


Key Metrics

Metric Value Notes
Discussions reviewed — last 7 days ~170+ Conservative floor from parseable titles; structured fields mostly null
Label changes applied — last 7 days ~54+ From titles only (#119: 18, #42: 36); many summaries unparsed
Change rate — last 7 days ~32% overall; 60% on 2026-04-07, 26% on 2026-04-08 High day-to-day variance; incomplete data
Change rate — previous 7 days Unknown (only 1 parsed summary with partial data) Cannot compare windows reliably
Correction-Collector runs — last 7 days 30 Up from 0 in previous 7-day window
Correction-Feedback runs — last 7 days 30 (all skipped) No signals being resolved
Open correction signals 79 2 closed (both from the previous window)

⚠️ Data quality note: The reviewed and changed fields are null for 9 of 10 daily summary issues in the last 7 days, even when titles contain the numbers (e.g., "70 discussions reviewed, 18 updated", "36 label changes across 60 discussions"). Figures above are manually inferred from titles and should be treated as conservative estimates.


Correction Pressure

Pressure is concentrated in two high-volume categories. Of the 83 open signals where category metadata is available:

Discussion Category Signal Count
other-feature-feedback-questions-ideas 24
github-education 15
actions 10
discussions 6
pull-requests 6
copilot-conversations 4
repositories 4
Other 14

The top two labels seen across open signals are "Other Features and Feedback" (12 signals) and "GitHub Education" (10 signals), suggesting the auto-labeller is struggling most with the boundary between those two categories and with Education-specific sub-labelling.

"ARC (Actions Runner Controller)" and bug tie at 6 signals each—ARC may be under-represented in labelling instructions relative to its discussion volume.

All open correction signals originate from Batch 02 (4 open parent intake issues, all created 2026-04-08). Pressure is concentrated into a single batch rather than spread across many.

Open correction signal label breakdown
Latest Label Count
Other Features and Feedback 12
GitHub Education 10
ARC (Actions Runner Controller) 6
bug 6
Profile 5
Actions 4
Copilot 4
Product Feedback 4
Code Search and Navigation 3
Pull Requests 3
Code Security 2
Discussions 2
GitHub Education Benefits 2
Mobile 2
Repositories 2
other 2
Remaining (1 each) 8

Open Instruction Debt

The correction backlog is growing rapidly. 79 of 81 signals are open; only 2 have been closed (both from the previous 7-day window). The oldest open signal is #47, created 2026-04-08—approximately 2 days old at report time.

Four open parent intake issues exist (all Batch 02), which is unusual duplication for a single batch. Batch 01 (#26) is closed, suggesting the intake mechanism itself was working previously.

The backlog is not yet stale (all signals are <3 days old), but it is entirely unactionable until the Correction Feedback workflow stops skipping. If the current rate continues (~30 new correction signals per day), the backlog will become unmanageable within 1–2 weeks.

Correction parent intake issues
Issue Title Created State
#50 Labelling Correction Intake — Batch 02 2026-04-08 Open
#49 Labelling Correction Intake — Batch 02 2026-04-08 Open
#46 Labelling Correction Intake — Batch 02 2026-04-08 Open
#45 Labelling Correction Intake — Batch 02 2026-04-08 Open
#26 Labelling Correction Intake — Batch 01 2026-03-31 Closed

Recommendations

  1. Investigate and unblock the Correction Feedback workflow. All 30 runs in the last 7 days concluded skipped. Until this is resolved, no correction signals can feed back into labelling instruction improvements. Check the workflow's trigger conditions and skip logic immediately.

  2. Fix daily summary structured-field extraction. Only 1 of 10 last-week summary issues had a parsed reviewed count; none had changed parsed despite titles containing both numbers. The parsing regex or body template has likely drifted. Fixing this is required for trend tracking to be reliable.

  3. Deduplicate the open Batch 02 parent intake issues. Four open issues (#45, #46, #49, #50) carry the identical title "Labelling Correction Intake — Batch 02". Close or merge the duplicates and investigate why the intake workflow created multiple parents for the same batch.

  4. Prioritise other-feature-feedback-questions-ideas and github-education in .github/instructions/community-discussion-labeling.md. These two categories account for ~47% of correction signal pressure. Tightening the boundary rules between "Other Features and Feedback", "Product Feedback", and category-specific labels—and clarifying GitHub Education sub-label assignment—would reduce the most common correction cases.


References

Recent daily summary issues (last 7 days)
Issue Date Reviewed Changed
#124 Daily Auto-labelling Summary – 2026-04-09 2026-04-09
#119 2026-04-08 — 70 discussions reviewed, 18 updated 2026-04-08 ~70 (title) ~18 (title)
#101 Daily Auto-labelling Summary – 2026-04-08 2026-04-08
#44 Discussion Label Run — 2026-04-08 2026-04-08
#42 2026-04-07 — 36 label changes across 60 discussions 2026-04-07 ~60 (title) ~36 (title)
#40 Discussion Label Run — 2026-04-06 2026-04-06
#38 Discussion Label Updates – 2026-04-05 2026-04-05
#36 Daily Auto-labelling Summary – 2026-04-04 2026-04-04 40
#34 Daily Auto-labelling Summary – 2026-04-03 2026-04-03
#33 2026-04-03 2026-04-03
Recent workflow runs

Label Discussions (last 7 days — 10 runs, all success)

Run Date
§35 2026-04-09
§34 2026-04-08
§33 2026-04-08
§32 2026-04-08
§31 2026-04-07

Labelling Correction Collector (last 7 days — 30 runs, all success)

Labelling Correction Feedback (last 7 days — 30 runs, all skipped)

§35 · §351 · §117

Generated by Labelling Health Report · ● 648.3K ·

  • expires on May 10, 2026, 3:39 AM UTC

Metadata

Metadata

Assignees

No one assigned

    Type

    No type
    No fields configured for issues without a type.

    Projects

    No projects

    Milestone

    No milestone

    Relationships

    None yet

    Development

    No branches or pull requests

    Issue actions