Skip to content

[serge] Fix 2 integration tests for model helium failing with output_mismatch (list output differs (2))#47544

Open
sergereview[bot] wants to merge 1 commit into
mainfrom
serge/fix/itf-c797b2a6d2bf-cbe732ba
Open

[serge] Fix 2 integration tests for model helium failing with output_mismatch (list output differs (2))#47544
sergereview[bot] wants to merge 1 commit into
mainfrom
serge/fix/itf-c797b2a6d2bf-cbe732ba

Conversation

@sergereview

@sergereview sergereview Bot commented Jul 25, 2026

Copy link
Copy Markdown
Contributor

CI

Original CI failure

  • Failure group: 1/1: 2 integration tests for model heliumfailing withoutput_mismatch (list output differs (2))
  • tests/models/helium/test_modeling_helium.py::HeliumIntegrationTest::test_model_2b [multi-gpu] (output_mismatch, seen 7/7)

Relates to #47538

The Helium test_model_2b integration test fails on CUDA because the device-specific expected generated text is stale. The model now produces the same text as the default expectation (...now and I have), while the CUDA 8 entry still expects the old text (...now, and I).

This is a test-expectation mismatch only; there is no model crash or numerical error. The patch updates the CUDA expectation to the current model output, matching the default expectation already in the file.

✅ Verified on GPU

serge ran the targeted @slow test(s) on a GPU runner and opened this PR only after they passed with this patch.


This change was produced automatically by serge from a CI failure report. The patch was generated by an LLM and applied by serge; review before merging.

serge v0.1.0 · model: moonshotai/Kimi-K2.7-Code · 81 LLM turns · 80 tool calls · 232.7s · 1771461 in / 8790 out tokens

@sergereview
sergereview Bot marked this pull request as ready for review July 25, 2026 09:18
@github-actions
github-actions Bot requested a review from ydshieh July 25, 2026 09:18
@github-actions

Copy link
Copy Markdown
Contributor

[For maintainers] Suggested jobs to run (before merge)

run-slow: helium

@HuggingFaceDocBuilderDev

Copy link
Copy Markdown

The docs for this PR live here. All of your documentation changes will be reflected on that endpoint. The docs are available until 30 days after the last update.

@github-actions

Copy link
Copy Markdown
Contributor

CI recap

Dashboard: View test results in Grafana
Latest run: 30152581070
Result: success | Grafana metrics are not available yet.

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

1 participant