Fix CUDA Graph breaking host to device copy from scalar tensor allocation - #47547
Conversation
Signed-off-by: Harry Mellor <19981378+hmellor@users.noreply.github.com>
|
The docs for this PR live here. All of your documentation changes will be reflected on that endpoint. The docs are available until 30 days after the last update. |
vasqu
left a comment
vasqu
left a comment
There was a problem hiding this comment.
Careful approval because it totally makes sense BUT 2 things
- Can we add a small test that catches this
- I think pretty much any of our placeholder mask code uses simple torch.tensor, so we could apply this across all models
cc @zucchini-nlp @IlyasMoutawwakil just in case
|
@vasqu I could add an mlinter rule? |
|
Sure why not tbh |
Signed-off-by: Harry Mellor <19981378+hmellor@users.noreply.github.com>
|
Can we update the title of the PR please 😬 |
hmellor
changed the title
|
Updated the title and description |
CI recapDashboard: View test results in Grafana |
Co-Authored-By: Claude Opus 5 (1M context) <noreply@anthropic.com>
|
[For maintainers] Suggested jobs to run (before merge) run-slow: aria, audioflamingo3, aya_vision, blip_2, bloom, chmv2, cohere2_vision, conditional_detr, cosmos3_edge, cosmos3_omni, d_fine, deepseek_ocr2, deepseek_vl, deepseek_vl_hybrid, deimv2, detr |
…tion (huggingface#47547) * Make Gemma 4 CUDA Graph compilable Signed-off-by: Harry Mellor <19981378+hmellor@users.noreply.github.com> * fix everywhere Signed-off-by: Harry Mellor <19981378+hmellor@users.noreply.github.com> * Run modular conversion for instructblipvideo Co-Authored-By: Claude Opus 5 (1M context) <noreply@anthropic.com> --------- Signed-off-by: Harry Mellor <19981378+hmellor@users.noreply.github.com> Co-authored-by: Claude Opus 5 (1M context) <noreply@anthropic.com>
3 participants
Fixed a CUDA-graph-capture crash in several models by replacing
torch.tensor(self.config.<x>_token_id, device=inputs_embeds.device)(a host→device copy that graph capture forbids) withtorch.full((), self.config.<x>_token_id, dtype=torch.long, device=inputs_embeds.device), which fills the scalar directly on-device.