[serge] Fix 8 integration tests for model blip_2 failing with other (other (8))

[sergereview]

Original CI failure

• Failure group: 1/1: 8 integration tests for model blip_2failing withother (other (8))
• tests/models/blip_2/test_modeling_blip_2.py::Blip2ModelIntegrationTest::test_inference_itm [multi-gpu] (other, seen 6/7)

Relates to #46906

The blip2-itm-vit-g integration tests fail because AutoTokenizer cannot instantiate a backend tokenizer for the checkpoint. The model's tokenizer_config.json declares BertTokenizer, while the blip-2 model type maps to GPT2Tokenizer. The mismatch path tries to use the hub-declared slow class, but no fast serialization is available, so TokenizersBackend initialization raises ValueError: Couldn't instantiate the backend tokenizer from one of:.

Root cause: Salesforce/blip2-itm-vit-g is not in MODEL_IDS_TO_TOKENIZERS_BACKEND, so it does not get routed to TokenizersBackend.from_pretrained, which is the intended loader for these BLIP-2 checkpoints.

Patch: add salesforce/blip2-itm-* to MODEL_IDS_TO_TOKENIZERS_BACKEND in tokenization_auto.py, alongside the existing blip2-opt-* and blip2-flan-t5-* patterns.

---

This change was produced automatically by serge from a CI failure report. The patch was generated by an LLM and applied by serge; review before merging.

serge v0.1.0 · model: moonshotai/Kimi-K2.7-Code · 31 LLM turns · 30 tool calls · 80.5s · 443747 in / 3535 out tokens

[github-actions]

[For maintainers] Suggested jobs to run (before merge)

run-slow: auto

[github-actions]

CI Dashboard: View test results in Grafana

[HuggingFaceDocBuilderDev]

The docs for this PR live here. All of your documentation changes will be reflected on that endpoint. The docs are available until 30 days after the last update.