Skip to content

[Test Coverage] Generation compatibility for TransformerBridge models lacking native generate() #1328

@jlarson4

Description

@jlarson4

Summary

TransformerBridge.generate() works for most autoregressive HF models, but some bridge models (e.g. encoder-only architectures, certain multimodal models) currently fall through to an AttributeError/RuntimeError. The compatibility test silently skips when this happens, hiding the gap.

Affected test

  • tests/unit/model_bridge/compatibility/test_utils.py:104 has inline pytest.skip("Generation not supported for this TransformerBridge model") inside an except (AttributeError, RuntimeError):

What the test verifies

That model.generate(prompt, max_new_tokens=5) returns one of (str, list, torch.Tensor) for every parameterized model.

Acceptance criteria

  • Models that legitimately don't support generation (BERT encoder-only) are filtered out of the parameterized set rather than handled with a runtime except + skip.
  • The inline except + pytest.skip is removed.
  • Remaining tests pass without a silent fallback.

Where to start

  • Find the parameterized model set (@pytest.mark.parametrize or params= on the fixture).
  • Categorize each by whether generation is supported. Filter the non-supporting ones out at parametrize time.
  • Investigate and fix models that should support generation, but currently raise.

Metadata

Metadata

Assignees

No one assigned

    Labels

    TransformerBridgeBug specific to the new TransformerBridge systemcomplexity-moderateModerately complicated issues for people who have intermediate experience with the codegood first issueGood for newcomershelp wantedExtra attention is neededlow-priorityMaintainers are not prioritising this work currently.

    Type

    No type
    No fields configured for issues without a type.

    Projects

    No projects

    Milestone

    No milestone

    Relationships

    None yet

    Development

    No branches or pull requests

    Issue actions