fix: remove unsound structured-output Tapioca compiler

[jbeckwith-oai]

Summary

• Remove the unreleased OpenAI::BaseModel Tapioca DSL compiler, its compiler-specific README section, and the compiler-only tests.
• Preserve the independently useful Rails/Tapioca load-order handling and nullable-array Sorbet conversion fixes from feat: add Tapioca typing for structured outputs #364.
• Add focused compatibility regressions proving application-defined structured-output models preserve raw nested hashes and arrays on construction/assignment, while Chat Completions and Responses both hydrate nested models in parsed API responses.
• Keep the nullable-array Sorbet regression in an isolated subprocess so sorbet-runtime remains optional for normal SDK usage.

Why this is the correct architectural rollback

#364 addressed the customer report in #309 by adding a compiler that generated readers such as Participant and T::Array[Participant]. Those signatures are unsound for the SDK's supported transport-model contract:

event = Event.new(participant: {name: "Ada"}, participants: [{name: "Grace"}])
event.participant       # => Hash
event.participants.first # => Hash

Parsed API responses correctly hydrate those same fields into application-defined model objects. The original compiler test only type-checked synthetic RBI usage; it never compared generated reader signatures with direct model construction, assignment, or both public parsed-response boundaries. The proposed runtime workaround in #385 introduced additional coercion and conflicting accessor/storage semantics instead of addressing the incorrect abstraction.

Architecture review concluded that SDK transport models should remain permissive transport models. Sound application-owned typed structured outputs should instead be designed at the structured-output integration boundary. #309 has therefore been reopened and rewritten around the original customer outcome rather than the rejected implementation.

The removed compiler has not appeared in a released gem, so removing it now avoids publishing an unsound API contract. The actual live Castiron generator also emitted this compiler; its coordinated upstream removal is https://github.com/openai/openai/pull/1291001.

Why retain the other #364 changes

• lib/openai.rb loader detection independently fixes Rails/Tapioca load order and retains its existing regression coverage.
• ArrayOf#to_sorbet_type correctly preserves nullable array elements independently of the deleted compiler; the new isolated subprocess test preserves coverage without making Sorbet a runtime dependency.
• Existing SDK request-model raw-value contract tests remain unchanged, and new public OpenAI::BaseModel tests cover both individual nested fields and nested arrays.

Validation

• Ruby 4.0.6: 695 tests, 3,134 assertions, zero failures/errors/skips.
• Ruby 3.4.10: 695 tests, 3,134 assertions, zero failures/errors/skips.
• Ruby 3.3.12: 695 tests, 3,134 assertions, zero failures/errors/skips.
• bundle exec rake lint: 2,618 RuboCop files clean, Sorbet clean, 1,212 RBS files valid.
• bundle exec rake build:gem; inspected the built gem to verify it contains no Tapioca compiler and does not add sorbet-runtime as a runtime dependency.
• Castiron companion: 574 Ruby renderer tests passed, including assertions that both generation profiles omit the compiler while preserving their intended loader behavior.

Related customer issue: #309. Do not auto-close it: the underlying typed structured-output design remains open.

[openai-sdks]

OkTest Summary

🟡 Running for Ruby SDK PR #405.

View OkTest run #31741832850

_{SDK merge (pending) · head (a75eb55520a3) · base (cbda8bc0e6e3) · OkTest (a845fa206fa4)}

[HAYDEN-OAI]

Reviewed the compiler rollback, transport-model compatibility, structured-output hydration, and optional Sorbet boundary. Looks good.

[HAYDEN-OAI]

Approved after reviewing all six changed files and the coordinated Castiron generator removal. The unreleased compiler can be removed without breaking published APIs; direct model construction and assignment still preserve caller-owned raw values, while both Chat Completions and Responses hydrate nested structured outputs. Nullable-array Sorbet types, optional sorbet-runtime, and Rails/Tapioca loader behavior remain intact.