Skip to content

[bot] OpenAI Batch API (client.batches.create) not instrumented (openai + ruby-openai gems) #196

Description

@braintrust-bot

Summary

Both the official openai gem and the community ruby-openai gem expose OpenAI's Batch API (client.batches.create), which submits up to 50,000 queued generation requests (chat completions, responses, embeddings, moderations, images, or videos) for asynchronous processing at reduced cost. This is the OpenAI-gem counterpart to two gaps already tracked in this repo for other providers/gems — Anthropic's Message Batches API (#169) and RubyLLM's RubyLLM.batch (#193) — but the equivalent surface for the openai/ruby-openai gems themselves has no instrumentation at all.

What is missing

OpenAI::Resources::Batches#create(completion_window:, endpoint:, input_file_id:, ...) (official gem) and the equivalent client.batches.create(...) on ruby-openai submit a batch job referencing an uploaded JSONL file of requests targeting one of /v1/responses, /v1/chat/completions, /v1/embeddings, /v1/completions, /v1/moderations, /v1/images/generations, /v1/images/edits, or /v1/videos. This is the generative-execution-initiating call for the batch — no Patcher in this repo wraps it.

batch = client.batches.create(
  completion_window: :"24h",
  endpoint: :"/v1/chat/completions",
  input_file_id: "input_file_id"
)

What a span should capture

  • Input: input_file_id, endpoint (which underlying generative API the batch targets), completion_window
  • Metadata: provider (openai), batch id, status on creation, any request-level metadata param
  • Metrics: request_counts once available (total/completed/failed), latency of the create call

Because results are retrieved asynchronously via file download (a CRUD/retrieval operation, out of scope per this audit), the actionable instrumentation point is the create call itself — matching the pattern already proposed for the Anthropic Messages Batches and RubyLLM.batch gaps in this repo.

Braintrust docs status

not_foundhttps://www.braintrust.dev/docs/integrations/ai-providers/openai documents chat completions, streaming, structured outputs, tool calling, audio transcription, and multimodal content, but does not mention the Batch API in any language.

Upstream sources

Local repo files inspected

  • lib/braintrust/contrib/openai/patcher.rb — defines ChatPatcher, ResponsesPatcher, ModerationsPatcher; no BatchesPatcher
  • lib/braintrust/contrib/openai/instrumentation/ — contains chat.rb, responses.rb, moderations.rb, common.rb; no batches.rb
  • lib/braintrust/contrib/ruby_openai/patcher.rb — same three patchers registered; no batches equivalent
  • lib/braintrust/contrib/ruby_openai/instrumentation/ — same file set as official gem integration; no batches.rb
  • Grep for batch, Batch, Batches across lib/braintrust/contrib/openai/ and lib/braintrust/contrib/ruby_openai/ returns zero matches
  • Issue [bot] Anthropic Messages Batch API (client.messages.batches) not instrumented #169 (Anthropic Messages Batch API) and [bot] RubyLLM provider-side batch completions (RubyLLM.batch) not instrumented #193 (RubyLLM.batch) establish the precedent that batch-submission APIs are in-scope generative execution surfaces for this SDK

Metadata

Metadata

Assignees

No one assigned

    Labels

    No labels
    No labels

    Fields

    No fields configured for Feature.

    Projects

    No projects

    Milestone

    No milestone

    Relationships

    None yet

    Development

    No branches or pull requests

    Issue actions