You signed in with another tab or window. Reload to refresh your session.You signed out in another tab or window. Reload to refresh your session.You switched accounts on another tab or window. Reload to refresh your session.Dismiss alert
Both the official openai gem and the community ruby-openai gem expose OpenAI's Batch API (client.batches.create), which submits up to 50,000 queued generation requests (chat completions, responses, embeddings, moderations, images, or videos) for asynchronous processing at reduced cost. This is the OpenAI-gem counterpart to two gaps already tracked in this repo for other providers/gems — Anthropic's Message Batches API (#169) and RubyLLM's RubyLLM.batch (#193) — but the equivalent surface for the openai/ruby-openai gems themselves has no instrumentation at all.
What is missing
OpenAI::Resources::Batches#create(completion_window:, endpoint:, input_file_id:, ...) (official gem) and the equivalent client.batches.create(...) on ruby-openai submit a batch job referencing an uploaded JSONL file of requests targeting one of /v1/responses, /v1/chat/completions, /v1/embeddings, /v1/completions, /v1/moderations, /v1/images/generations, /v1/images/edits, or /v1/videos. This is the generative-execution-initiating call for the batch — no Patcher in this repo wraps it.
Input: input_file_id, endpoint (which underlying generative API the batch targets), completion_window
Metadata: provider (openai), batch id, status on creation, any request-level metadata param
Metrics: request_counts once available (total/completed/failed), latency of the create call
Because results are retrieved asynchronously via file download (a CRUD/retrieval operation, out of scope per this audit), the actionable instrumentation point is the create call itself — matching the pattern already proposed for the Anthropic Messages Batches and RubyLLM.batch gaps in this repo.
Summary
Both the official
openaigem and the communityruby-openaigem expose OpenAI's Batch API (client.batches.create), which submits up to 50,000 queued generation requests (chat completions, responses, embeddings, moderations, images, or videos) for asynchronous processing at reduced cost. This is the OpenAI-gem counterpart to two gaps already tracked in this repo for other providers/gems — Anthropic's Message Batches API (#169) and RubyLLM'sRubyLLM.batch(#193) — but the equivalent surface for theopenai/ruby-openaigems themselves has no instrumentation at all.What is missing
OpenAI::Resources::Batches#create(completion_window:, endpoint:, input_file_id:, ...)(official gem) and the equivalentclient.batches.create(...)onruby-openaisubmit a batch job referencing an uploaded JSONL file of requests targeting one of/v1/responses,/v1/chat/completions,/v1/embeddings,/v1/completions,/v1/moderations,/v1/images/generations,/v1/images/edits, or/v1/videos. This is the generative-execution-initiating call for the batch — noPatcherin this repo wraps it.What a span should capture
input_file_id,endpoint(which underlying generative API the batch targets),completion_windowopenai), batchid,statuson creation, any request-levelmetadataparamrequest_countsonce available (total/completed/failed), latency of thecreatecallBecause results are retrieved asynchronously via file download (a CRUD/retrieval operation, out of scope per this audit), the actionable instrumentation point is the
createcall itself — matching the pattern already proposed for the Anthropic Messages Batches and RubyLLM.batch gaps in this repo.Braintrust docs status
not_found— https://www.braintrust.dev/docs/integrations/ai-providers/openai documents chat completions, streaming, structured outputs, tool calling, audio transcription, and multimodal content, but does not mention the Batch API in any language.Upstream sources
Batchesresource: https://developers.openai.com/api/reference/ruby/resources/batches/methods/createruby-openaigem (alexrudall), which documents batches support: https://github.com/alexrudall/ruby-openaiLocal repo files inspected
lib/braintrust/contrib/openai/patcher.rb— definesChatPatcher,ResponsesPatcher,ModerationsPatcher; noBatchesPatcherlib/braintrust/contrib/openai/instrumentation/— containschat.rb,responses.rb,moderations.rb,common.rb; nobatches.rblib/braintrust/contrib/ruby_openai/patcher.rb— same three patchers registered; no batches equivalentlib/braintrust/contrib/ruby_openai/instrumentation/— same file set as official gem integration; nobatches.rbbatch,Batch,Batchesacrosslib/braintrust/contrib/openai/andlib/braintrust/contrib/ruby_openai/returns zero matchesclient.messages.batches) not instrumented #169 (Anthropic Messages Batch API) and [bot] RubyLLM provider-side batch completions (RubyLLM.batch) not instrumented #193 (RubyLLM.batch) establish the precedent that batch-submission APIs are in-scope generative execution surfaces for this SDK