Skip to content

[bot] OpenAI legacy text completions API (client.completions) not instrumented #189

Description

@braintrust-bot

Summary

The OpenAI legacy text completions API (/v1/completions) is available in both the official openai gem and the ruby-openai gem but is not instrumented. This is the original OpenAI generation API, still used today for fine-tuned instruct models (gpt-3.5-turbo-instruct, babbage-002, davinci-002) and other text completion workflows. It is distinct from the chat completions API (/v1/chat/completions), which is already instrumented.

What is missing

Official openai gem

OpenAI::Resources::Completions exposes two generative execution methods, neither of which is patched:

  • client.completions.create(model:, prompt:, max_tokens:, temperature:, top_p:, n:, stop:, ...) — Synchronous text completion; returns a Completion object with choices[].text, usage, model, and id.
  • client.completions.create_streaming(model:, prompt:, ...) — Streaming variant; yields raw SSE chunks aggregating into the same final structure.

ruby-openai gem

OpenAI::Client exposes the equivalent via:

  • client.completions(parameters: { model:, prompt:, max_tokens:, ... }) — Calls /v1/completions directly. Accepts the same parameters as the official SDK.

What a span should capture

  • Input: the prompt (text string or array of strings/tokens)
  • Metadata: model, max_tokens, temperature, top_p, n, stop, frequency_penalty, presence_penalty, seed, logprobs, provider, endpoint (/v1/completions)
  • Output: choices[].text and finish_reason for each completion choice
  • Metrics: prompt_tokens, completion_tokens, total_tokens from the usage object; time_to_first_token

The pattern is identical to the existing ModerationsPatcher — a new CompletionsPatcher wrapping create() and create_streaming() on the respective resource class.

Braintrust docs status

not_found — The Braintrust OpenAI integration docs at https://www.braintrust.dev/docs/integrations/ai-providers/openai and the tracing guide at https://www.braintrust.dev/docs/guides/tracing focus exclusively on chat completions. The legacy /v1/completions endpoint is not mentioned.

Upstream sources

Local repo files inspected

  • lib/braintrust/contrib/openai/patcher.rb — defines ChatPatcher, ResponsesPatcher, ModerationsPatcher; no CompletionsPatcher for OpenAI::Resources::Completions
  • lib/braintrust/contrib/openai/instrumentation/ — contains chat.rb, responses.rb, moderations.rb, common.rb; no completions.rb
  • lib/braintrust/contrib/ruby_openai/patcher.rb — defines ChatPatcher, ResponsesPatcher, ModerationsPatcher; no CompletionsPatcher
  • lib/braintrust/contrib/ruby_openai/instrumentation/ — contains chat.rb, responses.rb, moderations.rb, common.rb; no completions.rb
  • test/fixtures/vcr_cassettes/alexrudall_ruby_openai/completions.yml — recorded cassette showing a POST https://api.openai.com/v1/completions call with gpt-3.5-turbo-instruct, confirming the endpoint is accessible via the ruby-openai gem
  • Grep for Completions (outside of Chat::Completions), instruct, /v1/completions across lib/braintrust/ returns zero instrumentation matches

Metadata

Metadata

Assignees

No one assigned

    Labels

    No labels
    No labels

    Fields

    No fields configured for Feature.

    Projects

    No projects

    Milestone

    No milestone

    Relationships

    None yet

    Development

    No branches or pull requests

    Issue actions