Skip to content

[bot] RubyLLM content moderation (RubyLLM.moderate) not instrumented #192

Description

@braintrust-bot

Summary

The ruby_llm gem provides RubyLLM.moderate(input, ...) for AI-powered content moderation. This API is not instrumented by this SDK. The current RubyLLM integration only instruments RubyLLM::Chat. Content moderation uses an AI classification model to screen text and images for harmful content, and is a distinct execution surface from chat completions.

This is a cross-library parity gap: the Braintrust Ruby SDK already instruments OpenAI's moderation API for both the openai gem (OpenAI::Resources::Moderations) and the ruby-openai gem (ModerationsPatcher). The equivalent ruby_llm surface has no instrumentation.

What is missing

RubyLLM.moderate(input, model:, with:, provider:, ...) / RubyLLM::Moderation.moderate(input, ...) — invokes a provider moderation model (e.g., OpenAI's omni-moderation-latest) and returns a RubyLLM::Moderation object containing:

  • id — provider-assigned identifier
  • model — model ID used for moderation
  • results — array of Result objects with category flags and scores

What a span should capture

  • Input: text string(s) and/or image attachments being moderated
  • Metadata: model, provider, endpoint
  • Output: flagged status (flagged?), flagged categories, per-category scores
  • Metrics: time_to_first_token (latency)

The implementation pattern is established: ModerationsPatcher for openai and ruby-openai gems both wrap a single create-style method. A new ModerationPatcher for ruby_llm would wrap Moderation.moderate.

Braintrust docs status

not_found — The Braintrust RubyLLM integration docs at https://www.braintrust.dev/docs/integrations/sdk-integrations/ruby-llm document only chat completions, tool calls, token usage, and streaming. Moderation is not mentioned. The OpenAI integration page does describe moderation support for the openai gem.

Upstream sources

Local repo files inspected

  • lib/braintrust/contrib/ruby_llm/patcher.rb — defines only ChatPatcher; no ModerationPatcher
  • lib/braintrust/contrib/ruby_llm/integration.rb — registers [ChatPatcher] only
  • lib/braintrust/contrib/openai/instrumentation/moderations.rb — existing OpenAI moderation instrumentation (pattern to follow)
  • lib/braintrust/contrib/openai/patcher.rbModerationsPatcher demonstrates the patcher pattern
  • lib/braintrust/contrib/ruby_openai/instrumentation/moderations.rb — ruby-openai moderation instrumentation
  • Grep for moderate, Moderation, ModerationPatcher across lib/braintrust/contrib/ruby_llm/ returns zero matches

Metadata

Metadata

Assignees

No one assigned

    Labels

    No labels
    No labels

    Fields

    No fields configured for Feature.

    Projects

    No projects

    Milestone

    No milestone

    Relationships

    None yet

    Development

    No branches or pull requests

    Issue actions