v0.4.0

What's Changed

• Add classifier support by Stephen Belanger (@Qard) in #154
• feat: Upwards-recursively read .braintrust.json containing BRAINTRUST_API_KEY by Luca Forstner (@lforst) in #166

Full Changelog: v0.3.2...v0.4.0

v0.3.2

Changelog

• fix(deps): allow openssl 4.x for compatibility (#151) (b5dcaa9)

v0.3.1

Changelog

• Added support for Ruby 4.0 (#129)
• Added parameter support for remote evals (#133)
• Added project_id support to Functions#list and Prompt.load (#131) (Nick Giancola (@patbenatar))
• Added Prompt#version (#131) (Nick Giancola (@patbenatar))
• Addressed some Eval edge cases with improved defaults (#127)
• Some internal refactors for gemspec and Evals (#128)

v0.3.0

Changelog

• Added a Rails engine for mounting a remote eval server onto your existing Rails applications (#87 #114) (Gabriel Kent (@gabe-kent), Alan R. Fachini (@alfakini))
• Scorers can return metadata tags as a part of their result (#115, #120) (Alan R. Fachini (@alfakini))
• Scorers can return multiple scores from a single scorer (#125)
• Adjustments to eval traces to make them consistent with ones from other SDKs (#119, #121)
• Fixed filter_ai_spans being bypassed for root spans (#123, #124) (Chidi Williams (@chidiwilliams))

v0.2.1

Changelog

• Added support for trace: in Scorer blocks: this allows you to score an Eval case by looking at the LLM activity during the task, not just the input or output. (#112)
• A Scorer can now be retrieved by its name + project name (not just by ID) (#113)
• Scorer and Task now support keyword arguments for their blocks (e.g. |output:, expected:| instead of |input, output expected|) to allow for more concise expressions. (#111)
  • Using positional arguments are still supported but will produce a deprecation warning.
• Scorer and Task are now independent and accessible outside of Eval (#111)
  • The scorer and task functions in the Eval namespace are still supported will produce a deprecation warning.

v0.2.0

Changelog

• Added dev server support for running remote evals in Ruby applications. See setup instructions for more details. A special thanks to Alan R. Fachini (@alfakini) and Gabriel Kent (@gabe-kent) for their contributions to this new feature! (#87, #108) (Alan R. Fachini (@alfakini) Gabriel Kent (@gabe-kent))

v0.1.4

Changelog

• Scorers using invoke api now support objects instead of just strings (#104) (Barrett Pyke (@barrettpyke) Almir Davletov (@davletovalmir))
• Fixed API calls not following redirects for Evals (#101 #106) (Alan R. Fachini (@alfakini))
• Link experiments with datasets when using remote datasets (#103) (Alan R. Fachini (@alfakini))

v0.1.3

Changelog

• Added origin tagging to Eval datasets (for proper attribution) (#97)
• Datasets are now a first-class API (#98)
• Evals no longer accept state, use api objects instead (#98)
• Fixed Anthropic instrumentation not recording system prompts correctly (#100) (Jessica Alder (@jessicaalder-lgtm))

v0.1.2

Changelog

• Added support for prompts (#70, #77)

v0.1.1

Changelog

• Added moderations API support for openai and ruby-openai (#89, #92)
• Added support for anthropic messages beta API (#90, #93)
  • This is "best effort" support only. Since the Anthropic API is "beta" and may change between minor versions, instrumentation may break unexpectedly and support for beta features may be limited.
• The tracer now attempts to flush traces at the end of the process by default. Useful for short-lived processes like lambdas. (#91)