Skip to content

Conversation

@c-ehrlich
Copy link
Collaborator

Stacks on #169 - please merge that first. If you already want to review this: changes are in eval.ts and eval.types.ts.

Changes

Previously collections were typed extends string | Record<string, any>

This limited the input/expected, and made it harder for types to flow between data and scorers.

This PR simplifies the types, which improves DX both in terms of flexibility and type inference.

Copilot AI review requested due to automatic review settings November 28, 2025 05:16
@pkg-pr-new
Copy link

pkg-pr-new bot commented Nov 28, 2025

Open in StackBlitz

npm i https://pkg.pr.new/axiomhq/ai/axiom@173

commit: 5b91039

Copilot finished reviewing on behalf of c-ehrlich November 28, 2025 05:20
Copy link
Contributor

Copilot AI left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Pull request overview

This PR simplifies type constraints in the evaluation framework by removing the extends string | Record<string, any> restrictions from EvalTask, CollectionRecord, and EvalParams type parameters. This improves type inference and developer experience by allowing more flexible data structures to flow between eval data and scorers. The PR also adds a comprehensive "kitchen-sink" example demonstrating the full Axiom AI SDK capabilities and updates several example projects with React 19 and dependency upgrades.

Key Changes

  • Removed type constraints from eval framework type parameters (TInput, TExpected, TOutput) to enable better type inference
  • Added kitchen-sink example with support agent capability showcasing instrumentation, app scope/flags, and comprehensive evals
  • Updated React to v19 and dependencies across example projects

Reviewed changes

Copilot reviewed 70 out of 78 changed files in this pull request and generated 3 comments.

Show a summary per file
File Description
packages/ai/src/evals/eval.types.ts Simplified type constraints by removing extends string | Record<string, any> from generic parameters
packages/ai/src/evals/eval.ts Updated Eval function overloads to remove OutputOf type and simplify generic parameters
pnpm-workspace.yaml Updated zod catalog version and reorganized dependency order
examples/kitchen-sink/* Added comprehensive kitchen-sink example with support agent capability, evals, and full SDK integration
examples/evals-minimal/* Added minimal eval example demonstrating basic usage patterns
examples/evals/* Removed old evals example in favor of new minimal and kitchen-sink examples
examples/telemetry-nextjs-ai-sdk-v4/package.json Updated React types to v19
examples/telemetry-nextjs-ai-sdk-v5/package.json Updated React and React types to v19

💡 Add Copilot custom instructions for smarter, more guided reviews. Learn how to get started.

@c-ehrlich c-ehrlich merged commit eeaeb6b into main Dec 1, 2025
6 checks passed
@c-ehrlich c-ehrlich deleted the cje/ai-113-scorers-shouldnt-be-string-recordstring-any branch December 1, 2025 12:59
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

2 participants