-
Notifications
You must be signed in to change notification settings - Fork 2
feat: improve data/scorer types, again #173
New issue
Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.
By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.
Already on GitHub? Sign in to your account
feat: improve data/scorer types, again #173
Conversation
commit: |
There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.
Pull request overview
This PR simplifies type constraints in the evaluation framework by removing the extends string | Record<string, any> restrictions from EvalTask, CollectionRecord, and EvalParams type parameters. This improves type inference and developer experience by allowing more flexible data structures to flow between eval data and scorers. The PR also adds a comprehensive "kitchen-sink" example demonstrating the full Axiom AI SDK capabilities and updates several example projects with React 19 and dependency upgrades.
Key Changes
- Removed type constraints from eval framework type parameters (
TInput,TExpected,TOutput) to enable better type inference - Added kitchen-sink example with support agent capability showcasing instrumentation, app scope/flags, and comprehensive evals
- Updated React to v19 and dependencies across example projects
Reviewed changes
Copilot reviewed 70 out of 78 changed files in this pull request and generated 3 comments.
Show a summary per file
| File | Description |
|---|---|
| packages/ai/src/evals/eval.types.ts | Simplified type constraints by removing extends string | Record<string, any> from generic parameters |
| packages/ai/src/evals/eval.ts | Updated Eval function overloads to remove OutputOf type and simplify generic parameters |
| pnpm-workspace.yaml | Updated zod catalog version and reorganized dependency order |
| examples/kitchen-sink/* | Added comprehensive kitchen-sink example with support agent capability, evals, and full SDK integration |
| examples/evals-minimal/* | Added minimal eval example demonstrating basic usage patterns |
| examples/evals/* | Removed old evals example in favor of new minimal and kitchen-sink examples |
| examples/telemetry-nextjs-ai-sdk-v4/package.json | Updated React types to v19 |
| examples/telemetry-nextjs-ai-sdk-v5/package.json | Updated React and React types to v19 |
💡 Add Copilot custom instructions for smarter, more guided reviews. Learn how to get started.
Stacks on #169 - please merge that first. If you already want to review this: changes are in
eval.tsandeval.types.ts.Changes
Previously collections were typed
extends string | Record<string, any>This limited the input/expected, and made it harder for types to flow between data and scorers.
This PR simplifies the types, which improves DX both in terms of flexibility and type inference.