Open
Description
Is your feature request related to a problem? Please describe.
Evaluation metrics existing as a separate plugin that needs to be explicitly installed may be a hurdle for user.
Describe the solution you'd like
Some considerations:
- Provide
genkitEvals
plugin as part of the core SDK. (no need to do a separate npm i) - Stretch, make some metrics integrated into the CLI to leverage a common platform. No need to port every metric to every runtime (instant parity). Considerations needed here are how to provide genkit primitives in CLI, or perhaps a workaround that references a CLI-metric from the runtime.
Describe alternatives you've considered
A clear and concise description of any alternative solutions or features you've considered.
Additional context
Add any other context or screenshots about the feature request here.
Metadata
Metadata
Assignees
Labels
Type
Projects
Status
No status