feat(go): Schema definition system with dotprompt #2793

ihan211 · 2025-04-22T00:02:16Z

This PR adds a schema definition system to genkit with Dotprompt reference support.
Key Components

Schema registration with Dotprompt framework
DefineSchema API for defining prompt data structures
Enhanced initialization process for schema registration
Schema resolution within templates

Example Usage

// Define a product schema
type ProductSchema struct {
Name string json:"name"
Description string json:"description"
Price float64 json:"price"
Category string json:"category"
InStock bool json:"inStock"
}

// Register the schema with genkit
genkit.DefineSchema("ProductSchema", ProductSchema{})

// Reference the schema in a Dotprompt template
// promptContent example:
// ---
// input:
// schema:
// theme: string
// output:
// schema: ProductSchema
// ---
// Generate a product that fits the {{theme}} theme.
// Make sure to provide a detailed description and appropriate pricing.

The implementation enables:

Automatic schema registration during Genkit initialization
Structured output parsing from model responses to Go structs
Schema reference resolution within Dotprompt templates
JSON Schema support for standardized type definitions

Testing
Unit tests in schema_test.go
Working example in samples directory

Checklist (if applicable):

PR title is following https://www.conventionalcommits.org/en/v1.0.0/
Tested (manually, unit tested, etc.)
Docs updated (updated docs or a docs bug requir

…d improve code organization - Move pendingSchemas management to core package to avoid validation duplication - Add RegisterGlobalSchemaResolver that follows DRY by calling registerSchemaResolver - Create recursive helper function for field-to-schema conversion to support nested structs - Fix import cycles between genkit, core, and registry packages - Standardize schema path construction with consistent separators - Add functions to retrieve and clear pending schemas - Update schema registration during initialization process - Fix LookupPrompt function signature to match usage This refactoring addresses code review feedback by: 1. Eliminating duplicate validation between genkit.DefineSchema and core.RegisterSchema 2. Following DRY principle for schema resolver registration 3. Improving support for nested struct types with recursive field conversion 4. Creating a cleaner architecture with proper separation of concerns

…lace directive

…ons and improve naming convention

hugoaguirre · 2025-04-24T18:40:10Z

go/core/schema.go

+// Package core provides core functionality for the genkit framework.
+
+package core


Suggested change

// Package core provides core functionality for the genkit framework.

package core

// Package core provides core functionality for the Genkit framework.

package core

hugoaguirre · 2025-04-24T18:44:36Z

go/core/schema.go

+
+// ClearSchemas removes all registered schemas.
+// This is primarily for testing purposes.
+func ClearSchemas() {


Where's this function used in the framework? if it's for testing purposes, shouldn't this be defined as a helper function in a test file?

moved to test file.

hugoaguirre · 2025-04-24T18:49:14Z

go/core/schema.go

+// Returns the schema for convenience in chaining operations.
+func RegisterSchema(name string, schema any) Schema {
+	if name == "" {
+		panic("core.RegisterSchema: schema name cannot be empty")


I believe we can replace these panic() calls with errors since these are recoverable errors

hugoaguirre · 2025-04-24T18:51:50Z

go/genkit/genkit.go

@@ -245,6 +250,22 @@ func Init(ctx context.Context, opts ...GenkitOption) (*Genkit, error) {
 	return g, nil
 }

+// Internal function called during Init to register pending schemas
+func registerPendingSchemas(reg *registry.Registry) error {
+	// Get pending schemas from core


nit: the code is pretty clear, we should avoid these kind of comments, same for line 264

hugoaguirre · 2025-04-24T18:52:32Z

go/genkit/schema.go

+//	personSchema := genkit.DefineSchema("Person", Person{})
+func DefineSchema(name string, schema Schema) Schema {
+	if name == "" {
+		panic("genkit.DefineSchema: schema name cannot be empty")


We should not panic, these errors are recoverable. Try to return a (Schema, error) pair

hugoaguirre · 2025-04-24T19:00:30Z

go/genkit/schema.go

+		return jsonSchema
+	}
+
+	// Register the resolver with Dotprompt


we can ignore this comment, same for the one at line 112

hugoaguirre · 2025-04-24T19:04:48Z

go/genkit/schema.go

+	schemaResolver := func(name string) any {
+		schema, exists := LookupSchema(name)
+		if !exists {
+			fmt.Printf("Schema '%s' not found in registry\n", name)


Use slog to log these kind of messages, here's an example of how it is used:

slog.Error("schema not found in registry", "name", name)

hugoaguirre · 2025-04-24T19:06:30Z

go/internal/registry/schema.go

+
+	r.Dotprompt.DefineSchema(name, jsonSchema)
+	r.RegisterValue(SchemaType+"/"+name, structType)
+	fmt.Printf("Registered schema '%s' with registry and Dotprompt\n", name)


If these logs are not for debugging purposes, use slog instead, otherwise I think you should remove them

hugoaguirre · 2025-04-24T19:08:20Z

go/internal/registry/schema.go

+}
+
+// DumpRegistrySchemas prints all schemas stored in the registry
+func (r *Registry) DumpRegistrySchemas() {


I cannot find where this function is used, why is it needed?

hugoaguirre · 2025-04-24T19:10:34Z

go/internal/registry/schema.go

+
+// convertStructToJsonSchema converts a Go struct to a JSON schema
+func convertStructToJsonSchema(structType any) (*jsonschema.Schema, error) {
+	fmt.Printf("Converting schema of type %T to JSON Schema\n", structType)


nit: not needed, we can remove this

- Move ClearSchemas to schema_test.go as clearSchemasForTest() - Replace panic with error returns in RegisterSchema and DefineSchema - Improve logging using structured slog instead of fmt.Printf - Remove debug prints and unused functions - Add proper error handling for schema not found cases

ihan211 added 2 commits April 15, 2025 17:52

feat(go/genkit): add DefineSchema and Dotprompt reference support

89b9518

github-project-automation bot added this to Genkit Backlog Apr 22, 2025

github-actions bot added the go label Apr 22, 2025

ihan211 self-assigned this Apr 22, 2025

ihan211 requested review from AbeJLazaro and hugoaguirre April 22, 2025 00:29

ihan211 force-pushed the ivenh/2702 branch from 0a7087c to e05a81c Compare April 22, 2025 01:10

feat(go/genkit): Fix dependency conflict with dotprompt by adding rep…

e18f609

…lace directive

ihan211 force-pushed the ivenh/2702 branch from e05a81c to e18f609 Compare April 22, 2025 16:44

AbeJLazaro changed the title ~~This PR adds a schema definition system to genkit with Dotprompt reference support.~~ feat(go): Schema definition system with dotprompt Apr 22, 2025

AbeJLazaro approved these changes Apr 22, 2025

View reviewed changes

ihan211 added 2 commits April 22, 2025 13:59

Update dotprompt dependency to use add-schema-support branch

50d7d6e

feat(go/genkit): Refactor schema registry: rename Get prefixed functi…

0aefcf3

…ons and improve naming convention

hugoaguirre requested changes Apr 24, 2025

View reviewed changes

Provide feedback

Saved searches

Use saved searches to filter your results more quickly

feat(go): Schema definition system with dotprompt #2793

feat(go): Schema definition system with dotprompt #2793

ihan211 commented Apr 22, 2025

hugoaguirre Apr 24, 2025

ihan211 Apr 25, 2025

hugoaguirre Apr 24, 2025

ihan211 Apr 25, 2025

hugoaguirre Apr 24, 2025

ihan211 Apr 25, 2025

hugoaguirre Apr 24, 2025

ihan211 Apr 25, 2025

hugoaguirre Apr 24, 2025

ihan211 Apr 25, 2025

hugoaguirre Apr 24, 2025

hugoaguirre Apr 24, 2025

ihan211 Apr 25, 2025

hugoaguirre Apr 24, 2025

ihan211 Apr 25, 2025

hugoaguirre Apr 24, 2025

ihan211 Apr 25, 2025

hugoaguirre Apr 24, 2025

ihan211 Apr 25, 2025

		// Package core provides core functionality for the genkit framework.

		package core

feat(go): Schema definition system with dotprompt #2793

Are you sure you want to change the base?

feat(go): Schema definition system with dotprompt #2793

Conversation

ihan211 commented Apr 22, 2025

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment