Sources for Eval Driven Development on the JVM talk

Conference talk repo demonstrating Eval-Driven Development (EDD) on the JVM using the Dokimos framework.

• Dokimos overview: https://dokimos.dev/overview
• Main idea: treat evals as first-class tests, run them locally, and inspect results in the Dokimos statistics dashboard.

---

Modules Overview

1) spring-ai/ (Spring Boot + Spring AI + Dokimos evals)

A Kotlin/Spring Boot app that:

• Exposes chat endpoints (text + audio) backed by OpenAI via Spring AI
• Uses pgvector for retrieval (conference session data)
• Emits traces to Langfuse via OpenTelemetry (optional but recommended)
• Contains Dokimos eval tests under spring-ai/src/test/kotlin/... (see e.g. dev.example.edd.ChatEval)

Key configuration lives in spring-ai/src/main/resources/application.properties (port 8082, pgvector on localhost:5430, etc.).

2) chatclient-kmp/ (Kotlin Multiplatform desktop client)

A Compose Desktop client (JVM target) that talks to the spring-ai server.

• Text chat UI
• Audio chat UI (records/uploads audio, plays back TTS)

3) docker-compose.yaml (pgvector)

A small Docker Compose setup that starts Postgres + pgvector on localhost:5430 and initializes the schema/data via docker-entrypoint-initdb.d/.

4) Dokimos Eval statistics server (external)

Dokimos provides a small server + dashboard to collect and visualize eval results.

---

Prerequisites

• Java 25 (both modules target JVM 25)
• Docker (for pgvector + Dokimos statistics server)
• An OpenAI API key
• (Optional but recommended) a free Langfuse Cloud account

---

Environment variables (required for spring-ai/)

Set these in your shell before running the Spring Boot app or the eval tests.

export OPENAI_API_KEY=<open-ai-key>

# Langfuse (requires a free account on https://cloud.langfuse.com/)
export LANGFUSE_BASE_URL=https://cloud.langfuse.com
export LANGFUSE_PUBLIC_KEY=<pk>
export LANGFUSE_SECRET_KEY=<sk>

# OpenTelemetry export to Langfuse
export OTEL_EXPORTER_OTLP_HEADERS='Authorization=Basic <pk:sk base64>'
export OTEL_EXPORTER_OTLP_TRACES_ENDPOINT=https://cloud.langfuse.com/api/public/otel/v1/traces

Notes:

• OPENAI_API_KEY is mandatory because spring-ai/src/main/resources/application.properties references it.
• To build the Basic header value, base64-encode the literal string "<pk>:<sk>".
• Traces are exported only when OTEL_EXPORTER_OTLP_TRACES_ENDPOINT is set. Spring Boot 4 ignores the OTEL_* variables, so observability/LangfuseTracingConfig.kt passes them to its exporter and adds the x-langfuse-ingestion-version: 4 header for Langfuse's v4 ingestion path. OTEL_EXPORTER_OTLP_ENDPOINT is not needed.
• Langfuse v4 queries observations directly, so the app copies the session ID and trace name onto every span and puts each request's input and output on the root observation (observability/LangfuseTraceAttributes.kt).
• The feedback-triage flow reads Langfuse through the Observations API v2 (langfuse/LangfuseFeedbackClient.kt).

---

Running the infrastructure

1) Start pgvector (for RAG)

From the repo root:

docker compose up -d

This uses ./docker-compose.yaml and exposes Postgres on localhost:5430.

2) Start Dokimos Eval statistics server (dashboard)

The Dokimos server compose file is vendored in dokimos/. From the repo root:

cd dokimos && docker compose up -d

Open http://localhost:8080 to view the dashboard.

---

Running spring-ai/

Start the Spring Boot app

From spring-ai/:

./mvnw spring-boot:run

The app starts on:

• http://localhost:8082

Useful endpoints

The controller is dev.example.AIController.

• POST /chat (JSON) – text chat
• POST /audio-chat (multipart) – audio in, audio out (TTS)
• POST /audio-in-text-out-chat (multipart) – audio in, text out
• POST /feedback (JSON) – user feedback

---

Running the Dokimos evals

The evals live under spring-ai/src/test/kotlin/.

A representative suite is dev.example.edd.ChatEval:

• Uses Dokimos experiment { ... } DSL
• Optionally reports results to the Dokimos statistics server at http://localhost:8080

Run tests from spring-ai/:

./mvnw test

Notes:

• Some tests are guarded with @EnabledIfEnvironmentVariable(named = "OPENAI_API_KEY", ...).
• For dashboard reporting, make sure the Dokimos statistics server is running (see above).

Dokimos skills for Claude Code

The Dokimos Claude Code plugins are enabled for this repo in .claude/settings.json. When you open the repo in Claude Code and trust the folder, it offers to add the dokimos marketplace and install them:

• Scaffolding: create-evaluator, create-dataset, create-tests, create-experiment, generate-goldens
• Integration: evaluate-agent, evaluate-spring-ai, evaluate-koog, evaluate-langchain4j, evaluate-openai, evaluate-embabel, evaluate-spring-ai-alibaba

To install them by hand instead:

/plugin marketplace add dokimos-dev/dokimos
/plugin install evaluate-spring-ai@dokimos

---

Running chatclient-kmp/ (desktop client)

From chatclient-kmp/:

./gradlew run

The UI expects the spring-ai server to be running on http://localhost:8082.

---