Skip to content

Add telemetry and cancellable inference - #10

Merged
TorstenDittmann merged 2 commits into
mainfrom
feat/telemetry-cancellation
Sep 8, 2026
Merged

Add telemetry and cancellable inference#10
TorstenDittmann merged 2 commits into
mainfrom
feat/telemetry-cancellation

Conversation

@TorstenDittmann

Copy link
Copy Markdown
Contributor

Summary

  • add structured JSON request logs, validated request IDs, and Prometheus HTTP/pipeline/runtime metrics
  • propagate request deadlines and client cancellation into ONNX Runtime with per-run cancellation options
  • make shutdown deterministic with draining readiness, request rejection, a configurable grace period, and forced inference cancellation
  • split liveness and readiness endpoints while retaining /healthz compatibility
  • embed release/build metadata and document the operational contract

Validation

  • go test -race ./...
  • go vet ./...
  • make build
  • npm run build --prefix docs
  • git diff --check

Docker validation could not run locally because the Docker daemon is not running; the existing CI matrix builds both amd64 and arm64 images. The integration suite will exercise real ONNX inference and the new cancellation assertion in CI.

@greptile-apps

greptile-apps Bot commented Sep 8, 2026

Copy link
Copy Markdown

Greptile Summary

The latest changes complete the telemetry and cancellation fixes requested by the previous review.

  • Moves request instrumentation to the server boundary so /metrics and unmatched requests receive request IDs, logs, and HTTP metrics.
  • Checks request cancellation after inference and focal-point calculation before reporting success.
  • Adds focused regression tests for post-inference cancellation and unmatched-route telemetry.

Confidence Score: 5/5

The PR appears safe to merge; both previous findings are resolved and the latest changes introduce no accepted new issue.

The handler now prevents successful responses after request cancellation, and server-boundary instrumentation covers registered, metrics, and unmatched routes without introducing an established functional failure.

Important Files Changed

Filename Overview
cmd/autogravity/main.go Wraps the complete HTTP mux with telemetry and adds cancellation checks around post-inference processing.
cmd/autogravity/main_test.go Adds regression coverage for cancellation after inference and instrumentation of unmatched routes.
cmd/autogravity/telemetry.go Derives bounded route labels when telemetry is applied at the server boundary.

Reviews (2): Last reviewed commit: "fix: cover post-inference cancellation a..." | Re-trigger Greptile

Comment thread cmd/autogravity/main.go
Comment thread cmd/autogravity/main.go
@TorstenDittmann
TorstenDittmann merged commit 79fc1c0 into main Sep 8, 2026
4 checks passed
@TorstenDittmann
TorstenDittmann deleted the feat/telemetry-cancellation branch September 8, 2026 09:07
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

2 participants