Skip to content

docs(openrouter-analytics): sync analytics skills with the query builder - #180

Open
devin-ai-integration[bot] wants to merge 5 commits into
mainfrom
devin/1788109516-update-openrouter-analytics-query
Open

devin-ai-integration[bot] wants to merge 5 commits into
mainfrom
devin/1788109516-update-openrouter-analytics-query

Conversation

@devin-ai-integration

@devin-ai-integration devin-ai-integration Bot commented Aug 30, 2026

Copy link
Copy Markdown
Contributor

Summary

Four rounds of analytics skill sync against packages/clickhouse/analytics/ and services/cfw-public-api/src/routes/analytics/ in openrouter-web. Every statement below is taken from a registry, schema constant, migration, or route schema in that source.

Round 1 (commit 7b1f643)time_range on POST /api/v1/analytics/query requires seconds (YYYY-MM-DDTHH:MM:SSZ, fractional allowed); 2026-05-01T00:00Z is rejected with 400.

Round 2 (commits 803e12f, 391dcea) — corrections and additions across the three analytics skills:

- up to 365 days                     # daily-MV cap, stated everywhere
+ up to 367 days                     # ANALYTICS_MAX_RANGE_MS_DAILY_MV = 367 days
- Maximum 10 classifier dimensions   # per query
+ Maximum 2 classifier dimension names   # MAX_CLASSIFIER_DIMENSIONS = 2
- Multiple dimension names → generic clf_dimension_name / clf_dimension_value columns
+ Two dimension names → one column each, cross-grouped
+ dimension_names omitted → generic clf_dimension_name / clf_dimension_value columns
+ data_region (Region), skin (API Skin), session_id      # generations-only dimensions
+ possible_cached_tokens, possible_cache_hit_rate, cache_capture_rate, blended_cost_per_million_tokens
+ include_unset on in / not_in
+ sentinel labels: api_key_id -1 → Chatroom, app -1 → Unknown, max-UUID workspace → Unattributed
+ hour granularity reads the minute MV, so it caps at 31 days
+ legacy all-zero workspace UUID folds into the account's default workspace

Round 3 (commit eed46f2) — server-tool accounting in request counts, following openrouter-web #39256:

+ request_count excludes server_tool_call / server_tool_checkpoint rows on both the MV and raw-generations paths
+ byok_request_count excludes them only on the raw-generations path (`countIf(provider_api_key_id IS NOT NULL AND notServerTool)`);
+   the MV column `byok_requests` still counts every row with a provider key, so the two paths can differ

Round 4 (commit 4812cc6) — latency SLO metrics and server-tool timing exclusions, following openrouter-web#42186, openrouter-web#42209, and openrouter-web#39842:

+ p95_ variant on every performance family (latency, throughput)
+ *_router_latency, *_total_time_to_first_token, *_generation_time families (avg/p50/p90/p95/p99), generations-only
+ *_total_time_to_first_token = latency + router_latency, streamed rows only (non-streamed `latency` is full response time)
+ all performance metrics exclude server-tool rows (`timing()` wraps every aggregate in `notServerTool`)
- blended_cost_per_million_tokens (up to 367 days)
+ blended_cost_per_million_tokens (31-day limit; mv expression removed, server-tool spend excluded from numerator)
+ streamed dimension (generations-only; values "true"/"false")
+ workspace labels resolve deleted workspaces by name (getWorkspacesByIds includeDeleted)

The classifier-column correction is the one an agent could previously get wrong silently: with two names the response has one column per dimension, not the generic EAV pair. The possible-cache metrics resolve to an hourly rollup that carries only model and provider, so any other dimension or filter, or minute granularity, returns 400.

Overlaps with #184 (schema skill only, same latency SLO metrics and streamed); this PR is the superset.

Verification: static review against the source registries, migrations 163/174, and route schemas. No live call was made against the analytics endpoints.

Link to Devin session: https://openrouter.devinenterprise.com/sessions/56737e0e76c84459936257ad4816d4ad
Open in Devin Desktop: https://openrouter.devinenterprise.com/desktop/session/56737e0e76c84459936257ad4816d4ad?variant=devin

…tamps

Co-Authored-By: Devin AI <158243242+devin-ai-integration[bot]@users.noreply.github.com>
@devin-ai-integration

Copy link
Copy Markdown
Contributor Author
Original prompt from Automation

You are the public skills sync pipeline. Your job is to scan PRs merged to main on OpenRouterTeam/openrouter-web in the last 24 hours, determine if any changes warrant updating the public agent skills in OpenRouterTeam/skills, and open PRs to the skills repo when updates are needed.

#``# Step 1: Identify relevant merged PRs

Clone both repos:

cd /home/ubuntu/repos/openrouter-web
git fetch origin main
git log --since='24 hours ago' --merges --first-parent origin/main --pretty=format:'%H|%s' --name-only

For each merged PR, collect the list of changed files. Only PRs that touch the paths in the mapping below are relevant. Skip bot commits (devin-ai-integration[bot], Devin AI, GitHub Actions Bot, openrouter[bot], opencode-agent[bot], OpenRouter SDK Bot, Claude, detail-app[bot], forge-the-devex-intern[bot], dependabot[bot]).

If no relevant PRs were merged, post a short message to the #devrel Slack channel (C08LQFVSC7Q) saying no skill-relevant changes were detected today, and exit.

#``# Step 2: Map changed files to affected skills

Use this mapping to determine which public skills are potentially affected:

openrouter-web paths Affected skill(s) in OpenRouterTeam/skills
sdks/typescript/, packages/llm-interfaces/ openrouter-typescript-sdk, openrouter-agent-migration
services/cfw-*/src/routes/*analytics*, analytics query/response schemas openrouter-analytics, openrouter-analytics-query, openrouter-analytics-schema
Generation routes, /api/v1/generation openrouter-generations
Image generation adapters, packages/image-generation/ openrouter-images
supabase/seeds/, model schemas, /api/v1/models route openrouter-models
OAuth/auth routes, PKCE flow code openrouter-oauth
STT adapters, services/cfw-stt-api/, /api/v1/audio/transcriptions openrouter-stt
TTS routes, /api/v1/audio/speech openrouter-tts
Video adapters, /api/v1/videos openrouter-video
``@`openr... (3383 chars truncated...)

@devin-ai-integration

Copy link
Copy Markdown
Contributor Author

🤖 Devin AI Engineer

I'll be helping with this pull request! Here's what you should know:

✅ I will automatically:

  • Address comments on this PR that start with 'DevinAI' or '@devin'.
  • Look at CI failures and help fix them

Note: I can only respond to comments from users who have write access to this repository.

⚙️ Control Options:

  • Disable automatic comment, CI, and merge conflict monitoring

… query builder

Co-Authored-By: Devin AI <158243242+devin-ai-integration[bot]@users.noreply.github.com>
…its own blockquote paragraph

Co-Authored-By: Devin AI <158243242+devin-ai-integration[bot]@users.noreply.github.com>
@devin-ai-integration devin-ai-integration Bot changed the title docs(openrouter-analytics-query): sync with openrouter-web changes docs(openrouter-analytics): sync analytics skills with the query builder Aug 31, 2026
…est counts

Co-Authored-By: Devin AI <158243242+devin-ai-integration[bot]@users.noreply.github.com>
… dimension, and server-tool exclusions

Co-Authored-By: Devin AI <158243242+devin-ai-integration[bot]@users.noreply.github.com>
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

0 participants