Skip to content

Commit bf91e2c

Browse files
itssimonitssimon
committed
Improve skill
1 parent 074d5cc commit bf91e2c

3 files changed

Lines changed: 48 additions & 91 deletions

File tree

‎AGENTS.md‎

Lines changed: 7 additions & 6 deletions
Original file line numberDiff line numberDiff line change
@@ -2,7 +2,7 @@
22

33
## Product Overview
44

5-
[Apitally](https://apitally.io) is an API monitoring and analytics tool. This is a CLI tool for humans and AI agents. It retrieves data from the Apitally API and outputs it in NDJSON format, or optionally stores it in a local DuckDB database and allows running arbitrary SQL queries against it.
5+
[Apitally](https://apitally.io) is an API monitoring and analytics tool. This is a CLI tool for AI agents and humans. It retrieves data from the Apitally API and outputs it in NDJSON format, or optionally stores it in a local DuckDB database and allows running arbitrary SQL queries against it.
66

77
## Repository Structure
88

@@ -20,6 +20,10 @@ src/
2020
npm/
2121
cli.js Thin wrapper that spawns the native binary
2222
install.js postinstall script that downloads the correct binary
23+
skills/
24+
apitally-cli/ Agent skill: guides AI agents through using the CLI to investigate API issues
25+
SKILL.md Workflow, command reference, investigation patterns, SQL examples
26+
references/ Detailed command and table schema references
2327
```
2428

2529
## Tech Stack
@@ -82,7 +86,7 @@ Triggered by publishing a GitHub release. Two jobs:
8286

8387
### Module `run()` functions
8488

85-
Each command module exposes a `pub fn run(...)` that does all the work: resolve auth, call the API, write output. The last parameter is always `impl Write` (stdout in production, `Vec<u8>` in tests). Data goes to the writer (stdout), human messages and progress go to stderr.
89+
Each command module exposes a `pub fn run(...)` that does all the work: resolve auth, call the API, write output. The last parameter is `impl Write` (stdout in production, `Vec<u8>` in tests), except `auth::run` which takes `&mut impl Read` (stdin) instead. Data goes to the writer (stdout), human messages and progress go to stderr.
8690

8791
### Error handling and exit codes
8892

@@ -100,10 +104,6 @@ Exit code 2 comes from clap (usage errors). `check_response` in `utils.rs` centr
100104

101105
API requests use `api_get`/`api_post` helpers in `utils.rs`. API key is sent as the `Api-Key` header.
102106

103-
### request-logs dual format
104-
105-
The `request-logs` command sends `format: "ndjson"` or `format: "arrow"` depending on whether `--db` is given. NDJSON mode is a raw passthrough (`io::copy` from response to stdout). Arrow mode streams Arrow IPC batches into a DuckDB staging table, then does `INSERT OR REPLACE` into the final table.
106-
107107
## Tests
108108

109109
Inline `#[cfg(test)] mod tests` in each source file. Run with `cargo test`.
@@ -123,3 +123,4 @@ Inline `#[cfg(test)] mod tests` in each source file. Run with `cargo test`.
123123
- Verify work compiles and passes `cargo clippy` before considering a task complete
124124
- Keep user-facing output concise
125125
- Use inline comments sparingly — only to explain non-obvious intent
126+
- When changing CLI commands, flags, output format, or DuckDB schemas, update the agent skill in `skills/apitally-cli/` (including its `references/` docs) to reflect those changes

‎skills/apitally-cli/SKILL.md‎

Lines changed: 27 additions & 71 deletions
Original file line numberDiff line numberDiff line change
@@ -11,25 +11,45 @@ description: >
1111

1212
# Apitally CLI
1313

14-
The Apitally CLI retrieves API request log data from [Apitally](https://apitally.io) and optionally stores it in a local DuckDB database for investigation with SQL. Each record is an individual API request with method, URL, status code, response time, consumer, headers, payloads, exceptions, and more. Request log retention is **15 days**. Older data is not available.
14+
The Apitally CLI retrieves API request log data from [Apitally](https://apitally.io) and optionally stores it in a local DuckDB database for investigation with SQL. Each record is an individual API request with method, URL, status code, response time, consumer, headers, payloads, exceptions, and more. Request log retention is **15 days**.
1515

16-
Run commands with npx (no install needed):
16+
Run commands with `npx` (no install needed):
1717

1818
```
1919
npx @apitally/cli <command> [--api-key <key>]
2020
```
2121

22+
A team-scoped API key is required to use the CLI. The `auth` command writes the provided API key to `~/.apitally/auth.json`. It's then used by all subsequent commands unless overridden by the `--api-key` flag.
23+
2224
All commands output NDJSON to stdout by default. With `--db`, data is written to a DuckDB database instead (`~/.apitally/data.duckdb` by default), enabling SQL queries via the `sql` command.
2325

24-
A team-scoped API key is required to use the CLI. The `auth` command writes the provided API key to `~/.apitally/auth.json`. It's then used by all subsequent commands unless overridden by the `--api-key` flag.
26+
## Key Concepts
27+
28+
- **App and environment** — An app is a monitored API application, identified by a numeric `app_id`. Each app has one or more environments (e.g. "prod", "dev"). The `env` field on request logs is a string matching the environment name.
29+
- **Consumer** — An API client or user tracked by Apitally. `consumer_id` is a numeric internal ID (surrogate key, used in request log filters and JOINs). `identifier` is a string set by the application (e.g. email, username) to uniquely identify the consumer. `name` is a display name (auto-generated from `identifier` if not explicitly set). `group` is an optional group name.
30+
- **Path vs URL** — `path` is the parameterized route template (e.g. `/users/{user_id}`), good for grouping by endpoint. `url` is the full request URL with actual values and query parameters (e.g. `https://api.example.com/users/123?limit=10`).
31+
- **Application logs** — Server-side log entries emitted by application code during request handling. Only available via `request-details` as the `logs` field.
32+
- **Spans** — OpenTelemetry trace spans representing units of work during request handling (e.g. database queries, external API calls, instrumented function calls). Only available via `request-details`. Form a tree via `parent_span_id`.
33+
34+
## Command Quick Reference
35+
36+
All commands are run via `npx @apitally/cli <command>`. For full details, see [references/commands.md](references/commands.md).
37+
38+
- `auth [--api-key <key>]` -- configure API key
39+
- `whoami` -- check auth, show team
40+
- `apps [--db [<path>]]` -- list apps (get app IDs)
41+
- `consumers <app-id> [--requests-since <dt>] [--db [<path>]]` -- list consumers for an app (get consumer IDs)
42+
- `request-logs <app-id> --since <dt> [--until <dt>] [--fields <json>] [--filters <json>] [--limit <n>] [--db [<path>]]` -- fetch request logs (max 1,000,000 rows at once)
43+
- `request-details <app-id> <request-uuid> [--db [<path>]]` -- fetch full details for a single request (including headers, payloads, exception info, application logs, and spans)
44+
- `sql "<query>" [--db <path>]` -- run SQL against local DuckDB
2545

2646
## Workflow
2747

28-
1. **Check authentication** -- run `npx @apitally/cli whoami`. If it fails, ask the user to provide their Apitally API key or run `npx @apitally/cli auth` themselves. Explain that API keys can be created in the Apitally dashboard under Settings > API keys (https://app.apitally.io/settings/api-keys). If the user provides a key, run `npx @apitally/cli auth --api-key <key>` to store it.
48+
1. **Check authentication** — run `npx @apitally/cli whoami`. If it fails, ask the user to provide their Apitally API key or run `npx @apitally/cli auth` themselves. Explain that API keys can be created in the Apitally dashboard under Settings > API keys (https://app.apitally.io/settings/api-keys). If the user provides a key, run `npx @apitally/cli auth --api-key <key>` to store it.
2949

30-
2. **Identify the app** -- run `npx @apitally/cli apps` to list apps and get their IDs. If there is only one app, use it. Otherwise, if the correct app can't be inferred from the user's previous messages, ask the user which app they mean.
50+
2. **Identify the app** — run `npx @apitally/cli apps` to list apps and get their IDs. If there is more than one app, and the correct app can't be inferred from the user's messages, ask the user which app they mean.
3151

32-
3. **Determine if consumers are involved** -- decide which scenario applies:
52+
3. **Determine if consumers are involved** — decide which scenario applies:
3353
- **(a) Specific consumer(s)**: the user is asking about specific consumers (e.g. by email, name, or group). Fetch consumers first, then query to find the matching `consumer_id`, then use it as a filter when fetching request logs.
3454
- **(b) Consumer context needed**: the investigation involves consumers but not specific ones known upfront (e.g. "which consumers cause the most errors"). Fetch consumers into DuckDB for later JOINs with request logs.
3555
- **(c) No consumer involvement**: skip fetching consumers.
@@ -65,66 +85,12 @@ A team-scoped API key is required to use the CLI. The `auth` command writes the
6585
npx @apitally/cli sql "SELECT method, path, status_code, COUNT(*) as n FROM request_logs WHERE status_code >= 400 GROUP BY ALL ORDER BY n DESC"
6686
```
6787

68-
7. **Iterate** -- refine filters, fetch additional fields (headers, bodies, exceptions), or widen the time range as needed
69-
70-
## Command Quick Reference
71-
72-
All commands are run via `npx @apitally/cli <command>`. For full details, see [references/commands.md](references/commands.md).
73-
74-
- `auth [--api-key <key>]` -- configure API key
75-
- `whoami` -- check auth, show team
76-
- `apps [--db [<path>]]` -- list apps (get app IDs)
77-
- `consumers <app-id> [--requests-since <dt>] [--db [<path>]]` -- list consumers for an app
78-
- `request-logs <app-id> --since <dt> [--until <dt>] [--fields <json>] [--filters <json>] [--limit <n>] [--db [<path>]]` -- fetch request logs (max 1,000,000 rows at once)
79-
- `request-details <app-id> <request-uuid> [--db [<path>]]` -- fetch full details for a single request (headers, body, logs, spans)
80-
- `sql "<query>" [--db <path>]` -- run SQL against local DuckDB
88+
7. **Iterate** — refine filters, fetch additional fields (headers, bodies, exceptions), or widen the time range as needed.
8189

8290
## Investigation Patterns
8391

8492
For DuckDB table schemas, see [references/tables.md](references/tables.md).
8593

86-
### Find failing requests
87-
88-
```sql
89-
SELECT timestamp, method, path, status_code, response_time_ms, client_ip
90-
FROM request_logs
91-
WHERE status_code >= 400
92-
ORDER BY timestamp DESC
93-
LIMIT 50
94-
```
95-
96-
### Error breakdown by endpoint
97-
98-
```sql
99-
SELECT method, path, status_code, COUNT(*) as count
100-
FROM request_logs
101-
WHERE status_code >= 400
102-
GROUP BY method, path, status_code
103-
ORDER BY count DESC
104-
```
105-
106-
### Find slow requests
107-
108-
```sql
109-
SELECT timestamp, method, path, status_code, response_time_ms
110-
FROM request_logs
111-
ORDER BY response_time_ms DESC
112-
LIMIT 20
113-
```
114-
115-
### Response time by endpoint (p50, p95)
116-
117-
```sql
118-
SELECT method, path,
119-
COUNT(*) as count,
120-
ROUND(quantile_cont(response_time_ms, 0.5)) as p50_ms,
121-
ROUND(quantile_cont(response_time_ms, 0.95)) as p95_ms,
122-
MAX(response_time_ms) as max_ms
123-
FROM request_logs
124-
GROUP BY method, path
125-
ORDER BY p95_ms DESC
126-
```
127-
12894
### Inspect a specific request
12995

13096
Use `request-details` to fetch full details (headers, body, exception, application logs, spans) for a single request:
@@ -203,16 +169,6 @@ WHERE response_body_json IS NOT NULL
203169
AND (response_body_json->>'error') IS NOT NULL
204170
```
205171

206-
### Requests by country
207-
208-
```sql
209-
SELECT client_country_iso_code, COUNT(*) as count
210-
FROM request_logs
211-
WHERE client_country_iso_code IS NOT NULL
212-
GROUP BY client_country_iso_code
213-
ORDER BY count DESC
214-
```
215-
216172
## Exit Codes
217173

218174
| Code | Meaning |

‎skills/apitally-cli/references/tables.md‎

Lines changed: 14 additions & 14 deletions
Original file line numberDiff line numberDiff line change
@@ -8,7 +8,7 @@ Tables are created automatically when using the `--db` flag with `apps`, `consum
88
CREATE TABLE apps (
99
app_id INTEGER NOT NULL UNIQUE,
1010
name TEXT NOT NULL,
11-
framework TEXT NOT NULL,
11+
framework TEXT NOT NULL, -- e.g. FastAPI, Express
1212
client_id TEXT NOT NULL,
1313
created_at TIMESTAMPTZ NOT NULL
1414
);
@@ -33,9 +33,9 @@ CREATE TABLE app_envs (
3333
CREATE TABLE consumers (
3434
app_id INTEGER NOT NULL,
3535
consumer_id INTEGER NOT NULL,
36-
identifier TEXT NOT NULL,
37-
name TEXT NOT NULL,
38-
"group" TEXT,
36+
identifier TEXT NOT NULL, -- e.g. email, username, API key name
37+
name TEXT NOT NULL, -- auto-generated from identifier if not set
38+
"group" TEXT, -- optional consumer group name
3939
created_at TIMESTAMPTZ NOT NULL,
4040
last_request_at TIMESTAMPTZ,
4141
UNIQUE (app_id, consumer_id)
@@ -51,11 +51,11 @@ CREATE TABLE request_logs (
5151
app_id INTEGER NOT NULL,
5252
timestamp TIMESTAMPTZ NOT NULL,
5353
request_uuid VARCHAR NOT NULL,
54-
env VARCHAR,
54+
env VARCHAR, -- environment name, e.g. "prod"
5555
method VARCHAR NOT NULL,
56-
path VARCHAR,
57-
url VARCHAR NOT NULL,
58-
consumer_id INTEGER,
56+
path VARCHAR, -- parameterized route template, e.g. /users/{user_id}
57+
url VARCHAR NOT NULL, -- full URL with actual path values, e.g. https://api.example.com/users/123
58+
consumer_id INTEGER, -- references consumers.consumer_id
5959
request_headers STRUCT(name VARCHAR, value VARCHAR)[],
6060
request_size_bytes BIGINT,
6161
request_body_json JSON,
@@ -70,7 +70,7 @@ CREATE TABLE request_logs (
7070
exception_message VARCHAR,
7171
exception_stacktrace VARCHAR,
7272
sentry_event_id VARCHAR,
73-
trace_id VARCHAR,
73+
trace_id VARCHAR, -- OpenTelemetry trace ID (hex)
7474
UNIQUE (app_id, request_uuid)
7575
);
7676
```
@@ -100,14 +100,14 @@ Populated by the `request-details` command when using `--db`.
100100
CREATE TABLE spans (
101101
app_id INTEGER NOT NULL,
102102
request_uuid VARCHAR NOT NULL,
103-
span_id VARCHAR NOT NULL,
103+
span_id VARCHAR NOT NULL, -- OpenTelemetry span ID (hex)
104104
parent_span_id VARCHAR,
105105
name VARCHAR NOT NULL,
106-
kind VARCHAR NOT NULL,
107-
start_time_ns BIGINT NOT NULL,
108-
end_time_ns BIGINT NOT NULL,
106+
kind VARCHAR NOT NULL, -- e.g. SERVER, CLIENT, INTERNAL
107+
start_time_ns BIGINT NOT NULL, -- Unix epoch nanoseconds
108+
end_time_ns BIGINT NOT NULL, -- Unix epoch nanoseconds
109109
duration_ns BIGINT NOT NULL,
110-
status VARCHAR NOT NULL,
110+
status VARCHAR NOT NULL, -- e.g. OK, ERROR, UNSET
111111
attributes JSON
112112
);
113113
```

0 commit comments

Comments
 (0)