Skip to content
Open
Show file tree
Hide file tree
Changes from all commits
Commits
File filter

Filter by extension

Filter by extension

Conversations
Failed to load comments.
Loading
Jump to
Jump to file
Failed to load files.
Loading
Diff view
Diff view
12 changes: 7 additions & 5 deletions docs/providers/zed.md
Original file line number Diff line number Diff line change
Expand Up @@ -16,14 +16,16 @@ One SQLite database with one row per agent thread (`zed.ts:19`):

## Storage format

The `threads` table stores each thread's `data` BLOB as zstd-compressed JSON (`data_type = "zstd"`; legacy rows may be uncompressed `"json"`, both are read, `zed.ts:117-127`). Decompression uses Node's built-in `zlib.zstdDecompressSync` (`zed.ts:17`), no extra dependency.
The `threads` table stores each thread's `data` BLOB as zstd-compressed JSON (`data_type = "zstd"`; legacy rows may be uncompressed `"json"`, both are read, `zed.ts:153-165`). Decompression uses Node's built-in `zlib.zstdDecompressSync` (`zed.ts:17`), no extra dependency.

The decompressed thread JSON carries:

- `model`: `{ "provider": ..., "model": ... }`
- `request_token_usage`: map of user-message id to `{ input_tokens, output_tokens, cache_creation_input_tokens, cache_read_input_tokens }` (zero-valued fields are omitted)
- `cumulative_token_usage`: same shape, whole-thread totals

Each row's `folder_paths` column carries the workspace folder roots the thread was created against — absolute paths, one per line, lexicographically sorted (Zed's `PathList` serialization; the `folder_paths_order` column is display-only and unused here). The column is absent on databases written by older Zed, which added it via `ALTER TABLE`; the parser detects it with `PRAGMA table_info` and degrades gracefully (`zed.ts:94-101`).

Token semantics match Anthropic's (separate cache-creation and cache-read fields), so pricing maps directly onto the LiteLLM engine. Shapes verified against Zed's serialization source (`crates/agent/src/db.rs`: `DbThread`, `TokenUsage`, `SerializedLanguageModel`, `DataType`) and a real store.

## Caching
Expand All @@ -32,18 +34,18 @@ None.

## Deduplication

Per `zed:<threadId>:<requestKey>` (`zed.ts:96`), where `requestKey` is the user-message id from `request_token_usage` or the synthetic `cumulative-remainder`.
Per `zed:<threadId>:<requestKey>` (`zed.ts:132`), where `requestKey` is the user-message id from `request_token_usage` or the synthetic `cumulative-remainder`.

## Quirks

- `request_token_usage` is keyed by user message and does not cover every request a thread made (verified on a real thread: cumulative was ~3x the map sum). One remainder entry per thread tops usage up to the exact `cumulative_token_usage` (`zed.ts:133-153`), so totals always match the store.
- `request_token_usage` is keyed by user message and does not cover every request a thread made (verified on a real thread: cumulative was ~3x the map sum). One remainder entry per thread tops usage up to the exact `cumulative_token_usage` (`zed.ts:170-192`), so totals always match the store.
- The per-request map carries no timestamps, so every call in a thread uses the thread's `updated_at`; day-level attribution inside long-running threads is approximate.
- Node's zlib gained zstd in 22.15. On older Nodes the provider skips with a notice instead of failing (`zed.ts:14-17`).
- All Zed usage currently lands under a single `zed` project; `folder_paths` is not yet mapped to per-project attribution.
- Project attribution mirrors Zed's own sidebar grouping (`zed.ts:79-101`): a thread with exactly one `folder_paths` entry maps to that folder's project (`projectPath` = folder path, `project` = folder basename); a thread with two or more entries maps to a synthetic project named after the joined basenames (`codeburn, website`, matching `ProjectGroupKey::display_name`), with no `projectPath` since a multi-folder set has no one path; rows without the column (older schemas) keep the single `zed` bucket. Zed records which workspace roots a thread was created against, but not which folder it actually used.

## When fixing a bug here

1. If discovery returns no sessions, confirm `threads.db` exists at the platform path and the `threads` table still has `id`, `summary`, `updated_at`, `data_type`, `data`.
1. If discovery returns no sessions, confirm `threads.db` exists at the platform path and the `threads` table still has `id`, `summary`, `updated_at`, `data_type`, `data` (`folder_paths` is optional and only read when present).
2. If threads are skipped, check `data_type` values on disk; only `zstd` and `json` are read.
3. If totals disagree with the store, compare against `cumulative_token_usage` per thread; the remainder logic must bring each thread exactly to it.
4. If model names stop pricing, inspect `model.model` strings in a real thread and add aliases if Zed introduces new hosted-model ids.
11 changes: 9 additions & 2 deletions src/daily-cache.ts
Original file line number Diff line number Diff line change
Expand Up @@ -5,6 +5,13 @@ import { homedir } from 'os'
import { join } from 'path'
import type { DateRange, ProjectSummary } from './types.js'

// Bumped to 17: Zed threads now attribute to their recorded workspace
// folder(s) — a single folder becomes that project, multi-folder workspaces
// become a joined-basename project ("codeburn, website") — instead of the
// shared `zed` bucket, so days finalized at v16 carry the old single-project
// split. Raising MIN_SUPPORTED_VERSION forces the one-time re-derivation of
// days whose threads.db still exists.
//
// Bumped to 16: Codex discovery is structural instead of originator-gated
// (#873/#626), so rollouts written by third-party frontends driving
// `codex app-server` ("t3code_desktop", "JetBrains.IntelliJ IDEA", ...) now
Expand Down Expand Up @@ -67,8 +74,8 @@ import type { DateRange, ProjectSummary } from './types.js'
// that older binaries skipped. v8 added local-model savings to the daily
// rollup; the `savingsConfigHash` field is invalidated separately when the
// user changes their `localModelSavings` mapping.
export const DAILY_CACHE_VERSION = 16
const MIN_SUPPORTED_VERSION = 16
export const DAILY_CACHE_VERSION = 17
const MIN_SUPPORTED_VERSION = 17
// Version-suffixed so different binaries each own a distinct file and never
// clobber an incompatible schema. Bumping the version mints a fresh filename;
// adoptOlderDailyCaches then unions days out of every previous file (including
Expand Down
98 changes: 96 additions & 2 deletions src/providers/zed.ts
Original file line number Diff line number Diff line change
Expand Up @@ -32,12 +32,22 @@ const THREADS_QUERY = `
ORDER BY updated_at ASC
`

// Newer Zed adds `folder_paths` (newline-separated absolute workspace roots,
// lexicographically sorted) via ALTER TABLE, so databases written by older
// versions do not have the column. Query it only when the schema has it.
const THREADS_QUERY_WITH_FOLDER_PATHS = `
SELECT id, summary, updated_at, data_type, data, folder_paths
FROM threads
ORDER BY updated_at ASC
`

type ThreadRow = {
id: string
summary: string | null
updated_at: string | null
data_type: string | null
data: Uint8Array | null
folder_paths: unknown
}

type TokenUsage = {
Expand Down Expand Up @@ -66,13 +76,84 @@ function usageIsEmpty(usage: TokenUsage): boolean {
)
}

// A thread carries the workspace folder roots it was created against as a
// newline-separated list. The display label is intentionally kept separate
// from the stable path-set key because basenames are not unique.
type ThreadProject = {
project: string
projectPath?: string
}

function pathComponents(path: string): string[] {
const normalized = path.replace(/\\/g, '/').replace(/\/+$/, '')
if (!normalized || /^[A-Za-z]:$/.test(normalized)) return []
const components = normalized.split('/').filter(Boolean)
const last = components.length - 1
if (last >= 0 && components[last]!.endsWith('.git')) {
components[last] = components[last]!.slice(0, -'.git'.length)
}
return components.filter(Boolean)
}

function pathDisplaySuffix(path: string, detail: number): string {
return pathComponents(path).slice(-(detail + 1)).join('/')
}

function displayNames(paths: string[]): string[] {
const names = paths.map(path => pathDisplaySuffix(path, 0))
const counts = new Map<string, number>()
for (const name of names) {
if (name) counts.set(name, (counts.get(name) ?? 0) + 1)
}

return paths.map((path, index) => {
const name = names[index] ?? ''
if (!name || counts.get(name) === 1) return name

const components = pathComponents(path)
for (let detail = 1; detail < components.length; detail++) {
const candidate = pathDisplaySuffix(path, detail)
const conflicts = paths.some((otherPath, otherIndex) => {
if (otherIndex === index) return false
const otherName = names[otherIndex] ?? ''
return otherName === name && pathDisplaySuffix(otherPath, detail) === candidate
})
if (!conflicts) return candidate
}
return pathDisplaySuffix(path, components.length - 1) || name
})
}

function resolveThreadProject(folderPaths: unknown): ThreadProject | undefined {
if (typeof folderPaths !== 'string') return undefined
const paths = folderPaths.split('\n').map(p => p.trim()).filter(Boolean)
if (paths.length === 0) return undefined

const names = displayNames(paths).filter(Boolean)
return {
project: names.length > 0 ? names.join(', ') : 'Empty Workspace',
...(paths.length === 1 ? { projectPath: paths[0]! } : {}),
}
}

function hasFolderPathsColumn(db: SqliteDatabase): boolean {
try {
const columns = db.query<{ name: string }>('PRAGMA table_info(threads)')
return columns.some(c => c.name === 'folder_paths')
} catch {
return false
}
}

function buildCall(opts: {
threadId: string
requestKey: string
usage: TokenUsage
model: string
timestamp: string
userMessage: string
project?: string
projectPath?: string
}): ParsedProviderCall {
const input = num(opts.usage.input_tokens)
const output = num(opts.usage.output_tokens)
Expand All @@ -96,16 +177,19 @@ function buildCall(opts: {
deduplicationKey: `zed:${opts.threadId}:${opts.requestKey}`,
userMessage: opts.userMessage,
sessionId: opts.threadId,
...(opts.project ? { project: opts.project } : {}),
...(opts.projectPath ? { projectPath: opts.projectPath } : {}),
}
}

function parseThreads(db: SqliteDatabase, seenKeys: Set<string>): ParsedProviderCall[] {
const calls: ParsedProviderCall[] = []
let skipped = 0
const withFolderPaths = hasFolderPathsColumn(db)

let rows: ThreadRow[]
try {
rows = db.query<ThreadRow>(THREADS_QUERY)
rows = db.query<ThreadRow>(withFolderPaths ? THREADS_QUERY_WITH_FOLDER_PATHS : THREADS_QUERY)
} catch {
return calls
}
Expand Down Expand Up @@ -153,8 +237,18 @@ function parseThreads(db: SqliteDatabase, seenKeys: Set<string>): ParsedProvider
if (!usageIsEmpty(remainder)) entries.push(['cumulative-remainder', remainder])
}

const project = resolveThreadProject(withFolderPaths ? row.folder_paths : null)

for (const [requestKey, usage] of entries) {
const call = buildCall({ threadId: row.id, requestKey, usage, model, timestamp, userMessage })
const call = buildCall({
threadId: row.id,
requestKey,
usage,
model,
timestamp,
userMessage,
...(project ?? {}),
})
if (seenKeys.has(call.deduplicationKey)) continue
seenKeys.add(call.deduplicationKey)
calls.push(call)
Expand Down
6 changes: 6 additions & 0 deletions src/session-cache.ts
Original file line number Diff line number Diff line change
Expand Up @@ -237,6 +237,12 @@ export const PROVIDER_PARSE_VERSIONS: Record<string, string> = {
'roo-code': 'worktree-project-grouping-v1',
warp: 'worktree-project-grouping-v1-est-cost',
antigravity: 'worktree-project-grouping-v5',
// folder-path-project-grouping-v1: threads attribute to the recorded
// workspace folder(s) — single folder becomes that project, multi-folder
// becomes a joined-basename project — instead of the shared `zed` bucket,
// so already-cached threads must re-parse once (the session cache would
// otherwise serve the old single-project turns without invoking the parser).
zed: 'folder-path-project-grouping-v1',
}

// ── Cache Dir ──────────────────────────────────────────────────────────
Expand Down
Loading