Skip to content

The agent logs as their own skill: @gemstack/skill-logs, command logs; agents-logs folded onto agent-data - #1771

Merged
suleimansh merged 1 commit into
mainfrom
skill-logs
Sep 9, 2026
Merged

The agent logs as their own skill: @gemstack/skill-logs, command logs; agents-logs folded onto agent-data#1771
suleimansh merged 1 commit into
mainfrom
skill-logs

Conversation

@suleimansh

Copy link
Copy Markdown
Contributor

The agent logs are the fourth skill, @gemstack/skill-logs, command logs. This is #1769 built as picked there. Agents read past runs. The daemon records every run through the skill. The agents-logs branch is gone: the runs and the routine locks live on agent-data.

What an agent gets:

npx logs [--ticket <file>] [--branch <name>] [--limit N]   the runs, newest first, as cards
npx logs show <id>                                          one run: its card, and what the agent said, its result, its ending, its cost

The SKILL.md says when to look: before you plan or work a ticket, read its runs. A stopped or failed run says what to avoid. A done run with a PR says the work may already be there.

Cutover is by hand after merge, per machine, on every project. The recipe and the converter are below. A daemon on the old build after this merges writes to agents-logs again, so every machine moves at once.


🤖 curated · Fable 5.1

The card and the diary

A run is two files under agents/<who>/ on agent-data. <who> is the git email the repo commits as, made safe, as before. Both files are pushed, as before.

The card, <id>.json, has eleven fields of the skill's own: id, startedAt, endedAt, status, intent, driver, model, branch, pr, ticket, cost. Everything else the daemon records sits under one key, caller. The skill stores it and never reads it. The command never prints it.

The diary, <id>.jsonl, is the event log as before, with four events written in the skill's shape: said (a driver text event), result (a driver result event), ended (the end event) and cost (a usage event). Every other event is written as it is. logs show prints only the four kinds.

Two facts are new on the framework's meta so the card can carry them: endedAt, set when the end event folds, and cost, summed over the usage events.

The framework side

One module maps both ways: packages/framework/src/store/run-record.ts. toRunCard/fromRunCard fold the meta into the card and back. toDiaryLine/fromDiaryLine map one event to one line and back. Every reader of a recorded run comes back through it: the History list and the run page (listAgents, loadAgentEvents), the tail of an ended run (dashboard-rpc/events.ts wraps the tailer's callback), a continuation's restore, and the retry's failure detail.

Teardown reads the run out of its worktree (readWorktreeAgent) and records it with the skill's writeRun. Adoption and the Open PR button patch the card with the skill's patchRun. Delete uses deleteRun. The transient archive under .the-framework/agents/ stays as it was, for projects that are not git repositories and for the crash rescue.

The boot-time reconcile used to edit a running record in place on the branch's checkout. The next sync's rebase refuses a dirty tree and the funnel resets it, so that edit was lost within a minute. It now ends the run through the skill's funnel, as a commit.

The fold

Gone: LOGS_BRANCH, LOGS_CHECKOUT_DIR, agent-archive.ts, archived-agent-patch.ts, the second pull in the data sync. The routine locks use DATA_BRANCH. The LAYOUT marker drops logs-branch and archive-dir and gains runs-dir: agents; .the-framework/LAYOUT is regenerated in this PR. Every SPEC, FEATURES-SPEC.md and the three dashboard comments say agent-data now. No file in the repo says agents-logs any more.

The daemon puts logs on every agent's PATH and links the skill into every checkout beside the other three. The system-prompt bridge for agents outside a daemon-made checkout carries the skill's text after the queue's. Both places are already marked TEMPORARY (#1748) in the code, and this PR adds to them without changing that.

Calls made while building, for review

These are the agent's, not picks from the issue:

  • logs prints the newest 20 runs unless --limit says otherwise. A full listing with every prompt is 126 KB on this repo.
  • --ticket <file> matches a card whose ticket is that exact path or ends in /<file>. The skill does not know where tickets live.
  • Diary lines carry no timestamp. The event log never had one, and adding it is the writer's change, not the skill's.
  • The framework's updatedAt sits under caller, like every other non-card field.
  • A history read with a cutoff keeps a run on the branch by its card's startedAt, not by its file name. The file-name shortcut stays for the transient archive only. The adoption pass reads about 200 small files per pass on this repo.

Proof

Tests: skill-logs 16, framework 1379 node + 865 vitest, agent-data 53, skill-branches 78, skill-tickets 47, skill-queue 16. Typecheck clean.

Real data: a scratch copy of this repo's agents-logs (208 runs, 415 files) converted with the script below. logs reads 20 cards in 0.9 s and all 208 in 2.3 s. The converter recovers the cost of 84 runs ($238.64) and an end time for all 208.

Rig: a throwaway repo with a bare origin, the daemon from this branch on its own port, one real claude-code run started over RPC on a ticket. The run was on origin's agent-data 15 s after it ended, as one commit logs: record run <id>. The card had the eleven fields on top and the daemon's under caller. The History RPC returned status, ticket, cost and end time. The run page replayed with driver, usage and end events. From a fresh clone: logs --ticket 2026-09-09_dogfood-logs.md found it by file name and by path, logs show <id> printed the card with its said, result, cost and ended lines, logs --branch found it, and an unknown id was refused no-run.

Cutover, by hand, after merge

Per project, on every machine that runs a daemon, before that machine's daemon runs the new build:

# 1. stop the daemon
# 2. in the project's agent-data checkout (or any clone on that branch)
git fetch origin agent-data agents-logs
git checkout agent-data && git pull
git checkout origin/agents-logs -- agents/
node /path/to/cutover-convert.mjs /path/to/gemstack/packages/framework .
git add -A && git commit -m "the runs from agents-logs, as the logs skill's" && git push origin agent-data
# 3. release any routine lock left on agents-logs (none on origin today), then
git push origin --delete agents-logs && git branch -D agents-logs
rm -rf .branches/agents-logs
# 4. pull, install, build, start the daemon on the new build

The converter (toRunCard/diaryOf from the built framework, so it is the same mapping the daemon uses):

cutover-convert.mjs
// Cutover, one-off, by hand: convert the runs copied from `agents-logs` onto `agent-data` into the
// `logs` skill's shape — each `agents/<who>/<id>.json` (the framework's flat meta) into a card, each
// `<id>.jsonl` (the framework's events) into a diary. Run inside the agent-data checkout after
// `git checkout agents-logs -- agents/`, before committing. Idempotent: a card already converted
// (has `caller`, or only card fields) converts to itself; a diary line already mapped stays.
//   node cutover-convert.mjs <path to the framework package> [checkout dir, default cwd]
import { readdir, readFile, writeFile } from 'node:fs/promises'
import { join, resolve } from 'node:path'
import { pathToFileURL } from 'node:url'

const [fw, dir = process.cwd()] = process.argv.slice(2)
const { toRunCard, fromRunCard, diaryOf } = await import(pathToFileURL(resolve(fw, 'dist/store/run-record.js')).href)
const { parseRunCard, formatRunCard, formatDiary, parseDiary } = await import(pathToFileURL(resolve(fw, 'node_modules/@gemstack/skill-logs/dist/index.js')).href)

const runs = join(dir, 'agents')
let cards = 0, diaries = 0
for (const who of await readdir(runs)) {
  for (const name of await readdir(join(runs, who))) {
    if (!name.endsWith('.json')) continue
    const path = join(runs, who, name)
    const raw = JSON.parse(await readFile(path, 'utf8'))
    // A flat meta: every non-card field moves under caller. An already-converted card unfolds and refolds to itself.
    const meta = raw.caller ? fromRunCard(parseRunCard(JSON.stringify(raw))) : raw
    // The two card fields the old meta never had, folded from the old event log: the cost summed over
    // the usage events, the end time as the last update of a run that ended.
    const diaryPath = path.replace(/\.json$/, '.jsonl')
    const lines = parseDiary(await readFile(diaryPath, 'utf8').catch(() => ''))
    const diary = diaryOf(lines)
    const usd = diary.filter(l => l.kind === 'cost' && typeof l.usd === 'number').reduce((sum, l) => sum + l.usd, 0)
    if (meta.cost === undefined && usd > 0) meta.cost = usd
    if (meta.endedAt === undefined && meta.status !== 'running' && meta.updatedAt) meta.endedAt = meta.updatedAt
    await writeFile(path, formatRunCard(toRunCard(meta)))
    cards++
    await writeFile(diaryPath, formatDiary(diary))
    diaries++
  }
}
console.log(`converted ${cards} cards and ${diaries} diaries under ${runs}`)

Not in this PR

…ogs`; `agents-logs` folded onto `agent-data`

The fourth skill (#1769). A run is a card (`agents/<who>/<id>.json`, eleven
plain fields, the framework's rest under `caller`) and a diary (`<id>.jsonl`,
four kinds of line the skill knows among the framework's events), both on the
`agent-data` branch. Agents read with `logs`, `logs --ticket <file>`,
`logs --branch <name>`, `logs show <id>`; the SKILL.md says to read a ticket's
runs before planning or working it. The framework maps its meta and events
onto the two shapes in one module, `store/run-record.ts`, and records every
run through the skill at teardown; the history, the run page, the tail of an
ended run and a continuation read back through the same mapping. The
`agents-logs` branch is gone: its runs and the routine locks live on
`agent-data`, `LOGS_BRANCH` and `LOGS_CHECKOUT_DIR` with it, and the LAYOUT
marker names `runs-dir` instead.

Co-Authored-By: Claude Fable 5.1 <noreply@anthropic.com>
Claude-Session: https://claude.ai/code/session_017hpLBHRrvmubeoLrEyqtWD
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

1 participant