diff --git a/README.md b/README.md index 96b144f..91d341e 100644 --- a/README.md +++ b/README.md @@ -49,6 +49,8 @@ codeshot [--path ] [--out ] [--limit ] [--max-re ```bash codeshot --architecture --path ~/code/myrepo --out architecture.svg --format svg +# ...or, on a repo with more files than fit in one readable picture: +codeshot --architecture --path ~/code/myrepo --group-depth 1 --out modules.svg --format svg ``` A second mode, distinct from the single-symbol trail above: instead of one @@ -66,6 +68,19 @@ slow operation (one sequential `codegraph` call per enumerated symbol), so two extra flags exist specifically for this mode: - `--max-symbols` — cap how many symbols get probed (default 500). Codeshot warns on stderr if this cuts the scan short. +- `--group-depth ` — roll files up into their first `n` directory segments + and draw *those* as the nodes (`--group-depth 1` on `src/api/user.js` → + `src/`), summing the call weights of every file pair that collapses into the + same pair of groups. This is the readable view of a repo big enough that the + per-file graph is a hairball. Unlike `--max-render`, which drops the + least-busy *files* outright — taking every edge that touched them with it — + grouping keeps every **cross-module** call and just draws it at module + resolution. Calls that become intra-group are dropped, for the same reason + same-file calls already are: this diagram is about coupling between modules, + not inside them. So the summed weights on a grouped diagram are legitimately + lower than the per-file one's, often much lower — that's the intra-module + traffic, not a lost edge. Repo-root files (no directory to roll into) stay + themselves. - `--depth` has no effect here and is rejected if passed — there's no multi-hop file-traversal concept to apply it to. diff --git a/TECHNICAL.md b/TECHNICAL.md index eed9f2f..32c303a 100644 --- a/TECHNICAL.md +++ b/TECHNICAL.md @@ -56,10 +56,11 @@ print outFile path to stdout **`--architecture` mode** is a separate pipeline through the same `main()`, dispatched on the boolean flag before the symbol-trail path runs: ``` -argv (--architecture, --path, --out, --limit, --max-render, --max-symbols, --format) +argv (--architecture, --path, --out, --limit, --max-render, --max-symbols, --group-depth, --format) | v reject if a positional was also given, or --depth was explicitly set +(--group-depth is rejected in the OTHER direction: only outside --architecture) | v enumerateSymbols(repoPath, maxSymbols) --- runCodegraph(['query', '--path', repoPath, @@ -67,7 +68,10 @@ enumerateSymbols(repoPath, maxSymbols) --- runCodegraph(['query', '--path', repo --- an empty-string query WITHOUT --limit silently caps around 50 results regardless of repo size; passing a large --limit instead makes codegraph return everything it has (see External Dependencies) --- the real cap is - applied client-side via unwrapQueryNodes + a slice to maxSymbols + applied client-side via unwrapQueryNodes + sortSymbolsForEnumeration + (codegraph's own result order is unspecified, so the slice is sorted by + filePath+name first — otherwise WHICH symbols survive a --max-symbols cut + varies run to run) + a slice to maxSymbols | v symbolBudgetWarning(...) --- warns on stderr if enumeration was capped by --max-symbols @@ -90,8 +94,18 @@ aggregateFileEdges(symbolEdges) --- dedupes/sums {fromFile,toFile} pairs into a real filePath on either end | v -renderTruncationNote('files', totalFiles, maxRender) --- warns on stderr if - --max-render will cut files +rollupFileEdges(fileEdges, groupDepth) --- only with --group-depth: rewrites each + endpoint to its first N directory + segments (groupPath) and re-sums the + weights, dropping self-group edges; + groupCollapseWarning fires if that ate + every edge (a blank diagram the FLAG + caused, which emptyArchitectureWarning + would otherwise blame on the index) + | + v +renderTruncationNote('files'|'groups', totalFiles, maxRender) --- warns on stderr if + --max-render will cut nodes | v buildArchitectureDot(fileEdges, { maxRender }) --- pure function, produces a DOT string @@ -120,7 +134,7 @@ It doubles as a live legend for the [Visual Encoding](#visual-encoding) rules be - **`render/callgraph.js`** — the entire tool. Exports `buildDot(symbol, callers, callees, { maxRender, transitiveEdges })` (pure: turns caller/callee arrays into a DOT digraph string, deduplicating entries with the same `name`+`filePath` so repeated JSON rows don't render as duplicate edges; via `nodeIdentities` it gives each drawn node a graphviz id unique per `name`+`filePath`, so two *distinct* symbols that share a name but live in different files render as two separate boxes instead of silently collapsing into one — graphviz keys a node by the exact string in its edge, so without this the second same-named caller/callee vanishes from the picture; a name that occurs in only one file keeps name-as-id, leaving collision-free graphs byte-for-byte identical to before this existed, and only a colliding name gains a file-qualified id plus a `name\n(basename)` label to tell the boxes apart; when `tooltips` is set (`main` sets it for `svg`/`svgz` output only, since graphviz renders node tooltips as `` there and ignores them for raster formats), every drawn node is declared with its `filePath` as a hover tooltip so you can read which file a symbol lives in without cluttering the box — the root symbol is left un-tooltipped since `buildDot` isn't passed its file; if `maxRender` is given, `allocateRenderBudget` splits it as one shared allowance across callers, callees, and `transitiveEdges` — spent in that priority order, so the direct trail is never starved to make room for deeper hops; a `"kind":"file"` node — codegraph's way of saying "this is a module-level/import reference, not a verified function call" — is styled dotted/gray/`"file"` via `edgeStyleAttrs` instead of looking like a real call edge; `transitiveEdges`, an optional array of `{ from, to, depth }` pairs from `--depth > 1` traversal, is colored by `depthColor(depth)` unless it's file-kind — omitting `transitiveEdges`/`maxRender` renders exactly as before either feature existed), `allocateRenderBudget(maxRender, counts)` (pure: the shared-budget split described above, also called from `main` so its stderr notes report what actually got drawn), `dedupeNodes(nodes)` (pure: collapses same-`name`+`filePath` entries, used by both `buildDot` and `main` so distinct-count logic has one source of truth), `dedupeEdges(edges)` (pure: the same idea as `dedupeNodes` but keyed on a `from`+`to` pair, used only for `transitiveEdges`), `depthColor(depth)` (pure: maps hop distance ≥2 to a progressively lighter shade, clamped at the palette's last entry for very deep hops), `isTestRef(node)` (true if a node's name or filePath looks test-related, used to render those edges dashed — applies to `transitiveEdges` too, checked against each edge's `from` node, and is overridden by file-kind styling when both apply; also reused by `buildArchitectureDot` below, passed `{name: '', filePath}` since architecture-mode nodes have no symbol name — the name-based heuristics degrade harmlessly to `false` on an empty name, leaving the path-based ones intact), `truncationWarning(kind, results, limit)` (pure: returns a warning string if `results.length` hit `limit` exactly, else `null` — the *fetch* cap), `renderTruncationNote(kind, distinctCount, cap)` (pure: returns a warning string if the deduplicated count exceeds `cap`, else `null` — the *render* cap; `main` passes each dimension's actual `allocateRenderBudget` allotment as `cap`, not the raw `--max-render`; also reused as-is by `--architecture` mode with `kind: 'files'`), `depthBudgetWarning(truncated, budget)` (pure: returns a warning string if `--depth` traversal hit its node-discovery cap (`--max-depth-nodes`) before finishing, else `null`), `formatMismatchWarning(outFile, format)` (pure: returns a warning string if `--out`'s extension is a real `dot`-recognized format that disagrees with `--format`, else `null`), and `matchSymbolNotFound(out)` (pure: extracts the symbol name from codegraph's plain-text "Symbol not found" message, or `null` if `out` doesn't match that shape). Everything else (`requireOnPath`, `runCodegraph`, `parseCodegraphOutput`, `resolveSymbol`, `collectTransitive`, `main`) is CLI plumbing, not exported — `resolveSymbol` and `collectTransitive` in particular do real I/O (`codegraph` calls), so like `runCodegraph` they're only exercised by the CLI-level tests, not unit-tested directly. - **`--architecture` mode adds:** `unwrapQueryNodes(queryResults)` (pure: unwraps `query`'s `{node, score}` result shape, dropping only a missing `node` — a `"kind":"file"` entry is deliberately KEPT rather than dropped: `codegraph callees ` is a real, working probe against it, and it's the only way to surface calls made from inside a top-level anonymous callback, which codegraph attributes to the enclosing file rather than any named function — see the note on `probeFileEdges` and Known Limitations. Also deliberately does NOT allowlist "callable" kinds like `function`/`method` for the non-file entries — probing a `constant` or `variable` just harmlessly returns an empty `callees` array, which is more robust across languages than maintaining a per-language kind list), `symbolBudgetWarning(truncated, budget)` (pure: same shape as `depthBudgetWarning`, fires when `--max-symbols` cut enumeration short), `duplicateNames(symbols)` (pure: the `Set` of names appearing in more than one distinct **file** — counting files rather than symbol occurrences is what keeps same-file collisions off the file-qualified route; shared by `probeFileEdges` and `duplicateNameWarning` so the fix and the warning can't disagree about what counts as a duplicate), `parseNodeCalls(out, expectedFile)` (pure: reads the file-qualified trail line out of `codegraph node -f`'s text output into `{name, filePath}` callees; returns `[]` for a recognized symbol that calls nothing, `null` on any of the four untrustworthy responses listed under Known Limitations so the caller falls back rather than under-reporting, and drops a callee whose name equals its own file's basename — the structural stand-in for the `"kind":"file"` filter, since the trail carries no kind), `duplicateNameWarning(symbols)` (pure: reports duplicate names in two halves — the symbol collisions `probeFileEdges` resolved via `node -f`, and the file-name collisions that remain genuinely ambiguous; see Known Limitations), `aggregateFileEdges(symbolEdges)` (pure: dedupes/sums `{fromFile, toFile}` pairs from every probed symbol into weighted `{from, to, weight}` file edges, dropping self-file edges and any edge missing a real `filePath` on either end — an unresolved external/stdlib callee has no file of its own and would otherwise render as a bogus `""` node), `topFilesByWeight(fileEdges, maxRender)` (pure: ranks files by total in+out edge weight and returns the top `maxRender` as a `Set`, or `null` meaning "no cap" — deliberately a simple weight cutoff, not a connected-component/centrality algorithm), `buildArchitectureDot(fileEdges, { maxRender })` (pure: the architecture-mode analog of `buildDot` — a dedicated function rather than a `buildDot` branch, since the semantics genuinely differ: no root-symbol highlight, no caller/callee direction split, no file-kind dotted-edge concept since every node already IS a file), and `architectureOutputBaseName(repoPath)` (pure: `sanitizeForFilename(path.basename(path.resolve(repoPath)))`, used for the default `--out` filename). Unexported CLI plumbing: `enumerateSymbols`, `probeFileEdges` (takes the file-qualified route for duplicate-named non-file symbols, the bare-name `callees --json` route for everything else), `probeCallsInFile`, `runArchitectureMode` (real I/O, exercised via the CLI-level tests — including a purpose-built fixture repo covering both a cross-file collision (two `handle`s, which must be split) and a same-file one (two `run`s, which must not be), since this repo's own index has no duplicate names to trip either path), and `renderDotToFile` (shared with symbol mode — the write-tempfile/`dot -T`/delete-tempfile tail, previously inline in `main`, extracted once a second call site needed it). + **`--architecture` mode adds:** `unwrapQueryNodes(queryResults)` (pure: unwraps `query`'s `{node, score}` result shape, dropping only a missing `node` — a `"kind":"file"` entry is deliberately KEPT rather than dropped: `codegraph callees ` is a real, working probe against it, and it's the only way to surface calls made from inside a top-level anonymous callback, which codegraph attributes to the enclosing file rather than any named function — see the note on `probeFileEdges` and Known Limitations. Also deliberately does NOT allowlist "callable" kinds like `function`/`method` for the non-file entries — probing a `constant` or `variable` just harmlessly returns an empty `callees` array, which is more robust across languages than maintaining a per-language kind list), `symbolBudgetWarning(truncated, budget)` (pure: same shape as `depthBudgetWarning`, fires when `--max-symbols` cut enumeration short), `duplicateNames(symbols)` (pure: the `Set` of names appearing in more than one distinct **file** — counting files rather than symbol occurrences is what keeps same-file collisions off the file-qualified route; shared by `probeFileEdges` and `duplicateNameWarning` so the fix and the warning can't disagree about what counts as a duplicate), `parseNodeCalls(out, expectedFile)` (pure: reads the file-qualified trail line out of `codegraph node -f`'s text output into `{name, filePath}` callees; returns `[]` for a recognized symbol that calls nothing, `null` on any of the four untrustworthy responses listed under Known Limitations so the caller falls back rather than under-reporting, and drops a callee whose name equals its own file's basename — the structural stand-in for the `"kind":"file"` filter, since the trail carries no kind), `duplicateNameWarning(symbols)` (pure: reports duplicate names in two halves — the symbol collisions `probeFileEdges` resolved via `node -f`, and the file-name collisions that remain genuinely ambiguous; see Known Limitations), `aggregateFileEdges(symbolEdges)` (pure: dedupes/sums `{fromFile, toFile}` pairs from every probed symbol into weighted `{from, to, weight}` file edges, dropping self-file edges and any edge missing a real `filePath` on either end — an unresolved external/stdlib callee has no file of its own and would otherwise render as a bogus `""` node), `groupPath(filePath, depth)` (pure: the group a file rolls up into at `--group-depth` — its first `depth` *directory* segments with a trailing slash, so a group reads as a directory rather than a file; a repo-root file has no directory to roll into and is returned unchanged, since grouping it under `""` would merge every root file into one nameless box, and a file shallower than `depth` simply keeps the directories it has, so an over-deep `--group-depth` degrades to per-file output instead of erroring), `rollupFileEdges(fileEdges, depth)` (pure: rewrites both endpoints of every file edge through `groupPath` and re-sums the weights, dropping edges that become self-group — the directory-level restatement of `aggregateFileEdges`'s self-file rule, since once a directory is the unit an intra-directory call is internal structure, not cross-module coupling; returns the same `{from,to,weight}` shape so `topFilesByWeight`, `emptyArchitectureWarning`, `isTestRef`'s dashed-test-node styling and `buildArchitectureDot` all operate on groups with no change, and an unset `depth` returns the input untouched), `groupCollapseWarning(beforeCount, afterCount, depth)` (pure: fires only when a rollup consumed *every* edge — a repo whose files all sit under one directory at `--group-depth 1` — because that blank diagram is the flag's doing, and `emptyArchitectureWarning` would otherwise blame a missing/stale index for it; `main` suppresses the latter when this one fires so the user gets the actionable cause, not two contradictory explanations), `sortSymbolsForEnumeration(symbols)` (pure: orders the enumerated symbol set by `filePath` then `name` before the `--max-symbols` slice — see the enumeration-order note under Known Limitations for why an unsorted slice made a truncated scan non-reproducible), `topFilesByWeight(fileEdges, maxRender)` (pure: ranks files by total in+out edge weight and returns the top `maxRender` as a `Set`, or `null` meaning "no cap" — deliberately a simple weight cutoff, not a connected-component/centrality algorithm), `buildArchitectureDot(fileEdges, { maxRender })` (pure: the architecture-mode analog of `buildDot` — a dedicated function rather than a `buildDot` branch, since the semantics genuinely differ: no root-symbol highlight, no caller/callee direction split, no file-kind dotted-edge concept since every node already IS a file), and `architectureOutputBaseName(repoPath)` (pure: `sanitizeForFilename(path.basename(path.resolve(repoPath)))`, used for the default `--out` filename). Unexported CLI plumbing: `enumerateSymbols`, `probeFileEdges` (takes the file-qualified route for duplicate-named non-file symbols, the bare-name `callees --json` route for everything else), `probeCallsInFile`, `runArchitectureMode` (real I/O, exercised via the CLI-level tests — including a purpose-built fixture repo covering both a cross-file collision (two `handle`s, which must be split) and a same-file one (two `run`s, which must not be), since this repo's own index has no duplicate names to trip either path), and `renderDotToFile` (shared with symbol mode — the write-tempfile/`dot -T`/delete-tempfile tail, previously inline in `main`, extracted once a second call site needed it). - **`test/run.js`** — assertion-based test suite (Node's built-in `assert`, no framework) covering all of the pure functions above directly. Run via `npm test`. - **`package.json`** — declares the `codeshot` bin pointing at `render/callgraph.js`, and the `test` script. - **`.runechoguardignore`** — false-positive suppression list for the RunEcho pre-commit symbol-resolution guard (a local hook, not part of codeshot itself). Bare-call identifiers the guard can't resolve (e.g. Node builtins passed as function parameters) get listed here instead of disabling the guard. @@ -147,20 +161,22 @@ No environment variables, no config file. All behavior is controlled by CLI argu | `--depth` | `1` (today's direct-only behavior) | How many hops of callers-of-callers / callees-of-callees to draw beyond the direct trail. Must be a positive integer. `codegraph`'s own `callers`/`callees` have no traversal depth of their own, so Codeshot implements this client-side: `collectTransitive` recursively calls `callers`/`callees` on each newly discovered node, sequentially (same concurrency-safety reason as the depth-1 calls — see External Dependencies), up to `--depth` hops or `--max-depth-nodes` (below) total discovered nodes, whichever comes first. Fan-out is multiplicative with depth and branching factor, so the budget exists specifically to stop a well-connected symbol at `--depth 3`+ from turning into hundreds of sequential `codegraph` calls; if the budget is hit first, `depthBudgetWarning` prints a stderr note that the graph beyond that point is incomplete. Each hop beyond the first is drawn in a progressively lighter edge color (`depthColor`) so distance from the symbol is visible at a glance; `--limit` and `--max-render` are NOT applied per-hop, only globally to the depth-1 fetch/render as before. Has no meaning with `--architecture` (no multi-hop file traversal concept) and is rejected if explicitly passed alongside it. | | `--max-depth-nodes` | `200` (`DEFAULT_NODE_BUDGET`) | Only meaningful with `--depth > 1`: raises or lowers `collectTransitive`'s total-discovered-node safety cap (formerly the fixed, unconfigurable `NODE_BUDGET` constant). Must be a positive integer. Exposed as a flag for the same reason `--max-symbols` below is — a genuinely well-connected symbol at `--depth 3`+ can legitimately need a higher cap to finish, and that's a coverage/speed tradeoff only the caller can make. Its `--architecture` rejection follows `--depth`'s precedent above, not `--max-symbols`'s: like `--depth`, it's rejected only when an *explicit, non-default* value is passed alongside `--architecture` (`codeshot --architecture --max-depth-nodes 200` is a silent no-op, same as `--depth 1`) — unlike `--max-symbols`, which has no `--architecture`-only guard at all and is simply ignored outside that mode. | | `--architecture` | `false` | Switches to whole-repo file-level dependency graph mode instead of a single symbol's trail — see the second pipeline diagram above. Mutually exclusive with the `` positional (rejected if both given); `` becomes optional-and-forbidden rather than required. | +| `--group-depth` | unset (one node per file) | `--architecture`-only: rolls each file up into its first N directory segments and draws those as the graph's nodes, summing the weights of every file pair that collapses into the same group pair (`rollupFileEdges`). The answer to the per-file hairball that `--max-render` only half-solves: `--max-render` is lossy at *every* level (the least-busy files are dropped outright, and any edge touching one goes with them), while grouping is lossless at the *module* level specifically — every cross-module call still appears, drawn between directories instead of files. It is not lossless in total: calls that become intra-group are dropped exactly as same-file calls already are, so a grouped diagram's summed weights are legitimately lower (often far lower) than the per-file one's. That difference is the repo's intra-module traffic, and the docs say so, because a user reconciling the two pictures would otherwise read it as a weight-aggregation bug. Composable with `--max-render`, which then caps groups rather than files (`renderTruncationNote` says `groups`). Must be a positive integer. Rejected — not silently ignored — outside `--architecture`, the opposite direction from `--depth`/`--max-depth-nodes` and unlike `--max-symbols`' quiet no-op: symbol mode has no file-level graph to roll up, so passing it there is always a mistake, and a silently-dropped flag reads as "grouping applied" in exactly the diagram you would then trust. Applied *after* probing, never during: probes must stay file-exact (`parseNodeCalls`' `expectedFile` check, `isTestRef`, duplicate-name attribution all key off the real path), so this stays a pure view over the same data rather than a second scan mode. With `--embed` it claims its own marker id (`codeshot:arch-d`) and default image name rather than the plain `codeshot:arch` — the per-file and per-module pictures answer different questions, so embedding one must not silently overwrite the other's committed block; unset `--group-depth` keeps the original id, so existing docs are unaffected. | | `--max-symbols` | `500` | `--architecture`-only: caps how many enumerated symbols get probed, exposed as a flag for the same reason `--max-depth-nodes` above is — `--architecture` is a multi-minute O(symbols) sequential scan by nature (confirmed: 500 symbols took over 5 minutes against a real ~1,900-node repo), so users legitimately need to trade coverage for speed themselves rather than wait on a hidden safety net tuned for an already-fast operation. Must be a positive integer. `symbolBudgetWarning` prints a stderr note when this cuts enumeration short. | ## Visual Encoding Beyond the direct-call edges (solid indigo) and dashed test-caller edges (see README), one more distinction is drawn straight from data codegraph already returns but previously went unused: an edge whose node has `"kind":"file"` — codegraph's way of reporting a module-level/import reference instead of a real function-level call site (common for framework dependency-injection patterns, e.g. FastAPI's `Depends()`, where codegraph can't resolve the real caller function) — is rendered dotted, gray (`#9ca3af`), and labeled `"file"`, so it reads as "codegraph found *some* relationship here, but not a verified call" rather than looking identical to a confirmed function call. This takes precedence over the test-dash styling when a node is both (rare, but a test file's own file-level reference would otherwise be ambiguous which signal wins). -`--architecture` mode's encoding is simpler, since every node is already a file (no root-symbol highlight, no caller/callee direction split): each edge is labeled with its call-count weight (how many distinct symbol-level calls between that pair of files were aggregated into it), and a file whose path matches `isTestRef`'s heuristics is drawn dashed — the same test-detection logic as symbol mode, just applied to the file itself rather than a caller node. +`--architecture` mode's encoding is simpler, since every node is already a file (no root-symbol highlight, no caller/callee direction split): each edge is labeled with its call-count weight (how many distinct symbol-level calls between that pair of files were aggregated into it), and a file whose path matches `isTestRef`'s heuristics is drawn dashed — the same test-detection logic as symbol mode, just applied to the file itself rather than a caller node. Under `--group-depth` the encoding is unchanged, only the unit is: a node is a directory (rendered with its trailing slash, `test/`, so it can't be misread as a file), the weight is the summed call count between two directories, and the dashed styling still comes from `isTestRef` — which reads path segments, so a `test/` group matches exactly as `test/run.js` did. ## Maintenance Commands ```bash -npm test # runs test/run.js — assertions against all exported pure functions, plus five CLI-level tests (depth 1, --depth 2, fuzzy-query resolution, --architecture, and two --architecture flag-rejection cases) against this repo's own codegraph index (skips the codegraph-dependent ones if not codegraph-indexed) +npm test # runs test/run.js — assertions against all exported pure functions, plus the CLI-level tests (depth 1, --depth 2, fuzzy-query resolution, --architecture, --architecture --group-depth 1, and the flag-rejection cases) against this repo's own codegraph index (skips the codegraph-dependent ones if not codegraph-indexed) node render/callgraph.js --path --out /tmp/out.png # manual smoke test, symbol mode node render/callgraph.js --architecture --path --out /tmp/arch.svg --format svg # manual smoke test, architecture mode +node render/callgraph.js --architecture --path --group-depth 1 --format dot # manual smoke test, directory-rollup view (prints DOT source) ``` There is no service to restart, no rollback beyond `npm uninstall -g codeshot` / reinstalling a prior git ref, and no other services, logs, or scheduled jobs to maintain — see [README.md](README.md#install) for the install command itself. @@ -174,7 +190,7 @@ There is no service to restart, no rollback beyond `npm uninstall -g codeshot` / - A cyclic call graph (recursion, or A and B calling each other) can cause `--depth`'s transitive traversal to rediscover the root symbol or an already-drawn depth-1 node as a "from"/"to" endpoint of a deeper edge. This is harmless (graphviz just draws the extra edge; `dedupeEdges` still collapses exact repeats) but can occasionally show what looks like a redundant edge back into an already-visible node. - **`--architecture` mode resolves colliding *symbol* names by file, but still cannot resolve colliding *file* names.** `codegraph callees ` takes a bare name with no way to disambiguate which file's symbol is meant, and — measured against codegraph 1.5.0 — it answers with the **union** of every same-named symbol's callees, so a bare-name probe doesn't merely pick the wrong file, it invents edges that exist in neither. On a two-`handle` fixture repo, 4 edges were drawn where only 2 were real. This matters at `--architecture`'s scale: generically-named methods (`render`, `init`, `get`, `run`, `String`) existing in more than one file is common, not rare, in most real codebases (confirmed: 12 duplicate names out of 500 probed symbols on a real ~1,900-node Go repo). `probeFileEdges` therefore re-probes exactly the symbols whose name appears in more than one **file** with `codegraph node -f `, the one file-qualified probe codegraph offers, and reads its trail line via `parseNodeCalls`. The "more than one file" precision is load-bearing: two symbols sharing a name *inside one file* (Go's `String()` on two types, two class methods) have no file-attribution ambiguity at all, the bare-name union is already exactly right for them, and routing them through `node -f` would *lose* edges — codegraph answers a same-file collision with two concatenated trail blocks. Costs, all deliberate: `node -f` returns the symbol's full source on every call, so this route is scoped to the duplicate subset rather than made the default; and it has **no `--json`**, so `parseNodeCalls` reads markdown text. That text is a human-facing summary, not an API, and is treated with matching suspicion — `parseNodeCalls` returns `null` (→ fall back to the bare-name probe, which over-reports rather than under-reports) on four measured conditions: no trail section at all (file-mode output, or a changed format); more than one trail section; a trail codegraph truncated with `+N more` (**it caps at 12 entries** — `main` in this repo has 23 callees and its trail shows 12, so taking the visible ones would trade fabricated edges for missing ones, the worse failure per the note above); or a reported `**Location:**` outside the file that was requested (`-f` is a *preference*, not a filter — measured: `node -f no/such/file.js -- buildDot` still answers with `render/callgraph.js`'s `buildDot`, exit 0). It also scans only after the trail marker, since the embedded source can itself contain trail-shaped lines, and `parseCodegraphOutput` matches codegraph's not-found message against only the first non-empty line for the same reason. **The residue that is still genuinely unfixable:** `unwrapQueryNodes` keeps *file* nodes in the probed set (see below), and `node -f` answers a file node in file mode — a different output shape — so two files named `index.js` in different directories remain indistinguishable to their bare-name probe. `duplicateNameWarning` reports both halves separately: which collisions were resolved, and which remain a real risk. - **`--architecture` mode probes file nodes' `callees`, not just named symbols', specifically to catch calls made from inside a top-level anonymous callback** (e.g. `test('...', () => { realCall() })` — a common pattern in test suites, including this repo's own `test/run.js`). codegraph attributes such a call to the enclosing file, not any named function, since no named function contains it; without probing the file node itself, `--architecture` mode would be structurally blind to that entire category of real cross-file dependency. This was confirmed against a real regression-turned-non-regression in codegraph itself: a codegraph 1.4.1 bug (fixed in 1.5.0, see `git log` for `LITERAL_RECEIVER_TYPES` in codegraph's history) briefly caused a *different*, spurious file-node-unrelated edge to appear in this project's own diagram — a call like `/regex/.test(x)` in `render/callgraph.js` got bare-name-matched to `test/run.js`'s own `test(name, fn)` helper purely by name collision. That edge is gone as of codegraph 1.5.0+ (correctly — it was never real); probing file nodes is what makes the *actual* dependency (`test/run.js` calling into `render/callgraph.js`) visible in its place. `probeFileEdges` skips any `"kind":"file"` *callee* it gets back from a probe — the same unverified-reference status that makes symbol mode draw it dotted/gray rather than as a real call (see Visual Encoding) means it must not be counted as a real cross-file edge here either, or this exact fabricated-edge problem reappears via a different mechanism. Verified empirically against this repo's own index (0 `"kind":"file"` callees among 100 probed) — but a repo where a file's *only* top-level reference is an unresolved `require(...)` with no other calls would exercise this path, and there's no dedicated test for it (real I/O against a live index would be needed to construct one). Two costs of probing file nodes, both real but not separately mitigated: enumeration and `--max-symbols` now compete real symbols against file nodes for the same fixed slot budget in an order codegraph doesn't guarantee (see the `--limit` note below) — on a repo near the cap, file nodes could crowd out real-symbol coverage with no warning distinguishing the two; and the probe count (and thus the already-"multi-minute" wall-clock cost) grows by roughly the repo's file count, since `probeFileEdges` is strictly sequential. -- **`--architecture` mode's enumeration query (`codegraph query --json --limit -- ''`) has confirmed, inconsistent `--limit` behavior worth knowing before trusting it.** Without `--limit`, an empty-string query silently caps around 50 results regardless of actual repo size (confirmed on a real 1,870-node index). Passing a large `--limit` (confirmed with both 500 and 2000 against that same index) instead returns *every* result codegraph has — more than the requested number, not capped at it. Codeshot works around this by always passing a very large `--limit` to force the "return everything" behavior, then applying the real `--max-symbols` cap client-side — but the *order* codegraph returns results in in that case is unknown (untested whether it's insertion order, alphabetical, ID-based, or something else), so on a repo larger than `--max-symbols`, the kept subset should not be assumed to sample evenly across the whole repo — it could be clustered by file, directory, or however codegraph happens to have stored them. +- **`--architecture` mode's enumeration query (`codegraph query --json --limit -- ''`) has confirmed, inconsistent `--limit` behavior worth knowing before trusting it.** Without `--limit`, an empty-string query silently caps around 50 results regardless of actual repo size (confirmed on a real 1,870-node index). Passing a large `--limit` (confirmed with both 500 and 2000 against that same index) instead returns *every* result codegraph has — more than the requested number, not capped at it. Codeshot works around this by always passing a very large `--limit` to force the "return everything" behavior, then applying the real `--max-symbols` cap client-side. The *order* codegraph returns results in in that case is unknown (untested whether it's insertion order, alphabetical, ID-based, or something else), which used to mean the kept subset was not even reproducible: the same unchanged repo could be probed on a different set of symbols run to run, yielding a different diagram — and a diagram guarded by `--check` in CI could therefore fail for no code reason at all. `sortSymbolsForEnumeration` now sorts by `filePath`+`name` before the slice, so the kept subset is a deterministic function of the index alone. What that does **not** fix, and can't from codeshot's side: the subset is still a *prefix*, now explicitly a path-ordered one, so on a repo larger than `--max-symbols` the scan covers the alphabetically-first files rather than an even sample across the repo. Raise `--max-symbols` if you need the whole picture. ![Architecture — generated by codeshot](docs/architecture.svg) diff --git a/USAGE.md b/USAGE.md index ebc16f9..ffb7879 100644 --- a/USAGE.md +++ b/USAGE.md @@ -41,10 +41,35 @@ them, labeled with how many calls. Boxes for test files render dashed, same as symbol mode. Unlike symbol mode, there's no `` argument — the `--architecture` flag replaces it, and combining the two is rejected. +**Too many files to read? Group them.** On anything bigger than a handful of +files, a box per file is a hairball. `--group-depth ` draws directories +instead — files roll up into their first `n` path segments, and the call counts +between them are summed: + +```bash +codeshot --architecture --path /path/to/repo --group-depth 1 --out modules.svg --format svg +``` + +`--group-depth 1` gives you the top-level module map (`src/` → `lib/`, +`test/` → `src/`); `--group-depth 2` splits one level finer (`src/api/` → +`src/db/`). Calls *inside* a group don't draw an arrow — the diagram is about +coupling between modules, the same reason calls inside one file don't draw one. +Prefer this over `--max-render` when the graph is dense: `--max-render` throws +the quiet files away entirely, while grouping keeps every *cross-module* call +and just zooms out. Calls between two files in the same group don't draw an +arrow, so the weights on a grouped diagram add up to less than the per-file +one's — that difference is the repo's intra-module traffic, not a dropped edge. +Files at the repo root have no directory to roll into, so they stay as +themselves. + This can genuinely take a few minutes on a mid-size-or-larger repo (one `codegraph` call per symbol, run sequentially) — `--max-symbols ` (default 500) trades completeness for speed if you want a faster, partial -scan; Codeshot tells you on stderr if it stopped early. `--limit` and +scan; Codeshot tells you on stderr if it stopped early. Note what "partial" +means: symbols are probed in path order, so the cut is a prefix, not a sample — +files late in path order go unprobed and can therefore appear to call nothing +when they really do. Raise `--max-symbols` before trusting a sparse-looking +corner of a big repo's diagram. `--limit` and `--max-render` carry over from symbol mode (see above); `--depth` doesn't apply here and is rejected if you pass it. @@ -70,7 +95,9 @@ block: Symbol mode works the same way, keyed by the symbol name (``), so several distinct diagrams can live in -one doc without clobbering each other. `--embed` **refreshes an existing doc — +one doc without clobbering each other. `--group-depth` gets its own key too +(``), so you can commit both the per-file and the +per-module architecture picture in the same doc and `--check` both. `--embed` **refreshes an existing doc — it won't create one**, and a stray/half-present marker pair is an error rather than a silent mangle. @@ -112,7 +139,8 @@ fall back to a raw byte-compare, which does require CI to use the same - **"codeshot: --depth traversal stopped early (safety cap of 200 discovered nodes)"** — The symbol is heavily connected enough that `--depth` hit its node-discovery cap before finishing; the graph you got is real but incomplete beyond that point. Try a smaller `--depth` (2 instead of 3), a lower `--limit`, a more specific, less-central symbol, or raise the cap itself with `codeshot --depth 3 --max-depth-nodes 500` (default is 200). - **`--depth` runs slowly** — Each additional hop makes one sequential `codegraph` call per newly discovered node (CodeGraph itself has no multi-hop traversal for `callers`/`callees`, so Codeshot does this client-side), so a well-connected symbol at `--depth 2` or higher can take noticeably longer than the default `--depth 1`. This is expected, not a bug. - **`--architecture` is taking a long time** — Expected on anything past a small repo: it's one sequential `codegraph` call per enumerated symbol, and there's no way to parallelize it (concurrent `codegraph` calls against one index race and fail). Rerun with a smaller `--max-symbols` (e.g. `--max-symbols 100`) for a faster, partial scan — Codeshot warns on stderr when the scan is cut short by the cap so you know the result is incomplete. -- **`--architecture`'s diagram is a hairball / unreadable** — Same fix as symbol mode: `--max-render ` (e.g. `--max-render 20`) keeps only the busiest N files by total call-edge weight and drops the rest. Some remaining files can end up with no surviving edges if all their edges pointed at a dropped file — that's expected, not a bug, at aggressive `--max-render` values. +- **`--architecture`'s diagram is a hairball / unreadable** — Reach for `--group-depth 1` first: it draws one box per top-level directory instead of one per file, so no *module-to-module* call is thrown away, the picture just zooms out (see "Generating a whole-repo architecture diagram" above). If it's still dense at group level, `--max-render ` (e.g. `--max-render 20`) then keeps only the busiest N nodes by total call-edge weight and drops the rest. Some remaining nodes can end up with no surviving edges if all their edges pointed at a dropped one — that's expected, not a bug, at aggressive `--max-render` values. +- **"codeshot: --group-depth N left no edges to draw"** — Every call in the repo is between files that share a group at that depth, so the grouped diagram is blank. The message tells you which of the two causes it is. *"every file falls into a single group"* (common at `--group-depth 1` where everything lives under `src/`) means the depth is too coarse — try 2 or 3. *"the N groups at this depth have no calls between them"* means the modules genuinely don't call each other at this depth; going deeper will stay blank, so drop the flag for the per-file graph. For anything not covered here, check `TECHNICAL.md` or open an issue on the GitHub repo. diff --git a/render/callgraph.js b/render/callgraph.js index aecdb9b..e0e8141 100755 --- a/render/callgraph.js +++ b/render/callgraph.js @@ -499,7 +499,13 @@ function unwrapQueryNodes(queryResults) { function symbolBudgetWarning(truncated, budget) { if (!truncated) return null; - return `codeshot: --architecture stopped enumerating after ${budget} symbols (--max-symbols) — the graph is incomplete; rerun with a larger --max-symbols to cover the rest of the repo.`; + // Names the SHAPE of the incompleteness, not just its existence: the kept + // subset is a path-sorted prefix (see sortSymbolsForEnumeration), so the + // missing symbols are not a random sample — they are the last files by path, + // and since only scanned symbols contribute outgoing edges, those files can + // render as sinks that appear to call nothing. "Incomplete" alone reads as + // "a few edges missing" and would leave that misreading in place. + return `codeshot: --architecture stopped enumerating after ${budget} symbols (--max-symbols) — the graph is incomplete; rerun with a larger --max-symbols to cover the rest of the repo. Note the cut is a path-sorted prefix, not a sample: files later in path order went unprobed, so they may appear to call nothing when they do.`; } // With zero cross-file edges, buildArchitectureDot emits a graph with no nodes — @@ -632,6 +638,61 @@ function aggregateFileEdges(symbolEdges) { return [...weights.values()]; } +// The group a file rolls up into at --group-depth : its first `n` DIRECTORY +// segments, with a trailing slash so a group reads as a directory rather than a +// file. A file with no directory at all (a repo-root file) has nothing to roll +// into and stays itself — grouping it under "" would invent a nameless node and +// silently merge every root file into one box. A file shallower than `n` keeps +// whatever directories it has, so a deeper --group-depth degrades to per-file +// output rather than erroring. Pure; posix and Windows separators both split. +function groupPath(filePath, depth) { + const parts = String(filePath || '').split(/[\\/]/).filter(Boolean); + const dirs = parts.slice(0, -1); + if (!dirs.length) return filePath; + return `${dirs.slice(0, depth).join('/')}/`; +} + +// Rewrites file-level edges as group-level edges, summing the weights of every +// file pair that collapses into the same group pair. Self-group edges are +// dropped for exactly the reason aggregateFileEdges drops self-file ones: once +// a directory is the unit, a call between two files inside it is intra-module +// structure, not the cross-module coupling this diagram is about. Returns the +// same {from, to, weight} shape, so everything downstream (topFilesByWeight, +// emptyArchitectureWarning, isTestRef's dashed test nodes, buildArchitectureDot) +// works on groups unchanged. Pure; `depth` unset returns the input untouched. +function rollupFileEdges(fileEdges, depth) { + if (!Number.isFinite(depth)) return fileEdges; + const weights = new Map(); + for (const e of fileEdges || []) { + const from = groupPath(e.from, depth); + const to = groupPath(e.to, depth); + if (from === to) continue; + const key = `${from} -> ${to}`; + weights.set(key, (weights.get(key) || { from, to, weight: 0 })); + weights.get(key).weight += e.weight; + } + return [...weights.values()]; +} + +// A rollup that eats every edge yields a blank image whose cause is the flag, +// not the code — emptyArchitectureWarning would blame a missing index instead. +// +// `groupCount` (how many distinct groups the pre-rollup endpoints mapped to) +// separates the two genuinely different causes, which want opposite advice: +// one group means the depth is too coarse and a deeper --group-depth will help; +// several groups means every call is intra-module at this depth, the diagram is +// a real (if boring) finding, and going deeper will keep returning blank. The +// earlier single-message version asserted "within one directory" in both cases, +// which contradicted the user's own tree and sent them down a dead end. +// Pure: takes the edge counts either side of the rollup, returns the string or null. +function groupCollapseWarning(beforeCount, afterCount, depth, groupCount) { + if (!Number.isFinite(depth) || beforeCount === 0 || afterCount > 0) return null; + const advice = groupCount > 1 + ? `the ${groupCount} groups at this depth have no calls between them, so there is genuinely no cross-module coupling to draw — a deeper --group-depth will stay blank; drop the flag for the per-file graph.` + : 'every file falls into a single group at this depth. Try a deeper --group-depth, or drop the flag for the per-file graph.'; + return `codeshot: --group-depth ${depth} left no edges to draw — all ${beforeCount} cross-file edge(s) collapsed within a group: ${advice}`; +} + // Top-N files by total in+out edge weight — simpler than a connected- // component/centrality algorithm, consistent with keeping v1 minimal. // `null` means "no cap" (mirrors allocateRenderBudget's no-op case). @@ -689,11 +750,37 @@ function architectureOutputBaseName(repoPath) { const ENUMERATION_QUERY_LIMIT = 100000; async function enumerateSymbols(repoPath, maxSymbols) { const results = await runCodegraph(['query', '--path', repoPath, '--json', '--limit', String(ENUMERATION_QUERY_LIMIT), '--', '']); - const symbols = unwrapQueryNodes(results); + const symbols = sortSymbolsForEnumeration(unwrapQueryNodes(results)); const truncated = symbols.length > maxSymbols; return { symbols: symbols.slice(0, maxSymbols), truncated }; } +// codegraph's order for the enumeration query is unspecified (untested whether +// it is insertion, alphabetical, or id order — see TECHNICAL.md), so on a repo +// larger than --max-symbols the slice above would keep a DIFFERENT subset run to +// run: the same repo, unchanged, could yield a different diagram each time, and +// a committed diagram guarded by --check could flap in CI for no code reason. +// Sorting by (filePath, name) makes the kept subset a deterministic function of +// the index alone. It does not make the subset representative — it is still a +// prefix, now explicitly a path-ordered one, so a truncated scan covers the +// code-point-first files rather than an even sample (symbolBudgetWarning says +// so). Pure. +// +// Compared by code point, deliberately NOT String#localeCompare: localeCompare +// with no explicit locale uses the *implementation-default* locale, which varies +// with the environment and with how the Node binary's ICU was built — so it +// would reintroduce, one layer down, exactly the run-to-run variability this +// function exists to remove (a dev box and a CI runner could keep different +// subsets of the same repo, and --check would fail with no code change). Code +// point order is machine-independent by construction. Its only cost is that +// 'Api.js' sorts before 'api.js'; nothing here needs human-facing collation. +function sortSymbolsForEnumeration(symbols) { + const byCodePoint = (a, b) => (a < b ? -1 : a > b ? 1 : 0); + return [...(symbols || [])].sort((a, b) => + byCodePoint(String(a.filePath || ''), String(b.filePath || '')) || + byCodePoint(String(a.name || ''), String(b.name || ''))); +} + // The file-qualified probe, used only for names that are actually ambiguous. // Returns null whenever the answer can't be trusted to be this file's complete // call list — see parseNodeCalls for the four cases — so the caller falls back @@ -754,7 +841,7 @@ async function probeFileEdges(symbols, repoPath, limit) { return edges; } -async function runArchitectureMode(repoPath, { limit, maxSymbols, maxRender }) { +async function runArchitectureMode(repoPath, { limit, maxSymbols, maxRender, groupDepth }) { const { symbols, truncated } = await enumerateSymbols(repoPath, maxSymbols); const symbolWarning = symbolBudgetWarning(truncated, maxSymbols); if (symbolWarning) console.error(symbolWarning); @@ -762,13 +849,26 @@ async function runArchitectureMode(repoPath, { limit, maxSymbols, maxRender }) { if (dupeWarning) console.error(dupeWarning); const symbolEdges = await probeFileEdges(symbols, repoPath, limit); - const fileEdges = aggregateFileEdges(symbolEdges); - + const rawFileEdges = aggregateFileEdges(symbolEdges); + // The rollup happens here, on aggregated edges, rather than by rewriting + // filePaths at probe time: probing must stay file-exact (parseNodeCalls' + // expectedFile check, isTestRef, the duplicate-name attribution all key off + // the real path), and collapsing afterwards keeps --group-depth a pure, + // testable view over the same data instead of a second scan mode. + const fileEdges = rollupFileEdges(rawFileEdges, groupDepth); + + const groupCount = Number.isFinite(groupDepth) + ? new Set(rawFileEdges.flatMap(e => [groupPath(e.from, groupDepth), groupPath(e.to, groupDepth)])).size + : 0; + const collapseWarning = groupCollapseWarning(rawFileEdges.length, fileEdges.length, groupDepth, groupCount); + if (collapseWarning) console.error(collapseWarning); const emptyWarning = emptyArchitectureWarning(fileEdges); - if (emptyWarning) console.error(emptyWarning); + if (emptyWarning && !collapseWarning) console.error(emptyWarning); + // Counted (and capped) after the rollup: --max-render bounds what is actually + // drawn, and with --group-depth the drawn nodes are groups, not files. const totalFiles = new Set(fileEdges.flatMap(e => [e.from, e.to])).size; - const note = renderTruncationNote('files', totalFiles, maxRender); + const note = renderTruncationNote(groupDepth ? 'groups' : 'files', totalFiles, maxRender); if (note) console.error(note); return buildArchitectureDot(fileEdges, { maxRender }); @@ -947,7 +1047,7 @@ function finishOutput(dot, { format, outFile, embedFile, check, markerId, alt }) console.log(outFile); } -const USAGE = 'Usage: callgraph.js [--path ] [--out ] [--limit ] [--max-render ] [--format ] [--depth ] [--max-depth-nodes ] [--embed [--check]]\n or: callgraph.js --architecture [--path ] [--out ] [--limit ] [--max-render ] [--max-symbols ] [--format ] [--embed [--check]]'; +const USAGE = 'Usage: callgraph.js [--path ] [--out ] [--limit ] [--max-render ] [--format ] [--depth ] [--max-depth-nodes ] [--embed [--check]]\n or: callgraph.js --architecture [--path ] [--out ] [--limit ] [--max-render ] [--max-symbols ] [--group-depth ] [--format ] [--embed [--check]]'; async function main() { let values, positionals; @@ -965,6 +1065,7 @@ async function main() { 'max-depth-nodes': { type: 'string' }, architecture: { type: 'boolean', default: false }, 'max-symbols': { type: 'string', default: String(DEFAULT_MAX_SYMBOLS) }, + 'group-depth': { type: 'string' }, embed: { type: 'string' }, check: { type: 'boolean', default: false }, }, @@ -991,6 +1092,10 @@ async function main() { const flagIndex = process.argv.indexOf('--max-symbols'); const badValue = flagIndex !== -1 ? process.argv[flagIndex + 1] : undefined; console.error(`codeshot: --max-symbols must be a positive integer, got '${badValue}'`); + } else if (err.code === 'ERR_PARSE_ARGS_INVALID_OPTION_VALUE' && /--group-depth/.test(err.message)) { + const flagIndex = process.argv.indexOf('--group-depth'); + const badValue = flagIndex !== -1 ? process.argv[flagIndex + 1] : undefined; + console.error(`codeshot: --group-depth must be a positive integer, got '${badValue}'`); } else { console.error(`codeshot: ${err.message}`); } @@ -1072,6 +1177,23 @@ async function main() { console.error(`codeshot: --max-symbols must be a positive integer, got '${values['max-symbols']}'`); process.exit(1); } + // Rejected outside --architecture rather than silently ignored (the stance + // --depth/--max-depth-nodes take in the opposite direction, not --max-symbols' + // quiet no-op): there is no file-level graph to roll up in symbol mode, so a + // --group-depth there is always a mistake, and a silently-dropped flag reads + // as "grouping applied" in exactly the diagram you'd then trust. + let groupDepth; + if (values['group-depth'] !== undefined) { + groupDepth = Number(values['group-depth']); + if (!Number.isInteger(groupDepth) || groupDepth <= 0) { + console.error(`codeshot: --group-depth must be a positive integer, got '${values['group-depth']}'`); + process.exit(1); + } + if (!values.architecture) { + console.error('codeshot: --group-depth only applies with --architecture (a symbol trail has no file-level graph to roll up)'); + process.exit(1); + } + } if (values.embed === '') { console.error('codeshot: --embed must not be empty'); process.exit(1); @@ -1092,7 +1214,8 @@ async function main() { const safeSymbol = values.architecture ? null : sanitizeForFilename(symbol); if (!outFile) { - const base = values.architecture ? `arch-${architectureOutputBaseName(repoPath)}` : `callgraph-${safeSymbol}`; + const archBase = groupDepth ? `arch-d${groupDepth}-${architectureOutputBaseName(repoPath)}` : `arch-${architectureOutputBaseName(repoPath)}`; + const base = values.architecture ? archBase : `callgraph-${safeSymbol}`; // With --embed the image must live at a STABLE path next to the doc — so the // relative link resolves, the file can be committed, and a re-run overwrites // the same file rather than littering tmp with timestamped copies. @@ -1113,15 +1236,23 @@ async function main() { if (healthWarning) console.error(healthWarning); if (values.architecture) { - const dot = await runArchitectureMode(repoPath, { limit, maxSymbols, maxRender }); + const dot = await runArchitectureMode(repoPath, { limit, maxSymbols, maxRender, groupDepth }); // Fixed, path-independent alt: deriving it from the checkout's directory // basename made the embedded markdown vary by where the repo was cloned // (a bare-worktree dir, "master", a branch name...), which both read wrong // and broke --check portability — a fresh clone under a different dir name // would report the committed diagram as drifted. The repo name is redundant // anyway; the diagram lives in that repo's own doc. - const alt = 'Architecture — generated by codeshot'; - finishOutput(dot, { format, outFile, embedFile, check: values.check, markerId: 'arch', alt }); + // The grouped view gets its own marker id (and default image name), so + // embedding it doesn't silently overwrite an ungrouped `codeshot:arch` + // block already committed in the same doc — the per-file and per-module + // pictures answer different questions and a repo may reasonably want both. + // Unset --group-depth keeps the plain 'arch' id, so existing docs are + // untouched. + const alt = groupDepth + ? `Architecture (grouped by directory, depth ${groupDepth}) — generated by codeshot` + : 'Architecture — generated by codeshot'; + finishOutput(dot, { format, outFile, embedFile, check: values.check, markerId: groupDepth ? `arch-d${groupDepth}` : 'arch', alt }); return; } @@ -1192,6 +1323,7 @@ module.exports = { depthBudgetWarning, allocateRenderBudget, formatMismatchWarning, matchSymbolNotFound, unwrapQueryNodes, symbolBudgetWarning, duplicateNameWarning, duplicateNames, parseNodeCalls, aggregateFileEdges, topFilesByWeight, buildArchitectureDot, architectureOutputBaseName, + groupPath, rollupFileEdges, groupCollapseWarning, sortSymbolsForEnumeration, applyEmbed, embedMarkers, embedRelLink, parseUnresolvedRefs, svgStructure, decodeXmlEntities, emptyGraphWarning, emptyArchitectureWarning, diff --git a/test/run.js b/test/run.js index e92b905..e448131 100644 --- a/test/run.js +++ b/test/run.js @@ -7,6 +7,7 @@ const { depthBudgetWarning, allocateRenderBudget, formatMismatchWarning, matchSymbolNotFound, unwrapQueryNodes, symbolBudgetWarning, duplicateNameWarning, duplicateNames, parseNodeCalls, aggregateFileEdges, topFilesByWeight, buildArchitectureDot, architectureOutputBaseName, + groupPath, rollupFileEdges, groupCollapseWarning, sortSymbolsForEnumeration, applyEmbed, embedMarkers, embedRelLink, parseUnresolvedRefs, svgStructure, decodeXmlEntities, emptyGraphWarning, emptyArchitectureWarning, @@ -879,6 +880,186 @@ test('architectureOutputBaseName sanitizes a repo path down to its basename', () assert.strictEqual(architectureOutputBaseName('/home/ericm/personal_projects/codeshot/master'), 'master'); }); +// --- --group-depth: rolling the file graph up into directory groups ------ + +test('groupPath rolls a nested file up to its first N directory segments', () => { + assert.strictEqual(groupPath('src/api/user.js', 1), 'src/'); + assert.strictEqual(groupPath('src/api/user.js', 2), 'src/api/'); +}); + +test('groupPath leaves a repo-root file as itself (no directory to roll into)', () => { + // Grouping it under '' would merge every root-level file into one nameless box. + assert.strictEqual(groupPath('index.js', 1), 'index.js'); +}); + +test('groupPath degrades to the file\'s own directories when depth exceeds the tree', () => { + assert.strictEqual(groupPath('render/callgraph.js', 5), 'render/'); +}); + +test('groupPath splits Windows separators too', () => { + assert.strictEqual(groupPath('src\\api\\user.js', 1), 'src/'); +}); + +test('rollupFileEdges sums the weights of every file pair collapsing into one group pair', () => { + const rolled = rollupFileEdges([ + { from: 'src/a.js', to: 'lib/x.js', weight: 2 }, + { from: 'src/b.js', to: 'lib/y.js', weight: 3 }, + ], 1); + assert.deepStrictEqual(rolled, [{ from: 'src/', to: 'lib/', weight: 5 }]); +}); + +test('rollupFileEdges drops edges that become self-group (intra-module calls)', () => { + const rolled = rollupFileEdges([ + { from: 'src/a.js', to: 'src/b.js', weight: 4 }, + { from: 'src/a.js', to: 'lib/x.js', weight: 1 }, + ], 1); + assert.deepStrictEqual(rolled, [{ from: 'src/', to: 'lib/', weight: 1 }]); +}); + +test('rollupFileEdges is a no-op when no --group-depth was given', () => { + const edges = [{ from: 'src/a.js', to: 'src/b.js', weight: 1 }]; + assert.strictEqual(rollupFileEdges(edges, undefined), edges); +}); + +test('groupCollapseWarning fires only when the rollup ate every edge', () => { + assert.match(groupCollapseWarning(7, 0, 1, 1), /--group-depth 1 left no edges to draw — all 7 cross-file edge\(s\)/); + assert.strictEqual(groupCollapseWarning(7, 3, 1, 2), null, 'edges survived — not a collapse'); + assert.strictEqual(groupCollapseWarning(0, 0, 1, 0), null, 'nothing to collapse — emptyArchitectureWarning owns this case'); + assert.strictEqual(groupCollapseWarning(7, 0, undefined, 0), null, 'no --group-depth — not the flag\'s doing'); +}); + +test('groupCollapseWarning gives opposite advice for one group vs several', () => { + // One group: the depth is too coarse, going deeper helps. + assert.match(groupCollapseWarning(3, 0, 1, 1), /every file falls into a single group.*Try a deeper --group-depth/s); + // Several groups: the modules genuinely don't call each other, so a deeper + // --group-depth stays blank — telling the user to go deeper would be a dead end. + const many = groupCollapseWarning(3, 0, 1, 2); + assert.match(many, /the 2 groups at this depth have no calls between them/); + assert.match(many, /a deeper --group-depth will stay blank/); + assert.doesNotMatch(many, /within one directory/, 'must not assert a single directory when there are several'); +}); + +test('sortSymbolsForEnumeration compares by code point, not locale collation', () => { + // localeCompare with no explicit locale uses the implementation-default + // locale, which varies with the environment and the Node binary's ICU build — + // that would reintroduce the run-to-run variance this sort exists to remove. + // Code point order puts uppercase before lowercase and '_' (U+005F) after + // uppercase; en-US collation does neither. + const sorted = sortSymbolsForEnumeration([ + { name: 'a', filePath: 'src/api.js' }, + { name: 'a', filePath: 'src/Api.js' }, + { name: 'a', filePath: 'src/_x.js' }, + ]); + assert.deepStrictEqual(sorted.map(s => s.filePath), ['src/Api.js', 'src/_x.js', 'src/api.js']); +}); + +test('symbolBudgetWarning names the cut as a path-sorted prefix, not a sample', () => { + // Otherwise "the graph is incomplete" reads as "a few edges missing", hiding + // that unprobed late-path files render as sinks that appear to call nothing. + const warning = symbolBudgetWarning(true, 500); + assert.match(warning, /path-sorted prefix, not a sample/); + assert.match(warning, /may appear to call nothing when they do/); +}); + +test('buildArchitectureDot still dashes a rolled-up test directory group', () => { + // isTestRef reads path segments, so the group's trailing slash must not defeat it. + const dot = buildArchitectureDot(rollupFileEdges([{ from: 'test/run.js', to: 'render/callgraph.js', weight: 1 }], 1)); + assert.match(dot, /"test\/"\s*\[style="rounded,filled,dashed"\]/); +}); + +test('sortSymbolsForEnumeration orders by filePath then name, so a --max-symbols slice is reproducible', () => { + const sorted = sortSymbolsForEnumeration([ + { name: 'zeta', filePath: 'b.js' }, + { name: 'beta', filePath: 'a.js' }, + { name: 'alpha', filePath: 'a.js' }, + ]); + assert.deepStrictEqual(sorted.map(s => `${s.filePath}:${s.name}`), ['a.js:alpha', 'a.js:beta', 'b.js:zeta']); +}); + +test('sortSymbolsForEnumeration does not mutate its input', () => { + const input = [{ name: 'z', filePath: 'b.js' }, { name: 'a', filePath: 'a.js' }]; + sortSymbolsForEnumeration(input); + assert.strictEqual(input[0].name, 'z'); +}); + +test('--group-depth is rejected outside --architecture (no file graph to roll up)', () => { + const { execFileSync } = require('child_process'); + let threw = false; + try { + execFileSync('node', [require('path').join(__dirname, '..', 'render', 'callgraph.js'), 'Foo', '--group-depth', '1'], { encoding: 'utf8', stdio: 'pipe' }); + } catch (err) { + threw = true; + assert.match(err.stderr, /--group-depth only applies with --architecture/); + } + assert.strictEqual(threw, true, 'expected --group-depth without --architecture to be rejected'); +}); + +test('--group-depth rejects a non-positive-integer value', () => { + const { execFileSync } = require('child_process'); + let threw = false; + try { + execFileSync('node', [require('path').join(__dirname, '..', 'render', 'callgraph.js'), '--architecture', '--group-depth', '0'], { encoding: 'utf8', stdio: 'pipe' }); + } catch (err) { + threw = true; + assert.match(err.stderr, /--group-depth must be a positive integer, got '0'/); + } + assert.strictEqual(threw, true, 'expected --group-depth 0 to be rejected'); +}); + +test('CLI --architecture --group-depth 1 draws directory groups, not files', () => { + const { execFileSync } = require('child_process'); + const path = require('path'); + const repoRoot = path.join(__dirname, '..'); + const callgraphJs = path.join(repoRoot, 'render', 'callgraph.js'); + + try { + execFileSync('codegraph', ['callers', '--path', repoRoot, '--limit', '1', '--json', '--', 'buildDot'], { stdio: 'pipe' }); + } catch { + console.log(' # skipped: `codegraph` not on PATH or this repo is not codegraph-indexed'); + return; + } + + // --format dot prints the DOT source itself, so the node ids are directly + // assertable — this repo's two source files live in render/ and test/. + const out = execFileSync('node', [callgraphJs, '--architecture', '--path', repoRoot, '--group-depth', '1', '--format', 'dot'], { encoding: 'utf8', stdio: 'pipe' }); + const dot = require('fs').readFileSync(out.trim(), 'utf8'); + try { + assert.match(dot, /"render\/"/, 'expected a render/ group node'); + assert.match(dot, /"test\/"/, 'expected a test/ group node'); + assert.doesNotMatch(dot, /callgraph\.js/, 'files must be rolled up, not drawn alongside their groups'); + } finally { + require('fs').rmSync(out.trim(), { force: true }); + } +}); + +test('CLI --embed --architecture --group-depth writes its own marker block, leaving an ungrouped one intact', () => { + const { execFileSync } = require('child_process'); + const path = require('path'); + const fs = require('fs'); + const os = require('os'); + const repoRoot = path.join(__dirname, '..'); + const callgraphJs = path.join(repoRoot, 'render', 'callgraph.js'); + + try { + execFileSync('codegraph', ['callers', '--path', repoRoot, '--limit', '1', '--json', '--', 'buildDot'], { stdio: 'pipe' }); + } catch { + console.log(' # skipped: `codegraph` not on PATH or this repo is not codegraph-indexed'); + return; + } + + const dir = fs.mkdtempSync(path.join(os.tmpdir(), 'codeshot-groupembed-')); + const doc = path.join(dir, 'DOC.md'); + fs.writeFileSync(doc, '# Doc\n\n\n![existing](existing.svg)\n\n', 'utf8'); + try { + execFileSync('node', [callgraphJs, '--architecture', '--path', repoRoot, '--group-depth', '1', '--embed', doc, '--format', 'svg'], { encoding: 'utf8', stdio: 'pipe' }); + const md = fs.readFileSync(doc, 'utf8'); + assert.match(md, //, 'grouped view must use its own marker id'); + assert.match(md, /!\[existing\]\(existing\.svg\)/, 'the pre-existing ungrouped block must be left alone'); + } finally { + fs.rmSync(dir, { recursive: true, force: true }); + } +}); + test('CLI --architecture runs end-to-end against this repo\'s own real codegraph index', () => { const { execFileSync } = require('child_process'); const path = require('path');