docs(fundamentals): add Architecture and Database pages [main]

[bloxster]

Summary

main twin of #21494. Same two new pages:

• /fundamentals/architecture — staged sync, modular processes, storage split, embedded Caplin, flat-KV state. Mermaid component diagram.
• /fundamentals/database — datadir layout, MDBX engine, snapshots, per-transaction history granularity, mainnet sizing, tuning flags, safe-to-delete table.

Stacked on #21496

Base is docs/prose-audit-main-2026-05-28 so links can target the renamed Prune Modes page. GitHub auto-retargets to main once #21496 merges.

Verification

• npm run build passes with onBrokenLinks/onBrokenAnchors: 'throw'
• docs-site / build (incl. llms verification) ✓ pass
• llms.txt and llms-full.txt regenerated

Notes

• Sourced from the Improving-public-docs spec and the README.md "Datadir structure" / "Erigon3 datadir size" / "Modularity" sections.
• Adds @docusaurus/theme-mermaid@^3.10.0 (used by the architecture diagram).

[yperbasis]

Content and structure are good, but I checked the load-bearing claims against the code and found factual errors that should be fixed before merge.

1. --internalcl does not exist (architecture.md). There is no internalcl flag anywhere in the tree — only externalcl (cmd/utils/flags.go:172). Caplin is embedded by default; you opt out with --externalcl. Reword to: "Caplin runs embedded by default; pass --externalcl to use an external CL."

2. --batchSize example contradicts itself (database.md). Default is 512M (node/cli/flags.go:48), so the suggested --batchSize 1G raises it, not "lower it". Also batchSize is the execution-stage RAM buffer — tying it to "chaindata/ grows unexpectedly" is a dubious rationale. Fix the number or the framing.

3. Cross-page contradiction on receipts. architecture.md pipeline says "Finalization → persist receipts", but database.md correctly says "Receipts are not stored — they are re-computed". Drop "persist receipts" from the pipeline.

4. Mermaid diagram edge TxPool <-- private gRPC --> Caplin is wrong. TxPool does not talk to Caplin (the CL). Its gRPC relationship is with the core/Sentry. This edge should not connect to Caplin.

5. Snapshot sizing doesn't add up (database.md). Listed dirs sum to ~1.13 TB (accessor 120 + domain 300 + history 280 + idx 430) but "snapshots TOTAL 2.3 TB". The gap is presumably the unlisted blocks/transactions .seg files — add those rows or note the breakdown is partial.

6. Likely dead link. https://github.com/erthink/libmdbx — that GitHub repo was taken down; Erigon vendors github.com/erigontech/mdbx-go. onBrokenLinks only checks internal links so the build won't catch it.

All are text edits, no restructuring needed.

[Copilot]

Pull request overview

Adds two new Fundamentals documentation pages (“Architecture” and “Database”) and enables Mermaid rendering in the Docusaurus site so the architecture page can include a component diagram. Also updates the generated llms*.txt artifacts to include the new pages.

Changes:

• Add new /fundamentals/architecture and /fundamentals/database pages (incl. Mermaid diagram and datadir/MDBX/snapshots details).
• Enable Mermaid support in the docs site via @docusaurus/theme-mermaid and config updates.
• Regenerate llms.txt / llms-full.txt (and docs/site/static/ copies) to include the new pages.

Reviewed changes

Copilot reviewed 8 out of 9 changed files in this pull request and generated 3 comments.

Show a summary per file

File	Description
llms.txt	Adds links to the new Fundamentals pages.
llms-full.txt	Adds full-text entries for the new pages.
docs/site/static/llms.txt	Mirrors llms.txt updates for the site static copy.
docs/site/static/llms-full.txt	Mirrors llms-full.txt updates for the site static copy.
docs/site/package.json	Adds Mermaid theme dependency.
docs/site/package-lock.json	Locks Mermaid/theme transitive dependencies.
docs/site/docusaurus.config.ts	Enables Mermaid in markdown and registers the Mermaid theme.
docs/site/docs/fundamentals/architecture.md	New Architecture page with Mermaid component diagram and system overview.
docs/site/docs/fundamentals/database.md	New Database page describing datadir layout, MDBX, snapshots, and sizing/tuning.

Files not reviewed (1)

• docs/site/package-lock.json: Language not supported

---

💡 Add Copilot custom instructions for smarter, more guided reviews. Learn how to get started.

[Copilot]

Copilot encountered an error and was unable to review this pull request. You can try again by re-requesting a review.

[yperbasis]

Thanks for the quick turnaround — the six items from my previous review are all correctly addressed (verified --externalcl at cmd/utils/flags.go:173, the --batchSize default 512M at node/cli/flags.go:51, the Caplin-edge removal, the sizing note, and the mdbx-go link). I re-checked the remaining load-bearing claims against main and found a few more factual issues — including one I got wrong in my last review. Requesting changes for these. All are text edits, no restructuring.

1. database.md: "Receipts are not stored — they are re-computed" is incorrect (and a correction to my earlier review)

In my previous review I claimed database.md "correctly says Receipts are not stored" and on that basis asked to drop "persist receipts" from the architecture pipeline. That premise was wrong on my part — receipts are stored:

• db/kv/tables.go:715-716: ReceiptDomain — "Tiny Receipts - without logs. Required for node-operations."; RCacheDomain — "Fat Receipts - with logs. Optional."
• Written every txn during Execution: db/rawdb/rawtemporaldb/accessors_receipt.go (AppendReceipt → DomainPut(kv.ReceiptDomain, …) for cumulative gas / blob gas / log-index) and db/rawdb/accessors_chain.go:1373 (DomainPut(kv.RCacheDomain, …) for full receipts).
• Both are registered aggregated domains, so they produce .kv files in snapshots/domain/ (db/state/statecfg/state_schema.go:50,53).
• The RPC path is a hybrid, not pure re-exec: rpc/jsonrpc/receipts/receipts_generator.go reads stored metadata via ReceiptAsOf + cached full receipts, and only re-executes to re-derive logs when the full receipt isn't cached.

Please reword to something like: full receipts (with logs) aren't persisted by default; Erigon stores compact per-txn receipt metadata (cumulative gas, blob gas, log index) in a required receipt domain and optionally caches full receipts, reconstructing receipts on demand and re-deriving logs by re-execution when needed. Dropping "persist receipts" from the Finalization step was still correct — it's Execution that writes the receipt domain, not Finalization.

2. database.md: "4 domains: account, storage, code, commitment" — there are 6

db/kv/tables.go:711-717 defines 6 domains. The four you list are StateDomains (:720); the other two are ReceiptDomain and RCacheDomain, and all six produce files under snapshots/domain/. Either say "the four state domains" or list all six. (Tied to #1 — receipts literally are domains in that directory.)

3. architecture.md: prune full is not "post-Merge blocks"

FullMode.Blocks = Distance(config3.DefaultPruneDistance) = 262,144 blocks (db/kv/prune/storage_mode.go:38-42, db/config3/config3.go:48) — a rolling window, not all post-Merge blocks. KeepPostMergeBlocksPruneMode is a separate sentinel that is no longer the full default. Suggest matching the flag usage ("keep only necessary blocks + latest state", node/cli/flags.go:79). The (default) label on full is correct.

4. architecture.md: pipeline lists "Commitment" as a separate stage, contradicting the next paragraph

Step 5 "Commitment" is shown as its own stage, but the following paragraph says Execution absorbs stage_trie into a single pass — and DefaultStages (execution/stagedsync/default_stages.go) has no Commitment stage; it runs inside Execution. Also, Snapshots/OtterSync is stage #1 in the real pipeline, not #3 after Headers/Bodies. Either fold Commitment into the Execution line or note it's part of Execution.

Minor

• architecture.md prune table omits the blocks mode. node/cli/flags.go:77-83 lists four modes (full, archive, minimal, blocks); the table shows three. Add a row or note it's a subset.
• database.md "rm -rf chaindata/ is recoverable / rebuilds from snapshots" — please double-check before we publish this as safe. chaindata/ holds recent blocks and canonical data not yet folded into snapshots, and block download over devp2p was just removed (#21505, engine-API-only), so the refill path may not behave as the doc implies.
• Mermaid Sentry --> Downloader edge: the Downloader is BitTorrent-based and independent of Sentry (devp2p). Low priority for a high-level diagram, but the linear Sentry→Downloader→Execution→RPC chain implies a data flow that doesn't exist.
• For the open Copilot comments: please ignore the suggestion to re-add the TxPool↔Caplin edge (it's reasoning from a since-regenerated llms-full; removing it was correct). Pinning @docusaurus/theme-mermaid to an exact version, as Copilot suggests, is reasonable given the other Docusaurus deps are pinned.

[bloxster]

Thanks @yperbasis — all verified against main and pushed in 32e1534. Point by point:

1. Receipts — corrected. ReceiptDomain (required) + RCacheDomain (optional) confirmed at db/kv/tables.go:715-716. Reworded to: compact per-txn receipt metadata (cumulative gas, blob gas, log index) persisted in a required domain, full receipts optionally cached, logs re-derived by re-execution only when uncached. Thanks for catching your own earlier note too.
2. 6 domains, not 4 — fixed in both the tree comment and the snapshots/domain/ table row; now lists the 4 state domains + receipt/rcache.
3. prune full — fixed to "latest state + a rolling window of recent blocks (~262k, the default prune distance)" per FullMode.Blocks = Distance(config3.DefaultPruneDistance).
4. Pipeline / Commitment — Snapshots is now stage Pull from go-ethereum up to 26aea73 (7 Feb 2019) #1, added Senders, and folded Commitment into Execution with an explicit note that there's no separate Commitment stage (matches DefaultStages). Also fixed the "Commitment stage" reference further down the page.

Minor:

• Added the blocks prune mode to the table (now all four).
• rm -rf chaindata/ — softened. Now: recoverable but not free — re-derives state from snapshots and resyncs the tip from the CL over the Engine API, with devp2p block download removed (execution: remove legacy headerdownload, bodydownload and dataflow packages #21505) called out. Framed as a tip resync, not a quick rebuild.
• Mermaid — removed the Sentry → Downloader → Execution chain; Downloader now independently writes snapshots to the datadir, and blocks flow from Caplin via the Engine API. Sentry feeds the TxPool (gossip).

On Copilot: left the TxPool↔Caplin edge out as you advised. @docusaurus/theme-mermaid is already pinned exact (3.10.0, matching the other Docusaurus deps in package.json), so no lockfile change was needed there.

[bloxster]

@yperbasis Thanks again for the careful code-level review. All items from the latest round are now addressed in the branch (commits 32e1534, a7d67d0, c8d6ed7), and I re-verified each load-bearing claim against main:

1. Receipts are stored (database.md) — reworded. The doc now states that Erigon stores compact per-txn receipt metadata (cumulative gas, blob gas, log index) in a required receipt domain, and that full receipts (with logs) live in a separate cache domain persisted by default for full/minimal/blocks and skipped by default for archive, reconstructed on demand (re-deriving logs by re-execution) when not cached.

• db/kv/tables.go:715-716 — ReceiptDomain ("Required for node-operations") / RCacheDomain ("Fat Receipts - with logs. Optional").
• node/cli/flags.go:238 — persistenceReceiptsV2 := ctx.String(PruneModeFlag.Name) != prune.ArchiveMode.String(), overridable via --persist.receipts. So the per-mode default is exactly as documented.

2. Six domains, not four — database.md now reads "6 domains: the 4 state domains (account, storage, code, commitment) plus 2 receipt domains (receipt, rcache)", matching db/kv/tables.go:711-717 (StateDomains at :720 is the 4-domain subset).

3. Prune full is a rolling window — architecture.md now says "latest state + a rolling window of recent blocks (the default prune distance, ~262k blocks)". Confirmed: FullMode.Blocks = Distance(config3.DefaultPruneDistance) and DefaultPruneDistance = 262_144 (db/kv/prune/storage_mode.go:38-40, db/config3/config3.go:48).

4. Commitment folded into Execution — the separate Commitment pipeline step is gone; the doc now lists Snapshots as stage #1 and notes explicitly there is no separate Commitment stage (trie/state-root runs inside Execution). Matches DefaultStages in execution/stagedsync/default_stages.go (Snapshots → Headers → BlockHashes → Bodies → Senders → Execution → TxLookup → Finish; no Commitment stage).

Minor

• blocks mode added to the prune table in architecture.md (all four modes now listed).
• rm -rf chaindata/ reworded — it's now described as recoverable-but-not-free: a resync of the post-snapshot tip from the CL over the Engine API, explicitly noting blocks are no longer pulled over devp2p (per execution: remove legacy headerdownload, bodydownload and dataflow packages #21505). No longer implies a quick rebuild.
• Mermaid diagram fixed: removed the Sentry --> Downloader edge; flow is now Caplin -->|new blocks (Engine API)| Execution and Sentry -->|tx gossip| TxPool.
• Copilot comments: did not re-add the TxPool↔Caplin edge (per your note); @docusaurus/theme-mermaid pinned to an exact version to match the other Docusaurus deps.

llms-full.txt has also been regenerated so the docs build's up-to-date check passes. Ready for another look when you have a moment.

[Copilot]

Pull request overview

Copilot reviewed 9 out of 10 changed files in this pull request and generated no new comments.

Files not reviewed (1)

• docs/site/package-lock.json: Language not supported