Skip to content

Generate per-package llms-{pkg}.txt from MkDocs build #196

@rororowyourboat

Description

@rororowyourboat

Summary

llms-full.txt is 1.6 MB+ — a complete dump of all documentation across 14+ packages. For agents working on a single DSL (the common case), this is context dilution: loading OWL/SHACL internals when working on a control-systems problem wastes tokens and increases hallucination risk.

Generate per-package segmented files alongside the existing full dump.

Proposed artifacts

File Contents Approximate size
site/llms.txt Index (exists) ~10 KB
site/llms-full.txt Everything (exists) ~1.6 MB
site/llms-framework.txt Framework reference + architecture ~100 KB
site/llms-stockflow.txt Stock-flow DSL guide + reference ~60 KB
site/llms-control.txt Control DSL guide + reference ~60 KB
site/llms-business.txt Business DSL guide + reference ~80 KB
site/llms-software.txt Software DSL guide + reference ~80 KB
site/llms-games.txt Games DSL guide + reference ~80 KB
site/llms-sim.txt Simulation (discrete + continuous) + analysis ~80 KB

Each segmented file includes:

  1. The framework overview prelude (shared — every segment needs the composition algebra basics)
  2. Only the sections from mkdocs.yml nav relevant to that package
  3. The "What GDS Is Not" prelude (once feat: add gds-proof subpackage #193 lands)

Implementation options

Option A: mkdocs-llmstxt plugin sections filter
Check if the llmstxt plugin supports multiple output files or section filters. If so, configure in mkdocs.yml directly.

Option B: Post-build script
A Python script in scripts/ that reads site/llms-full.txt, splits by heading markers (which correspond to package sections), and writes per-package files. Wire into mkdocs build as a post-build hook or Makefile target.

Option C: Multiple llmstxt plugin instances
If the plugin supports it, declare one instance per package with different section lists. Likely not supported — check docs.

Acceptance criteria

  • At least the 7 segmented files listed above are generated during mkdocs build
  • Each segmented file includes the shared framework prelude
  • site/llms.txt index links to all segmented files (not just full)
  • Generation is automated (part of build, not manual)
  • No hand-maintained files — all derived from MkDocs source

Dependencies

Metadata

Metadata

Assignees

No one assigned

    Labels

    documentationImprovements or additions to documentationtier-2Tier 2: Medium Priority

    Type

    No type

    Projects

    No projects

    Milestone

    No milestone

    Relationships

    None yet

    Development

    No branches or pull requests

    Issue actions