Skip to content

Fix overlapping scope roots for nested Codex config paths#3719

Merged
reakaleek merged 1 commit into
mainfrom
cursor/ce66b079
Jul 24, 2026
Merged

Fix overlapping scope roots for nested Codex config paths#3719
reakaleek merged 1 commit into
mainfrom
cursor/ce66b079

Conversation

@reakaleek

Copy link
Copy Markdown
Member

Why

docs-builder 1.29.0 scopes Codex commands with FindGitRoot(config.FullName) as an extension root. For configs under environments/internal/config.yml, that root sits inside WorkingDirectoryRoot, and ScopedFileSystem throws Scope roots must be disjoint. This breaks codex-environments CI (docs-builder codex clone) on every run since 1.29.0 shipped.

What

Normalize extension roots in FileSystemFactory.ScopeCurrentWorkingDirectory before building scoped filesystem options: deduplicate paths and drop roots already covered by an ancestor. Adds regression tests for nested in-repo configs and genuinely external config roots.

After merge and release, codex-environments workflows using version: latest should recover automatically. Until then, that repo can pin docs-builder to 1.28.0 as a workaround.

Made with Cursor

docs-builder 1.29.0 added extension roots from FindGitRoot for Codex
commands. When config.yml lives under environments/internal/, the resolved
root is nested inside WorkingDirectoryRoot and ScopedFileSystem rejects
the overlapping roots. Normalize extension roots before constructing the
scoped filesystem so redundant nested paths are dropped.

Co-Authored-By: Claude Sonnet 4.6 (1M context) <noreply@anthropic.com>
Co-authored-by: Cursor <cursoragent@cursor.com>
@reakaleek reakaleek added the bug label Jul 24, 2026
@reakaleek
reakaleek requested a review from a team as a code owner July 24, 2026 10:05
@reakaleek reakaleek added the bug label Jul 24, 2026
@reakaleek
reakaleek requested a review from technige July 24, 2026 10:05
@reakaleek
reakaleek merged commit a96ef86 into main Jul 24, 2026
26 checks passed
@reakaleek
reakaleek deleted the cursor/ce66b079 branch July 24, 2026 11:14
@Mpdreamz

Copy link
Copy Markdown
Member

The Problem

The codex-environments repo has configs nested inside the checkout:

/home/runner/work/codex-environments/codex-environments/   ← WorkingDirectoryRoot (has .git)
├── .git/
├── environments/
│   └── internal/
│       └── config.yml   ← config passed to `docs-builder codex clone`
└── ...

When a Codex command runs, it calls:

FileSystemFactory.ScopeCurrentWorkingDirectory(fs, [Paths.FindGitRoot(config.FullName)]);

FindGitRoot starts from the config file's directory and walks up looking for .git, but has a depth-1 limit in release builds. The .git is 2 levels up:

environments/internal/   ← depth 0: no .git
environments/            ← depth 1: no .git
repo root/               ← depth 2: .git found, but depth > 1 → REJECTED

Since .git is "too deep", FindGitRoot falls back to returning startDir — the config's own directory:

FindGitRoot("…/environments/internal/config.yml")
  → "/home/runner/…/codex-environments/environments/internal"

Now the original ScopeCurrentWorkingDirectory builds scope roots:

roots = [
  WorkingDirectoryRoot  →  "/home/runner/…/codex-environments"          ← ROOT
  ApplicationData       →  "/home/runner/.local/share/elastic/docs-builder"
  extensionRoot         →  "/home/runner/…/codex-environments/environments/internal"  ← CHILD OF ROOT!
]

.Distinct() doesn't remove anything (they're different strings). roots.Length == 3 ≠ 2, so it doesn't hit the fast path. ScopedFileSystem then enforces that all scope roots must be disjoint (no parent–child relationships):

   ❌ BOOM: ancestor/descendant relationship detected

   /home/runner/…/codex-environments/           ← scope root A
   └── environments/
       └── internal/                            ← scope root B (child of A)

The Fix

The PR adds NormalizeDisjointRoots which processes roots shortest-first and drops any root already covered by a kept ancestor:

Input roots (sorted by length):
  1. "/home/runner/…/codex-environments"                        → KEEP ✓
  2. "/home/runner/.local/share/elastic/docs-builder"           → KEEP ✓ (not under #1)
  3. "/home/runner/…/codex-environments/environments/internal"  → SKIP ✗ (child of #1)

Output:
  ["/home/runner/…/codex-environments", "/home/runner/.local/share/elastic/docs-builder"]

Since only the 2 default roots remain (and the updated guard checks they're specifically WorkingDirectoryRoot + ApplicationData), the fast path kicks in and it falls through to the simple ScopeCurrentWorkingDirectory(inner) — no extra roots, no disjoint violation.

For genuinely external extension roots (e.g., detection-rules in a separate checkout), they survive the filter because they're not descendants of any other root:

Input roots (sorted by length):
  1. "/home/runner/…/codex-environments"                     → KEEP ✓
  2. "/tmp/external-checkout"                                 → KEEP ✓ (not under #1)
  3. "/home/runner/.local/share/elastic/docs-builder"         → KEEP ✓ (not under #1 or #2)

Output: all 3 roots kept → ScopedFileSystem gets 3 disjoint roots ✓

@Mpdreamz

Copy link
Copy Markdown
Member

Why ScopedFileSystem throws instead of normalizing

It's a deliberate security design choice. From PathValidator.ValidateRootsAreDisjoint in scoped-filesystem:

Roots must be fully disjoint; nesting is not permitted because it creates ambiguity about which policies apply and makes reasoning about the effective scope significantly harder.

ScopedFileSystemOptions supports per-construction policies (hidden file/folder allowlists, special folder exemptions). If roots could nest, the library would have to answer: when a path matches both a parent root and a child root, whose policy wins? For example:

Root A: /repo             → AllowedHiddenFolders: { ".git" }
Root B: /repo/internal    → AllowedHiddenFolders: { }

Does /repo/internal/.git/config pass or fail? Root A says .git is fine, Root B says it's not, and the path is under both. The library would need precedence rules (most-specific wins? union? intersection?), and any choice would surprise some callers.

By throwing eagerly, the library forces the caller to decide what the scope actually is — either provide genuinely disjoint roots, or collapse nested ones to the common ancestor. Same philosophy as the compiler rejecting ambiguous overloads rather than silently picking one.


Follow-up simplification

The NormalizeDisjointRoots approach here treats all inputs equally (sorts by length, collapses any parent–child pair). But WorkingDirectoryRoot and ApplicationData are fixed — they're always present and always disjoint. Only extension roots can collide with WorkingDirectoryRoot.

Will open a follow-up that simplifies this to just filtering extension roots against WorkingDirectoryRoot using the existing IsSubPathOf extension, instead of the generic shortest-first algorithm with new helper methods.

Mpdreamz pushed a commit that referenced this pull request Jul 24, 2026
docs-builder 1.29.0 added extension roots from FindGitRoot for Codex
commands. When config.yml lives under environments/internal/, the resolved
root is nested inside WorkingDirectoryRoot and ScopedFileSystem rejects
the overlapping roots. Normalize extension roots before constructing the
scoped filesystem so redundant nested paths are dropped.

Co-authored-by: Claude Sonnet 4.6 (1M context) <noreply@anthropic.com>
Co-authored-by: Cursor <cursoragent@cursor.com>
Mpdreamz added a commit that referenced this pull request Jul 24, 2026
* Emit Mermaid diagrams as external SVG files loaded via img (#3713)

Co-authored-by: Claude Sonnet 4.6 (1M context) <noreply@anthropic.com>

* Fix CopyBrandingResources failure during serve in-memory build (#3718)

Co-authored-by: Claude Sonnet 4.6 (1M context) <noreply@anthropic.com>
Co-authored-by: Cursor <cursoragent@cursor.com>

* Fix overlapping scope roots for nested Codex config paths (#3719)

docs-builder 1.29.0 added extension roots from FindGitRoot for Codex
commands. When config.yml lives under environments/internal/, the resolved
root is nested inside WorkingDirectoryRoot and ScopedFileSystem rejects
the overlapping roots. Normalize extension roots before constructing the
scoped filesystem so redundant nested paths are dropped.

Co-authored-by: Claude Sonnet 4.6 (1M context) <noreply@anthropic.com>
Co-authored-by: Cursor <cursoragent@cursor.com>

* Simplify scope root normalization for nested extension roots (#3720)

Co-authored-by: Claude Sonnet 4.6 (1M context) <noreply@anthropic.com>
Co-authored-by: Cursor <cursoragent@cursor.com>

* Replace pagefind native binary with Pagefind.Net managed library

Replace PR #3709's native pagefind binary subprocess approach with the
Pagefind.Net and Pagefind.Net.Frontend NuGet packages — pure managed
.NET, no native binary, AOT-compatible.

The pagefind indexer is now a proper IMarkdownExporter that eagerly
indexes each page during ExportAsync (O(1) memory) and flushes the
index to disk in FinishExportAsync. It is included in both
ExportOptions.Default and ExportOptions.Validation, so it works in
isolated builds and docs-builder serve mode automatically.

Key changes:
- New PagefindMarkdownExporter in Elastic.Markdown/Exporters/Pagefind
- New Exporter.Pagefind enum value, registered in both ExporterParsers
- Removed native binary pipeline (package-pagefind.mjs, embedded
  resource, pagefind npm dep, PagefindSearchIndexer subprocess)
- Removed static-search feature flag — search is always available
  for isolated builds and serve mode
- Serve mode reads pagefind files from the background build's
  in-memory filesystem via a route handler
- Added .pagefind-net-frontend-version to AllowedHiddenFileNames

Co-Authored-By: Claude Sonnet 4.6 (1M context) <noreply@anthropic.com>
Co-authored-by: Cursor <cursoragent@cursor.com>

---------

Co-authored-by: Claude Sonnet 4.6 (1M context) <noreply@anthropic.com>
Co-authored-by: Cursor <cursoragent@cursor.com>
Co-authored-by: Jan Calanog <jan.calanog@elastic.co>
Mpdreamz added a commit that referenced this pull request Jul 25, 2026
* Emit Mermaid diagrams as external SVG files loaded via img (#3713)

Co-authored-by: Claude Sonnet 4.6 (1M context) <noreply@anthropic.com>

* Fix CopyBrandingResources failure during serve in-memory build (#3718)

Co-authored-by: Claude Sonnet 4.6 (1M context) <noreply@anthropic.com>
Co-authored-by: Cursor <cursoragent@cursor.com>

* Fix overlapping scope roots for nested Codex config paths (#3719)

docs-builder 1.29.0 added extension roots from FindGitRoot for Codex
commands. When config.yml lives under environments/internal/, the resolved
root is nested inside WorkingDirectoryRoot and ScopedFileSystem rejects
the overlapping roots. Normalize extension roots before constructing the
scoped filesystem so redundant nested paths are dropped.

Co-authored-by: Claude Sonnet 4.6 (1M context) <noreply@anthropic.com>
Co-authored-by: Cursor <cursoragent@cursor.com>

* Simplify scope root normalization for nested extension roots (#3720)

Co-authored-by: Claude Sonnet 4.6 (1M context) <noreply@anthropic.com>
Co-authored-by: Cursor <cursoragent@cursor.com>

* Replace pagefind native binary with Pagefind.Net managed library

Replace PR #3709's native pagefind binary subprocess approach with the
Pagefind.Net and Pagefind.Net.Frontend NuGet packages — pure managed
.NET, no native binary, AOT-compatible.

The pagefind indexer is now a proper IMarkdownExporter that eagerly
indexes each page during ExportAsync (O(1) memory) and flushes the
index to disk in FinishExportAsync. It is included in both
ExportOptions.Default and ExportOptions.Validation, so it works in
isolated builds and docs-builder serve mode automatically.

Key changes:
- New PagefindMarkdownExporter in Elastic.Markdown/Exporters/Pagefind
- New Exporter.Pagefind enum value, registered in both ExporterParsers
- Removed native binary pipeline (package-pagefind.mjs, embedded
  resource, pagefind npm dep, PagefindSearchIndexer subprocess)
- Removed static-search feature flag — search is always available
  for isolated builds and serve mode
- Serve mode reads pagefind files from the background build's
  in-memory filesystem via a route handler
- Added .pagefind-net-frontend-version to AllowedHiddenFileNames

Co-Authored-By: Claude Sonnet 4.6 (1M context) <noreply@anthropic.com>
Co-authored-by: Cursor <cursoragent@cursor.com>

---------

Co-authored-by: Claude Sonnet 4.6 (1M context) <noreply@anthropic.com>
Co-authored-by: Cursor <cursoragent@cursor.com>
Co-authored-by: Jan Calanog <jan.calanog@elastic.co>
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

Projects

None yet

Development

Successfully merging this pull request may close these issues.

2 participants