feat: replace Algolia search with self-contained llms.txt search

[mattpodwysocki]

Summary

search_mapbox_docs_tool no longer depends on the Algolia third-party service. The hosted server at mcp-docs.mapbox.com shares a single Algolia free-tier quota across every user, making it prone to throttling as adoption grows. This replaces it with a self-contained keyword search over the llms.txt index files that now exist at every product level on docs.mapbox.com (following the docs restructure in subdomain-docs-root#576).

How it works

On first search, 12 product llms.txt files are fetched in parallel (~220KB total). Each file contains a structured list of documentation pages with titles, URLs, and one-line descriptions. These are cached via the existing docCache for the 1-hour TTL. All subsequent searches are pure in-memory keyword matching — zero network calls after warm-up.

Products indexed (~220KB total, all cached):

Product	File	Size
API Reference	api/llms.txt	5.7KB
Mapbox GL JS	mapbox-gl-js/llms.txt	34KB
Help Center	help/llms.txt	39KB
Style Specification	style-spec/llms.txt	4.2KB
Studio Manual	studio-manual/llms.txt	2.5KB
Mapbox Search JS	mapbox-search-js/llms.txt	9.3KB
Maps SDK for iOS	ios/maps/llms.txt	21KB
Maps SDK for Android	android/maps/llms.txt	31KB
Navigation SDK for iOS	ios/navigation/llms.txt	16KB
Navigation SDK for Android	android/navigation/llms.txt	42KB
Mapbox Tiling Service	mapbox-tiling-service/llms.txt	6.8KB
Tilesets	data/tilesets/llms.txt	7.3KB

Scoring: Title matches rank 3×, description and URL path matches 1× each. Results are deduplicated by URL across sources (same page can appear in multiple product indexes) and capped at limit.

Reliability: Failed sources are silently skipped — if any single llms.txt is unreachable, the remaining 11 still return results. No single point of failure.

Trade-offs vs Algolia

	Algolia	llms.txt search
Relevance ranking	Full-text, weighted	Keyword frequency
Coverage	All indexed pages	12 curated products
Third-party dependency	Yes (quota risk)	None
Cold start	~100ms	~500ms (12 parallel fetches)
Warm queries	~100ms	<1ms (in-memory)
Rate limiting	Yes	None

For typical developer queries ("directions API parameters", "add marker GL JS", "style spec fill color") keyword matching against structured titles and descriptions is effective. The output format is the same shape — numbered list with title, URL, and description — so get_document_tool chaining works identically.

Also included

fetchCachedText(url, httpRequest) — new helper in docFetcher.ts for fetching and caching plain-text URLs. Shared between docsSearchIndex.ts and (in PR #27) the resource layer. Fixes a subtle bug where empty-string cache entries weren't recognized as hits due to a falsy check on '' — now uses !== null.

Test plan

• npm test — 68 tests pass (including 8 new SearchDocsTool tests, 7 new docsSearchIndex tests)
• CI green
• Manual: search "directions" → returns Directions API page
• Manual: search "add marker" → returns GL JS markers guide
• Manual: search "style fill color" → returns Style Spec fill-color reference

"Search Mapbox docs for isochrone"

🤖 Generated with Claude Code

[zmofei]

LGTM — good call removing the shared Algolia quota dependency.

Two minor notes (neither blocking):

1. Merge coordination with #27 — both PRs modify docFetcher.ts, the 5 resource classes, and docFetcher.test.ts. This PR's fetchCachedText uses the more correct !== null cache check (handles empty-string entries) vs #27's falsy check — make sure the better version survives the rebase.

2. fetchCachedText reads full body before size check — PR #27's version uses readBodyWithLimit() which caps reads early. Not a concern for llms.txt files (< 50KB each), but worth noting if this helper gets reused for larger URLs.

Search engine is clean — scoring weights, dedup, and graceful degradation on failed sources are all well-handled. Good test coverage across parsing, ranking, and resilience.

[mattpodwysocki]

On the merge coordination — the !== null check from this branch survived the merge into #27 (confirmed in the resolution commit). On response.text() vs readBodyWithLimit() — agreed it's a minor inconsistency, but since all the llms.txt files are well under 50KB it's not a practical concern. Could unify in a follow-up.