feat: replace Algolia search with self-contained llms.txt search

.gitignore

@@ -145,3 +145,4 @@ vite.config.ts.timestamp-*
 # Test artifacts
 test-results.xml
 coverage/
+.claude/

CHANGELOG.md

@@ -1,5 +1,32 @@
 ## Unreleased
 
+### Replace Algolia search with self-contained llms.txt search
+
+`search_mapbox_docs_tool` no longer depends on the Algolia third-party service. The hosted server shares a single Algolia free-tier quota across all users, making it prone to throttling as usage grows. The new implementation searches directly against the `llms.txt` index files that now exist at every product level on docs.mapbox.com.
+
+**How it works:**
+
+On first search, the tool fetches 12 product `llms.txt` files in parallel (~220KB total). Each file contains a structured list of documentation pages with titles, URLs, and one-line descriptions. These files are cached for the standard 1-hour TTL, so subsequent searches are pure in-memory keyword matching — no network calls.
+
+**Products indexed:**
+
+- API Reference (`api/llms.txt`)
+- Mapbox GL JS (`mapbox-gl-js/llms.txt`)
+- Help Center (`help/llms.txt`)
+- Style Specification (`style-spec/llms.txt`)
+- Studio Manual (`studio-manual/llms.txt`)
+- Mapbox Search JS (`mapbox-search-js/llms.txt`)
+- Maps SDK for iOS and Android
+- Navigation SDK for iOS and Android
+- Mapbox Tiling Service
+- Tilesets
+
+**Scoring:** Title matches (3×) outrank description matches (1×) and URL path matches (1×). Results are deduplicated by URL across sources and capped at the requested `limit`.
+
+**Reliability:** Failed sources are silently skipped — if any single product `llms.txt` is unreachable, the remaining sources still return results.
+
+**`fetchCachedText(url, httpRequest)`** — new helper in `docFetcher.ts` that fetches a URL and stores the response in `docCache`. Used by `docsSearchIndex.ts` to share the cache with the resource layer (which also caches `llms.txt` files). Fixed a subtle bug where empty-string responses (`''`) were not treated as cache hits due to falsy check — now uses `!== null`.
+
 ### Raise `docCache` per-entry limit to 5 MB with size warnings
 
 - **Hard cap raised from 2 MB → 5 MB** — allows `llms-full.txt` files (Style Spec 466 KB, iOS Nav 696 KB, GL JS 1.6 MB) to be cached after being fetched via `get_document_tool`
@@ -10,6 +37,22 @@
 
 - **Upgrade `tshy` to `^4.1.1`, `vitest` to `^4.1.4`, `typescript` to `^6.0.2`** — removed deprecated `baseUrl` from `tsconfig.base.json` (TS6), added `"types": ["node"]` (required because tshy compiles from `.tshy/` and does not auto-discover `@types/node` in CI); downgraded `@types/node` to `^22.0.0` for LTS consistency with other repos; bumped `typescript-eslint` packages to `^8.58.2` for TypeScript 6 support
 
+### Resources — use sublevel `llms.txt` per product
+
+docs.mapbox.com restructured its documentation so that `llms.txt` files now exist at every product level (e.g. `docs.mapbox.com/api/llms.txt`, `docs.mapbox.com/help/llms.txt`, `docs.mapbox.com/mapbox-gl-js/llms.txt`) alongside `llms-full.txt` files containing full page content. The root `docs.mapbox.com/llms.txt` is now a pure index of links to these sublevel files rather than a monolithic content file. The previous resources all filtered the root file by category keyword — now that the root contains only link lists, they were effectively returning empty or useless content.
+
+Updated resources to use the appropriate sublevel `llms.txt` files:
+
+- **`resource://mapbox-api-reference`** now fetches `docs.mapbox.com/api/llms.txt` — a clean, structured index of every Mapbox REST API grouped by service (Maps, Navigation, Search, Accounts) with links to full API reference pages
+- **`resource://mapbox-guides`** now fetches `docs.mapbox.com/help/llms.txt` (39KB) — the full Mapbox Help Center index with troubleshooting guides, how-to tutorials, and walkthroughs
+- **`resource://mapbox-sdk-docs`** now fetches `docs.mapbox.com/mapbox-gl-js/llms.txt` (34KB) — the GL JS documentation index listing all guides, API reference pages, and examples for the primary web mapping SDK
+- **`resource://mapbox-reference`** now fetches the root `llms.txt` without filtering and returns the complete product catalog — useful for discovering what documentation exists and finding `llms.txt` URLs for any product
+- **`resource://mapbox-examples`** continues to extract playground/demo/example sections from the root index (API Playgrounds, Demos & Projects)
+
+**`docFetcher.fetchCachedText`** — new shared helper that fetches a URL and stores it in `docCache`, used by all five resources to avoid duplicating the fetch+cache pattern.
+
+**`docFetcher.toMarkdownUrl`** — no longer rewrites URLs already ending in `.txt`, `.md`, or `.json`. Previously `get_document_tool` would try to fetch `llms.txt.md` before falling back; now it fetches `llms.txt` directly on the first attempt.
+
 ## 0.2.1 - 2026-04-01
 
 ### Security

cspell.json

@@ -21,6 +21,8 @@
     "Tilesets",
     "tilestats",
     "Tilequery",
+    "isochrone",
+    "Isochrone",
     "aerialway",
     "aeroway",
     "housenum",

src/resources/mapbox-api-reference-resource/MapboxApiReferenceResource.ts

@@ -8,22 +8,24 @@ import type {
   ServerRequest
 } from '@modelcontextprotocol/sdk/types.js';
 import type { HttpRequest } from '../../utils/types.js';
-import { docCache } from '../../utils/docCache.js';
 import { BaseResource } from '../BaseResource.js';
-import {
-  parseDocSections,
-  filterSectionsByCategory,
-  sectionsToMarkdown
-} from '../utils/docParser.js';
+import { fetchCachedText } from '../../utils/docFetcher.js';
 
 /**
- * Resource providing Mapbox API reference documentation
+ * Resource providing Mapbox API reference documentation.
+ * Fetches the aggregated API reference index from docs.mapbox.com/api/llms.txt,
+ * which lists all Mapbox REST API endpoints grouped by service category
+ * (Maps, Navigation, Search, Accounts, etc.) with direct links to each
+ * API reference page.
  */
 export class MapboxApiReferenceResource extends BaseResource {
   readonly name = 'Mapbox API Reference';
   readonly uri = 'resource://mapbox-api-reference';
   readonly description =
-    'Mapbox REST API reference documentation including endpoints, parameters, rate limits, and authentication for all Mapbox APIs (Geocoding, Directions, Static Images, Tilequery, etc.)';
+    'Mapbox REST API reference index organized by service (Maps, Navigation, Search, Accounts). ' +
+    'Lists all API endpoints with links to detailed reference pages covering parameters, ' +
+    'rate limits, authentication, and response formats (Geocoding, Directions, Static Images, ' +
+    'Tilequery, Matrix, isochrone, Optimization, Styles, Uploads, Datasets, and more).';
   readonly mimeType = 'text/markdown';
 
   private httpRequest: HttpRequest;
@@ -38,36 +40,17 @@ export class MapboxApiReferenceResource extends BaseResource {
     _extra: RequestHandlerExtra<ServerRequest, ServerNotification>
   ): Promise<ReadResourceResult> {
     try {
-      const LLMS_TXT_URL = 'https://docs.mapbox.com/llms.txt';
-      let content = docCache.get(LLMS_TXT_URL);
-      if (!content) {
-        const response = await this.httpRequest(LLMS_TXT_URL, {
-          headers: {
-            Accept: 'text/markdown, text/plain;q=0.9, */*;q=0.8'
-          }
-        });
-
-        if (!response.ok) {
-          throw new Error(
-            `Failed to fetch Mapbox documentation: ${response.statusText}`
-          );
-        }
-
-        content = await response.text();
-        docCache.set(LLMS_TXT_URL, content);
-      }
-
-      // Parse and filter for API sections only
-      const allSections = parseDocSections(content);
-      const apiSections = filterSectionsByCategory(allSections, 'apis');
-      const apiContent = sectionsToMarkdown(apiSections);
+      const content = await fetchCachedText(
+        'https://docs.mapbox.com/api/llms.txt',
+        this.httpRequest
+      );
 
       return {
         contents: [
           {
             uri: uri.href,
             mimeType: this.mimeType,
-            text: `# Mapbox API Reference\n\n${apiContent}`
+            text: content
           }
         ]
       };

src/resources/mapbox-examples-resource/MapboxExamplesResource.ts

@@ -8,22 +8,26 @@ import type {
   ServerRequest
 } from '@modelcontextprotocol/sdk/types.js';
 import type { HttpRequest } from '../../utils/types.js';
-import { docCache } from '../../utils/docCache.js';
 import { BaseResource } from '../BaseResource.js';
+import { fetchCachedText } from '../../utils/docFetcher.js';
 import {
   parseDocSections,
   filterSectionsByCategory,
   sectionsToMarkdown
 } from '../utils/docParser.js';
 
 /**
- * Resource providing Mapbox examples and playgrounds
+ * Resource providing links to Mapbox interactive examples and playgrounds.
+ * Extracts the examples/playground/demo sections from the root llms.txt
+ * catalog, which lists API playgrounds, demo apps, and open-code projects.
  */
 export class MapboxExamplesResource extends BaseResource {
   readonly name = 'Mapbox Examples';
   readonly uri = 'resource://mapbox-examples';
   readonly description =
-    'Mapbox code examples, API playgrounds, and interactive demos for testing and learning';
+    'Mapbox interactive API playgrounds, demo applications, and code examples. ' +
+    'Includes playground URLs for Directions, Search Box, Static Images, ' +
+    'isochrone, Matrix APIs, and demo apps for real estate, store locator, etc.';
   readonly mimeType = 'text/markdown';
 
   private httpRequest: HttpRequest;
@@ -38,26 +42,12 @@ export class MapboxExamplesResource extends BaseResource {
     _extra: RequestHandlerExtra<ServerRequest, ServerNotification>
   ): Promise<ReadResourceResult> {
     try {
-      const LLMS_TXT_URL = 'https://docs.mapbox.com/llms.txt';
-      let content = docCache.get(LLMS_TXT_URL);
-      if (!content) {
-        const response = await this.httpRequest(LLMS_TXT_URL, {
-          headers: {
-            Accept: 'text/markdown, text/plain;q=0.9, */*;q=0.8'
-          }
-        });
-
-        if (!response.ok) {
-          throw new Error(
-            `Failed to fetch Mapbox documentation: ${response.statusText}`
-          );
-        }
-
-        content = await response.text();
-        docCache.set(LLMS_TXT_URL, content);
-      }
+      const content = await fetchCachedText(
+        'https://docs.mapbox.com/llms.txt',
+        this.httpRequest
+      );
 
-      // Parse and filter for example sections only
+      // Extract playground/demo/example sections from the catalog index
       const allSections = parseDocSections(content);
       const exampleSections = filterSectionsByCategory(allSections, 'examples');
       const exampleContent = sectionsToMarkdown(exampleSections);

src/resources/mapbox-guides-resource/MapboxGuidesResource.ts

@@ -8,22 +8,22 @@ import type {
   ServerRequest
 } from '@modelcontextprotocol/sdk/types.js';
 import type { HttpRequest } from '../../utils/types.js';
-import { docCache } from '../../utils/docCache.js';
 import { BaseResource } from '../BaseResource.js';
-import {
-  parseDocSections,
-  filterSectionsByCategory,
-  sectionsToMarkdown
-} from '../utils/docParser.js';
+import { fetchCachedText } from '../../utils/docFetcher.js';
 
 /**
- * Resource providing Mapbox guides and tutorials
+ * Resource providing Mapbox guides, tutorials, and help articles.
+ * Fetches from docs.mapbox.com/help/llms.txt which contains the full
+ * Mapbox Help Center index: troubleshooting guides, how-to tutorials,
+ * glossary, account setup, billing, and platform-specific walkthroughs.
  */
 export class MapboxGuidesResource extends BaseResource {
   readonly name = 'Mapbox Guides';
   readonly uri = 'resource://mapbox-guides';
   readonly description =
-    'Mapbox guides, tutorials, and how-tos including Studio Manual, map design guides, and best practices';
+    'Mapbox Help Center documentation: troubleshooting guides, how-to tutorials, ' +
+    'glossary, account and billing setup, and walkthroughs for common developer tasks. ' +
+    'Use this for conceptual guidance and step-by-step instructions.';
   readonly mimeType = 'text/markdown';
 
   private httpRequest: HttpRequest;
@@ -38,36 +38,17 @@ export class MapboxGuidesResource extends BaseResource {
     _extra: RequestHandlerExtra<ServerRequest, ServerNotification>
   ): Promise<ReadResourceResult> {
     try {
-      const LLMS_TXT_URL = 'https://docs.mapbox.com/llms.txt';
-      let content = docCache.get(LLMS_TXT_URL);
-      if (!content) {
-        const response = await this.httpRequest(LLMS_TXT_URL, {
-          headers: {
-            Accept: 'text/markdown, text/plain;q=0.9, */*;q=0.8'
-          }
-        });
-
-        if (!response.ok) {
-          throw new Error(
-            `Failed to fetch Mapbox documentation: ${response.statusText}`
-          );
-        }
-
-        content = await response.text();
-        docCache.set(LLMS_TXT_URL, content);
-      }
-
-      // Parse and filter for guide sections only
-      const allSections = parseDocSections(content);
-      const guideSections = filterSectionsByCategory(allSections, 'guides');
-      const guideContent = sectionsToMarkdown(guideSections);
+      const content = await fetchCachedText(
+        'https://docs.mapbox.com/help/llms.txt',
+        this.httpRequest
+      );
 
       return {
         contents: [
           {
             uri: uri.href,
             mimeType: this.mimeType,
-            text: `# Mapbox Guides\n\n${guideContent}`
+            text: content
           }
         ]
       };

src/resources/mapbox-reference-resource/MapboxReferenceResource.ts

@@ -8,22 +8,25 @@ import type {
   ServerRequest
 } from '@modelcontextprotocol/sdk/types.js';
 import type { HttpRequest } from '../../utils/types.js';
-import { docCache } from '../../utils/docCache.js';
 import { BaseResource } from '../BaseResource.js';
-import {
-  parseDocSections,
-  filterSectionsByCategory,
-  sectionsToMarkdown
-} from '../utils/docParser.js';
+import { fetchCachedText } from '../../utils/docFetcher.js';
 
 /**
- * Resource providing Mapbox reference documentation
+ * Resource providing the complete Mapbox documentation catalog.
+ * Fetches the root llms.txt which is now a structured index of every
+ * Mapbox product and documentation section, each with a link to its
+ * own llms.txt file. Use this to discover what documentation is available
+ * and find URLs to pass to get_document_tool for deeper exploration.
  */
 export class MapboxReferenceResource extends BaseResource {
   readonly name = 'Mapbox Reference';
   readonly uri = 'resource://mapbox-reference';
   readonly description =
-    'Mapbox reference documentation including tilesets, data products, accounts, pricing, and other reference materials';
+    'Complete catalog of all Mapbox products and documentation. ' +
+    'Lists every product section (Maps SDKs, Navigation APIs, Search APIs, ' +
+    'Studio, Style Spec, Tilesets, Data products, Help Center, Atlas, Unity, etc.) ' +
+    "with links to each product's own llms.txt index. Use this to discover " +
+    'what documentation exists and find the right URLs to fetch full docs.';
   readonly mimeType = 'text/markdown';
 
   private httpRequest: HttpRequest;
@@ -38,39 +41,17 @@ export class MapboxReferenceResource extends BaseResource {
     _extra: RequestHandlerExtra<ServerRequest, ServerNotification>
   ): Promise<ReadResourceResult> {
     try {
-      const LLMS_TXT_URL = 'https://docs.mapbox.com/llms.txt';
-      let content = docCache.get(LLMS_TXT_URL);
-      if (!content) {
-        const response = await this.httpRequest(LLMS_TXT_URL, {
-          headers: {
-            Accept: 'text/markdown, text/plain;q=0.9, */*;q=0.8'
-          }
-        });
-
-        if (!response.ok) {
-          throw new Error(
-            `Failed to fetch Mapbox documentation: ${response.statusText}`
-          );
-        }
-
-        content = await response.text();
-        docCache.set(LLMS_TXT_URL, content);
-      }
-
-      // Parse and filter for reference sections only
-      const allSections = parseDocSections(content);
-      const referenceSections = filterSectionsByCategory(
-        allSections,
-        'reference'
+      const content = await fetchCachedText(
+        'https://docs.mapbox.com/llms.txt',
+        this.httpRequest
       );
-      const referenceContent = sectionsToMarkdown(referenceSections);
 
       return {
         contents: [
           {
             uri: uri.href,
             mimeType: this.mimeType,
-            text: `# Mapbox Reference\n\n${referenceContent}`
+            text: content
           }
         ]
       };