How to use DefaultMarkdownGenerator with requests only?

[barvhaim]

Hi, what's the payload parameters for using the DefaultMarkdownGenerator without installing the whole library? I use the REST API directly against an instance running on docker

[hafezparast]

The /md endpoint is exactly what you need — it uses DefaultMarkdownGenerator internally and returns markdown directly.

Basic request:

curl -X POST http://your-docker-host:11235/md \
  -H "Content-Type: application/json" \
  -H "Authorization: Bearer YOUR_API_TOKEN" \
  -d '{
    "url": "https://example.com",
    "f": "fit"
  }'

Filter modes available:

• "fit" (default) — readability-based extraction, gives you clean content (uses PruningContentFilter inside DefaultMarkdownGenerator)
• "raw" — direct DOM-to-markdown conversion, no filtering
• "bm25" — BM25 relevance ranking (pass a query with "q": "your search terms")
• "llm" — LLM-based filtering (requires LLM provider config)

Response:

{
  "url": "https://example.com",
  "filter": "fit",
  "query": null,
  "cache": "0",
  "markdown": "# Page Title\n\nContent here...",
  "success": true
}

If you need more control (CSS selectors, excluded tags, etc.), use the /crawl endpoint with crawler_config — the markdown output is included in the response by default, no extra config needed.