how to filter network_requests when set capture_network_requests=True

[wangning182]

page has a lot of requests, i wan't ge all of them

[hafezparast]

capture_network_requests=True captures everything. They're in result.network_requests as a list of dicts.

To reduce noise at the source, use BrowserConfig to block resources you don't need — this prevents them from loading entirely:

browser_config = BrowserConfig(
    text_mode=True,    # disables images and rich content
    avoid_ads=True,    # blocks ad/tracker requests
    avoid_css=True,    # blocks CSS files
)

To filter captured requests after the crawl:

config = CrawlerRunConfig(capture_network_requests=True)
result = await crawler.arun(url, config=config)

all_requests = result.network_requests or []

# Filter by type
api_calls = [r for r in all_requests if r.get("event_type") == "response" 
             and "/api/" in r.get("url", "")]
xhr_only = [r for r in all_requests if r.get("resource_type") in ("xhr", "fetch")]
failed = [r for r in all_requests if r.get("event_type") == "requestfailed"]

Available resource_type values (from Playwright): document, stylesheet, image, media, font, script, xhr, fetch, websocket, other.

There's no built-in pre-filter on capture (like capture_network_types=["xhr", "fetch"]) — it's all-or-nothing. But between BrowserConfig blocking and post-crawl filtering, you can get what you need.