Skip to content

[Bug]: AsyncHTTPCrawlerStrategy passes page_timeout (ms) directly to aiohttp ClientTimeout (seconds) #1894

@coderJASFK

Description

@coderJASFK

crawl4ai version

0.8.0

Expected Behavior

page_timeout=5000 (5 seconds) in CrawlerRunConfig should cause AsyncHTTPCrawlerStrategy to timeout HTTP requests after approximately 5 seconds.

Current Behavior

AsyncHTTPCrawlerStrategy._handle_http passes page_timeout directly to aiohttp.ClientTimeout(total=page_timeout). But page_timeout is in milliseconds (as documented: "Timeout in ms for page operations") while aiohttp.ClientTimeout.total expects seconds. So page_timeout=5000 sets a 5000-second timeout (1.4 hours) instead of 5 seconds. The default page_timeout=60000 becomes 60000 seconds (16.7 hours).

Is this reproducible?

Yes

Inputs Causing the Bug

Steps to Reproduce

Code snippets

OS

Windows

Python version

3.12

Browser

No response

Browser version

No response

Error logs & Screenshots (if applicable)

No response

Metadata

Metadata

Assignees

No one assigned

    Labels

    ⚙️ In-progressIssues, Features requests that are in Progress🐞 BugSomething isn't working

    Projects

    No projects

    Milestone

    No milestone

    Relationships

    None yet

    Development

    No branches or pull requests

    Issue actions