-
Notifications
You must be signed in to change notification settings - Fork 906
New issue
Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.
By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.
Already on GitHub? Sign in to your account
[BUG] asyncio.exceptions.InvalidStateError: invalid state thrown by exit in async context manager #2238
Comments
I was able to repro in 1 out of 5 runs. However, I was not able to repro with the following snippet. Not yet sure what's going on. from playwright.async_api import async_playwright
import asyncio
async def doit(url):
print(f"Processing {url}")
async with async_playwright() as p:
browser_type = p.chromium
browser = await browser_type.launch(
headless=True,
)
try:
page = await browser.new_page(
bypass_csp=True,
ignore_https_errors=True,
)
res = await page.goto(url, wait_until="load", timeout=30 * 1000)
await page.wait_for_load_state(state="networkidle")
except Exception as e:
print(f"Got exception {e}")
raise e
finally:
await browser.close()
asyncio.run(doit("https://www.streetinsider.com/Press+Releases/Radius+Recycling+Reports+First+Quarter+Fiscal+2024+Financial+Results/22593061.html")) |
It appears that the |
I'm unfortunately not able to reproduce it. I tried to repro running 10 times on macOS with Python 3.10 and Python 3.12. |
Closing for now since we can't reproduce it. |
I don't think this should be closed. I can reproduce the error. Whenever there is a timeout error it appears that the event loop is closing, resulting in an Invalid state.
|
I am facing a similar problem with my scraper as well. The entire code base is really large so I can't post it here. Where this error happens isn't exactly consistent, but it seems to occur somewhere after about 3 hours of scraping, at around 350 links. It only throws the error when I stop the python programme, and does not stop the python file automatically like an error. Some measures taken to workaround:
Edit: Happens on Python 3.10 on MacOS and Python 3.11 on Windows. |
Another stacktrace:
With async with (
async_playwright() as p,
create_task_group() as tg
):
browser = await p.chromium.launch()
list_spider = await SpiderAPI[ListingLink, ListPageLink].create(browser)
tg.start_soon(list_spider.run, spider_list(config)) # curried
await sleep(5)
tg.cancel_scope.cancel() |
also facing this issue:
Process SpawnProcess-3: During handling of the above exception, another exception occurred: Traceback (most recent call last): |
I have just randomly encountered a very similar bug just in almost bare asyncio with Python 3.11 without playwright or any other significant library. With that, this may very easily be an asyncio bug itself. Gonna check more and return back as soon as i find more. |
Any update I have been facing the same invalidate state error. |
The unresolved and closed issue #2612 may explain this bug. As mentioned,
And I suspect this is related, as the underlying issue appears to be a I will look into this further. |
System info
Source code
Steps
Expected
It should complete without error.
Actual
The text was updated successfully, but these errors were encountered: