Future exception was never retrieved

[anomam]

Hi friends,

We've been using aiohttp to send multipart data to some workloads (we own) with decently high concurrency, but nothing crazy.
In one of our clusters we started seeing the following error:

Future exception was never retrieved
future: <Future finished exception=ClientOSError(104, 'Connection reset by peer') ... >

After enabling the debug mode for our async loop in the Python app inside that cluster, we were able to find where the future was created; see following screenshot:

We have tried very hard to reproduce this error locally (artificially) to find ways to retrieve the future exception, catch it, and maybe retry our calls, but without success...

So we're hoping you could help us with the following:

1. do you have any idea on how to retrieve that future exception (catch it) in order to retry the http request?

2. do you have any idea on how to reproduce the error locally and artificially (with some dummy server/app)? We've tried very hard but the best we could do was get an asyncio.TimeoutError raised within aiohttp, which was catchable and different from the issue described above... this would hopefully help us optimize our workloads to minimize the issue.

Thank you for any help/suggestion 🙏

[Dreamsorcerer]

I think there are issues already open. I don't really remember much about it, but it's one of those tricky ones I don't think we've resolved yet.