Can the “Invalid header token” error be overcome?

[plmilord]

I'm currently developing a custom component for Home Assistant and overcoming an annoying error using aiohttp.

Here is my code:

    async def get_data(self):
        url = f"{self.base_url}data"
        while True:
            async with aiohttp.ClientSession() as session:
                try:
                    async with session.get(url) as response:
                        result = await response.text()
                        if response.status == 200:
                            self._data = result
                            self._online = True
                        else:
                            _LOGGER.error("Error, unable to get data")
                except aiohttp.ClientError as e:
                    self._online = False
            await asyncio.sleep(5)

...and here is the error I get:

aiohttp.http_exceptions.BadHttpMessage: 400, message:
  Invalid header token:

    b'HTTP/1.1 200 OK\r'
          ^

The above exception was the direct cause of the following exception:

Traceback (most recent call last):
  File "/config/custom_components/ikamand/ikamand.py", line 93, in get_data
    async with session.get(url) as response:
  File "/usr/local/lib/python3.11/site-packages/aiohttp/client.py", line 1141, in __aenter__
    self._resp = await self._coro
                 ^^^^^^^^^^^^^^^^
  File "/usr/local/lib/python3.11/site-packages/aiohttp/client.py", line 560, in _request
    await resp.start(conn)
  File "/usr/local/lib/python3.11/site-packages/aiohttp/client_reqrep.py", line 901, in start
    raise ClientResponseError(
aiohttp.client_exceptions.ClientResponseError: 400, message="Invalid header token:\n\n  b'HTTP/1.1 200 OK\\r'\n        ^", url=URL('http://192.168.2.41/cgi-bin/data')

Do you have any ideas on how I can overcome this problem with aiohttp? Need to report this as a bug?

I'm using the latest aiohttp version included in Home Assistant Core 2023.12.0.dev20231026.

Thanks,

Here is the raw printout of the response headers while using the DevTools in Chrome:

HTTP/1.0 200 OK
HTTP/1.1 200 OK

[Dreamsorcerer]

I think home assistant reverted their version of aiohttp after finding a regression. I'd retest with aiohttp 3.8.6 or 3.9.0 and confirm if there's still an issue then.

[Dreamsorcerer]

Actually, if that's the response, it's literally 2 HTTP status lines, which would be clearly wrong.
So, that probably won't be accepted even after upgrading. Can't you fix whatever is sending the invalid HTTP message?

[plmilord]

Thank-you @Dreamsorcerer for your follow-up.

Even with aiohttp 3.8.6 or 3.9.0, I still get this error:

aiohttp.http_exceptions.BadHttpMessage: 400, message:
  Invalid header token:

    b'HTTP/1.1 200 OK'
          ^

The above exception was the direct cause of the following exception:

Traceback (most recent call last):
  File "c:/Users/plmil/Desktop/test/test.py", line 42, in <module>
    asyncio.run(ikamand_instance.fetch_data())
  File "C:\Users\plmil\AppData\Local\Programs\Python\Python38\lib\asyncio\runners.py", line 43, in run
    return loop.run_until_complete(main)
  File "C:\Users\plmil\AppData\Local\Programs\Python\Python38\lib\asyncio\base_events.py", line 616, in run_until_complete
    return future.result()
  File "c:/Users/plmil/Desktop/test/test.py", line 23, in fetch_data
    async with session.get(url, headers=self.headers) as response:
  File "C:\Users\plmil\AppData\Local\Programs\Python\Python38\lib\site-packages\aiohttp\client.py", line 1186, in __aenter__
    self._resp = await self._coro
  File "C:\Users\plmil\AppData\Local\Programs\Python\Python38\lib\site-packages\aiohttp\client.py", line 600, in _request
    await resp.start(conn)
  File "C:\Users\plmil\AppData\Local\Programs\Python\Python38\lib\site-packages\aiohttp\client_reqrep.py", line 925, in start
    raise ClientResponseError(
aiohttp.client_exceptions.ClientResponseError: 400, message="Invalid header token:\n\n  b'HTTP/1.1 200 OK'\n        ^", url=URL('http://192.168.2.41/cgi-bin/data')

The page viewed still works very well in browsers... There is a redirection to an image which seems wrong, but I can visually consult the content of what is published in clear text.

As I'm trying to parse this message from a third party module, I don't have access to modify anything on this server...

Using the requests module works, but its asynchronous management is rather hellish!

Any ideas for continuing with aiohttp? Report as a bug?

Thanks,

[Dreamsorcerer]

You can report an issue at https://github.com/nodejs/llhttp/ asking for a lenient option that allows for multiple status lines. If they accept it, then we'll just need to enable the new lenient option in aiohttp. If they reject it, then we can't really do anything about it, it is a badly malformed HTTP request (I mean, are we supposed to parse it as HTTP 1.0, or 1.1?, it's not valid as either anyway...). Feel free to @ me, so I can track the issue.

[Dreamsorcerer]

It would be best to get the full raw message first (maybe using curl -I --raw).