fix: strip misleading Content-Encoding header from mocked responses - #812
fix: strip misleading Content-Encoding header from mocked responses#812Sohel2309 wants to merge 2 commits into
Conversation
responses always serves `body` as literal, uncompressed bytes -- it never actually applies the encoding named in a Content-Encoding header. When such a header was present on a registered response (e.g. copied verbatim from a real recorded response into a hand-edited/legacy fixture file, or passed directly via headers=), requests/urllib3 would try to decompress the already-uncompressed body and raise ContentDecodingError/DecodeError instead of returning the mocked response. Strip the header in BaseResponse.get_headers(), the single choke point used by both Response and CallbackResponse, so every path that constructs a mocked response (add(), add_callback(), and _add_from_file()) is protected, not just freshly recorded and re-dumped fixture files. Adds two regression tests: one for the direct responses.add() usage and one for the _add_from_file() replay path used by the recorder documentation, both of which reproduce the exact failure from the reported issue before this fix and pass after it. Fixes getsentry#724
| responses.add( | ||
| responses.GET, | ||
| "https://example.org/", | ||
| body='{"first_name": true}', |
There was a problem hiding this comment.
Couldn't the response mock be gzip encoded though?
There was a problem hiding this comment.
Yes, you're right. I initially handled this too broadly. Genuine gzip-encoded mocks are valid and already work correctly, so stripping Content-Encoding when serving the response would break that use case. I've reverted that part and added a regression test to ensure real gzip-encoded mocks continue to be decoded correctly.
| body: '{"first_name": true}' | ||
| content_type: application/json | ||
| headers: | ||
| content-encoding: gzip |
There was a problem hiding this comment.
Why not correct the fixture file?
There was a problem hiding this comment.
You're right. The fixture itself is inconsistent. I also found that the recorder was creating the same inconsistency in-memory by storing requests' already-decompressed body together with the original Content-Encoding header. The fix now removes the header at recording time, while leaving response serving and valid gzip mocks unchanged.
2 participants
Summary
Mocked responses can preserve a
Content-Encodingheader even when the response body is not actually encoded.This causes requests/urllib3 to attempt decoding the body and fail.
Fix
Strip
Content-Encodingfrom mocked response headers before serving the response.This covers both direct
responses.add()usage and responses loaded from fixtures.Tests
Fixes #724