reject control characters in the request target - #13212
Conversation
Merging this PR will degrade performance by 9.13%
|
| Benchmark | BASE |
HEAD |
Efficiency | |
|---|---|---|---|---|
| ❌ | test_resolve_gitapi |
596.6 ms | 656.5 ms | -9.13% |
Tip
Investigate this regression by commenting @codspeedbot fix this regression on this PR, or directly use the CodSpeed MCP with your agent.
Comparing arshsmith1:request-target-ctl-reject (2474ffc) with master (d8b943b)
Footnotes
-
83 benchmarks were skipped, so the baseline results were used instead. If they were deleted from the codebase, click here and archive them to remove them from the performance reports. ↩
Codecov Report✅ All modified and coverable lines are covered by tests. Additional details and impacted files@@ Coverage Diff @@
## master #13212 +/- ##
==========================================
+ Coverage 98.98% 98.99% +0.01%
==========================================
Files 131 132 +1
Lines 48955 49297 +342
Branches 2550 2567 +17
==========================================
+ Hits 48458 48802 +344
+ Misses 373 371 -2
Partials 124 124
Flags with carried forward coverage won't be shown. Click here to find out more. ☔ View full report in Codecov by Harness. |
| _FIELD_VALUE_FORBIDDEN_CTL_RE: Final[Pattern[str]] = re.compile( | ||
| r"[\x00-\x08\x0a-\x1f\x7f]" | ||
| ) | ||
| # https://www.rfc-editor.org/rfc/rfc9112#section-3.2-3 |
There was a problem hiding this comment.
No whitespace is allowed in the request-target. Unfortunately, some user agents fail to properly encode or exclude whitespace found in hypertext references, resulting in those disallowed characters being sent as the request-target in a malformed request-line.
How is this related to the control characters regex?
There was a problem hiding this comment.
Good catch, that anchor is the wrong paragraph. 3.2-3 is about whitespace specifically; the real basis is that a control char makes the target unmatchable by the request-target ABNF, so it's an invalid request-line, and 3.2-4 is the SHOULD-400 rule for that. I've repointed the comment (and the test) at #section-3.2-4 and reworded it to say why.
What do these changes do?
HttpRequestParser.parse_messagevalidates the method withTOKENREand the version withVERSRE, but hands the request target straight toURL.build()/URL()without looking at it. RFC 9112 §3.2 forbids control characters there, and llhttp'sURL_CHARtable has none of them, so the Cython parser already rejects\x00, HTAB, bare LF, bare CR,\x1fand DEL in a target. The pure-Python fallback accepted all of them, in origin-form, absolute-form and authority-form alike, and they landed inrequest.path/request.raw_path/match_info.That means the same bytes on the wire get opposite accept/reject decisions depending on whether the compiled extension is present (
AIOHTTP_NO_EXTENSIONS, PyPy, source builds), which is the divergence row 1.12 ofTHREAT_MODEL.mdis about. A bare CR or LF in the start line is also the usual desync primitive against an intermediary that treats it as a line terminator.Verified both ways with the C extension built locally:
After the patch both backends reject all of them. The new test is parameterised over
REQUEST_PARSERS, so it covers the pure-Python and Cython parsers; it fails on 9 of 18 cases (the pure-Python half) without the fix.Non-ASCII bytes in the target are a separate divergence that the pure-Python parser tolerates deliberately (
test_http_request_parser_utf8_request_linexfails the C parser), so I left that alone.Are there changes in behavior for the user?
A request whose target contains a control character now gets a 400 instead of being routed. That was already the behaviour on wheel installs; this only brings the pure-Python parser in line. Valid targets are untouched.
Is it a substantial burden for the maintainers to support this?
No, it is one regex and one guard next to the existing method/version checks, plus a parameterised regression test alongside the other bad-URI cases.
Related issue number
None.
Checklist
THREAT_MODEL.mdrow 1.12 updated perAGENTS.mdCONTRIBUTORS.txt- already listedCHANGES/folder