Skip to content

reject control characters in the request target - #13212

Draft
arshsmith1 wants to merge 3 commits into
aio-libs:masterfrom
arshsmith1:request-target-ctl-reject
Draft

reject control characters in the request target#13212
arshsmith1 wants to merge 3 commits into
aio-libs:masterfrom
arshsmith1:request-target-ctl-reject

Conversation

@arshsmith1

Copy link
Copy Markdown
Contributor

What do these changes do?

HttpRequestParser.parse_message validates the method with TOKENRE and the version with VERSRE, but hands the request target straight to URL.build() / URL() without looking at it. RFC 9112 §3.2 forbids control characters there, and llhttp's URL_CHAR table has none of them, so the Cython parser already rejects \x00, HTAB, bare LF, bare CR, \x1f and DEL in a target. The pure-Python fallback accepted all of them, in origin-form, absolute-form and authority-form alike, and they landed in request.path / request.raw_path / match_info.

That means the same bytes on the wire get opposite accept/reject decisions depending on whether the compiled extension is present (AIOHTTP_NO_EXTENSIONS, PyPy, source builds), which is the divergence row 1.12 of THREAT_MODEL.md is about. A bare CR or LF in the start line is also the usual desync primitive against an intermediary that treats it as a line terminator.

Verified both ways with the C extension built locally:

bare LF in target      [py] ACCEPTED path='/a\nb'      [c] rejected BadHttpMessage
bare CR in target      [py] ACCEPTED path='/a\rb'      [c] rejected InvalidURLError
NUL in target          [py] ACCEPTED path='/a\x00b'    [c] rejected InvalidURLError
DEL in target          [py] ACCEPTED path='/a\x7fb'    [c] rejected InvalidURLError
TAB in target          [py] ACCEPTED path='/a\tb'      [c] rejected InvalidURLError

After the patch both backends reject all of them. The new test is parameterised over REQUEST_PARSERS, so it covers the pure-Python and Cython parsers; it fails on 9 of 18 cases (the pure-Python half) without the fix.

Non-ASCII bytes in the target are a separate divergence that the pure-Python parser tolerates deliberately (test_http_request_parser_utf8_request_line xfails the C parser), so I left that alone.

Are there changes in behavior for the user?

A request whose target contains a control character now gets a 400 instead of being routed. That was already the behaviour on wheel installs; this only brings the pure-Python parser in line. Valid targets are untouched.

Is it a substantial burden for the maintainers to support this?

No, it is one regex and one guard next to the existing method/version checks, plus a parameterised regression test alongside the other bad-URI cases.

Related issue number

None.

Checklist

  • I think the code is well written
  • Unit tests for the changes exist
  • Documentation reflects the changes - N/A, no public API change; THREAT_MODEL.md row 1.12 updated per AGENTS.md
  • If you provide code modification, please add yourself to CONTRIBUTORS.txt - already listed
  • Add a new news fragment into the CHANGES/ folder

@psf-chronographer psf-chronographer Bot added the bot:chronographer:provided There is a change note present in this PR label Jul 22, 2026
@codspeed-hq

codspeed-hq Bot commented Jul 22, 2026

Copy link
Copy Markdown

Merging this PR will degrade performance by 9.13%

⚠️ Different runtime environments detected

Some benchmarks with significant performance changes were compared across different runtime environments,
which may affect the accuracy of the results.

Open the report in CodSpeed to investigate

❌ 1 regressed benchmark
✅ 83 untouched benchmarks
⏩ 83 skipped benchmarks1

Warning

Please fix the performance issues or acknowledge them on CodSpeed.

Performance Changes

Benchmark BASE HEAD Efficiency
test_resolve_gitapi 596.6 ms 656.5 ms -9.13%

Tip

Investigate this regression by commenting @codspeedbot fix this regression on this PR, or directly use the CodSpeed MCP with your agent.


Comparing arshsmith1:request-target-ctl-reject (2474ffc) with master (d8b943b)

Open in CodSpeed

Footnotes

  1. 83 benchmarks were skipped, so the baseline results were used instead. If they were deleted from the codebase, click here and archive them to remove them from the performance reports.

@codecov

codecov Bot commented Jul 22, 2026

Copy link
Copy Markdown

Codecov Report

✅ All modified and coverable lines are covered by tests.
✅ Project coverage is 98.99%. Comparing base (8e814f5) to head (2474ffc).
⚠️ Report is 82 commits behind head on master.
✅ All tests successful. No failed tests found.

Additional details and impacted files
@@            Coverage Diff             @@
##           master   #13212      +/-   ##
==========================================
+ Coverage   98.98%   98.99%   +0.01%     
==========================================
  Files         131      132       +1     
  Lines       48955    49297     +342     
  Branches     2550     2567      +17     
==========================================
+ Hits        48458    48802     +344     
+ Misses        373      371       -2     
  Partials      124      124              
Flag Coverage Δ
Autobahn 22.07% <21.42%> (-0.05%) ⬇️
CI-GHA 98.90% <100.00%> (+<0.01%) ⬆️
OS-Linux 98.68% <100.00%> (+<0.01%) ⬆️
OS-Windows 97.03% <100.00%> (+0.01%) ⬆️
OS-macOS 97.93% <100.00%> (+<0.01%) ⬆️
Py-3.10 98.12% <100.00%> (+0.01%) ⬆️
Py-3.11 98.38% <100.00%> (+<0.01%) ⬆️
Py-3.12 98.47% <100.00%> (+<0.01%) ⬆️
Py-3.13 98.45% <100.00%> (+<0.01%) ⬆️
Py-3.14 98.47% <100.00%> (+<0.01%) ⬆️
Py-3.14t 97.56% <100.00%> (+<0.01%) ⬆️
Py-pypy-3.11 97.40% <85.71%> (-0.03%) ⬇️
VM-macos 97.93% <100.00%> (+<0.01%) ⬆️
VM-ubuntu 98.68% <100.00%> (+<0.01%) ⬆️
VM-windows 97.03% <100.00%> (+0.01%) ⬆️
cython-coverage 38.14% <100.00%> (+0.24%) ⬆️

Flags with carried forward coverage won't be shown. Click here to find out more.

☔ View full report in Codecov by Harness.
📢 Have feedback on the report? Share it here.

Comment thread aiohttp/http_parser.py Outdated
_FIELD_VALUE_FORBIDDEN_CTL_RE: Final[Pattern[str]] = re.compile(
r"[\x00-\x08\x0a-\x1f\x7f]"
)
# https://www.rfc-editor.org/rfc/rfc9112#section-3.2-3

Copy link
Copy Markdown
Member

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

No whitespace is allowed in the request-target. Unfortunately, some user agents fail to properly encode or exclude whitespace found in hypertext references, resulting in those disallowed characters being sent as the request-target in a malformed request-line.

How is this related to the control characters regex?

Copy link
Copy Markdown
Contributor Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Good catch, that anchor is the wrong paragraph. 3.2-3 is about whitespace specifically; the real basis is that a control char makes the target unmatchable by the request-target ABNF, so it's an invalid request-line, and 3.2-4 is the SHOULD-400 rule for that. I've repointed the comment (and the test) at #section-3.2-4 and reworded it to say why.

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

bot:chronographer:provided There is a change note present in this PR

Projects

None yet

Development

Successfully merging this pull request may close these issues.

2 participants