Skip to content

fix quadratic backtracking in Forwarded and Link header parsing - #13285

Draft
arshsmith1 wants to merge 1 commit into
aio-libs:masterfrom
arshsmith1:header-list-quadratic-backtracking
Draft

fix quadratic backtracking in Forwarded and Link header parsing#13285
arshsmith1 wants to merge 1 commit into
aio-libs:masterfrom
arshsmith1:header-list-quadratic-backtracking

Conversation

@arshsmith1

Copy link
Copy Markdown
Contributor

What do these changes do?

HeadersDictProxy.getall splits a comma-separated header into elements with _LIST_ELEMENT_RE, whose unquoted-element group used a lazy quantifier ahead of a trailing [ \t]*(?:,|\Z). When an element carries a long run of whitespace before a non-space character (e.g. for=a followed by many spaces then b), the engine backtracks quadratically. That regex sits on two attacker-reachable paths: request.forwarded (a request header, server side) and response.links (a response header, client side), so a single ~8 KB header value burns hundreds of ms of CPU per access.

response.links also parsed each Link parameter with (.*?)(\2)\s*$, which backtracks the same way on an unquoted value that ends in trailing whitespace. Making the split-regex group greedy fixes the first case with no change in output (the captured group is stripped afterwards anyway), and the Link parameter is now captured greedily with the surrounding quotes and whitespace stripped in Python. Both go from O(n^2) to O(n).

Are there changes in behavior for the user?

No. Parsed results are identical for every input; only the worst-case timing changes. I ran a differential fuzz of the old and new getall regex over a few hundred thousand generated inputs with zero divergence, and the same for the Link parameter parsing.

Is it a substantial burden for the maintainers to support this?

No. It is a one-character change to the split regex plus a small rewrite of the Link parameter parse, each covered by a regression test that fails on the unpatched tree.

Related issue number

N/A

Checklist

  • I think the code is well written
  • Unit tests for the changes exist
  • Documentation reflects the changes — N/A, no public API change
  • If you provide code modification, please add yourself to CONTRIBUTORS.txt
  • Add a new news fragment into the CHANGES/ folder
Before/after timing (pure-Python parser)
request.forwarded, header = "for=a" + " "*N + "b"
  N= 8000   before: 306 ms    after: 0.7 ms
  N=16000   before: ~1.2 s    after: 1.6 ms

response.links, Link = "<http://example.com/>; rel=a" + " "*15000 + "b"
  before: ~1.5 s    after: 1.2 ms

@psf-chronographer psf-chronographer Bot added the bot:chronographer:provided There is a change note present in this PR label Jul 29, 2026
@codspeed-hq

codspeed-hq Bot commented Jul 29, 2026

Copy link
Copy Markdown

Merging this PR will not alter performance

✅ 84 untouched benchmarks
⏩ 83 skipped benchmarks1


Comparing arshsmith1:header-list-quadratic-backtracking (d9eee13) with master (c3f07fc)

Open in CodSpeed

Footnotes

  1. 83 benchmarks were skipped, so the baseline results were used instead. If they were deleted from the codebase, click here and archive them to remove them from the performance reports.

@codecov

codecov Bot commented Jul 29, 2026

Copy link
Copy Markdown

Codecov Report

✅ All modified and coverable lines are covered by tests.
✅ Project coverage is 98.98%. Comparing base (c3f07fc) to head (d9eee13).
✅ All tests successful. No failed tests found.

Additional details and impacted files
@@           Coverage Diff           @@
##           master   #13285   +/-   ##
=======================================
  Coverage   98.98%   98.98%           
=======================================
  Files         132      132           
  Lines       49023    49050   +27     
  Branches     2551     2552    +1     
=======================================
+ Hits        48526    48553   +27     
  Misses        373      373           
  Partials      124      124           
Flag Coverage Δ
Autobahn 22.12% <13.79%> (-0.01%) ⬇️
CI-GHA 98.90% <100.00%> (+<0.01%) ⬆️
OS-Linux 98.67% <100.00%> (+<0.01%) ⬆️
OS-Windows 97.03% <100.00%> (+<0.01%) ⬆️
OS-macOS 97.93% <100.00%> (+<0.01%) ⬆️
Py-3.10 98.12% <100.00%> (+<0.01%) ⬆️
Py-3.11 98.37% <100.00%> (+<0.01%) ⬆️
Py-3.12 98.46% <100.00%> (+<0.01%) ⬆️
Py-3.13 98.44% <100.00%> (-0.01%) ⬇️
Py-3.14 98.46% <100.00%> (-0.01%) ⬇️
Py-3.14t 97.56% <100.00%> (+<0.01%) ⬆️
Py-pypy-3.11 97.39% <100.00%> (+<0.01%) ⬆️
VM-macos 97.93% <100.00%> (+<0.01%) ⬆️
VM-ubuntu 98.67% <100.00%> (+<0.01%) ⬆️
VM-windows 97.03% <100.00%> (+<0.01%) ⬆️
cython-coverage 37.95% <0.00%> (-0.03%) ⬇️

Flags with carried forward coverage won't be shown. Click here to find out more.

☔ View full report in Codecov by Harness.
📢 Have feedback on the report? Share it here.

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

bot:chronographer:provided There is a change note present in this PR

Projects

None yet

Development

Successfully merging this pull request may close these issues.

1 participant