Skip to content

gh-150942: Speed up pyexpat attribute parsing - #154860

Open
Punisheroot wants to merge 1 commit into
python:mainfrom
Punisheroot:perf/js/optimize-pyexpat-attributes
Open

gh-150942: Speed up pyexpat attribute parsing#154860
Punisheroot wants to merge 1 commit into
python:mainfrom
Punisheroot:perf/js/optimize-pyexpat-attributes

Conversation

@Punisheroot

Copy link
Copy Markdown

Summary

my_StartElementHandler() builds a new dictionary for unordered element
attributes. Use _PyDict_SetItem_Take2() there so the dictionary insertion
consumes the newly created key and value directly, removing the matching
Py_DECREF() calls on both the success and error paths.

This keeps the existing behavior and error handling while doing less reference
count work.

Benchmark

Benchmarks used pyperf 2.10.0 on CPython main 3.16.0a0 at revision
8b048eb35e, built for Windows x64 with the official PCbuild PGO/PGUpdate
configuration and link-time code generation. Baseline and patched runs used the
same interpreter and PGO profile; only the pyexpat extension binary was
swapped. Each worker used CPU 1 affinity and high process priority.

The workloads parse complete XML documents with:

  • 2,000 elements and 16 attributes per element
  • 20,000 elements and 2 attributes per element
  • 200 elements and 64 attributes per element

Regular build:

Benchmark Baseline Patched Change
many attributes per element 3.42 ms 3.35 ms 1.02x faster
few attributes, many elements 6.22 ms 6.02 ms 1.03x faster
wide attribute dictionaries 1.33 ms 1.31 ms 1.02x faster
geometric mean reference 1.02x faster

Free-threaded build (Py_GIL_DISABLED=1, sys._is_gil_enabled() == False):

Benchmark Baseline Patched Change
many attributes per element 3.59 ms 3.49 ms 1.03x faster
few attributes, many elements 6.68 ms 6.48 ms 1.03x faster
wide attribute dictionaries 1.42 ms 1.39 ms 1.03x faster
geometric mean reference 1.03x faster

pyperf check reported both free-threaded result files as stable. The benchmark
is single-threaded, including on the free-threaded interpreter, so it measures
the direct parsing cost rather than concurrent scalability.

Validation

  • PCbuild\amd64\python.exe -m test test_pyexpat: passed (86 tests, 1 skipped)
  • PCbuild\amd64t\python.exe -m test test_pyexpat: passed (86 tests, 1 skipped)
  • PCbuild\amd64\python_d.exe -m test -R 3:3 test_pyexpat: passed
  • python.bat -m patchcheck: passed
  • git diff --check: passed
  • regular and free-threaded PGO training: passed (10,501 tests in each build)

@bedevere-app

bedevere-app Bot commented Jul 29, 2026

Copy link
Copy Markdown

Most changes to Python require a NEWS entry. Add one using the blurb_it web app or the blurb command-line tool.

If this change has little impact on Python users, wait for a maintainer to apply the skip news label instead.

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Projects

None yet

Development

Successfully merging this pull request may close these issues.

1 participant