Repository navigation
Add an opt-in cache for deterministic production frontend builds - #7455
FarhanAliRaza wants to merge 19 commits into
Conversation
Two exports (or an export and a prod/preview start) in the same project used to race on .web: the second replaced build/ and env.json while the first was still compiling or zipping, so one of them shipped the other's frontend. Hold an exclusive, reentrant file lock on .web/.reflex-build.lock from compilation through ZIP creation, release it before serving, and never create .web for a backend-only export that has nothing to protect. Split out of #7054 (lock only; the opt-in build cache that needs it follows separately). Claude-Session: https://claude.ai/code/session_018nWB9r2HcC5tkT72UxWgGj
With REFLEX_FRONTEND_BUILD_CACHE=true, a production build fingerprints its inputs (generated frontend sources, assets, env.json, lockfiles, the effective build environment, runtime binaries, and installed-dependency metadata) and, on a match, restores the pristine Vite output instead of running it again. Post-build plugins, fallback generation, compression, and frontend-path processing still run on every build. Snapshots are verified before restore and published only after a complete successful build; a disabled cache discards any snapshot so a fresh build is guaranteed. Disabled by default, macOS/Linux only. Split out of #7054; depends on the workspace lock from the previous commit. Claude-Session: https://claude.ai/code/session_018nWB9r2HcC5tkT72UxWgGj
Codex Review SummaryThis comment shows the latest Codex review activity on this pull request.
ℹ️ About Codex in GitHubYour team has set up Codex to review pull requests in this repo. Reviews are triggered when you
Codex reacts with 👀 while any review is running, comments if it has suggestions, and reacts with 👍 once all reviews finish with no findings. |
![cubic-dev-ai[bot]](https://avatars.githubusercontent.com/in/1082092?s=60&v=4){kind=small}
There was a problem hiding this comment.
All reported issues were addressed across 7 files
Reply with feedback, questions, or to request a fix.
Turn on auto-fix | Re-trigger cubic
![chatgpt-codex-connector[bot]](https://avatars.githubusercontent.com/in/1144995?s=60&v=4){kind=small}
There was a problem hiding this comment.
💡 Codex Review
Here are some automated review suggestions for this pull request.
Reviewed commit: 9d997ab081
ℹ️ About Codex in GitHub
Your team has set up Codex to review pull requests in this repo. Reviews are triggered when you
- Open a pull request for review
- Mark a draft as ready
- Comment "@codex review".
If Codex has suggestions, it will comment; otherwise it will react with 👍.
Codex can also answer questions or update the PR. Try commenting "@codex address that feedback".
Merging this PR will not alter performance
|
 [Medium impact] The reviewed changes appear safe to merge. SummaryAdds an opt-in cache for deterministic production frontend builds. Cache hits restore pristine output before post-build work runs again.
Reviews (9) · Last reviewed commit: "Discard the build snapshot when caching ..." · Reviewed by Greptile |
![greptile-apps[bot]](https://avatars.githubusercontent.com/in/867647?s=60&v=4){kind=small}
A cached production build that was killed after capturing its pristine snapshot but before publishing it left reflex.build-cache/pending-* behind. Deploy timeouts, CI cancellation and the OOM killer all do this, because the production build installs no SIGTERM handler and the cleanup in `finally` never runs. Nothing removed such a snapshot while the cache stayed enabled, so every kill leaked one full copy of .web/build. Capture now removes any other pending snapshot before it creates its own. It runs under the exclusive build lock that every snapshot writer holds, so no live build can own one. Removal is best-effort, like close(), so a leftover that cannot be deleted never disables caching. Claude-Session: https://claude.ai/code/session_01PXcFKAgnzhgdnSvhuCmxWE
The build cache reads environment.REFLEX_FRONTEND_BUILD_CACHE, which this branch adds to reflex-base, so reflex now needs an unreleased reflex-base change. Following the sibling dependency floor rule in AGENTS.md, raise the floor to the .dev0 of reflex-base's next version, 0.9.13.dev0, which excludes reflex-base 0.9.12 and earlier. Claude-Session: https://claude.ai/code/session_01PXcFKAgnzhgdnSvhuCmxWE
Brings in #7449, which drops Python 3.10 and moves ruff to py311. Claude-Session: https://claude.ai/code/session_01PXcFKAgnzhgdnSvhuCmxWE
PERF203 only applies before Python 3.11, where try/except in a loop costs per iteration. With ruff targeting py311 since #7449 the directive is unused, which RUF100 rejects. Claude-Session: https://claude.ai/code/session_01PXcFKAgnzhgdnSvhuCmxWE
There was a problem hiding this comment.
All reported issues were addressed across 3 files (changes from recent commits).
Reply with feedback, questions, or to request a fix.
Turn on auto-fix | Re-trigger cubic
The cleanup of snapshots left by killed builds ran in capture(), which a cache hit skips, so a leftover outlived every hit until the next miss. Constructing the cache handle runs under the build lock on every cached build, so the cleanup moves there. Claude-Session: https://claude.ai/code/session_01PXcFKAgnzhgdnSvhuCmxWE
|
CodSpeed Performance Analysis (red on 01009c4) is not this PR's. It flags only The benchmark has been jumping between about 113 µs and 121 µs on every run this hour, with CodSpeed warning that the runtime environments differed:
This run compares against the base's fast reading (113.4 µs). CodSpeed isn't a required check, and I can't re-run it from here. Generated by Claude Code |
masenf
added
documentation
ChatGPT Codex Connector
The production build lock had its own copy of the platform-specific locking (flock / msvcrt) next to the sidecar-based json_file_lock, with different Windows polling, a different failure mode and a lock file inside .web. Keep one implementation: json_file_lock becomes file_lock, usable for files and directories, and gains the reentrancy and fork handling the build lock needed. Exports and production/preview startup now lock the .web directory through it. The sidecar lives in REFLEX_DIR, so locking never creates .web and the lock inode survives .web being replaced. Backend-only exports therefore always take the lock instead of skipping it when .web is absent, which closes the race the review bots flagged. The lock is released explicitly on Windows before the descriptor closes. Claude-Session: https://claude.ai/code/session_017XAFCkYB3Z8w3S6pgJrLCh
# Conflicts: # pyproject.toml
The dedicated build lock module is gone; the cache now serializes through the shared sidecar lock, which no longer lives in .web and so needs no fingerprint exclusion. The symlinked-lock and lock-failure tests covered behavior that lock no longer has. Claude-Session: https://claude.ai/code/session_017XAFCkYB3Z8w3S6pgJrLCh
The backend ZIP walks the app root, and REFLEX_DIR sat inside it, so the exporter archived its own held lock file. Windows refuses reads of a byte-range-locked file, failing the zip with PermissionError. Claude-Session: https://claude.ai/code/session_01PXcFKAgnzhgdnSvhuCmxWE
# Conflicts: # docs/hosting/self-hosting.md
Hash relative POSIX strings from os.scandir instead of building and relativizing a Path per entry, hash file content with hashlib.file_digest, and reuse the path file_lock already resolved. The input fingerprint of a 13.6k-entry .web drops from 308 ms to 73 ms.
masenf
left a comment
masenf
left a comment
There was a problem hiding this comment.
i'm still not 100% convinced that this is a feature we want. i'm struggling to see what users would use this
file_lock now yields a handle naming the resolved path and whether the OS lock was acquired, directly or through an outer block in the same thread. The build cache reads it and runs a plain build without restoring, capturing, publishing or cleaning snapshots when the lock fell back to unlocked, since every snapshot step assumes exclusive access. The docs also state that source and asset fingerprints omit modification times and other filesystem metadata. Claude-Session: https://claude.ai/code/session_017XAFCkYB3Z8w3S6pgJrLCh
There was a problem hiding this comment.
All reported issues were addressed across 5 files (changes from recent commits).
Reply with feedback, questions, or to request a fix.
View guided diff | Turn on auto-fix | Re-trigger cubic
Turning the cache off promises a fresh build and no leftover snapshot. Deleting one without the workspace lock can only lose a snapshot, never reuse a wrong one, so the discard no longer waits on the lock. Claude-Session: https://claude.ai/code/session_017XAFCkYB3Z8w3S6pgJrLCh
This branch has not been deployed
3 participants
Split out of #7054. Built on #7453 (the workspace lock), which has merged; this PR now targets
mainand its diff is the cache alone. This is the big one and the one @masenf was nervous about, so the numbers below are the case for it.Changes
REFLEX_FRONTEND_BUILD_CACHE=true(default off, macOS/Linux only) makesbuild.build()fingerprint its inputs and, on a match, copy the previous pristine Vite output back into.web/buildinstead of running Vite:env.json, lockfiles,reflex.jsonminus private timestamps, the effective build environment (everythingfrontend_env()passes to the subprocess exceptREFLEX_LOGLEVELandSHLVL), runtime binaries, Reflex version, and installed-dependency file metadata (dev/inode/size/mtime/ctime; content is not hashed fornode_modules).frontend_pathprocessing still run on every build, on the restored output.REFLEX_FRONTEND_BUILD_CACHE=falsediscards the snapshot so a fresh build is guaranteed. Symlinks leaving.web, or into ignored caches, bypass the cache..weblock throughpath_ops.file_lockfrom Serialize production frontend work sharing one .web directory #7453, so reload builds and exports serialize with each other.docs/hosting/self-hosting.md; news fragments forreflexandreflex-base.Measurement
Unchanged app, repeat
reflex export, bun:Same with npm: 1.63 s total / 0.88 s build phase on a hit, versus 7.6 s / 6.9 s when Vite runs.
A cache miss (deploy URL changed, so
env.jsondiffers) costs the fingerprinting plus a snapshot copy on top of a normal build: 9.22 s total versus 7.35 s with the cache off (+1.9 s, two input fingerprints plus a snapshot copy of.web/build), so the cache only pays for itself when at least every third build is a hit.What a user gets: the Vite step is the whole export on a repeat deploy, so a hit removes roughly 85% of it. What it costs: ~450 lines in
build_cache.py, a new env var to document, and the lock in #7453. It only helps repeat deploys of an unchanged frontend on one machine, and it must stay off for builds that read the clock, the network, or files outside.web.Measurement setup
Five-page app from the
blanktemplate (Radix Themes + Tailwind v4 plugins), Linux x86_64, 4 vCPU, Python 3.14, Node 22.22, npm 10.9, bun 1.4.2. Each run is a fresh interpreter callingreflex.utils.export.export()the same wayreflex exportdoes; durations are the phase timings the export already reports to telemetry (compileincludes the dependency install). Medians of 3 runs unless noted. Vite dominates every export (~6.5 s); these numbers isolate the part this PR touches.Type of change
Checklist
tests/unitssuite passes locally except the 14reflex_bench/drivers/test_browser.pycases that need a Playwright browser launch, which fail identically on untouchedmainin this sandboxruff check,ruff format --check,codespell, andpyright reflex testspass locallyhttps://claude.ai/code/session_018nWB9r2HcC5tkT72UxWgGj
https://claude.ai/code/session_017XAFCkYB3Z8w3S6pgJrLCh