Support the uv package manager

[mikepink]

uv is a drop-in replacement for pip which runs very quickly. We've replaced pip with uv locally and have noticed a significant speed up in our Python build time in our development and CI environments. By adopting support for uv, we anticipate our Heroku builds would complete ~5 minutes more quickly.

Because uv is a drop-in replacement, we found adoption to be very straightforward in our case. pip install -r requirements.txt becomes uv pip install -r requirements.txt. The promise of significantly improved performance with a very simple migration path was enough to convince us to give it a try internally.

Happy to attempt a PR if that would be welcome/well received.

[edmorley]

@mikepink Hi! Thank you for opening this - I'd love to see uv support too 😄

Since this issue was opened, uv has added support for project management (uv.lock + the uv sync command etc). As such, I think it would make most sense to support using uv sync rather than the uv pip compatibility commands (on Heroku the latter wouldn't speed up warm builds by much due to the current caching of site-packages, plus wouldn't give any of the determinism benefits etc).

I don't yet have an exact timeline for when I'll be able to add support - I personally would like it to be as soon as possible! (The buildpack refactoring/improvements made as part of adding support for Poetry recently will at least make it much faster to add uv support than for adding Poetry support.)

See also the related issue for the Python CNB (Cloud Native Buildpack; the next-generation buildpack that is due to replace this classic Heroku buildpack in the future):
heroku/buildpacks-python#248

[edmorley]

Btw the more thumbs up I can get on both this GitHub issue and the roadmap issue (heroku/roadmap#323) the easier it will be to make a case for it during internal roadmap/product planning - so please everyone who is interested in uv support add a 👍 to both GitHub issues!

[MarounMaroun]

We're considering migrating to uv, but the buildpack is one of our main concerns. Any updates on this one? 🤔

[edmorley]

@MarounMaroun Hi! I've been experimenting with uv support locally the last couple of days.

It's mostly looking good so far - though I ran into an issue with the way uv sync creates entrypoint scripts with symlinked Python installs (something the classic Python buildpack has to do due to the Cedar-generation build system using /tmp/build_* as the directory at build time rather than /app) - which I've reported here:
astral-sh/uv#11048

We could try and work around it in the classic buildpack by manually rewriting all entrypoint scripts, but I'd prefer not to if possible, so will see what I hear back first.

[edmorley]

The upstream uv issue mentioned above has since been fixed 🎉

With that fixed, the proof-of-concept I have locally mostly works - though I found two more items that will need looking at next:

1. uv's link mode option not supporting a "hardlink OR clone" option - which makes supporting the variety of build environments and configurations harder, since we have some environments which need to be using one or the other (eg ext4 only supports same mount hardlink, but xfs supports cross-mount when using reflinks only) - and I'd rather not have to force them all to copy or have to write my own reflink detection etc. I'll need to write up a clear issue upstream justifying why the feature request is necessary (likely referencing pnpm as prior art too).
2. Deciding best approach for apps that don't have a .python-version file, since uv's Python discovery can pick the wrong Python (eg distro Python in base image) or give a confusing error message if an app uses runtime.txt that doesn't match their pyproject.toml requires-python (or doesn't specify a Python version at all and the buildpack default/cached version doesn't match requires-python) etc. The solution here will likely be to make .python-version mandatory when using uv, though (a) I need to decide whether long term we want to make .python-version mandatory for all apps regardless of package manager for consistency (or at least apps using a lockfile-supporting package manager), (b) I'll need to write up some clear error message copy for the various error cases.

I'll also need to work on the Python CNB implementation, since we'd really need to add support to both buildpacks at once.

I was working on the proof-of-concept during our hack week last week, so for this week I need to pivot back to some other work - but I hope to come back to this very soon!

In other news, it looks like Dependabot support for uv will be coming soon (which I imagine is another blocker for many adopters):
dependabot/dependabot-core#10478
github/roadmap#1100