Specify dependency groups in a way that doesn't require poetry #733
Conversation
Poetry is required to use dependency groups. To make things compatible with other ways of dependency management, switch to PEP621 optional dependencies, but specified using the Poetry configuration aliases to retain support for poetry. TODO: Docs
As long as we're using it we should be validating! :)
mfisher87
force-pushed
the
poetry-groups-to-pep621-optional-deps
branch
from
e36aa7c
to
1b84b1f
Compare
Maybe we could use Nox instead of scripts at some point? Then the behaviors can be coupled to the correct environments.
These settings are already specified in pyproject.toml
mfisher87
force-pushed
the
poetry-groups-to-pep621-optional-deps
branch
from
6157ee0
to
6040f4d
Compare
mfisher87
force-pushed
the
poetry-groups-to-pep621-optional-deps
branch
from
439cb98
to
faa4571
Compare
| - repo: https://github.com/python-poetry/poetry | ||
| rev: '1.8.3' | ||
| hooks: | ||
| - id: poetry-check |
There was a problem hiding this comment.
this will fail if the lockfile hasn't been updated, or if there's a typo in a package name in poetry config.
I have been exploring hatch w/ hatchling, and it is both PEP compliant and also (like Poetry) avoids using extras (a.k.a optional-dependencies) for managing development environments. Information about extras is built into the distribution metadata and primarily provides extra packages users might want (i.e. |
|
Just to clarify, I'm for including these optional dependency groups in the package metadata, as that's in my experience a very common method and is compatible with all environment / package managers, and as far as I know there is no standard way to specify dependency groups that do not go in the package metadata. I'd like to avoid pros and cons of environment / package managers here (i.e. Hatch, Poetry), as we can isolate that concern from our build tooling and configuration. Regardless of whether we decide down the road to document / encourage Hatch or Poetry or something else for package management, if our build configuration is PEP621 compliant, contributors can choose any tool. |
|
To make things more confusing, PDM has "dev dependencies", their own uniquely named version of Poetry's "dependency groups" and Hatch's "environment dependencies". Wow! The "same" thing three times with three names and three config keys. I'd really like to treat this Python environment management feature as non-standard and avoid it until there's a unified interface, which I'm sure there's already an open PEP for 😆 |
|
Found it: https://peps.python.org/pep-0735/ |
"Dependency groups" are a poetry-specific feature that's different from PEP621 "optional dependencies", aka "extras". To make things compatible with other ways of dependency management, switch to PEP621 optional dependencies, but specified using the Poetry configuration aliases to retain support for poetry. While the Poetry "dependency groups" functionality may be more aimed at development workflows, it's not compatible with other tools and in my experience the dominant convention in the ecosystem is to use optional dependencies. As @itcarroll points out below, Hatch supports a similar concept called "environment dependencies" and specifies them with a different tool-specific configuration key. PDM also calls it something different, "dev dependencies", and uses another proprietary config table. 😓
I really don't like the way poetry requires optional dependencies to be specified. It requires duplication.
Thoughts? I don't want to add all this duplication to our codebase. Can we go forward with replacing Poetry with good ol' setuptools or maybe hatchling for builds and switch the optional dependency groups to the standard format? Anything PEP621 compatible would be a step forward IMO. I can take that on as part of this PR. The PEP621 example shows our use case, test dependencies, implemented in the conventional way.
Closes #734
Closes #619
📚 Documentation preview 📚: https://earthaccess--733.org.readthedocs.build/en/733/