The `.pkg` environment has no project dependencies, raise ModuleNotFoundError: No module named 'xxx'

[YaoJusheng]

Issue

Environment

Provide at least:

• OS:

Output of pip list of the host Python, where tox is installed

 + anyio==4.6.2.post1
 + certifi==2024.8.30
 + cffi==1.17.1
 + coverage==7.6.3
 + cryptography==43.0.1
 + exceptiongroup==1.2.2
 + execnet==2.1.1
 + factory-boy==3.3.1
 + faker==30.6.0
 + h11==0.14.0
 + httpcore==1.0.6
 + httpx==0.27.2
 + idna==3.10
 + iniconfig==2.0.0
 + orjson==3.10.7
 + packaging==24.1
 + pluggy==1.5.0
 + pycparser==2.22
 + pytest==8.3.3
 + pytest-asyncio==0.24.0
 + pytest-cov==5.0.0
 + pytest-xdist==3.6.1
 + python-dateutil==2.9.0.post0
 + six==1.16.0
 + sniffio==1.3.1
 + tomli==2.0.2
 + typing-extensions==4.12.2

Output of running tox

Output of tox -rvv

py310: 877 I exit 0 (0.07 seconds) /Users/yaojusheng/sdk_light> uv pip install factory_boy pytest pytest-asyncio pytest-cov pytest-xdist -r /Users/yaojusheng/sdk_light/requirements.txt pid=99627 [tox/execute/api.py:286]
.pkg: 879 W venv> .tox/.tox/bin/uv venv -p /Users/yaojusheng/sdk_light/.tox/.tox/bin/python --allow-existing -v /Users/yaojusheng/sdk_light/.tox/.pkg [tox/tox_env/api.py:427]
DEBUG uv 0.4.24 (b9cd54913 2024-10-17)
DEBUG Checking for Python interpreter at path `.tox/.tox/bin/python`
Using CPython 3.10.12 interpreter at: .tox/.tox/bin/python
Creating virtual environment at: .tox/.pkg
DEBUG Allowing existing directory
.pkg: 891 I exit 0 (0.01 seconds) /Users/yaojusheng/sdk_light> .tox/.tox/bin/uv venv -p /Users/yaojusheng/sdk_light/.tox/.tox/bin/python --allow-existing -v /Users/yaojusheng/sdk_light/.tox/.pkg pid=99628 [tox/execute/api.py:286]
.pkg: 892 W install_requires> .tox/.tox/bin/uv pip install 'setuptools>=40.8.0' wheel -v [tox/tox_env/api.py:427]
DEBUG uv 0.4.24 (b9cd54913 2024-10-17)
DEBUG Searching for default Python interpreter in system path
DEBUG Found `cpython-3.10.12-macos-aarch64-none` at `/Users/yaojusheng/sdk_light/.tox/.pkg/bin/python3` (active virtual environment)
Using Python 3.10.12 environment at .tox/.pkg
DEBUG Acquired lock for `.tox/.pkg`
DEBUG At least one requirement is not satisfied: wheel
DEBUG Using request timeout of 30s
DEBUG Solving with installed Python version: 3.10.12
DEBUG Solving with target Python version: >=3.10.12
DEBUG Adding direct dependency: setuptools>=40.8.0
DEBUG Adding direct dependency: wheel*
DEBUG Found fresh response for: https://pypi.org/simple/wheel/
DEBUG Found fresh response for: https://pypi.org/simple/setuptools/
DEBUG Searching for a compatible version of setuptools (>=40.8.0)
DEBUG Found fresh response for: https://files.pythonhosted.org/packages/1b/d1/9babe2ccaecff775992753d8686970b1e2755d21c8a63be73aba7a4e7d77/wheel-0.44.0-py3-none-any.whl.metadata
DEBUG Selecting: setuptools==75.2.0 [compatible] (setuptools-75.2.0-py3-none-any.whl)
DEBUG Found fresh response for: https://files.pythonhosted.org/packages/31/2d/90165d51ecd38f9a02c6832198c13a4e48652485e2ccf863ebb942c531b6/setuptools-75.2.0-py3-none-any.whl.metadata
DEBUG Searching for a compatible version of wheel (*)
DEBUG Selecting: wheel==0.44.0 [compatible] (wheel-0.44.0-py3-none-any.whl)
DEBUG Tried 2 versions: setuptools 1, wheel 1
DEBUG Split specific environment resolution took 0.002s
Resolved 2 packages in 2ms
DEBUG Requirement already cached: setuptools==75.2.0
DEBUG Requirement already cached: wheel==0.44.0
Installed 2 packages in 11ms
 + setuptools==75.2.0
 + wheel==0.44.0
DEBUG Released lock at `/Users/yaojusheng/sdk_light/.tox/.pkg/.lock`
.pkg: 917 I exit 0 (0.02 seconds) /Users/yaojusheng/sdk_light> .tox/.tox/bin/uv pip install 'setuptools>=40.8.0' wheel -v pid=99629 [tox/execute/api.py:286]
.pkg: 919 W _optional_hooks .> python /Users/yaojusheng/sdk_light/.tox/.tox/lib/python3.10/site-packages/pyproject_api/_backend.py True setuptools.build_meta __legacy__ [tox/tox_env/api.py:427]
Backend: run command _optional_hooks with args {}
Backend: Wrote response {'return': {'get_requires_for_build_sdist': True, 'prepare_metadata_for_build_wheel': True, 'get_requires_for_build_wheel': True, 'build_editable': True, 'get_requires_for_build_editable': True, 'prepare_metadata_for_build_editable': True}} to /var/folders/8q/rp1fz7gx78qc7lpd3mb2dhdr0000gn/T/pep517__optional_hooks-o160l1gc.json
.pkg: 1087 I exit None (0.17 seconds) .> python /Users/yaojusheng/sdk_light/.tox/.tox/lib/python3.10/site-packages/pyproject_api/_backend.py True setuptools.build_meta __legacy__ pid=99630 [tox/execute/api.py:286]
.pkg: 1212 W get_requires_for_build_editable .> python /Users/yaojusheng/sdk_light/.tox/.tox/lib/python3.10/site-packages/pyproject_api/_backend.py True setuptools.build_meta __legacy__ [tox/tox_env/api.py:427]
Backend: run command get_requires_for_build_editable with args {'config_settings': None}
Traceback (most recent call last):
  File "/Users/yaojusheng/sdk_light/.tox/.tox/lib/python3.10/site-packages/pyproject_api/_backend.py", line 93, in run
    outcome = backend_proxy(parsed_message["cmd"], **parsed_message["kwargs"])
  File "/Users/yaojusheng/sdk_light/.tox/.tox/lib/python3.10/site-packages/pyproject_api/_backend.py", line 34, in __call__
    return getattr(on_object, name)(*args, **kwargs)
  File "/Users/yaojusheng/sdk_light/.tox/.pkg/lib/python3.10/site-packages/setuptools/build_meta.py", line 477, in get_requires_for_build_editable
    return self.get_requires_for_build_wheel(config_settings)
  File "/Users/yaojusheng/sdk_light/.tox/.pkg/lib/python3.10/site-packages/setuptools/build_meta.py", line 332, in get_requires_for_build_wheel
    return self._get_build_requires(config_settings, requirements=[])
  File "/Users/yaojusheng/sdk_light/.tox/.pkg/lib/python3.10/site-packages/setuptools/build_meta.py", line 302, in _get_build_requires
    self.run_setup()
  File "/Users/yaojusheng/sdk_light/.tox/.pkg/lib/python3.10/site-packages/setuptools/build_meta.py", line 516, in run_setup
    super().run_setup(setup_script=setup_script)
  File "/Users/yaojusheng/sdk_light/.tox/.pkg/lib/python3.10/site-packages/setuptools/build_meta.py", line 318, in run_setup
    exec(code, locals())
  File "<string>", line 10, in <module>
  File "/Users/yaojusheng/sdk_light/noeliclight/__init__.py", line 10, in <module>
    from .crypto import AESUtil, RSAUtil, HashUtil  # noqa: F401
  File "/Users/yaojusheng/sdk_light/noeliclight/crypto.py", line 11, in <module>
    from anyio import to_thread
ModuleNotFoundError: No module named 'anyio'
Backend: Wrote response {'code': 1, 'exc_type': 'ModuleNotFoundError', 'exc_msg': "No module named 'anyio'"} to /var/folders/8q/rp1fz7gx78qc7lpd3mb2dhdr0000gn/T/pep517_get_requires_for_build_editable-xpglx41s.json
.pkg: 1326 I exit None (0.11 seconds) .> python /Users/yaojusheng/sdk_light/.tox/.tox/lib/python3.10/site-packages/pyproject_api/_backend.py True setuptools.build_meta __legacy__ pid=99647 [tox/execute/api.py:286]
py310: 1438 E packaging backend failed (code=1), with ModuleNotFoundError: No module named 'anyio'
Traceback (most recent call last):
  File "/Users/yaojusheng/sdk_light/.tox/.tox/lib/python3.10/site-packages/pyproject_api/_backend.py", line 93, in run
    outcome = backend_proxy(parsed_message["cmd"], **parsed_message["kwargs"])
  File "/Users/yaojusheng/sdk_light/.tox/.tox/lib/python3.10/site-packages/pyproject_api/_backend.py", line 34, in __call__
    return getattr(on_object, name)(*args, **kwargs)
  File "/Users/yaojusheng/sdk_light/.tox/.pkg/lib/python3.10/site-packages/setuptools/build_meta.py", line 477, in get_requires_for_build_editable
    return self.get_requires_for_build_wheel(config_settings)
  File "/Users/yaojusheng/sdk_light/.tox/.pkg/lib/python3.10/site-packages/setuptools/build_meta.py", line 332, in get_requires_for_build_wheel
    return self._get_build_requires(config_settings, requirements=[])
  File "/Users/yaojusheng/sdk_light/.tox/.pkg/lib/python3.10/site-packages/setuptools/build_meta.py", line 302, in _get_build_requires
    self.run_setup()
  File "/Users/yaojusheng/sdk_light/.tox/.pkg/lib/python3.10/site-packages/setuptools/build_meta.py", line 516, in run_setup
    super().run_setup(setup_script=setup_script)
  File "/Users/yaojusheng/sdk_light/.tox/.pkg/lib/python3.10/site-packages/setuptools/build_meta.py", line 318, in run_setup
    exec(code, locals())
  File "<string>", line 10, in <module>
  File "/Users/yaojusheng/sdk_light/noeliclight/__init__.py", line 10, in <module>
    from .crypto import AESUtil, RSAUtil, HashUtil  # noqa: F401
  File "/Users/yaojusheng/sdk_light/noeliclight/crypto.py", line 11, in <module>
    from anyio import to_thread
ModuleNotFoundError: No module named 'anyio'
Backend: run command get_requires_for_build_editable with args {'config_settings': None}
Backend: Wrote response {'code': 1, 'exc_type': 'ModuleNotFoundError', 'exc_msg': "No module named 'anyio'"} to /var/folders/8q/rp1fz7gx78qc7lpd3mb2dhdr0000gn/T/pep517_get_requires_for_build_editable-xpglx41s.json [tox/session/cmd/run/single.py:54]
  py310: FAIL code 1 (1.25 seconds)
  evaluation failed :( (1.38 seconds)

Minimal example

tox.ini

[tox]
envlist = py310
install_command = uv pip install {opts} {packages}
requires = 
    tox>=4
    tox-uv
    wheel
    setuptools >= 65
setupdir = .
toxworkdir = {toxinidir}/.tox
temp_dir = {toxworkdir}/.tmp
isolated_build = True
package_env = .pkg

[testenv]
usedevelop = True
package = wheel
wheel_build_env = .pkg
setenv =
  PYTHONPATH = {toxinidir}
  PIP_INDEX_URL = {env:PIP_INDEX_URL:https://pypi.tuna.tsinghua.edu.cn/simple/}
  PIP_EXTRA_INDEX_URL = {env:PIP_EXTRA_INDEX_URL:https://pypi.org/simple/}
install_command = uv pip install {opts} {packages}
deps = 
    pytest
    -r{toxinidir}/requirements.txt

[testenv:.pkg]
pass_env =
    PKG_CONFIG
    PKG_CONFIG_PATH
    PKG_CONFIG_SYSROOT_DIR
basepython = python3.10
usedevelop = false
; deps = -r{toxinidir}/requirements.txt
commands = python setup.py sdist bdist_wheel

[testenv:dev]
deps = setuptools >= 65
package = editable

When I add deps = -r{toxinidir}/requirements.txt in .pkg, a configuration parsing error will be raised:
...
packaging.requirements.InvalidRequirement: Expected package name at the start of dependency specifier
-r /Users/yaojusheng/sdk_light/requirements.txt
^

[jugmac00]

I can confirm that deps works with a package name, but not with -r requirements.txt.

@gaborbernat I assume we want to support the -r requirements.txt syntax, and the current behavior is a bug?

Could you give me some examples when you would want to specify packaging dependencies yourself, and not relying on the build backend to take care of them?

[gaborbernat]

I would say that specifying any dependencies to the package environment outside of the build backend way is just not supported.

[jugmac00]

Then we probably should error out when deps is configured in .pkg?

[gaborbernat]

This is now available in version 4.36.0 🎉