Poetry install: broken dependencies #3687
Comments
Going with nixos is the safe bet. You can install nix on your Mac and then enter |
|
PyYAML issue fixed here: #3720 |
|
iiiiii don't think we even need the |
|
curve25519-donna is used in crypto tests -- and in some strange manner too, i'm having trouble replacing it with any other library. but I don't think breaking the whole env just for two test cases is the right thing to do @onvej-sl you're probably the person who can best look at |
This doesn't work because _ed225519.py implements ed25519 instead of curve25519. Nevertheless, both the curves are birationally equivalent. This means the underlying groups are isomorphic and the isomorphism is easily computable.
I think we can use the package cryptography which is already used in device tests. |
See the diff:--- a/crypto/tests/test_curves.py
+++ b/crypto/tests/test_curves.py
@@ -5,9 +5,9 @@ import hashlib
import os
import random
-import curve25519
import ecdsa
import pytest
+from cryptography.hazmat.primitives.asymmetric.x25519 import X25519PrivateKey
def bytes2num(s):
@@ -344,17 +344,17 @@ def test_validate_pubkey_direct(point):
def test_curve25519(r):
sec1 = bytes(bytearray(r.randbytes(32)))
sec2 = bytes(bytearray(r.randbytes(32)))
- pub1 = curve25519.Private(sec1).get_public()
- pub2 = curve25519.Private(sec2).get_public()
+ pub1 = X25519PrivateKey.from_private_bytes(sec1).public_key()
+ pub2 = X25519PrivateKey.from_private_bytes(sec2).public_key()
session1 = r.randbytes(32)
- lib.curve25519_scalarmult(session1, sec2, pub1.public)
+ lib.curve25519_scalarmult(session1, sec2, pub1.public_bytes_raw())
session2 = r.randbytes(32)
- lib.curve25519_scalarmult(session2, sec1, pub2.public)
+ lib.curve25519_scalarmult(session2, sec1, pub2.public_bytes_raw())
assert bytearray(session1) == bytearray(session2)
- shared1 = curve25519.Private(sec2).get_shared_key(pub1, hashfunc=lambda x: x)
- shared2 = curve25519.Private(sec1).get_shared_key(pub2, hashfunc=lambda x: x)
+ shared1 = X25519PrivateKey.from_private_bytes(sec2).exchange(pub1)
+ shared2 = X25519PrivateKey.from_private_bytes(sec1).exchange(pub2)
assert shared1 == shared2
assert bytearray(session1) == shared1
assert bytearray(session2) == shared2
@@ -362,10 +362,10 @@ def test_curve25519(r):
def test_curve25519_pubkey(r):
sec = bytes(bytearray(r.randbytes(32)))
- pub = curve25519.Private(sec).get_public()
+ pub = X25519PrivateKey.from_private_bytes(sec).public_key()
res = r.randbytes(32)
lib.curve25519_scalarmult_basepoint(res, sec)
- assert bytearray(res) == pub.public
+ assert bytearray(res) == pub.public_bytes_raw()
def test_curve25519_scalarmult_from_gpg(r): |
Perfect. Let's do this and drop both curve-donna and ed25519 from pyproject.toml -> #3730 |
Describe the bug
After cloning the main branch poetry won't install pyyaml depenency with python version
3.12.2and older (see bellow).Firmware version and revision
mainbranch. Just clonedgit clone --recurse-submodules https://github.com/trezor/trezor-firmware.gitpoetry install
Desktop setup:
macOS Sonoma 14.4.1 (23E224)3.12.2also occurs on3.11.9,3.8.1and3.8.2To Reproduce
Steps to reproduce the behavior just follow the build instructions from docs:
git clone --recurse-submodules https://github.com/trezor/trezor-firmware.gitcd trezor-firmwarepoetry installExpected behavior
No error. I can proceed to
4.
cd coreScreenshots
Installing pyyaml (6.0): Failed ChefBuildError Backend subprocess exited when trying to invoke get_requires_for_build_wheel running egg_info writing lib/PyYAML.egg-info/PKG-INFO writing dependency_links to lib/PyYAML.egg-info/dependency_links.txt writing top-level names to lib/PyYAML.egg-info/top_level.txt Traceback (most recent call last): File "/Users/martin/Library/Application Support/pipx/venvs/poetry/lib/python3.12/site-packages/pyproject_hooks/_in_process/_in_process.py", line 353, in <module> main() File "/Users/martin/Library/Application Support/pipx/venvs/poetry/lib/python3.12/site-packages/pyproject_hooks/_in_process/_in_process.py", line 335, in main json_out['return_val'] = hook(**hook_input['kwargs']) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/Users/martin/Library/Application Support/pipx/venvs/poetry/lib/python3.12/site-packages/pyproject_hooks/_in_process/_in_process.py", line 118, in get_requires_for_build_wheel return hook(config_settings) ^^^^^^^^^^^^^^^^^^^^^ File "/private/var/folders/zh/s5jmjfgn78vcz6ryn05ghls00000gn/T/tmpwonam_m1/.venv/lib/python3.12/site-packages/setuptools/build_meta.py", line 325, in get_requires_for_build_wheel return self._get_build_requires(config_settings, requirements=['wheel']) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/private/var/folders/zh/s5jmjfgn78vcz6ryn05ghls00000gn/T/tmpwonam_m1/.venv/lib/python3.12/site-packages/setuptools/build_meta.py", line 295, in _get_build_requires self.run_setup() File "/private/var/folders/zh/s5jmjfgn78vcz6ryn05ghls00000gn/T/tmpwonam_m1/.venv/lib/python3.12/site-packages/setuptools/build_meta.py", line 311, in run_setup exec(code, locals()) File "<string>", line 288, in <module> File "/private/var/folders/zh/s5jmjfgn78vcz6ryn05ghls00000gn/T/tmpwonam_m1/.venv/lib/python3.12/site-packages/setuptools/__init__.py", line 104, in setup return distutils.core.setup(**attrs) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/private/var/folders/zh/s5jmjfgn78vcz6ryn05ghls00000gn/T/tmpwonam_m1/.venv/lib/python3.12/site-packages/setuptools/_distutils/core.py", line 185, in setup return run_commands(dist) ^^^^^^^^^^^^^^^^^^ File "/private/var/folders/zh/s5jmjfgn78vcz6ryn05ghls00000gn/T/tmpwonam_m1/.venv/lib/python3.12/site-packages/setuptools/_distutils/core.py", line 201, in run_commands dist.run_commands() File "/private/var/folders/zh/s5jmjfgn78vcz6ryn05ghls00000gn/T/tmpwonam_m1/.venv/lib/python3.12/site-packages/setuptools/_distutils/dist.py", line 969, in run_commands self.run_command(cmd) File "/private/var/folders/zh/s5jmjfgn78vcz6ryn05ghls00000gn/T/tmpwonam_m1/.venv/lib/python3.12/site-packages/setuptools/dist.py", line 967, in run_command super().run_command(command) File "/private/var/folders/zh/s5jmjfgn78vcz6ryn05ghls00000gn/T/tmpwonam_m1/.venv/lib/python3.12/site-packages/setuptools/_distutils/dist.py", line 988, in run_command cmd_obj.run() File "/private/var/folders/zh/s5jmjfgn78vcz6ryn05ghls00000gn/T/tmpwonam_m1/.venv/lib/python3.12/site-packages/setuptools/command/egg_info.py", line 321, in run self.find_sources() File "/private/var/folders/zh/s5jmjfgn78vcz6ryn05ghls00000gn/T/tmpwonam_m1/.venv/lib/python3.12/site-packages/setuptools/command/egg_info.py", line 329, in find_sources mm.run() File "/private/var/folders/zh/s5jmjfgn78vcz6ryn05ghls00000gn/T/tmpwonam_m1/.venv/lib/python3.12/site-packages/setuptools/command/egg_info.py", line 550, in run self.add_defaults() File "/private/var/folders/zh/s5jmjfgn78vcz6ryn05ghls00000gn/T/tmpwonam_m1/.venv/lib/python3.12/site-packages/setuptools/command/egg_info.py", line 588, in add_defaults sdist.add_defaults(self) File "/private/var/folders/zh/s5jmjfgn78vcz6ryn05ghls00000gn/T/tmpwonam_m1/.venv/lib/python3.12/site-packages/setuptools/command/sdist.py", line 102, in add_defaults super().add_defaults() File "/private/var/folders/zh/s5jmjfgn78vcz6ryn05ghls00000gn/T/tmpwonam_m1/.venv/lib/python3.12/site-packages/setuptools/_distutils/command/sdist.py", line 251, in add_defaults self._add_defaults_ext() File "/private/var/folders/zh/s5jmjfgn78vcz6ryn05ghls00000gn/T/tmpwonam_m1/.venv/lib/python3.12/site-packages/setuptools/_distutils/command/sdist.py", line 336, in _add_defaults_ext self.filelist.extend(build_ext.get_source_files()) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "<string>", line 204, in get_source_files File "/private/var/folders/zh/s5jmjfgn78vcz6ryn05ghls00000gn/T/tmpwonam_m1/.venv/lib/python3.12/site-packages/setuptools/_distutils/cmd.py", line 107, in __getattr__ raise AttributeError(attr) AttributeError: cython_sources at ~/Library/Application Support/pipx/venvs/poetry/lib/python3.12/site-packages/poetry/installation/chef.py:164 in _prepare 160│ 161│ error = ChefBuildError("\n\n".join(message_parts)) 162│ 163│ if error is not None: → 164│ raise error from None 165│ 166│ return path 167│ 168│ def _prepare_sdist(self, archive: Path, destination: Path | None = None) -> Path: Note: This error originates from the build backend, and is likely not a problem with poetry but with pyyaml (6.0) not supporting PEP 517 builds. You can verify this by running 'pip wheel --no-cache-dir --use-pep517 "pyyaml (==6.0)"'.Additional context
The problem is caused by pyyaml version 6.0 as described here. The problem is fixed in pyyaml version 6.0.1 which, for some reason, is not installed during the poetry installation process.
I suspect some dependency requires exactly 6.0.0 version. I have tried to install the pyyaml in an empty dummy project with the exact dependency from trezor-firmware/poetry.lock :
and the latest version
6.0.1was installed without problems.As a workaround I have tried to remove the
poetry.lockfile and add pyyaml version6.0.1as the project dependency in pyproject.toml:Which installs the pyyaml correctly but throws another error. The ed25519 installation now failing:
The problem with
ed25519could be caused by changing the configparser API in python3.12as described here. The problem seems to be with packageversioneerin particular which does not seem to be supporting the latest python as described hereSo, I tried several python environments older than 3.12 (
3.11.9,3.8.1and3.8.2).With these older version I got error when installing
curve25519-donnapackage:In sum, it is very nice dependency hell and I wonder if there is anyone who could confirm this bug and/or tell me what is the working python version setup on mac OS for the current
masterbranch.The text was updated successfully, but these errors were encountered: