META: spksrc framework python TODO list

[th0ma7]

Non-exhaustive TODO whishlist to facilitate python package mainteanance

Wheel compile specific

• spksrc.python-wheel.mk Allow using *_ARGS arguments (similar to python-wheel-meson.mk)
• spksrc.wheel-compile.mk Migrate to new *_ARGS arguments (similar to python-wheel-meson.mk)
• Migrate to newer pip (depends on above, otherwise stuck on v24.x)
• Re-evaluate the need for abi3 requirement files
• Add status-build.log entries for wheel builds
• When calling make WHEELS="rpds_py==0.25.1" wheel-comcerto2k-7.1 creates log filename build---wheel.log

Wheel download specific

• Fix wheel source download in github-action
• Use flock to avoid parallel builds to fail

crossenv

• Allow automated version management for wheelhouse:numpy==<version> similar use cases
• Migrate per python version crossenv definition files to a common location

Ideas on this could be a combinaion of

1. spkrsc/python/crossenv for usual requirements*.txt manage wheels
2. Directly within `spksrc/python//requirement.txt for directory specific wheels

Automation

• Automate the updating of wheels in python31*-wheels/src/requirements*.txt files using github automation procesess (Automate wheel updates through github-action #6629)
• Migrate native/python31* to use requirements.txt file instead of embeded within its Makefile (Automate wheel updates through github-action #6629)

Others

• Removal of python38 (Removal of python38 and domoticz #6631)
• Removal of python311

[th0ma7]

@mreid-tt @hgy59 I've created this issue to track python related framework changes we'd like to see in the future. There was a lot of more than overdue changes over the last few months and a few more would be needed in order to facilitate future compatibility and maintenance.

Feel free to add things I may have overlook.

[mreid-tt]

@th0ma7 Other possible objectives we could consider:

• Make Python packages self-contained by default to speed up installation (aside from package size, is there any reason we can’t avoid using the index entirely?)
• Simplify the bundled requirements so it doesn’t attempt to reinstall included packages like setuptools, etc.

[th0ma7]

@mreid-tt interesting, I was mostly thinking of the framework side of things, but indeed there may be things on the client side as well.

• Make Python packages self-contained by default to speed up installation (aside from package size, is there any reason we can’t avoid using the index entirely?)

I did tried that in the past and there was a super long thread on that matter. It may since have been resolved? I tried finding back that issue but couldn't, too many opened items in pip github issue page.

• Simplify the bundled requirements so it doesn’t attempt to reinstall included packages like setuptools, etc.

pip does handle that so it isn't re-installing an already installed wheel. Beyond that what did you had in mind?

[mreid-tt]

• Simplify the bundled requirements so it doesn’t attempt to reinstall included packages like setuptools, etc.

pip does handle that so it isn't re-installing an already installed wheel. Beyond that what did you had in mind?

What I was thinking is that the packages include multiple requirements.txt files, which may overlap with the base Python installation. This could lead to attempts to reinstall packages that are already present, even if they’ve been commented out in the build process.

Also, not sure if you saw my earlier note — this line might contain a bug:

spksrc/mk/spksrc.wheel.mk

Line 46 in f158587

ifneq ($(and $(WHEEL_NAME),$(or (WHEEL_VERISON),$(WHEEL_URL))),)

Specifically, shouldn't or (WHEEL_VERISON) be or $(WHEEL_VERISON)?