Automate wheel updates through github-action

[th0ma7]

Description

Intent is to automate wheel updates through github-action so a PR can be made for every wheel needing updating as found into our requirement files. IMPORTANT: Said PR would contain a wheel update for all package where it was found.

How it works:

1. A .github/actions/generate_dependabot.py script generates a .github/dependabot.yml including all detected requirement files such as:

version: 2
updates:
- package-ecosystem: pip
  directory: /spk/python310-wheels/src
  requirements-file: requirements-abi3.txt
  schedule:
    interval: weekly
  groups:
    all-python-deps:
      patterns:
      - '*'
- package-ecosystem: pip
  directory: /spk/python311-wheels/src
  requirements-file: requirements-abi3.txt
  schedule:
    interval: weekly
  groups:
    all-python-deps:
      patterns:
      - '*'
...

2. A .github/workflows/generate-dependabot.yml calls the generate_dependabot.py script every week.
3. github-action ends-up checking potential update candidates from the .github/dependabot.yml and creates associated PR

Extra tid-bits:

• Manages exclusion list namely for pip, Cython and msgpack
• native/python3* now uses a requirements.txt file and capture pip version at processing time

Relates to : #6619

Checklist

• Build rule all-supported completed successfully
• New installation of package completed successfully
• Package upgrade completed successfully (Manually install the package again)
• Package functionality was tested
• Any needed documentation is updated/created

Type of change

• Bug fix
• New Package
• Package update
• Includes small framework changes
• This change requires a documentation update (e.g. Wiki)

[th0ma7]

@hgy59 and @mreid-tt this is a little outside my normal area of work... I'm hoping with this to automate wheel updates. One area I'm unsure it will work ok is in the crossenv definition files as there is a build|cross|wheelhouse: prefix. Also it may require an exclude list such as pip (until fixed), Cython and msgpack...

Your 👀 on this would be appreciated, thnx.

EDIT: Exclude portion now in theory solved... as for the prefix, I'll need to further think about it.

[th0ma7]

@mreid-tt and @hgy59 you review on this would be much appreciated, thnx.

[mreid-tt]

@th0ma7 This is outside my area of expertise as well, but if I’m understanding correctly, you're reviewing the requirements for various Python versions and triggering a PR whenever updates are available? That seems a bit heavy; wouldn’t this potentially lead to users installing multiple new Python versions every month? Could you help me understand the benefit of this approach?

[th0ma7]

The idea is for the wheels requirements definitions for python to be auto updated. This in turn would test python to confirm it still builds ok. When we are ready to release a new version it becomes straight forward as the requirements have already been tested.

[mreid-tt]

The idea is for the wheels requirements definitions for python to be auto updated. This in turn would test python to confirm it still builds ok. When we are ready to release a new version it becomes straight forward as the requirements have already been tested.

Ah! Okay. That would include the requirements we have in the python/* directory? i.e. are we updating the python-wheels as well? I'm not sure updating Python alone is comprehensive enough, but then again doing everything would make for long build times.

[th0ma7]

It takes so much time and manipulations to fully update things that automating the wheel requirements process would simplify things greatly.

Personally I'd also automate python versions... But that could be for a later time.