[FLINK-37776][python] Drop python 3.8 support

[autophagy]

What is the purpose of the change

Python 3.8 is reached end-of-life on 2024-10-07. This PR drops support for Python 3.8, making PyFlink's minimum Python version 3.9.

Brief change log

• Dropped support for Python 3.8 in PyFlink.
• Corrected some out of date Python version dependency documentation.

Verifying this change

This can be verified with the existing Python tests.

Does this pull request potentially affect one of the following parts:

• Dependencies (does it add or upgrade a dependency): (no)
• The public API, i.e., is any changed class annotated with @Public(Evolving): (no)
• The serializers: (no)
• The runtime per-record code paths (performance sensitive): (no)
• Anything that affects deployment or recovery: JobManager (and its components), Checkpointing, Kubernetes/Yarn, ZooKeeper: (no)
• The S3 file system connector: (no)

Documentation

• Does this pull request introduce a new feature? (no)
• If yes, how is the feature documented? (not applicable)

[flinkbot]

CI report:

• 7642a4c Azure: SUCCESS

Bot commands

The @flinkbot bot supports the following commands:

• @flinkbot run azure re-run the last Azure build

[autophagy]

Just going to close this for now - worth a discussion on the mailing list around deprecating/dropping old version support.

[autophagy]

After some discussion on the mailing list, it seems like theres a decent consensus for just dropping 3.8 support, and not deprecating it first, since we're already 6 months out of EOL. Will reopen this PR.

[davidradl]

I am curious - why not 3.13? Should we not be recommending this level as it is the only one in active support? https://endoflife.date/python

[autophagy]

We do not currently support 3.13. Currently one of our main dependencies, pemja which is used for Python UDFs in threaded exec mode, only supports Python versions up to 3.12.

[davidradl]

Thanks - I see alibaba/pemja#50 is looking at supporting 3.13. I guess we wait for that.

[davidradl]

I suggest that we document that Python versions will no longer be supported by Flink when the security support ends for that Python level. Is https://endoflife.date/python the definitive link for this - if it is then we should include the link in the docs.

So in 5 months we will remove 3.9. The worry there is that 3.9 is the last release with Python 2 backward compatibility layers. Is there anything we can do to warn users that Python 2 programs will no longer work or be supported when we remove 3.9. Maybe some wording to encourage moving to Python 3 APIs.

[autophagy]

We will likely just deprecate 3.9 when it hits EOL, and give users time to migrate over before we remove it (likely after 6 months, like we are doing with Python 3.8).

It's a good point on the Python 2 deprecation - I'll make a discussion/ticket for encoding our deprecation/removal policy in the Python docs and mention this explicitly w/r/t 3.9.