[SPARK-55306][PYTHON][TESTS][FOLLOW-UP] Skip Kafka streaming RTM tests when dependencies are not installed

[zhengruifeng]

What changes were proposed in this pull request?

Skip StreamingKafkaTests in test_streaming_kafka_rtm.py when testcontainers or kafka-python is not installed, instead of raising an ImportError that crashes the entire test module.

Why are the changes needed?

The CI job pyspark-structured-streaming fails because test_streaming_kafka_rtm.py raises ImportError at module level when testcontainers is not installed. This causes the entire test runner to exit with a non-zero code, failing the CI build. See: https://github.com/apache/spark/actions/runs/24128422095/job/70399041605

Does this PR introduce any user-facing change?

No.

How was this patch tested?

The test class is now decorated with @unittest.skipIf(not have_kafka_deps, ...), which gracefully skips when the dependencies are missing — the same pattern used for Docker availability in the same file.

Was this patch authored or co-authored using generative AI tooling?

Generated-by: Claude Code

[viirya]

cc @jerrypeng

[HeartSaVioR]

#55266

Can we check whether this PR is doing the same work with above?

[viirya]

This causes the entire test runner to exit with a non-zero code, failing the CI build. See: https://github.com/apache/spark/actions/runs/24128422095/job/70399041605

The PR added the test code passed all CIs. Is the failed pipeline not running on the PR?

[zhengruifeng]

@HeartSaVioR just notice #55266, I think we don't need this PR now.

@viirya

Is the failed pipeline not running on the PR?

yes, it is failed on scheduled workflows.

[jerrypeng]

yes, it is failed on scheduled workflows.

@zhengruifeng why would it succeed for PRs but fail for scheduled workflows?

[zhengruifeng]

@jerrypeng because you didn't install the dependency in all scheduled workflow, e.g.

https://github.com/apache/spark/actions/workflows/build_python_3.12
https://github.com/apache/spark/actions/workflows/build_python_minimum.yml