[Data][CI] Add lint to ensure tests actually run

[bveeramani]

Why are these changes needed?

Our CI runs tests with Bazel. Rather than running the tests with pytest, Bazel's executes the test Python files directly. So, to ensure that the tests actually run, we need to invoke pytest explicitly in our test files like this:

if __name__ == "__main__":
    import sys

    sys.exit(pytest.main(["-v", __file__]))

This PR adds a lint to enforce this.

Related issue number

Checks

• I've signed off every commit(by using the -s flag, i.e., git commit -s) in this PR.
• I've run pre-commit jobs to lint the changes in this PR. (pre-commit setup)
• I've included any doc changes needed for https://docs.ray.io/en/master/.
  • I've added any new APIs to the API Reference. For example, if I added a
    method in Tune, I've added it in doc/source/tune/api/ under the
    corresponding .rst file.
• I've made sure the tests are passing. Note that there might be a few flaky tests, see the recent failures at https://flakey-tests.ray.io/
• Testing Strategy
  • Unit tests
  • Release tests
  • This PR is not tested :(

[gemini-code-assist]

Code Review

This pull request introduces a valuable linting check to ensure all tests in ray.data are executable, which is a great improvement for CI reliability. The changes correctly add the necessary pytest.main call to several test files. My review includes a few minor suggestions to improve code style consistency in the newly added code snippets by grouping imports, following PEP 8 guidelines. This also includes updating the semgrep rule's suggested snippet to reflect this style.

[bveeramani]

Needed to upgrade the version to make this work properly

[cursor]

Bug: Test File Commit Causes CI and Lint Failures

A temporary test file was accidentally committed. It contains an assert False that will cause CI failures, and it's missing the pytest.main() invocation, which violates a new lint rule and will also cause lint failures.

 

[aslonnie]

I think we already have a lint for this?
https://github.com/ray-project/ray/blob/master/ci/lint/check-pytest-format.sh

[aslonnie]

leaving this for @elliot-barn to review. hail lord of lints!

I think we already have a lint for that, if we want to migrate to use semgrep, maybe remove the old one?

[bveeramani]

leaving this for @elliot-barn to review. hail lord of lints!

I think we already have a lint for that, if we want to migrate to use semgrep, maybe remove the old one?

@aslonnie do you remember what the old one is called? Don't think it's working properly, because there are several Data tests that aren't actually run

[aslonnie]

ray/ci/lint/lint.sh

Line 96 in cef177f

pytest_format() {

https://buildkite.com/ray-project/microcheck/builds/28197#0199c9f9-72b5-4352-842a-b068bd23b377

[elliot-barn]

https://github.com/ray-project/ray/blob/master/ci/lint/check-pytest-format.sh

Lets use semgrep and get rid of the pytest_checker script? Ill open a PR with these changes and pre-commit checks

[bveeramani]

Closing in favor of #57617