Skip to content

Conversation

bveeramani
Copy link
Member

Why are these changes needed?

test_hanging_detector_detects_issues checks that Ray Data emits a log if one task takes a lot longer than the others. The issue is that the test doesn't capture the log output correctly, and so the test fails even though Ray data correctly emits the log.

To make this test more robust, this PR uses pytest's caplog fixture to capture the logs rather than a bespoke custom handler.

[2025-10-08T09:00:41Z] >           assert hanging_detected, log_output
  | [2025-10-08T09:00:41Z] E           AssertionError:
  | [2025-10-08T09:00:41Z] E           assert False
  | [2025-10-08T09:00:41Z]
  | [2025-10-08T09:00:41Z] python/ray/data/tests/test_issue_detection_manager.py:153: AssertionError
  |  

Related issue number

Checks

  • I've signed off every commit(by using the -s flag, i.e., git commit -s) in this PR.
  • I've run pre-commit jobs to lint the changes in this PR. (pre-commit setup)
  • I've included any doc changes needed for https://docs.ray.io/en/master/.
    • I've added any new APIs to the API Reference. For example, if I added a
      method in Tune, I've added it in doc/source/tune/api/ under the
      corresponding .rst file.
  • I've made sure the tests are passing. Note that there might be a few flaky tests, see the recent failures at https://flakey-tests.ray.io/
  • Testing Strategy
    • Unit tests
    • Release tests
    • This PR is not tested :(

Signed-off-by: Balaji Veeramani <[email protected]>
@bveeramani bveeramani requested a review from a team as a code owner October 8, 2025 20:05
Copy link
Contributor

@gemini-code-assist gemini-code-assist bot left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Code Review

This pull request significantly improves the robustness of test_hanging_detector_detects_issues by replacing a complex, custom log-capturing mechanism with the standard pytest caplog fixture. This change simplifies the test, makes it easier to understand, and less prone to errors. The addition of the restore_data_context fixture is also a great improvement for test isolation.

I have one suggestion to make the test assertion even more specific and robust.

Copy link
Contributor

@omatthew98 omatthew98 left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Thanks!

@aslonnie aslonnie added the go add ONLY when ready to merge, run all tests label Oct 8, 2025
@ray-gardener ray-gardener bot added the data Ray Data-related issues label Oct 9, 2025
Signed-off-by: Balaji Veeramani <[email protected]>
This reverts commit c338363.

Signed-off-by: Balaji Veeramani <[email protected]>
@bveeramani bveeramani merged commit 070820e into master Oct 10, 2025
6 checks passed
@bveeramani bveeramani deleted the refacot-issue-detection branch October 10, 2025 22:29
joshkodi pushed a commit to joshkodi/ray that referenced this pull request Oct 13, 2025
…roject#57567)

<!-- Thank you for your contribution! Please review
https://github.com/ray-project/ray/blob/master/CONTRIBUTING.rst before
opening a pull request. -->

<!-- Please add a reviewer to the assignee section when you create a PR.
If you don't have the access to it, we will shortly find a reviewer and
assign them to your PR. -->

## Why are these changes needed?

<!-- Please give a short summary of the change and the problem this
solves. -->

`test_hanging_detector_detects_issues` checks that Ray Data emits a log
if one task takes a lot longer than the others. The issue is that the
test doesn't capture the log output correctly, and so the test fails
even though Ray data correctly emits the log.

To make this test more robust, this PR uses pytest's `caplog` fixture to
capture the logs rather than a bespoke custom handler.

```
[2025-10-08T09:00:41Z] >           assert hanging_detected, log_output
  | [2025-10-08T09:00:41Z] E           AssertionError:
  | [2025-10-08T09:00:41Z] E           assert False
  | [2025-10-08T09:00:41Z]
  | [2025-10-08T09:00:41Z] python/ray/data/tests/test_issue_detection_manager.py:153: AssertionError
  |  
```

## Related issue number

<!-- For example: "Closes ray-project#1234" -->

## Checks

- [ ] I've signed off every commit(by using the -s flag, i.e., `git
commit -s`) in this PR.
- [ ] I've run pre-commit jobs to lint the changes in this PR.
([pre-commit
setup](https://docs.ray.io/en/latest/ray-contribute/getting-involved.html#lint-and-formatting))
- [ ] I've included any doc changes needed for
https://docs.ray.io/en/master/.
- [ ] I've added any new APIs to the API Reference. For example, if I
added a
method in Tune, I've added it in `doc/source/tune/api/` under the
           corresponding `.rst` file.
- [ ] I've made sure the tests are passing. Note that there might be a
few flaky tests, see the recent failures at https://flakey-tests.ray.io/
- Testing Strategy
   - [ ] Unit tests
   - [ ] Release tests
   - [ ] This PR is not tested :(

---------

Signed-off-by: Balaji Veeramani <[email protected]>
Signed-off-by: Josh Kodi <[email protected]>
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

data Ray Data-related issues go add ONLY when ready to merge, run all tests

Projects

None yet

Development

Successfully merging this pull request may close these issues.

4 participants