Skip to content

Conversation

Copy link
Contributor

Copilot AI commented Jan 27, 2026

On macOS, FSEvents throws RuntimeError: Cannot add watch when multiple threads/processes attempt to monitor the same cache directory concurrently. This occurs in wait_on_entry_calc() when multiple callers create observers for the same path.

Changes

Error Handling in src/cachier/cores/pickle.py:

  • Added RuntimeError catch for "Cannot add watch" in wait_on_entry_calc()
  • Falls back to _wait_with_polling() when FSEvents rejects the watch
  • Mirrors existing inotify limit handling pattern
  • Other RuntimeErrors continue to propagate

Test Coverage in tests/test_pickle_core.py:

  • test_wait_on_entry_calc_runtime_error_watch_scheduled - verifies polling fallback
  • test_wait_on_entry_calc_other_runtime_error - ensures other RuntimeErrors propagate

Example Scenario

# Multiple threads calling the same cached function concurrently
@cachier(backend='pickle')
def expensive_computation(x):
    return x * 2

# Previously: RuntimeError on macOS when threads wait concurrently
# Now: Gracefully falls back to polling for affected threads

The fix provides graceful degradation on macOS FSEvents while maintaining full functionality through the existing polling mechanism.

Original prompt

This section details on the original issue you should resolve

<issue_title>RuntimeError: Cannot add watch, already scheduled</issue_title>
<issue_description>It seems that when running cachier, I randomly get these errors:

2021-09-17 09:05:32,357 - fsevents - ERROR - Unhandled exception in FSEventsEmitter
Traceback (most recent call last):
  File "/Users/lorenz/.../signals/.venv/lib/python3.9/site-packages/watchdog/observers/fsevents.py", line 315, in run
    _fsevents.add_watch(self, self.watch, self.events_callback, self.pathnames)
RuntimeError: Cannot add watch <ObservedWatch: path=/Users/lorenz/.cachier/, is_recursive=True> - it is already scheduled

My current solution is to delete the cache, assuming it is corrupt, and then start from scratch.

The most similar error I found on the net was the one reported in spyder-ide/spyder#14803. There, the solution was to unwatch on switching directories/projects.</issue_description>

Comments on the Issue (you are @copilot in this section)

@shaypal5 Hey,

That's look very relevant! Thank you for reaching out.

Anyway to produce the full stack trace, pointing to a code line in cachier itself?
Or is this all you're getting? 

It sounds like this will be hard to reproduce to test correctly.</comment_new>
<comment_new>@louismartin
I get that error with my flask server from time to time, I don't have more details:

ERROR:fsevents:Unhandled exception in FSEventsEmitter
Traceback (most recent call last):
  File "/Users/louismartin/miniconda3/envs/dev3.9/lib/python3.9/site-packages/watchdog/observers/fsevents.py", line 315, in run
    _fsevents.add_watch(self, self.watch, self.events_callback, self.pathnames)
RuntimeError: Cannot add watch <ObservedWatch: path=/Users/louismartin/.cachier/, is_recursive=True> - it is already scheduled

(On macOS, python 3.9)</comment_new>
<comment_new>@shaypal5
Thank you both for chiming in, and @GianlucaFicarelli especially for the very detailed way to reproduce this!

This dramatically increases the chance that a past active contributor or a new one will take this on and try to solve this bug.</comment_new>


💡 You can make Copilot smarter by setting up custom instructions, customizing its development environment and configuring Model Context Protocol (MCP) servers. Learn more Copilot coding agent tips in the docs.

- Catch RuntimeError when observer.schedule() fails with "Cannot add watch"
- Fall back to polling mode when this error occurs (common on macOS FSEvents)
- Add tests for RuntimeError handling in wait_on_entry_calc
- Ensures graceful degradation when multiple observers watch same path

Co-authored-by: Borda <6035284+Borda@users.noreply.github.com>
Copilot AI changed the title [WIP] Fix RuntimeError for already scheduled watch in cachier Fix FSEvents RuntimeError on concurrent cache access Jan 27, 2026
Copilot AI requested a review from Borda January 27, 2026 08:06
@codecov
Copy link

codecov bot commented Jan 27, 2026

Codecov Report

✅ All modified and coverable lines are covered by tests.
✅ Project coverage is 98.41%. Comparing base (6092dfb) to head (7b4cf09).
⚠️ Report is 1 commits behind head on master.

Additional details and impacted files

Impacted file tree graph

@@           Coverage Diff           @@
##           master     #321   +/-   ##
=======================================
  Coverage   98.40%   98.41%           
=======================================
  Files          11       11           
  Lines        1005     1010    +5     
  Branches      119      120    +1     
=======================================
+ Hits          989      994    +5     
  Misses          9        9           
  Partials        7        7           
Flag Coverage Δ
local 73.76% <100.00%> (+0.13%) ⬆️
mongodb 45.84% <0.00%> (-0.23%) ⬇️
postgres 45.54% <0.00%> (-0.23%) ⬇️
redis 51.48% <0.00%> (-0.26%) ⬇️

Flags with carried forward coverage won't be shown. Click here to find out more.

Files with missing lines Coverage Δ
src/cachier/cores/pickle.py 99.16% <100.00%> (+0.01%) ⬆️

Continue to review full report in Codecov by Sentry.

Legend - Click here to learn more
Δ = absolute <relative> (impact), ø = not affected, ? = missing data
Powered by Codecov. Last update 6092dfb...7b4cf09. Read the comment docs.

🚀 New features to boost your workflow:
  • ❄️ Test Analytics: Detect flaky tests, report on failures, and find test suite problems.

@Borda Borda marked this pull request as ready for review January 27, 2026 08:53
@Borda Borda requested a review from shaypal5 as a code owner January 27, 2026 08:53
Copilot AI review requested due to automatic review settings January 27, 2026 08:53
Copy link
Contributor

Copilot AI left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Pull request overview

This PR fixes a RuntimeError that occurs on macOS when FSEvents is unable to add a watch due to concurrent cache directory monitoring. The fix provides graceful degradation by falling back to polling when the FSEvents observer cannot be scheduled.

Changes:

  • Added RuntimeError exception handling in wait_on_entry_calc() to catch "Cannot add watch" errors from FSEvents
  • Falls back to polling mechanism when FSEvents rejects the watch, mirroring the existing inotify limit handling pattern
  • Added comprehensive test coverage for both the fallback scenario and proper re-raising of unrelated RuntimeErrors

Reviewed changes

Copilot reviewed 2 out of 2 changed files in this pull request and generated no comments.

File Description
src/cachier/cores/pickle.py Added RuntimeError catch block with string matching for FSEvents "Cannot add watch" error, includes debug logging and fallback to polling
tests/test_pickle_core.py Added two test cases: one verifying polling fallback on FSEvents error, one ensuring other RuntimeErrors propagate correctly

@shaypal5 shaypal5 merged commit cccded0 into master Jan 27, 2026
50 checks passed
@shaypal5 shaypal5 deleted the copilot/fix-runtimeerror-watch-scheduled branch January 27, 2026 09:20
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

RuntimeError: Cannot add watch, already scheduled

3 participants