Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Deadlock in Subscriber impl of Drop #1515

Open
ardouglas opened this issue Dec 23, 2024 · 0 comments
Open

Deadlock in Subscriber impl of Drop #1515

ardouglas opened this issue Dec 23, 2024 · 0 comments
Labels

Comments

@ardouglas
Copy link

Hello, I encountered a deadlock while toying around with the Subscriber API. The scenario is as follows:

One thread performing inserts on a Tree.

One or more threads subscribed to updates on the Tree and forwarding those events to another thread over a channel. If the subscriber's end of the channel is closed, I break out of the subscriber loop and fall through to the end of the function (or so I thought).

What I'm actually seeing is that the impl of Drop gets deadlocked on the call to self.home.write(). To test this I added some prints to the drop method like:

impl Drop for Subscriber {
    fn drop(&mut self) {
        println!("entering drop for subscriber");
        let mut w_senders = self.home.write();
        w_senders.remove(&self.id);
        println!("exiting drop for subscriber");
    }
}

When I have two subscribers I will often see one of them make it through both printlns, but the other will not. Furthermore, the thread performing the inserts is no longer able to make forward progress in this scenario.

I found two ways around this:

  1. force the subscribers to continue reading until the inserts are complete
  2. edit the impl of Drop and drop the rx field of Subscriber prior to calling self.home.write() like:
impl Drop for Subscriber {
    fn drop(&mut self) {
        let (_tx, mut rx) = sync_channel(1);
        std::mem::swap(&mut rx, &mut self.rx);
        drop(rx);
        let mut w_senders = self.home.write();
        w_senders.remove(&self.id);
    }
}

The above code does seem to alleviate the deadlock though I'm not sure if there might be a better way to remove the cycle between the two fields.

Expected Result

Dropping a Subscriber does not impact writing threads

Actual Result

Dropping a Subscriber deadlocks the subscriber and the writer threads

Sled Version

0.34.7

Rustc Version

1.83.0 (90b35a623 2024-11-26)

Operating System

Rocky Linux 8.5

Minimal Code Sample

I don't immediately have one I can submit but if you're willing to accept a PR I will try to make a minimal test case.

logs, panic messages, stack traces

There are none

@ardouglas ardouglas added the bug label Dec 23, 2024
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
Projects
None yet
Development

No branches or pull requests

1 participant