A problem with spinlocks' interlocking

[jpszczolowski]

Usually context switching while holding a lock leaves the lock acquired (and owned). And that's an intended behavior. But spinlocks (meaning either spinning or disabling interrupts) are different -- if a thread acquires a spinlock and then voluntarily gives up the processor, we would like the kernel to automatically release the spinlock and re-acquire it when the thread is put onto the processor again.

This is in fact already implemented but only for each thread's td_spin spinlock (inside mips/switch.S).

Let's look at an example (based on #550 problem) explaining why we need such behavior. Let's say there are two threads:

• thread 1 inserting elements into spinlock-guarded tail queue:

WITH_SPIN_LOCK (&spinlock) {
  TAILQ_INSERT_TAIL(&tailq, elem, elem_link);
  sleepq_signal(wchan);
}

• thread 2 taking elements from spinlock-guarded tail queue:

WITH_SPIN_LOCK (&spinlock) {
  while (TAILQ_EMPTY(&tailq))
    sleepq_wait(wchan, NULL);
  elem = TAILQ_FIRST(&tailq);
  ...
}

And now consider the following sequence of actions that cause kernel failure:

• tail queue is empty,
• thread 2 acquires the spinlock,
• thread 2 "goes to sleep" using sleepq_wait,
• thread 1 wants to add an element to the tail queue,
• thread 1 tries to acquire the spinlock but fails in function spin_lock (assert that the spinlock's s_owner is NULL fails because thread 2 is still the owner).