Skip to content

litebox_shim_linux: fix race in prepare_for_exit child TID write - #1155

Open
Leon Schuermann (lschuermann) wants to merge 1 commit into
microsoft:mainfrom
lschuermann:dev/litebox-shim-linux-thread-detach-race
Open

litebox_shim_linux: fix race in prepare_for_exit child TID write#1155
Leon Schuermann (lschuermann) wants to merge 1 commit into
microsoft:mainfrom
lschuermann:dev/litebox-shim-linux-thread-detach-race

Conversation

@lschuermann

@lschuermann Leon Schuermann (lschuermann) commented Aug 10, 2026

Copy link
Copy Markdown
Contributor

This fixes a bug in prepare_for_exit in the LiteBox Linux shim. When a thread exits, it may indicate this by writing its child TID to a location in user memory. However, prior to this, detach_from_process is run, which decrements nr_threads. This is a signal for other tasks that this task no longer holds any references to process memory, and thus is ready to be deallocated. In the case of, e.g., an exec system call, this in turn introduces a "confused deputy" style write where the LiteBox kernel will either attempt to write to some unmapped pages (caught by the exception handlers and silently turned into a no-op), or writes into and corrupts the new process' memory.

The Race Condition

#Thread A (main / exec'ing)Thread B (worker / dying)
1 pthread_create(), passing CLONE_CHILD_CLEARTID = &pd->tid
2 starts running
  • clear_child_tid = address inside B's stack
  • nr_threads increment to 2
3 recv(fd, buf, ...), blocks
4 execve("/new/program") blocked
5 kill_other_threads()
  • sets is_exiting, interrupts B
  • waits for nr_threads == 1
blocked
6 waiting recv returns EINTR
  • drops Task, drops ThreadState, runs prepare_for_exit()
7 waiting detach_from_process()
  • nr_threads 2 → 1, wakes A
  • declared done; guest write still pending
8 kill_other_threads() returns
  • process assumed quiesced
9 descheduled
  • still owes write to clear_child_tid
10 release_memory(), UNMAP
  • destroys all old mappings, incl. B's stack
  • ranges immediately reusable, no quarantine
descheduled
11 load_program(), REMAP
  • new ELF / stack / heap from same pool
  • may land on the freed range
descheduled
12 new program runs clear_child_tid.write_at_offset(0, 0)
  • confused-deputy write into new image
13 new program runs sys_futex(Wake), wake_robust_list()
  • more accesses via old-image addresses

The Fix

Don't decrement the nr_threads counter prior to completing any writes to that process' address space.

memcpy_fallible provides fault safety, but not lifetime safety. I used Claude Opus 5 inspect the codebase for this class of bug, and help me verify that it can be triggered, as well as validate the fix.

This fixes a bug in `prepare_for_exit` in the LiteBox Linux shim. When a
thread exits, it may indicate this by writing its child TID to a
location in user memory. However, prior to this, `detach_from_process`
is run, which decrements `nr_threads`. This is a signal to other tasks
that this task no longer holds any references to process memory, and
thus that the memory is ready to be deallocated.

In the case of, e.g., an `exec` system call, this in turn introduces a
"confused deputy" style write where the LiteBox kernel will either
attempt to write to some unmapped page (caught by the exception handlers
and silently turned into a no-op), or write into and corrupt the new
process' memory.

The Race Condition
------------------

+----+------------------------------+------------------------------+
| #  | A (main / exec-ing thread)   | B (worker / dying thread)    |
+----+------------------------------+------------------------------+
| 1  | pthread_create(), passing    |                              |
|    | CLONE_CHILD_CLEARTID =       |                              |
|    | &pd->tid                     |                              |
+----+------------------------------+------------------------------+
| 2  |                              | starts running               |
|    |                              | - clear_child_tid = addr     |
|    |                              |   inside B's stack           |
|    |                              | - nr_threads -> 2            |
+----+------------------------------+------------------------------+
| 3  |                              | recv(fd, buf, ...), blocks   |
+----+------------------------------+------------------------------+
| 4  | execve("/new/program")       | blocked                      |
+----+------------------------------+------------------------------+
| 5  | kill_other_threads()         | blocked                      |
|    | - sets is_exiting,           |                              |
|    |   interrupts B               |                              |
|    | - waits nr_threads == 1      |                              |
+----+------------------------------+------------------------------+
| 6  | waiting                      | recv returns EINTR           |
|    |                              | - drops Task ->              |
|    |                              |   prepare_for_exit()         |
+----+------------------------------+------------------------------+
| 7  | waiting                      | detach_from_process()        |
|    |                              | - nr_threads 2 -> 1,         |
|    |                              |   wakes A                    |
|    |                              | - DECLARED DONE, but         |
|    |                              |   guest write pending        |
+----+------------------------------+------------------------------+
| 8  | kill_other_threads()         |                              |
|    | RETURNS                      |                              |
|    | - process assumed            |                              |
|    |   quiesced                   |                              |
+----+------------------------------+------------------------------+
| 9  |                              | descheduled                  |
|    |                              | - still owes write to        |
|    |                              |   clear_child_tid            |
+----+------------------------------+------------------------------+
| 10 | release_memory() UNMAP       | descheduled                  |
|    | - destroys all old           |                              |
|    |   mappings, incl. B's        |                              |
|    |   stack                      |                              |
|    | - ranges immediately         |                              |
|    |   reusable, no               |                              |
|    |   quarantine                 |                              |
+----+------------------------------+------------------------------+
| 11 | load_program() REMAP         | descheduled                  |
|    | - new ELF/stack/heap         |                              |
|    |   from same pool             |                              |
|    | - may land on the            |                              |
|    |   freed range                |                              |
+----+------------------------------+------------------------------+
| 12 | new program runs             | clear_child_tid              |
|    |                              | .write_at_offset(0, 0)       |
|    |                              | - confused-deputy write      |
|    |                              |   into new image             |
+----+------------------------------+------------------------------+
| 13 | new program runs             | sys_futex(Wake),             |
|    |                              | wake_robust_list()           |
|    |                              | - more accesses via          |
|    |                              |   old-image addresses        |
+----+------------------------------+------------------------------+

The Fix
-------

Don't decrement the `nr_threads` counter prior to completing any writes
to that process' address space.

`memcpy_fallible` provides fault safety, but not lifetime safety.

I used Claude Opus 5 to inspect the codebase for this class of bug, help
me verify that it can be triggered, and validate the fix.
@lschuermann
Leon Schuermann (lschuermann) force-pushed the dev/litebox-shim-linux-thread-detach-race branch from ef98547 to 93fa9eb Compare August 10, 2026 18:23
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

1 participant