`Illegal instruction` on arm64 Android, when the `wasi-http` response body is larger.

[Heap-Hop]

Test Case

Use the original bytecodealliance/sample-wasi-http-rust's /echo API is enough to reproduce.

Environment

Android arm64 device or emulator.
Latest release of wasmtime-v42.0.1-aarch64-android.tar.xz and bytecodealliance/sample-wasi-http-rust.

Steps to Reproduce

adb shell mkdir -p /data/local/tmp
adb push wasmtime sample-wasi-http-rust.wasm /data/local/tmp
adb forward tcp:8080 tcp:8080
adb shell HOME=/data/local/tmp \
  /data/local/tmp/wasmtime serve \
  -Scli -Shttp \
  /data/local/tmp/sample-wasi-http-rust.wasm

Open a new terminal:

curl -d "$(printf 'data%.0s' {1..5000})"  http://127.0.0.1:8080/echo > /dev/null

output:

  % Total    % Received % Xferd  Average Speed   Time    Time     Time  Current
                                 Dload  Upload   Total   Spent    Left  Speed
100 28038    0  8038  100 20000   197k   491k --:--:-- --:--:-- --:--:--  702k
curl: (18) transfer closed with outstanding read data remaining

And the wasmtime process crashed and showed:

Serving HTTP on http://0.0.0.0:8080/
Illegal instruction 

android-wasi-sigill.mov

Also tried on Termux, same results.

Expected Results

wasmtime process should not exit

Actual Results

Illegal instruction

Versions and Environment

Wasmtime version or commit: wasmtime-v42.0.1-aarch64-android

Operating system: Android 16

Architecture: arm64

Extra Info

Maybe related to #11295

[Heap-Hop]

lldb:

(lldb) c
Process 8704 resuming
Process 8704 stopped
* thread #2, name = 'tokio-runtime-w', stop reason = signal SIGILL: illegal operand
    frame #0: 0x00000059d5fa3c30 wasmtime`wasmtime_internal_fiber::stackswitch::aarch64::wasmtime_fiber_switch_::h5b6d4250d8a8bee0 + 100
wasmtime`wasmtime_internal_fiber::stackswitch::aarch64::wasmtime_fiber_switch_::h5b6d4250d8a8bee0:
->  0x59d5fa3c30 <+100>: autiasp 
    0x59d5fa3c34 <+104>: ret    

wasmtime`wasmtime_internal_fiber::FiberStack::from_custom::h1d78d71118b22fbd:
    0x59d5fa3c38 <+0>:   sub    sp, sp, #0x50
    0x59d5fa3c3c <+4>:   stp    x29, x30, [sp, #0x30]
Target 0: (wasmtime) stopped.

[Heap-Hop]

Just tried through Copilot, GPT-5.4.
Heap-Hop@c5245c1

Using the artifact bins-aarch64-android from Artifacts.
It works fine now.

This is the summary by AI:
(I'm not an expert of this area, please review these changes)

---

Pointer Authentication (PAuth) and Android SIGILL crash

Background

Wasmtime uses a small assembly stub to perform stack switching for lightweight fibers.
On AArch64 this stub historically emitted the ARM Pointer Authentication (PAuth) instructions
paciasp / autiasp to protect return addresses as a form of control-flow integrity.

On many Android arm64 devices the CPU does not support the Pointer Authentication extension.
In that case, executing autiasp causes a fatal SIGILL (Illegal Instruction) and crashes the Wasmtime process.

This is the issue reported in:

• Illegal instruction on arm64 Android, when the wasi-http response body is larger. #12778
• pointer authentication failed on arm64 android #11295

What changed

The fiber stack-switch implementation in crates/fiber/src/stackswitch/aarch64.rs
now only emits the PAuth instructions if the Rust compiler is configured with one of the
PAuth target features:

• +paca (Armv8.3-A Pointer Authentication)
• +pacg (Armv8.3-A Pointer Authentication)

When neither feature is enabled (the default for aarch64-linux-android), the generated
code does not include paciasp / autiasp, avoiding the SIGILL crash.

Security implications

✅ Safe for devices without PAuth support

If the CPU does not support PAuth, the CPU cannot execute the instructions anyway.
This change prevents Wasmtime from crashing on such devices, which is a correctness fix.

🔐 CFI is preserved when PAuth is enabled

If a build enables -C target-feature=+paca / +pacg, Wasmtime will still emit the
PAuth instructions and gain the same control-flow integrity protection as before.

In other words:

• No PAuth target feature → No PAuth instructions → No crash (but no CFI)
• PAuth target feature enabled → PAuth instructions emitted → CFI is intact

Relevant files

• crates/fiber/src/stackswitch/aarch64.rs (the assembly stub that was modified)

[Heap-Hop]

Heap-Hop@c5245c1 causes a new failure:
https://github.com/Heap-Hop/wasmtime/actions/runs/23087440874/job/67066363560#step:22:849
This might not be a complete fix yet.

[Heap-Hop]

https://github.com/Heap-Hop/wasmtime/actions/runs/23099865150
CI passed and it works fine on Android.

The changes were suggested by AI, and I'm not very familiar with this area, so they definitely need review.

Should I open a PR for this, or would someone else like to start one based on this attempt?

[cfallin]

Thanks for filing the issue. Looking at the diff you liked, the fundamental bug appears to be that we don't conditionalize use of pointer-authentication instructions on target features, as you say. Cranelift-compiled code will work fine because we have cranelift-native that does feature detection; but the fiber-switch routines are hardcoded.

That said, the diff has kind of made a mess of the code -- it removes the cfg_if entirely and messes up the logic. Could you (human, not LLM) refactor the patch so that it retains the cfg_if, and splits the else branch into else if (pac features) { existing non-Apple defs } else { empty defs }? I think we'd be happy to review a patch with that. Thanks!

[alexcrichton]

This might also be something we need an expert/someone knowledgable about Android to weigh-in. My impression was that the pointer authentication instructions are noops on CPUs that don't support them, which means that I don't know what the issue is here. What pointer authentication is expected to happen on Android?

Basically I'm not sure why we'd want a duplicate path without pointer authentication. I'd expect the pointer-authentication bits to be self-contained in this file where if the external systems uses them then it respects the ABI, but if the external system has pointer authentication disabled then this would be a bunch of noops.

[cfallin]

Basically I'm not sure why we'd want a duplicate path without pointer authentication. I'd expect the pointer-authentication bits to be self-contained in this file where if the external systems uses them then it respects the ABI, but if the external system has pointer authentication disabled then this would be a bunch of noops.

My understanding is that it's a little more subtle than that: the instructions may be present on the CPU (it's part of ARMv8.3+) but the OS might not configure the secret keys, etc. That configuration is part of the target triple aka Rust target features. So we really do need the handwritten assembly to follow the same conventions that rustc itself follows in codegen'ing the runtime (for example), which means we conditionally emit the instructions.

[cfallin]

(To confirm why Cranelift-generated code gets this right, in cranelift-native we have a check for the target feature paca which configures use of the feature.)

[cfallin]

(Or, actually, I'm not sure if that tests the CPU-level feature or a feature vector provided by the OS; I don't have an Android device handy to test. In any case it seems like having our inline assembly follow the same prologue/epilogue conventions as the rest of the runtime seems sensible.)

[alexcrichton]

Do you know where exactly the illegal instruction here is coming from? Can these instructions trap if the OS has them disabled? Are we using the wrong key since it was detected wrong? Are we in a situation where the fiber code is doing pointer authentication, the rest of the system isn't, and that causes issues?

[cfallin]

In #11295 there's a debugger trace showing the fault on autiasp (authenticate return address before return). These instructions can certainly trap (that's their purpose!) -- the OS doesn't "disable" them, it fails to set up the kernel state that configures the secret keys used for pointer signing.

Are we in a situation where the fiber code is doing pointer authentication, the rest of the system isn't, and that causes issues?

Yes, exactly. The target triple appears not to have the paca feature, which means that rustc codegens the rest of the runtime (and all of userspace is compiled) without these instructions. The path I suggested above is to make our inline assembly match what rustc codegens for any other function.

[alexcrichton]

What I don't understand is that the fiber code should work if pointer authentication is supported/implemented (aka not a nop) and the rest of the system isn't using it (e.g. Rust compiled without it). My understanding is that the inline asm we have encodes lr at the start of the function, and then decodes it at the end of the function (e.g. paciasp at the start, autiasp at the end). The trapping instruction in #11295 (and this issue) is autiasp, but the input to autiasp is, I thought, entirely controlled by wasmtime-fiber. Given that I don't understand why something needs to be conditional, but I must be missing something...