ffi: exercise authentic generic scalar CALLXS

Author: Frityet

notes/ffi-authentic-generic-callxs-2026-07-18.md

@@ -0,0 +1,154 @@
+# Authentic generic scalar CALLXS checkpoint (2026-07-18)
+
+## Status and scope
+
+This checkpoint executes the real generic x64 `IR_CALLXS` seam between the
+XSAVE-consuming exact-frame entry/leave helpers. Activation is deliberately
+test-only under `LJ_FFI_CALLXS_TEST_ACTIVATE`; the default recorder still raises
+`LJ_TRERR_BLACKL`. No `plan/` file is changed.
+
+The path is declaration-independent. It adds no signature enum, C declaration
+matcher, wrapper catalogue, or per-shape dispatcher. CType-driven argument
+conversion and the existing x64 ABI classifier continue to build one generic
+`CARG` tree and lower one `CALLXS`.
+
+## Generated lifecycle
+
+For the currently admitted nonallocating scalar result-handoff classes,
+recording builds every CType argument conversion before the native boundary,
+then emits:
+
+```text
+XSAVE
+CALLS lj_ffi_native_trace_enter(L, exact_trace, raw_function)
+guard enter != 0                 -- pre-call/replaying snapshot
+CALLXS generic_CARG_tree
+pure result normalization
+caller-state snapshot
+CALLS lj_ffi_native_trace_leave(L)  -- first C call after the foreign call
+guard leave == 0                 -- post-call/non-replaying snapshot
+```
+
+The admitted result normalizations are nonthrowing machine conversions and do
+not alter the foreign error pair. Leave remains the first C call after the
+foreign return. Its `CCI_T` unwind and its forced guard now share the caller
+snapshot because that snapshot's IR reference names leave itself. The enter and
+leave IRCALL descriptors intentionally do not claim preserved FP registers;
+normal register allocation must move or spill the normalized result across
+leave. The authentic matrix proves both GPR and XMM result survival.
+
+Both native-boundary guard snapshots carry `SNAPCOUNT_DONE`, used here as an
+explicit never-record marker. LOOP copy-substitution preserves this marker.
+The pre-call rejection stays in the interpreter, while the completed-call exit
+must pass through central trace-exit cleanup; linking either exit to a side
+trace could otherwise cross or bypass native lifecycle ownership.
+
+The opt-in boundary currently accepts ordinary `CALL`, `CALLM`, and `ITERC`
+Lua frames only. Bool, pointer, enum, i64 and u64 results stay interpreted:
+their current result paths branch or allocate cdata after the foreign side
+effect. Protected, continuation, vararg-frame and tail-return shapes also stay
+interpreted until each has an exact post-return snapshot contract. These are
+return/frame-class safety boundaries, not declaration matching.
+
+## Exact trace constant on loop traces
+
+The first authentic loop showed that KTRACE patching in `asm_tail_link()` was
+not sufficient: normal loop traces skip tail linking, so their placeholder
+remained null and native entry rejected every execution. That looked correct at
+the Lua level because the pre-call exit replayed the call in the interpreter.
+
+KTRACE is now patched immediately after each assembly attempt selects the
+copied `J->curfinal->ir`. This covers loop and non-loop traces plus IR-growth,
+alignment and RENAME retries which allocate a fresh final body. The generated
+fixture inspects the enter CARG and proves its KGC is the exact finalized body.
+
+## Caller-state post-call snapshot
+
+The first forced leave exit exposed a real continuation bug. `lj_record_ret()`
+had already moved `J->base`, `baseslot`, `framedepth` and result slots back to
+the Lua caller, but `L->base/top` and `J->pc/fn/pt` still described the live
+fast-function retry. The snapshot therefore combined caller values with the
+synthetic `BC_FUNCC` PC. Its side trace started at source line zero and restore
+resumed `lj_BC_FUNCC` with a Lua closure where a C closure was expected,
+jumping through a Lua upvalue word into non-executable GC memory.
+
+`ffrecord_postcall_snap()` now presents the matching Lua caller view only while
+the snapshot is built. It does not shift the recorder stack a second time. It
+validates the caller PC against the caller prototype, snapshots with caller
+`L->base/top` and `J->pc/fn/pt`, and then restores the still-live fast-function
+view. Terminal trace links and unsupported physical frame shapes fail recording
+instead of appending a guard to an invalid continuation.
+
+For its i32 side-effect target, the C fixture mechanically verifies that every
+activated CALLXS is immediately followed by native leave, every leave guard
+owns a snapshot inside the expected Lua prototype, and the root snapshot's
+frame/top encoding matches that prototype. It then forces a handshake-epoch
+change while the final frame sequence is odd. The real guard restores the
+caller, releases the retained pin once, preserves the exact foreign side-effect
+count, and leaves the root safely re-enterable.
+
+## One-shot XSAVE ownership
+
+Every native-entry attempt consumes all three XSAVE staging words. Earlier
+rejected entries left them intact for the pre-call snapshot exit, but a hot
+exit can link directly to side code and bypass central trace-exit cleanup.
+Authentic occupied-depth injection exposed the stale raw
+`ffi_xsave_root/baseslot/nslots` geometry after interpreter fallback.
+
+`lj_ffi_native_trace_enter()` now performs the complementary owner clear on
+every valid-carrier return, including capacity, occupied-depth, malformed
+geometry, trace-pin and publication rejection. This keeps dormant XSAVE-only
+instrumentation independent of unrelated trace exits. The injected rejection
+proves that CALLXS and generated leave do not run, the interpreter performs each
+foreign side effect exactly once, and all staging is zero afterward.
+
+## Deterministic evidence
+
+`m7_ffi_callxs_authentic` builds the runtime with the explicit activation macro
+and runs two fixtures:
+
+- `t-ffi-callxs-authentic.lua` covers void, signed and unsigned 8/16/32-bit
+  scalar returns, float, double, mixed GPR/XMM/pointer/u64 arguments, C
+  varargs, and generated-call `errno` preservation. It requires real XSAVE and
+  CALLXS IR, while proving bool, pointer, enum, i64 and u64 results remain
+  interpreted with their exact values intact.
+- `t-ffi-callxs-postcall.c` proves exact finalized KTRACE identity, immediate
+  leave ordering, caller snapshot PC/frame geometry, permanent no-side-trace
+  markers, repeated forced POSTCALL cleanup under `hotexit=1`, exact side
+  effects, a deterministic fresh-STOPREQ injection between generated native
+  leave and its throw decision from the same caller snapshot, re-entry,
+  rejected-entry fallback, zero native depth, zero frame depth, zero trace pins,
+  restored callback/function mirrors and cleared XSAVE staging.
+
+Focused validation passed:
+
+- `m7_ffi_callxs_authentic`;
+- default-gated `m7_ffi_ccall_native` and `m7_ffi_native_frames`;
+- `m6_jit_xsave`;
+- `m5_jit_trace_publish`;
+- Clang 19 ASan with fail-fast and leak checking disabled for both authentic
+  fixtures;
+- Clang 19 UBSan with the repository's documented alignment, function,
+  pointer-overflow and shift exclusions;
+- default, `LUAJIT_DISABLE_JIT`, and `LUAJIT_DISABLE_FFI` builds.
+
+## Production gate remains closed
+
+This checkpoint does not claim safe default generic FFI tracing. The remaining
+coupled gates are:
+
+1. callback `ACTIVE -> SUSPENDED -> ACTIVE` publication, nested callback and
+   error-unwind cleanup, plus nonwaiting carrier leasing;
+2. pre-rooted boxed result storage and nonthrowing bool/result handoff;
+3. exact snapshot contracts for pcall, continuation, vararg and tail/RETF
+   return shapes;
+4. remote trace-flush admission for a certified pinned ACTIVE frame, followed
+   by forced POSTCALL exit from the retired exact body;
+5. descriptor-driven aggregate argument and multi-register result lowering;
+6. Linux stress and sanitizer proof followed by Win64/Wine and macOS/Darling
+   ABI and error-state coverage.
+
+Unknown foreign functions can callback on their first invocation regardless of
+their declaration, so historical callback blacklisting cannot justify opening
+the default gate. The next lifecycle tranche is callback suspension, not a new
+signature allowlist.

src/lj_asm.c

@@ -2397,12 +2397,6 @@ static void asm_tail_link(ASMState *as)
   emit_addptr(as, RID_BASE, 8*(int32_t)baseslot);
   checkmclim(as);
 
-  if (as->J->ktrace) {  /* Patch ktrace slot with the final GCtrace pointer. */
-    IRIns *ir = IR(as->J->ktrace);
-    ir_kgc_store_rel(ir, obj2gco(as->J->curfinal));
-    la_store8_rel((uint8_t *)&ir->o, IR_KGC);  /* 05 section 5.7.4 trace root. */
-  }
-
   /* Sync the interpreter state with the on-trace state. */
   checkmclim(as);
   asm_stack_restore(as, snap);
@@ -2412,6 +2406,22 @@ static void asm_tail_link(ASMState *as)
     asm_stack_check(as, as->topslot, NULL, as->freeset & RSET_GPR, snapno);
 }
 
+/* Patch the exact-body constant in every assembly attempt. Loop traces skip
+** asm_tail_link(), and an IR-growth retry allocates a fresh curfinal copy, so
+** neither tail linking nor a one-time patch is sufficient. Assembly proceeds
+** backwards after this point and therefore observes the final KGC value when
+** lowering native-entry arguments. */
+static void asm_patch_ktrace(ASMState *as)
+{
+  if (as->J->ktrace) {
+    IRIns *ir = IR(as->J->ktrace);
+    lj_assertA(ir->o == IR_KNUM || ir->o == IR_KNULL || ir->o == IR_KGC,
+	       "bad exact trace constant op %d", ir->o);
+    ir_kgc_store_rel(ir, obj2gco(as->J->curfinal));
+    la_store8_rel((uint8_t *)&ir->o, IR_KGC);  /* Trace-owned GC root. */
+  }
+}
+
 /* -- Trace setup --------------------------------------------------------- */
 
 /* Clear reg/sp for all instructions and add register hints. */
@@ -2772,6 +2782,7 @@ void lj_asm_trace(jit_State *J, GCtrace *T)
     as->mcp_prev = as->mcp;
 #endif
     as->ir = J->curfinal->ir;  /* Use the copied IR. */
+    asm_patch_ktrace(as);
     as->curins = J->cur.nins = as->orignins;
 #ifdef LUAJIT_RANDOM_RA
     as->prngstate = J2TG(J)->prng;  /* Must (re)start from identical state. */

src/lj_ccall.c

@@ -1719,12 +1719,19 @@ LJ_STATIC_ASSERT((LJ_FFI_NATIVE_LEAVE_FORCE_EXIT & 0x0000ffffu) == 0);
 
 #if defined(LJ_XSAVE_TEST_HELPERS)
 static LJFFINativeTraceFinishHook ffi_native_trace_finish_hook;
+static LJFFINativeTraceLeaveHook ffi_native_trace_leave_hook;
 
 void lj_ffi_native_trace_test_set_finish_hook(
   LJFFINativeTraceFinishHook hook)
 {
   la_storefunc_rel(&ffi_native_trace_finish_hook, hook);
 }
+
+void lj_ffi_native_trace_test_set_leave_hook(
+  LJFFINativeTraceLeaveHook hook)
+{
+  la_storefunc_rel(&ffi_native_trace_leave_hook, hook);
+}
 #endif
 
 /* Validate owner-private staging entirely through integer byte geometry. This
@@ -1786,6 +1793,17 @@ static int ccall_jit_geometry(lua_State *L, TGState *tg,
   return 1;
 }
 
+/* XSAVE staging is a single generated-entry attempt, not persistent carrier
+** state. Consume it after successful publication or before any ordinary
+** rejection returns. This avoids leaving raw stack geometry live while the
+** pre-call exit resumes in the interpreter. */
+static void ccall_jit_xsave_clear(TGState *tg)
+{
+  la_storeptr_rel((void **)&tg->ffi_xsave_root, NULL);
+  la_store32_rel(&tg->ffi_xsave_baseslot, 0);
+  la_store32_rel(&tg->ffi_xsave_nslots, 0);
+}
+
 /* The executing trace's published jit_base is the independent lifetime proof
 ** required before the first T read. SMR then validates the exact public slot
 ** while native-pin admission races retirement without waiting. */
@@ -1823,8 +1841,9 @@ int lj_ffi_native_trace_enter(lua_State *L, GCtrace *T, void *func)
   CCallErrorState err;
   CCallJITGeometry geo;
   LJFFINativeFrame frame;
-  global_State *g;
-  TGState *tg;
+  global_State *g = NULL;
+  TGState *tg = NULL;
+  TGState *current = NULL;
   CCallbackRuntime *cb;
   TraceNo traceno;
   uint32_t depth;
@@ -1833,11 +1852,19 @@ int lj_ffi_native_trace_enter(lua_State *L, GCtrace *T, void *func)
   /* This must remain the first operation: entry validation and SMR admission
   ** must not replace the errno/LastError pair observed by the foreign call. */
   ccall_error_save(&err);
-  if (!L || !T || !func)
+  if (!L)
     goto out;
-  tg = L2TG(L);
   g = G(L);
-  if (!tg || !g)
+  if (!g)
+    goto out;
+  /* IR_XSAVE writes through RID_DISPATCH, which the x64 VM loads directly
+  ** from L->tg_hint. Require that staged/dispatch carrier to be the current
+  ** TLS carrier before reading or clearing any owner-private word. */
+  tg = L->tg_hint;
+  current = G2TG(g);
+  if (!tg || tg != current || lj_tg_load_cur_L(tg) != L)
+    goto out;
+  if (!T || !func)
     goto out;
   /* Capacity is the expected nonexceptional failure. Check it before the raw
   ** trace constant so a full stack always side-exits without touching it. */
@@ -1881,17 +1908,21 @@ int lj_ffi_native_trace_enter(lua_State *L, GCtrace *T, void *func)
     goto out;
   }
   /* The exact even frame precedes every remotely acknowledged native state.
-  ** Consume staging only after publication; every earlier failure leaves it
-  ** intact for the pre-call side exit. */
-  la_storeptr_rel((void **)&tg->ffi_xsave_root, NULL);
-  la_store32_rel(&tg->ffi_xsave_baseslot, 0);
-  la_store32_rel(&tg->ffi_xsave_nslots, 0);
+  ** Successful entry consumes staging here after publication; every ordinary
+  ** earlier rejection consumes it at the common return. */
+  ccall_jit_xsave_clear(tg);
   ccallback_slot_rel(cb, ~0u);
   lj_tg_ffi_call_func_rel(tg, func);
   ccallback_native_had_stopreq_rel(cb, had_stopreq);
   lj_native_enter(tg);
   ok = 1;
 out:
+  /* A valid current-carrier route owns the staged words even when admission
+  ** rejects before geometry, pinning or frame publication. The equality checks
+  ** above prevent a stale hint from mutating a different carrier. */
+  if (!ok && tg != NULL && tg == current &&
+      lj_tg_load_cur_L(tg) == L)
+    ccall_jit_xsave_clear(tg);
   ccall_error_restore(&err);
   return ok;
 }
@@ -1901,7 +1932,7 @@ uint32_t lj_ffi_native_trace_leave(lua_State *L)
   CCallErrorState err;
   LJFFINativeFrame frame;
   global_State *g;
-  TGState *tg;
+  TGState *tg, *current;
   CCallbackRuntime *cb;
   LJFFINativeFrame *src;
   GCtrace *T;
@@ -1914,9 +1945,10 @@ uint32_t lj_ffi_native_trace_leave(lua_State *L)
   ccall_error_save(&err);
   if (LJ_UNLIKELY(L == NULL))
     abort();
-  tg = L2TG(L);
   g = G(L);
-  if (LJ_UNLIKELY(tg == NULL || g == NULL ||
+  tg = L->tg_hint;
+  current = g ? G2TG(g) : NULL;
+  if (LJ_UNLIKELY(tg == NULL || tg != current || g == NULL ||
       lj_tg_load_cur_L(tg) != L))
     abort();
   depth = lj_ffi_native_frame_depth_acq(tg);
@@ -1936,6 +1968,16 @@ uint32_t lj_ffi_native_trace_leave(lua_State *L)
   ** leader owns its consumed poll. Do not change any payload or pin until
   ** native_leave has closed native state and returned from that wait. */
   actions = lj_native_leave(L);
+#if defined(LJ_XSAVE_TEST_HELPERS)
+  {
+    /* Deterministic injection point after the native poll and before the
+    ** fresh-STOPREQ decision. Production builds contain no hook load/call. */
+    LJFFINativeTraceLeaveHook hook =
+      la_loadfunc_acq(&ffi_native_trace_leave_hook);
+    if (hook)
+      hook(tg);
+  }
+#endif
   /* This predicate may perform one final owner poll, but never throws. If it
   ** reports a fresh STOPREQ, no generated post-call guard can be relied upon:
   ** the frame and pin must be released before checkstop unwinds the trace. */

src/lj_ccall.h

@@ -215,12 +215,14 @@ LJ_FUNC uint64_t lj_ffi_native_frame_sequence_acq(const TGState *tg);
 LJ_FUNC uint32_t lj_ffi_native_frame_depth_acq(const TGState *tg);
 
 #if LJ_HASJIT
-/* Owner-only XSAVE consumer for the future generic IR_CALLXS seam. Entry is
-** allocation-free and nonwaiting; zero requests a pre-call side exit without
-** consuming XSAVE state. Ordinary leave removes the frame. A forced leave
-** converts it to POSTCALL and retains its exact pin until the unconditional
-** caller-state trace exit invokes cleanup. Nested/callback entry remains gated
-** until suspended-frame publication exists. No recorder emits these calls yet.
+/* Owner-only XSAVE consumer for the generic IR_CALLXS seam. Entry is
+** allocation-free and nonwaiting; every attempt consumes XSAVE staging and
+** zero requests a pre-call interpreter exit. Ordinary leave removes the frame.
+** A forced leave converts it to POSTCALL and retains its exact pin until the
+** non-side-linkable caller-state trace exit invokes cleanup. The default
+** recorder remains gated; explicit test-only activation exercises the admitted
+** scalar lifecycle. Nested/callback entry remains gated until suspended-frame
+** publication exists.
 */
 #define LJ_FFI_NATIVE_LEAVE_FORCE_EXIT	0x80000000u
 LJ_FUNC int lj_ffi_native_trace_enter(lua_State *L, struct GCtrace *T,
@@ -231,8 +233,11 @@ LJ_FUNC int lj_ffi_native_trace_exit_cleanup(lua_State *L,
 					     uint32_t traceno);
 #if defined(LJ_XSAVE_TEST_HELPERS)
 typedef void (*LJFFINativeTraceFinishHook)(TGState *tg);
+typedef void (*LJFFINativeTraceLeaveHook)(TGState *tg);
 LJ_FUNC void lj_ffi_native_trace_test_set_finish_hook(
   LJFFINativeTraceFinishHook hook);
+LJ_FUNC void lj_ffi_native_trace_test_set_leave_hook(
+  LJFFINativeTraceLeaveHook hook);
 #endif
 #endif
 #if defined(LJ_FFI_NATIVE_FRAME_TEST_HELPERS)