Skip to content

fix(ir): keep concrete scalar foreach values through the loop local (#405)#550

Open
mirchaemanuel wants to merge 2 commits into
illegalstudio:mainfrom
mirchaemanuel:fix/405-foreach-value-concrete-element
Open

fix(ir): keep concrete scalar foreach values through the loop local (#405)#550
mirchaemanuel wants to merge 2 commits into
illegalstudio:mainfrom
mirchaemanuel:fix/405-foreach-value-concrete-element

Conversation

@mirchaemanuel

Copy link
Copy Markdown
Contributor

Fixes #405.

Problem

Appending the by-value foreach loop variable into another array and returning that array from a function produced a corrupted result: $r[0] printed nothing (or a garbage integer), and iterating the result died with Fatal error: heap memory exhausted. On current main the minimal repro is fatal already at echo $r[0].

Root cause (two layers)

1. Signature/layout mismatch across the return boundary. foreach_value_type() degraded the loop value local to a boxed Mixed for every element type except Callable/Object. Appending that box widened the array under construction to array<mixed> at runtime, while the checker-side signature kept the concrete element type (array<string>). The caller then read the returned array through the concrete layout: for int elements it printed the box address; for string elements it read the box pointer as a (ptr,len) pair and the garbage length turned the next read into a huge copy → heap exhaustion. Inline code was unaffected because the widened slot type is used consistently inside one function — only the call boundary miscompiled.

2. Ownership asymmetry for borrowed Str iterator values. With concrete elements preserved, iter_current_value results were still modeled as owning temporaries. That is correct for the boxed-Mixed path (the box is a fresh owned cell) and for objects/callables (the codegen retains them), but a concrete Str current value is a borrowed pointer into the source array's payload — exactly like ArrayGet string results, which value_is_owned_index_read_temp already documents as borrowed. The retaining store's source release freed the source array's string block while the array still referenced it; the very next __rt_str_persist reused the freed block, and the source array's deep-free at foreach.exit zeroed it out from under its new owner. Verified with an lldb watchpoint: the element byte is overwritten by _rt_heap_free's free-list link write, called from _rt_array_free_deep in the foreach.exit block.

Fix

  • foreach_value_type() (src/ir_lower/stmt/mod.rs): preserve concrete scalar element types (Int, Float, Str, Bool) for the by-value loop variable. The codegen already supports unboxed concrete elements on both arches (load_current_array_value_{aarch64,x86_64} handle Int/Bool/Float/Str); only the EIR result type was missing.
  • value_is_owning_temporary() (src/ir_lower/context.rs): IterCurrentValue/IterCurrentKey results stay owning except when the result is a concrete Str, which is borrowed from the source container payload (same rule the ownership model already applies to ArrayGet string results).

QA (macOS ARM64, debug build, all with --heap-debug)

  • Issue Appending a foreach value into a new array and returning it corrupts the array (dangling elements / heap exhaustion) #405 repro: prints a / abc, heap clean (was: empty + Fatal error: heap memory exhausted).
  • Int variant: prints 123 (was: box addresses like 4304419072).
  • 15 discriminating probes (function/inline, literal/explode source, local/temporary source, subscript-based control cases) all correct and heap-clean.
  • Edge cases: value var read after the loop, owned reassignment inside the body, nested string foreach, assoc source (hash path unchanged, still Mixed), by-ref foreach (unchanged), zero-iteration loop, float elements — all correct and heap-clean.
  • 4 new regression tests in tests/codegen/arrays/foreach_value_append.rs.
  • Targeted test slices covering the touched areas — arrays::, foreach*, spl::, optimizer::, control_flow::, iterator*, strings::, generators::, zval:: plus the full ir_backend_smoke_test target: 1708 passed, 0 failed (the full suite compiles a native fixture per test and runs >1h locally; deferring the rest to CI).

x86_64: the concrete-element iterator loads and __rt_array_push_str paths are pre-existing and symmetric on both arches; verified by inspection here (local Mach-O toolchain can't assemble the ELF runtime), CI shards cover it.

…llegalstudio#405)

A by-value foreach over an array with a concrete scalar element type
degraded the loop variable to a boxed Mixed local, so appending it
widened the array under construction to array<mixed> at runtime while
the checker-side function signature kept the concrete element type.
Callers then read the returned array through the concrete layout:
box addresses for int elements, and a garbage (ptr,len) reinterpretation
for string elements that made the next read exhaust the heap.

foreach_value_type() now preserves Int/Float/Str/Bool elements (the
codegen already loads unboxed concrete iterator values on both arches),
and IterCurrentValue/IterCurrentKey results are modeled as owning
temporaries only when they really bind a fresh owned copy: a concrete
Str current value is a borrowed pointer into the source array's payload
(the same rule the ownership model applies to ArrayGet string results),
so the retaining store must not release it — that release freed the
source array's string block, the next __rt_str_persist reused it, and
the source's deep-free at foreach.exit zeroed it under its new owner.
@github-actions github-actions Bot added area:eir Touches EIR definitions, lowering, validation, or passes. size:s Small pull request. type:fix Corrects broken or incompatible behavior. labels Jul 16, 2026
@mirchaemanuel

Copy link
Copy Markdown
Contributor Author

Note on the single red shard (Eval Codegen Tests macos-aarch64 13/16): the failure is codegen::eval::test_eval_declared_method_return_type_values timing out at the 60s per-try limit (both tries), not an assertion failure. I can't rerun the job myself (no repo admin rights), so leaving the analysis here:

  • Locally (macOS ARM64, debug build) the test passes in 37.7s on this branch and 38.0s on pristine main — identical timing, so this PR does not slow it down; it's just an intrinsically slow fixture that clears the 60s bar only barely, and the shared macos-14 runner (with -j 2) pushed it over.
  • The same test passed in the linux-x86_64 and linux-aarch64 eval shards of this run, and every other macOS shard (codegen + eval) is green.

A rerun of the failed shard should go green. Happy to adjust anything if you'd rather bump that fixture's timeout instead.

…hod_return_type_values

The test intentionally exercises five complete compile-link-run eval
programs in one test (~38s on a local ARM64 debug build, identical on
pristine main), which clears the global 60s slow-timeout only barely
and timed out on a loaded shared macos-14 CI runner. Same category and
same 180s override as the eval regression tests already listed here.
@github-actions github-actions Bot added the area:tooling-ci Touches CI, development tooling, Docker, or repository scripts. label Jul 16, 2026
@mirchaemanuel

Copy link
Copy Markdown
Contributor Author

Pushed fb2f4622f fixing the red shard: test_eval_declared_method_return_type_values is now on the same 180s slow-timeout override list in .config/nextest.toml as the other eval regression tests that intentionally run several complete compile-link-run programs in one test. It runs five of them (~38s locally, identical on pristine main, so not a regression of this PR) — that clears the global 60s guard only barely and timed out on a loaded shared macos-14 runner. Same pattern and same 180s value as the existing entries.

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

area:eir Touches EIR definitions, lowering, validation, or passes. area:tooling-ci Touches CI, development tooling, Docker, or repository scripts. size:s Small pull request. type:fix Corrects broken or incompatible behavior.

Projects

None yet

Development

Successfully merging this pull request may close these issues.

Appending a foreach value into a new array and returning it corrupts the array (dangling elements / heap exhaustion)

1 participant