Skip to content

feat(fuzz): report symbolic frontier failures#15646

Closed
figtracer wants to merge 2 commits into
masterfrom
fig/hybrid-symbolic-fuzz
Closed

feat(fuzz): report symbolic frontier failures#15646
figtracer wants to merge 2 commits into
masterfrom
fig/hybrid-symbolic-fuzz

Conversation

@figtracer

@figtracer figtracer commented Jul 8, 2026

Copy link
Copy Markdown
Member

What

This makes targeted symbolic frontier solving surface replay-confirmed fuzz failures immediately.

Before this change, --symbolic-use-fuzz-frontiers could solve a branch frontier, replay the candidate concretely, persist the resulting input into --fuzz-corpus-dir, and then still let the current forge test invocation pass. Users had to discover the failure through a later forge fuzz replay run.

After this change, if the symbolic frontier candidate replays as a fuzz failure, Forge reports it as the fuzz counterexample for the current run and still persists the seed for later corpus replay.

Why

This is the smallest useful closed-loop behavior for the merged branch-frontier pipeline:

  1. fuzzing captures a frontier comparison;
  2. symbolic execution flips that comparison;
  3. concrete replay confirms the input fails;
  4. the fuzz command reports that failure immediately.

That makes the hybrid flow useful without requiring a separate manual replay step.

Benchmark

I used a small hard-interval fuzz target where baseline fuzz misses the bug with a fixed seed, while frontier-guided symbolic execution flips the recorded comparison.

function testFuzz_magic(uint256 x) public pure {
    uint256 target = uint256(keccak256(abi.encodePacked("hybrid-magic-frontier")));
    if (x > target) {
        if (x < target + 100) {
            assert(false);
        }
    }
}

Commands were run with compilation already warm and runtime caches cleared per sample.

mode command shape result mean
baseline fuzz --fuzz-runs 10000 --fuzz-seed 0x1234 --threads 1 passes, misses bug 343.8 ms ± 2.9 ms
hybrid frontier 64 fuzz runs to capture frontier, then one targeted symbolic frontier solve fails with replay-confirmed counterexample 144.3 ms ± 0.6 ms

hyperfine reports the hybrid flow as 2.38x ± 0.02x faster for this repro.

Notes

This is deliberately much smaller than #15591. It builds on the already-merged stateless frontier capture/consume path and does not depend on the broader stateful invariant symbolic PR.

My read after this benchmark is that #15591 should be kept only if we want the separate stateful-invariant symbolic engine work on its own merits. It should not be the main vehicle for proving the hybrid fuzzer loop: this PR demonstrates the stateless frontier loop with a 3-file behavior change and a focused benchmark, while #15591 remains a broader 22-file symbolic/invariant change.

The second commit is a local clippy hygiene fix for an existing forge fuzz parser stack-frame warning that reproduces on clean origin/master with current local nightly. It is kept separate from the behavior change.

Validation

  • cargo +1.96.0 build -p forge
  • cargo +1.96.0 check -p forge
  • cargo +1.96.0 test -p forge --test cli test_cmd::symbolic::symbolic_fuzz_frontier_seeding_persists_branch_flipping_input -- --exact --nocapture
  • cargo +nightly fmt --all -- --check
  • git diff --check
  • cargo +nightly clippy -p forge --test cli -- -D warnings

Authored with AI assistance.

@figtracer

Copy link
Copy Markdown
Member Author

Closing: I opened this while misreading the retarget request. The intended work is to retarget #15631 instead.

@figtracer figtracer closed this Jul 8, 2026
@figtracer
figtracer deleted the fig/hybrid-symbolic-fuzz branch July 8, 2026 14:37
@github-project-automation github-project-automation Bot moved this to Done in Foundry Jul 8, 2026
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

Status: Done

Development

Successfully merging this pull request may close these issues.

1 participant