Add page about creating unit tests from crash reproduction data.

[julianharty]

I realised it's feasible to create unit tests based on the reproduction tests and the contents of the relevant fuzz target. I've documented the process based on the tutorial in this book. You're welcome to keep whatever's useful and modify it to suit the rest of the book. I'm also happy to revise the contents based on your feedback.

[fitzgen]

Thanks!

Another thing you can do is check the crashes into your repo (or a submodule) and write a test that does something like

// my_crate/src/lib.rs

pub fn my_fuzz(data: &[u8]) {
    // ...
}

#[test]
fn crash_regression_tests() -> anyhow::Result<()> {
    for entry in fs::read_dir("path/to/crashes")? {
        let entry = entry?;
        let data = fs::read(entry.path())?;
        my_fuzz(data);
    }
    Ok(())
}

// fuzz/fuzz_targets/my_fuzz.rs

libfuzzer_sys::fuzz_target!(|data| {
    my_crate::my_fuzz(data);
});

Better yet, use libtest_mimic to make each file its own test so that they can run in parallel, be ran one at a time, etc.

Mind expanding the docs to include this technique?

[julianharty]

@fitzgen Thanks these great suggestions which complement the approach I've described. I'm happy to incorporate them into this PR.

Big picture

I envisage 3 options devs could use:

1. Create unit tests as per my approach - All the data would be incorporated into the unit test(s) and no external crash files need to be checked in to the repo to run these unit regression tests. Devs would need to hard-code the data (thankfully cargo fuzz fmt fuzz_target_1 fuzz/artifacts/fuzz_target_1/crash-420742a84803bc64cbeb97952d220909c7f91d98 provides the raw data bytes).
2. Relocate the body of the fuzz test target into the core codebase akin to my_fuzz with a suitable function name, add the crash files to the repo or as a sub-module, and implement crash_regression_tests() - This would read each crash file sequentially and call my_fuzz with the contents. Devs wouldn't need to implement individual unit tests, the data payload is read from whatever crash files are found on the local filesystem.
3. Would be similar to option 2 but wrap the tests in libtest_mimic to improve the flexibility and potentially the speed of execution. Again data would be read from crash files found on the local filesystem. As new crashes are found, the crash filenames would need to be explicitly added as tests to the libtest_mimic source file in order to run them.

Have I captured your suggestions adequately? And what else would be useful for readers of this topic in the book?

Implementation details

[Background context]
I expect you know cargo fuzz's internals and how it works better than I do, I've only used it with several projects so far. In terms of your suggestion to read the crashes and call a my_fuzz(data) At least one of the projects I'm familiar with has distinct implementations for each fuzz target e.g. https://github.com/holo-routing/holo/blob/master/fuzz/fuzz_targets/bgp/attribute/aggregator_decode.rs calls a different method to https://github.com/holo-routing/holo/blob/master/fuzz/fuzz_targets/bgp/attribute/as_path_decode.rs in the same package and the project has 25 fuzz targets currently that generally call distinct methods.

[Questions for you]

• Do you envisage developers create distinct my_fuzz(data: &[u8])'s for each method they've used in fuzz targets? in other words there'd be a my_fuzz for each fuzz target?
• Perhaps OT? I've encountered some issues running cargo fuzz targets in parallel (They were: Blocking waiting for file lock on build directory so perhaps more of a build issue?); are you aware of any limitations or constraints running cargo fuzz in parallel? (I'm aware that your suggestions wouldn't be calling cargo fuzz so they shouldn't be affected by any such issues).

I'm unfamiliar with libtest_mimic so please forgive what might seem an obvious question... Would something like the following suit? (again with one implementation per fuzz target rather than a behemoth file containing all the crashes):

use libtest_mimic::{Arguments, Trial};
use std::fs;
use std::path::Path;

fn crash_regression_test_single<P: AsRef<Path>>(crash_file: P) -> anyhow::Result<()> {
    let data = fs::read(crash_file)?;
    my_fuzz(data);
    Ok(())
}

fn main() {
    let args = Arguments::from_args();
    
    // Create tests for three crash files with realistic fuzzing artifact names
    let tests = vec![
        Trial::test("crash_test_420742a84803bc64", move || {
            crash_regression_test_single("fuzz/artifacts/fuzz_target_1/crash-420742a84803bc64cbeb97952d220909c7f91d98")
                .map_err(|e| format!("Test failed: {}", e).into())
        }),
        Trial::test("crash_test_7d3f8a9c2e154b87", move || {
            crash_regression_test_single("fuzz/artifacts/fuzz_target_1/crash-7d3f8a9c2e154b87fa623c810b7e95a4d3c2f6e1")
                .map_err(|e| format!("Test failed: {}", e).into())
        }),
        Trial::test("crash_test_b5e9c4d1a6f82370", move || {
            crash_regression_test_single("fuzz/artifacts/fuzz_target_1/crash-b5e9c4d1a6f82370eaf7b3d95c4e1029f8a7b6c5")
                .map_err(|e| format!("Test failed: {}", e).into())
        }),
    ];

    libtest_mimic::run(&args, tests).exit();
}