Add cargo-fuzz / OSS-Fuzz coverage for DataFusion's parser and analyzer

[youichi-uda]

Add cargo-fuzz / OSS-Fuzz coverage for DataFusion's parser and analyzer

Background

DataFusion already has excellent property-based fuzzers for runtime correctness — sort_fuzz, aggregation_fuzzer, join_fuzz, and friends in datafusion/core/tests/fuzz_cases/. They generate random queries against valid data and cross-check results, which is exactly the right tool for catching aggregation/sort/join semantic bugs.

What we don't yet have is byte-level (libFuzzer / cargo-fuzz) coverage. That's a different attack surface: feeding arbitrary bytes into entry points like DFParser::parse_sql exercises the SQL tokenizer/parser, the dialect-extension code, and any error-recovery paths the parser walks while building the AST. These are the kinds of bugs that show up as panics or pathological allocations rather than wrong results.

This is the same delta arrow-rs#5332 is closing on the format-parser side; DataFusion benefits from byte-level fuzzing for its own reasons.

Concrete proposal

1. Add per-crate fuzz/ directories with cargo-fuzz harnesses, starting with the smallest blast radius and growing outward as we land. Two are already prototyped on my fork at fuzz/initial-harnesses:

   • datafusion/sql/fuzz/parse_sql — DFParser::parse_sql over UTF-8-validated &[u8]. 22 s sanity run reaches 3,520 edges / 12,788 features / 2,342 corpus entries from an empty start, RSS peak 117 MiB, no crashes.
   • datafusion/sql/fuzz/parse_expr — DFParser::parse_sql_into_expr (different production set). 22 s sanity reaches 3,980 edges / 14,500 features / 2,492 corpus entries at 9,090 runs/s, no crashes.

   Then in subsequent PRs, depending on what shakes out: a LogicalPlanBuilder harness in datafusion/expr, a SessionContext::sql end-to-end harness, and possibly per-format datasource readers (csv/json/parquet/arrow/avro), though those overlap with the Fuzz tests for Arrow/Parquet arrow-rs#5332 work and we'd want to dedupe.

2. Wire ClusterFuzzLite into GitHub Actions so PR-time fuzz + nightly batch + corpus pruning run automatically. Catches regressions before they merge instead of relying on me to remember to run cargo-fuzz locally.

3. Open the OSS-Fuzz integration PR (projects/datafusion/) once the upstream fuzz/ lands and the harnesses are stable. Mirrors the layout I'm proposing for arrow-rs in google/oss-fuzz. Coverage target ≥80% on harness-touched modules.

Why this is worth doing

• DataFusion is widely used as a SQL engine inside other projects (InfluxDB, Ballista, Coralogix, DataFusion Comet, ...). A panic or pathological allocation in DFParser::parse_sql can DoS any service that exposes a SQL endpoint to untrusted input.
• The SQL parser walks an attacker-controlled token stream through DataFusion's extensions on top of sqlparser-rs. The sqlparser crate itself does not appear to be fuzzed in OSS-Fuzz, so DataFusion sits behind two unfuzzed parser layers right now.
• Coverage cost is modest: cargo-fuzz harnesses are typically tens of lines each, and the OSS-Fuzz/ClusterFuzzLite plumbing is a one-time setup.

Status

I'm prototyping this in my fork at masumi-ryugo/datafusion@fuzz/initial-harnesses and have a working parse_sql harness building under nightly cargo-fuzz. Happy to send the upstream PR once the layout question is settled.

Two questions for the maintainers (cc @alamb @andygrove @comphead @ozankabak):

1. Layout preference: per-crate fuzz/ dirs (gitoxide-style — independent buildable units, easy to add per-crate seed corpora) vs. a single top-level fuzz/ workspace? I'd lean per-crate but happy to do whatever you prefer.

2. primary_contact / auto_ccs for OSS-Fuzz project.yaml: OSS-Fuzz wants Google-account emails on file for crash notifications. Who from the PMC would want to be on that list? I can default to a single contact if a PMC alias isn't preferred.

xref apache/arrow-rs#5332 (sibling work for arrow-rs side, same author, same OSS-Fuzz integration pattern).

[2010YOUY01]

Thank you — here is some additional context:

There is already an existing fuzzer at https://github.com/apache/datafusion-sqlparser-rs/tree/main/fuzz.

I think a single top-level fuzzer would be preferable, unless there are specific crates that make good standalone fuzz targets.

I’ve tried general-purpose byte-level fuzzers before, but they don’t seem to work well for SQL. Ideally, test cases that target edge cases should be valid SQL, or at least close to valid queries. However, general-purpose fuzzers struggle to generate such inputs, likely because they cannot infer the underlying SQL structure, which is significantly more complex than the input formats that work well with byte-level fuzzing.

Overall, I think it’s great to have basic coverage, but this direction may not be worth investing too deeply in. Though, using those traditional byte-level fuzzers to mutate existing test case slightly would be a good idea to strengthen the test coverage.

2. primary_contact / auto_ccs for OSS-Fuzz project.yaml: OSS-Fuzz wants Google-account emails on file for crash notifications. Who from the PMC would want to be on that list? I can default to a single contact if a PMC alias isn't preferred.

How does that work 🤔? Is it the same as a local fuzz runner, except Google provides compute resources to run it for you?

[youichi-uda]

Thanks @2010YOUY01 — that's a fair read of where byte-level fuzzing pays off and where it doesn't, and I agree with most of it. A couple of clarifications and a smaller proposal.

On the existing apache/datafusion-sqlparser-rs/fuzz

I missed that — sorry. It uses honggfuzz with a single fuzz_parse_sql target that round-robins across all 14 dialects, which is a perfectly reasonable design for the parser and is more comprehensive than the parse_sql harness I'd prototyped. I don't think layering a second cargo-fuzz target on top of that crate buys much.

On byte-level fuzzing for SQL specifically

Your read matches mine. For comparison — the binary-format harnesses I've been prototyping in parallel (apache/arrow-rs#5332 / pola-rs/polars#27488) found 6 distinct DoS-class bugs over the past two days (fix PRs: arrow-rs#9883/9884/9886/9887, polars#27489/27490). For the SQL parser side, a 5-minute libFuzzer run on DFParser::parse_sql reached 4,377 edges and 20,378 features but produced no crashes — entirely consistent with what you'd expect from a parser that's been under property-based + honggfuzz coverage already.

So the leverage isn't comparable, and I don't think DataFusion needs the kind of harness-density push the binary-parser side does. Withdrawing the broad proposal.

Smaller scope I'd actually suggest

Two narrower things, both fine to decline:

1. Bring apache/datafusion-sqlparser-rs/fuzz under OSS-Fuzz. The honggfuzz harness already exists; OSS-Fuzz would just give it persistent corpus, 24/7 compute, and an automated crash-notification path. ~zero net work in this repo or in datafusion-sqlparser-rs itself; net change is a new projects/datafusion-sqlparser-rs/ directory in google/oss-fuzz that drives the existing harness. Happy to send that as a standalone proposal in datafusion-sqlparser-rs if it's useful.

2. Corpus-mutation harness only. As you suggested, a small target that takes a short byte slice and uses it to mutate the existing test SQL corpus already in datafusion/sql/tests, then re-parses. Stays in this repo, tiny surface, no big harness fleet. Sketch:

   fuzz_target!(|data: &[u8]| {
       let base = TEST_SQL_CORPUS[(data.first().copied().unwrap_or(0) as usize) % TEST_SQL_CORPUS.len()];
       let s = mutate_corpus_string(base, &data[1..]);  // small bit-flip mutator
       let _ = DFParser::parse_sql(&s);
   });

   This addresses your "valid SQL or close to it" point — every input is a near-valid query rather than random bytes.

Either or neither — happy to follow whatever suits the project.

On how OSS-Fuzz works

Good question. Short version:

• It's basically Google-hosted ClusterFuzz, free for OSS projects.
• You ship a projects/<name>/ directory in google/oss-fuzz with a project.yaml (contacts, sanitizers, fuzzing engines), a Dockerfile that fetches the source, and a build.sh that builds the harnesses into $OUT/.
• Their infra rebuilds your harnesses daily, runs them on a fleet 24/7 with persistent corpus, and reports new crashes to the email addresses in project.yaml (primary_contact + auto_ccs). Crashes get auto-bisected, deduped, and filed as private issues that flip public after the standard 90-day disclosure window.
• Local workflow is python infra/helper.py {build_image, build_fuzzers, check_build, run_fuzzer, coverage} <project>, which mirrors what their infra does so you can iterate on the harness before pushing.
• Apache projects that already use it for comparison: projects/arrow (Arrow C++), projects/librdkafka, projects/parquet-mr, etc. Same shape, just pointed at a Rust crate instead.

The closest precedent for the Rust workspace shape we'd want here is projects/gitoxide (auto-discovers per-crate fuzz/ dirs in a workspace and builds each) or projects/fontations. Both are referenced from my prototype projects/datafusion/ if you want to look — though as above, I'd actually recommend pointing it at datafusion-sqlparser-rs instead and leaving DataFusion proper alone.

---

Net: I'd close this issue or downscope it to "OSS-Fuzz integration of the existing datafusion-sqlparser-rs honggfuzz harness, tracked there." Whichever you prefer. cc @alamb @andygrove @comphead @ozankabak in case anyone has a different read.

[alamb]

I think any additional sql fuzzing coverage that finds issues is valuable. I don't think there is any particular reason the fuzz testing has to be added to this repository / maintained by the same maintainers as the core repo. Fuzz teseting can be done using only DataFusion's pubic APIs

In fact it is likely valuable to run more than one fuzzer to get larger coverage

[comphead]

more tests even better, however I'm not sure about asserting random array of bytes, if it would be reasonable, I was thinking about more specialized fuzz testing for SQL inputs:

• generate thousands of SQL queries covering built in functions, joins, etc
• check analyzer/parser speed and correctness, but without output correctness checks

[youichi-uda]

Thanks @alamb and @comphead.

@alamb — your "fuzz can use only public APIs / doesn't need to live in this repo" framing is exactly the lighter-touch model I'd default to, so that's helpful to hear.

@comphead — agreed that SQL-aware query generation (SQLsmith-style — generate thousands of valid-ish queries covering joins, built-ins, etc., then assert parser/analyzer don't panic and meet basic perf bounds) is more valuable per-input than byte-level on a mature SQL surface. I'd treat it as complementary to byte-level corpus mutation rather than a replacement, since they fail in different ways: a grammar generator finds bugs in the combinations the grammar can express, and a corpus mutator finds bugs in malformed/edge bytes near the corpus. But they're different enough harnesses that I think they're best authored by separate people — I'm not the right person for the grammar-generator one (no SQLsmith experience), so I'll scope my own follow-through to the byte-level side only.

Given the two narrower options I floated:

Going to start with (a) — wrap datafusion-sqlparser-rs/fuzz under OSS-Fuzz. It's the least intrusive: zero code change in this repo or in datafusion-sqlparser-rs, just a new projects/datafusion-sqlparser-rs/ directory in google/oss-fuzz that drives the existing honggfuzz harness. Concrete next step before any PR: I'll open a small issue on apache/datafusion-sqlparser-rs asking for primary_contact + auto_ccs Google-account emails (OSS-Fuzz needs them for crash notifications and there's no PMC alias path — same prerequisite I hit on apache/arrow-rs#5332). Once I have those I can send the google/oss-fuzz PR; it's mechanical. I'll cross-link both back here.

Parking (b) — corpus-mutation harness in datafusion/sql/tests. Won't touch this repo's code without an explicit go from @2010YOUY01, since this one does require an in-repo crate and that's the part you sounded skeptical about. Happy to drop it entirely if you'd rather, or revisit once (a) has produced (or not produced) anything interesting to bring back.

No code action from me until the contact-email issue on datafusion-sqlparser-rs lands. Will report back when that's filed.