The Harness Compiled. That Was the Easy Part: LLM Fuzzing

Consider a composite lab scenario. At 03:17 UTC, an agent-generated harness turns the dashboard green. It fixed three compiler errors, linked against AddressSanitizer, and produced an input that crashes in under a second. The report looks ready for escalation.

One line changes the case. The invalid pointer belongs to the harness, not the target library.

That distinction is the operational problem in LLM fuzzing. A model can write code, repair a build, run a target, and explain a stack trace. None of those actions makes its conclusion trustworthy. The useful unit is not a plausible report or even a crash. It is an evidence chain that survives a clean replay, identifies the owning defect, and binds every artifact to the exact source, toolchain, flags, corpus, and execution environment that produced it.

This is the same evidence-binding problem that appears in privileged automation. The difference is that a fuzzing agent also generates and executes untrusted code. If the pipeline treats compilation as proof, it will reward shallow harnesses, report its own bugs, and eventually turn the research runner into the asset under attack.

The first green signal is the most dangerous one

Google’s OSS-Fuzz team reported 26 vulnerabilities found with AI-generated or AI-enhanced fuzz targets, including CVE-2024-9143 in OpenSSL. The same work increased coverage across 272 C and C++ projects by more than 370,000 lines, with one project moving from 77 covered lines to 5,434.^[1] Those are material results, not a demo.

The mechanism matters more than the headline. The agent did not issue one prompt and wait for a vulnerability. It followed a developer-like loop: draft a harness, repair compilation failures, run short health checks, fix semantic mistakes, execute for longer periods, and triage crashes. Google also documented a case where the model correctly classified a heap-use-after-free as a bug in the fuzz driver rather than the project.^[1]

The open OSS-Fuzz-Gen framework exposes the same distinction in its evaluation model. It measures compilability, runtime crashes, runtime coverage, and line-coverage delta against existing human-written targets. Its published repository describes more than 1,300 benchmarks from 297 projects, valid targets with nonzero coverage gains for 160 C and C++ projects, and a maximum reported line-coverage increase of 29 percent.^[3]

Compilation is therefore Gate 3, not the finish line.^[1][3] A harness that builds but never reaches meaningful target state is a polished no-op. A harness that crashes before consuming input is noise. A harness that increases line coverage may still repeat the same state transitions under different syntax. The operator must keep asking a narrower question: what new behavior did this artifact make observable?

Two search systems, two failure modes

An LLM agent searches through source-level hypotheses. It reads declarations, cross-references, examples, tests, compiler diagnostics, and runtime feedback. This makes it good at reconstructing API preconditions and writing scaffolding that a mutation engine would not invent easily.

A specialized fuzzer searches a different space.^[4] Fuzzilli, for example, represents JavaScript programs in FuzzIL, a custom intermediate language designed to support meaningful mutation while preserving correctness properties. It combines mutation and generative components, lifts FuzzIL to JavaScript only for execution, uses a persistent execution mechanism, and can emit history files that record the mutations, splicing, and generation steps behind each saved program.^[4]

The contrast is useful.^[1][4] The agent can reason about why a target requires a table, callback, object graph, or protocol state before fuzzed data becomes interesting. The fuzzer can keep exploring variations without getting bored, rationalizing a weak test, or changing the success criterion mid-run.

Their failure modes are also complementary:

• The agent can hallucinate APIs, overfit to examples, or repair the harness until it validates its own assumption.
• The fuzzer can optimize a measurable signal that is only loosely connected to security impact.
• Both can generate duplicate crashes that look novel when stack signatures, configurations, or ownership are compared poorly.
• Both can become dangerous when the execution environment exposes credentials, network access, writable caches, or shared build infrastructure.

This is why an agent should feed a fuzzing system, not replace its evidence model.

Build an evidence gate, not a prompt

The inline diagram below treats the workflow as seven gates. Each gate either promotes the artifact with new evidence or rejects it before expensive triage.

1. Target selection: identify the API surface, reachable states, ownership rules, and expected invariants. A function name is not a test strategy.
2. Harness synthesis: generate the smallest state machine that reaches the target. Keep setup deterministic and separate fuzz-controlled bytes from fixed scaffolding.
3. Build: compile and link with pinned dependencies, explicit sanitizer flags, and a recorded toolchain digest.
4. Runtime health: prove the harness survives ordinary inputs, resets state correctly, and does not create lifetime bugs of its own.
5. Coverage delta: compare against the current baseline. Track new states or semantic features when line coverage stops being informative.
6. Crash evidence: require deterministic replay, minimization, deduplication, and a clean distinction between harness defects and project defects.
7. Closure: attribute the root cause, test the fix, search for variants, and retain enough provenance to reproduce the result after the agent session is gone.

A minimal run manifest can be small, but it must be immutable:

target:
  repository: https://example.invalid/authorized-lab.git
  commit: <full-commit-sha>
build:
  container_digest: sha256:<digest>
  compiler: clang-<pinned-version>
  sanitizers: [address, undefined]
execution:
  network: deny
  uid: nonroot
  timeout_seconds: 30
evidence:
  harness_sha256: <sha256>
  corpus_sha256: <sha256>
  reproducer_sha256: <sha256>
  prompt_model_run_id: <immutable-run-id>

ClusterFuzz already demonstrates the mature parts of this discipline: crash deduplication, testcase minimization, regression bisection, statistics, and automated issue lifecycle management.^[5]

OSS-Fuzz combines multiple coverage-guided engines with sanitizers and distributed execution.^[6]

The LLM belongs upstream of those controls, where it can improve harness quality without redefining what counts as a confirmed finding.^[1][5][6]

This also mirrors the lesson from the site’s analysis of GitHub Actions TOCTOU and artifact poisoning: approval must bind to immutable evidence, not to a mutable label that still looks familiar.

The research runner is part of the attack surface

The authorized target may still contain attacker-controlled material. Repository comments, issue fixtures, generated documentation, test names, build scripts, sample files, and submodules can all enter the agent’s context or execution path.

A realistic abuse chain looks like this:

1. Untrusted repository content steers the planner toward a malicious helper or build step.
2. The agent invokes the project’s build system because that action appears necessary to compile the harness.
3. The runner exposes a package token, cloud credential, writable cache, or unrestricted egress.
4. The build script exfiltrates data, modifies shared state, or plants a result that later looks like agent-generated evidence.
5. The report inherits trust from the security pipeline and reaches a maintainer, issue tracker, or release gate.

This is not hypothetical as an architecture class. Security tooling routinely executes content from the assets it inspects. The Nuclei scanner trust analysis shows why loaders, embedded runtimes, and response evaluation must be treated as privileged boundaries. The PraisonAI sandbox policy gap makes the adjacent point: a declared policy is irrelevant when the runtime does not enforce it.

For an LLM fuzzing runner, the minimum containment profile is blunt:

• disposable workspace and unprivileged UID;
• read-only source snapshot, with a separate writable output volume;
• no production credentials, SSH agent, cloud metadata, or inherited developer environment;
• denied network by default, with tightly scoped fetch phases outside the execution sandbox;
• allowlisted commands and resource ceilings for subprocesses, CPU, memory, disk, and wall time;
• complete command, file-write, process-tree, and egress-attempt telemetry;
• a trusted validation component that sees proposed actions and metadata, but not raw untrusted repository text.

That last separation is increasingly practical.^[2] Google’s agentic Chrome architecture describes a distinct critic isolated from untrusted page content, constrained origin sets, work logs, confirmations, and continuous red-team testing.^[2] A fuzzing platform can apply the same principle without copying the browser design: separate the component that proposes an action from the component that authorizes its capabilities.

Detection should focus on evidence drift

Traditional endpoint alerts are necessary but late. The higher-value detections live inside the research pipeline:

• a harness stops calling the declared target after an agent repair;
• compile flags, sanitizer set, or dependency versions drift between discovery and replay;
• crashes disappear under a clean environment or require hidden files, network state, or timing;
• the first input crashes immediately across many targets, which often indicates a harness defect;
• coverage rises only in setup code or error handling;
• the agent edits the harness after seeing the crash without invalidating the prior evidence manifest;
• the same root cause appears under many stack signatures and inflates the unique-finding count;
• a build or test process attempts egress, reads credential paths, or writes outside the run-owned workspace.

Metrics should follow the same logic. Compilation rate is useful for debugging the agent, not for measuring security value.^[1][3] Track stable execution rate, coverage delta beyond the existing baseline, deterministic unique crashes, confirmed root causes per compute unit, and human triage minutes per confirmed finding. The best agent is not the one that produces the most reports. It is the one that reduces the cost of trustworthy evidence.

Strategic takeaways

LLM fuzzing is already operationally useful. The public results show real coverage gains and real vulnerabilities in mature projects.^[1][3] The mistake is to treat model output as a new trust anchor.

Use agents to reconstruct state, propose harnesses, repair build failures, and prioritize suspicious crashes. Keep execution disposable. Keep validation deterministic. Preserve a manifest that can outlive the model session. Let established fuzzing infrastructure own minimization, deduplication, replay, and regression tracking.

Most importantly, make the human review boundary explicit. An autonomous pentest agent can accelerate exploration, but scope control and evidence quality remain engineering responsibilities. The same is true here: autonomy changes throughput. It does not change what proof means.

Sources

• [1] Leveling Up Fuzzing: Finding more vulnerabilities with AI
• [2] Architecting Security for Agentic Capabilities in Chrome
• [3] google/oss-fuzz-gen
• [4] How Fuzzilli Works
• [5] google/clusterfuzz
• [6] google/oss-fuzz