Coding Interview: How do you turn Go fuzz failures into regression corpus?
Question and suitable scenarios
You maintain a Go input parser that occasionally sees Unicode, truncated bytes, or extreme lengths in production. Explain how to design a testing.F fuzz target, choose seeds, express properties when exact output is unknowable, and preserve a failure as a regression after the fix.
This fits backend, infrastructure, and test-tooling roles. Microsoft’s technical-interview guidance explicitly evaluates testing, boundaries, and security implications; Amazon’s SDE guidance lists programming and core software topics. The interview signal is the engineering loop, not memorizing a command.
What the interviewer is testing
- Distinguishing example-test expected values from fuzz-test properties.
- Choosing a fast target without external side effects.
- Explaining seed corpus, coverage guidance, minimization, and replay.
- Handling invalid UTF-8, empty input, oversized input, and resource limits.
- Connecting discovery, diagnosis, repair, rerun, and committed corpus.
A weak answer says “generate random inputs.” A strong answer names the invariant, commands, failure path, and CI split.
Clarifications before answering
- Is the parser input a
string,[]byte, or several fields? This determines fuzz arguments and corpus encoding. - Which invalid inputs are expected errors? An error contract changes the assertion.
- What is the per-call resource budget? A slow input may be a denial-of-service signal.
- Are we seeking panics, semantic bugs, or compatibility regressions? Each needs different properties and seeds.
A 30-second answer framework
I start with a checkable property, such as “encoding after parsing preserves the same structure” or “invalid input returns a controlled error.” I add real boundary cases with f.Add, then keep f.Fuzz fast, deterministic, and bounded. Ordinary go test runs seeds and saved failures; a separate CI job runs go test -fuzz with a time budget. Go minimizes a failing input into testdata/fuzz/<name>. After fixing the root cause, I replay that input, run the full test suite, and commit the corpus so the discovery becomes a regression constraint.
Step-by-step deep answer
1. Start with a property, not an expected answer
For a parser, use a round-trip property: Encode(Parse(x)) should represent x after normalization. For a string transform, use idempotence, length preservation, or UTF-8 validity. Allow specified normalization; do not mistake byte layout for the contract.
2. Build a bounded fuzz target
func FuzzParseRoundTrip(f *testing.F) {
f.Add([]byte("name=alice"))
f.Add([]byte{})
f.Fuzz(func(t *testing.T, input []byte) {
t.Helper()
if len(input) > 1<<20 {
t.Skip()
}
got, err := Parse(input)
if err != nil {
return
}
again, err := Parse(Encode(got))
if err != nil || !Equal(got, again) {
t.Fatalf("round trip failed: %v", err)
}
})
}The f.Add types and order must match the callback. The target should not write files, call a network, or depend on time; otherwise parallel fuzzing produces non-reproducible failures.
3. Seed business boundaries
Seeds should cover protocol versions, empty values, duplicate fields, non-ASCII text, truncation, and sanitized production samples. Go runs seeds during ordinary tests, so each seed must be cheap and stable.
4. Split execution modes
Before a commit, run go test -run=FuzzParseRoundTrip to check seeds. A dedicated job can run go test -fuzz=FuzzParseRoundTrip -fuzztime=10s. The time flag bounds exploration; CI owns parallelism and package budgets rather than production code.
5. Replay and diagnose the minimized input
Go minimizes an input that still triggers the failure and writes it under testdata/fuzz/FuzzParseRoundTrip/. Replay with go test -run=FuzzParseRoundTrip/<id>, then classify the root cause: assertion, panic, resource exhaustion, or test nondeterminism. The sample should explain the bug, not be an opaque blob.
6. Make the fix permanent
Replay the failure after the fix, then run all go test. Saved failures also run without -fuzz, so review the corpus for secrets, redaction, and size limits before committing it.
7. Measure fuzz quality
If coverage stops growing, add structured seeds or improve generation instead of only increasing duration. Frequent timeout failures call for smaller inputs or isolation of expensive paths; treat them as performance defects. A falling discovery rate is not proof of correctness.
High-quality sample answer
I define fuzzing around a reproducible property, not an expected business result for every random input. For a parser, valid input should parse and re-encode to the same structure; invalid input should return a controlled error and never panic. I seed empty input, the maximum accepted size, duplicate fields, Unicode, and sanitized production cases. The target is bounded and side-effect free. Developers run go test -run=FuzzX; CI explores with a fixed -fuzztime. When Go writes a minimized failure, I replay it, identify the root cause, fix the implementation, run ordinary tests and fuzzing, and commit testdata/fuzz/FuzzX. One discovery then becomes a regression check on every change.
Common mistakes
- Treating fuzzing as random load → no oracle exists → define an invariant and error contract first.
- Calling a live service from the target → network and state make failures flaky → use a deterministic fake.
- Running saved failures only in fuzz mode → fixes can regress → keep them in
testdata/fuzz. - Accepting unbounded input → one case consumes the budget → enforce a limit and record skips.
- Committing every generated case → repository noise and growth → keep cases that reproduce bugs or cover key branches.
Follow-up questions and responses
Is a round-trip assertion too strict when parsing normalizes input?
Yes. Compare normalized ASTs or field sets and state which ordering, whitespace, or case differences are intentionally ignored.
Can a failing sample containing user secrets be committed?
No. Check credentials and personal data, redact, and verify the redacted sample still fails. If it cannot be safely committed, retain controlled internal reproduction and submit a synthetic equivalent.
The CI budget is five minutes. How should fuzzing fit?
Run seeds and known failures on every build. Bound exploration by time, packages, and resources; record timeouts instead of silently skipping them.
How do you tell whether the fuzz test is broken?
Replay the minimized case and inspect global state, randomness, and goroutine timing. Then write a deterministic unit test; if it is flaky, fix isolation before production code.
Can multiple targets share a corpus?
Share conversion logic only when input format and semantics match. Keep separate directories and invariants so one target’s corpus cannot hide another’s coverage gap.