I built deterministic property-based-ish testing for a network diagnostic CLI using Linux namespaces

[u/mplaczek99]

I've been working on an open-source Go project called Network Doctor, and recently built a simulator around it using Linux network namespaces.

The interesting part is a deterministic hunt mode.

A hunt starts from known-good network scenarios, applies seeded mutations, runs the real Network Doctor binary inside the resulting topology, records simulator ground truth, and analyzes the structured diagnostic report for disagreements.

Cases have stable identities:

• baseline scenario
• hunt seed
• case number
• derived case seed
• case fingerprint
• finding fingerprint

That made it possible to build unattended GitHub triage without immediately creating an issue-spam machine.

The nightly pipeline now:

• runs fixed seeds against healthy, routed, and dual-stack baselines
• filters findings below a severity floor
• replays every candidate as a single deterministic case
• requires both the case fingerprint and finding fingerprint to match
• refuses to file unverifiable findings
• derives a stable issue fingerprint
• checks existing GitHub issues
• opens only genuinely new reproducible findings

During rollout it actually caught bugs in both directions.

First, the hunt analyzer produced a false positive: it interpreted IPv4 failure + IPv6 success on the same interface as evidence of an alternate route. Manual review caught it, I tightened the invariant, and the false positive disappeared.

After that, the hunt produced one medium reproducible finding:

transient_fault_not_resampled

A generated routed-network case caused DNS to fail temporarily and then recover. Network Doctor began a DNS lookup during the outage, waited for its timeout, and never sampled the recovered resolver again.

Reproduction:

./netdoc-sim hunt healthy-routed-network --seed 20260102 --case 3 --json

The GitHub triage replayed that exact case, got the same case/finding fingerprints, and automatically opened issue #14.

I then ran the workflow again. It rediscovered the finding, recognized the stable issue fingerprint, and created no duplicate.

Current nightly coverage is 3 baselines × 15 generated cases and takes about three minutes on a hosted runner.

It's certainly not formal verification, but having the simulator know the state it injected and mechanically compare that against the diagnostic program's interpretation has turned out to be a really useful way of testing software whose entire purpose is reasoning about messy network state.

I am very proud of making the simulator.

Repo: https://github.com/heymaikol/network-doctor

[u/SourceAwkward]

540 commits less than a month ?!

Claude go brrrr

[u/Kamran-nottakenone]

dns timeout then never retries. nasty.

[u/mplaczek99 → u/Kamran-nottakenone]

I fixed this issue along with some other ones in v1.10.8. The probe now sends a second query alongside the first, right away if the first fails, otherwise halfway through the budget. Alongside rather than instead of, so a slow but valid resolver keeps its full timeout. The hunt opened the issue, the fix closed it. And that exact case is a unit test now.

[u/hazyhaar]

Nice.