Live product
Mayhem Monkies
mayhem.feedback
A QA department, rebuilt as infrastructure.
Mayhem Monkies is twelve specialist agents — auth, injection, business logic, API contracts, race conditions, performance, static analysis, active testing, cloud config — plus a fixer who chains their findings into attack paths. They aren't API calls. Each one is a real coding CLI driven inside its own tmux session, and every hunt is run across three different vendors' models on rotation, so you get three independent readings of the same target rather than one model's blind spots repeated eight times.
The hard part was never finding bugs. It was letting an agent run --dangerously-skip-permissions against a hostile target on a box that also holds Stripe keys and other customers' private repos. So the constraints are kernel-level, not prompt-level. Hunters run in a network namespace where every internal range is hard-dropped by iptables and the only route out is a proxy that pins the screened IP against DNS rebinding and sends anything off-target to a fail-closed exfiltration judge. They run inside a bubblewrap sandbox that masks the secrets directory, the database, other tenants' data, other providers' credentials, and the shared tmux socket — and fails closed, showing an empty world, if its scoping inputs look wrong. Concurrency isn't a tuning knob; it's the live sum of un-rate-limited credential capacity, so the fleet backs off and scales up on its own.
Then there's the honesty problem. A bug hunter that cries wolf is ignored within a week. Every bill needs a reproduction; a static-analysis finding gets routed by bug class to the black-box hunter who owns that class, who has to confirm it from the wire; and when hunters started citing blockchain transaction hashes as evidence, we shipped a verifier that checks each one against mainnet and rewrites the bill to say the signature appears fabricated. The best proof is the repo itself: the crew was pointed at its own product, filed a gzip decompression bomb, a timing oracle on login, a client-trusted authorization flag and a cross-agent prompt-injection path, and the fixes are commits — with the hunter's name in the code comment.
// Technical highlights
- Kernel-enforced egress: agents live in a netns where RFC1918, Tailscale, loopback and cloud-metadata ranges are iptables
DROP, and the only way out is a proxy that judges every off-target request with a fail-closed LLM classifier. - 253-line bubblewrap sandbox that fails closed on a malformed hunt slug, masks the shared tmux socket (one
capture-panewas another customer's whole transcript), and caps its own mask tmpfs so it isn't an OOM primitive. - Automated hallucination catcher: walks every bill, checks cited Solana signatures against mainnet with full-history search, and rewrites
Confidence: validated→claimed-unverified (signatures appear fabricated). - Every hunt runs on three vendors' models in rotation; per-model concurrency is the live sum of un-cooled account capacity, so a rate limit auto-backs-off and a new account auto-scales up. No copied OAuth tokens, ever.
- 35-entry bug-class routing table sends each static-analysis finding to the black-box specialist who owns it for wire confirmation; the result carries a
Validates:cross-reference the UI renders as a green tick. - Compute follows signal: a 60-90s teaser sets each hunter's deep-turn budget by severity, and siege mode keeps extending a hunter while its bill count rises, stopping the turn it goes dry.
- 38 of 284 commits were authored by the hunters themselves — including a 1007x gzip amplification DoS and a 2.8x login timing oracle, both found in this product.