AI writing useless tests is a context problem, and it's mostly fixable.
β The core issue:
You cannot trust tests produced in the same context that produced the code.
With the implementation in context, the session is conditioned on the very thing it's supposed to check. So the most likely token sequence for the test is a description of what the code does, not what it should do. The result is a self-confirming test that checks nothing.
The obvious answer to that is subagents. But if you look at how harnesses actually launch them, turns out that's not really a viable solution.
Subagents still inherit the AGENTS.md / CLAUDE.md, the tools, the skills, and a prompt written by the context you're trying to escape. If you look closely at the default prompts harnesses like Claude Code launch their subagents with, it's usually layers of text and tightening definitions.
So you contaminate the context of a subagent anyway through the same-harness layer. Not a clean solution.
What actually works is graph engineering: a pipeline of minimal, isolated nodes, each getting only the data it needs and nothing more.
My setup:
β’ Minimal harness (I use Pi)
β’ CLI invocation
β’ Orchestration logic on top
A quick example with e2e test generation and a simple three-step graph:
0. PRD βΒ 1. User Flows βΒ 2. Tests
Node 1 sees only the PRD. Its one job is to invent the user flows that will actually happen when this ships, edge cases included. No code and no prior PRD-generation context. The flows get reviewed by a human (or an LLM judge, extra node) and fixed here.
Node 2 gets the flows along with the codebase, and writes the corresponding tests. It reads the code, but what to test was already decided upstream by a node that never saw the code. So there's nothing (or at least much less) left to cause self-consistency bias.
Then you get tests that check real behavior instead of confirming the code.
The name of the game is isolation. Each node should be minimally scoped and blind to the others' reasoning.
Want sharper tests? Add more nodes to mirror your architecture, and keep each new node just as blind.
ai writes so many useless tests