New Princeton + Sentient Labs paper shows a coding agent can cost 40x more per solved task by changing only the harness around it, leaving the model untouched.
and controlled swaps show those scaffold differences barely move accuracy.
that the same model passes about the same tasks on any harness, so a leaderboard score does not have to mean the cost you will pay.
The problem is that leaderboards rank by model name while leaving the scaffold undisclosed. The agent may keep taking turns, but those turns can stop editing files or running commands and just burn context.
The paper fixes this by holding model, prompt, sandbox and task set constant while varying only the harness, then reporting tokens per solved task, idle turns and failure mix next to pass rate.
This lets a developer pick the harness and model pair that fits token and latency budget.