We all say the same thing about overfitting: the strategy was "fitted to the noise."
So test that directly. Change the noise and see if the strategy survives.
This is the entire idea behind the Noise Test.
Your price history has two parts. Signal, which is the part that repeats. Noise, which is the part that happened once and will never happen the same way again. Every strategy you build is fit to both, and your backtest cannot tell which one it learned.
So change the noise and re-run the strategy.
Here's how it works:
Pick how much of the data to adjust and by how much. Say 50% of the bars, by up to 25% of average true range. Opens, highs, lows and closes get nudged up or down inside that band. Same market, same trend, same volatility (and clustering) but a different roll of the dice.
Create 1,000 noise adjusted data series and re-trade the strategy 1,000 times.
Now you have a distribution instead of a single backtest.
How to read it:
PASS The noise adjusted results do not deviate from your backtest. Most (or all) stay profitable and the distribution sits meaningfully above zero. Your rules were tracking something real.
FAIL The noise adjusted results fall off a cliff. Your original backtest sits way above the distribution and the rest cluster near or below zero. No edge. You fit the one path history happened to take.
If adjusting the noise erases your edge, your edge was never there, and no amount of out-of-sample data would have told you, because your out-of-sample sample data has exactly the same problem: it is one path only.
Reminder: Passing is not proof of edge. It rules out one specific failure and there are plenty of others, which is why this test is only one of a dozen tests (all automate-able) in Build Alpha.
Build Alpha can also optimize your parameters across the noise-adjusted series instead of the single historical one, so you are selecting parameters that work across many versions of history rather than the exact history we lived.
Your price history is one sample. Your strategy should work on the others too!