The Backtest Trap · Lesson 1 of 5
The Seductive Curve
A backtest is not evidence that a strategy works. It is evidence that a strategy can be made to have worked.
Load ten years of price data, test an idea, adjust it, test again. Two evenings later a beautiful equity curve climbs across your screen — steady, smooth, victorious. Here is the uncomfortable truth about that curve: its beauty is evidence of your editing, not the market's generosity.
The mechanism is called overfitting, and it needs no dishonesty — only iteration. Every time you tweak a parameter and re-run, you are not testing a strategy against the past; you are selecting for whatever combination best matches the past's accidents. Historical data contains real, repeatable structure — and mountains of one-off noise. A strategy fitted hard enough absorbs the noise, and noise, by definition, does not repeat. The curve is a portrait of a decade that will never happen again.
The tell is degrees of freedom. Count your adjustable choices: indicator settings, thresholds, session filters, stop and target sizes, "and skip December". Each is a dial, and with enough dials any dataset can be fitted perfectly — statisticians joke that with four parameters they can fit an elephant. A strategy with nine tuned conditions did not discover nine truths about markets; it made nine concessions to one particular history.
Two questions expose an overfit curve in seconds:
"Why does this rule exist?" A robust rule has a market reason (news moves prices; liquidity dries at rollover). An overfit rule has a data reason ("removing Tuesdays improved the Sharpe"). If the only justification is that it helped the backtest, it IS the backtest.
"What happens if I nudge it?" Change the 14 to a 15, the 40 pips to 38. A real edge degrades gently; an overfit one falls off a cliff, because the exact values were the fit. (Lesson four turns this into a formal tool.)
Our research phase produced exactly one seductive discovery: an exit combination showing +151 pips — the best cell in the whole grid — which happened to involve trading with no stop-loss at all, on a sample that contained no catastrophe. Perfectly fitted to a past with no disasters in it. We refused it, in writing, and the reasoning is on our site. That refusal is this whole course in miniature.