Trade Like a Machine · Lesson 7 of 8
Test Honestly, or Fool Yourself Slowly
The most dangerous person in your trading is the version of you that grades its own homework.
Every trader who backtests discovers the same intoxicating thing: with enough adjustments, the past can be made to pay beautifully. Tweak the stop, exclude that weird week, add one more filter — and the historical curve climbs like a staircase to heaven. What you have built is not a trading system. It is a description of one particular past, and the future does not take requests.
Honest testing rests on a handful of disciplines, and every one of them exists because we, and everyone else, got burned without it:
Pre-register your ideas. Write the hypothesis down — dated — before the data that will judge it exists. When we found a promising pattern in our research ledger ("take the top-ranked picks at the morning windows"), we wrote it down, locked the current rules, and let only future weeks vote on it. A pattern found in old data and graded on the same old data has an examiner it went to school with.
Score what you refused, not just what you took. Most traders only ever see the trades they made. Our ledger scores every refused signal at full weight, so a filter must prove it blocks losers more than winners. This is how we caught one of our own gates being worthless: a macro filter that felt essential turned out, on the refused pile, to be blocking trades that netted roughly zero — all cost, no protection. It was demoted from veto to footnote. Feelings said keep it; the refused pile said otherwise.
Respect provisional numbers. A result measured on trades still open is a compliment, not a fact. We watched a fresh variant show +149 pips in the morning and +99 by evening as its open trades finished their day. Neither number was a lie — one was just unfinished. Publish final figures; label everything else provisional, loudly.
Never delete a bad chapter. Our research phase lost money — its full ledger, over a thousand pips of scored mistakes, stays published on our site permanently, archived beneath the live record that learned from it. The moment you allow yourself one quiet deletion, every number you ever publish — including to yourself — becomes negotiable.
The research fortnight's brutal headline: the trades our old rules qualified lost −576 pips while the signals they ignored gained +1,692. The filter was worse than useless — it was anti-selecting. Painful to publish; priceless to know. Every rule in our current live system exists because that ugly number was allowed to be true. A flattering ledger teaches nothing.
The deeper principle: your test process is itself a system, and it can be corrupted by the same forces as your trading — the wish to be right, the pain of being wrong. Pre-registration, refused-pile scoring and permanent archives are to your judgement what stops are to your trades: constraints placed in advance, where the weak moment cannot reach them.