Backtesting Without Fooling Yourself

A backtest is the easiest thing in trading to get wrong while feeling completely confident. With enough tweaking, almost any idea can be made to look brilliant on historical data. The skill is not producing a great-looking curve — it is producing one you can actually believe.

Overfitting: the big one

If you keep adjusting settings until the past looks perfect, you have not found an edge — you have memorised history. The more parameters you tune and the more you optimise, the more likely the result is a coincidence that will not repeat live.

• Prefer fewer parameters and robust, round-number settings.
• Test on data the settings were not tuned on (out-of-sample).
• Be suspicious of results that look too good.

Look-ahead bias

This happens when a test accidentally uses information that would not have been available in real time — like acting on a candle before it closed. It produces impossible results that vanish the moment the bot goes live.

Realistic costs

Spreads, commissions and slippage are real and they add up. A backtest run with zero spread is a fantasy. Always test with realistic spreads and, ideally, modelled slippage so the numbers reflect what a broker would actually have done.

Forward-test on demo

The honest final step is a forward test: run the bot on a demo account, in real time, for long enough to see it handle real conditions. Past performance and demo results never predict the future — but a forward test catches problems a backtest simply cannot.

The bottom line

Backtesting is for disproving ideas, not for selling them to yourself. Treat a clean, realistic, out-of-sample result as a reason to keep testing — never as a guarantee.