Why backtests fail in live trading

By John, builder of Wise Apple · Updated August 27, 2026

The backtest said 62% winners and a smooth curve. Three weeks of real money said something else. Nothing about the rules changed. So what did?

This is the most common story in systematic trading, and the usual explanation — "the market changed" — is usually wrong. The market changes constantly and most strategies survive it. What actually happened is narrower and more fixable: the backtest and the live run were not the same experiment, and one of seven differences between them ate the edge.

Below are the seven, ordered by how much damage each one does when it is the culprit. After that, a diagnostic order — because guessing which one it was is how people spend six months re-tuning a strategy that was never broken in the way they thought.

The seven causes, ranked by damage

1. The backtest saw the future

Leakage is first because it is the only cause that manufactures an edge from nothing. Every other item on this list makes a real edge look bigger. Leakage makes a non-existent edge look real. If a strategy was broken from the first live day — not degraded, broken — start here.

It gets in through fills at the signal bar's close, through indicators that repaint, through scaling fitted on the whole series, through labels that overlap their own features, and through you, tuning on the test period without noticing. The leakage guide has a check for each route. The fastest single test: delay every signal by one bar. If the edge vanishes, it was never yours.

2. Fills you could not have had

The backtest bought at the close. You bought at the next open, after a gap. The backtest paid no spread. You paid it twice per round trip. The backtest's limit orders always filled. Yours filled only when price went through them — which, for a limit order, means price was already moving against you.

This is the cause that most often turns a small real edge into a small real loss, and it is invisible in the equity curve because the curve's shape is right. Only the level is wrong.

3. Costs that were never charged

Related to fills but separate: commission, exchange fees, funding on perpetuals, borrow on shorts, and the slippage that scales with your size and the market's volatility. High-frequency strategies die here almost exclusively. A strategy that trades forty times a day cannot afford to be wrong about a fraction of a basis point per trade. The costs guide lists every item and the multiplier test that catches it.

4. Overfitting to one history

The rules were tuned until they fit the past perfectly, including the part of the past that was noise. Overfit strategies have a signature: they usually fail immediately out of sample rather than decaying gradually, because the patterns they learned were never patterns. The rest of the signature — parameter cliffs, profit concentrated in a handful of trades, a suspiciously smooth curve — is in is my strategy overfit?

5. Data that was quietly cleaned

Survivorship is the big one: the universe was built from what exists today, so every delisting, every bankruptcy, every dead coin was removed before the test began. But there are quieter versions. Splits and dividends back-adjusted so that price levels never existed. Sessions that were missing and silently interpolated. A 4-hour bar series where the exchange changed its bar boundary in 2021. The survivorship guide covers how to test your universe rather than trust it.

6. The market did change

It happens, and it is real — but it is sixth, not first, because regime change produces a specific pattern that the causes above do not. A strategy hit by a regime shift usually worked live for a while, matched its backtest, and then stopped. If yours was broken on day one, this is not your cause. If it ran well for four months and then went flat, it might be — and why trading strategies stop working is about telling regime change apart from the other explanations.

7. You did not run the system

Last because it is embarrassing, not because it is rare. You skipped an entry because it "felt wrong". You held a loser past the stop because it was "about to turn". You sized up after a winning streak. The backtest ran the rules. You ran a different strategy that used the rules as suggestions, and that strategy has never been tested.

Symptom to likely cause

Each cause leaves a different fingerprint. Before you touch a parameter, match what you saw to the table:

The diagnostic order

The expensive mistake is starting at the model. The cheap discipline is starting at the trades, because most of the causes above show up as a mismatch between what the backtest did and what your account did, and that mismatch is a list you can read line by line.

1. Reconcile the trades. Put the backtest's trade list next to your broker or exchange statement for the same period. Same entries? Same exits? If the lists differ, the problem is upstream of the strategy — fills, data, or you — and nothing about the rules needs to change yet.
2. Reconcile the prices. For the trades that match, compare fill prices. A consistent gap in your disfavour is spread and slippage, and its size tells you how much of the edge it consumed.
3. Rerun the backtest at higher costs. If 1.5× kills it, stop here; the strategy was a cost assumption.
4. Delay every signal one bar and rerun. If the edge disappears, it was leakage. This one test rules out the most damaging cause on the list in under a minute.
5. Shuffle the labels and retrain. If a model trained on nonsense scores nearly as well as your real model, the real model learned nothing.
6. Walk it forward. Re-tune on rolling windows and grade each following window. If the graded windows look nothing like the tuned ones, it was overfit. If they look fine until recently, it was regime.
7. Only now, look at the model. If every step above passed, the strategy is probably real and the market probably did change. That is a different problem with a different fix.

What a survivable result looks like

Strategies that hold up live share a shape. The out-of-sample curve looks like the in-sample curve, only worse — not different, worse. Profit is spread across many trades rather than concentrated in three lucky ones. Nudging any parameter changes the result a little, not a lot. Costs at 2× hurt but do not kill. And the strategy has already been through a stretch you would have hated, and you know exactly how deep that stretch went, because you counted.

That last part matters more than it sounds. Half of the strategies abandoned live are abandoned inside a drawdown the backtest predicted. The trader saw the number and did not believe it applied to them.

How Wise Apple approaches this

Most of the seven are settings, and in Wise Apple they default to the honest side. Fills are scheduled for the bar after the decision. Fees, Slippage and Market Impact are charged on every simulated trade. The train/test split is chronological with Embargo Bars between the halves, and the Walk-Forward Window grades the strategy on windows it never trained on. Every trade in the report is inspectable candle by candle, which is what makes step one of the diagnostic — reconciling the trades — a five-minute job instead of a spreadsheet weekend. It does not place trades and it cannot fix cause seven; that one is still yours.