How to backtest a trading strategy without fooling yourself

By John, builder of Wise Apple · Updated August 27, 2026

You have a rule you half-believe in. Buy the dip when RSI is under 30, sell when it crosses 70 — or whatever yours is. You have tried it on a few charts, it looked good, and now there is a number in your head that says it works.

That number is the most expensive thing you own. Not because it is wrong — it might not be — but because you got it the way everyone gets it: by looking at the past with the answer already showing. A backtest done that way does not test the rule. It tests your ability to fool yourself, and you will pass.

This guide is the whole loop, start to finish: how to turn the rule into something a computer can run, where to get data that does not lie, which costs to charge yourself, how to split time so the test never sees the future, and how to read the report when it comes back. No code is required. Skepticism is.

What a backtest is, and what it is not

A backtest is a simulation. You hand a set of rules to a program, point it at a stretch of history, and it pretends to trade — buying and selling exactly when the rules say, at the prices that were available, paying the costs a real account would pay. At the end it hands you a report: how many trades, how many won, how much was made or lost, how deep the worst stretch went.

That is all it is. It is not a prediction. It is not proof. It is one run of one set of rules over one history, and the entire skill of backtesting is arranging that run so the result is allowed to be bad. If your process cannot produce a bad result, it cannot produce a meaningful good one either.

Step 1: Write the rule down before you look

Most bad backtests are lost before the first candle loads, because the rule was built by staring at a chart until something fit. That is not a hypothesis. That is a memory of what already happened, written as if it were a forecast.

So write it down first — every part of it, in words a stranger could execute without asking you a question:

• Market and timeframe. BTC on 4-hour bars. SPY on daily bars. One instrument, one interval, to start.
• Entry. The exact condition, evaluated on a closed bar. "RSI(14) closes below 30" is a rule. "Looks oversold" is not.
• Exit. All of them: the target, the stop, and the give-up ("if neither hits in 20 bars, close"). A strategy with no exit rule has a hidden one — you, panicking — and the backtest cannot model that.
• Size. How much per trade, and whether it scales with volatility or account size.
• Direction. Long only, short only, or both.

If a part of the rule is missing, the backtest will fill it in with the most flattering default it has. Do not let it. And once the rule is written, resist the urge to peek at a chart and "just check" — every peek is a small optimisation you will forget you made.

Step 2: Get data that can say no

Data quality is not a detail; it decides what the backtest is capable of telling you. Four problems come up constantly, and each one pushes the result in the flattering direction:

• Survivorship. If your universe is "stocks in the S&P 500 today" or "the top 50 coins by market cap now", every failure has already been removed for you. Whatever you test will look better than it would have at the time. The survivorship guide covers how to check for it.
• Adjustments. Stock data is usually adjusted for splits and dividends after the fact. That is correct for returns and wrong for price levels — a rule like "buy under $50" can fire on prices that never existed.
• Gaps and holes. Missing hours in crypto data, missing sessions in equity data. A moving average across a gap is a moving average of the wrong bars.
• Timezone and session alignment. A daily bar that closes at midnight UTC is a different bar from one that closes at 4 p.m. New York. Mixing them creates signals no live trader could have acted on.

Ten years is a reasonable minimum for daily bars, because it usually contains at least one stretch you would not have wanted to live through. Where to get it, and the catch attached to every free source, is in the data guide.

Step 3: Decide the fill, then the costs

This is where honest backtests and flattering ones part company, and the difference is usually one setting.

When does the trade happen?

Your rule is evaluated on a closed bar. In real life, you can act on it no earlier than the open of the next bar. A backtest that fills you at the close of the bar that generated the signal has given you a trade at a price you could not have had — and on daily bars, that is a full day of hindsight per trade. Set the fill to next-bar open, always. If your tool does not let you, that tells you something about the tool.

What does the trade cost?

Charge yourself for all of it: the bid/ask spread (you buy at the ask and sell at the bid, so a round trip costs one spread before anything else happens), commission or exchange fees, slippage (the price moves between your decision and your fill, and it moves against you more often than not), and, for shorts or perpetuals, borrow cost or funding.

Then run the test again at 1.5× and 2× your cost estimate. If the strategy is fine at 1× and dead at 1.5×, you did not find an edge. You found a rounding error in your cost model. The costs guide walks through each item and how to estimate it.

Step 4: Split time so the test cannot see the future

Here is the mistake that makes machine-learning backtests in particular look miraculous: the same history that chose the parameters is the history that graded them. Of course it scored well. It was marking its own homework.

The fix is to split the timeline. Tune on the earlier part, grade on the later part, and never let the grading part influence a decision. Three refinements make this hold up on market data:

1. Split by time, never at random. A random shuffle puts Tuesday in the training set and Wednesday in the test set, and Wednesday looks almost exactly like Tuesday. The test is contaminated before it starts. The out-of-sample guide explains why this matters more for trading data than almost anywhere else.
2. Leave a gap. If your labels look ahead N bars ("did price rise over the next 10 bars?"), the last training rows overlap the first test rows. Put an embargo of at least N bars between them.
3. Roll it forward. One split is one opinion. Walk-forward analysis re-tunes on a sliding window and grades each following window, then stitches the graded pieces into a single out-of-sample curve. It is the closest thing to a rehearsal of live trading that a backtest can give you. See walk-forward analysis.

Step 5: Check for leaks

Even with a clean split, information from the future can get into the past through side doors. An indicator that recomputes when new bars arrive. A normalisation step fitted on the whole series. A "volatility" feature that quietly includes the current bar's close. None of these look like cheating. All of them are.

The tell is direction. Leaks do not add noise; they add skill. If your result is suspiciously good and gets better the more you refine it, look for a leak before you look for a reason. The leakage guide lists the six routes it takes and a test for each, and the repainting guide covers the one that lives inside indicators.

Step 6: Count the trades before you believe the numbers

A 70% win rate on 20 trades is not a 70% win rate. It is a coin that came up heads 14 times, which happens to fair coins more often than people think. Every statistic in the report has an error bar, and the error bar shrinks slowly — roughly with the square root of the trade count, which means going from 25 trades to 100 only halves it.

Worse, trades are not independent. Twenty trades in one bull run are one observation of one regime. Count trades per market condition — rising, falling, sideways, high volatility, low — and if any bucket is nearly empty, you have not tested that condition at all. The sample size guide puts numbers on this.

Step 7: Read the report in the right order

The report will lead with whatever number is largest. Read it in this order instead, and stop at the first failure:

Notice that win rate is not on the list until you already know the payoff per trade. A 90% win rate with tiny wins and one catastrophic loss is a losing strategy, and it will look like a great one right up until the loss. The win-rate myth has the arithmetic; how to read a backtest report covers every row above in detail.

Step 8: Try to break it

A result you have not tried to destroy is a result you do not understand. Four cheap attacks, in order of how often they succeed:

1. Nudge every parameter. If RSI 30 works and RSI 28 and 32 do not, you found a spike, not a plateau. Real edges are blunt.
2. Shift the signal one bar later. If performance collapses, the strategy depended on acting at a moment you could not have acted.
3. Multiply costs. Covered above. Do it anyway.
4. Shuffle the labels. Randomise which bars were "up" and which were "down", retrain, and see what score a model gets on nonsense. If your real score is not clearly above that, you have nothing.

If it survives all four, run the full checklist in is my strategy overfit? — the seven signs there are the ones that get past the first pass.

Step 9: Rehearse it live without trading it

The last test is time. Freeze the rules, stop touching them, and let them run on data that did not exist when you built them. This is the only holdout that cannot be contaminated by you, because you cannot peek at bars that have not happened yet.

You do not have to trade it to do this. Have the rules generate signals — alerts to your phone, a log, anything with a timestamp — and compare what they would have done to what the backtest said they would do. If the live signals and the backtested signals agree and the outcomes are in the same neighbourhood, you have something. If they disagree, why backtests fail live is the diagnostic order for finding out why.

What this looks like in Wise Apple

Wise Apple was built around this loop rather than around the equity curve. You describe the rule — market, entry, exit, sizing — in PowerCore Studio without writing code, and the guardrails are on before you run anything: fills happen after the decision bar, a Train/Test Split by time with Embargo Bars between them, a Walk-Forward Window that rolls the out-of-sample test forward, Train-Only PCA so scaling never sees the test period, and Fees, Slippage and Market Impact models charged on every fill. The report leads with trade count, expectancy and a buy-and-hold comparison, and every trade is inspectable candle by candle. Training runs on your own computer, so the twentieth attempt costs the same as the first: nothing.

It is research software. It does not place trades, and no backtest — here or anywhere — tells you what happens next. What it does is make the honest version of the test the default, so the number in your head has to earn its place.