Is my strategy overfit? Seven signs of curve fitting, and four tests that settle it

By John, builder of Wise Apple · Updated August 27, 2026

Every time you optimise, the curve gets prettier. Every time you trade it forward, it gets uglier. You already suspect what that means. Here is how to stop suspecting and find out.

Overfitting — curve fitting, in the older trading vocabulary — is what happens when a strategy learns the noise in one stretch of history as if it were signal. It is not a bug in your tool. It is what any optimiser does when you give it enough freedom and one dataset: it finds the settings that would have been perfect for that dataset, which is a different thing from the settings that will be good for the next one.

The frustrating part is that an overfit strategy looks exactly like a great one. Same smooth curve, same high win rate, same confident report. The differences are structural, and you have to go looking for them.

Seven signs

1. The parameter surface is a cliff

RSI 30 works. RSI 28 and RSI 32 do not. A real edge is blunt — it comes from something structural in how the market moves, and structure does not care whether you measured it with a 14-period or a 15-period window. A spike in the parameter surface means the setting was chosen to fit a handful of specific bars. Plateaus are what edges look like.

2. Too many rules for too few trades

Each parameter, each filter, each exception is a chance to memorise. With enough of them, any history can be fit exactly. The question is not "how many parameters is too many" — it depends on how much data constrains them — but every rule you add should be able to justify itself on a fresh window, and if it cannot, it was fitting noise.

3. Excellent in sample, poor out of sample

The classic signature. The tuned window looks wonderful; the graded window looks like a different strategy. Some degradation is normal — the in-sample result was selected, so it is biased upward — but a collapse, or a change in character (different win rate, different trade length, different drawdown shape), means the rules learned something that was only ever true once.

4. Profit concentrated in a handful of trades

Remove the best 5% of trades and see what is left. If the strategy goes from excellent to break-even, it did not have an edge; it had three lucky trades, and the optimiser found the settings that caught them. A real edge is spread thin across many trades, because that is what a small, persistent bias in your favour looks like.

5. Small data changes swing the result

Start the backtest one week later. Drop a random 5% of bars. Use a different data vendor. If the result moves a lot, the strategy is sensitive to individual bars, which means it is sensitive to noise.

6. Every rule has a story about one bad stretch

"I added the volatility filter because of March 2020." "The time-of-day rule is there because of that week in 2022." Each fix was for one event. Together they are a list of events the strategy has been taught to avoid, and the next event will not be on the list.

7. The equity curve is too smooth

Markets are noisy and a strategy that trades them should have a noisy equity curve. One that glides upward with tiny, brief dips has either found something extraordinary or has had its history sculpted. The second is more common by a wide margin.

Four tests that settle it

Signs raise suspicion. These answer it, and they escalate in cost — the first takes minutes, the last takes months.

Test 1: Parameter sensitivity

Take each parameter and run the backtest at the values around your chosen one — two steps down, two steps up. Plot the results. You are looking for a plateau: a region where every nearby value is about as good. A single spike means overfit. A sawtooth, where neighbouring values alternate good and bad, means the parameter is meaningless and the good values are coincidence.

Test 2: Walk-forward

Re-optimise on a rolling window, grade on the following window, repeat, and stitch the graded windows together. This tests whether your process for choosing parameters keeps working, not whether one set of parameters was lucky. It is the standard tool for this question and the walk-forward guide covers how to set it up and — importantly — how you can overfit the walk-forward itself.

Test 3: Permutation

Shuffle the outcomes so that the relationship between your features and the result is destroyed, then run the whole optimisation again on the shuffled data. Do this many times. Each run produces a "best" result found on pure noise. If your real result does not sit well outside that distribution, your optimiser can find results as good as yours in data that contains nothing.

This test is the one that most people skip and most need, because it measures the thing that actually matters: how good a result your search procedure produces when there is nothing to find.

Test 4: The sealed holdout

Keep a final stretch of history — the most recent year, say — that you never open until the strategy is frozen. Not "barely looked at". Never opened. Then open it once. Whatever it says is the answer, and you do not get to change the strategy and look again, because the second look makes it in-sample.

The only holdout stricter than this is time itself: freeze the rules and let them run on bars that do not exist yet.

What to do when the answer is yes

Do not tune harder. That is what caused it. Instead:

1. Remove rules. Take out every rule that exists because of one event. The strategy will look worse in sample and usually better out of sample.
2. Widen the parameters. Move from the peak to the plateau. Use round numbers. If the plateau does not exist, the parameter should not either.
3. Get more trades. More history, more instruments, a shorter timeframe if the idea supports it. Overfitting is a ratio of freedom to data, and adding data is the honest way to change it.
4. Re-derive the idea. Ask what the strategy is supposed to be exploiting, in one sentence. If you cannot say, the optimiser was never looking for anything in particular, and it will find noise every time.

And count the attempts. The sample-size guide explains why the best of thirty strategies is not a strategy, and the leakage guide covers the case where a result is too good for even overfitting to explain.

How Wise Apple approaches overfitting

Wise Apple's defaults are built around tests 2 and 4. The Walk-Forward Window rolls a chronological train/test split forward with Embargo Bars between the halves, so the headline result is a stitched out-of-sample curve rather than an in-sample fit. Models are scored on precision, recall and MCC on the test windows, and the 1–100 Score is capped for any configuration that trails simply holding the asset — an overfit strategy that beats nothing gets told so. Because training runs locally, running the parameter neighbourhood is free, which is the only way most people will actually do test 1.