LSTM stock prediction: why the tutorial curve is a lie

By John, builder of Wise Apple · Updated August 27, 2026

The notebook took forty minutes. The predicted price tracks the real price so closely you have to zoom in to see daylight between them. You are one step from trading it — and the step you are missing is the one where you check what it actually predicted.

Long short-term memory networks — LSTMs — are a kind of recurrent neural network built for sequences, and "predict stock prices with an LSTM" is one of the most-followed tutorials in applied machine learning. The results always look the same: a plot with two lines, real and predicted, almost on top of each other. And the live results always look the same too, which is why the forum threads exist.

This guide explains exactly what that plot is showing, the two separate ways the tutorial cheats without meaning to, why the problem is structural rather than a matter of tuning, and what to do with the forty minutes instead.

What the curve is actually showing

Take a price series and plot it against itself shifted one day to the right. The two lines will hug each other almost perfectly, because tomorrow's price is nearly always close to today's. That is not prediction. It is autocorrelation, and it is the entire content of most LSTM tutorial plots.

The network is asked to predict tomorrow's close from the last sixty closes, and it learns the easiest thing that minimises the error: output something very near the most recent close. The resulting "prediction" lags the real series by one step, matches it closely on every error metric, and contains no information about whether the next move is up or down — which is the only thing a trade needs.

The second cheat: the scaler saw the future

Almost every tutorial normalises the price series to 0–1 before splitting it into training and test sets. That means the scaler's minimum and maximum were computed over the whole series — including the test period. Every training example now carries a piece of information about where prices went later: the range they will span. A model trained on that has been told the future's envelope, and it will look better on the test set than it has any right to.

Fit the scaler on the training window only, apply it forward, and watch the test-period fit degrade. This is the global-scaling leak, and it is not specific to LSTMs — but LSTM tutorials are where most people meet it.

Why it is structural, not a tuning problem

Suppose you fix both cheats: predict the direction of the next move instead of the price, and scale causally. The network now faces the real problem, and it is badly equipped for it:

1. Not enough data. Ten years of daily bars is about 2,500 rows. A modest LSTM has tens of thousands of weights. There is nothing to constrain them, so the network memorises the training period and generalises to nothing.
2. Too much noise. The next day's direction is mostly unpredictable. A model powerful enough to find subtle structure will find subtle noise with equal enthusiasm, and cannot tell them apart on this little data.
3. Non-stationarity. Whatever relationship held in 2016 has drifted by 2022. Sequence models trained end to end on the whole history learn an average of regimes that no longer exist.
4. The tabular mismatch. The features that carry information about direction — momentum, volatility, regime, calendar — are a table, not a sequence. Tree models are built for tables.

That last point is the general finding behind the specific failure. Grinsztajn, Oyallon and Varoquaux (2022) showed tree-based models still beat deep learning across a wide range of tabular datasets, and market data is tabular, small and noisy — the conditions under which the gap is largest. The model families guide covers what each family is actually good for.

Can deep learning ever work here?

In narrow settings, with resources most people do not have: order-book data at millisecond resolution, where there are millions of rows and genuine sequential structure; alternative data — text, images — where deep learning's strengths apply and the output becomes one feature in a tabular model; or very large cross-sections of assets treated jointly. None of these is "an LSTM on ten years of daily closes", and a tutorial that suggests otherwise is selling the plot, not the method.

What to do with the forty minutes

1. Change the target from price to direction — or better, to which of a target, stop, or time limit a trade would hit first. The triple-barrier guide explains why.
2. Build a table of features computed only from closed bars: momentum, realised volatility, a regime measure, calendar effects.
3. Split chronologically with a purge gap. Fit every scaler on the training window alone.
4. Train a regularised linear model. Record its out-of-sample precision and MCC. That is the floor.
5. Train a regularised boosted tree. If it does not beat the floor, the features have no signal, and no network will find one.
6. Charge costs, delay fills to the next bar, and read the report in order.

The result will be less impressive than the LSTM plot. It will also be a number you can trade on, which the plot never was.

A minimal honest comparison you can run

If you already have the tutorial notebook, three changes turn it into a test that can say no:

1. Replace the target. Instead of tomorrow's close, label each bar +1 if the close two bars later is above today's close by more than a small threshold, −1 if below, 0 otherwise. The threshold should be a fraction of recent volatility, not a fixed percentage.
2. Add the baselines. Before training anything, compute the precision of "always predict the majority class" and of a logistic regression on the same features, both on the test window. Write them down.
3. Move the scaler. Fit it on the training rows only, transform the test rows with those parameters, and never touch the test window again.

Then train the LSTM and read its test-window precision and MCC next to the two baselines. In our experience the network lands within noise of the logistic regression and both land within noise of the base rate, which is the honest result: on this data, at this resolution, there was not much to find. The plot that started the exercise was never evidence of anything else.

Why Wise Apple has no neural network on the bench

Wise Apple's nineteen models are trees, forests, boosted ensembles, nearest-neighbour classifiers and an SVM — the families that earn their place on tabular market data — and there is no LSTM among them, for the reasons above. The targets are direction and triple-barrier outcomes, never price levels; Train-Only PCA and feature scaling are fitted on the training window; and every model is reported on precision and MCC out of sample, next to the trade count, so a model that has learned to copy the base rate is visible for what it is. If a network ever beats the boosted trees on that test, it will get a seat. It will have to earn it the way the others did.