Can ChatGPT backtest a trading strategy?

By John, builder of Wise Apple · Updated August 27, 2026

You describe the rule, ask for a backtest, and thirty seconds later there is a table: 63% win rate, 2.1 profit factor, 14% max drawdown. It looks exactly like a real result. That is the problem.

"ChatGPT trading strategy" is one of the most searched phrases in retail trading, and the videos under it split into two camps: people who let a language model write a strategy and report astonishing returns, and people who tried that and lost money. Both camps are describing the same tool doing what it does. The gap between them is understanding what that is.

This guide is the honest answer to the question in the title. It covers what a language model can genuinely do for strategy research — which is more than skeptics allow — the specific ways it fabricates backtest results, what the peer-reviewed evidence says about LLM stock prediction, and a workflow that gets the benefit without the invented numbers.

What a language model genuinely can do

• Turn a vague idea into precise rules. "Buy pullbacks in an uptrend" becomes a definition of uptrend, a definition of pullback, an entry condition, a stop and a target. This is genuinely useful and most people skip it.
• Write the code. Given the rules and a description of your data, it will produce a working Python backtest — usually with the standard mistakes, which is covered below.
• Explain a result. Paste a report and ask what the numbers mean. It is a good tutor for concepts like expectancy, drawdown, and walk-forward.
• Review your logic. Ask it to find look-ahead bias in your rules. It will often catch the obvious cases.
• Run real code on real data, in the versions that have a code-execution sandbox, if you upload the data. The result is then a computation, not a guess — though the computation inherits whatever mistakes the generated code contains.

The four ways it fabricates a backtest

1. It has no price data

A language model does not contain a database of historical prices. It contains patterns of text. Ask it what SPY did in March 2020 and it will answer from things it has read, approximately. Ask it to backtest a strategy over ten years without giving it the data, and there is nothing to compute on. The numbers it returns are what a backtest result usually looks like in the text it was trained on — which is why they are always plausible and always slightly flattering.

2. It reports numbers it did not compute

Even when asked to "simulate" a backtest, the model produces the shape of an answer, not an answer. A 63% win rate is not a computation; it is a typical number. The tell is that the same prompt asked twice gives different results, and neither is reproducible from anything.

3. Its code makes the standard mistakes

When it does write real code, the code tends to reproduce the errors most common in the tutorials it learned from: fills at the signal bar's close, no costs, a scaler fitted on the whole series, a random train/test split. The leakage guide lists these; every one of them appears regularly in generated backtests. The code runs. It runs a flattering test.

4. It agrees with you

Language models are tuned to be helpful, and "helpful" in a conversation about your strategy means encouraging. Ask whether a result is good and the answer leans yes. Ask it to improve the strategy and it will add rules — each one an overfitting opportunity — with no sense of how many you have tried.

What the evidence says about LLM prediction

The most-cited paper is Lopez-Lira and Tang's "Can ChatGPT Forecast Stock Price Movements?" (2023), which asked the model to score news headlines as good or bad for a stock and found the scores predicted next-day returns in a backtest over late 2021 to 2022. It is a careful study, and it is about sentiment scoring of headlines — one feature, used in a tabular model — not about the model generating or testing strategies. The authors also noted the effect was concentrated in smaller stocks and would be sensitive to costs, and later work has found such effects decay once widely known.

That is the realistic ceiling: a language model as a feature generator for information that is hard to quantify otherwise, plugged into an honest tabular pipeline. Not as an oracle, and not as the pipeline.

A workflow that keeps it useful

1. Use it to write the rules down precisely, before you look at a chart. Make it ask you every question a stranger would need answered to execute the strategy.
2. Ask it to list every way the rules could use future information. Fix each one.
3. Run the backtest somewhere that actually has the data and charges costs — your own code, a spreadsheet, or software built for it. Never accept a result the model produced without data.
4. If it wrote the code, check four lines: when fills happen, whether costs are charged, where the scaler is fitted, and how the split is made.
5. Paste the real report back and ask it to explain the numbers. Do not ask whether they are good.
6. Count every variant it suggests. Each one is a trial, and the multiple-testing discount applies to all of them.

Prompts that work, and prompts that do not

The difference is whether the prompt asks for text or for a result. Prompts that ask for text — precision, critique, explanation — get the model's genuine strength. Prompts that ask for a result the model cannot compute get its genuine weakness, dressed as a strength.

Where Wise Apple fits

Wise Apple is the third step in the workflow above: the place the rules go to be tested against real data with the honest settings already on. You describe the strategy in PowerCore Studio — no code, so nothing for a language model to get subtly wrong — and the backtest runs on ten-plus years of actual candles on your own machine, with fills after the decision bar, Fees and Slippage charged, a chronological split with Embargo Bars, and the result reported on out-of-sample windows. Every trade is inspectable, and the report exports to CSV, so if you want a language model to explain the numbers, you can hand it real ones.