Your AI backtest may have read tomorrow's newspaper: lookahead bias, explained

Ask an AI model to backtest a strategy on 2019 to 2022 data and it can produce a beautiful equity curve. The problem is that the model may already know how 2019 to 2022 turned out. It was trained on years of news, filings and market commentary written after those dates. This is lookahead bias, and in AI trading research it has moved from a footnote to a central issue.

Here is what the published evidence says.

1. Models remember exact market numbers

In "The Memorization Problem" (arXiv, first version April 2025, revised December 2025), Alejandro Lopez-Lira, Yuehua Tang and Mingyin Zhu of the University of Florida show that GPT-4o, before its October 2023 knowledge cutoff, can recall specific S&P 500 index levels with perfect precision on certain dates, unemployment rates to a tenth of a point, and precise quarterly GDP growth figures. After the cutoff, they observe no recall.

Their theoretical point is the important one. When a model has seen the realized values, its forecasting skill cannot be identified from its outputs: any good forecast is consistent with both genuine skill and memory. And any test that finds memorization only gives a lower bound, since a different prompt may unlock more.

2. Telling the model to forget does not work

The same paper tested the most common fix: instructing the model to ignore everything after a date. The authors told GPT-4o to use no information after 2010 and asked it to call whether quarterly U.S. GDP growth would exceed 2.5%, over March 1990 to June 2023. Accuracy was 97.6% before the artificial cutoff and 98.0% after it (observation counts and intervals not restated in the passage). On real post-cutoff data, accuracy was only 40%. In plain terms, the model ignored the instruction and used what it knew. nContext: n: quarterly observations not stated · Mar 1990 to Jun 2023 · Lopez-Lira, Tang and Zhu, arXiv:2504.14765 · margin: not disclosed

Masking company names and dates did not fully solve it either: the authors find that models can reconstruct the entity and the period from minimal context.

3. Sometimes names hurt more than memory

Paul Glasserman and Caden Lin of Columbia University tested a sentiment strategy on news headlines with GPT-3.5 (arXiv, September 2023). They used two datasets: 129,431 scraped headlines on 6,723 companies, and 181,908 Thomson Reuters headlines on 678 S&P 500 companies. They compared original headlines with versions where the company's name was removed.

The surprise: inside the model's training window, the anonymized headlines performed better. The authors call this a distraction effect. General knowledge about a famous company interferes with reading the sentiment of the actual news, and the effect was strongest for large companies. Out of sample, they found lookahead bias was not a concern, while distraction could remain. Their suggestion: strip company identifiers before asking the model, both to test for bias and possibly to improve live signals.

4. A test that catches contamination

Zhenyu Gao, Wenxi Jiang and Yutong Yan of CUHK Business School built a cheap diagnostic (arXiv, first draft December 2025, revised June 2026). They send the model only a firm name, ticker and date, with no news at all, and measure how confident it is about what happened next. They call that score Lookahead Propensity.

Their example is memorable. When Llama-3.3-70B received only Kodak's name, ticker and the date July 29, 2020, it put a probability of about 0.9999 on the stock going up. That was the day after Kodak announced a $765 million U.S. government loan, when the stock rose roughly 318%. No headline was needed: the model remembered. nContext: n=1 firm-date (Kodak, Jul 29, 2020) · Llama-3.3-70B date-only query · Gao, Jiang and Yan, arXiv:2512.23847 · margin: not applicable

Across two tasks, news headlines predicting returns and earnings calls predicting capital spending, the model's forecasts looked most powerful precisely on the firm-dates it was most likely to remember, and that extra power lost significance after the model's December 2023 training cutoff.

5. Statistics alone will not save you

An August 2026 preprint by independent researcher Eray Gençay adds a warning for anyone who thinks overfitting tests are enough. He planted a deliberately leaky strategy, one with access to future information, with a Sharpe ratio of 35. It passed the Deflated Sharpe ratio and probability-of-backtest-overfitting tests completely. Those tools correct for searching through many strategies; they cannot detect data from the future. nContext: n=453 stocks and 39 ETFs, up to 100 candidates, 5 runs · held-out windows of 4 and 9 years · Gençay, arXiv:2608.27734 · margin: not disclosed

Once both leakage and the number of strategies tried were controlled, his framework certified passive benchmarks and rejected every strategy the language models discovered, across two frontier models, search budgets of up to 100 candidates and five repeated runs, on a 453-stock U.S. universe and a 39-ETF universe with realistic costs.

A checklist before trusting an AI backtest

• Check the dates. Was the entire test period after the model's training cutoff? If not, the result cannot separate skill from memory.
• Do not rely on prompts. "Ignore information after X" did not work in the published test.
• Try anonymizing. Remove names and tickers and see whether results change.
• Ask how many strategies were tried. The best of 100 backtests is not a forecast.
• Look for live or post-cutoff evidence. Papers that use post-cutoff samples, such as the ChatGPT headline study, are built to avoid this trap.