AI earnings-call summaries can flip the bull or bear read. Here is how often, and how to prompt around it

Summaries are the most common way investors now meet earnings: AI digests in broker apps, a 'summarize this call' prompt in a chatbot. A summary can be accurate sentence by sentence and still leave you with the wrong overall picture. The paper When Summaries Distort Decisions, by 18 authors, first posted June 28, 2026 and revised September 17, 2026 (version 3), set out to measure exactly that. It is listed for the EMNLP 2026 Industry Track.

How they measured it

The authors took 300 10-Q MD&A sections and 297 earnings-call transcripts from S&P 100 companies for fiscal 2025 Q1 to Q3. A 'decision model' (Gemini-3.1-Flash-Lite or Claude-Haiku-4.5) read each full document and produced a bear, neutral or bull view. Then a compressor model summarized the document, the decision model read only the summary, and the authors counted how often the top view flipped. The key control: the same decision model rereading the full original, with no summary, to measure plain model randomness.

The results

The plain summaries ran to roughly 4% of the original length (4.1% for MD&A, 4.6% for calls) nContext: n=597 documents (300 MD&A, 297 calls) · fiscal 2025 Q1 to Q3 · arXiv:2606.29251 v3, Table 2 · margin: not disclosed, and every simple compression method flipped the read more than twice as often as rereading did. Pruning methods that just drop low-information words did worst, flipping about half the reads under the Gemini decision model nContext: n=597 documents · fiscal 2025 Q1 to Q3 · arXiv:2606.29251 v3, Table 4 · s.d. 0.7 to 1.9 across decision runs.

One footnote on how numbers drift: a September 6 summary of this paper by Prism Data Lab quotes the Gemini-only figures (33.0% and 23.9%) and a reread floor of 11.0% and 8.8%. The current version reports the averaged figures above and a lower floor nContext: n=597 documents · Prism Data Lab post of Sep 6, 2026 vs arXiv v3 of Sep 17, 2026 · Prism Data Lab; arXiv:2606.29251 · margin: not disclosed. Always check which version and which table a number comes from.

Why a correct summary can mislead

The authors name two mechanisms:

• Decontextualization. The summary keeps the headline claim but drops the caveat, comparison or expectation framing that gave it meaning. A beat that management called one-time, or growth that lagged guidance, can read as plain good news.
• Model dependency. Different summarizer models expose different views of the same document, so the read you get depends on which tool summarized it.

Their best fix, 'Agentic Context Compression', generates several candidate summaries with different models and audits their disagreements against the original. It cut flips meaningfully, but not to the reread floor.

What this means for the summaries you actually use

Broker digests and chatbot summaries are the same operation the paper tested: a long disclosure squeezed into a few lines. The study did not test any specific consumer product, so we cannot say how often a given app's digest flips the read. The mechanism, though, applies to all of them.

Prompting around it

• Ask for caveats explicitly. 'List every qualifier, one-time item and comparison to guidance next to the claim it modifies' targets decontextualization directly; the paper's contextualization prompt cut flips under both decision models.
• Summarize twice, with two models. Where two summaries disagree, go back to that passage in the transcript. That is a manual version of the paper's best method.
• Keep the length honest. A 4% summary nContext: n=597 documents (300 MD&A, 297 calls) · fiscal 2025 Q1 to Q3 · arXiv:2606.29251 v3, Table 2 · margin: not disclosed of a call drops a lot; ask for a longer one on the sections that drive your view (guidance, margins, segment commentary).
• Form your view from the source on positions that matter. Use the summary to find where to read, not what to think.