BloombergGPT vs GPT-4: what the finance-only AI model taught the industry

Whenever an AI trading product says its model is "trained on financial data," it is echoing a bet made in March 2023. That month, Bloomberg researchers posted BloombergGPT, a language model built specifically for finance. The idea was intuitive: a model that has read decades of financial news and filings should understand markets better than a general chatbot. The evidence that followed was less kind to that idea.

What Bloomberg built

According to the paper (Wu et al., arXiv, revised December 2023), BloombergGPT has 50 billion parameters. Its training set combined a 363 billion token dataset built from Bloomberg's own data sources, which the authors describe as perhaps the largest domain-specific dataset yet, with 345 billion tokens from general-purpose datasets. The team started from a compute budget of 1.3 million GPU hours on 40GB Nvidia A100 chips, running on 64 machines with eight GPUs each.

The paper's claim was measured: the mixed training produced a model that outperformed existing models on financial tasks by significant margins without losing ground on general benchmarks. But the comparison set matters. BloombergGPT was compared with open models of similar or larger size: GPT-NeoX, OPT-66B and BLOOM-176B. Against those, it won.

Then GPT-4 entered the comparison

In May 2023, researchers from Queen's University and J.P. Morgan AI Research posted "Are ChatGPT and GPT-4 General-Purpose Solvers for Financial Text Analytics?" (Li et al., later accepted to the EMNLP 2023 industry track). They ran ChatGPT and GPT-4, versions current as of July 7, 2023, on eight financial datasets across five task types, and set the results next to BloombergGPT's published numbers.

GPT-4 led on question answering over financial tables, sentiment and headline classification. BloombergGPT kept an edge on named entity recognition, picking out company and legal names in SEC agreements, where generalist models struggled. On the Financial PhraseBank sentiment test, the gap looked enormous (GPT-4 at 0.86 F1 with five examples against 0.51 reported for BloombergGPT), but the two papers used different data splits, so that comparison is the least reliable of the set.

The bigger lesson: specialists still mattered

The same study shows something the "GPT-4 crushed BloombergGPT" headlines skipped. Small models fine-tuned on one task often beat both giants. On headline classification, a fine-tuned BERT model scored 95.36 against GPT-4's 86.00. On named entity recognition, a classic CRF model trained on similar SEC data scored 82.70, far above every language model. On FinQA, GPT-4 with step-by-step reasoning reached 78.03 accuracy, ahead of the fine-tuned FinQANet at 68.90 but well short of human experts at 91.16.

So the lesson was not that domain knowledge does not matter. It was more specific:

• Scale and general reasoning moved faster than domain pretraining. A general model trained by a lab racing on capabilities overtook a finance model months after its release.
• Narrow, well-defined tasks still reward narrow models. Classification and extraction were where small fine-tuned models won.
• Numerical reasoning remained the weak point for everyone. Even the best model trailed human experts by 13 points on FinQA.

Why it still matters in 2026

Three years on, the pattern holds in newer benchmarks. On the FrontierFinance benchmark, the biggest score differences came from the tools around the model, not from finance-specific training. When a product leads with "trained on financial data," the useful follow-up question is what it is compared against and on which tasks.