A large language model (LLM) is a type of machine learning model designed for natural language processing tasks such as language generation. LLMs are language models with many parameters, and are trained with self-supervised learning on a vast amount of text.

List For the training cost column, 1 petaFLOP-day equals 1 petaFLOP/sec × 1 day, or (floating point operations). Only the cost of the largest model is shown. The number of parameters is measured in billions, and the training cost is measured in petaFLOP-days.

2018 Name Release date Developer of Corpus size Training cost License Notes GPT-1 OpenAI 1 BERT Google words

2019 Name Release date Developer of Corpus size Training cost License Notes T5 Google 34B tokens XLNet Google words 330 GPT-2 OpenAI 40GB (~ tokens) 28

2020 Name Release date Developer of Corpus size Training cost License Notes GPT-3 OpenAI tokens 3640Table D.1 in

2021 Name Release date Developer of Corpus size Training cost License Notes GPT-Neo EleutherAI 825 GiB HyperCLOVA Naver 82B 560B Tokens Unknown Unreleased GPT-J EleutherAI 825 GiB 200 Megatron-Turing NLG Microsoft and Nvidia tokens 38000 Ernie 3.0 Titan Baidu 4TB rowspan="4" Claude Anthropic tokens GLaM (Generalist Language Model) Google tokens 5600 Gopher Google DeepMind tokens 5833Table 20 and page 66 of PaLM: Scaling Language Modeling with Pathways

2022 Name Release date Developer of Corpus size Training cost License Notes LaMDA (Language Models for Dialog Applications) Google 1.56T words, tokens 4110 GPT-NeoX EleutherAI 825 GiB 740 Chinchilla scaling Google DeepMind tokens 6805 rowspan="2" PaLM (Pathways Language Model) Google tokens OPT (Open Pretrained Transformer) Meta tokens 310 YaLM 100B Yandex 1.7TB rowspan="5" Minerva Google 38.5B tokens from webpages filtered for math content and from arXiv BLOOM Large collaboration led by Hugging Face tokens (1.6TB) Galactica Meta tokens AlexaTM (Teacher Models) Amazon

2023 Name Release date Developer of Corpus size Training cost License Notes Llama Meta AI 6300 Stanford Alpaca Stanford University GPT-4 OpenAI Cerebras-GPT Cerebras 270 Falcon Technology Innovation Institute 1T tokens, from RefinedWeb (filtered web text corpus) plus some "curated corpora". 2800 UAE's Falcon 40B, World's Top-Ranked AI Model from Technology Innovation Institute, is Now Royalty-Free , 31 May 2023 BloombergGPT Bloomberg L.P. 363B tokens from Bloomberg's proprietary data sources, plus 345B tokens from general purpose datasets rowspan="3" PanGu-Σ Huawei 329B tokens OpenAssistant LAION 1.5T tokens Vicuna LMSYS Jurassic-2 AI21 Labs rowspan="3" PaLM 2 (Pathways Language Model 2) Google tokens YandexGPT Yandex Phi-1 Microsoft 7B tokens Llama 2 Meta AI tokens Claude 2 Anthropic rowspan="8" rowspan="8" rowspan="2" Granite 13b IBM Mistral 7B Mistral AI YandexGPT 2 Yandex rowspan="2" Claude 2.1 Anthropic Grok-1 xAI Gemini 1.0 Google DeepMind Mixtral 8x7B Mistral AI DeepSeek-LLM DeepSeek 2T tokens Phi-2 Microsoft 1.4T tokens 419

2024 Name Release date Developer of Corpus size Training cost License Notes Gemini 1.5 Google DeepMind rowspan="2" rowspan="2" rowspan="14" rowspan="2" Gemini Ultra Gemma 6T tokens OLMo Allen Institute for AI 2T tokens Claude 3 Anthropic DBRX Databricks and Mosaic ML 12T tokens YandexGPT 3 Pro Yandex Fugaku-LLM Fujitsu, Tokyo Institute of Technology, Tohoku University, RIKEN, etc. 380B tokens Chameleon Meta AI Mixtral 8x22B Mistral AI Phi-3 Microsoft 4.8T tokens Granite Code Models IBM YandexGPT 3 Lite Yandex Qwen2 Alibaba Cloud 3T tokens DeepSeek-V2 DeepSeek 8.1T tokens Nemotron-4 Nvidia 9T tokens Claude 3.5 Anthropic Llama 3.1 Meta AI 15.6T tokens Grok-2 xAI rowspan="6" OpenAI o1 OpenAI Sarvam-1 Sarvam AI ~2T tokens YandexGPT 4 Lite and Pro Yandex Mistral Large Mistral AI rowspan="2" rowspan="2" Pixtral OLMo 2 Allen Institute for AI 6.6T tokens 15,000 Phi-4 Microsoft tokens rowspan="2" DeepSeek-V3 DeepSeek 14.8T tokens Amazon Nova Amazon

2025 Name Release date Developer of Corpus size License Notes DeepSeek-R1 DeepSeek Qwen2.5 Alibaba MiniMax-Text-01 Minimax Gemini 2.0 Google DeepMind rowspan="7" rowspan="7" rowspan="7" Grok 3 xAI Claude 3.7 Anthropic YandexGPT 5 Lite Pretrain and Pro Yandex GPT-4.5 OpenAI Gemini 2.5 Google DeepMind YandexGPT 5 Lite Instruct Yandex Llama 4 Meta AI OpenAI o3 and o4-mini OpenAI Qwen3 Alibaba Cloud Claude 4 Anthropic Sarvam-M Sarvam AI Grok 4 xAI Param-1 BharatGen GLM-4.5 Z.ai GPT-OSS OpenAI Claude 4.1 Anthropic rowspan="2" GPT-5 OpenAI DeepSeek-V3.1 DeepSeek YandexGPT 5.1 Pro Yandex Apertus ETH Zurich and EPF Lausanne Claude Sonnet 4.5 Anthropic rowspan="4" GLM-4.6 Z.ai Alice AI LLM 1.0 Yandex rowspan="2" Gemini 3 Google DeepMind Olmo 3 Allen Institute for AI Claude Opus 4.5 Anthropic rowspan="4" DeepSeek-V3.2 DeepSeek GPT 5.2 OpenAI GLM-4.7 Z.ai

2026 + Name Release date Developer of Corpus size License Notes Qwen3-Max-Thinking Alibaba Cloud K2 Think V2 MBUZAI / Kimi K2.5 Moonshot AI Step 3.5 Flash StepFun rowspan="3" Claude Opus 4.6 Anthropic rowspan="2" rowspan="2" GPT-5.3-Codex OpenAI GLM-5 Z.ai Qwen3.5 Alibaba Cloud rowspan="3" Qwen3.5-Plus rowspan="2" rowspan="2" Claude Sonnet 4.6 Anthropic Param-2 BharatGen Sarvam-105B Sarvam AI rowspan="2" Sarvam-30B GPT-5.4 OpenAI rowspan="5" Mistral Small 4 Mistral AI MiMo-V2-Pro Xiaomi rowspan="2" Qwen3.6-Plus Alibaba Cloud Gemma 4 Google DeepMind GLM-5.1 Z.ai Muse Spark Meta rowspan="6" Qwen3.6 Alibaba Cloud Claude Opus 4.7 Anthropic Kimi K2.6 Moonshot AI GPT-5.5 OpenAI Hy3 preview Tencent MiMo-V2.5-Pro Xiaomi rowspan="2" MiMo-V2.5 Mistral Medium 3.5 Mistral AI128B rowspan="7" Composer 2.5 Cursor rowspan="4" Gemini 3.5 Flash Google DeepMind rowspan="3" Qwen3.7-Max Alibaba Cloud Claude Opus 4.8 Anthropic Step 3.7 Flash StepFun MiniMax-M3 MiniMax Nemotron 3 Ultra Nvidia Claude Fable 5 Anthropic rowspan="2" Kimi K2.7 Code Moonshot AI GLM-5.2 Z.ai rowspan="2" LongCat-2.0 Meituan Hy3 Tencent rowspan="5" Grok 4.5 SpaceXAI rowspan="4" rowspan="4" SWE-1.7 Cognition AI Muse Spark 1.1 Meta GPT-5.6 OpenAI Inkling Thinking Machines Lab Kimi K3 Moonshot AI rowspan="5" Qwen3.8 Alibaba Cloud Gemini 3.5 Flash Lite Google DeepMind rowspan="2" rowspan="2" Gemini 3.6 Flash Google DeepMind Laguna S 2.1 Poolside AI Motif 3 Motif Technologies Solar Open 2 Upstage Fugu-Ultra v1.1 Sakana AI rowspan="2" rowspan="2" rowspan="2" Claude Opus 5 Anthropic A.X K2 SK Telecom rowspan="3" Inkling Small Thinking Machines Lab rowspan="2" K-Exaone 2.0 LG DeepSeek-V4-Flash DeepSeek Muse Glimmer Meta rowspan="2" Gemini 3.7 Flash Google DeepMind DeepSeek-V4-Pro DeepSeek GLM-5.3 Z.ai 40B active GLM-5.3-flash 28 August 320B 30T 18B active Hy4 preview Tencent rowspan="5" Claude Mythos 5.1 Anthropic rowspan="4" rowspan="4" Claude Fable 5.1 Anthropic Gemini 3.8 Flash Google DeepMind GPT-6 Astra OpenAI K2 Horizon MBUZAI / IFM 0.9B to 375B DeepSeek-V4.1-Flash DeepSeek rowspan="1" Grok 4.7 SpaceXAI rowspan="2" rowspan="2" rowspan="2" GPT-6 Sol OpenAI MiMo-V2.6-Pro Xiaomi rowspan="2" MiMo-V2.6-Flash Claude Opus 5.5 Anthropic rowspan="4" rowspan="4" rowspan="4" Claude Sonnet 5.5 GPT 6.1 OpenAI Gemini 4 Argon Google DeepMind Mistral Large 4 Preview Mistral AI

See also Comparison of deep learning software Comparison of machine learning software List of AI-assisted software development tools List of chatbots List of language model benchmarks OpenRouter – LLM gateway to multiple AI providers Open weights Small language model

Notes

References