In artificial intelligence, a foundation model (FM), also known as large x model (LxM, where "x" is a variable representing any text, image, sound, etc.), is a machine learning or deep learning model trained on vast datasets so that it can be applied across a wide range of use cases. Generative AI applications like large language models (LLMs) are common examples of foundation models.
Building foundation models is often highly resource-intensive, with the most advanced models costing hundreds of millions of dollars to cover the expenses of acquiring, curating, and processing massive datasets, as well as the computational power required for training.Nestor Maslej, Loredana Fattorini, Erik Brynjolfsson, John Etchemendy, Katrina Ligett, Terah Lyons, James Manyika, Helen Ngo, Juan Carlos Niebles, Vanessa Parli, Yoav Shoham, Russell Wald, Jack Clark, and Raymond Perrault, "The AI Index 2023 Annual Report," AI Index Steering Committee, Institute for Human-Centered AI, Stanford University, Stanford, California, April 2023. These costs stem from the need for sophisticated infrastructure, extended training times, and advanced hardware, such as graphics processing units. In contrast, adapting an existing foundation model for a specific task or using it directly is far less costly, as it uses pre-trained capabilities and typically requires only fine-tuning on smaller, task-specific datasets.
Early examples of foundation models are language models like OpenAI's GPT series. In addition to text, foundation models have been developed for visual images, including DALL-E and Stable Diffusion.
Definitions The Stanford Institute for Human-Centered Artificial Intelligence's (HAI) Center for Research on Foundation Models (CRFM) coined the term "foundation model" in August 2021 to mean "any model that is trained on broad data (generally using self-supervision at scale) that can be adapted (e.g., fine-tuned) to a wide range of downstream tasks". This was based on their observation that preexisting terms, while overlapping, were not adequate, stating that "'large language model' was too narrow given the focus is not only language; 'self-supervised model' was too specific to the training objective; and 'pretrained model' suggested that the noteworthy action all happened after 'pretraining." The term "foundation model" was chosen over "foundational model" because "foundational" implies that these models provide fundamental principles in a way that "foundation" does not. The term vision-language model (VLM) is also used as a near-synonym.
As governments regulate foundation models, new legal definitions have emerged. In the United States, the now revoked Executive Order on the Safe, Secure, and Trustworthy Development and Use of Artificial Intelligence defined a foundation model as "an AI model that is trained on broad data; generally uses self-supervision; contains at least tens of billions of parameters; is applicable across a wide range of contexts". In the United States, the proposed AI Foundation Model Transparency Act of 2023 by House Representatives Don Beyer (D–VA) and Anna Eshoo (D, CA) defines a foundation model as "an artificial intelligence model trained on broad data, generally uses self supervision, generally contains at least 1,000,000,000 parameters, is applicable across a wide range of contexts, and exhibits, or could be easily modified to exhibit, high levels of performance at tasks that could pose a serious risk to security, national economic security, national public health or safety, or any combination of those matters." In the European Union, the European Parliament's negotiated position on the E.U. AI Act defines a foundation model as an "AI model that is trained on broad data at scale, is designed for generality of output, and can be adapted to a wide range of distinctive tasks". In the United Kingdom, the Competition and Markets Authority's AI Foundation Models: Initial Report defines foundation models as "a type of AI technology that are trained on vast amounts of data that can be adapted to a wide range of tasks and operations."
The United States's definitions are the only ones to make reference to the size of a foundation model, and differ on magnitude. Beyer and Eshoo's definition also specifies that foundation models must achieve a level of performance as to be a potential danger. In contrast, the E.U. definition requires the model to be designed for generality of output. All definitions agree that foundation models must be trained on a broad range of data with potential applications in many domains.
History Technologically, foundation models are built using established machine learning techniques like deep neural networks, transfer learning, and self-supervised learning. Foundation models differ from previous techniques as they are general purpose models that function as a reusable infrastructure, instead of bespoke and one-off task-specific models.
Advances in computer parallelism (e.g., CUDA GPUs) and new developments in neural network architecture (e.g., Transformers), and the increased use of training data with minimal supervision all contributed to the rise of foundation models. Foundation models began to materialize as the latest wave of deep learning models in the late 2010s. Relative to most prior work on deep learning, these language models demonstrated the potential of training on much larger web-sourced datasets using self-supervised objectives (e.g. predicting the next word in a large corpus of text). These approaches, which draw upon earlier works like word2vec and GloVe, deviated from prior supervised approaches that required annotated data (e.g. crowd-sourced labels).
The 2022 releases of Stable Diffusion and ChatGPT (initially powered by the GPT-3.5 model) led to foundation models and generative AI entering widespread public discourse. Further, releases of LLaMA, Llama 2, and Mistral in 2023 contributed to a greater emphasis placed on how foundation models are released with open foundation models garnering a lot of support and scrutiny.
Related concepts
Frontier models Certain highly advanced foundation models are termed "frontier models", whether because they expand the frontier (state of the art) or because they are considered above a certain threshold of risk.
There is no single standard for assessing risks from frontier models. Some examples of dangerous capabilities include: Designing and synthesizing new biological or chemical weapons Enabling novel offensive cyber attacks Evading human control through deceptive means It may be difficult to regulate the development and deployment of frontier models, due to limited interpretability, unforeseen capability improvements, ease of misuse, and rapid proliferation. If a frontier model is open-weight and publicly available online, the model can spread more rapidly, while the combination of greater customizability and non-revocable access exacerbates problems of accountability and regulatory oversight.
Mitigating all harms that might arise from already-deployed frontier models becomes more difficult due to post-release modifications and attacks such as API fine-tuning and prompt injection, as well as capabilities enhancements such as the agent harness, access to external resources (data, tools, systems), and orchestration among AI systems.
The industry-led Frontier Model Forum was founded in July 2023 by OpenAI, Anthropic, Google DeepMind, and Microsoft, with a stated goal of supporting safety research, best-practice standards, and information-sharing about advanced foundation models. A network of national Artificial intelligence safety institutes (AISIs) have formed to assess risk from frontier models.
General-purpose AI Foundation models adaptable to a wide range of use-cases are sometimes referred to as general-purpose AI. In designing the EU AI Act, the European Parliament has stated that a new wave of general-purpose AI technologies shapes the overall AI ecosystem.
World models
World models are sometimes described as foundation models. World models are a representation of an environment intended to predict the state of that environment after taking a set of actions, as well as to implicitly model physical concepts such as gravity. Input prompts for world models can include text or images, as well as videos or 3D scenes, and the resulting 3D environments can be exported. World models, alongside embodied AI, multi-agent models, and neuroscience models of the brain, are seen as alternatives to large language models for achieving general artificial intelligence.
World models do not have a fully agreed definition, but have been divided into two scopes: one for representing and understanding the current environment, and another for predicting the future state of that environment. In the former view, world models are developed using model-based reinforcement learning and a Markov decision process, using model predictive control or Monte Carlo tree search to create policies. With the latter, (multimodal) large language models or video generation models can be used. In addition, these environments can be immersive simulations for training AI agents that can interact in the real world.
History Quanta Magazine traced world models back to a 1943 publication by Kenneth Craik on mental models and the blocks world of SHRDLU in the 1960s. Business Insider traced world models to a 1971 paper by Jay Wright Forrester. A related idea of organizing world knowledge, the frame representation, was proposed by Marvin Minsky in 1974.
In 2018, researchers David Ha and Jürgen Schmidhuber defined world models in the context of reinforcement learning: an agent with a variational autoencoder model V for representing visual observations, a recurrent neural network model M for representing memory, and a linear model C for making decisions. They suggested that agents trained on world models in environments that simulate reality could be applied to real world settings.
In 2022, Yann LeCun saw a world model (defined by him as a neural network that acts as a mental model for aspects of the world that are seen as relevant) as part of a larger system of cognitive architecture – other neural networks that are analogous to different regions of the brain. In his view, this framework could lead to commonsense reasoning. LeCun has estimated that world models would be fully functional by the late 2020s to mid 2030s.
Training World models are trained on a variety of data modalities, including text, images, audio and video, and have been applied to video generation. One open source dataset for world models includes 1 billion data points across multiple modalities (text, images, audio, video and point clouds), including 1 million manual annotations.
Examples TechCrunch saw Sora as an example of a world model, while in January 2025, Nvidia released its own set of world models. The South China Morning Post wrote that Manycore Tech was another example of companies aiming to build a world model, viewing their work as an example of spatial intelligence. In May 2025, Mohamed bin Zayed University of Artificial Intelligence released a world model for building simulations to test AI agents.
Google DeepMind has also released two world models in two-dimensional space and three-dimensional space, respectively, that were trained on video data, with Google claiming that the latter can be a training environment for AI agents. Meta released a world model in June 2025, Tencent released an open source world model in July 2025. Niantic, Inc. spinoff, Niantic Spatial, is developing a world model using anonymized player scans from Pokémon GO. Other companies that are planning as of 2025 to build world models include ByteDance and xAI.
Applications