Guide

What Is an LLM?

A plain-English guide to large language models: what an LLM is, how LLMs work, how they are trained, what a token is, what inference means, how to read parameters and context windows, examples of LLMs, why they hallucinate, and how LLMs got here.

October 7, 2026 · The AI Rankings

Quick answer: An LLM, or large language model, is an AI model trained on trillions of words of text to predict the next small chunk of text, called a token, and that single skill is what lets it answer questions, write, translate, summarise and write code. ChatGPT, Claude and Gemini are apps built on LLMs: the app is the interface, and the LLM is the model doing the work underneath. Almost every LLM is built on the transformer, a neural-network design published by Google researchers in June 2017. The one caveat: an LLM generates the most plausible text, not verified fact, so it can state a wrong answer with confidence, and OpenAI’s own research says standard training rewards guessing over admitting uncertainty.

This page is the explainer: what an LLM is, how it works, and the vocabulary you need to read a model page. It does not rank models. For the current ranking, see best AI models. For the developer view of pricing and providers, see best LLM APIs. For models you can download and run yourself, see best open-source LLMs. For one-line definitions of the rest of the vocabulary, see the AI glossary.

What is an LLM?

A large language model is a neural network trained on very large amounts of text to predict which token comes next in a sequence.

The name breaks down simply:

• Large refers to the size of the model and its training data. The largest LLMs have hundreds of billions to trillions of learned numerical values, called parameters, and are trained on tens of trillions of tokens. Alibaba trained Qwen3 on about 36 trillion tokens, and DeepSeek trained DeepSeek-V4-Pro on more than 32 trillion.
• Language refers to what the model was trained on and produces: text, including programming code. Many LLMs also accept images, audio or video as input, but the name has stuck.
• Model means a trained mathematical function. An LLM takes text in, runs it through billions of calculations, and returns a probability for every possible next token.

Predicting the next token sounds like a narrow skill. It is not, because predicting the next word of a physics textbook, a contract or a Python file accurately requires a working model of physics, law or Python. Training an LLM on enough text to predict it well forces the model to pick up grammar, facts, reasoning patterns and coding conventions along the way. Nobody programs those abilities in directly.

LLM vs AI vs generative AI vs chatbot

These four terms are often used interchangeably. They are not the same thing.

Every LLM is generative AI, and all generative AI is AI. The reverse is not true: an image generator such as Midjourney is generative AI but is usually built on a diffusion model, not an LLM.

The distinction between the model and the app matters when you compare products. ChatGPT is an app, and the LLM inside it changes over time as OpenAI releases new GPT models. The same LLM can sit behind many apps: Anthropic’s Claude models are available in the Claude app, through the Anthropic API, and inside third-party tools such as coding assistants.

How do LLMs work?

An LLM works by turning text into tokens, running those tokens through a stack of transformer layers, and producing a probability for every possible next token, one token at a time.

Step by step: what happens when you send a prompt

1. Tokenisation. The app splits your prompt into tokens, which are word fragments taken from the model’s fixed vocabulary. “Unbelievable” might become “un”, “believ” and “able”.
2. Embedding. Each token is converted into a long list of numbers, called a vector, that represents its meaning in a form the model can calculate with.
3. Attention. The vectors pass through dozens of transformer layers. In each layer, an attention mechanism lets every token weigh how relevant every earlier token is to it, which is how the model connects a pronoun to the noun it refers to several paragraphs earlier.
4. Prediction. The final layer outputs a probability for every token in the vocabulary. “The capital of France is” puts most of the probability on ” Paris”.
5. Sampling. The model picks a token from that distribution. A setting called temperature controls how often it picks less likely tokens: lower temperature gives more predictable output, higher temperature gives more varied output.
6. Repeat. The chosen token is added to the sequence, and the whole process runs again to predict the next one. A 500-word answer is roughly 650 of these steps.

The model has no separate store of facts that it looks up. Everything it “knows” from training is encoded in its parameters, which is why an LLM can produce a fluent answer about a topic and still get a specific fact wrong.

The transformer and attention

The transformer is the neural-network architecture behind essentially every LLM. Google researchers introduced it in the paper “Attention Is All You Need”, first posted on 12 June 2017. The paper’s design was “based solely on attention mechanisms, dispensing with recurrence and convolutions”, which let models process a whole passage in parallel instead of word by word. That parallelism is what made it practical to train models on trillions of tokens using thousands of chips at once.

Mixture of experts: why parameter counts got confusing

Many large LLMs use a mixture-of-experts (MoE) design, in which the model is split into many specialised sub-networks, called experts, and only a few experts run for each token. The model’s total parameter count and the parameters actually used per token are therefore very different numbers.

MoE lets a lab build a model with a very large store of knowledge while keeping the computing cost of each answer closer to that of a much smaller model. It is also why total parameter count is a weak guide to how capable or expensive a model is.

Is an LLM just autocomplete?

An LLM is trained on next-token prediction, so in a narrow sense it is a very large autocomplete. Research into what happens inside the model suggests the description undersells it. In research published on 27 March 2025, Anthropic traced the internal activity of its Claude 3.5 Haiku model while it wrote rhyming poetry and found that the model chose the rhyming word for the end of a line before writing the line, which Anthropic describes as evidence that “Claude will plan what it will say many words ahead”. Whether that amounts to understanding is an open philosophical question; it does show that predicting one token at a time does not mean thinking only one token ahead.

How LLMs are trained

LLMs are trained in two broad phases: pre-training, which teaches a model language and knowledge from raw text, and post-training, which turns that raw model into an assistant that follows instructions.

Pre-training: scale

Pre-training is the expensive part. It runs for weeks or months on thousands of GPUs or TPUs, and it is where most of a model’s knowledge comes from. Training data has grown fast: DeepSeek trained DeepSeek-V3 on 14.8 trillion tokens, Alibaba trained Qwen3 on about 36 trillion tokens, and DeepSeek trained DeepSeek-V4-Pro on more than 32 trillion tokens. Anthropic, OpenAI and Google do not publish training-data sizes or parameter counts for their flagship models.

DeepMind’s Chinchilla paper (29 March 2022) set the rule of thumb that shaped this growth: for compute-optimal training, “for every doubling of model size the number of training tokens should also be doubled”. Its 70-billion-parameter Chinchilla model, trained on about 1.4 trillion tokens, outperformed much larger models trained on less data.

Post-training: from text predictor to assistant

A base model straight out of pre-training will happily continue your question as though it were the start of a document. Post-training fixes that. OpenAI’s InstructGPT paper (4 March 2022) showed how much it matters: people preferred outputs from a 1.3-billion-parameter model trained with human feedback over outputs from the 175-billion-parameter GPT-3, a model more than 100 times larger. Nearly nine months later, OpenAI launched ChatGPT on 30 November 2022 using the same approach.

Reasoning models

A reasoning model is an LLM trained to work through a problem in a hidden or visible chain of intermediate steps before giving its final answer. OpenAI introduced the first widely used one, o1-preview, on 12 September 2024, saying it had trained the models “to spend more time thinking through problems before they respond”. DeepSeek’s R1 paper (22 January 2025) showed that this reasoning behaviour can be produced “through pure reinforcement learning”. Many reasoning models let the user or developer set how much effort the model spends, and benchmark boards such as Artificial Analysis list the same model separately at each effort level, with higher effort scoring higher and costing more.

What is a token in AI?

A token is the unit of text an LLM actually reads and writes: usually a word, part of a word, a punctuation mark or a space, taken from a fixed vocabulary the model learned during training.

LLMs do not see letters or words. They see token IDs. Common words such as “the” are usually a single token; rarer or longer words are split into several. Every limit and price that matters to an LLM user is counted in tokens: the context window, the maximum answer length, rate limits and the API bill.

How many words is a token?

The vendors publish slightly different rules of thumb for English text, because each uses a different tokeniser.

The practical conversion most people use is 1,000 tokens to about 750 English words, so a 100,000-word book is roughly 133,000 tokens. Anthropic is the important exception: its token-counting documentation says the tokeniser used by Claude Opus 4.7 and later models produces “approximately 30 percent more tokens” for the same text, so the same document uses more tokens on a newer Claude model than on an older one.

Why tokens matter

• Price. API access to an LLM is priced per million tokens, split between input tokens (what you send) and output tokens (what the model writes). See the worked example in the inference section below.
• Context window. The context window is the maximum number of tokens a model can hold at once, including your prompt, any attached files, the conversation so far and the answer. Windows of about 1 million tokens exist: Anthropic’s pricing page lists a 1-million-token window for Claude 4.6 and later models.
• Other languages cost more. Tokenisers are built mostly from English text, so other languages often split into more tokens per word. A NeurIPS 2023 study by Petrov and colleagues found differences “up to 15 times” in the number of tokens needed to encode the same content in different languages. That means higher cost and less usable context for the same text.
• Odd failures. Because the model sees tokens rather than letters, it can struggle with tasks such as counting the letters in a word or reversing a string, even when it handles much harder reasoning well.

Reasoning tokens

Reasoning models also generate thinking tokens before their visible answer. Those tokens are generally billed as output tokens, even when the app hides them. This is why cost per completed task, not price per token, is the fairer comparison between reasoning models: a model with a lower price per token can cost more per task if it uses more thinking tokens to get there.

What is inference in AI?

Inference is the process of running a trained model to produce an output. Training builds the model once; inference happens every time anyone sends a prompt.

Prefill and decode: why output tokens cost more

Inference runs in two phases. NVIDIA’s inference engineering guide describes the first, prefill, as “highly parallelized”: the model reads every input token at once. The second, decode, generates output tokens one at a time, and NVIDIA calls it “a memory-bound operation”. Because each output token needs its own pass through the model, while input tokens are processed together, output is the slower and more expensive phase. That is our explanation of why API providers price output tokens higher than input; the providers’ pricing pages do not state a reason. On Anthropic’s pricing page, every Claude model’s output price is five times its input price.

Two techniques do much of the work of making inference affordable:

• KV cache. The model stores intermediate calculations for tokens it has already processed, which avoids “recomputing all these tensors for all tokens at each time step” (NVIDIA). Providers pass some of the saving on as prompt caching: Anthropic’s standard price for reading cached input is 10% of its normal input price (Anthropic pricing).
• Speculative decoding. A small, fast draft model guesses several tokens ahead, and the large model checks them in parallel, accepting the correct ones (NVIDIA).

What inference costs

A worked example, using illustrative prices: summarising a 7,500-word report (about 10,000 input tokens) into a 750-word summary (about 1,000 output tokens) on a model priced at $2 per million input tokens and $10 per million output tokens costs $0.02 for input plus $0.01 for output, or $0.03 in total. The same job on a model priced at $10 and $50 costs $0.15. A reasoning model’s thinking tokens would add to the output figure in both cases. Current prices for each model are on best LLM APIs.

The price of a given level of capability has fallen fast. Epoch AI reported in March 2025 that the price of reaching a fixed level of LLM performance fell between 9-fold and 900-fold per year depending on the task, and 40-fold per year for matching GPT-4 on PhD-level science questions.

How much energy inference uses

Google reported that the median text prompt in its Gemini apps used 0.24 watt-hours of energy and about 0.26 millilitres of water, measured in May 2025, and that energy per prompt fell 33-fold between May 2024 and May 2025. OpenAI chief executive Sam Altman gave a figure of 0.34 watt-hours for an average ChatGPT query in a June 2025 blog post. Neither figure covers training, and neither has been independently audited.

How to read an LLM’s spec sheet

Every model page on this site, and every vendor announcement, uses the same handful of numbers. This is what each one tells you.

Types of LLMs

LLMs differ in who can access them, how they are built to answer, and how large they are.

Open-weight vs open-source LLMs

Most “open-source LLMs” are open-weight models: the trained parameters are published, but the training data and full training code are not. That still lets you run the model on your own hardware, keep your data private and fine-tune it. For the full open field, see best open-source LLMs; for what runs on a laptop or a single graphics card, see best local LLMs.

Small language models

There is no agreed size cut-off for a small language model. NVIDIA researchers argued in a June 2025 paper that small models are “sufficiently powerful, inherently more suitable, and necessarily more economical” for many of the repetitive sub-tasks inside AI agents. In practice, the term is used for models that run on a phone, a laptop or a single consumer graphics card.

Examples of LLMs

LLMs are released as model families, with new versions replacing old ones several times a year. This table lists the main families by maker. It is not a ranking, and it carries no scores or prices: for which model is strongest and what each costs, see best AI models.

Independent leaderboards do not agree on a single winner, because they measure different things. The Artificial Analysis Intelligence Index combines a set of hard evaluations into one score. The LMArena text leaderboard ranks models by blind human preference votes.

To compare the apps rather than the models, see best AI chatbots and ChatGPT vs Claude.

What LLMs are good at, and where they fail

LLMs are strong at tasks where fluent, well-structured text is the goal and where errors are easy to spot: drafting, summarising, translating, explaining, brainstorming and writing code that can be tested. They are weakest where a single wrong fact matters and nobody checks.

Hallucination

A hallucination is a confident, plausible-sounding answer that is false. It is the defining weakness of LLMs, and no model has eliminated it.

• Why it happens. OpenAI’s September 2025 research argues that “standard training and evaluation procedures reward guessing over acknowledging uncertainty”: a model that guesses scores better on most benchmarks than one that says “I don’t know”.
• How often it happens. Vectara’s hallucination leaderboard measures invented content in document summaries: the lowest rate listed is 1.8%, and some widely used models exceed 10%.

Grounding an LLM in real sources reduces hallucination: web search, retrieval from your own documents (a technique called retrieval-augmented generation, or RAG) and asking for citations you then check.

Context rot

A model’s accuracy can fall as its input grows, even well inside the advertised context window. Chroma tested 18 models in July 2025 and found that their “performance grows increasingly unreliable as input length grows”. A 1-million-token window is a capacity, not a promise of the same accuracy on the millionth token as on the first.

Knowledge cutoff

An LLM knows nothing that happened after its training data ends unless the app gives it a search tool. A model is usually released some months after its cutoff, so even a new model is behind on recent events. Vendors list the cutoff for each model in their documentation, as Anthropic does on its models overview.

Other limits

• No memory between sessions by default. The model itself remembers nothing once a request ends; memory features in apps work by saving notes and feeding them back in as context.
• Sensitivity to wording. Small changes to a prompt can change the answer, which is why the same question can get different results on different days.
• Letter-level tasks. Tokenisation makes spelling, letter counting and some arithmetic on long numbers unreliable without a tool.

LLMs vs AI agents

An LLM on its own only produces text. An AI agent is an LLM placed in a loop with tools, such as a browser, code runner or file system, so it can take actions, check the results and continue until a task is finished. The LLM is the reasoning engine; the agent is the system built around it. For more, see what is agentic AI and best AI agents.

How to use an LLM

There are three ways to use an LLM, and the right one depends on what you are doing.

To choose a model rather than an app, start with best AI models, which ranks the frontier on independent evidence and cost per task. For coding specifically, see best AI for coding. If you are building search or retrieval on top of an LLM, you will also need an embedding model: see best embedding models.

A short history of large language models

For what has launched since, see the AI changelog.

Frequently asked questions

What is an LLM in simple terms?

An LLM, or large language model, is an AI model trained on huge amounts of text to predict the next word fragment in a sequence. By repeating that prediction thousands of times, it can answer questions, write, summarise, translate and write code. ChatGPT, Claude and Gemini are apps built on LLMs.

What does LLM stand for?

LLM stands for large language model. “Large” refers to the size of the model, up to trillions of parameters, and to its training data, up to tens of trillions of tokens. “Language” refers to the text it is trained on and produces.

How do LLMs work?

An LLM splits your prompt into tokens, converts each token into numbers, and passes them through many transformer layers that use attention to work out how each token relates to the others. The model then outputs a probability for every possible next token, picks one, adds it to the text and repeats until the answer is complete.

What is a token in AI?

A token is the unit of text an AI language model reads and writes, usually a word or part of a word. OpenAI’s rule of thumb is that one token is about four characters or three-quarters of an English word, so 1,000 tokens is about 750 words. Context windows, rate limits and API prices are all counted in tokens.

How many words is 1,000 tokens?

1,000 tokens is about 750 English words by OpenAI’s rule of thumb, and 600 to 800 words by Google’s. The figure varies by model and language: Anthropic says the tokeniser in Claude Opus 4.7 and later models fits roughly 555 words into 1,000 tokens, and many non-English languages need more tokens per word than English.

What is inference in AI?

Inference is the process of running a trained AI model to produce an output, such as an answer to a prompt. Training builds the model once; inference happens every time someone uses it. API prices are charges for inference, billed per million input and output tokens.

Is ChatGPT an LLM?

ChatGPT is an app built on LLMs, not an LLM itself. OpenAI’s GPT models do the work inside it, and ChatGPT adds the chat interface, memory, web search, file handling and image generation around them.

What is the difference between AI and an LLM?

AI is the whole field of building systems that perform tasks associated with human intelligence. An LLM is one specific kind of AI: a generative model trained on text that produces text by predicting the next token. Every LLM is AI, but most AI systems, such as spam filters and recommendation engines, are not LLMs.

What is the best LLM right now?

The answer changes with almost every major release, so this guide does not name one. Our current ranking, which weighs benchmarks, reviews and cost per task, is on best AI models.

What is a context window in an LLM?

A context window is the maximum number of tokens an LLM can work with in one request, covering the prompt, attached documents, the conversation so far and the answer. The largest context windows are about 1 million tokens. Accuracy can still fall on very long inputs, an effect researchers call context rot.

Why do LLMs hallucinate?

LLMs hallucinate because they generate the most plausible next text rather than looking up verified facts. OpenAI’s 2025 research adds that standard training and evaluation reward guessing over admitting uncertainty, so models learn to give a confident answer instead of saying they do not know. Web search, document retrieval and checking citations reduce the problem but do not remove it.

Are LLMs open source?

Some LLMs are open-weight, meaning the trained model can be downloaded and run, but very few are fully open source with training data and code released. The DeepSeek, Qwen, Kimi and Llama families, Google’s Gemma and OpenAI’s gpt-oss are open-weight. Anthropic’s Claude, Google’s Gemini and OpenAI’s main GPT models are proprietary and available only through their makers’ apps and APIs.

What is a small language model?

A small language model is an LLM compact enough to run on a phone, a laptop or a single graphics card, with no agreed size cut-off. Microsoft’s 14-billion-parameter Phi-4 and Google’s Gemma models are examples. Small models are cheaper and more private to run but less capable than frontier models on hard tasks.

Do LLMs understand what they say?

There is no agreed answer. LLMs are trained only to predict the next token, but Anthropic’s 2025 interpretability research found Claude planning a rhyming word before writing the line that leads to it, which shows more internal structure than simple word-by-word autocomplete. Whether that counts as understanding is still debated.

← All guides