Category leaders

Guide

What Is a Neural Network?

A plain-English guide to neural networks: what a neural network is, how artificial neurons, weights and layers work, how a network learns through backpropagation, the main types from CNNs to transformers, how the transformer behind ChatGPT works, and how neural networks got here.

November 10, 2026 · Updated November 21, 2026 · The AI Rankings

Quick answer: A neural network is a computer model made of layers of simple connected units, called artificial neurons, that learns to do a task by adjusting the strength of the connections between them until its outputs match the examples it was trained on. Neural networks are the technology underneath chatbots such as ChatGPT, Claude, Gemini, and underneath image generators, speech recognition and face unlock. The design behind those chatbots is the transformer, a neural-network architecture that Google researchers published in June 2017. The one caveat: a neural network is loosely inspired by the brain but works very differently from one, and even its builders cannot fully explain how a large network reaches a given answer.

This page is the explainer: what a neural network is, how it learns, the main types, and how the transformer fits in. For how transformers become chatbots, see what is an LLM. For one-line definitions of the rest of the vocabulary, see the AI glossary.


What is a neural network?

A neural network is a model built from many small calculating units, arranged in layers, where each unit’s output feeds into units in the next layer and the strength of every connection is learned from data rather than written by a programmer.

In one sentence that could stand on its own: a neural network takes numbers in, passes them through layers of weighted connections, and produces numbers out, and training is the process of finding the connection weights that make those outputs useful.

The numbers can stand for almost anything. A photo is a grid of pixel brightness values. A sound clip is a sequence of air-pressure readings. A sentence is a sequence of tokens, each converted into a list of numbers. Because everything is numbers, the same basic machinery can learn to recognise cats, transcribe speech or write an email.

Neural network vs machine learning vs deep learning

These terms are nested, not interchangeable.

TermWhat it meansRelationship
Artificial intelligence (AI)The whole field of building systems that perform tasks associated with human intelligenceThe umbrella
Machine learningAI that learns from examples instead of following hand-written rulesOne approach to AI
Neural networkA machine learning model made of layers of connected artificial neuronsOne family of machine learning models, alongside decision trees, regression and others
Deep learningMachine learning with neural networks that have many layersThe modern, large-scale use of neural networks
TransformerA neural-network architecture built around a mechanism called attentionThe design behind large language models such as the ones in ChatGPT, Claude and Gemini
Large language model (LLM)A very large transformer trained on text to predict the next tokenThe best-known application of transformers (LLM guide)

Every LLM is a neural network, but most neural networks are not LLMs. The network that unlocks your phone with your face and the one that recommends your next video are neural networks too.

Is a neural network like the brain?

Only loosely. The idea came from neuroscience: in 1943, Warren McCulloch and Walter Pitts published “A Logical Calculus of the Ideas Immanent in Nervous Activity”, a mathematical model of a neuron that fires or stays silent depending on its inputs. Modern artificial neurons keep that basic idea and little else.

Biological brainArtificial neural network
Basic unitNeuron, a living cellA simple function: multiply, add, apply a threshold-like curve
ScaleAbout 86 billion neurons on average, measured in adult male brains (Azevedo et al., 2009)Learned values, called parameters, range from thousands to more than a trillion
How signals travelElectrical and chemical spikes, timedPlain numbers passed layer to layer
How it learnsMany mechanismsAlmost always one method: backpropagation with gradient descent
EnergyAbout 20 watts (NIST)The 2017 transformer was trained on eight GPUs for 3.5 days (paper); larger models use far more

The useful takeaway: “neural” describes where the idea came from, not how the system works. A neural network does not think the way a person does, and comparing parameter counts to neuron counts tells you very little.


How does a neural network work?

A neural network works by passing numbers forward through layers of artificial neurons, where each neuron multiplies its inputs by learned weights, adds them up, applies a simple non-linear function and passes the result on to the next layer.

The artificial neuron

Each artificial neuron does four small things:

  1. Takes inputs. A list of numbers, either the raw data or the outputs of neurons in the previous layer.
  2. Multiplies each input by a weight. A weight is a learned number that says how much that input matters. A large positive weight amplifies an input; a negative weight suppresses it.
  3. Adds them up, plus a bias. The bias is another learned number that shifts the total up or down.
  4. Applies an activation function. A simple curve that decides how strongly the neuron “fires”. The most common is ReLU (rectified linear unit), which passes positive numbers through unchanged and turns negative numbers into zero.

On its own, a single neuron does almost nothing useful. The power comes from wiring millions or billions of them together and tuning all their weights at once.

Why the activation function matters

Without an activation function, stacking layers would be pointless: a chain of multiply-and-add steps collapses mathematically into a single multiply-and-add step, however many layers you use. The small bend that the activation function adds is what lets a deep network represent complicated patterns. Mathematicians have shown that a network with even one hidden layer and enough neurons can approximate almost any continuous function, a result known as the universal approximation theorem (George Cybenko proved an early version in 1989). In practice, deep networks with many layers learn such functions far more efficiently than wide, shallow ones.

Layers: input, hidden and output

LayerWhat it doesExample: recognising a handwritten digit
Input layerReceives the raw numbers784 inputs, one per pixel of a 28-by-28 image
Hidden layersTransform the data step by step into more useful featuresEarly layers respond to strokes and edges; later layers to loops and line combinations
Output layerProduces the answer10 outputs, one score for each digit from 0 to 9

“Deep” in deep learning simply means many hidden layers. Each layer builds on the one before: in an image network, early layers pick up edges, middle layers combine edges into shapes such as eyes or wheels, and late layers combine shapes into objects. Nobody programs those features in. The network discovers them because they help it get the right answer.

Parameters: what a model’s size means

A network’s weights and biases together are its parameters. When a model is described as having 7 billion or 1.6 trillion parameters, that is the number of learned values inside it.

NetworkParametersSource
AlexNet (2012 image classifier)About 60 millionKrizhevsky, Sutskever and Hinton
GPT-3 (2020 language model)175 billionOpenAI
DeepSeek-V4-Pro1.6 trillion total, 49 billion active per tokenDeepSeek model card

More parameters give a network more capacity to store patterns, but size alone does not decide quality. Training data, training method and architecture matter as much.


How do neural networks learn?

A neural network learns by making a prediction, measuring how wrong it was, working out how much each weight contributed to the error, and nudging every weight slightly in the direction that reduces the error, then repeating that millions of times.

The training loop, step by step

  1. Start random. Every weight begins as a small random number, so the untrained network’s answers are noise.
  2. Forward pass. Feed in a training example, such as a photo labelled “cat”, and let the numbers flow through the layers to an output.
  3. Measure the error. A loss function turns the gap between the network’s answer and the correct answer into a single number. Lower is better.
  4. Backward pass (backpropagation). Work backwards from the output layer, using calculus, to calculate how much each weight in the network contributed to the error. That set of values is called the gradient.
  5. Update the weights (gradient descent). Move every weight a tiny step in the direction that would have made the error smaller. The size of the step is set by the learning rate.
  6. Repeat. Do it again with the next batch of examples, many millions of times, until the error stops falling.

Backpropagation is the key idea. David Rumelhart, Geoffrey Hinton and Ronald Williams popularised it in a 1986 paper in Nature, and it is the standard way to train neural networks, from small image classifiers to large language models.

A simple way to picture gradient descent

Imagine standing on a hillside in thick fog, trying to reach the valley floor. You cannot see the valley, but you can feel which way the ground slopes under your feet, so you take a small step downhill and check again. Gradient descent does the same thing in a landscape with millions or billions of dimensions, one per weight. The “height” is the error, and the valley floor is a set of weights where the network makes few mistakes.

Training vs inference

TrainingInference
What happensWeights are adjusted using backpropagationWeights are fixed; the network just runs forward
How oftenOnce per model versionEvery time someone uses the model
CostHeavy: days on eight GPUs for the 2017 transformer, and far more for larger modelsLight by comparison: one forward pass per output

When you send a prompt to a chatbot, nothing is learned from it in that moment. The network only runs forward. The LLM guide covers inference, tokens and what it costs in more detail.

Overfitting

A network with enough parameters can memorise its training examples instead of learning the general pattern, and then fail on new data. This is called overfitting. Builders guard against it by testing on data the network never saw during training, by using more varied data, and with techniques such as dropout, which randomly switches off neurons during training so the network cannot rely on any single path.


Types of neural networks

Neural networks come in several architectures, each built around the shape of the data it handles. The architecture is the wiring plan: how the layers connect and what each kind of layer does.

ArchitectureBuilt forHow it works, in one lineWhere you meet it
Feedforward network (multilayer perceptron)Tables of numbers, simple classificationData flows straight through fully connected layersCredit scoring, inside larger models as a building block
Convolutional neural network (CNN)Images and videoSmall filters slide across the image, detecting the same pattern wherever it appearsFace unlock, photo tagging, medical scans
Recurrent neural network (RNN) and LSTMSequences such as speech and textReads one step at a time, carrying a memory forwardSpeech recognition and translation systems of the 2010s
TransformerText, and increasingly images, audio and videoAttention lets every part of the input look at every other part at onceChatGPT, Claude, Gemini, translation, transcription
Diffusion modelGenerating images, video and audioLearns to turn random noise into an image step by stepMost AI image and video generators
Generative adversarial network (GAN)Generating realistic imagesTwo networks compete: one generates, one judgesEarlier image generators and face-swap tools
State-space model (such as Mamba)Very long sequencesProcesses a sequence with a compressed running state instead of full attentionResearch models and hybrid language models

Convolutional neural networks

A convolutional neural network scans an image with small learned filters, each tuned to a pattern such as a vertical edge or a patch of texture, and stacks layers of these filters to recognise larger structures. Yann LeCun and colleagues used this approach to read handwritten digits in the late 1980s and 1990s. CNNs then drove the 2012 breakthrough: an entry built on AlexNet, a CNN with about 60 million parameters, won the ImageNet competition with a top-5 error rate of 15.3%, against 26.2% for the next-best entry. ACM said the result “almost halved the error rate for object recognition and reshaped the computer vision field” (ACM).

Recurrent networks and LSTMs

A recurrent neural network reads a sequence one element at a time and passes a hidden state, a kind of running memory, from each step to the next. Plain RNNs struggle to remember things from many steps earlier. The long short-term memory (LSTM) network, published by Sepp Hochreiter and Jürgen Schmidhuber in 1997, added a memory cell with gates that control what it stores and passes on, and LSTMs powered much of speech recognition and machine translation in the 2010s. Their weakness is that they must process a sequence in order, one step after another, which makes them slow to train on very large datasets. The transformer removed that bottleneck.

Diffusion models

A diffusion model is trained by adding random noise to images in many small steps and teaching a neural network to predict and remove that noise. To generate a new image, it starts from pure noise and removes it step by step, guided by your text prompt, until a picture appears. The neural network doing the denoising is often itself a transformer or a CNN. For the tools built on diffusion, see best AI image generators and best AI video generators.


Transformers: the architecture behind ChatGPT

A transformer is a neural-network architecture that processes a whole sequence at once and uses a mechanism called attention to let every token weigh how relevant every other token is to it. It is the “T” in GPT, which stands for generative pre-trained transformer, and it is the design behind the large language models in ChatGPT, Claude and Gemini.

Where the transformer came from

Google researchers introduced the transformer in the paper “Attention Is All You Need”, first posted on 12 June 2017. Its design was “based solely on attention mechanisms, dispensing with recurrence and convolutions”. In practice that meant a model no longer had to read a sentence word by word, as an RNN does. It could process every word in parallel, which suited the GPU chips used for training and made it practical to train on vastly more text. The paper’s largest model reached 28.4 BLEU on the WMT 2014 English-to-German translation benchmark, a new state of the art at the time, after 3.5 days of training on eight GPUs.

How attention works, in plain English

Take the sentence “The trophy did not fit in the suitcase because it was too big.” To understand “it”, a model has to connect it to “trophy”, not “suitcase”. Attention is the mechanism that makes that connection.

  1. Every token becomes a vector. Each word fragment is turned into a list of numbers that represents its meaning, plus information about its position in the sentence.
  2. Each token asks a question. The network creates three versions of each vector: a query (what this token is looking for), a key (what this token offers) and a value (the information it carries).
  3. Tokens score each other. Each token’s query is compared with every other token’s key. A high score means “this other token is relevant to me”. For “it”, the score against “trophy” comes out high.
  4. Blend the information. Each token’s new representation is a weighted mix of the other tokens’ values, weighted by those scores.
  5. Do it many times in parallel. A transformer runs many attention “heads” side by side, each free to track a different kind of relationship (grammar, meaning, what refers to what), and stacks dozens of these layers.

Between the attention steps, each layer also has a small feedforward network that processes each token on its own. Attention moves information between tokens; the feedforward layers transform it.

Encoders, decoders and GPT

Transformer typeWhat it doesExamples
Encoder-onlyReads a whole input and builds a rich representation of it; good for classifying and searching textGoogle’s BERT, many embedding models used for search
Decoder-onlyGenerates text one token at a time, each token attending only to the tokens before itThe GPT, Claude, Gemini, Llama and DeepSeek families
Encoder-decoderReads an input with one stack and writes an output with anotherThe original 2017 translation model, OpenAI’s Whisper speech recogniser

Chatbots use decoder-only transformers. They predict the next token, add it to the text, and run again, which is why answers appear word by word. The LLM guide walks through that loop, plus tokens, context windows and training, step by step.

Why transformers took over

Transformer limits, and what might come next

Attention compares every token with every other token, so its cost grows roughly with the square of the input length: doubling the text quadruples the attention work. That is the main reason long context windows are expensive to serve. Researchers have proposed alternatives. Mamba, a state-space model published by Albert Gu and Tri Dao in December 2023, scales linearly with sequence length, and its authors report “5× higher throughput than Transformers” at inference. AI21 Labs’ Jamba, released in March 2024, is a hybrid that mixes transformer and Mamba-style layers.


Neural network examples in everyday life

Where you meet itWhat the network doesArchitecture
Chatbots and AI assistantsAnswers questions, writes and writes codeTransformer (decoder-only)
Phone face unlock and photo searchRecognises faces and objectsCNN or vision transformer
Dictation and voice assistantsTurns speech into textTransformer, earlier RNN/LSTM
Translation appsTranslates text and speechTransformer
Image and video generatorsCreates pictures and clips from a promptDiffusion model, often with a transformer inside
Streaming and shopping appsRecommends what to watch or buy nextFeedforward and transformer-based recommenders
BankingFlags payments that look fraudulentFeedforward networks, alongside other machine learning
Search enginesMatches the meaning of a query to pagesTransformer encoders and embedding models

For the products built on these networks, see best AI models for the models themselves and best AI chatbots for the apps. For neural networks you can download and run on your own hardware, see best open-source LLMs and best local LLMs, and what is open-source AI for what “open” means.


Limitations of neural networks


A short history of neural networks

YearMilestone
1943Warren McCulloch and Walter Pitts publish a mathematical model of the neuron
1958Frank Rosenblatt describes the perceptron, a single-layer network that learns from examples
1969Marvin Minsky and Seymour Papert’s book Perceptrons shows the limits of single-layer networks, and interest in the field cools
1982John Hopfield publishes a paper on the network now named after him, the Hopfield network, which stores and recalls patterns
1986Rumelhart, Hinton and Williams publish their backpropagation paper in Nature, showing how to train multi-layer networks
1997Hochreiter and Schmidhuber publish the long short-term memory (LSTM) network
1998Yann LeCun and colleagues publish LeNet-5, a convolutional network for reading handwritten characters
2012AlexNet wins ImageNet by a wide margin
2017Google researchers publish the transformer
2019Yoshua Bengio, Geoffrey Hinton and Yann LeCun receive the 2018 ACM Turing Award for their work on deep neural networks
2020The Vision Transformer applies transformers to images
2022OpenAI launches ChatGPT on 30 November, built on a transformer language model
2023Gu and Dao publish Mamba, an alternative to attention for long sequences
2024John Hopfield and Geoffrey Hinton win the Nobel Prize in Physics “for foundational discoveries and inventions that enable machine learning with artificial neural networks”

How to learn more about neural networks

If you want toStart here
Understand how neural networks become chatbotsWhat is an LLM
Look up a term quicklyAI glossary
See how networks are used as agents that take actionsWhat is agentic AI
Take a structured courseBest AI courses
Run a neural network on your own computerBest local LLMs
Work in the fieldMachine learning jobs

Frequently asked questions

What is a neural network in simple terms?

A neural network is a computer model made of layers of simple connected units that learns a task from examples. Each connection has a strength, called a weight, and training adjusts those weights until the network’s answers match the examples. Neural networks power chatbots, image generators, speech recognition and face unlock.

How does a neural network work?

A neural network passes numbers through layers of artificial neurons. Each neuron multiplies its inputs by learned weights, adds them up, applies a simple activation function and passes the result to the next layer. The final layer produces the answer, such as a label for an image or the next word in a sentence.

How does a neural network learn?

A neural network learns through backpropagation and gradient descent. It makes a prediction, measures how wrong it was, calculates how much each weight contributed to the error, and adjusts every weight slightly to reduce it. Repeating this over millions of examples gradually turns random weights into a useful model.

What is the difference between a neural network and deep learning?

A neural network is a type of machine learning model made of layers of connected artificial neurons. Deep learning is the use of neural networks with many layers, often dozens or more. All deep learning uses neural networks, but a small network with one or two layers is not usually called deep learning.

Are neural networks the same as AI?

Neural networks are one approach within AI, not the whole of it. AI also includes rule-based systems, search algorithms and other kinds of machine learning such as decision trees. Neural networks are the approach behind chatbots, image generators and speech recognition.

What is a transformer model?

A transformer is a neural-network architecture that processes a whole sequence at once and uses attention to let each part of the input weigh every other part. Google researchers introduced it in 2017. It is the architecture behind the large language models in ChatGPT, Claude and Gemini, and it is the “T” in GPT.

Is ChatGPT a neural network?

ChatGPT is an app built on neural networks. The models inside it are large transformer neural networks, called large language models, that generate text one token at a time. ChatGPT adds the chat interface, memory, web search and other tools around them.

What are the main types of neural networks?

The main types are feedforward networks for simple data, convolutional neural networks (CNNs) for images, recurrent networks and LSTMs for sequences, transformers for language and increasingly everything else, diffusion models for generating images and video, and generative adversarial networks (GANs).

How is a neural network different from the human brain?

A neural network borrows the idea of connected neurons from the brain but works very differently. An adult human brain has about 86 billion living neurons that communicate with timed electrical and chemical signals. An artificial neural network is a set of mathematical functions passing plain numbers between layers, and it learns almost entirely through one method, backpropagation.

Who invented neural networks?

No single person invented neural networks. Warren McCulloch and Walter Pitts published the first mathematical model of a neuron in 1943, Frank Rosenblatt described the perceptron in 1958, and David Rumelhart, Geoffrey Hinton and Ronald Williams popularised backpropagation in 1986. Hinton and John Hopfield won the 2024 Nobel Prize in Physics for foundational work on neural networks.

What are parameters in a neural network?

Parameters are the learned numbers inside a neural network, mainly the weights on its connections and the biases on its neurons. A model’s size is usually quoted in parameters: the 2012 image network AlexNet had about 60 million, OpenAI’s GPT-3 had 175 billion, and DeepSeek-V4-Pro, released in April 2026, has 1.6 trillion.

← All guides