Guide

What Is an NPU?

A plain-English guide to NPUs, the AI chips in phones and laptops: what a neural processing unit is, how it differs from a CPU, GPU and TPU, what TOPS means and why it misleads, which chips have one, what actually runs on it, and whether you need one.

November 4, 2026 · Updated November 14, 2026 · The AI Rankings

Quick answer: An NPU, or neural processing unit, is a part of a chip built to run AI models: it does the huge volume of simple multiply-and-add maths that neural networks need, at low precision, using far less power than a CPU or GPU would for the same job. Almost every new phone and most new Windows laptops have one, and it is what lets features such as live captions, background blur, photo search and small on-device chatbots run without sending your data to a server. Microsoft requires an NPU rated at 40 TOPS (trillion operations per second) or more for a laptop to be sold as a Copilot+ PC. The one caveat: TOPS is a peak figure each chipmaker calculates its own way, so it tells you whether a laptop meets that bar, not which chip runs AI fastest in practice.

This page is the explainer for NPUs and on-device AI hardware. It does not rank devices. For which phone does AI best, see best AI phones. For the models themselves, see what is an LLM, and for models you can run on your own computer, see best local LLMs. For one-line definitions of the rest of the vocabulary, see the AI glossary.

What is an NPU?

A neural processing unit is a processor designed specifically to run neural networks, the type of model behind modern AI. In a phone or laptop it is usually not a separate chip: it is one block on the same system-on-a-chip as the CPU and GPU.

IBM describes NPUs as “tailored to accelerate AI tasks and workloads”, with dedicated hardware for matrix multiplication, convolution and dot products. Those three operations are most of what a neural network does when it runs. An NPU puts thousands of small arithmetic units side by side so it can do huge numbers of them at once.

The second trick is precision. A CPU normally works with 32-bit or 64-bit numbers. NPUs, IBM notes, often work in 8-bit or lower, which is precise enough for a trained model to give nearly the same answer and much cheaper in power and memory.

Vendors use their own names for the same idea:

Where the name came from

Huawei announced the Kirin 970 phone chip with a dedicated NPU on 2 September 2017, calling it Huawei’s first mobile AI computing platform. Apple announced the A11 Bionic with a two-core Neural Engine capable of 600 billion operations per second ten days later, on 12 September 2017. Intel brought the NPU to mainstream laptops with its first Core Ultra processors on 14 December 2023, and Microsoft made it a headline requirement when it announced Copilot+ PCs on 20 May 2024.

NPU vs CPU vs GPU vs TPU

The CPU, GPU and NPU all sit on the same chip in a modern phone or laptop, and each is good at a different kind of work.

The practical difference between a GPU and an NPU is power. IBM puts it as NPUs doing similar parallel processing “with far better power efficiency”. That matters for jobs that run constantly, such as blurring your background on a two-hour video call or transcribing a recording, where a GPU would work but would cost battery life and fan noise.

The GPU still wins on raw muscle. Apple has added AI-specific hardware to its GPUs: the M5 chip has “a Neural Accelerator in each core” of its GPU alongside a 16-core Neural Engine, and Apple claims more than four times the peak GPU AI compute of the M4. Nvidia’s RTX Spark Windows laptops lead with the GPU too, rated by Nvidia at up to 1 petaflop of AI performance, while the laptop models also include an Nvidia NPU.

Google’s TPU is the data-centre ancestor of the category. Google described its first TPU in a 2017 paper, “In-Datacenter Performance Analysis of a Tensor Processing Unit”, and the Tensor chips in Pixel phones carry a smaller TPU of their own.

What does an NPU actually do on your device?

An NPU runs AI inference: it takes a model that has already been trained and uses it to produce an answer. Training a model happens in data centres on thousands of GPUs or TPUs; your phone or laptop NPU only runs the finished result.

On a laptop

Microsoft’s Copilot+ PC page lists the Windows features built around the NPU:

• Recall (preview): takes regular snapshots of your screen so you can search for something you saw earlier by describing it.
• Click to Do: suggests actions for the text or image under your cursor.
• Live Captions with translation: captions any audio playing on the PC and translates it in real time. Captions alone work on all Windows 11 PCs; translation is the Copilot+ part.
• Windows Studio Effects: background blur, eye-contact correction and automatic framing for your webcam, which requires a qualified NPU.
• Improved Windows search: find files and settings by describing them rather than remembering their names.

The Copilot assistant itself is mostly a cloud service and runs on any PC.

On Macs, Apple’s Neural Engine and GPU handle Apple Intelligence features on M-series chips. Apple does not publish a list of which features run on which part of the chip.

On a phone

Phone NPUs run camera processing (portrait blur, night mode, object removal), voice typing and call transcription, photo search, live translation and small on-device language models.

• Google runs Gemini Nano, its smallest Gemini model, on Android phones through a system service called AICore that uses the phone’s hardware accelerators. The full Gemini app runs mostly in the cloud.
• Apple’s third generation of Apple Foundation Models includes AFM 3 Core, a 3-billion-parameter model built for on-device use.
• MediaTek says the main NPU in its Dimensity 9600 Pro processes a prompt 51% faster than its previous chip, the step a phone runs before a language model starts answering.

For context, the frontier models behind ChatGPT, Claude and Gemini are far larger and run only in data centres. An NPU runs small, specialised models; it does not put a frontier chatbot in your pocket. Most phone AI features are hybrid: the phone answers simple requests locally and sends harder ones to the cloud. Best AI phones covers which features run where on each flagship.

Glasses, earbuds and other devices

Smart glasses, translation earbuds and AI voice recorders lean on small NPUs or on the paired phone for wake words, noise cancellation and quick commands, and send heavier requests to the cloud. See best AI glasses and best AI translator earbuds.

What is TOPS?

TOPS stands for trillions of operations per second, and it is the headline number chipmakers use to describe NPU performance. Microsoft’s Copilot+ bar is an NPU of 40 TOPS or more.

The figure is a theoretical peak. It is broadly the number of arithmetic units in the NPU, multiplied by how many operations each does per clock cycle, multiplied by the clock speed. Four things make it a weak guide to real-world speed:

1. Precision is chosen by the vendor. Qualcomm labels its laptop figure “TOPS (INT8)”, meaning 8-bit integer maths. Not every vendor states the precision in its headline figure, and Lenovo notes that NPUs often reach their highest ratings using 8-bit operations. Halve the precision and the same hardware can quote a higher number.
2. GPU figures use different units. Nvidia rates RTX Spark at up to 1 petaflop of FP4 (4-bit floating point) performance. That cannot be compared directly with an 8-bit NPU TOPS figure.
3. Memory is often the real limit. Running a language model means reading its weights from memory for every token it produces, so memory bandwidth and capacity frequently matter more than peak TOPS. Apple, for example, quotes the M5’s 153GB/s memory bandwidth alongside its AI claims rather than a TOPS number.
4. Software decides whether the NPU is used at all. An app has to be built for a specific NPU, or use a framework that supports it, or the work falls back to the CPU or GPU.

Phone chipmakers mostly describe a new NPU as a percentage improvement over their own previous chip, as the phone table below shows, which makes cross-brand comparison on paper impossible.

Platform TOPS vs NPU TOPS

Intel and others also quote “platform TOPS”, which adds the CPU, GPU and NPU together. Intel rates its top Core Ultra Series 3 chips at 50 NPU TOPS and at up to 180 platform TOPS. The Copilot+ requirement is about the NPU figure alone.

Which chips have an NPU?

Every current flagship phone chip and most new laptop chips do. These are the figures their makers publish.

Laptop and desktop chips

Sources: Qualcomm X2 Elite product brief, Qualcomm X2 Plus, Intel, AMD Ryzen AI 400, AMD Ryzen AI 300, Apple M5, Apple M5 Pro and Max, Nvidia RTX Spark specifications.

AMD’s top laptop part, the Ryzen AI 9 HX 475, is the 60 TOPS model; the HX 470 is rated at 55 and the rest of the range at 50. AMD’s desktop Ryzen AI 400 chips carry a 50 TOPS NPU (Notebookcheck).

Phone chips

Every figure in the phone table is the chipmaker’s own claim, measured against its own previous chip.

How are NPUs benchmarked?

Because TOPS figures are not comparable, independent testers use benchmarks that run real AI models on the device and time them.

MLPerf Client shows how differently each vendor uses its hardware: AMD runs the test across its NPU and GPU together, Intel across NPU and GPU through its OpenVINO software, Qualcomm across NPU and CPU, and Apple Macs on the GPU through Apple’s MLX framework, not the Neural Engine.

Do you need an NPU?

For most people the answer is that you will get one anyway, and it matters less than the rest of the spec sheet.

The rule of thumb: an NPU makes AI features cheaper to run on battery and keeps some of them private, but it does not make a cloud chatbot smarter or faster, and a big GPU with plenty of memory still matters more if you want to run serious models locally.

Frequently asked questions

What is an NPU in simple terms?

An NPU, or neural processing unit, is the part of a phone or laptop chip built to run AI. It does the repetitive multiply-and-add maths that AI models depend on, at low precision, using much less power than the main processor or graphics chip would. That lets features such as live captions, webcam background blur and photo search run on the device.

What does NPU stand for?

NPU stands for neural processing unit. The name comes from neural networks, the type of model behind modern AI. Apple calls its version the Neural Engine, Qualcomm the Hexagon NPU, AMD the XDNA NPU and Google, in Pixel phones, the TPU.

What is the difference between an NPU and a GPU?

A GPU is a general parallel processor originally built for graphics, and it has the most raw AI throughput but draws a lot of power. An NPU is built only for neural-network maths at low precision, so it delivers less peak performance but far better performance per watt. Laptops and phones use the NPU for AI features that run for long periods and the GPU for the heaviest AI jobs.

What is the difference between an NPU and a CPU?

A CPU is a general-purpose processor that handles complex tasks one step after another on a small number of powerful cores. An NPU has thousands of simple arithmetic units that do the same calculation in parallel, which suits AI models. The CPU can run AI, but more slowly and with more power.

Is an NPU the same as a TPU?

A TPU, or tensor processing unit, is Google’s name for its own AI accelerators, so it is one kind of NPU. Google’s large TPUs run in its data centres and power services such as Gemini, and its Tensor phone chips include a smaller TPU that does the job of an NPU in Pixel phones.

What does TOPS mean?

TOPS stands for trillions of operations per second, the peak AI throughput a chipmaker claims for its NPU. Microsoft requires 40 TOPS or more for a Copilot+ PC. The number depends on the precision the vendor chooses, usually 8-bit integers, so two chips with the same TOPS rating can perform very differently on real models.

How many TOPS do I need?

You need an NPU rated at 40 TOPS or more for a Windows laptop to qualify as a Copilot+ PC and run features such as Recall, Click to Do and live caption translation. The current top laptop chips from Qualcomm, Intel and AMD are rated at 50 to 80 TOPS, so they clear that bar comfortably. Beyond it, extra TOPS rarely changes what you can do.

Do I need an NPU for ChatGPT?

No. ChatGPT, Claude, Gemini and Microsoft Copilot run on their makers’ servers, so they work the same on any phone or computer with an internet connection. An NPU only matters for AI features that run on the device itself.

Does the iPhone have an NPU?

Yes. Apple calls it the Neural Engine, and every iPhone since the iPhone 8 and iPhone X, which used the A11 Bionic chip announced in September 2017, has one. The A20 Pro in the iPhone 18 Pro has two 16-core Neural Engines, 32 cores in total, which Apple says doubles the AI processing power of the A19 Pro.

Do Macs have an NPU?

Yes. Every Apple silicon Mac has a Neural Engine, and the M5, M5 Pro and M5 Max each have a 16-core Neural Engine. From the M5 onwards Apple also builds a Neural Accelerator into each GPU core, and Apple’s headline AI claim for the M5, more than four times the peak GPU AI compute of the M4, is about the GPU rather than the Neural Engine.

Can an NPU run a large language model?

An NPU can run small language models of a few billion parameters, such as Google’s Gemini Nano or Apple’s 3-billion-parameter on-device model. Frontier models with hundreds of billions of parameters are far too large and run only in data centres. On laptops, most tools for running open-weight models locally still rely on the GPU, because memory capacity and bandwidth matter more than NPU throughput.

What is a Copilot+ PC?

A Copilot+ PC is a Windows 11 laptop or desktop that meets Microsoft’s hardware bar for on-device AI, including an NPU rated at 40 TOPS or more. Microsoft announced the category on 20 May 2024. Copilot+ PCs get Windows features that run on the NPU, such as Recall, Click to Do, live caption translation and Windows Studio Effects; the Copilot assistant itself works on any Windows PC.

← All guides