---
title: "What Is an NPU? How Neural Processing Units Power On-Device AI"
slug: what-is-an-npu-neural-processing-unit-explained
category: ai
category_label: "AI"
author: "BrainWavePost Staff"
date: 2026-06-10
tags: ["NPU", "AI hardware", "on-device AI", "Copilot+ PC", "Apple Neural Engine", "machine learning"]
read_time_minutes: 9
canonical_url: https://brainwavepost.com/article/what-is-an-npu-neural-processing-unit-explained
source: BrainWavePost
---

# What Is an NPU? How Neural Processing Units Power On-Device AI

*AI · 2026-06-10 · BrainWavePost Staff · 9 min read*

> A Neural Processing Unit (NPU) is a specialized chip built to run AI workloads efficiently on-device. Here is a sourced, plain-English explainer of what NPUs are, how they work, and why they now ship in nearly every modern phone and laptop.

> **How this article is sourced** _(info)_
>
> Definitions and technical claims below are taken from primary vendor documentation (Intel, Qualcomm, Apple, Arm, Microsoft) and established outlets (IBM, IEEE Spectrum, AnandTech). Citations are linked at the end. [1][2][3][4][5][6][7][8]

A Neural Processing Unit, or NPU, is a hardware accelerator designed specifically to execute the math behind machine-learning models — primarily large numbers of multiply-accumulate (MAC) operations on tensors — at much higher energy efficiency than a general-purpose CPU or GPU. [1][2][3] Intel defines an NPU as 'a specialized hardware accelerator designed to speed up AI and machine learning tasks' that handles 'complex AI computations like neural network inference' while offloading work from the CPU and GPU. [1]

NPUs are now standard silicon. Apple's A- and M-series chips include a Neural Engine, Qualcomm's Snapdragon platforms ship the Hexagon NPU, Intel Core Ultra includes an integrated NPU, and AMD Ryzen AI uses the XDNA NPU. [2][3][6][7] Microsoft's 'Copilot+ PC' brand requires an NPU capable of at least 40 TOPS (trillion operations per second) of sustained AI performance. [5]

## Why a separate chip for AI?

Modern neural networks are dominated by matrix multiplications and convolutions. CPUs are optimized for branchy, sequential code; GPUs are optimized for highly parallel floating-point math (originally graphics). NPUs are narrower still: they are optimized for low-precision integer (INT8, INT4) and small floating-point (FP16, BF16) tensor math that dominates inference. [1][3][4]

- Energy efficiency: an NPU can deliver the same inference using a fraction of the power of a CPU or GPU, which matters on phones and laptops running on battery. [1][2][3]
- Latency: inference happens on the device, so there is no round-trip to a cloud server. [1][5]
- Privacy: data such as voice, photos, and keystrokes can stay on the device. [2][5]
- Concurrency: the CPU and GPU stay free for the rest of the operating system while the NPU handles the AI workload. [1][3]

## How an NPU works (the short version)

At the core of every NPU is a large array of MAC units — small circuits that compute a × b + c in a single cycle. Thousands of these are arranged so that a tile of an input tensor and a tile of weights can be multiplied and summed in parallel, often using a systolic-array layout pioneered by Google's TPU. [4][8] On-chip SRAM keeps weights and activations close to the MAC array to avoid the energy cost of fetching from DRAM. A scheduler streams tensor tiles in, runs the activation function (ReLU, GELU, etc.), and writes results back out. [3][4][8]

![Simplified NPU architecture diagram showing input tensor and weights flowing through on-chip SRAM into a MAC array, then accumulator, activation function, and output tensor](npu-architecture-diagram.png)

*Simplified NPU dataflow: tensors and weights are staged in on-chip SRAM, multiplied and accumulated in a MAC array, passed through an activation function, and written back out. Architecture inspired by published descriptions of TPU-style accelerators. [4][8]*

### Lower precision, more throughput

NPUs trade numeric precision for speed. Most inference runs in INT8 or even INT4 instead of the FP32 used during training, which lets the same silicon area fit far more MAC units and move far less data per operation. [3][4] Modern NPUs from Qualcomm, Intel, and Apple advertise mixed-precision support (INT4 / INT8 / FP16 / BF16) so models can be quantized post-training with minimal accuracy loss. [2][3][6]

## NPU vs CPU vs GPU

- CPU — a few powerful, general-purpose cores. Great for OS logic and branchy code; inefficient for tensor math. [1]
- GPU — thousands of simpler cores optimized for parallel floating-point math. Excellent for both training and inference, but power-hungry. [1][3]
- NPU — a domain-specific accelerator built almost entirely around MAC arrays and on-chip memory for low-precision tensor math. Highest performance-per-watt for inference. [1][2][3][4]

## Who makes NPUs?

NPUs ship in most consumer silicon released since 2024:

- Apple — the Neural Engine has shipped in every A-series chip since the A11 Bionic (2017) and in every M-series Mac chip. Apple cites 16-core Neural Engine designs in current generations. [6]
- Qualcomm — the Hexagon NPU in Snapdragon X Elite is rated at 45 TOPS, meeting Microsoft's Copilot+ PC bar. [2][5]
- Intel — Core Ultra (Meteor Lake and successors) integrates an NPU based on the Movidius VPU lineage. Intel's Lunar Lake Core Ultra Series 2 advertises 48 TOPS on its NPU. [1][3]
- AMD — Ryzen AI uses the XDNA NPU, derived from the Xilinx acquisition; Ryzen AI 300 series advertises 50 TOPS. [7]
- Google — Tensor SoCs in Pixel phones include a TPU, and Google's data-center TPU family is the canonical published example of a large-scale tensor accelerator. [8]

- **40+ TOPS** — Minimum NPU performance for a Microsoft Copilot+ PC [5]
- **INT8 / INT4** — Typical inference precisions on consumer NPUs [3][4]
- **2017** — Year Apple shipped the first Neural Engine in the A11 Bionic [6]

## What NPUs are actually used for today

On phones and laptops, NPUs power features users already touch every day: live captions and translation, background blur and eye contact in video calls, computational photography, voice assistants, on-device search, and increasingly small language models running locally. [2][3][5][6] Microsoft's Copilot+ features — Recall, Live Captions with translation, Cocreator in Paint, Windows Studio Effects — explicitly require an NPU. [5]

> **TOPS is not the whole story** _(tip)_
>
> Vendors love to quote peak TOPS, but real-world performance depends on supported data types, memory bandwidth, the software stack (DirectML, Core ML, ONNX Runtime, OpenVINO, QNN), and how well a given model maps to the hardware. Always check independent benchmarks before comparing chips on TOPS alone. [3][4]

## The bottom line

An NPU is not a magic AI chip — it is a focused accelerator that trades generality for efficiency on the specific math that neural networks need. As models shrink (quantization, distillation) and NPUs grow, more of the AI you use every day will quietly run on the device in your hand rather than in a data center. [1][2][3][5]

## References (clickable)

- [1] Intel — 'What Is a Neural Processing Unit (NPU)?': https://www.intel.com/content/www/us/en/products/docs/processors/what-is-a-npu.html
- [2] Qualcomm — 'Hexagon NPU' product page: https://www.qualcomm.com/products/technology/processors/hexagon
- [3] IBM — 'What is an NPU (neural processing unit)?': https://www.ibm.com/think/topics/npu
- [4] IEEE Spectrum — 'How Google Built the TPU' (systolic arrays and tensor accelerators): https://spectrum.ieee.org/google-tpu
- [5] Microsoft — 'Copilot+ PCs' overview and NPU requirement: https://www.microsoft.com/en-us/windows/copilot-plus-pcs
- [6] Apple — 'Apple silicon' (Neural Engine in A- and M-series): https://www.apple.com/mac/m-series/
- [7] AMD — 'Ryzen AI' (XDNA NPU): https://www.amd.com/en/products/processors/consumer/ryzen-ai.html
- [8] Google Cloud — 'Introduction to Cloud TPU': https://cloud.google.com/tpu/docs/intro-to-tpu

---

_Canonical article: [https://brainwavepost.com/article/what-is-an-npu-neural-processing-unit-explained](https://brainwavepost.com/article/what-is-an-npu-neural-processing-unit-explained) — © BrainWavePost. Educational content; see the article page for full disclaimers._