GPU, TPU or NPU? How AI Chips Differ, and What TOPS and FP4 Mean

Every week brings another huge number about AI chips. This week it was $63 billion: the size of a financing deal to buy Broadcom-made AI accelerators. Chip names fly past in the same stories: GPUs, TPUs, XPUs, NPUs, often with a score in TOPS and a number format like FP4 attached.

They are all answers to the same question: what is the fastest, cheapest way to do the one kind of maths that AI needs, over and over again? The differences come down to how much flexibility you are willing to give up for speed, and where the chip has to live.

For the bigger picture of supply, memory and power, see The AI Compute Stack: Why Chips Decide the Race.

Why it matters now

On 26 October 2026 Broadcom announced that Bank of America, Blackstone, Citi and Morgan Stanley would back a $63 billion deal to buy Broadcom “XPU AI accelerators” and networking equipment, to be leased to an AI lab, with deliveries expected from 2027 (our brief). XPU is Broadcom’s name for these accelerators, which our brief describes as custom chips rather than off-the-shelf graphics processors.

Deals like this decide which AI services can run, how fast and at what price. Knowing what a GPU, a custom accelerator and an NPU each do makes those stories, and the spec sheets of new laptops, much easier to read.

How it works, step by step

Step 1: What AI actually asks a chip to do

A trained AI model is, underneath, a vast collection of numbers called parameters. Every time the model reads a word or produces one, the chip multiplies the incoming numbers by those parameters and adds the results together. One multiply plus one add is called a multiply-accumulate, or MAC.

Picture a times-table worksheet the size of a football pitch, and every answer needs filling in before the model can say its next word. The question for any AI chip is how many of those sums it can do at once, and how quickly it can fetch the numbers.

Step 2: CPUs and GPUs

The main processor in a computer, the CPU, is like a few brilliant chefs who can cook anything, one complicated order after another. NVIDIA’s own description is that a CPU has “just a few cores with lots of cache memory”, while a GPU has “hundreds of cores that can handle thousands of threads simultaneously”. Today’s chips have far more cores than when that was written, but the contrast still holds.

GPUs were built to draw video-game graphics, where every pixel on the screen can be worked out at the same time. That same habit of doing thousands of simple sums in parallel turned out to be exactly what AI needs. Data-centre GPUs now sit next to stacks of very fast memory called HBM, because feeding the numbers in is often harder than doing the maths.

Step 3: TPUs and other custom accelerators

A GPU can still run many kinds of program. A custom accelerator gives up that flexibility to do AI maths more efficiently. These chips are a type of ASIC, an application-specific integrated circuit: a chip designed for one job.

Google’s TPU (tensor processing unit) is the best-documented example. A 2017 paper by Google engineers said the first TPU had been running in Google’s data centres since 2015, that its heart was a block of 65,536 units each doing 8-bit multiply-accumulates, and that it reached a peak of 92 TOPS. On Google’s own workloads it was on average 15 to 30 times faster than the GPUs and CPUs of the time, and 30 to 80 times better in operations per watt.

The catch is cost and time. Designing a chip takes years and a large team, so custom accelerators make sense mainly for the biggest buyers, who run enough AI to pay that back. Broadcom’s XPUs are this kind of chip, built for particular customers.

Step 4: NPUs, the AI chip in your pocket

An NPU (neural processing unit) is a small AI block built into the main chip of a phone or laptop. It is designed for low power, so features such as live captions or blurring your background on a video call can run on the device itself, without sending data to the cloud and without draining the battery.

When Microsoft launched its Copilot+ PCs in May 2024, it described the new chips as capable of “40+ TOPS (trillion operations per second)”, and that NPU figure became the bar for the range. An NPU is far less powerful than a data-centre GPU; it is built to run small models efficiently, not giant ones.

Step 5: Reading the numbers: TOPS and FP4

TOPS means trillions of operations per second. It is a peak figure, like the top speed printed on a car brochure, and it depends on how big each number is. A chip will quote far more TOPS for small numbers than for large ones.

That is where formats such as FP4 come in. Computers store numbers using a set number of bits. A 16-bit number is like a price written to the penny; a 4-bit number is like a price rounded to the nearest pound. Fewer bits means less memory and more sums per second, at some cost in precision. This shrinking is called quantisation.

To stop the rounding doing damage, newer 4-bit formats give each small group of numbers a shared “scale”, a bit like noting “all prices on this shelf are in thousands”. NVIDIA’s NVFP4 gives each block of 16 values its own scale, and NVIDIA says this cuts a model’s memory footprint by about 3.5 times compared with 16-bit numbers. Chipmakers including AMD, Arm, Intel, Meta, Microsoft, NVIDIA and Qualcomm also agreed shared “microscaling” formats in 2023, including a 4-bit one called MXFP4.

A worked example: why the same model needs different chips

Take an imaginary AI model with 70 billion parameters and follow the arithmetic.

1. At 16 bits, each parameter takes 2 bytes, so the parameters alone fill about 140 gigabytes. That is more than any laptop has, so the model lives in a data centre, spread across GPUs or custom accelerators with fast HBM memory.
2. At 4 bits, each parameter takes half a byte, so the same model needs about 35 gigabytes, plus a little extra for the scales. It now fits on fewer chips, so each answer costs less to produce.
3. On your laptop, 35 gigabytes is still too much for most machines. But a small model with 8 billion parameters at 4 bits needs about 4 gigabytes, which a modern laptop can hold and its NPU or GPU can run.
4. The trade-off. Each step down in precision saves memory and money, but risks small errors. NVIDIA reported 1% or less accuracy loss on key language tasks for one large model moved from 8-bit to NVFP4; other models and tasks have to be tested case by case.

The lesson: there is no single best AI chip. The giant model goes to the data centre, the small one to your laptop, and the number format decides how many chips each one needs.

What people get wrong

“More TOPS means faster AI.” TOPS is a peak for one number size. Real speed often depends more on how fast the chip can fetch numbers from memory, and on the software.

“GPUs are graphics cards.” They started that way. Many data-centre GPUs today are built almost entirely for AI and scientific maths, as our compute stack explainer sets out.

“Custom chips are always better.” They are better at the jobs they were designed for. A GPU can switch to a new kind of model without waiting years for a new chip, which is why big buyers often use both.

“4-bit means a quarter as smart.” Precision and intelligence are not the same thing. Formats like NVFP4 add shared scales precisely to limit the loss, though it always needs measuring.

Where it is heading

• Custom silicon at huge scale. Financing deals such as Broadcom’s $63 billion arrangement, with deliveries from 2027, show how much money is now going into accelerators built for particular customers.
• Smaller numbers. Shared 4-bit formats, from the 2023 microscaling standard to NVIDIA’s NVFP4, are making low-precision AI routine rather than experimental.
• AI on the device. NPUs with tens of TOPS are now a selling point for ordinary laptops, which lets more features run without the cloud.
• Follow the chips. Our Silicon tracker lists AI chips from every major maker, with their specs.

Key terms

• GPU
• TPU (tensor processing unit)
• ASIC
• NPU
• Inference chip
• TOPS
• NVFP4
• Quantisation
• HBM
• Parameter

Sources

• Broadcom, news release on the $63 billion XPU financing, 26 October 2026. investors.broadcom.com
• Jouppi et al., “In-Datacenter Performance Analysis of a Tensor Processing Unit”, arXiv:1704.04760, 2017. arxiv.org
• NVIDIA, “What’s the Difference Between a CPU and a GPU?”, 16 December 2009. blogs.nvidia.com
• Microsoft, “Introducing Copilot+ PCs”, 20 May 2024. blogs.microsoft.com
• NVIDIA Technical Blog, “Introducing NVFP4 for Efficient and Accurate Low-Precision Inference”, 24 June 2025. developer.nvidia.com
• Open Compute Project, “AMD, Arm, Intel, Meta, Microsoft, NVIDIA, and Qualcomm Standardize Next-Generation Narrow Precision Data Formats for AI”, 17 October 2023. opencompute.org
• AIooze: NVFP4 explained in 2 minutes