TPU (Tensor Processing Unit)
Last Updated: July 29, 2026 | By Mihail Sebastian | AI Dictionary
A custom chip Google designed to accelerate the tensor operations inside neural networks, built for the matrix math that dominates training and inference.
What is a TPU (Tensor Processing Unit)?
A TPU is a chip Google designed to accelerate tensor operations, the large matrix multiplications that dominate neural network training and inference.
Google announced the TPU in 2016, after running the chips in its own data centers to power services like Search and Translate. Today they are available for rent through Google Cloud.
How TPUs Work
Where a general-purpose processor spends silicon on flexibility, a TPU spends nearly all of it on multiplying matrices. The chip trades the ability to run arbitrary programs for higher throughput and better energy efficiency on the one workload neural networks consist of.
TPUs are programmed through frameworks, chiefly TensorFlow and JAX, which compile a model’s operations down to the hardware. For very large models, Google connects many TPU chips into pods that train a single network across hundreds of chips at once.
TPU vs GPU
The practical difference: a GPU is a general-purpose parallel processor that runs anything from graphics to CUDA kernels, while a TPU does one thing – tensor math – on hardware built for nothing else. GPUs come from several vendors, run everywhere, and carry the most mature software ecosystem in machine learning. TPUs exist only on Google Cloud and perform best inside Google’s frameworks.
The choice usually reduces to this: GPUs for flexibility and portability, TPUs for cost and energy efficiency at scale if your stack already fits Google’s tooling.
Example of a TPU
The chip’s public debut came with AlphaGo. When Google revealed the TPU in May 2016, it said the chips had powered AlphaGo’s match against Go champion Lee Sedol two months earlier.
The workload shows why the hardware fit. AlphaGo evaluated board positions with deep neural networks, and each evaluation is a stack of matrix multiplications. Running those on processors built solely for that operation let the system search more positions within the time limits of a live match.
The same trade plays out in routine cloud training jobs today: shard the data across a TPU pod, let each chip compute on its slice, and exchange gradients over the pod’s dedicated interconnect.
Related AI terms: CUDA · GPU Memory · Tensors · TensorFlow · Deep Learning
Did you like the TPU (Tensor Processing Unit) gist?
Learn about 250+ need-to-know artificial intelligence terms in the AI Dictionary.
Mihail Sebastian — Writes about AI governance, regulation, and the technology behind them. Placeholder bio — replace with a real credential line. About