Tensors
Last Updated: July 29, 2026 | By Mihail Sebastian | AI Dictionary
Arrays with any number of axes, from a single number to a batch of images, and the one data structure every deep learning framework computes with.
What are Tensors?
A tensor is an array of numbers with any number of axes. One number is a tensor with no axes, a list has one, a table has two, and a stack of tables has three; deep learning frameworks use the single word for all of them.
In PyTorch and TensorFlow, a tensor carries more than its contents. It knows which device holds it, CPU or GPU, and it can record the operations applied to it so gradients can flow back through them during backpropagation.
Types of Tensors
- Scalar (0 axes): A single number, such as a learning rate of 0.001 or a loss value of 2.7.
- Vector (1 axis): An ordered list, such as the 768 numbers of a sentence embedding.
- Matrix (2 axes): Rows and columns, such as a weight matrix connecting 512 inputs to 256 outputs.
- Higher-order tensor (3+ axes): Everything else. A color image needs three axes for height, width, and channel; a batch of images needs four; a batch of video clips needs five.
Example of Tensors
A vision model receives 32 photographs resized to 224 by 224 pixels with three color channels. PyTorch stores that batch as a tensor of shape (32, 3, 224, 224): batch first, then channels, then height and width.
A convolutional layer with 64 filters turns it into (32, 64, 224, 224). Pooling halves the spatial axes to (32, 64, 112, 112). Repeated layers shrink height and width while growing the channel count, until a final flatten and linear layer produce (32, 10), one score per class per image.
Almost every early bug in this pipeline is a shape bug. TensorFlow’s default convolution layout puts channels last, (32, 224, 224, 3), so code ported between frameworks without transposing the axes will either crash or silently train on nonsense.
Related AI terms: PyTorch · TensorFlow · NumPy · Vectorization · Deep Learning
Did you like the Tensors gist?
Learn about 250+ need-to-know artificial intelligence terms in the AI Dictionary.
Mihail Sebastian — Writes about AI governance, regulation, and the technology behind them. Placeholder bio — replace with a real credential line. About