Recurrent Neural Network (RNN)
Last Updated: July 29, 2026 | By Mihail Sebastian | AI Dictionary
A neural network that processes data one step at a time, carrying a hidden state forward so earlier inputs shape how it reads later ones in a sequence.
What is a Recurrent Neural Network (RNN)?
A Recurrent Neural Network (RNN) is a neural network that processes a sequence one step at a time, feeding a summary of everything seen so far back into itself at each step.
A standard network treats every input as independent. An RNN does not: the word it reads now is interpreted in light of the words before it, which is what sequence tasks like language and time series forecasting demand.
How an RNN Works
- Hidden state: At each step, the network combines the new input with its current hidden state to produce an output and an updated state. The state is the network’s running memory of the sequence.
- Backpropagation Through Time (BPTT): Training unrolls the network across the sequence and applies backpropagation through every step, so errors at the end adjust weights that acted at the beginning.
- The long-range problem: As sequences grow, gradients flowing back through many steps shrink toward zero – the vanishing gradient problem. Gated variants such as LSTM were invented to fix this, and transformers later sidestepped recurrence entirely.
RNN vs CNN
The practical difference: an RNN reads sequential data one step at a time, while a convolutional neural network reads spatial data all at once. An RNN asks “what came before this”; a CNN asks “which pixels sit next to which”.
| Criterion | RNN | CNN |
|---|---|---|
| Data shape | Sequential: text, audio, time series | Grid-like: images, video frames, spectrograms |
| What it exploits | Order and context from earlier steps | Spatial locality between neighboring values |
| Processing | Step by step, carrying a hidden state | Whole input in parallel |
| Typical tasks | Language modeling, forecasting | Image classification, object detection |
Applications of RNNs
- Language modeling: Predicting the next word from the words so far, the basis of text generation before transformers took over.
- Speech recognition: Turning a stream of audio frames into text, where each sound depends on the sounds around it.
- Time series forecasting: Projecting future values, such as electricity demand or sensor readings, from their history.
Example of a Recurrent Neural Network
Give an RNN trained on English text the fragment “The cat sat on the”. It reads one word per step: after “The” its hidden state holds little, after “cat” it expects an action, after “sat on the” it expects a surface.
At the final step it outputs a probability for every word in its vocabulary, and “mat” ranks near the top. The prediction works only because the hidden state carried the earlier words forward; shown “the” alone, the network would have no basis to choose.
Related AI terms: Convolutional Neural Network · LSTM · NLP Transformer · Sequence-to-Sequence · Time Series
Did you like the Recurrent Neural Network (RNN) gist?
Learn about 250+ need-to-know artificial intelligence terms in the AI Dictionary.
Mihail Sebastian — Writes about AI governance, regulation, and the technology behind them. Placeholder bio — replace with a real credential line. About