Gaussian Process
Last Updated: July 29, 2026 | By Mihail Sebastian | AI Dictionary
A probabilistic model that defines a distribution over functions, so each prediction comes with both a value and a measure of its own uncertainty.
What is a Gaussian Process?
A Gaussian process is a probabilistic model that defines a distribution over entire functions, so every prediction arrives with both a value and a measure of the model’s uncertainty about it. Most models give you a number; a Gaussian process gives you a number plus an honest error bar.
That uncertainty estimate is the reason to choose one. Where a wrong-but-confident prediction is expensive, knowing what the model does not know is worth extra computation.
How a Gaussian Process Works
A GP starts from an assumption of smoothness: similar inputs should produce similar outputs. A kernel function encodes what “similar” means – the same machinery described under kernel methods.
Formally, the model assumes any finite set of outputs follows a multivariate normal distribution, written
\[ f(x) \sim \mathcal{GP}\big(m(x),\, k(x, x')\big) \]where \(m\) is the mean function and \(k\) the kernel. Conditioning on observed data pins the distribution down: predictions tighten near training points and the uncertainty band widens far from them.
The price is computation. Exact GP inference scales with the cube of the dataset size, which confines the method to small and medium datasets, or to sparse approximations for anything larger.
Example of a Gaussian Process
A team tuning a neural network’s learning rate treats validation accuracy as an unknown function of that rate. Each training run is one expensive evaluation of the function, so they want as few runs as possible.
They fit a Gaussian process to the first five runs. The GP now predicts accuracy across the whole range of learning rates, with tight bands near tested values and wide bands in unexplored regions.
The next run goes wherever predicted accuracy and uncertainty together look most promising; the GP updates, and the loop repeats. This routine, called Bayesian optimization, is the engine inside many hyperparameter tuning tools.
Related AI terms: Kernel Method · Normal Distribution · Bayesian Network · Regression Analysis · Hyperparameter Tuning
Did you like the Gaussian Process gist?
Learn about 250+ need-to-know artificial intelligence terms in the AI Dictionary.
Mihail Sebastian — Writes about AI governance, regulation, and the technology behind them. Placeholder bio — replace with a real credential line. About