Black Box Model
Last Updated: July 29, 2026 | By Mihail Sebastian | AI Dictionary
An AI model whose internal decision logic a person cannot read or trace, whether because the model is too complex or because its design is proprietary.
What is a Black Box Model?
A black box model is an AI model whose internal decision logic a person cannot read or trace. You see what goes in and what comes out; the reasoning in between is inaccessible.
The label matters because black boxes now make decisions people are entitled to contest. An applicant denied credit by a model no one can inspect has no way to check the decision or correct it.
How Models Become Black Boxes
Opacity arrives by two separate routes, and they demand different remedies.
The first is complexity. A deep learning network spreads its logic across millions of weights, and an ensemble method averages hundreds of trees; no person can follow either computation, including the people who built it.
The second is secrecy. A proprietary scoring model may even be structurally simple, but if the vendor guards it as a trade secret, it is a black box to everyone it judges. Complexity is a technical problem; secrecy is a legal and commercial one.
Working With Black Boxes
One response is to explain the box from outside: XAI techniques such as SHAP and LIME reconstruct which inputs drove a given output, producing explainability after the fact. The reconstruction approximates the model and can miss what it actually computed.
The stricter response is to refuse the box. Cynthia Rudin’s widely cited 2019 argument holds that high-stakes decisions should use models with built-in interpretability, whose logic is readable by design, rather than explained black boxes.
Example of a Black Box Model
COMPAS, a recidivism risk score used in US courts, shows the secrecy route. Its methodology is a trade secret, so defendants scored by it could not examine how their risk level was computed.
ProPublica’s 2016 “Machine Bias” investigation analyzed its outputs and reported that Black defendants were more likely than white defendants to be wrongly flagged as high risk. The vendor disputed the analysis, and the dispute itself illustrates the problem: with the model sealed, outsiders could argue only from its outputs.
The same year, in State v. Loomis, the Wisconsin Supreme Court allowed judges to consult the score but required warnings about its limits, in part because the methodology could not be examined. A sentencing input existed that no defendant could audit; that is what a black box means in practice.
Related AI terms: Interpretability · Explainability · XAI · Transparency · Deep Learning
Did you like the Black Box Model gist?
Learn about 250+ need-to-know artificial intelligence terms in the AI Dictionary.
Mihail Sebastian — Writes about AI governance, regulation, and the technology behind them. Placeholder bio — replace with a real credential line. About