Let's Talk

AI GLOSSARY

Consider this your AI Primer, sans the hype.

A working reference for the words that decide AI budgets. 161 terms, each defined in plain language and paired with what it actually changes about how you buy, build, measure and govern AI. Written for the people in the room when the money is committed — not for the people building the model. Every term has its own link, so you can send someone exactly one definition.

01 — Foundations

The words underneath every other word here. If a vendor deck uses these interchangeably, that tells you something.

Artificial IntelligenceAI

Software that does tasks normally needing human judgment — recognizing, classifying, predicting, generating, deciding. John McCarthy coined the name in 1955, in the proposal for the Dartmouth conference held the next summer. He wrote it with Marvin Minsky, Nathaniel Rochester and Claude Shannon. The name was chosen to christen a new field, not to describe a technique, and it has never had a technical meaning since. That is why it stretches from a 1970s rules engine to a frontier language model, and means almost nothing on its own.

Why it matters

"We use AI" is not a capability claim. It never was one, by design. Ask what the system does when it is wrong, how often that happens, and who finds out. Those answers separate a product from a demo.

See also

Context

Also called Prompt context

Everything supplied to a model for a single call — the system prompt, the user's message, retrieved documents, tool results and the conversation so far. It is the model's entire working memory: nothing outside it exists for that request, and nothing inside it persists to the next one unless you send it again.

Why it matters

Most "the AI does not know our business" complaints are context problems rather than model problems. The model is not failing to remember, because it was never told — and what it was told is a design decision somebody owns.

See also

Context Window

Also called Context length

The maximum amount of text a model can consider at once — the prompt, any retrieved documents, the conversation so far and the answer it is producing, all counted together. Exceed it and the earliest content is dropped or the request fails outright.

Why it matters

A large context window is not a substitute for retrieval. Filling one with everything you own is slow, expensive, and measurably worse than supplying the twenty passages that were actually relevant.

See also

Deep Learning

Machine learning that uses neural networks with many layers. The layers work out for themselves which features of the data matter, rather than a person choosing them first. It is what powers image recognition, speech-to-text and language models. The trade is raw power on messy inputs, paid for with more data and much less ability to explain the answer.

Why it matters

Deep models are where explainability gets hard. If a decision has to stand up to a regulator or a credit committee, that belongs in the design conversation, not the launch review.

See also

Foundation Model

Also called Frontier model

A large, general-purpose model trained once at great expense on broad data, then adapted to many downstream tasks by prompting, fine-tuning or tool use. The economic point is that the expensive part is shared: organizations build on a foundation model instead of training one, the way they build on a cloud platform instead of a data center.

Why it matters

Building on a foundation model means inheriting someone else's release schedule, pricing and deprecation policy. Design the system so the model is a swappable component, and that dependency stays commercial rather than structural.

See also

Generative AIGenAI

AI that produces new content — text, code, images, audio, video, structured data — rather than only scoring or classifying existing content. Generative systems predict plausible continuations of their input, which is why they are fluent, fast and capable of being confidently wrong in the same breath.

Why it matters

Generation is the easy half. The work that produces returns is everything around it: grounding the output in your data, constraining it to a task, and checking it before it reaches a customer.

See also

Inference

Running a trained model to get an answer, as opposed to training it in the first place. Training is a large one-off cost; inference is the recurring one, paid on every request for as long as the feature is live.

Why it matters

Programs are budgeted on build cost and then surprised by run cost. Inference is the line that scales with adoption — success makes it bigger, which is exactly when nobody wants to discover it.

See also

Knowledge Cutoff

Also called Training cutoff

The date after which a model's training data stops. Anything that happened later — a price change, a regulation, a reorganisation, a competitor launch — is simply absent, and the model has no way to know it is missing. Models are also unreliable narrators of their own cutoff, so asking one is not a check.

Why it matters

A large share of "the AI got it wrong" turns out to be this rather than hallucination, and the fix is different: retrieval and tools, not a better prompt or a bigger model. Worth establishing before anyone debugs the wrong problem.

See also

Large Language ModelLLM

A model trained on huge amounts of text to predict what comes next. That turns out to be enough to summarize, translate, reason over documents, write code and hold a conversation. An LLM has no database and no memory of your company. Everything it seems to know was either baked into its weights during training or handed to it in the prompt.

Why it matters

That split is the whole architecture decision. Knowledge that has to be current, private or auditable does not belong in the weights. It belongs in retrieval, where you can update it and cite it.

See also

Machine LearningML

A family of techniques in which a system learns patterns from data rather than following rules a person wrote. You supply examples and a target; the training process fits a model that maps one to the other, and the model is then used on inputs it has never seen. Forecasting, fraud scoring, churn prediction and recommendation are all classical ML, and most of them long predate generative AI.

Why it matters

A great deal of the highest-ROI work in an enterprise is still classical ML on tabular data. Reaching for a language model because it is the current news is how organizations pay frontier prices for a regression problem.

See also

Multimodal AI

A model that handles more than one kind of input at once — text with images, audio, video or files. It does this in a single context, rather than chaining separate tools together. So it can read a scanned contract, a chart and the email about them as one thing, then answer a question across all three.

Why it matters

Most company knowledge is stuck in formats that were never text: scans, screenshots, slides, recordings. Multimodality is what makes that backlog reachable, without a transcription project in front of it.

See also

Neural Network

Also called Artificial neural network, ANN

A model made of layers of simple units, each passing a weighted sum of its inputs through the next layer, where the weights are learned from data rather than written by hand. Nothing about it is neurological beyond the metaphor — it is arithmetic at scale, and the "learning" is a search for weights that make the output match the training examples.

Why it matters

It is the substrate under almost everything else here, which is why so many vendor claims quietly reduce to the same architecture with different training data. When two products describe themselves differently but both run a transformer, the difference you are buying is the data, the evaluation and the controls — not the maths.

See also

Token

The unit a language model actually reads and writes — roughly a word fragment. English averages about four characters per token, so 750 words is near 1,000 tokens. Context limits, latency and price are all denominated in tokens, not words or pages.

Why it matters

Every cost model for a generative feature starts here. Tokens in plus tokens out, times volume, times price, is the unit economics of the feature — and it is knowable before a line of code is written.

See also

02 — Models & Architectures

What a model actually is, how it comes to be, and the levers that change its cost and behavior.

Attention

Also called Attention mechanism, Self-attention

The mechanism that lets a model weigh every token against every other when deciding what each one means, so "it" can be connected to the right noun forty words back. It is the core idea of the transformer, and its cost grows with the square of the sequence length, which is why long contexts are expensive rather than merely large.

Why it matters

That quadratic cost is the reason context-window pricing works the way it does, and the reason a bigger window is not free capacity. Most efficiency work at the frontier is an attempt to get around this one curve.

See also

Distillation

Training a smaller model to reproduce the behavior of a larger one on a target task, keeping most of the quality at a fraction of the cost and latency. Distillation is how a capability proven on an expensive model becomes economic to run at scale.

Why it matters

A sensible arc for a high-volume feature: prove it on the best model available, instrument it, then distil once you know exactly what good looks like. Doing it in the other order optimizes something nobody has validated.

See also

Effort

Also called Reasoning effort, Thinking budget

A dial that says how hard a reasoning model should work on this one request. Low answers fast and cheap. High thinks longer, tries more approaches and checks itself before replying. It is the setting that turns test-time compute from a property of the model into a choice you make per call. Every major provider exposes one now, under slightly different names and scales.

Why it matters

It is the only AI cost control that also changes quality, in both directions. One prompt run at two settings can differ several-fold in price and seconds. The mistake is picking one and applying it everywhere. Match it to the decision: high on a contract review, low on a subject line. If nobody has set it, you are paying the default on every call.

See also

Embedding

Also called Vector representation

A list of numbers that stands in for a piece of content — a sentence, a document, an image — placed so that things with similar meaning sit close together. This is what lets a system search by meaning instead of by keyword. It sits underneath retrieval, clustering, de-duplication and recommendation.

Why it matters

Two documents can share no words at all and still mean nearly the same thing. Embeddings are how a system finds the policy that answers a question phrased in words the policy never uses.

See also

Fine-Tuning

Continuing to train an existing model on your own examples so it adopts a format, a house style or a specialized task. Fine-tuning changes behavior reliably; it is a poor way to teach facts, because the knowledge is frozen at training time and cannot be updated, cited or removed on request.

Why it matters

The common mistake is fine-tuning to fix wrong answers. Wrong answers are usually a retrieval problem, and retrieval is cheaper to build, cheaper to change and possible to audit.

See also

KV Cache

Also called Key-value cache

The saved working state from tokens already processed, kept so the model does not redo the whole sequence for each new token. It is what makes generation fast enough to be usable. It also eats memory in proportion to how long the context is and how many people are using it at once.

Why it matters

This is the hidden capacity limit behind self-hosting. How many conversations one GPU can handle is usually a KV cache question, not a model size question. Long contexts cost memory per user, not just tokens.

See also

Mixture of ExpertsMoE

An architecture that splits a model into many specialized sub-networks and activates only a few per token, so a very large model can be served at the cost of a much smaller one. It is a primary reason capability has kept climbing while price per token has kept falling.

See also

Open-Weight Model

Also called Open-source model, Self-hosted model

A model whose trained weights are published for download. You can run it on your own infrastructure, inspect it, and adapt it without an API call leaving your network. Open weights are not the same as open source: the training data and code usually stay private, and licences run from genuinely permissive to sharply restricted.

Why it matters

Read the licence before the benchmark. Limits on what you may use it for, buried in a permissive-sounding model, have killed more deployments than capability gaps have.

See also

Parameters

Also called Model size, Weights

The learned weights inside a model, counted in billions — the "7B" or "70B" in a model's name. More parameters generally means more capability and always means more memory and more cost to run: at sixteen-bit precision a 70B model needs roughly 140GB just to hold, before any serving overhead. Parameter count says nothing about training data quality, so a smaller model trained well routinely beats a larger one trained carelessly.

Why it matters

This is the number that decides whether you can self-host at all, and what it costs if you can. It is also the number vendors lead with because it sounds like capability — ask for benchmark results on your task instead, then ask what it costs per thousand requests.

See also

Quantization

Storing a model's parameters at lower numerical precision so it needs less memory and runs faster, with a small and measurable loss of quality. It is the standard technique for fitting a capable model onto modest hardware or a device.

See also

Reasoning Model

Also called Thinking model

A model trained to spend extra computation working a problem through before it answers, thinking in a scratchpad the user may never see. These models are much stronger on multi-step maths, code and analysis. They are also much slower and cost more per answer.

Why it matters

Not every task deserves deliberation. Sending the hard ten percent to a reasoning model and the rest to a fast one is one of the biggest cost levers you have. It is an architecture choice, not a model choice.

See also

Reinforcement Learning from Human FeedbackRLHF

A training stage in which human raters compare model outputs and their preferences are used to steer the model toward helpful, honest and harmless responses. It is the step that turns a raw text predictor into something that follows instructions and declines the requests it should decline.

Why it matters

A model's manners are a training artifact, not a law of nature. Two models with similar benchmark scores can behave very differently on refusal, tone and hedging — which is a product decision, so evaluate it like one.

See also

Sampling

Also called Top-p, Top-k, Nucleus sampling, Decoding

How the next token is picked from the probabilities the model produces. Temperature flattens or sharpens the distribution; top-p keeps only the most likely options that together reach a probability threshold; top-k keeps a fixed number. Greedy decoding always takes the most likely token and is the closest thing to deterministic output.

Why it matters

These settings, not the model, are usually what people mean by "it gives different answers every time". If reproducibility matters for an audit trail, it is a sampling decision and it should be recorded alongside the output.

See also

Small Language ModelSLM

A compact model — typically a few billion parameters — designed to run cheaply, quickly, and often on your own infrastructure or a device. Small models trail frontier models on open-ended reasoning but can match them on narrow, well-specified tasks such as classification, extraction and routing.

Why it matters

Most production volume is narrow, well-specified tasks. Sizing the model to the job rather than to the hardest thing the system ever does is where run-rate cost quietly halves.

See also

Speculative Decoding

Also called Draft model

A speed technique in which a small fast model drafts several tokens ahead and the large model verifies them in one pass, keeping what it agrees with. Output is identical to what the large model would have produced alone; only the wall-clock time changes.

Why it matters

It is one of the few optimisations with no quality trade-off to weigh, which is why providers apply it invisibly. Worth knowing so that "faster" in a release note is not mistaken for "different".

See also

Temperature

A sampling control over how much randomness enters generation. Low values make the model favor its most likely next token and produce repetitive, predictable text; higher values widen the field. Note that temperature 0 is not the same as deterministic — batching and floating-point ordering mean identical inputs can still diverge.

Why it matters

It is the knob teams reach for when output is wrong, and it is almost never the cause. Wrong facts are a grounding problem and wrong shape is a schema problem; temperature only changes how adventurous the wording is.

See also

Test-Time Compute

Also called Inference-time compute, Inference-time scaling

Spending more computation at the moment of answering rather than during training — letting a model reason for longer, sample several attempts, or check its own work before replying. It is the axis reasoning models scale on, and it converts money into accuracy per request instead of per run.

Why it matters

It moves a cost that used to be capital into operations. A model that thinks for thirty seconds may well be worth it on a loan decision and absurd on an autocomplete, so the question becomes per-task rather than per-model.

See also

Tokenizer

Also called Tokenization, Byte-pair encoding

The component that cuts text into the tokens a model actually consumes, using a fixed vocabulary learned before training. Common English words are usually one token; rare words, names, code and most non-English text fragment into several.

Why it matters

It is why the same sentence costs different amounts in different languages — text in a language the vocabulary handles poorly can take two or three times the tokens, and you pay per token. Worth checking before pricing a multilingual deployment.

See also

Transformer

The neural network architecture behind essentially every modern language model, introduced by Google researchers in 2017. Its defining move is attention: instead of reading a sequence strictly in order, the model weighs how much each part of the input should inform each other part. That is what allows a model to connect a pronoun to a name forty sentences earlier.

See also

Vision-Language-Action ModelVLA

Also called VLA model, Robot foundation model

A model that takes in what a camera sees and an instruction in plain language, and puts out the actual movements — where to send a joint, when to close a gripper. It is the same architecture family as a large language model, trained to produce motion instead of text. Google's RT-2 in 2023 is the one usually cited first.

Why it matters

This is the piece that made robots feel like they joined the AI boom. Before it, a new task meant new engineering; now it can mean new examples. The limits are real and worth naming: these models need a lot of demonstration data, they are slower than a hand-written controller, and they fail in ways that are hard to predict — which matters more when the failure has mass.

See also

03 — Training & Learning

How a model is actually made — the runs, the stages and the knobs. Further from a purchase order than the rest of this glossary, and worth knowing anyway: most of what a vendor can and cannot change about a model is decided here.

Backpropagation

Also called Backprop

The algorithm that works out how much each weight in the network contributed to the error, by propagating it backwards from the output layer to the input. It is what makes gradient descent tractable on a network with billions of weights rather than merely theoretical.

Why it matters

Mostly it is worth knowing as the reason training and inference cost completely different amounts. Running a model is one forward pass; training it is a forward pass plus a backward one, over and over.

See also

Base Model

Also called Foundation weights, Pretrained model

The raw output of pre-training, before any instruction tuning or preference work. It continues text rather than answering questions, has no notion of a conversation, and will happily produce anything its training data contained. Every assistant you have used is a base model with post-training on top.

Why it matters

Open-weight releases sometimes ship both, and the distinction matters: a base model is a component, not a product, and wiring one directly into a customer-facing surface is how organizations discover what "no guardrails" means in practice.

See also

Batch Size

How many examples the model sees before each weight update. Larger batches give a steadier estimate of the right direction and need proportionally more memory, which is usually what caps it in practice.

Why it matters

It is a hardware constraint wearing a modeling costume — the batch size a team can use is mostly a statement about the GPUs they have. Relevant when a quote for a training run turns out to be a quote for memory.

See also

Catastrophic Forgetting

Also called Forgetting

The tendency of a model to lose previously learned capability when trained on something new. Fine-tune hard on legal documents and general reasoning degrades; the model has not been extended so much as bent.

Why it matters

It is the single most common unpleasant surprise in a fine-tuning project, and it is invisible unless you kept a general evaluation set. Run the old tests after tuning, not only the new ones.

See also

Compute-Optimal Training

Also called Chinchilla scaling, Chinchilla-optimal

Given a fixed compute budget, the best split between model size and training data. Hoffmann et al. showed in 2022 that the models of the preceding era were substantially too large for the data they had seen, and that a smaller model trained on more tokens beats a larger one trained on fewer.

Why it matters

It is why parameter count stopped being the headline and why capable small models exist at all. A well-trained smaller model that runs on hardware you can afford is frequently the better commercial answer.

See also

Constitutional AI

Also called RLAIF, AI feedback

Training a model to critique and revise its own outputs against a written set of principles, so that much of the feedback comes from the model rather than from human labelers. The principles are explicit and published, which makes the target of the training inspectable in a way a pile of preference labels is not.

Why it matters

A written constitution is a governance artifact: you can read it, disagree with it and cite it. When comparing vendors on safety, ask whether the behavior was specified in a document or emerged from labeling instructions nobody kept.

See also

Direct Preference OptimizationDPO

Also called DPO

A way of tuning a model on human preferences that skips the reward model RLHF needs, optimizing directly on pairs of responses where one was judged better. Simpler to run and far cheaper, which is why most teams outside the frontier labs use it rather than full RLHF.

Why it matters

It put preference tuning within reach of organizations that could never have run RLHF. If a vendor says a model is "aligned to your preferences", this is usually the mechanism, and the question to ask is who wrote the preference pairs.

See also

Epoch

One complete pass over the training set. Small fine-tuning jobs run for a handful; frontier pre-training runs typically see most data once or twice, because at that scale fresh data beats repetition.

Why it matters

On a fine-tuning job it is the dial most likely to be set wrong. Too few and nothing is learned; too many and the model memorises the examples instead of the pattern, which looks like success on the training set and failure everywhere else.

See also

Federated Learning

Also called Federated training

Training one shared model across many places without moving the data. Each site trains on what it holds and sends back only the changes to the model; a coordinator averages them. The raw records never leave. Google published the method in 2016 and used it on phone keyboards.

Why it matters

It is the answer when the data that would make a model good is data you cannot pool. Patient records across hospitals. Transactions across banks. Anything sitting behind a residency rule. Two honest caveats. It is slower and fiddlier than normal training. And the updates themselves can leak something about the data, so "the data never moved" is not the same as "nothing got out".

See also

Gradient Descent

Also called Stochastic gradient descent, SGD

The optimization method behind essentially all model training: work out which direction each weight should move to reduce the loss, take a small step that way, repeat several million times. The "gradient" is the slope of the loss with respect to every weight at once.

Why it matters

It explains why training is expensive and why it is not a one-off command — it is millions of small corrections, and the cost is the number of steps times the size of the model.

See also

Instruction Tuning

Also called Instruction following

Training a base model on examples of instructions paired with good responses, so it answers questions instead of continuing text. It is the first step of post-training and the one that turns a text predictor into something recognisably an assistant.

Why it matters

It is also the cheapest place to teach a model your formats and conventions — a house style, a required output shape, a refusal policy — without touching anything deeper.

See also

Learning Rate

Also called Learning rate schedule

The size of the step taken at each update. Too large and training overshoots and destabilises; too small and it crawls or settles somewhere mediocre. It is usually scheduled rather than fixed — larger early, decayed as the run proceeds.

Why it matters

It is the hyperparameter most responsible for a fine-tune that produced nothing, and the first thing to ask about when one does. Also the clearest example of why fine-tuning is an experimental process with a cost per attempt, not a build step.

See also

LoRA

Also called Low-Rank Adaptation, PEFT, Parameter-efficient fine-tuning, Adapter

Low-Rank Adaptation — fine-tuning that freezes the original weights and trains a small set of new ones alongside them, typically well under one percent of the model's size. The result is a lightweight adapter file rather than a whole new model, and several can be swapped in and out against one shared base. The broader family is parameter-efficient fine-tuning.

Why it matters

It is the reason fine-tuning stopped being a frontier-lab activity. One hosted base model can serve many tuned variants, so the cost question moves from "can we afford to train" to "can we maintain the evaluation set for each adapter".

See also

Loss Function

Also called Objective function, Cost function

The number training is trying to make smaller — a measure of how wrong the model's output was on each example. Every weight update is a step toward reducing it, so the loss function is the operational definition of "better" for the entire run.

Why it matters

It is where a modeling objective and a business objective quietly diverge. A model with excellent loss can be useless if the loss rewarded something adjacent to what you wanted, which is the same problem as a team hitting its KPI and missing the point.

See also

Model Collapse

Also called Model autophagy

The degradation that sets in when models are trained on output from earlier models, generation after generation. Rare cases at the tails of the distribution disappear first, so the result grows blander and more confident while looking superficially fine.

Why it matters

It is the structural argument for knowing your data provenance, and it is getting harder as more of the web becomes model output. If you generate synthetic training data, keep the human-authored set and keep measuring against it.

See also

Overfitting

Also called Underfitting, Memorisation

When a model learns the training examples rather than the pattern behind them — excellent on data it has seen, poor on anything new. The mirror failure is underfitting, where it has not learned enough to be useful either. A held-out test set is what tells the two apart.

Why it matters

Any accuracy figure measured on data the model trained on is meaningless, and this is how that happens honestly rather than dishonestly. Ask which data the number came from before you ask how high it is.

See also

Post-training

Also called Alignment training

Everything done to a base model to make it useful and safe. First instruction tuning, then preference tuning through RLHF or DPO, plus safety work and often distillation. Set against pre-training it costs almost nothing. It is also responsible for nearly everything a user experiences as the model's character.

Why it matters

Two products built on the same base model can behave nothing alike, and this is where the difference comes from. If you are comparing vendors running the same weights, post-training and evaluation are the whole comparison.

See also

Pre-training

Also called Pretraining

The first and by far the most expensive stage: one very long run over a vast corpus of text, in which the model learns to predict what comes next and, in doing so, absorbs grammar, facts, reasoning patterns and a great deal of the internet's opinions. It produces a base model that completes text but does not yet follow instructions or refuse anything.

Why it matters

This is the stage you will never repeat and cannot meaningfully influence — it costs tens of millions and months of compute. Everything a vendor offers you happens after it, which is why "we trained our own model" almost always means they fine-tuned someone else's.

See also

Reinforcement LearningRL

Also called RL

Learning by trial against a reward signal rather than from labeled examples: the system acts, gets scored, and adjusts toward whatever scores well. It is how game-playing systems were trained, and through RLHF and its successors, how language models are shaped after pre-training.

Why it matters

The whole method turns on the reward being right. Reinforcement learning optimizes what you measure with more discipline than any organization ever does, which is why a badly chosen reward produces a system that is technically excellent and useless.

See also

Reward Model

Also called Preference model

A second model trained to predict which of two responses a human would prefer, used as a stand-in for human judgment so the main model can be optimized against it at scale. It is the piece that makes RLHF possible and the piece most likely to be gamed.

Why it matters

Optimizing hard against an imperfect proxy is the textbook route to reward hacking — the model learns what scores well rather than what is good. It is the same failure any organization invites when it ties incentives to a single metric.

See also

Self-Supervised Learning

Also called Next-token prediction

Learning from unlabelled data by hiding part of it and predicting the hidden part — for language models, predicting the next token. The labels come free from the data's own structure, which is what made training on internet-scale text possible at all.

Why it matters

This is the trick behind the whole generation of models. It is also why training data quality is a capability question rather than a compliance one: with no labeler in the loop, whatever is in the corpus is what gets learned.

See also

Sim-to-Real

Also called Sim2real, Domain randomization

Training a robot in simulation, then moving what it learned onto the real machine. Simulation is cheap and fast, so a policy can practise years of simulated time in a weekend. The catch is that no simulator is right. Friction, light, wear and the odd shove are all a little off, and a policy tuned to the simulator can fall apart outside it.

Why it matters

This gap is where robotics projects actually fail, and it is not a hardware problem. The standard answer is to vary the simulation deliberately — change the physics, the lighting, the textures on every run — so the policy learns what does not matter. Ask any robotics vendor how much of their training was simulated and how they closed the gap. A vague answer is the answer.

See also

Supervised Fine-TuningSFT

Also called SFT

Continuing training on a curated set of input–output pairs that demonstrate the behavior you want. "Supervised" means every example carries the right answer, written by someone. It is the workhorse of post-training and the form of fine-tuning most vendors actually sell.

Why it matters

Quality beats quantity by a wide margin here — a few thousand carefully written examples routinely outperform a hundred thousand scraped ones, and the cost is in the writing, not the compute. Budget for the people, not the GPUs.

See also

Supervised Learning

Also called Classification

Learning from examples that carry the right answer — emails labeled spam or not, invoices labeled approved or rejected. The model learns the mapping from input to label, and the labels have to come from somewhere, usually people.

Why it matters

When a use case reduces to supervised learning, a small classical model often beats a language model on cost, latency and accuracy together. The binding constraint is almost never the algorithm; it is whether anyone labeled enough examples.

See also

Training Data

Also called Training corpus, Training set

The corpus a model learns from. For pre-training it is web text, books, code and licensed collections at a scale nobody reads end to end; for fine-tuning it is whatever you supply. Composition determines capability, bias and much of the legal exposure, and frontier labs disclose it in varying and generally decreasing detail.

Why it matters

Two questions cover most of the risk: what was in it, and can yours end up in the next one. The first is a provenance and copyright question you may not get a full answer to; the second is a contract term, and the default differs by provider and tier.

See also

Transfer Learning

Taking a model trained on one task and adapting it to another, rather than starting from nothing. The general capability built during pre-training carries over, and only the last stretch is task-specific — which is the premise the entire foundation-model industry rests on.

Why it matters

It is why you do not need millions of examples any more. A task that would once have required a bespoke model and a labeling program now often needs a few hundred examples on top of something that already exists.

See also

Unsupervised Learning

Also called Clustering

Finding structure in data with no labels — clustering customers, spotting anomalies, reducing dimensions. Nobody says what the right answer is, so the model reports patterns and a person decides whether they mean anything.

Why it matters

It is useful exactly where you do not yet know what you are looking for, and dangerous for the same reason: a clustering will always return clusters. The finding needs a human test before it becomes a decision.

See also

04 — Building with AI

The craft layer — how instructions, tools and constraints turn a general model into a system that does one job reliably.

AI-Native

A system or process designed around what AI can do, rather than an existing workflow with AI added to one step. AI-native design assumes the model in the architecture from the start — which typically means fewer handoffs and different roles, not the same process running faster.

Why it matters

Bolting a model onto an unchanged process caps the return at whatever that one step was worth. The compounding gains come from redesigning the work, which is an operating decision before it is a technical one.

See also

Chain-of-ThoughtCoT

Prompting a model to work through intermediate steps before committing to an answer, which improves accuracy on problems that need more than recall. The stated reasoning is a useful artifact but not a guaranteed account of how the answer was produced.

Why it matters

Do not treat a model's explanation as an audit trail. If a decision must be defensible, the evidence has to come from the sources and checks around the model, not from its narration of itself.

See also

Context Engineering

Deciding what actually occupies the context window on each call — which retrieved passages, how much conversation history, which tool results, what memory, in what order — as distinct from how the instruction is worded. As systems moved from single prompts to agents and retrieval, this became the larger half of the job.

Why it matters

Most quality problems blamed on the prompt are really about what was in the window beside it. It is also where cost and latency are decided, since every token assembled here is paid for on every call.

See also

Edge AI

Also called On-device AI, Edge inference

Running the model where the data is made, instead of sending it to a data centre. On the camera. On the machine. On the vehicle, or the store's own server. On-device AI is the tightest version, where it runs on the end device itself, like a phone. This is why models get shrunk: quantization, distillation and small models all exist to fit something into a small place.

Why it matters

Four reasons people do it, and only one is speed. The data never leaves, so a residency rule stops being a problem. The round trip disappears, so the answer is there in milliseconds. There is no per-token bill. And it still works when the network does not. The cost is that you now have models deployed on hardware you have to update, which is a fleet problem, not an AI one.

See also

Few-Shot Prompting

Also called In-context learning

Including a handful of worked examples in the prompt so the model infers the pattern you want instead of being told it in the abstract. Showing three correctly formatted answers is usually more effective than three paragraphs describing the format.

See also

Graph Engineering

Also called Task graph, Agent graph

Designing the shape of a system that several agents work inside: which units of work exist, what may pass between them, what they share, and who decides where the work goes next. The name arrived in mid-2026 and is not settled, and it gets used for two different things. Say which you mean. A task graph is how work moves through one run. A knowledge graph is what the system knows about the business, and it outlives every run.

Why it matters

It is where independent review becomes real. One agent in a loop checks its own work in the same context that produced it, which is not a check. Split the work across separate steps and the reviewer can be handed a clean context and the power to send the job back. The cost is that you have to decide the structure before you run it, instead of finding out later where it broke.

See also

Guardrails

Checks wrapped around a model that limit what goes in and what comes out. They cover input filters, output checks, required formats and refusal rules. They also cover caps on rate and spend, and when to hand a case to a person. Guardrails are plain, predictable code around an unpredictable part.

Why it matters

You cannot make a generative system incapable of being wrong. You can make it incapable of being wrong in ways that reach a customer unchecked. That is the difference between a pilot and a deployment.

See also

Harness

Also called Agent harness, Harness engineering

Everything around the model that makes one agent run: the tools it may call, the files it can see, the permissions it holds, the sandbox it works in, and what gets assembled into its context each turn. The model is bought. The harness is built. Two teams on the same model get very different results because of it.

Why it matters

When an agent underperforms, the harness is usually the cause and the model gets the blame. It could not see the file. The tool was not wired up. It was never told the convention. Swapping in a better model fixes none of that. The useful question is what this agent can actually reach, and what happens when it reaches for something it should not.

See also

Human in the LoopHITL

Also called Human oversight

A design in which a person reviews, approves or corrects the system's output before it takes effect — usually on the consequential subset rather than everything. Well built, the review queue is also a training signal: every correction is a labeled example of what good looks like.

Why it matters

The right question is not whether to keep a human in the loop but which decisions require one. Answer it explicitly and you get a defensible control; leave it implicit and you get a rubber stamp.

See also

LatencyTTFT

Also called Time to first token, Response time

How long the system takes to answer. Two numbers matter and they behave differently: time to first token, which is when the response starts appearing, and total completion time. With streaming, the first is what a user experiences as speed, even when the second is unchanged.

Why it matters

Latency decides whether a feature gets used inside a live conversation or gets routed around. Set the target before choosing the architecture — reasoning models, multi-step agents and large retrieved contexts each cost seconds, and the decision is far more expensive to revisit afterwards.

See also

Loop Engineering

Designing the system that runs an agent, rather than prompting the agent yourself. You set the goal, the tools it may call, what it keeps between passes, how its output gets checked, and what makes it stop. Geoffrey Huntley showed the pattern in July 2025 with a coding agent in an endless shell loop. Addy Osmani named the practice in June 2026.

Why it matters

It is the third floor of the same building. A prompt shapes one call. Context engineering shapes what the model sees each turn. A loop shapes whether the agent ever reaches a good answer at all. The risk moved with the leverage: a loop running unattended is a loop making mistakes unattended, so the check on its output now matters more than the wording of its instruction.

See also

Markdown

Also called MDX

A plain-text way of marking up writing — asterisks for bold, `#` for a heading, hyphens for a list. It predates AI by two decades. Models default to it because so much of what they learned from was written in it, which is why answers arrive full of `##` and `**` when nothing renders them. It is now also the format agent instructions are written in.

Why it matters

It is the human-facing half of a story whose machine half is structured output. JSON is what another system consumes. Markdown is what a person reads. Knowing which you asked for is the difference between output you can pipe and output you can paste. It is also why agent setup is plain files rather than a console. A skill, a memory file, a set of house rules: all markdown. So they live in your repo, get reviewed in a pull request, and diff like code.

See also

Model Context ProtocolMCP

Also called MCP server

An open standard, introduced by Anthropic in late 2024 and since adopted across the industry, for connecting AI applications to external tools and data sources through a common interface. It replaces bespoke per-integration glue with one protocol that any compliant client and server can speak.

Why it matters

Integration work is where AI budgets actually go. A shared protocol turns each connector into an asset you keep rather than plumbing you rewrite the next time the model or the vendor changes.

See also

Model Routing

Classifying each request and sending it to the cheapest model that can handle it, rather than sending everything to the best one. A small model takes the routine majority; a frontier or reasoning model takes the hard remainder.

Why it matters

Most production traffic is easier than the hardest thing a system has to do, so paying frontier prices for all of it is a choice. Routing is usually the difference between a feature that survives scale and one that gets switched off in a cost review — and it is an architecture decision, not a model decision.

See also

Orchestration

The layer that decides what runs in what order: which model handles a request, which tools it may call, how retries and failures are handled, what gets logged, and when a human is brought in. It is ordinary software engineering, and it is most of the system.

Why it matters

The model is the component teams argue about and the orchestration is the component that determines whether the thing works. Building it well is also what keeps the model swappable.

See also

Prompt Engineering

Designing the instructions, examples and structure you give a model. The goal is output that is right every time, not right now and then. In production it looks less like clever phrasing and more like writing a spec: the task, the format, the edge cases, and what to do when the input is unusable.

Why it matters

A prompt is production code. Version it, test it against a fixed set of cases, and change it as carefully as anything else customers depend on. A one-word edit can move accuracy several points.

See also

Skill

Also called Agent skill, SKILL.md

A folder of written instructions an agent loads when the job calls for it — conventions, steps, the thing you do not do any more because of that one incident. Usually a `SKILL.md` file plus any scripts or reference material it needs. The agent reads it on the runs where it is relevant and ignores it otherwise. Anthropic defined the format and other tools have adopted it.

Why it matters

Without one, an agent starts every session knowing nothing about you, and fills the gaps with confident guesses. A skill is that knowledge written down on the outside. Once, in a place it gets read every run. It is also the cheapest of the three options. A prompt lasts one conversation. Fine-tuning changes the model and is priced accordingly. A skill is a file in your repo that anyone can read and edit.

See also

Structured Output

Also called JSON mode, Constrained decoding

Constraining a model to return data in a fixed shape — JSON matching a schema, a defined set of labels, a specific field list — rather than prose a downstream system then has to parse. Most providers now enforce this during generation rather than hoping the instruction is followed.

Why it matters

This is the line between a demo and something another system can consume. Note what it does and does not buy you: the shape is guaranteed, the contents are not. A schema-valid response can still be entirely wrong, so validation of values is still yours.

See also

System Prompt

The standing instruction that governs a model's behavior across every request in an application — its role, its boundaries, its tone, its output format. Users supply the question; the system prompt supplies the job description.

Why it matters

This is where most of a product's behavior actually lives, and where policy gets encoded in practice. It belongs under review and version control, not in a text box someone edits on a Friday.

See also

Tool Use

Also called Function calling

Giving a model the ability to call software — a search index, a database query, a pricing API, a calculator — and use the result in its answer. The model decides which tool fits and with what arguments; your code decides what tools exist and what they are permitted to do.

Why it matters

This is the line between a chatbot and a system that does work. It is also where permissions become real: a tool that can write is a tool that can be wrong in production, so scope it like a service account.

See also

Workflow Recomposition

Rebuilding a process once AI is in it. Some steps automate. Some become human-AI loops. Some vanish, because they only existed to work around a constraint that is now gone. This is not augmentation, where every step stays and gets help, and not replacement, where the whole process is handed over.

Why it matters

Which of the three patterns a workflow needs is the strongest single predictor of whether the investment pays back. It is also the question most programs never actually ask.

See also

05 — Agents & Automation

Software that decides its own next step. The most oversold category in the market, and the most consequential when it works.

Agent Memory

How a system carries information across steps and sessions, since the model itself remembers nothing between calls. Short-term memory is the running context; long-term memory is an external store the agent writes to and retrieves from deliberately.

Why it matters

Memory is a data asset and a liability at once. Whatever an agent remembers about a customer is now personal data you hold, with a retention policy, an access model and a deletion obligation attached.

See also

Agentic AI

The broad category of AI systems that act rather than only answer. In practice the label spans everything from a model calling two tools in sequence to a long-running process that manages a workflow for days, so it says far more about ambition than about architecture.

Why it matters

The useful question is never "is it agentic". It is how many steps it takes unsupervised, what it can change, and what happens on step seven when step three was wrong.

See also

AI Agent

A system that pursues a goal over multiple steps, choosing its own actions along the way — calling tools, reading results, revising its approach — rather than producing one answer to one prompt. The distinguishing feature is not intelligence but discretion: nobody specified the sequence in advance.

Why it matters

Discretion is the thing to scope. An agent that can read is a research assistant; an agent that can write to a system of record is an employee, and it needs the permissions, logging and review that implies.

See also

Copilot

AI that assists a person inside the work they are already doing — suggesting, drafting, completing — where the person invokes it and decides whether to accept what comes back. The distinguishing feature is that a human is in the loop by construction, not by policy.

Why it matters

The honest comparison for most enterprise decisions is copilot versus agent, and it is a question about accountability rather than capability. A copilot's output has a named person's judgment attached to it before it goes anywhere. An agent's does not, which is what makes the governance work bigger.

See also

Loop

Also called Agent loop, Agentic loop

An agent's basic working cycle: take a goal, plan, call a tool, look at what came back, decide whether the job is done, and go again. A single model call answers a question. A loop keeps going until a stopping rule fires. Almost everything sold as an AI agent is a loop with tools attached.

Why it matters

The stopping rule is the part that gets skipped. A loop without one either declares the job finished when it is half done, or never stops and bills you all night. Ask what ends the loop before you ask what the model can do.

See also

Multi-Agent System

An architecture in which several specialized agents work on parts of a problem and coordinate — one researches, one drafts, one checks — usually under a coordinating agent. It buys parallelism and separation of concerns at the cost of more failure modes and much harder debugging.

Why it matters

Most problems sold as multi-agent are one agent with better tools. Add agents when the work genuinely parallelises, not because the diagram looks more impressive with boxes in it.

See also

Robotic Process AutomationRPA

Rule-based automation that replays a fixed sequence of user-interface or API actions — the generation of automation that preceded AI agents. RPA is deterministic and auditable, and it breaks the moment a screen, a form or an exception falls outside its script.

Why it matters

The interesting pattern is not replacing RPA but pairing it: the model handles the judgment and the unstructured input, the deterministic path handles the transaction. Keeping the irreversible step deterministic is usually the right instinct.

See also

06 — Data & Retrieval

How a model gets access to what your organization knows — the part that determines whether output is grounded or invented.

Agentic RAG

Also called Agentic retrieval, Iterative retrieval

Retrieval where the model decides what to look for. It can rewrite a vague question into a better search, look again, notice that what came back does not answer the question, and try once more. Ordinary RAG runs one search and works with whatever it gets.

Why it matters

It is the usual answer to questions that need two or three hops, where the first document only tells you what to search for next. It also makes cost and speed hard to predict, because the model chooses how many searches to run. Cap the loop before production. Log the searches it actually ran, or you will not be able to explain an answer later.

See also

Chunking

Splitting documents into passages small enough to retrieve precisely and large enough to still make sense on their own. Chunk on the document's actual structure — sections, clauses, rows — rather than every N characters, or you will routinely retrieve half a sentence and half an answer.

Why it matters

When a retrieval system gives confidently incomplete answers, chunking is the first place to look. It is unglamorous, it is rarely the thing anyone demos, and it moves quality more than swapping models does.

See also

Context Rot

Also called Lost in the middle

Degrading answer quality as the context grows, even well inside the window. Liu and colleagues documented the shape of it in 2023: models attend most reliably to material at the beginning and the end of a long context and can miss what sits in the middle, so adding more passages can make a system worse.

Why it matters

It is the direct rebuttal to "we have a large context window, so just put everything in". Fewer, better-ranked passages beat more of them, and where a critical document sits in the prompt is a design decision rather than an implementation detail.

See also

Data Readiness

Whether the data an AI use case depends on actually exists, is accessible, is current, and is trustworthy enough to act on. Readiness is assessed per use case, not per organization: the same company can be fully ready for one workflow and nowhere near it for the one next to it.

Why it matters

This is the most common reason a promising use case stalls, and it is knowable in days rather than quarters. Assessing readiness before committing to a build is the cheapest risk reduction available.

See also

GraphRAG

Also called Graph RAG, Knowledge graph RAG

A retrieval method that builds a knowledge graph before it answers anything. A model reads your documents and pulls out the people, places and things in them, plus how they connect. It groups the closely linked ones and writes a summary of each group. Questions are then answered from those summaries, not from single passages. Microsoft Research published the method in April 2024.

Why it matters

It answers what ordinary RAG cannot. "What are the main themes here" is a summarizing job, not a lookup. Fetching the top ten passages will always miss it. The trade is real: a model has to read your whole corpus first, and again each time the corpus changes. Worth it when people ask broad questions of a lot of text. Wasteful when they are just looking things up.

See also

Grounding

Tying a model's output to verifiable sources — retrieved documents, tool results, database records — so each claim can be traced back to something outside the model. Grounding is the mechanism; citation is the visible part of it.

Why it matters

Ungrounded output cannot be checked at the speed it is produced, which means it cannot safely be used at that speed either. Grounding is what makes review possible without re-doing the work.

See also

Knowledge Graph

A structured representation of entities and the relationships between them — this company owns that subsidiary, which holds this contract, which references that clause. Where retrieval finds passages that look relevant, a graph answers questions about how things connect.

Why it matters

Questions that traverse relationships — exposure across a portfolio, dependency across a supply chain — are exactly where pure retrieval underperforms and where the graph earns its keep.

See also

OpenRAG

Also called Open RAG

An open-source package for building agentic RAG systems, started at IBM. It bundles three existing projects into one stack: Langflow to build and edit the flows, OpenSearch for hybrid search and vector storage, and Docling to parse documents. Every part is meant to be swapped — the model, the search engine, the wiring between them.

Why it matters

"Open" here means you can take it apart and host it yourself, and that is the choice it puts in front of you. Run the whole thing inside your network. Or keep the documents and the index in-house and send only the prompt to a hosted model. Or use a vendor's managed stack. Those three options put your data in three different places, which is a governance answer before it is a technical one.

See also

Permission-Aware Retrieval

Also called Access-controlled retrieval, Entitlement-aware search

Retrieval that applies the asker's own access rights before deciding what the model may see. Without it, a search index built across everything the organization holds will happily surface a passage from a document the person asking was never cleared to open — and the model will summarize it for them in plain language.

Why it matters

This is the most common serious failure in enterprise RAG, and it does not look like a bug: the system works exactly as built. Permissions have to be enforced at retrieval time against the requesting user, not bolted on as a filter over results, and certainly not left to a prompt asking the model to be discreet.

See also

Reranking

Also called Re-ranker, Cross-encoder

A second pass that reorders retrieved results before they reach the model, using a slower and more accurate model than the one that fetched them. The first pass optimizes for recall over a large corpus; the reranker reads the shortlist properly and puts the genuinely relevant items at the top.

Why it matters

Context is finite and position matters, so what reaches the model in the first few slots is what it answers from. Reranking is usually the cheapest single improvement available to a retrieval system that is fetching roughly the right documents and still answering badly.

See also

Retrieval-Augmented GenerationRAG

Also called Retrieval augmentation, Grounded generation

A pattern where the system searches your own content for passages relevant to a question, then puts them in the prompt. The model answers from the evidence supplied rather than from memory. It is the standard way to make a general model useful on private, current or regulated information, and the standard way to get citations.

Why it matters

RAG turns "the AI said so" into "this document says so, here it is". That one property is what makes a generated answer acceptable when somebody has to stand behind it.

See also

Semantic Knowledge Graph

Also called Ontology, RDF graph

A knowledge graph that also writes down what its own terms mean. Not only that this company owns that one, but what counts as a customer, that every subsidiary is a company, that "supplies" runs one way and not the other. That set of rules is called an ontology. It is written in a standard a machine can read, so software can work out facts nobody ever stored. Told that Santiago is a capital, and that capitals are cities, it knows Santiago is a city.

Why it matters

It is the difference between a graph that holds your data and one that can police it. The rules reject a question the business would call nonsense before it returns rows, which matters most when a model is writing the query. The cost is the modelling itself, and it is real work. A plain graph is faster to stand up and cannot tell you when an answer is meaningless.

See also

Synthetic Data

Data generated by a model rather than collected from the world, used to augment scarce training sets, cover rare cases, or stand in for records that cannot leave a controlled environment. It is genuinely useful and it inherits every bias and blind spot of whatever produced it.

Why it matters

Synthetic data is a good way to test a pipeline and a poor way to establish that a system works. Validation has to happen against real cases or it has not happened.

See also

Vector Database

A store built to hold embeddings and find the nearest ones to a query fast. It is the index behind semantic search and retrieval, and it is increasingly a feature of databases you already run rather than a separate product you must buy.

Why it matters

Check what your existing data platform already does before adding a system of record for meaning. A second store is a second thing to secure, sync, back up and explain to an auditor.

See also

07 — Evaluation & Measurement

How you know it works, and how you prove it kept working. The discipline most AI programs skip, then fail on.

Baseline

The measured performance of the current process before AI — cycle time, error rate, cost per case, throughput. Captured before the build, not reconstructed afterwards from memory or from the best week anyone remembers.

Why it matters

A missing baseline is the most common reason a working deployment cannot prove it worked. Half a day of measurement up front is what makes every later ROI claim defensible rather than rhetorical.

See also

Benchmark

A standardized public test used to compare models — reasoning, coding, retrieval, domain knowledge. Benchmarks are useful for tracking the frontier and weak evidence about your workflow, since a model tuned to score well on a public test has been optimized for that test, not your task.

Why it matters

Nobody buys a model. They buy performance on a workflow. Benchmark leadership is a reason to try something and never a reason to deploy it.

See also

Calibration

Also called Confidence calibration

Whether a model's stated confidence means anything. A well-calibrated system is right about 70% of the time when it says 70%. Base models are often reasonably calibrated. The alignment training that makes a model helpful and polite has been shown to make this worse, which is why a fluent, confident tone is not evidence of anything.

Why it matters

Any design that routes on confidence — escalate below a line, auto-approve above it — is only as good as the calibration underneath. That has to be measured on your own task, not assumed. An uncalibrated threshold quietly sends the wrong cases to the wrong place.

See also

EvaluationsEvals

Also called Evaluation harness, Test set

A fixed set of test cases with known-good outcomes, run automatically against the system so you can tell whether a change made it better or worse. Evals are to an AI feature what a test suite is to software, and they are the only way to upgrade a model without guessing.

Why it matters

Without evals, every prompt edit and model upgrade is a coin flip nobody can score. Teams that build them early ship faster within a quarter, because they stop re-litigating whether the last change helped.

See also

Four-Lens ROI

Also called Four-Lens Reconciliation

The AWSM LABS framework for scoring an AI deployment on four instruments that measure different things: operational, financial, experiential and custodial. The discipline is not to average them. It is to find where they disagree. Agreement confirms what you already suspected; a wide spread shows you where the value is really sitting, or leaking.

Why it matters

Measuring on one lens does not under-report AI value. It gives you a confident wrong answer. The gap between your highest and lowest lens tells you more than any single score.

Ground Truth

Also called Gold set, Labeled data, Labelled data

The set of answers you have agreed are correct, against which a system's output is scored. It is produced by people, it costs real time to build, and it is the part of an evaluation most likely to be missing — which is why so many AI programs report accuracy figures with nothing underneath them.

Why it matters

No ground truth, no measurement — only impressions. Before accepting any accuracy claim, internal or from a vendor, ask what it was measured against, who decided those answers were right, and how many there were. A hundred well-chosen labeled cases from your own work beat any public benchmark.

See also

Hallucination

Also called Confabulation

Output that is fluent, plausible and wrong — an invented citation, a policy clause that does not exist, a number with no source. It is not a bug to be patched out but a property of systems that predict likely continuations, which is why the engineering response is grounding, validation and review rather than a fix.

Why it matters

The danger is not that a model is wrong. It is that wrong output is indistinguishable from right output at a glance — so the control has to be structural, not vigilance.

See also

LLM-as-Judge

Using one language model to score another model's output against a rubric. It lets you evaluate at a scale human review could never reach. On clear criteria it lines up reasonably well with human judgment. It also picks up the judge's own biases — toward length, toward confidence, toward its own house style.

Why it matters

Calibrate the judge against a human-scored sample before you trust it, then re-check it now and then. An uncalibrated judge gives you a quality score that is precise, cheap, and unrelated to what your users experience.

See also

Model Drift

Also called Data drift, Performance drift

Degradation in a deployed system's performance over time as the world moves away from the conditions it was built for — new products, new language, new customer behavior, or a provider updating the model underneath you. Nothing breaks; results quietly get worse.

Why it matters

Drift is why an AI deployment needs monitoring rather than a launch date. The failure mode is silent, and the first person to notice is usually a customer.

See also

Perplexity

A measure of how surprised a model is by text it did not generate — lower means it predicted the words better. It is the standard metric for comparing language models during training, and it says nothing about whether answers are correct, useful or safe.

Why it matters

It appears in model cards and research papers and should never appear in a procurement decision on its own. A model with better perplexity can be worse at your task; the only measurement that settles that is your own evaluation set.

See also

Precision and Recall

Also called F1 score, False positives

The two halves of accuracy, which trade against each other. Precision is how much of what the system flagged was actually right; recall is how much of what was there it actually caught. A fraud check tuned for recall stops more fraud and more legitimate customers; tuned for precision it annoys fewer people and misses more fraud. F1 is their harmonic mean, useful as shorthand and misleading when the two costs differ.

Why it matters

"99% accurate" is not a claim, because it does not say which kind of wrong. Which error costs you more is a business decision, not a modeling one, and answering it before procurement is how you avoid discovering the trade-off through complaints.

See also

Workflow–Tool Fit

Scoring an AI tool against one specific workflow instead of ranking tools in the abstract. The same tool can be excellent on one task and actively harmful on the next. AWSM RSRCH scores the pair on five dimensions: outcome efficacy, adoption velocity, integration fit, total cost of ownership and deployment risk.

Why it matters

Tool selection debates usually cannot be settled because the question is malformed. Name the workflow and the same debate turns into a measurable comparison.

See also

08 — Governance & Risk

The obligations, frameworks and failure modes a board, a regulator, an LP or an auditor will eventually ask about.

AI Governance

The structures that determine who may deploy AI, on what, under what controls, and who answers for the outcome. In practice it is an inventory of what is running, a risk tier per system, named owners, documented evaluation, and monitoring after launch — not a policy document.

Why it matters

The most common governance finding is not a bad control. It is that nobody can say who owns a system that is already making decisions — which is an unbounded position, not an oversight.

See also

Algorithmic Bias

Also called Fairness

Consistent, unfair differences in how a system performs across groups of people. It comes from the training data, from how the goal was framed, or from how the output gets used. Bias can be measured — but only once you have decided in advance which differences would be unacceptable.

Why it matters

A model that is accurate on average can still be unusable for a subgroup that matters legally or commercially. Aggregate accuracy hides exactly the failure that produces a complaint.

See also

Audit Trail

Also called Traceability

A lasting record of what the system did. It holds the inputs, the evidence it pulled, the model and prompt version, the tools it called, the output, and any human review. The point is that someone who was not there can rebuild the decision months later.

Why it matters

The first serious question after an AI incident is what the system saw and what it did. Logging designed in from the start answers that in an afternoon. Logging added afterwards cannot answer it at all.

See also

Data Provenance

Also called Data lineage

Where a piece of data came from, how it has been transformed since, and what you are permitted to do with it. Distinct from an audit trail, which records what your system did: provenance records what the data is and whether it should have been there at all.

Why it matters

Training or retrieval quietly pulls data across boundaries it was never licensed to cross — a customer contract, a third-party feed, a scraped corpus. The question arrives later, from a customer's counsel or a regulator, and it is unanswerable retrospectively if nobody recorded the lineage at ingest.

See also

Data Residency

Where data is physically processed and stored, and which jurisdiction's law therefore applies to it. For AI this covers prompts, retrieved documents, logs and any content retained for training — and the answers differ by provider, by product tier and by region.

Why it matters

Whether your prompts can be used to train someone else's model is a contract term, not a technical property. Read it, and read what changes on the enterprise tier.

See also

EU AI Act

Also called Regulation (EU) 2024/1689

The European Union's law on AI. It came into force in August 2024 and arrives in stages. It sorts systems into four risk levels — prohibited, high-risk, limited and minimal — and sets duties to match, with separate rules for general-purpose models. Bans and AI-literacy duties started in February 2025. Rules for general-purpose models followed in August 2025. Most high-risk requirements land in August 2026.

Why it matters

It reaches companies outside the EU if their systems affect people inside it. The duties attach to what you use AI for, not to the technology you use. So the first step is narrow: work out which of your systems would count as high-risk.

See also

Explainability

Also called Interpretability, XAI

The ability to say why a system produced a particular output in terms a person can evaluate and, where required, contest. For modern models this is rarely a look inside the weights; it is the evidence, retrieved sources, inputs and rules surrounding the decision.

Why it matters

If a decision affects someone's credit, care, employment or eligibility, explanation is an obligation rather than a feature. Design the evidence trail in at the start — it cannot be added convincingly afterwards.

See also

GDPR

Also called General Data Protection Regulation, Regulation (EU) 2016/679

The European Union's General Data Protection Regulation, in force since May 2018. It governs how personal data is collected, used and stored. Article 22 gives people the right not to be subject to a purely automated decision that has legal or similar effects — the clause AI deployments hit first. Fines reach the greater of €20 million or 4% of worldwide annual turnover.

Why it matters

It is routinely confused with the EU AI Act, and they do different jobs. GDPR governs the personal data. The AI Act governs the system. Both can apply to one deployment, and meeting one says nothing about the other.

See also

ISO/IEC 42001

Also called AI management system

The international standard for managing AI, published in December 2023. It sets out how a company should build, run and keep improving an AI management system. Unlike a framework, an outside auditor can certify you against it.

Why it matters

In regulated sectors, buyers are starting to treat the certificate as a shortcut — the way ISO 27001 works for security. Worth knowing whether yours are about to start asking.

See also

Model Card

Also called System card

A short standard document describing a model or AI system: what it is for, what it was evaluated on, how it performed, where it should not be used, and what is known about its limitations. The system-level equivalent of a datasheet.

Why it matters

Writing one forces the questions a governance review will ask anyway, at the point when the answers are cheap to act on rather than expensive.

See also

Model Registry

Also called Model inventory

One list of every model and version you are running. For each one: what it is, which version is live and where, who owns it, what it was tested against, and what it is approved to do.

Why it matters

Every governance framework assumes you have one. Most companies do not. That is why the honest answer to "what AI are we running" is usually a spreadsheet someone builds over a fortnight. Without it you cannot scope an incident or plan a shutdown.

See also

NIST AI Risk Management FrameworkAI RMF

A voluntary framework from the US National Institute of Standards and Technology, published in January 2023. It is built on four functions: Govern, Map, Measure, Manage. Together they cover finding and handling AI risk across a system's whole life. A companion profile for generative AI followed in July 2024.

Why it matters

Voluntary, widely cited, and the thing most enterprise security questionnaires are quietly based on. Mapping to it early costs far less than retrofitting it mid-deal.

See also

Personally Identifiable InformationPII

Also called Personal data, Sensitive data

Data that identifies a person, on its own or combined with other data you hold. The term is American and narrower than it sounds. European law says "personal data", which covers anything relating to an identifiable person — IP addresses and device IDs included. In an AI system it turns up in places nobody listed: prompts, retrieved documents, chat logs and test sets.

Why it matters

Most AI data incidents are not a breach of the model. They are a breach of the logs around it. Decide before launch what you keep, for how long, and who can read it. Then check whether your provider trains on it — that is a contract term, not a technical property.

See also

Prompt Injection

Also called Indirect prompt injection

An attack in which instructions hidden in content the model reads — a web page, a document, an email, a calendar invite — are treated as commands. The model cannot reliably tell your instructions from text that merely looks like instructions, which makes this a design constraint rather than a bug to be patched.

Why it matters

Risk scales with what the system can do, not with what it can read. An agent that can send mail, move money or change records needs the untrusted input path and the privileged action path kept apart by construction.

See also

Shadow AI

AI tools used inside an organization without its knowledge or approval — personal accounts, unvetted browser extensions, features switched on inside software already deployed. It is usually a sign that the sanctioned path is slower than the work requires.

Why it matters

You cannot govern an inventory you do not have, and the data already left. The durable fix is a sanctioned path that is genuinely easier to use, backed by visibility into what is actually running.

See also

SOC 2

Also called SOC 2 Type II, Service Organization Control 2

An audit report on how a supplier handles customer data, written by an accounting firm. It scores five trust criteria: security, availability, processing integrity, confidentiality and privacy. Type I checks the controls on a single date. Type II tests that they worked over a period, usually three to twelve months. It is an attestation, not a certification, and the company being audited picks the scope.

Why it matters

It is the first thing an enterprise buyer asks an AI vendor for, and the easiest to wave through. Ask for the Type II report, not the badge. Check which systems are in scope. Then read the exceptions — a clean opinion on a narrow scope tells you very little.

See also

Sovereign AI

Also called AI sovereignty

Running AI on infrastructure you control, inside a border you choose, under laws you answer to — the chips, the data, the model weights and whoever operates them. Nations use the term about national capability; companies use it about not being a tenant. It is a spectrum, not a switch: most who claim it own some layers and rent the rest.

Why it matters

For a regulated business this is the real reason to look at self-hosting, and it outranks cost. Rent a model and the provider can change it, reprice it, or cut you off. Your data also crosses a border to be read. Ask which layers you actually control: data, weights, hardware, operations. A local model on someone else's cloud closes fewer gaps than it looks like.

See also

09 — Economics & Adoption

What AI costs, what it returns, and why the gap between the two is almost never a technology problem.

Adoption Rate

The share of intended users genuinely using a deployed AI system for real work, as against licences issued or accounts activated. Measured properly it is depth and frequency per role, not a headline percentage.

Why it matters

Licences bought is the number most often reported and the least informative. A tool at ten percent real adoption has not half-worked; it has produced a cost with no offsetting return.

See also

AI Maturity

How ready a company is to get value from AI — across its data, workflows, skills, governance and who owns it at the top. Maturity is always uneven. Most companies are further ahead in one function than the company-wide score suggests, and that gap is where the sequencing decision sits.

Why it matters

A maturity assessment earns its keep when it ends in a ranked, costed list of use cases. Done as a scorecard it produces a slide. Done properly it produces a plan.

See also

AI Tool Sprawl

Overlapping AI subscriptions, licences and built-in features piling up across a company, bought team by team with nobody keeping a central list. The cost is rarely any one licence. It is paying twice for the same thing, data split across tools, and no one able to say what is running.

Why it matters

Sprawl is a visibility problem before it is a spend problem. Companies that measure adoption keep finding the same capability paid for three times, with two of the three barely used.

See also

Build vs. Buy

The choice between licensing an AI product, building a system you own, or switching on capability already sitting in software you pay for. The honest framing is three options, not two. The deciding question is whether the workflow sets you apart or is the same everywhere.

Why it matters

Buy the commodity. Build the differentiator. And switch on what you already pay for before doing either. Most companies find the third option last.

See also

Change Management

The work of getting people to actually use a new system — redefining roles, retraining, resetting expectations about what good work looks like, and deciding explicitly where the freed-up time goes. For AI it is heavier than for conventional software, because the tool changes judgment rather than just tooling.

Why it matters

Time saved does not become value until somebody with authority says where it goes. Absent that sentence, the hours dissipate and the financial lens reads flat — correctly.

See also

Pilot Purgatory

Also called Proof-of-concept purgatory

The state in which AI pilots keep succeeding on their own terms and none reaches production. The usual causes are structural rather than technical: no named owner, no baseline to prove value against, unresolved data access, and no route through security and procurement.

Why it matters

The escape is decided before the pilot starts — by naming the production owner, the success threshold and the integration path up front. A pilot without those is a demonstration, and demonstrations do not graduate.

See also

Prompt Caching

Also called Context caching

Reusing the provider's processed form of a prompt prefix across requests, so a long system prompt, a policy document or a fixed set of examples is not re-processed every call. Cached input is billed at a substantial discount and returns faster.

Why it matters

For a high-volume feature with a long, stable preamble this is usually the largest single cost lever available, and it is a change to request structure rather than to the product. It rewards putting the unchanging content first — which is not how most prompts get written.

See also

Rate Limit

Also called Quota, Throughput limit

The ceiling a provider puts on how much you can send — requests per minute, tokens per minute, concurrent jobs — enforced per account or per key. Separate from price: you can be willing to pay and still be refused.

Why it matters

This is the constraint that turns a successful launch into an outage, because traffic arrives in bursts and limits apply to the peak rather than the average. Find out your ceiling and your provider's process for raising it before the launch, not during it.

See also

Straight-Through ProcessingSTP

The share of cases a system completes end to end with no human touching them. Borrowed from payments and back-office operations, where it has been the headline automation number for decades.

Why it matters

For an operations buyer this is the number, and it is far more honest than adoption or time saved: it is a ratio nobody can narrate their way around. Worth pairing with the exception rate, because a high STP achieved by pushing hard cases into a queue nobody works is not automation.

See also

Time to Value

How long from starting work to the point where a business number measurably moves — not first demo, not launch. It is the metric that most cleanly separates AI programs that compound from ones that consume budget.

Why it matters

Short cycles create evidence, and evidence is what funds the next one. A program structured around one large delivery has a single point of failure with a twelve-month fuse.

See also

Token Economics

Also called Cost per token, Inference cost

The unit cost model of a generative feature: tokens in plus tokens out, multiplied by price per token, multiplied by volume. Retrieved context, conversation history, reasoning steps and retries all count, which is why real cost per request is routinely several times the naive estimate.

Why it matters

This is calculable before you build. Model the cost of the tenth-thousandth request, not the first — that is the one that decides whether the feature survives success.

See also

Total Cost of OwnershipTCO

Everything an AI system costs over its life. Licences and inference. Building it in, and the data work behind it. Evaluation, monitoring and human review. Retraining, and the eventual move off it. Model spend is usually a minority of the total, and almost always the only line in the business case.

Why it matters

The costs left out are the recurring ones. That is why year-two economics disappoint so reliably. A build estimate with no run-rate attached is half a number.

See also

Vendor Lock-in

The cost of leaving. For AI systems it accrues in specific, identifiable places: prompts tuned to one model's quirks, evals written against its outputs, fine-tunes that do not transfer, embeddings in a proprietary vector space, and agent tooling bound to one provider's interfaces.

Why it matters

It is not an argument against committing to a provider — it is an argument for knowing the number. Keep the model swappable at the orchestration layer and keep your evals portable, and a price rise or a deprecation becomes a negotiation rather than a rebuild.

See also

10 — Frontier & Safety

The vocabulary of the research frontier, which arrives in board papers and vendor decks long before it arrives in a product. Defined here the same way as everything else — by what it changes for someone deciding where to spend.

Alignment

Also called AI alignment

Getting a system to pursue what its operator actually intends, rather than a proxy that merely correlates with it. A model optimized for the wrong objective can be entirely accurate and still wrong — it does the thing you measured instead of the thing you wanted.

Why it matters

This is not only a frontier concern. Any deployment with a scoring function has an alignment problem in miniature: the moment a team optimizes a number, that number stops measuring what it used to. Name what the metric is standing in for before you tune against it.

See also

Alignment Tax

Also called Safety tax, Over-refusal

The capability given up in exchange for safety and steerability. Training a model to refuse, hedge and stay on task also makes it more cautious on legitimate edge cases, and the cost shows up as over-refusal and blander output rather than as a visible switch.

Why it matters

It is the real trade-off underneath "the safer model is worse at my task". Both halves are adjustable, so treat it as a dial to set deliberately for your risk profile rather than a flaw to complain about — and measure refusals on your own work, since that is where the tax is actually levied.

See also

Artificial General IntelligenceAGI

A system that matches or exceeds human performance across essentially any cognitive task, rather than doing one thing well. There is no agreed definition and no agreed test, which is why credible people looking at the same systems disagree on how close it is — they are not measuring the same thing.

Why it matters

A contract, a roadmap or a valuation that turns on "AGI" turns on nothing enforceable, because no two parties mean the same thing by it. If it appears in something you are signing, get the claim restated as a capability on a named task at a measurable standard.

See also

Embodied AI

Also called Embodied intelligence

The research position that intelligence needs a body. What a system can learn is shaped by what it can sense and do, so a model that only reads and writes is missing something a model with hands and eyes has. The academic term, in use since the 1990s, for what industry started calling physical AI in 2024.

Why it matters

The two names are close enough to be treated as the same thing, and the register tells you who is talking. "Embodied AI" is how a paper puts it; "physical AI" is how a deck puts it. The useful split underneath is simpler: software that only produces information, versus software whose mistakes move something heavy.

See also

Emergent Capabilities

Also called Emergence

Abilities that show up in larger models and are missing from smaller ones, appearing all at once rather than improving bit by bit. The framing is contested. Schaeffer and colleagues showed in 2023 that some of it is an artifact of the scoring — measure the same runs with a continuous metric and the jump smooths out.

Why it matters

"It will emerge at the next model size" is a forecast dressed as a roadmap. Treat a capability as missing until you have measured it on your own task, whatever the scaling story around it.

See also

Humanoid

Also called Humanoid robot

A robot shaped roughly like a person — two legs, two arms, hands. The case for the shape is that the world is already built for it: our doors, stairs, tools and benches need no changing. The case against is that legs are hard, expensive and mostly unnecessary when a wheeled base would do.

Why it matters

The most funded and least proven corner of physical AI. Demos are strong and deployments are small, and the honest gap between them is the whole story. Two questions cut through a pitch: how many hours does it run unattended between human interventions, and what does it cost per hour against the job it replaces. Most answers are not ready.

See also

Jailbreak

Getting a model to break its own rules through what the user types. Role-play framing, made-up scenarios, coded instructions, or simply wearing it down. This is not prompt injection. There the hostile instruction hides inside content the model reads on the user's behalf.

Why it matters

The difference decides who you are defending against. A jailbreak is your user working around you. An injection is a third party working through your user. Different threat, different control. A guardrail built for one will not catch the other.

See also

Mechanistic Interpretability

Research into what a model is actually computing inside — which features it holds and which circuits produce a given behavior. It is not the same as explainability. Explainability accounts for a decision after the fact, using evidence from outside the model. This tries to read the mechanism itself.

Why it matters

You cannot buy it or apply it to a deployment yet. It matters because it is the most likely route to models that can genuinely be audited. The gap between it and today's explainability is the honest answer when someone asks whether you can prove why the system did that.

See also

Physical AI

Also called Physical artificial intelligence

AI that senses the world, decides what to do, and then does it — through cameras, sensors, motors and grippers rather than through text on a screen. The output is a change in the world, not a document. Jensen Huang of Nvidia put the phrase into wide use at GTC in 2024, and it now covers self-driving vehicles, warehouse robots, drones and humanoids.

Why it matters

It is mostly a new name for embodied AI, aimed at buyers rather than researchers, and its edges are not agreed. That is worth knowing before you price anything: a vendor saying "physical AI" may mean a learned policy that adapts, or a rebadged robot arm running a fixed script. The question that separates them is what happens when conditions change.

See also

Recursive Self-ImprovementRSI

Also called Self-improving AI, Intelligence explosion

A system improving its own ability to improve itself, so each gain makes the next one easier and progress compounds rather than plateaus. The term does two very different jobs. In the research discussion it names a hypothetical fast takeoff, where a model redesigns its own architecture with no human in the loop. In practice today it describes something far more ordinary: models being used to help write training code, generate training data, and grade the next model's outputs — narrow, supervised loops with people at every gate.

Why it matters

Conflating the two is how a roadmap conversation goes sideways. When a vendor or a board paper invokes it, ask which one they mean, and what a human is still required to approve. The mundane version is real and already shows up in release cadence; the dramatic version is a forecast, not a plan.

See also

Red Teaming

Deliberately attacking your own system to find how it fails before someone else does — adversarial prompts, edge-case inputs, attempts to extract data or bypass policy. Distinct from evaluation, which measures whether it works on the cases you expect.

Why it matters

Evals tell you the system is good. Red teaming tells you how it is bad, which is the half a customer, a journalist or a regulator will find. Budget for it separately, because it gets cut first when it sits inside a QA line item.

See also

Scaling Laws

Measured relationships showing that model loss falls in a predictable way as compute, data and parameters go up. Kaplan and colleagues set them out in 2020. Hoffmann and colleagues revised the balance in 2022, showing that most models of the day were badly undertrained for their size.

Why it matters

They drive the release cadence and the prices you are planning around, so they are worth following as a supplier dynamic. But they predict loss, not business value. Nothing in them says a lower-loss model moves your number.

See also

Superintelligence

A hypothetical system substantially beyond the best human performance across essentially all domains, including the ability to improve itself and to plan. It is a research and policy concept rather than a description of anything that exists.

Why it matters

Nothing you procure this year is this. It is worth knowing because it frames the safety debate your regulators and your enterprise customers are reading — not because it bears on a purchase decision.

See also

Training Compute

Also called FLOP, Compute budget

The total arithmetic used to train a model, counted in floating-point operations. It is the closest thing the field has to one number for how costly and how capable a model is. That is why regulators reach for it: under the EU AI Act, a general-purpose model is presumed to carry systemic risk once training compute passes 10^25 FLOP.

Why it matters

A threshold in law makes the compute a vendor used a compliance fact about what you are buying, not a technical footnote. Worth knowing which side of it your supplier sits on.

See also

World Model

Also called World models

An internal picture of how things work, good enough to predict what happens next. Push a glass and it falls. A system with one can try an action in its head before trying it for real. The idea is old — the psychologist Kenneth Craik described a "small-scale model" of the world in 1943 — and it now covers both hand-built physics simulators and models that learn to predict video.

Why it matters

It is the difference between a robot that has been shown a task and one that can work out a task it has not seen. It is also why a lot of robotics money goes into simulation rather than hardware. Note the term is escaping robotics: people now use it about text-only agents too, meaning roughly "does it understand the situation, or just the words".

See also

Type size

Knowing the words is not the same as knowing where to start.

This glossary grows from the questions we get asked in engagements. If a term you keep hearing isn't here, tell us and we'll add it — and if the real question is which of these belongs in your stack, that is a shorter conversation than it looks.