The wrong starting point
Many AI projects start from a technology: a demo that impressed the leadership team, a vendor's roadmap, or a budget line labelled generative AI. Then someone goes looking for a problem to fit it. That is how a demand forecast ends up with a chat interface it did not need, and how a document review problem gets a hand-built classifier when a retrieval system would have answered the question directly.
Our position: frame the decision first and choose the technique last. The output the work has to produce, and the person who acts on it, tell you most of what you need to know. If neither technique is the answer, which happens more often than vendors suggest, a rule, a report or a better process may be.
The rule of thumb
If the answer is a number, a score, a ranking or a plan, it is usually machine learning. If the work is reading, writing or conversing over documents and knowledge, it is usually generative AI. Many systems use both, and should be scoped together when they do.
| The question | What it produces | Usually |
|---|---|---|
| Which customers are about to leave? | A score per customer | Machine learning: predict |
| How much will we need, and when? | A forecast with a range | Machine learning: forecast |
| Which of these transactions is not normal? | A flag and a reason | Machine learning: detect |
| What is the best plan within our limits? | A schedule, route or allocation | Optimization |
| What should we offer this person next? | A ranked list | Machine learning: recommend |
| What does this contract say about termination? | An answer with its source | Generative AI with retrieval |
| Draft a reply to this support ticket | Text for a person to review | Generative AI |
| Pull the key fields out of these documents | Structured data | Generative AI, then rules or machine learning |
The table is a starting point, not a verdict. The real test is what someone does with the output, and what a wrong output costs them.
Why the difference matters after the build
The two families do not only differ in how they are built. They differ in how you prove they work, what they cost to run and how you keep them under control. Those differences should shape the scope, the budget and the risk review.
Proving it works
A predictive model can usually be tested against history. If the outcome has been recorded reliably, you can backtest on past periods the model never saw and measure the result against the decision it supports. If the outcome has not been recorded, that is the first problem to solve, before any modelling.
Generative output is open-ended. There is rarely a single right answer to compare against, so you need an evaluation harness: a set of real test cases, agreed criteria for a good answer, and review by people who know the work. Plausible but wrong output is a known risk. NIST's Generative AI Profile (NIST AI 600-1), published in July 2024 as a companion to the AI Risk Management Framework, lists confabulation among the risks that are unique to generative AI or made worse by it.
Paying for it
The cost of a scoring model sits mostly in building it and keeping it healthy; each prediction is usually a small part of the bill. A generative system also costs something on every call, and that cost grows with volume and with how much text it reads and writes. Measure the cost per decision at realistic volume before you scale, not after.
Keeping it in bounds
A predictive model is controlled mainly through its threshold, which is set with the people who act on the score, and through monitoring once it is live. A generative system also needs limits on what it may say, what data it may see and what it may do. Once it can take actions through other systems, it needs credentials and permissions of its own. We cover that in our article on agents, credentials and least privilege.
When the answer is both
Some of the most useful systems combine the two, each doing the part it is good at.
- Read, then score. Generative AI turns unstructured documents, such as claims, contracts or notes, into structured fields. A predictive model then scores them.
- Score, then explain. A model ranks cases or produces a forecast. Generative AI drafts the explanation or the next message for a person to review and send.
- Ask the forecast. A conversational layer lets planners query a forecast in plain language, while the numbers still come from a model that was validated on its own.
When a system combines both, scope and evaluate them together. Each part needs its own test, and the whole needs a test against the decision.
Three common mismatches
- A language model asked to predict from tables. Asking a general-purpose language model to forecast demand or score risk from structured data usually gives you an answer that is hard to validate, costly to run and difficult to explain. A purpose-built model, trained and backtested on your history, is the better tool.
- A classifier asked to answer questions. Building a custom model to route or tag documents, when the actual need is to find and quote the relevant passage, is a lot of labelling for less value. Retrieval with a generative model fits the job.
- Either one, where neither is needed. If a well-defined rule, a cleaner report or a fixed process step solves the problem, build that. It is cheaper to run and easier to trust.
Questions to answer before you choose
- What exactly does the output look like, and who acts on it?
- Has the outcome you care about been recorded reliably, and for long enough to learn from?
- How will you know an answer is right, and who will check?
- What does a wrong answer cost, and how would it be caught?
- What is the cost per decision at the volume you expect?
- Does the system need to take actions, or only to inform a person?
- Would a rule, a report or a better process do the job?
If you can answer these, the choice of technique is usually clear. If you cannot, the project is not ready to build, whichever technique you choose. Once it is running, it also needs an owner and a routine; see MLOps in practice.
