Start with the simplest AI system that works

Every AI pitch you hear this year will mention agents. Some problems in a plant need one. Most don’t. The fastest way to get value from AI, and to avoid paying for complexity you don’t need, is to use the simplest system that does the job and prove it works before you scale it.

The ladder

Think of AI systems as a ladder with four rungs. Each rung up can do more, and each one also adds delay, cost, new ways to fail, and more testing.

1. One model call, with your information attached. You ask a question, and the model answers using documents or data you hand it. A maintenance tech asks why a pump keeps tripping, and the answer comes from the equipment manual and the last two years of work orders.

2. A workflow. Code runs a fixed set of steps, calling the model where it helps. Every morning it pulls yesterday’s downtime events, groups them by cause, and sends a one-page summary to the shift leads. The steps never change; only the data does.

3. An agent. The model decides its own next step, in a loop, until the job is done. This fits a problem where you can’t write the steps down in advance, such as working through an unusual quality problem where the next thing to check depends on what the last check showed.

4. Several agents. One agent directs others, each with its own specialty. It is powerful, and it is the hardest to test and the most expensive to run. In a plant it is rarely justified.

A simple test tells you which rung you need: if you can write the steps down, it’s a workflow. The most effective AI system I’ve built was a workflow, not an agent: a conveyor vision system that classified and measured material and handed the result to the plant’s controls, the same steps every time. It ran unattended and anchored the autonomous-operations program I led, which cut operator interactions on those systems by about 96%. Only when the steps genuinely depend on what the system finds along the way do you need an agent.

A demo is not a working system

Almost any of these can be made to look good in a demo. Before one goes into service at your plant, it needs five things a demo usually skips:

• A test set. A fixed list of real questions or cases with known right answers, run again after every change, so you know an improvement didn’t break something else.
• Permissions first. Data is filtered by what the person asking is allowed to see before it ever reaches the model.
• Limits. Caps on how many steps, retries, and how much text the system can use, so a confused system stops instead of running up a bill.
• Cost and speed controls. You know what each answer costs and how long it takes, and you decide what’s acceptable.
• Tracing. A record of each request through every step, so when an answer is wrong you can see where it went wrong.

On the production floor there is one more rule: AI advises, and it can reject a part, but it is never placed in a safety loop.

When it gets something wrong

It will. When it does, find the layer that failed before you fix anything. Was the instruction unclear? Did it pull the wrong documents? Or is the model simply not strong enough for the task? A fix aimed at the wrong layer does nothing. Change one thing at a time, and run the same test set after each change, so you know which change helped.

What this means for your first project

Start low on the ladder, on one problem with a clear payback. Agree on the baseline and the success criteria before anything is built. Then measure it against those numbers. That list of five things above is how we decide a pilot is ready to hand over.

Not sure which problem to start with? That is what an AI Assessment is for: a ranked list of opportunities in your operation, each with a dollar figure, and a recommendation for the first one to build.