We have been building with AI agents for a while now. Claude Code, custom automation pipelines, multi-step workflows that do real work. This is not theoretical. We use these systems daily.
What follows are observations from that experience. Not benchmarks. Not vendor claims. What we have actually seen happen when agents run unsupervised.
Small errors compound
The most consistent pattern we observe is that small errors early in a workflow create larger problems downstream. An agent misunderstands a file structure, makes an incorrect assumption about a variable name, or misreads a requirement. Each subsequent step builds on that error.
By step five or six, the output can be confidently wrong in ways that are hard to trace back to the original mistake. The agent does not know it has gone off course. It continues with the same certainty it had at the start.
Confidence does not track accuracy
Agents sound equally confident whether they are correct or not. There is no hesitation, no hedging, no signal that something has gone wrong. This makes human oversight harder than it sounds.
When you review agent output, you are not looking for obvious errors. You are looking for plausible-sounding output that happens to be wrong. That requires active verification, not passive review.
Context degradation is real
Long-running agents lose track of earlier context. Instructions given at the start of a session may be forgotten or deprioritised by the end. The agent does not flag this. It continues operating as if it remembers everything.
We have seen agents contradict their own earlier output, repeat work they have already done, or forget constraints that were clearly stated. The longer the session, the more likely this becomes.
Tool use creates new failure modes
When agents can use tools, they can fail in ways that pure text generation cannot. File operations can go wrong. Commands can be misformed. Assumptions about system state can be incorrect.
The failure mode shifts from "incorrect text" to "incorrect action". An agent that writes bad code is one thing. An agent that executes bad code is another. The stakes change when agents can act on the world.
Recovery is harder than prevention
When an agent goes wrong mid-workflow, getting it back on track is difficult. You can correct it, but the correction often does not propagate cleanly. The agent may acknowledge the correction while continuing to make related errors.
Starting over is frequently faster than fixing. This is not ideal, but it is what we have observed.
What we do about it
Our approach is pragmatic:
Shorter workflows. Break complex tasks into smaller steps with human checkpoints. Do not let agents run unattended for long sequences.
Explicit verification. Build verification into the workflow itself. Do not assume the agent got it right. Check.
State management. Keep track of what the agent knows and what it has done. Do not rely on the agent to maintain this context reliably.
Graceful failure. Design workflows to fail safely. Make it easy to restart from a known good state.
The opportunity is real
None of this means agents are not useful. They are. We use them constantly. But the useful version requires understanding the failure modes and designing around them.
The agents that work well in our experience are the ones where humans remain in the loop at critical points, where verification is built in, and where the workflow is designed with failure in mind.
This is not a limitation to work around. It is just how the technology works right now. Build accordingly.