Carousel
Beyond the LLM: Agent Harness Engineering and AI governance
The biggest challenge was never the model. It was the agent harness.
Over the past year I took part in developing AI agents for industrial environments. And the experience left me with a conclusion I did not expect.
The beginning
The initial idea seemed simple. Build an assistant connected in real time to:
- Plant signals (OT)
- Operational reports and incidents (IT)
- Technical documentation and multiple data sources
At first, with few processes and a bounded set of documents, everything worked incredibly well. The agent answered well and the team was enthusiastic.
Then came the inevitable question: what if we scale?
The twist
We went from dozens to thousands of signals. From a few PDFs to entire repositories. We added RAG, multi-agent systems and MCP.
And that is where reality hit us: the more it grew, the harder consistency became.
What answered well one day failed the next. Instructions "got lost". Behavior became unpredictable.
Our first intuition? Blame the model. We thought the prompt was wrong. We thought the tools were failing. We thought we needed a more advanced LLM.
We were looking in the wrong direction.
The diagnosis
The problem was not the model. It was the design of the whole system around it.
What academia now studies under the name Agent Harness Engineering describes exactly what we lived through. Elements that end up being as important as, or more important than, the LLM itself:
- Architecture: memory, context management, orchestration and observability.
- Governance: permissions, business rules and real understanding of the process.
An agent is not traditional software
Add to this the clash of expectations. Many leaders expect an agent to behave deterministically, like traditional software. But agents do not work that way. They carry uncertainty and variability.
Those of us who come from the world of probabilistic models, and I have worked with machine learning for more than 15 years, recognize this behavior immediately: it is not a failure, it is the nature of the system.
The lesson
Can an organization scale agents without an AI strategy and governance model? My experience says no.
The conversation always starts with the models, but it should start with the architecture, the operating model and the process. Successful agents are not just a collection of prompts; they are complex systems. Without that context, even the best model will reach wrong conclusions.
Most problems do not come from the AI. They come from how we design the system around it.
Is your organization already thinking about the harness, or still looking for the perfect model?
To go deeper
Anthropic, Effective Context Engineering for AI Agents (2025) · Chroma, Context Rot: How Input Tokens Impact LLM Performance · Agent Harness for LLM Agents: A Survey (2026).
More in this topic