As artificial intelligence agents enter production systems, the difficulty of diagnosing failures grows sharply. Unlike traditional software, an agent can continue running while pursuing the wrong path, making mistakes that compound through subsequent steps without triggering recognizable error signals.

Nvidia VP of Product Adel el Hallak recently outlined the visibility challenges developers face as agents take on increasingly complex tasks. The company is also participating in an industry initiative called the Secure Agent Findings Exchange, or SAFE, which brings together roughly 140 organizations to build shared infrastructure for reporting agent failures—a model borrowed from how the software industry handles vulnerability disclosure.

When we find these vulnerabilities, it's not just for one company. It's for everyone to patch across.

Adel el Hallak, Nvidia VP of Product

Why traditional observability falls short

Conventional software debugging typically starts with a clear signal—an exception, a failed request, or a service outage. Agents operate differently. They can continue executing while moving in the wrong direction, carrying forward errors from earlier steps without producing anything resembling a standard software failure. An agent might simply decide to "get creative" when it should not.

Even the highest-performing coding agents fail more than 60% of the time on tasks from real codebases. Detecting failure is straightforward; understanding its cause is not.

It's not enough to just look at the logs or the inputs and the outputs. It is important to figure out how it got to the answer. What were the reasoning traces? What tools did it utilize? Where did it get stuck? Where did it decide to try a new approach?

Adel el Hallak, Nvidia VP of Product

Debugging may require replaying the agent's execution path to identify where it diverged. What appears to be a model failure might originate elsewhere in the system stack. This distinction matters: agent bugs are not always model bugs.

Runtime as collection point

Nvidia positions the runtime layer as the natural location for capturing execution visibility. The company's OpenShell agent runtime, which operates beneath the NemoClaw platform, handles sandboxing and policy enforcement while exposing how an agent executes its work.

El Hallak identified OpenShell as essential across Nvidia's reference architectures. "You can change whatever harness you need. I'm even open to using whatever models you need," he said. "But the governance, the secure and open runtime that we want to leverage at all times is OpenShell."

Nvidia divides the agent stack into three layers: the model supplies intelligence, the harness orchestrates operations, and the runtime enforces governance. When an agent fails, the model may not be responsible.

Nvidia's NOAH research demonstrated that adjusting the harness while keeping the model constant can boost agent performance—meaning a poorly fitted harness can undermine an otherwise strong model. "Every model's different. Some could be more chatty than others," el Hallak noted. "Making sure those two things are either co-developed together or have profiles that are specific to models is a new unlock."

Safety as systems engineering

Nvidia CEO Jensen Huang has framed AI safety as an engineering discipline, a perspective el Hallak compared to conventional software quality assurance. "If there's a bug in your software, you don't release it," he said. "You work until it's fixed and it passes all your tests."

Agents complicate this model because reproducing failures often requires reconstructing system-wide behavior. That instrumentation carries costs: OpenAI found that monitoring adds roughly 20% to inference compute for its most advanced persistent agents.

Nvidia's strategy combines governed harnesses, sandboxed runtimes and confidential computing to protect models and user data. "There are ways where you make guarantees all the way down to the silicon," el Hallak stated.

SAFE extends this engineering discipline beyond individual organizations by establishing infrastructure for sharing what companies discover when agents fail.

Agents debugging other agents

CrowdStrike is fine-tuning Nvidia's Nemotron models using years of security data to build paired agents—one designed to discover exploits, the other to apply patches. When either agent malfunctions, the output alone may not reveal the source. A faulty patch could stem from the model, the execution path, or the tools the agent selected.

"I don't need general purpose for a given task. I need specialization," el Hallak said. As organizations develop agents for specialized workflows, failures may not surface in general-purpose model benchmarks or safety evaluations, placing greater responsibility on teams to understand what occurred during execution.

Toward shared failure reporting

Platform teams face two distinct challenges: locating the failure and reconstructing enough of the agent's execution to determine its cause. SAFE aims to make those discoveries actionable across organizations. Traditional software has established mechanisms for sharing vulnerabilities and fixes; no equivalent system yet exists for agent failures. The objective is to prevent every team from independently encountering the same problem.