Nvidia releases open-source software to fence in AI agents

Nvidia on Monday released open-source software meant to keep AI agents inside the lines, and laid out a hardware design that it says can catch an agent trying to break out and shut it down.

The package, which Nvidia calls its Open Agent Safety Platform, has two parts. The first, OpenShell, sits between an agent and the computer it runs on. Nvidia says it records everything the agent does and enforces rules about what it may touch, whichever company’s model is doing the work. Nvidia also says it runs with little slowdown on its new Vera processor and that, as open-source software, it can be extended to work with Arm and Intel chips. Nvidia says OpenShell is available now through GitHub and its developer site.

The second part, Sentry, is a watchdog that Nvidia says runs on its BlueField-4 networking chips, separate from the machine the agent runs on. Nvidia says it watches agent traffic continuously and, if an agent tries to move outside its software boundary, quarantines and stops it “in milliseconds.” Sentry is part of what Nvidia calls a reference system design, a blueprint rather than a product. Nvidia’s release gives no ship date, price or list of systems you can buy with it.

What it does: puts a lock outside the agent itself. Nvidia says companies need an enforceable boundary outside the model and the software wrapped around it, because agents in recent incidents got around controls at the application layer to finish their tasks. In Nvidia’s telling, OpenShell sets the limits in software and Sentry enforces them from separate hardware.

What it can’t do, so far: Nvidia published no independent tests of its speed or coverage claims. The hardware half needs Nvidia gear. And a boundary only helps if someone draws it in the right place; the rules are still up to each company.

Who it’s for: businesses running agents for long stretches on real systems. Nvidia names banks, software companies and power companies among the organizations working with it. According to the release, Anthropic’s Claude Managed Agents now integrate with OpenShell and BlueField so companies can tightly control what agents can reach. “Companies are giving AI agents more of their most important work, and they need to direct and verify what those agents do,” Anthropic chief commercial officer Paul Smith said in the release. Nvidia says Salesforce has integrated OpenShell with Slack, where teams can watch an agent’s activity and approve or reject its requests for more access, and that SAP is embedding OpenShell in its Joule Studio runtime. Microsoft and Hugging Face are among more than 100 organizations Nvidia lists as working with the technology.

The announcement comes on the same day as other news about agent safety. As we reported this morning, OpenAI has paused training its latest models after its agents went beyond their tasks on federal websites, AP reported. Nvidia’s release doesn’t mention OpenAI or connect the two. It refers only to unnamed “recent security incidents” and says that across them, “the pattern is the same — the agent circumvented security controls at the application layer to complete its assigned task.”

“Safety and security require full-stack engineering,” Nvidia CEO Jensen Huang said in the release.

— Rowan Keel