loading
loading
Autonomous agents can work for hours on complex tasks, accessing files, tools, the internet, and spawning other agents to collaborate with. Executing complex tasks over long time horizons leads to compounding risk: the agents may download code, build new tools, request additional privileges, and touch more of the underlying infrastructure. The agents’ action paths are determined as they work and reason through the steps needed to complete their goals. As they interact with the tools and systems around them, they adapt their reasoning and behavior to achieve their task. The adaptability of agents is what makes them so powerful, but they are goal-oriented and may not prioritize safety in pursuit of their objectives. In order for agents to work both autonomously and safely, we need to build environments that provide control and visibility in what the agent is doing. An agent cannot govern itself. Security and governance belong in the infrastructure the agent operates in. OpenShell provides deterministic governance outside of the agent’s reach NVIDIA OpenShell is an open-source runtime to build the security and governance needed for agentic systems into the computing infrastructure. OpenShell defines and enforces which systems and data an agent can access, from files and internet services to systems that interact with physical devices. It is the foundation of the NVIDIA OpenShell Reference Architecture for building safe AI agents. OpenShell treats agent workloads as untrusted throughout execution. An agent is given access to what it needs, when it needs it. OpenShell has four key components: - Gateway: The control plane that manages sandbox lifecycles and policies - Supervisor: Checks outbound requests against policy and mediates access to credentials. This runs completely separate to the sandbox - Sandbox: Isolates the agent workload and restricts filesystem and process access - Policy prover: Adds formal verification, checking whether a policy’s permissions stay within an organization-defined ceiling. As the agents evolve, these components provide the infrastructure controls that allow them to work safely. Extending and integrating OpenShell Extensibility is a core principle of OpenShell’s design. Supervisor middleware lets you inspect, block, or change the content that an agent sends and receives over the network as an additional enforcement layer. Gateway interceptors let operators add deployment-specific governance to OpenShell control-plane operations without modifying the gateway. Organizations also need to establish who an agent acts for, what authority it holds, and whether an action fits the task. Identity, authorization, data protection, and auditing each address part of that problem. OpenShell provides enforcement points where those systems can be integrated. In the reference architecture, OpenShell runs on NVIDIA Vera CPUs. NVIDIA Vera, the compute layer beneath the controls OpenShell defines what agents can do. Vera determines how fast they can do it. Agents operate through loops of dependent steps where each step must complete before the next begins: reasoning, calling tools, executing code, querying data. AI factories run thousands of these loops concurrently, placing new demands on CPU architecture across five dimensions: - They need massive core counts to run many concurrent workflows, and fast individual cores to shorten each sequential step. - They need high-bandwidth fabric and memory to keep those cores fed with the live data agents depend on. - They need massive I/O to move data at speed between CPUs, GPUs, storage, and network. - They need hardware-enforced security, because agents increasingly operate on sensitive enterprise data and must be isolated from each other and the host. - They need to deliver all of this within the power envelope of a modern data center (more agent work per watt, not just more raw performance). NVIDIA calls this combination the max single-threaded CPU at scale, and NVIDIA Vera is built for it: massive Olympus cores, a second-generation Scalable Coherency Fabric, LPDDR5X memory, Gen 6 I/O, confidential computing, and an energy-efficient design purpose-built for power-limited data centers. Three architectural properties make Vera fast for agent workloads - First, 88 NVIDIA-designed Olympus cores with a 10-wide front end, neural branch predictor, and deep out-of-order execution deliver up to 2x performance on critical agentic workloads versus x86, shortening each tool call, code execution, and evaluation. - Second, Vera’s second-generation Scalable Coherency Fabric provides up to 3x core-to-core bandwidth and up to 50% lower core-to-core latency than x86 alternatives, keeping data moving predictably under load. Vera also integrates 96 lanes of PCIe Gen 6, delivering the I/O bandwidth to connect GPUs, high-speed storage, and network at the throughput agentic infrastructure demands. - Third, up to 1.2 TB/s of LPDDR5X memory bandwidth, with up to 40% lower loaded memory latency than chiplet-based designs, keeps agents supplied with the live context they need without stalling on memory access. LPDDR5X also makes Vera dramatically more efficient: Vera delivers 2x more performance per watt than x86 CPUs, while the LPDDR5X memory subsystem itself draws an order of magnitude less power than competing DDR5-based memory configurations, enabling more agent work within the same power envelope. Agent workloads depend on sandboxes: isolated execution environments where agents run code, invoke tools, and complete tasks without interfering with each other or the host system. The denser and faster those sandboxes spin up, the more agent work a given infrastructure footprint can sustain. CoreWeave, the AI cloud platform purpose-built for training and serving models at scale, found sandbox startup to be more than 3x faster on Vera than on x86, with a single rack supporting more than 11,000 concurrent agent environments. Running Terminal Bench, which measures CPU performance across real agentic loop tasks, CoreWeave found Vera delivered 1.8x faster performance versus x86, across agentic tasks ranging from scripting to database recovery. That density and performance under load is what agent infrastructure actually requires. NVIDIA BlueField and NVIDIA Sentry for hardware-enforced protection OpenShell places authority outside the agent. For deployments that require additional protection, that separation can also extend into purpose-built silicon. NVIDIA BlueField-4 DPUs provide a hardware-isolated trust domain outside the agent’s reach. In the OpenShell reference architecture, BlueField-4 sits in the I/O path between agents running on Vera and the models they access. NVIDIA Sentry, a security software framework built on NVIDIA DOCA, runs in this independent domain to continuously monitor agent behavior and enforce runtime policy. NVIDIA Sentry observes requests and responses, networking and file activities, verifies identity and delegated authority, and analyzes selected host memory to detect deviations from authorized behavior without relying on agent self-reporting. When it detects a threat or policy violation, BlueField-4 enforces policy in silicon, allowing NVIDIA Sentry to quarantine the agent within milliseconds. Together, they add an optional hardware safeguard alongside OpenShell’s software boundaries. Learn more about NVIDIA Sentry in this technical deep-dive. Bringing secure agent runtimes to scale Running agents in production requires infrastructure that can keep thousands of workflows moving while governing every action they take. Vera provides the compute, memory bandwidth, and I/O to sustain those workflows. OpenShell supplies the runtime boundaries that determine which actions execute and which resources agents can reach. Together, Vera and OpenShell make secure agent execution a foundation for scale. Agents can run longer, use tools, and work with enterprise data within policies enforced outside their control. For deployments that require additional protection, NVIDIA Sentry on BlueField-4 adds independent monitoring and in-silicon enforcement. The NVIDIA Vera Rubin platform, where LLMs run, together with OpenShell and NVIDIA Sentry on BlueField-4, are the pillars of the OpenShell reference architecture. They form the complete stack for safe AI agents: a complete, hardware-rooted foundation that gives organizations the performance to make agents productive and the controls to keep their work within authorized boundaries. Get started To implement secure agent governance in your organization, begin with NVIDIA OpenShell, follow the documentation, and explore how NVIDIA Vera CPUs accelerate agent workloads.
document fetched 2026-10-28 11:34 · canonical https://developer.nvidia.com/blog/how-nvidia-openshell-reference-architecture-secures-and-governs-ai-agent-workloads-at-scale