Anthropic’s Managed Agents with Memory Are Reshaping AI Workloads

extract

Anthropic’s Managed Agents with Memory Are Reshaping AI Workloads Persistent memory shifts AI performance toward storage, networking, and data movement, not just GPU throughput. Anthropic is positioning managed agents with memory as a developer-facing infrastructure layer for building and running AI agents, shifting workloads from isolated inference calls to coordinated, stateful workflows. In an April 23 blog post, the company describes agents that retain context across interactions, call tools, and execute multi-step tasks with persistent state. That model changes how AI workloads are structured inside the data center. Instead of discrete inference requests, agents operate as coordinated workflows that retrieve, update, and reuse information across steps – making data movement and system coordination part of execution, not just an output of the model. Memory Turns Inference into a Multi-Step Workflow In the blog post, Anthropic said agents carry context forward, including past prompts, tool outputs, and structured data, and reuse that information over time. Memory allows agents to store information about users, tasks, and prior results, then access it in subsequent steps or sessions. Each request can involve retrieving stored context, incorporating it into model prompts, and updating that state based on new outputs. The result is a persistent layer of data alongside model execution, where state is read and written as part of the workflow. In this paradigm, inference extends beyond a single prompt-response cycle. Tasks can unfold across multiple steps, with context maintained and refined as the agent progresses. Why Data Movement, Not GPU, Becomes the Bottleneck Anthropic’s description of managed agents highlights multi-step execution in which agents call tools, process intermediate results, and continue iterating. Those stages require moving data among models, memory systems, and execution environments. These workloads “turn a single inference into a multi-step, distributed workflow [where] each user request now triggers repeated internal hops between models, memory stores, tool sandboxes, and schedulers," said Sameh Boujelbene, vice president at Dell’Oro Group. She added that traffic grows through “orchestration loops, frequent state reads and writes, and service-to-service calls, not heavier GPU compute." That pattern reflects how agent workflows execute in practice. Each step introduces additional data exchanges – retrieving stored context, passing results between components, or invoking tools – making those interactions central to how the workload runs. Boujelbene also noted that these systems are more sensitive to latency because requests depend on multiple sequential and parallel steps. “They become materially more sensitive often by an order of magnitude,” she said. Networking's Bigger Role in the Data Center Stack Anthropic frames managed agents as a way to handle long-running, multi-step tasks with persistent context and tool use – an approach that puts orchestration and system design at the forefront. From a network standpoint, Boujelbene said the shift requires “low tail-latency lossless fabrics” and a clearer split between tightly coupled compute domains and broader system connectivity. “Scale up powers the brain; scale out connects the brain to memory, tools, and state,” she said. She also expects networking to account for a larger share of data center spending over time, estimating a roughly 10 percentage-point gain in overall data center IT spend by the end of the decade. Taken together, managed agents with memory introduce a workload pattern built around coordination, persistence, and multi-step execution – factors that increasingly shape how infrastructure supports AI alongside the models themselves.