# Interview: Aria Networks CEO on Why Inference Is Reshaping the Network — retained record

> Retained news record from news:datacenterknowledge.

Canonical: https://highsignal.app/data/records/aedeb42399a00df1

[← sources](https://highsignal.app/data)


source: [news](https://highsignal.app/data/news) (news:datacenterknowledge)  published 2026-05-11 12:07 ingested 2026-05-12 08:37

[original source ↗](https://www.datacenterknowledge.com/infrastructure/aria-networks-ceo-inference-is-reshaping-data-center-networking)

## extract

Interview: Aria Networks CEO on Why Inference Is Reshaping the Network
Startup founder Mansour Karam argues token efficiency and distributed inference are turning networking into a central AI infrastructure battleground.
AI networking startup Aria Networks emerged from stealth in April with a $125 million funding round and a blunt argument about the future of AI infrastructure: the next battle won’t hinge solely on bigger GPU clusters, but on networks that squeeze more work out of them.
Founded by former Arista and Juniper executives, the Palo Alto, Calif.-based company says the rapid rise of distributed inference, reasoning models, and agentic AI systems is reshaping how AI clusters are designed and operated. Instead of treating the network as passive plumbing, Aria argues operators increasingly need adaptive networking systems that optimize traffic flows, reduce congestion, and improve Model FLOP Utilization (MFU) – a measure of how effectively GPUs are being used – alongside token efficiency and cost-per-token economics.
“Inference is the most complex, most interesting use case,” founder and CEO Mansour Karam told Data Center Knowledge in an interview following the company’s launch.
Aria’s platform combines Ethernet switching hardware, SONiC-based software, telemetry collected at microsecond resolution, and AI-driven operational tooling designed to dynamically tune AI networks in real time. Karam argues that as operators shift focus from raw cluster scale to token efficiency and cost-per-token, networks that can adapt to changing workloads will become critical.
The following transcript has been lightly edited for length and clarity.
Data Center Knowledge: You describe Aria as “the network that thinks.” What does that actually mean operationally inside a production AI cluster?
Karam: Basically, we have all the components you need to build the network, including the hardware. We build switches using the Broadcom Tomahawk 5 and Tomahawk 6 chipsets, which are the workhorses of these AI clusters. We use the SONiC switch operating system, which is now standard.
But the differentiated piece is really the cluster software.
What has blown away our customers is what we call "Deep Networking." The reason we call it Deep Networking is that it is very deep. It is multi-layered and starts with this foundation of telemetry across all components of your networking infrastructure – the switches, cables, optics, NICs, hosts – all the way up to the NCCL and RCCL layers.
We’re collecting telemetry at a resolution far beyond what you would get in other systems. Historically, telemetry systems sampled every 30 seconds. Maybe every second if you pushed it. Here, we’re collecting telemetry from the ASIC at microsecond resolution across thousands of parameters.
Then we process those signals, extract insights, and take action. The way you evaluate signals and react depends on where you are operating in the stack. At the ASIC layer, you have to react extremely quickly. At the cluster layer, you have more reasoning involved.
At the top layer, we use LLMs and an agentic approach where operators collaborate with the network itself.
DCK: Why did you decide to frame the company around token efficiency and MFU instead of traditional networking metrics like latency and throughput?
Karam: If you think of traditional networking, networking vendors historically sold to networking teams. Those teams cared about networking metrics – latency, throughput, packet performance.
But AI factory engineers don’t think that way. They’re thinking about how to optimize the AI factory as a whole. They care about token efficiency. They care about MFU. They care about cost per token. For them, the network is often viewed as plumbing. Our challenge was to explain that the network is actually one of the highest leverage components in the system because it touches everything else.
You can optimize your LLM router. You can optimize your scheduling. You can optimize your GPUs. But if the network isn’t performing well, all of those optimizations suffer.
The network is one component of this machine that has the highest leverage.
DCK: You spent a lot of time discussing inference. Why do you think the industry underestimated inference networking complexity?
Karam: A year and a half ago, very competent people were confidently saying inference was trivial from a networking standpoint. The assumption was that training created all the networking load, while inference just sent traffic to one node.
You look at it today, and inference is the most complex, most interesting use case.
With the advent of reasoning models and reinforcement learning, models have evolved dramatically. Distributed inference introduces completely different traffic patterns and operational requirements.
Now you’re dealing with many agents being fanned out for every query. Every one of those requests is traversing multiple networks. You have frontend networks, backend networks, storage systems, and KV cache transfers all interacting with each other.
The network is in the path of every one of those operations.
DCK: What actually breaks first as inference clusters scale?
Karam: It depends on the inference use case. Some inference workloads are simple – one query goes to one node and comes back. Others are much more distributed and memory-intensive.
In many of these distributed inference use cases, what we’re finding is that you need much more memory. The storage system becomes the bottleneck. Memory on the compute nodes becomes the bottleneck. That leads to architectures where you disaggregate the prefill phases and the decode phases and move KV caches between them.
At any point in this cycle, a different part of the system can become the bottleneck.
The one constant across all of this is the network. The network plays a critical role across every component in the system.
DCK: You mentioned “noisy neighbor” problems in inference clusters. What does that look like operationally?
Karam: Imagine you have thousands of users simultaneously interacting with a distributed inference cluster. One user might ask a simple question. Another user might ask a very complex question that branches into many agents and generates a huge amount of traffic. That complex workload can create congestion that affects everyone else. That’s the noisy neighbor problem.
The network has to manage all those traffic flows smoothly and efficiently so that one bursty workload doesn’t create catastrophic delays for another user.
You’ve probably experienced this yourself using reasoning models where the system suddenly stalls while generating a response. The model can’t produce the next token until all of the underlying operations are complete.
That’s fundamentally a distributed systems problem, and networking becomes central to solving it.
DCK: You’ve argued that networking directly impacts AI economics. How do operators think about that?
Karam: In inference, the cost per token becomes critical.
All of these AI factories and neoclouds are competing with each other. If I can serve a model at a lower token cost than others, then I have a structural advantage.
That gives me flexibility in pricing, flexibility in business models, and flexibility in how I deliver services to end users.
Many operators are now optimizing around token economics rather than simply maximizing cluster size.
DCK: You’ve compared Deep Networking to self-driving systems. Why?
Karam: Because older networking architectures were fundamentally rule-based. Ten years ago, self-driving software was rule-based, too. It only worked in highly constrained environments because engineers had to define all the rules manually.
The same thing existed in networking. You could build automation around predefined scenarios, but the moment the environment changed, the system struggled.
AI infrastructure is evolving too quickly for static rule-based networking architectures. Models change. Traffic patterns change. Cluster architectures change. Inference workloads change. You need systems that adapt dynamically.
That’s why we believe probabilistic systems and AI-driven optimization become critical.
DCK: How autonomous is the platform today?
Karam: We’re approaching this conservatively because networking has to be reliable above all else.
Job No. 1 is that the network stays up.
There are certain things we already automate completely. If a link goes down, traffic reroutes automatically.
But in more complicated scenarios – for example, if an optical transceiver is degrading – we keep operators deeply involved. The system proposes strategies, surfaces telemetry, collaborates with operators, and helps guide remediation.
Over time, the system will become increasingly autonomous.
But we believe trust develops gradually. Operators need visibility into how the system works and confidence that it’s making the right decisions.
DCK: How do operators maintain trust in a system making real-time networking decisions?
Karam: The operators have to feel deeply involved in the functioning of the system. They need visibility into the telemetry. They need to understand why the system is making decisions. They need the ability to inject their own expertise and operational preferences into the platform.
Over time, as they see the system perform effectively, trust develops naturally.
It’s similar to the evolution of self-driving software. Early on, people were skeptical. Today, many people trust autonomous driving systems more than human drivers in certain conditions.
I think the same evolution will happen in networking.
DCK: Has the industry already decided that Ethernet wins the AI fabric market?
Karam: For scale-out networking, I think Ethernet has clearly won.
InfiniBand still exists in certain use cases, but even Nvidia is now aggressively pushing Ethernet.
I’ve been in networking for 30 years, and I like to say: never bet against Ethernet. Ethernet is ubiquitous. There’s massive operational expertise around it. There’s an enormous ecosystem around it. There’s a huge economy of scale around it. At the end of the day, a lot of the differentiation between networking technologies comes down to latency and throughput, and Ethernet can deliver both.
My belief is that Ethernet will continue to eat the networking world in AI clusters just as it has everywhere else.
DCK: Three years from now, what metric will AI infrastructure operators care about most?
Karam: The industry is evolving so quickly that I’m cautious about making predictions. A lot of assumptions people made even a year ago turned out to be completely wrong.
What I am extremely confident about is that the network is only going to become more important. AI factories are becoming increasingly complex systems. They’re becoming more dynamic, more distributed, and less specialized. The network is the connective tissue across all of those systems.
Whatever happens with models, workloads, or architectures, you’re only going to need more networking.

## retained text

No document body was retained for this record — only its metadata was stored.