Utilities May Get an AI Boom the Grid Wasn’t Built For

extract

Utilities May Get an AI Boom the Grid Wasn’t Built For Utilities are planning around giant AI training campuses, but distributed inference workloads could alter where and how future power demand emerges across the grid. For the past two years, the AI infrastructure boom has revolved around a simple premise: bigger clusters win. Hyperscalers, utilities, developers, and power providers raced to secure gigawatts of electricity for massive AI campuses built around frontier model training, where tens of thousands of GPUs operate inside tightly synchronized clusters. Utilities in multiple regions revised their demand forecasts upward amid AI-driven data center growth. Grid planners modeled steep increases in projected load. Developers assembled giant land banks across Texas, Virginia, Ohio, Georgia, and the Carolinas. Training drove the first wave of grid planning around giant, synchronized campuses; inference now threatens to reshape demand geographically while refocusing policy toward interconnection rules, flexible/interruptible tariffs, and demand response. Now some utilities are discovering how unstable those assumptions may be. In a February filing with Ohio regulators, AEP Ohio said a speculative data center queue initially exceeding 30,000 MW fell to 5,642 MW of signed commitments once customers were required to provide binding financial and legal obligations under the utility’s new data center tariff (DCT) structure. “The primary purpose of the DCT proceeding was … to flush out the speculative and uncommitted data center load,” AEP Ohio wrote in the filing. The utility said the remaining committed load will now be used for PJM Interconnection transmission planning purposes. That attrition underscores a deeper question utilities now face: which AI workloads create durable, transmission-scale load, and which do not. Training vs. Inference: Two Load Shapes, Two Playbooks Much of that uncertainty centers on inference. While frontier AI training depends on tightly coupled GPU clusters linked through ultra-fast fabrics – where latency and synchronization overhead directly affect performance – some inference workloads can operate closer to population centers or regional power markets without those same synchronization requirements. Former DOE Loan Programs Office director Jigar Shah told Data Center Knowledge that utilities are planning around the wrong category of AI load. “The training versus inference distinction doesn’t just affect where load lands – it affects which policy levers actually matter,” Shah said. Training campuses often depend on long-term power procurement and behind-the-meter generation strategies tied to gigawatt-scale deployment. Distributed inference infrastructure, Shah argued, may depend more heavily on flexible interconnection rules, interruptible tariffs, and demand response programs that allow smaller facilities to operate across broader portions of the grid. Epoch AI research further complicates the picture. The organization recently estimated Amazon, Google, Meta, Microsoft, and Oracle collectively controlled roughly 71% of global AI compute capacity by late 2025, up from 63% in early 2024. That creates a growing contradiction inside the industry. Ownership of AI compute infrastructure is becoming increasingly centralized, even as some inference workloads may become more geographically fragmented over time. Nvidia CEO Jensen Huang has increasingly framed inference as the next major phase of AI infrastructure growth rather than a lighter-weight successor to training demand. During Nvidia’s February earnings call, Huang argued that “compute equals revenues” as companies scale AI services. At GTC earlier this year, Huang declared that “the inference inflection has arrived,” while describing a future where agentic AI systems dramatically increase token generation and sustained compute demand. Inference may prove more geographically flexible than frontier training clusters, but that does not necessarily mean infrastructure demand shrinks. It may instead change the physical shape of where the load emerges across the grid. What Utilities Are Seeing: Early Signals in Interconnection Queues EPRI Principal Technical Executive Tom Wilson told Data Center Knowledge that the vast majority of current data center connection requests remain far smaller than the multi-gigawatt projects dominating headlines. “Current data center connection requests average 100 to 200 MW, rather than the 1 to 5 GW requests that make headlines,” Wilson said. He added that customer-facing inference facilities clustered near population centers could still create concentrated local load growth even at a smaller scale. Over time, he added, efficiency improvements in AI hardware could allow much smaller data centers to handle critical inference workloads closer to end users. EPRI, Nvidia, InfraPartners, and Prologis are now collaborating on a distributed compute demonstration project designed around networks of 5 MW to 20 MW AI data centers sited near available grid capacity. Wilson said developers of smaller AI facilities increasingly seek distribution-level connections either permanently or as an interim path toward transmission access, potentially shifting some infrastructure pressure away from bulk transmission systems and toward local distribution networks. Utilities are also trying to manage the forecasting risks created by volatile AI demand assumptions. A recent E3 whitepaper cautioned that utilities are making large and long-term infrastructure investment decisions even as AI demand patterns, utilization rates, and deployment models remain highly uncertain. The report highlighted multi-gigawatt swings in utility planning assumptions tied to AI infrastructure forecasts and warned that overestimating demand could create stranded costs while underestimating demand could threaten reliability. A recent Duke University study also found roughly 100 GW of stranded grid capacity could potentially become available if data centers curtailed just 0.5% of annual load during peak demand periods. EPRI’s DCFlex initiative is piloting flexible-load concepts through grid-interactive data center demonstrations across the US and Europe to evaluate how AI workloads could help utilities use existing infrastructure more efficiently. Getting the Load Shape Right: Planning Amid Uncertainty For utilities, the stakes extend far beyond data centers. Transmission corridors, gas plants, substations, water systems, tax incentives, and land acquisitions are all being planned around assumptions about how AI workloads will physically organize themselves over the next decade. The first wave of AI planning assumptions centered on giant synchronized training campuses. The next phase may depend on whether inference behaves more like a hyperscale industrial load, a flexible cloud service, or something in between.