← sources

TPU IR snapshot

source: ir (ir:TPU)published 2026-06-07 08:44ingested 2026-06-07 08:49

extract

Build, optimize, and scale training, inference, and reinforcement learning workloads to power autonomous reasoning agents Overview TPUs are custom-designed accelerators purpose-built for AI workloads such as agents, code generation, large language models, media content generation, synthetic speech, vision services, recommendation engines, and personalization models, among others. TPUs power Gemini, and all of Google AI powered applications like Search, Photos, and Maps, all serving over 1 Billion users. The shift to Agentic AI requires infrastructure capable of multi-step reasoning and continuous reinforcement learning. TPUs break the inference "memory wall" by hosting massive KV caches entirely on-silicon, utilizing expanded on-chip SRAM with TPU 8i. Combined with our SparseCore engine to offload communication tasks, this architecture reduces core idle time. The result is low-latency, predictable performance that powers complex reasoning loops. Speed up your deployment time by reducing training timelines for frontier models. Cloud TPUs maximize [goodput](https://cloud.google.com/blog/products/ai-machine-learning/goodput-metric-as-measure-of-ml-productivity), ensuring that nearly every compute cycle is spent on active learning. This is supported by an high-speed Inter-Chip Interconnect, Optical Circuit Switching, and Virgo Network, so accelerators operate as a highly reliable, unified system. TPUs are engineered to improve value and power consumption by focusing on the computational demands of AI, eliminating the operational overhead found in multi-purpose architectures. Integrated power management dynamically adjusts to real-time request volume, delivering high [performance-per-watt](https://cloud.google.com/blog/topics/systems/ironwood-tpus-deliver-37x-carbon-efficiency-gains) and supports complex AI workloads sustainably. Build on an open ecosystem using familiar libraries and tools. Cloud TPUs provide native, high-performance support for PyTorch and JAX, and support the vLLM engine for fast inference. Manage and scale these deployments reliably across global clusters with Google Kubernetes Engine (GKE). Cloud TPU versions | Cloud TPU version | Description | Availability | |---|---|---| TPU 8i | TPU 8i is optimized for post-training and inference while provides an 80% performance-per-dollar improvement over previous generations for low-latency inference for large MoE models. | Coming soon | TPU 8t | TPU 8t is built for large-scale pre-training and embedding-heavy workloads at a scale of 9,600 chips in a single superpod, provides to 2.7x performance-per-dollar improvement over Ironwood for large-scale training. | Coming soon | Ironwood | 7th-generation energy-efficient TPU engineered for large-scale training, reasoning, and inference. Features 9,216 liquid-cooled chips per pod, provides 42.5 ExaFlops and 4X better performance per chip over Trillium. | Ironwood is generally available in North America (Central) and Europe (West region) | Trillium | Sixth-generation TPU featuring improved energy efficiency and peak compute performance for training and inference. Operates with 67% more energy efficiency and provides 4.7x higher peak compute performance per chip compared to previous generation TPU v5e. | Trillium is generally available in North America (US East region), Europe (West region), and Asia (Northeast region) | Additional information on [Cloud TPU versions](https://cloud.google.com/tpu/docs/system-architecture-tpu-vm#versions) TPU 8i TPU 8i is optimized for post-training and inference while provides an 80% performance-per-dollar improvement over previous generations for low-latency inference for large MoE models. Coming soon TPU 8t TPU 8t is built for large-scale pre-training and embedding-heavy workloads at a scale of 9,600 chips in a single superpod, provides to 2.7x performance-per-dollar improvement over Ironwood for large-scale training. Coming soon Ironwood 7th-generation energy-efficient TPU engineered for large-scale training, reasoning, and inference. Features 9,216 liquid-cooled chips per pod, provides 42.5 ExaFlops and 4X better performance per chip over Trillium. Ironwood is generally available in North America (Central) and Europe (West region) Trillium Sixth-generation TPU featuring improved energy efficiency and peak compute performance for training and inference. Operates with 67% more energy efficiency and provides 4.7x higher peak compute performance per chip compared to previous generation TPU v5e. Trillium is generally available in North America (US East region), Europe (West region), and Asia (Northeast region) Additional information on [Cloud TPU versions](https://cloud.google.com/tpu/docs/system-architecture-tpu-vm#versions) How It Works Get an inside look at the magic of Google Cloud TPUs, including a rare inside view of the data centers . Customers use Cloud TPUs to run some of the large-scale AI workloads and that capacity comes from much more than just a chip. In this video, take a look at the components of the TPU system, including data center networking, optical circuit switches, water cooling systems, biometric security verification and more. Reduce pre-training timelines for massive foundation models. The TPU 8t provides high-performance compute power within a single pod and scales via the Virgo network. Paired with rapid storage access and Axion-powered NUMA isolation, the architecture achieves high Goodput—ensuring compute cycles are spent on active model building rather than idling during data transfer or hardware resets. Reduce pre-training timelines for massive foundation models. The TPU 8t provides high-performance compute power within a single pod and scales via the Virgo network. Paired with rapid storage access and Axion-powered NUMA isolation, the architecture achieves high Goodput—ensuring compute cycles are spent on active model building rather than idling during data transfer or hardware resets. Build base models into intelligent agents through intensive post-training workflows. The 8th generation of TPU system rapidly processes continuous reinforcement learning trials, rewarding the best reasoning paths without the cycle delays common to previous generations. This allows you to efficiently fine-tune world models, enabling agents to refine their reasoning in simulated environments before executing in the real world. Build base models into intelligent agents through intensive post-training workflows. The 8th generation of TPU system rapidly processes continuous reinforcement learning trials, rewarding the best reasoning paths without the cycle delays common to previous generations. This allows you to efficiently fine-tune world models, enabling agents to refine their reasoning in simulated environments before executing in the real world. Break the inference memory wall. The TPU 8i expands on-chip SRAM and high-bandwidth memory, hosting high-capacity KV caches entirely on-silicon. By using the SparseCore-Collectives Acceleration Engine (SC-CAE) to offload global communication tasks, this architecture significantly reduces on-chip latency, freeing the main compute cores for pure, low-latency token generation. Break the inference memory wall. The TPU 8i expands on-chip SRAM and high-bandwidth memory, hosting high-capacity KV caches entirely on-silicon. By using the SparseCore-Collectives Acceleration Engine (SC-CAE) to offload global communication tasks, this architecture significantly reduces on-chip latency, freeing the main compute cores for pure, low-latency token generation. Business Case Autonomous reasoning agents TPUs provide the memory bandwidth and low-latency inference required to run continuous, multi-step reasoning loops for real-time coding assistants, autonomous customer service, and security operations. Foundation models and multimodal generative AI Delivering continuous, high-throughput compute, TPUs efficiently build and serve massive foundation models across text, image, audio, and video modalities. Precision science and healthcare TPUs manage complex, matrix-heavy mathematics to accelerate computationally intensive simulations for structural biology, genomic sequencing, and drug discovery. Physical AI Build physical agents that interact with and adapt to the real world. Simulate and train robots, autonomous agents, and industrial machines faster and more efficiently with synthetic and real-world data.

retained text

Engineered for next-generation AI
Powered by AI Hypercomputer
A decade of Tensor Processing Units (TPUs)
TPUs are purpose-built for advanced AI workloads—from large language models and code generation to intelligent agents. They are the engine behind Gemini and Google’s flagship billion-user products, including Search, Photos, and Maps.
Open, flexible, and reliable operations
Build on an open ecosystem using familiar libraries and tools. TPUs provide native, high-performance support for PyTorch and JAX, and support the vLLM engine for fast inference. Manage and scale these deployments reliably across global clusters with Google Kubernetes Engine (GKE).
Performance without compromise
Turn months of training into weeks for frontier models. With up to 1 million TPU chips in a single cluster, TPUs maximize [goodput](https://cloud.google.com/blog/products/ai-machine-learning/goodput-metric-as-measure-of-ml-productivity), ensuring nearly every compute cycle goes toward active learning.
TPU 8t
TPU 8t is built for large-scale pre-training and embedding-heavy workloads at a scale of 9,600 chips in a single superpod, delivering the high-density compute required for frontier models with nearly 3x the compute performance per pod over the previous generation.
Coming soon.
TPU 8i
TPU 8i is optimized for post-training and inference. It provides an 80% performance-per-dollar improvement over previous generations for low-latency inference for large MoE models.
Ironwood - 7th generation
7th-generation energy-efficient TPU engineered for large-scale training, reasoning, and inference. Features 9,216 liquid-cooled chips per pod, provides 42.5 ExaFlops and 4X better performance per chip over Trillium.
Generally available.
Trillium - 6th generation
Sixth-generation TPU featuring improved energy efficiency and peak compute performance for training and inference. Operates with 67% more energy efficiency and provides 4.7x higher peak compute performance per chip compared to previous generation TPU v5e.
TPU software stack is engineered to bridge the gap between high-level machine learning code and custom silicon to optimize raw hardware efficiency.
JAX
Compile complex mathematical operations at near-native hardware speed on TPUs. This open-source library for numerical computation and automatic differentiation scales effortlessly across massive distributed TPU clusters using an explicit device mesh.
TorchTPU & PyTorch
Run PyTorch workloads directly on TPUs using standard syntax. TorchTPU is an open source, PyTorch backend for TPUs. It lets you train, fine-tune, and serve models on Cloud TPUs with the code and tools you already use, without changing PyTorch semantics or learning a TPU-specific API.
OpenXLA
Reduce compilation bottlenecks to extract high performance from TPU hardware. This open-source machine learning compiler automatically fuses operations, optimizes memory usage, and eliminates framework overhead for linear algebra computations across TPU backends.
MaxText
Slash training timelines for massive foundation models. This JAX-based reference implementation provides a highly scalable, out-of-the-box blueprint to achieve peak hardware utilization during LLM pre-training on TPUs.
Tunix
Streamline LLM post-training and alignment on TPU infrastructure. This lightweight, open-source JAX library delivers scalable workflows for Supervised Fine-Tuning (SFT) and Reinforcement Learning (RL) to efficiently turn raw models into production-ready agents.
vLLM
Maximize serving concurrency and drive down operational costs for real-time TPU inference. This high-throughput open-source engine leverages memory management techniques like PagedAttention to eliminate waste and deliver highly efficient LLM serving on TPU architecture. Supported inference for both JAX and PyTorch models on Cloud TPU.
SGLang
Fast serving framework for large language models with optimized backend execution for Cloud TPUs.
TPUs are powering the innovators
TPUs are custom-designed application-specific integrated circuits (ASICs) developed by Google. They are purpose-built from the ground up to accelerate machine learning and artificial intelligence workloads. TPUs are optimized specifically for the heavy matrix multiplication and tensor operations that act as the fundamental building blocks of neural networks, powering everything from large language models to complex reasoning agents.
At the heart of a TPU is a specialized Matrix Multiplier Unit (MXU) that processes massive amounts of matrix operations simultaneously. TPUs utilize a "systolic array" architecture that reads data once and flows it continuously through thousands of arithmetic logic units (ALUs), accumulating results without needing to constantly read and write to memory. They also support reduced precision arithmetic (like 16-bit or 8-bit floating-point), allowing for millions of computations per second without sacrificing the accuracy required for AI models.
Cloud TPUs offer native, high-performance support for leading machine learning frameworks, primarily TensorFlow, PyTorch, and JAX. Code written in these frameworks is compiled by the Accelerated Linear Algebra (XLA) compiler, which automatically optimizes the computational graph to run efficiently on the underlying TPU hardware.
TPUs excel at deep learning tasks that require massive parallel matrix operations. They are highly recommended for:
A TPU Pod is a massive physical cluster of TPU chips connected together over a specialized, high-speed network (such as Google's Optical Circuit Switching or Virgo network). A single superpod can contain thousands of interconnected chips. A "slice" is a smaller, dedicated subset of a pod that you can rent. This networking architecture allows developers to scale their AI workloads across massive amounts of compute with little to no code changes, effectively operating the entire slice as a single unified machine.
[Goodput](https://cloud.google.com/blog/products/ai-machine-learning/goodput-metric-as-measure-of-ml-productivity) refers to the actual, productive time your infrastructure spends actively training your model, rather than idling. Training massive foundation models involves complex orchestration across thousands of chips, meaning compute cycles can easily be wasted on data transfer delays, hardware resets, or checkpointing. Cloud TPUs are engineered to maximize Goodput through high-bandwidth interconnects and optimized memory caching, ensuring that paid compute cycles contribute directly to model advancement.
Users can support reliability, comprehensive monitoring, and management using Google Kubernetes Engine (GKE) and Cluster Director. By leveraging GKE for TPU, customers operate in a cloud-native ecosystem featuring Inference Gateway for load balancing and the ability to scale deployments up to 130,000 GKE nodes.
Cloud TPUs support high-throughput, low-latency LLM serving primarily through popular open-source serving engines: vLLM and SGLang:
Both projects provide optimized, package-specific libraries tailored for TPU hardware to maximize serving efficiency. For setup guides see the [Cloud TPU Inference Documentation](https://docs.cloud.google.com/tpu/docs/tpu-inference).

document fetched 2026-10-22 02:58