Inside OpenAI's custom Jalapeño AI accelerator: interview with Richard Ho, VP Hardware at OpenAI
retained text
It is an inference accelerator rather than a training chip, and it carries 216 GiB of HBM4 at 15.4 TB/s alongside a compute die and an IO chiplet. Chip power is rated at 700 W peak with measured sustained draw closer to 550 W. A local domain is 128 accelerators, a full system is 2,048 accelerators, and at four-bit precision that system reaches 27 EFLOP/s. At HotChips 2026, Richard Ho presented the first measured performance from working silicon, alongside Ravi Narayanaswami and Chris Leary. The numbers claimed against NVIDIA GB200/GB300: - 1.5x to 1.9x better performance per watt at peak throughput - 1.7x to 3.6x better latency - Up to 104x better at operating points NVIDIA struggles to reach.
document fetched 2026-10-03 00:35 · canonical https://reddit.com/r/hardware/comments/1wvoh2v/inside_openais_custom_jalape%C3%B1o_ai_accelerator