← sources

Inside OpenAI's custom Jalapeño AI accelerator: interview with Richard Ho, VP Hardware at OpenAI

source: reddit (reddit:hardware)published 2026-10-02 08:32ingested 2026-10-03 02:39

retained text

It is an inference accelerator rather than a training chip, and it carries 216 GiB of HBM4 at 15.4 TB/s alongside a compute die and an IO chiplet. Chip power is rated at 700 W peak with measured sustained draw closer to 550 W. A local domain is 128 accelerators, a full system is 2,048 accelerators, and at four-bit precision that system reaches 27 EFLOP/s.

At HotChips 2026, Richard Ho presented the first measured performance from working silicon, alongside Ravi Narayanaswami and Chris Leary. The numbers claimed against NVIDIA GB200/GB300:

- 1.5x to 1.9x better performance per watt at peak throughput
- 1.7x to 3.6x better latency
- Up to 104x better at operating points NVIDIA struggles to reach.

document fetched 2026-10-03 00:35 · canonical https://reddit.com/r/hardware/comments/1wvoh2v/inside_openais_custom_jalape%C3%B1o_ai_accelerator