← sources

dev.to: Running Gemma 4 on EC2 G5g: Graviton2 AMD with NVIDIA GPU

source: dev-ecosystems (dev-ecosystems:devto)published 2026-08-13 18:42ingested 2026-08-14 07:29

retained text

A field report on serving Gemma 4 E2B under vLLM on AWS G5g — the only aarch64 + SM 7.5 hardware there is. No published build covers that combination, AWS quietly solves half of it, and the thing that actually blocks you is 64 KiB of shared memory.

tags: aws, vllm, cuda, machinelearning

reactions=7 | comments=0

document fetched 2026-08-19 06:44