Anyscale, in partnership with Google Cloud, announced major performance improvements for Ray Serve on Google Kubernetes...

Anyscale, in partnership with Google Cloud, announced major performance improvements for Ray Serve on Google Kubernetes Engine (GKE). The optimizations—including HAProxy integration, direct token streaming, and a v2 Ray executor backend for vLLM—yield up to 5x higher throughput and 8x lower latency for LLM inference compared to prior versions (Google Cloud Blog). Benchmarks on GKE A4 VMs (NVIDIA B200) show Ray Serve now approaching native vLLM performance while retaining its developer-friendly Python APIs and ecosystem. This collaboration strengthens Anyscale's competitive position in the AI inference serving market, as it eliminates the historical performance gap that limited Ray adoption in production. The improvements are available in Ray 2.56 and later. Anyscale's website continues to highlight its role as the engine behind Ray for multimodal data curation, training, and inference (Anyscale IR snapshot). The partnership with Google validates Anyscale's technology and may accelerate enterprise adoption of Ray on GKE.