Liquid AI's RIPL Engine Beats vLLM by 28x on Real-Time Inference

retained text

Liquid AI's RIPL Engine Beats vLLM by 28x on Real-Time Inference
Liquid AI's new RIPL inference engine runs small Liquid Foundation Models in under 50ms end-to-end, targeting search, ads, fraud, and payments.
- Liquid AI launched RIPL, a sub-50ms inference engine for LFM2.5 models on NVIDIA GPUs.
- Benchmarks show 5.6 to 28.4x lower p95 latency than Qwen3.5 on vLLM.
- Shopify runs it in production search with sub-20ms inference and ~50% fewer parameters.
- Targets ad retrieval, reranking, fraud, payments, credit decisions, and image alt text.
- Supports multimodal LFMs and Liquid's d1 decision models for classifier replacement.
- Enterprise-only on H100, H200, B200, and RTX PRO 6000; access by request.
Liquid AI has announced Liquid RIPL, short for Real-time Inference Pipeline, an engine for serving Liquid Foundation Models inside a live request path. Liquid says selected workloads return a complete generated response in under 50 milliseconds, with stable tail latency under continuous peak traffic.
Engineering teams often reserve ranking, fraud scoring, payment checks, and query classification for gradient-boosted trees or compact encoder models because autoregressive generation exceeds the latency budget. RIPL aims to place small generative models in those paths, where an application must wait for the result before continuing.
Milliseconds set the architecture
Search ranking, ad retrieval, and payment authorization operate under strict latency ceilings. A model can improve an offline relevance metric and still be unusable if it adds hundreds of milliseconds to every request.
General-purpose inference servers such as vLLM emphasize throughput, broad model support, and efficient batching. Liquid designed RIPL around its smaller LFM2.5 models and short, latency-sensitive requests. The supported models range from 230 million to 2.6 billion parameters.
Read the benchmark by request shape
Liquid reports that RIPL produced p95 latency between 5.6 and 28.4 times lower than Qwen3.5 served through vLLM across tested workloads, including requests with 32 input and 32 output tokens and requests with 256 input and 128 output tokens. The company ran the tests on a single NVIDIA B200 GPU.
P95 latency means 95 percent of requests completed at or below the reported time. The results show why the sub-50 ms claim must be tied to a specific model, prompt length, output length, request rate, and GPU. Longer generations with larger models reached hundreds of milliseconds in Liquid’s own tests.
The benchmark attributes gains to the serving engine and the models. On the same LFM2.5-350M model, Liquid’s charts place RIPL below vLLM across tested request shapes, isolating part of the engine’s contribution. The announcement provides company-run results without an independent reproduction or a TensorRT-LLM comparison, so the reported ratios should not be treated as universal across serving stacks.
Serving speed and task quality require separate evaluation. Liquid reports that LFM2 models were pretrained on 10 to 12 trillion tokens using knowledge distillation, difficulty-ordered training data, supervised fine-tuning, preference optimization, and model merging. LFM2-2.6B scores 79.56 percent on the instruction-following benchmark IFEval and 82.41 percent on the grade-school mathematics benchmark GSM8K. Those aggregate scores do not establish accuracy for a production ranking, fraud, or classification task.
Shopify puts RIPL in search
Liquid says Shopify uses LFMs for search enhancement, with production inference below 20 milliseconds. According to Liquid, the deployment uses roughly 50 percent fewer parameters than the compared alternatives and runs two to ten times faster. The public figures do not include enough workload and baseline detail for independent reproduction.
Workloads built around tiny outputs
The announced use cases share a common shape: constrained tasks that consume structured context and produce a short completion, score, label, or reason code.
Liquid labels the reranking workload “prefill only,” meaning the model derives a score while processing the prompt instead of decoding a longer sequence token by token. That pattern removes much of the serial generation cost that makes conventional LLM requests slow.
A label-token model could replace a compact text classifier while allowing teams to change tasks through prompts. Production validation still needs to measure label stability, calibration, malformed outputs, and accuracy under prompt changes.
RIPL also supports multimodal LFMs and Liquid’s d1 decision models. Credit and payment deployments require additional controls for auditability, policy enforcement, bias testing, deterministic fallbacks, and human review where applicable.
Enterprise access, closed engine
Enterprise customers can request RIPL for deployments on NVIDIA H100, H200, B200, and RTX PRO 6000 GPUs. Liquid has not published pricing, a self-service download, or an open-source release of the engine. Customers request access and deploy it on their own hardware.
Liquid’s latency calculator estimates end-to-end performance from the selected GPU, request shape, and request rate. The estimate can narrow an initial hardware plan, but production testing must include real prompt distributions, concurrency bursts, warm-up behavior, and failure recovery.
What to verify before adoption
An evaluation should establish the following details against production traces:
- p50, p95, and p99 latency at expected and burst concurrency
- Throughput for the exact input and output token distributions
- Precision or quantization settings and their effect on task accuracy
- API compatibility, model packaging, observability, and deployment tooling
- Capacity requirements, failover behavior, and cold-start time
- Output constraints, calibration, and fallback handling for structured decisions
RIPL’s strongest fit is a short prompt, a tiny output, a small LFM, and a fixed accelerator fleet. Capacity planning needs the exact request shape and concurrency profile because Liquid’s published examples range from single-digit milliseconds to 449.1 milliseconds at p95.

document fetched 2026-10-23 16:26