I measured 44 object detection checkpoints on one RTX 4090. At batch 1, YOLO26-n and YOLO26-x are only 2.7 ms apart end to end
retained text
Disclosure first: the site is mine, and it's a solo side project. It's free, with no ads and no sign-up, and there's nothing money-related planned at all. I used to work in computer vision, so I wanted to build something in that area that people would actually find useful, and I happened to have a spare RTX 4090 sitting around. That's the whole story. **The finding that surprised me most** YOLO26, measured as one full `predict()` call at batch 1 (pre-processing + forward + post-processing) on an RTX 4090: |Size|n|s|m|l|x| |:-|:-|:-|:-|:-|:-| |COCO box AP (measured)|40.5|47.9|52.6|54.4|57.0| |Median latency (ms)|14.2|14.3|13.3|17.1|16.9| That's +16.5 AP for +2.7 ms, and m was the fastest of the five. YOLOv8 shows the same pattern (n 11.6 ms to x 12.9 ms) and so does RF-DETR (nano 9.8 ms to large 10.7 ms). My reading is that, in this setting, the model's size isn't what sets the latency. So if you deploy on a desktop GPU at batch 1, the smallest size may not buy you much. Edge devices and batched or TensorRT pipelines are a different story, and I didn't measure those. **A few representative checkpoints** Each one ran at its official default input size and precision. AP is my own score on COCO val2017 (5,000 images, pycocotools), with the authors' number in brackets. |Model|Input|Prec.|COCO AP, measured (paper)|Latency median, ms|VRAM peak|License (code / weights)| |:-|:-|:-|:-|:-|:-|:-| |YOLO26-n|640x448|fp16|40.5 box (40.9)|14.2|571 MiB|AGPL-3.0| |YOLO26-x|640x448|fp16|57.0 box (57.5)|16.9|1.1 GiB|AGPL-3.0| |RF-DETR large|704x704|fp16|56.3 box (56.5)|10.7|1.0 GiB|Apache-2.0| |GTR-x|640x640|fp16|59.1 box (59.4)|33.5|732 MiB|MIT| |DEIMv2-x|640x640|fp32|57.6 box (57.8)|35.1|808 MiB|DEIMv2 License, non-commercial| |D-FINE-x|640x640|fp32|55.6 box (55.8)|42.6|925 MiB|Apache-2.0| |RF-DETR-Seg 2xlarge|768x768|fp16|49.6 mask (49.9)|16.4|2.5 GiB|Apache-2.0| |YOLO26-seg-x|640x448|fp16|46.4 mask (47.0)|18.5|1.2 GiB|AGPL-3.0| The chart shows all 44 scored detection checkpoints, colored by license. **Other things I noticed** * Across the 68 scored checkpoints that have an authors' number, my score came out lower every time, by 0.1 to 1.2 AP (median 0.3). * Licenses drift. DEIMv2 moved to its own DEIMv2 License on 2026-08-24, which allows research, education, benchmarking and evaluation only; earlier copies were Apache-2.0. Depth Anything V2 code is Apache-2.0, but only the Small weights are; Base and Large weights are CC-BY-NC-4.0. * The open-vocabulary models (Grounding DINO, MM-Grounding-DINO, LLMDet) took 118 to 468 ms at 1199x800 fp32. LLMDet and Grounding DINO base saw COCO during training, so their COCO scores aren't zero-shot. **How it's measured (short version)** * Latency: one `predict()` call, image decode excluded, batch 1. fp16 if the official code supports it, otherwise bf16, otherwise fp32. Three separate runs of 10 warm-ups and 100 timed calls each; the site shows the median and p90. * VRAM: device peak minus the idle baseline taken before loading, polled every 10 ms or faster. * Accuracy: full COCO val2017 with the reference scorer, at the same precision I time it. My numbers and paper numbers are stored and shown separately. * License: code license from the repo, weights license from the model card, per checkpoint, with the date I checked. The verdict is the stricter of the two. This is a summary, not legal advice. **Limits, honestly** * One GPU (RTX 4090, driver 580.173.02). The card also drives a display, so every run records its own idle baseline. * Official PyTorch code paths only, with no TensorRT or ONNX numbers. Precision follows each repo, so some models are fp32 and others fp16, which affects comparisons. * Input sizes differ between models (official defaults), so this isn't an equal-resolution comparison. * Pose, depth and matting models have latency and VRAM numbers but no accuracy scores yet. * RT-DETRv4 needed small fixes to run; its page links the patch. Site: [https://modelsthatrun.dev](https://modelsthatrun.dev). The measuring tool is open ([https://github.com/modelsthatrun/runner](https://github.com/modelsthatrun/runner)) if you want to run the same protocol on your own NVIDIA GPU and send the record. **Questions for you** 1. For batch-1 deployment, would you rather see a TensorRT/ONNX column next to the PyTorch numbers, or a forward-pass-only time next to the end-to-end one? 2. Is comparing models at their official default input size the right call, or would a fixed-resolution view (for example everything at 640) be more useful to you? 3. Which detection or segmentation models are missing that you'd want measured next?
document fetched 2026-11-10 00:23 · canonical https://reddit.com/r/computervision/comments/1xri3xk/i_measured_44_object_detection_checkpoints_on_one