fastforks: faster drop-in versions for many computer vision libraries. Tensorrt speed without compiling or leaving PyTorch
retained text
Over the last few months I built fastforks, a library with faster versions of published vision models behind one API. It covers stereo matching, monocular and multi-view depth, segmentation (SAM), object detection, classification and feature backbones like DINOv2 and DINOv3. You load a model by its id and get the original network's output within a small tolerance, just faster.
Every model has two backends:
* **portable**: the network restructured in plain PyTorch. Runs on CPU, CUDA and Apple MPS, so every model works on a Mac.
* **kernels**: custom CUDA kernels on top, plus Metal kernels for 19 model families on Apple silicon. If it can't run on your machine, it falls back to portable automatically.
Most of the portable speedup comes from five things: the weights are converted to fp16 once instead of wrapping the model in autocast, matmuls use fp16 accumulation (PyTorch's allow\_fp16\_accumulation), attention uses PyTorch's fused scaled-dot-product kernel, constant layers (BatchNorm, LayerScale) are folded into their neighbours, and resizing and post-processing run on the GPU instead of in NumPy/OpenCV on the CPU. It is all plain PyTorch, so the models still export to ONNX and TensorRT. The kernels backend then replaces the remaining hot spots with custom CUDA and Metal kernels. These were developed with heavy AI assistance and tuned by benchmarking on real hardware. Every model is checked against the authors' code, and the output differences are listed on each model page.
The image shows end-to-end latency on an RTX 5060 Ti. "Original" is the authors' code running in fp16 with autocast, so the baseline already uses half precision. Memory drops too: Depth Anything 3 Large goes from 3.4 GB to under 1 GB peak. To be upfront, these are some of the better results. Across all models and sizes I measured, the median speedup over the original is 1.4x for portable and 2.9x for kernels. A few plain ViT classifiers gain nothing from the kernels.
fastforks also works with the usual tools. Every model supports torch.compile, and all but 4 export to TensorRT. These changes pay off there too: for Depth Anything 3 Mono Large (the mono variant, not the one in the chart), a TensorRT fp16 engine built from the original takes 18.1 ms, one built from the fastforks version takes 14.4 ms, and the kernels take 14.6 ms. So TensorRT on fastforks is about as fast as the kernels, but the engine takes a few minutes to build, while the kernels are fast from the first call.
from fastforks import depth
model = depth.load("da3-large-1.1")
result = model("room.jpg")
result.depth # [H, W] float32 tensor
I benchmarked everything on seven devices: RTX 5060 Ti, L4, A10, L40S, A100, RTX PRO 6000 and an Apple M1 Pro. The website lets you switch between them. The CUDA kernels need Linux x86-64 and an Nvidia GPU from the RTX 30 series or A100 onward. Everything else uses portable. Note that some of the underlying models have non-commercial licences. Each model page lists the code and weights license.
I'm also curious which models you'd want added next, and whether anyone is interested in AMD or Jetson support.
Repo: [https://github.com/kleinicke/fastforks](https://github.com/kleinicke/fastforks)
Benchmarks: [https://fastforks.dev/models](https://fastforks.dev/models)
document fetched 2026-11-11 00:23 · canonical https://reddit.com/r/computervision/comments/1xssnwq/fastforks_faster_dropin_versions_for_many