# High-Performance llama.cpp fork for Legacy GPUs: Kepler Support, CUDA Graphs & Windows NCCL! — retained record

> Retained reddit record from reddit:LocalLLaMA.

Canonical: https://highsignal.app/data/records/d680b9c509f07c2a

[← sources](https://highsignal.app/data)


source: [reddit](https://highsignal.app/data/reddit) (reddit:LocalLLaMA)  published 2026-10-24 20:01 ingested 2026-10-25 02:39

[original source ↗](https://www.reddit.com/r/LocalLLaMA/comments/1xeicw2/highperformance_llamacpp_fork_for_legacy_gpus/)

## retained text



```
I've been working on improving `llama.cpp` support and performance on older NVIDIA GPUs, particularly Kepler and other pre-Ampere architectures.

The goal is simple: **get more life out of older CUDA hardware instead of treating it as e-waste.**

# Achieved Performance

On **MTP Qwen3.6-35B-A3B with Q8\_0 quantization**, I've observed the following (generation) speeds using 4x Tesla K40c GPUs (Kepler, compute capability 3.5):

* **Previously unable to start → 35 tps** Fixes to cuBLAS compute-type selection for legacy gpu in mainline llama cpp to run on hardware that previously failed during initialization (cc35-50).
* Approximately **35 → 55 tps** after enabling CUDA graphs on my four-K40c setup, under the same reported clock configuration.
* **\~5–10% end-to-end improvement:** testing with **Windows NCCL integration** compared with the previous configuration. Throughput reaching above **60 tps** while P.P. speeds are variable between **70-100 tps** in my testing. 

These are preliminary results from my own system, not universal benchmarks. More systematic testing across architectures and workloads is still needed.  
  
I'm especially interested in hearing from people running Kepler, Maxwell, Pascal, Volta, and other older NVIDIA GPUs. If you have legacy hardware, I'd love to compare results and test compatibility across different configurations!

**Project / source:** [https://github.com/theIvanR/llama-on-legacy-gpu](https://github.com/theIvanR/llama-on-legacy-gpu)

**Llama.cpp pull request**: [https://github.com/ggml-org/llama.cpp/pull/30727](https://github.com/ggml-org/llama.cpp/pull/30727)

**Windows NCCL fork:** [https://github.com/theIvanR/nccl-on-windows](https://github.com/theIvanR/nccl-on-windows)

*Feedback, reproducible benchmarks, and contributions are very welcome!*


```



document fetched 2026-10-25 00:36 · canonical https://reddit.com/r/LocalLLaMA/comments/1xeicw2/highperformance_llamacpp_fork_for_legacy_gpus
