# graphsignal-top: a `top` for inference GPUs in the open-source Graphsignal profiler, kernels ranked by time, live, with readable names — retained record

> Retained reddit record from reddit:LocalLLaMA.

Canonical: https://highsignal.app/data/records/8da8e64da7d9e45e

[← sources](https://highsignal.app/data)


source: [reddit](https://highsignal.app/data/reddit) (reddit:LocalLLaMA)  published 2026-10-12 13:35 ingested 2026-10-13 02:42

[original source ↗](https://www.reddit.com/r/LocalLLaMA/comments/1x421ow/graphsignaltop_a_top_for_inference_gpus_in_the/)

## retained text



```


A few weeks ago we posted the open-source Graphsignal profiler here: wrap `vllm serve` / `sglang serve` / whatever in `graphsignal-run` and it serves everything it measures as JSON, mainly so an agent can read it in a loop.

Turns out we kept opening the JSON ourselves to see what the GPU was doing. So now there is `graphsignal-top`:  
  
`graphsignal-run vllm serve <model> --port 8000   # one terminal`  
`graphsignal-top                                  # another`  
  
What you get: per-GPU util over time, memory, power, throttling; the engine's running/waiting requests, tok/s and TTFT/TPOT p50/p95; a GPU time split (kernels / graphs / memcpy / sync); and every kernel ranked by time over the last 5 s with a readable name and category, not just the mangled symbol. Tabs for CUDA graph replays, transfers and syncs.

Screenshot is Qwen3-32B on 2x RTX PRO 6000, tp 2, over PCIe. NCCL all-reduce is a quarter of GPU time and half the wall time is host sync waiting on it. You see that in one glance.

Works remotely too: `graphsignal-top --host gpu-box` against a profiler started with `--listen-host`, or port-forward over ssh. `--once` prints a single frame if you just want it in a log.

Repo: [https://github.com/graphsignal/graphsignal](https://github.com/graphsignal/graphsignal)
```



document fetched 2026-10-13 00:43 · canonical https://reddit.com/r/LocalLLaMA/comments/1x421ow/graphsignaltop_a_top_for_inference_gpus_in_the
