# Qwen3.8-27B Q4_K_M on 2x3060 — retained record

> Retained reddit record from reddit:LocalLLaMA.

Canonical: https://highsignal.app/data/records/c2d48a96539293cf

[← sources](https://highsignal.app/data)


source: [reddit](https://highsignal.app/data/reddit) (reddit:LocalLLaMA)  published 2026-09-26 06:54 ingested 2026-09-27 02:40

[original source ↗](https://www.reddit.com/r/LocalLLaMA/comments/1wqjr3f/qwen3827b_q4_k_m_on_2x3060/)

## retained text



```
For the last few days, I've been using two computers to run multiple agents.

My 4x3090 machine is running Qwen 3.8 27B with parallel=2, so I can run two agents at the same time. My pi instances are running on a machine with 2x3060, running/managing smaller models such as Gemma 26B A4B or Qwen 35B A3B.

Today, however, I needed my 4x3090 machine for some vLLM work, so I was missing my AI. I decided to try running Qwen 3.8 27B on the 2x3060 machine instead.

Here is the command:

    #!/bin/bash
    
    ~/git/llama.cpp/build/bin/llama-server \
      -sm tensor \
      -lv 4 \
      -m ~/LLMs-huge/Qwen3.8-27B-UD-Q4_K_M.gguf \
      -c 50000 \
      --host 0.0.0.0 \
      --jinja \
      -fa on \
      --keep 4096 \
      -b 8192 \
      -ub 512 \
      --no-kv-unified \
      --fit-target 1024 \
      --ctx-checkpoints 12 \
      --temp 0.6 \
      --top-p 0.95 \
      --top-k 20 \
      --min-p 0 \
      --presence-penalty 0 \
      --repeat-penalty 1.0 \
      --spec-type ngram-mod \
      --spec-type draft-mtp \
      --spec-draft-n-max 3 \
      --chat-template-kwargs '{"preserve_thinking":true}'

And here are some real-world speeds from actual usage:

    7.21.365.095 I slot print_timing: id  3 | task 4179 | prompt eval time =     873.12 ms /    48 tokens (   18.19 ms per token,    54.98 tokens per second)
    7.21.365.099 I slot print_timing: id  3 | task 4179 |        eval time =    2760.36 ms /   139 tokens (   20.00 ms per token,    49.99 tokens per second)
    7.21.365.099 I slot print_timing: id  3 | task 4179 |       total time =    3633.48 ms /   187 tokens
    (...)
    7.37.423.312 I slot print_timing: id  3 | task 4225 | prompt eval time =    1640.55 ms /   325 tokens (    5.05 ms per token,   198.10 tokens per second)
    7.37.423.315 I slot print_timing: id  3 | task 4225 |        eval time =   
    14163.13 ms /   630 tokens (   22.52 ms per token,    44.41 tokens per second)
    7.37.423.316 I slot print_timing: id  3 | task 4225 |       total time =   15803.68 ms /   955 tokens
    (...)
    7.44.951.668 I slot print_timing: id  3 | task 4445 | prompt eval time =     888.79 ms /    40 tokens (   22.22 ms per token,    45.01 tokens per second)
    7.44.951.672 I slot print_timing: id  3 | task 4445 |        eval time =    6364.50 ms /   277 tokens (   23.06 ms per token,    43.37 tokens per second)
    7.44.951.673 I slot print_timing: id  3 | task 4445 |       total time =    7253.29 ms /   317 tokens

Hopefully this helps anyone wondering how usable 3060s still are for local LLM, the main problem is short context (too short for long agentic session)

https://preview.redd.it/bjtsld02btrh1.png?width=1854&format=png&auto=webp&s=e974f8668ba9ba9c5b16c8ab27273dd1562900bd
```



document fetched 2026-09-27 00:39 · canonical https://reddit.com/r/LocalLLaMA/comments/1wqjr3f/qwen3827b_q4_k_m_on_2x3060
