← sources

Which coding agent gets the most out of a local Qwen 3.8 27B? I lined up 10 and logged tokens, cache and sandbox blocks

source: reddit (reddit:AI_Agents)published 2026-10-29 17:09ingested 2026-10-30 02:36

retained text

Part 2 of my agent test. Last time it was a 4B model, and you asked for the hardware, token counts and more agents. (Disclosure: I work on Atomic Agent, one of the ten.)

Used: MacBook Pro M5 Max, 64 GB. Qwen 3.8 27B Q4 on llama.cpp, thinking off.

**Results**

|\#|Agent|Passed|Median time|Median tokens|Cache reuse|Tried to go online|
|:-|:-|:-|:-|:-|:-|:-|
|1|Pi|5/5|38 s|9.2k|72%|no|
|2|DeepSeek Harness|5/5|60 s|35.3k|83%|no|
|3|Codex CLI|5/5|94 s|36.3k|79%|every run|
|4|OpenClaw|5/5|106 s|39.0k|72%|unclear|
|5|Cline|5/5|152 s|29.6k|79%|every run|
|6|OpenCode|4/5|50 s|32.4k|75%|every run|
|7|Hermes|4/5|80 s|64.9k|93%|no|
|8|Atomic Agent|5/5|56 s|30.4k|86%|no|
|9|goose|4/5|98 s|20.3k|76%|every run|
|10|Qwen Code|4/5|183 s|107.9k|74%|unclear|

**How it ran**

* same model and server for everyone
* 5 small coding tasks, each with a little trap
* each agent locked in a sandbox: it can only touch its own folder and only talk to the model
* auto-approve on, telemetry off, 5 min limit

**What I noticed**

Pi is crazy lean: about 9k tokens per task, the lowest on every single task. Qwen Code was the hungriest at about 108k per task.

Atomic Agent is the odd one out in how it works. It was the only one talking to llama.cpp's native endpoint with a grammar that forces the tool calls into shape. The other nine all went through the OpenAI-style API. It's also built to keep its prompt the same from step to step, which is why it reused 86% of the cache (2rd of 10). It stayed offline the whole time and passed 5/5.

The log task got three agents. They grepped "ERROR" and got 244 instead of 64, because some normal lines just mention errors. OpenCode, goose and Atomic Agent fell for it.

Codex, Cline, goose and OpenCode tried to reach the internet in every run, even with telemetry off. The sandbox blocked it, so I don't know what they were calling. OpenClaw and Qwen Code are "unclear" because they run as plain node and I couldn't tell them apart.

**Setup notes**

* Cline and Qwen Code have usage stats on by default, DeepSeek Harness uploads session logs (per their docs). I turned it all off
* Codex needs llama-server's /v1/responses now
* Hermes needs at least 64K context

Is anyone running these on a smaller model, like a 9B or 14B? Curious how the ranking changes.

document fetched 2026-10-30 00:18 · canonical https://reddit.com/r/AI_Agents/comments/1xipvla/which_coding_agent_gets_the_most_out_of_a_local