← sources

5 Frontier LLM's grade 39 local LLM's on 50 prompts

source: reddit (reddit:LocalLLaMA)published 2026-10-29 01:10ingested 2026-10-30 02:40

retained text

This is just something i had running for about 10-20 hours (kept adding models), various tests/benchmarks and had 4 (Claude kind of graded, his tokens are too precious) frontier LLM's grade the answers of the local LLM's.

Big models got about 44 and smaller models got 66 prompt/tests. Some written by me, most by AI, most Ai generated tests based off work ive done with ai agents.

Grading took about $30-40 for argon and 2x5 hour limits for codex grading and who cares about antigravity Gemini 3.8.

Astra, sol, terra, argon, Gemini and opus 5.5 all wrote some tests, we all per capita probably wrote about the same amount of test/benchmarks

All tested on a 5060 ti 16GB, 64GB ram, 5800x3d

Outputs are 3k tokens and some tests like debugging only had 1 prompt, for debugging specifically it was a real bug I came across in a project

After grading, each of the four judges looked at its own score table, agent counts, speeds and six sample answers, and named two models: its favorite (most capable today) and the best foundation (the one to fine-tune and give specialist models).

Gemini 3.8

* Top 3 capability: Bonsai-2 27B PQ2, Underdog-Saluki 27B, Qwen3.6 35B-A3B
* Top 3 foundation: Nanbeige4.2 3B, Gemma-4 12B Q6, Bonsai-2 27B PQ2

Terra (GPT-6.1)

* Top 3 capability: Bonsai-2 27B PQ2, Underdog-Saluki 27B, Gemma-4 12B Q6
* Top 3 foundation: Bonsai-2 27B PQ2, Underdog-Saluki 27B, Nanbeige4.2 3B

Sol (GPT-6.1 Sol)

* Top 3 capability: Bonsai-2 27B PQ2, Underdog-Saluki 27B, Qwen3.6 35B-A3B
* Top 3 foundation: Underdog-Saluki 27B, Bonsai-2 27B PQ2, Qwen3.6 35B-A3B

Argon (Gemini 4)

* Top 3 capability: Underdog-Saluki 27B, Bonsai-2 27B PQ2, Gemma-4 12B Q6
* Top 3 foundation: Gemma-4 12B Q6, Qwen3.6 35B-A3B, OmniCoder 9B Q8

document fetched 2026-10-30 00:45 · canonical https://reddit.com/r/LocalLLaMA/comments/1xi6edm/5_frontier_llms_grade_39_local_llms_on_50_prompts