# SPARK 2.6 4b and 1.7b Fixed a reasoning loop by editing one row of weights. — retained record

> Retained reddit record from reddit:LocalLLaMA.

Canonical: https://highsignal.app/data/records/2a95a7446f8effc7

[← sources](https://highsignal.app/data)


source: [reddit](https://highsignal.app/data/reddit) (reddit:LocalLLaMA)  published 2026-11-14 07:37 ingested 2026-11-15 02:36

[original source ↗](https://www.reddit.com/r/LocalLLaMA/comments/1xvwi81/spark_26_4b_and_17b_fixed_a_reasoning_loop_by/)

## retained text



```
Just put my first two models up on Hugging Face.

I used Qwen4 to do direct weight editing and fixed the reasoning loop problem. Got up to a 97% reduction on the 1.7B and a solid drop on the full 4B.

# The problem: the “but wait…” spiral

Spark-X2.6 has this nasty failure mode where it reaches the correct answer… and then just refuses to commit. It starts spiraling into re-decision: “but wait, should I…”, re-derives the same conclusion in slightly different words, re-lists its tools, and never actually acts or stops. It’s a semantic loop, so normal repetition metrics barely catch it. In an agent setting it’s fatal — the model knows the answer and still won’t pull the trigger.

# The fix: surgery, not training

No fine-tuning. No sampling hacks. Just a no-training surgical weight edit — literally one row of the output head.

* 4B: 2,560 weights changed (≈ 6×10⁻⁷ of the model)
* 1.7B: 2,048 weights (untied head, so the input embeddings are completely untouched)

The diagnosis came from watching activations at the exact moment the model reaches a conclusion but before it starts looping. The closure token (</think>) was basically missing (rank \~54, P ≈ 7×10⁻⁶) while restart vocabulary dominated. Since the head is just logit(t) = emb\[t\] · h, that closure deficit is a pure head-readout gap. So we solved for the minimum-norm row delta that raises P(close) by \~14 nats at those conclusion states, dequantized the single row, added the delta, requantized, and spliced it back in. Every other byte is bit-identical to the original. The edit is conclusion-selective — it doesn’t truncate legitimate reasoning.

# Results

Loop rate on a controlled branch-point benchmark (12 seeds per prompt):

|Model|Before|After|Reduction|
|:-|:-|:-|:-|
|4B|0.750|0.312|58.4%|
|1.7B|0.875|0.062|92.9%|

And it doesn’t cost general capability. Same eval suites, before vs after:

|Suite|4B Δ|1.7B Δ|
|:-|:-|:-|
|ARC-Challenge|−0.040|\+0.020|
|WinoGrande|\+0.030|\+0.020|
|PIQA|−0.010|\+0.010|
|HellaSwag|0.000|0.000|
|MMLU|\+0.022|0.000|
|GSM8K|−0.017|\+0.233|
|**Mean**|−0.002|\+0.047|

The 4B is basically neutral overall. The 1.7B actually got a bit smarter — GSM8K more than doubled (0.200 → 0.433). Fixing the loop didn’t break the brain.

All that said: this specifically fixes the thinking loop. Broader agent tool-use looping can still show up on some tasks. That’s a separate problem I’m still characterizing.

# Links

Both GGUFs include the exact delta and a CPU-only offline script that reproduces the checkpoint bit-for-bit from the upstream quant.

* **4B**: [Spark-X2.6-4B-heretic-noloop-headrow-GGUF](https://huggingface.co/graydini/Spark-X2.6-4B-heretic-noloop-headrow-GGUF)
* **1.7B**: [Spark-X2.6-1.7B-noloop-headrow-GGUF](https://huggingface.co/graydini/Spark-X2.6-1.7B-noloop-headrow-GGUF)

Full credit to XHToken (base), the heretic finetune, and the upstream quantizers — this is just a derivative that adds the head-row edit. The diagnosis, the activation surgery, and the benchmark rig were all done by Qwen4. That was the genuinely hard part: finding the actual causal lever in the weights instead of guessing at sampling knobs. I just pointed it at the problem and reviewed the work.
```



document fetched 2026-11-15 00:36 · canonical https://reddit.com/r/LocalLLaMA/comments/1xvwi81/spark_26_4b_and_17b_fixed_a_reasoning_loop_by
