← sources

a local Jev-style decision head onto Qwen 2.5 1.5B

source: reddit (reddit:LocalLLaMA)published 2026-09-20 11:06ingested 2026-09-21 02:38

retained text

Jev from TypeSafe AI got me thinking about non-autoregressive decision engines, so I built a small proof-of-concept called AES to test the concept locally on open weights.

It uses Qwen/Qwen2.5-1.5B-Instruct as the base. Instead of generating tokens, it runs a single prefill pass, pulls the final hidden state through a custom FP32 SwiGLU probe, and calculates cosine similarity directly against candidate token embeddings. It implements the same NOUL, CHOICE, and SCORE decision primitives.

Full disclosure: I only trained it on a few datasets (\~3,200 samples from BoolQ, ARC, and Yelp) for 1 epoch on free Kaggle T4 GPUs, so do not expect broad zero-shot generalization. It is strictly a small experiment.

It runs in \~28ms batched and works well on simple, unambiguous multiple-choice problems as long as you stick to single-token anchors (A, B, C, D or true/false) rather than long multi-word strings.

Hugging Face repo: https://huggingface.co/Brinij/aes

document fetched 2026-09-21 00:37 · canonical https://reddit.com/r/LocalLLaMA/comments/1wle746/a_local_jevstyle_decision_head_onto_qwen_25_15b