# Qwen3-VL 8B on a laptop vs Opus 5.5 / Sonnet 5 / GPT-5.6 on 137 messy documents: beat GPT-5.6 on tax forms, lost badly on Indian date formats[R] — retained record

> Retained reddit record from reddit:MachineLearning.

Canonical: https://highsignal.app/data/records/15d08249642e102d

[← sources](https://highsignal.app/data)


source: [reddit](https://highsignal.app/data/reddit) (reddit:MachineLearning)  published 2026-09-28 11:11 ingested 2026-09-29 02:39

[original source ↗](https://www.reddit.com/r/MachineLearning/comments/1wsbqni/qwen3vl_8b_on_a_laptop_vs_opus_55_sonnet_5_gpt56/)

## retained text



```
I benchmarked Qwen3-VL 8B Instruct (Q4\_K\_M, Ollama, M5 24GB, \~30s/doc) against Claude Opus 5.5, Sonnet 5 and GPT-5.6 Terra on:

\- receipts: CORD (Indonesia) and SROIE (Malaysia), 30 each

\- 20 scanned 1980s-90s invoices , answer keys human-verified

\- 32 real IRS forms, 4 damage levels (generated this week, so not in anyone's training data)

\- 10 synthetic Indian bank statements, 15 CUAD contracts

Results (documents fully right):   
\- Opus 89%,   
\- Sonnet 85%,   
\- Qwen 8B 59%,   
\- GPT-5.6 Terra 57%.



Qwen-specific findings:

\- W-2s: 21/32 fully right vs GPT-5.6 Terra 7/32

\- Indian bank statements: 2/10. Every amount and balance correct, but dd-mm-yyyy read as mm-dd.

\- Contracts (long text): 2/15, mostly wrong expiry dates

\- The default qwen3-vl:8b tag in Ollama is the thinking variant and ignores think:false. On long contracts it spent all 4,096 tokens thinking and returned nothing. Use :8b-instruct.



Other things I didn't expect:

\- GPT-5.6 Terra "corrects" unusual spellings (Rachael -> Rachel, Kelleyland -> Kellyland)

\- Asking a model to check its own output changed almost nothing (119/137 identical)

\- At least 4 of the 30 SROIE receipts have a wrong published answer key (e.g. B1750 where the receipt prints 81750)

Next I'm fine-tuning the 8B to fix the date and spelling failures and will post the result either way.



https://preview.redd.it/vmwm4920v8sh1.png?width=2200&format=png&auto=webp&s=173dcd046cdcb7c2d26ba9027f1ef4b9c24486e0

Prompts, keys, scorer tests and every raw output: [https://github.com/TashonBraganca/messy-docs-bench](https://github.com/TashonBraganca/messy-docs-bench)  

```



document fetched 2026-09-29 00:28 · canonical https://reddit.com/r/MachineLearning/comments/1wsbqni/qwen3vl_8b_on_a_laptop_vs_opus_55_sonnet_5_gpt56
