# dev.to: Count It or Compute It: When a Tool Returns Rows, the Models That Count Them Right Spend the Tokens — retained record

> Retained dev-ecosystems record from dev-ecosystems:devto.

Canonical: https://highsignal.app/data/records/0cc0d12921afe9c8

[← sources](https://highsignal.app/data)


source: [dev-ecosystems](https://highsignal.app/data/dev-ecosystems) (dev-ecosystems:devto)  published 2026-09-28 19:04 ingested 2026-10-01 02:33

[original source ↗](https://dev.to/gde/count-it-or-compute-it-when-a-tool-returns-rows-the-models-that-count-them-right-spend-the-tokens-2hae)

## retained text



```
A Kaggle benchmark of the step agents rarely test: counting what a tool returns. Ten models, 68 questions, one tool that returns the count and one that returns the rows. With the count, every model is right at a flat cost. With the rows, models that reason through the list count 330 ids right and spend 6 to 26 times the tokens doing it; models that answer straight away get 0 to 10 of 21.

tags: devchallenge, kagglechallenge, ai, machinelearning

reactions=15 | comments=5
```



document fetched 2026-10-01 02:33
