# Analysis: from 2019 to 2025, gains in pretraining compute efficiency came mostly from data improvements rather than model improvements (Dwarkesh Podcast) — retained record

> Retained techmeme record from techmeme.

Canonical: https://highsignal.app/data/records/eb81022e1c3b7926

[← sources](https://highsignal.app/data)


source: [techmeme](https://highsignal.app/data/techmeme) published 2026-09-09 02:10 ingested 2026-09-09 02:43

[original source ↗](https://www.dwarkesh.com/p/pretraining-progress-is-mostly-data)

## retained text



```
Techmeme permalink: https://www.techmeme.com/260908/p49#a260908p49

<a href="https://www.dwarkesh.com/p/pretraining-progress-is-mostly-data"><img align="RIGHT" border="0" hspace="4" src="http://www.techmeme.com/260908/i49.jpg" vspace="4" /></a>
<p><a href="https://www.techmeme.com/260908/p49#a260908p49" title="Techmeme permalink"><img height="12" src="http://www.techmeme.com/img/pml.png" style="border: none; padding: 0; margin: 0;" width="11" /></a> <a href="https://www.dwarkesh.com/">Dwarkesh Podcast</a>:<br />
<span style="font-size: 1.3em;"><b><a href="https://www.dwarkesh.com/p/pretraining-progress-is-mostly-data">Analysis: from 2019 to 2025, gains in pretraining compute efficiency came mostly from data improvements rather than model improvements</a></b></span>&nbsp; &mdash;&nbsp; Breaking down 6 years of pretraining progress into data vs model improvements&nbsp; &mdash;&nbsp; How much of the rapid progress in AI that we've seen over the last few years 1 &hellip; </p>
```



document fetched 2026-09-09 11:52 · canonical https://dwarkesh.com/p/pretraining-progress-is-mostly-data
