# One of the biggest overlooked inefficiencies and bottlenecks of modern AI - website viewing — retained record

> Retained reddit record from reddit:ClaudeCode.

Canonical: https://highsignal.app/data/records/216aca1100d92849

[← sources](https://highsignal.app/data)


source: [reddit](https://highsignal.app/data/reddit) (reddit:ClaudeCode)  published 2026-10-18 21:46 ingested 2026-10-19 02:42

[original source ↗](https://www.reddit.com/r/ClaudeCode/comments/1x9fcur/one_of_the_biggest_overlooked_inefficiencies_and/)

## retained text



```
The whole thing where most websites are barred from being crawled/read by AI and the whole Cloudflare protection against LLMs is a big impediment to intelligence and the full capability of AI and will surely change in the future. One of the most glaring inefficiencies out now. Imagine if you could just cleanly read the data on like 200 websites without any of the bullshit involved, any website whatsoever. Even the best ones (which are definitely not Claude) hit problems around this. Even local solutions with complicated scrapers often have difficulty with "protection" and even in non-headless browser mode it's very slow and sluggish to process all of it if you can even bypass capchas.

Anyways, just something thats clearly 'legacy internet' preventing progress.
```



document fetched 2026-10-19 00:35 · canonical https://reddit.com/r/ClaudeCode/comments/1x9fcur/one_of_the_biggest_overlooked_inefficiencies_and
