# MuckScraper 1.0: self-hosted news aggregator that groups coverage into stories, shows bias balance, and summarizes locally with Ollama — retained record

> Retained reddit record from reddit:selfhosted.

Canonical: https://highsignal.app/data/records/dfb6e22259d0dab9

[← sources](https://highsignal.app/data)


source: [reddit](https://highsignal.app/data/reddit) (reddit:selfhosted)  published 2026-10-21 00:20 ingested 2026-10-22 02:39

[original source ↗](https://www.reddit.com/r/selfhosted/comments/1xb8nb5/muckscraper_10_selfhosted_news_aggregator_that/)

## retained text



```
I'm glad to announce MuckScraper 1.0. It's a personal project I've been building for about six months, and it's finally out of beta. I posted it here a few months ago and a lot of your suggestions were a great help in getting it here.

The name is a nod to the old muckrakers. It doesn't do their job, but it does the digging through the coverage for you: it pulls articles from RSS feeds (and NewsAPI/GNews if you add keys) and scrapes the full article where it can. It groups coverage from different outlets into a single story using vector embeddings, then labels each outlet left, center or right on the American scale (AllSides ratings where they exist, a local model otherwise). Finally it writes a summary and an analysis of how each side is covering the story, all on your own hardware through Ollama by default.

**Try it first**

* [**https://sample.muckscraper.news**](https://sample.muckscraper.news): a public, read-only copy of the self-hosted app, refreshed after every fetch run. It's the same thing you'd run at home, minus the article text (only summaries, with links back to the publishers).
* [**https://muckscraper.news**](https://muckscraper.news): the newspaper site from the first post. Two editions of 20 stories a day, chosen to be balanced across the spectrum. Story pages have an "Aggregator View" button that opens the same story in the sample. It also has a daily horoscope from Astrid, the newsroom's AI astrologer, worked out from the real sky that day. She's the only one on staff who's allowed to make things up.

**What's new since the first post**

* **Syndicated copies** (u/ExcellentTrust4433, and thanks for crediting the repo): you were right that a lot of "multi-source" stories were one wire report under three mastheads. Copies are now detected by text overlap and counted once. The aggregator labels them "Same report as ...", and the newspaper site leaves them out.
* **Op/Ed:** opinion pieces are kept out of the news but saved separately, from a set of feeds balanced left and right. Each column is linked to the news story it argues about, with a one-line stance. A story with enough columns gets a short roundup of what each side is arguing.
* **Search** with Meilisearch: stories and articles, filtered by time window, with left/center/right counts per story.
* **Configured from the admin UI, not code:** feeds, topics, LLM prompts, the run schedule, and blocked sources and headline patterns.
* **LLM providers:** Ollama is still the default and everything can stay local. For people without a GPU, Gemini, Groq or any OpenAI-compatible endpoint is optional, and you can split the cheap and heavy calls between two providers.

**What I run it on:** the stack (Postgres + pgvector, Meilisearch, Flask, the scheduler) runs in containers on 2 cores of an old i7-4790K with a 60 GB disk. It uses about 1.5 GB of RAM between runs; I give it 8 GB so a run has room for headless Chromium and the search indexer. Ollama runs on a separate box with a 10 GB GPU, using Gemma 4 12B (QAT) and nomic-embed-text. A full run handles roughly 200-300 new articles in 25-35 minutes, and the GPU box powers off between runs, woken by Wake-on-LAN. Since April it's stored about 120,000 articles (80,000 with the full text), grouped into 69,000 stories, 15,000 of them covered by two or more outlets, and published 369 editions.

I built this with a lot of help from Claude Code, but it isn't something I prompted once and walked away from. I'm there every step of the way: I decide what gets built, how it should work and which route to take when there's a choice, test it, and run it every day. When the AI gets something wrong, I'm the one who catches it.

**Links**

* GitHub: [https://github.com/grregis/MuckScraper](https://github.com/grregis/MuckScraper)
* Sample aggregator: [https://sample.muckscraper.news](https://sample.muckscraper.news)
* Newspaper site: [https://muckscraper.news](https://muckscraper.news)

I'd like to hear what's missing. I'm especially after a non-US bias reference, if anyone knows a good one for the UK, Europe or Australia.
```



document fetched 2026-10-22 00:32 · canonical https://reddit.com/r/selfhosted/comments/1xb8nb5/muckscraper_10_selfhosted_news_aggregator_that
