← sources

Homemade data platform frameworks - bloated nonsense?

source: reddit (reddit:dataengineering)published 2026-10-02 09:04ingested 2026-10-03 02:36

retained text

Did any of you work in companies where engineers built custom frameworks that actually deliver?

I recently started at yet another company with such framework (about 13k lines of boilerplate python/pyspark code sitting on top of their Azure Databricks Delta Lakehouse). The thing doesn't seems to improve any aspect of what a data platform should offer. 

  
Previous such company was even worse with 30k lines, plus they tried to implement a data vault.



I'm clearly biased towards keeping things simple. So I may be a bit unfair towards this approach. Hence my question: **Have you worked with great homemade frameworks? What were the secret ingredients to make it work?** 

document fetched 2026-10-03 00:24 · canonical https://reddit.com/r/dataengineering/comments/1wvoy6o/homemade_data_platform_frameworks_bloated_nonsense