Homemade data platform frameworks - bloated nonsense?
retained text
Did any of you work in companies where engineers built custom frameworks that actually deliver? I recently started at yet another company with such framework (about 13k lines of boilerplate python/pyspark code sitting on top of their Azure Databricks Delta Lakehouse). The thing doesn't seems to improve any aspect of what a data platform should offer. Previous such company was even worse with 30k lines, plus they tried to implement a data vault. I'm clearly biased towards keeping things simple. So I may be a bit unfair towards this approach. Hence my question: **Have you worked with great homemade frameworks? What were the secret ingredients to make it work?**
document fetched 2026-10-03 00:24 · canonical https://reddit.com/r/dataengineering/comments/1wvoy6o/homemade_data_platform_frameworks_bloated_nonsense