# Migration of tables and dependencies from SQL Server of 75gb size to Postgres — retained record

> Retained reddit record from reddit:database.

Canonical: https://highsignal.app/data/records/a8508b2215e4a690

[← sources](https://highsignal.app/data)


source: [reddit](https://highsignal.app/data/reddit) (reddit:database)  published 2026-11-17 15:41 ingested 2026-11-18 02:41

[original source ↗](https://www.reddit.com/r/Database/comments/1xynofe/migration_of_tables_and_dependencies_from_sql/)

## retained text



```
We planned it to run in batches and stages with dependencies, using workers on 300 million rows distributed across multiple tables. This breakdown prevents memory overflow, isolates failures, and maintains data integrity from the legacy infrastructure to the target destination.

Despite these architectural considerations and concurrent workers, benchmark testing and predictive modeling reveal a bottleneck: an estimated time close to 30 hrs. This execution window introduces risks like network interruptions, extended maintenance, and delayed synchronization with ongoing transactional workloads.

Faced with this timeline, architects and database administrators question the technology stack. Is there a better way than through python scripts? While Python offers versatility and rapid development for Extract, Transform, Load pipelines, its interpreted nature, Global Interpreter Lock limitations, and single-threaded bottlenecks may not provide the raw processing speed required for hundreds of millions of records.
Exploring alternative paradigms—such as distributed computing frameworks, native database migration utilities, bulk insert operations, compiled programming languages, or massively parallel processing engines—might unlock the performance gains necessary to compress this thirty-hour window, ensuring a seamless and efficient migration experience.
```



document fetched 2026-11-18 00:42 · canonical https://reddit.com/r/Database/comments/1xynofe/migration_of_tables_and_dependencies_from_sql
