Dicky Faisal Dramar
← Back to projects

Healthcare

Data Migration & Change Data Capture System

Client
BPJS Kesehatan
Year
2024
Industry
Healthcare

A migration platform with real-time change data capture that moved millions of healthcare records without losing a single row.

Technologies

  • Python
  • Apache Airflow
  • Debezium
  • SQL Server
  • PostgreSQL

The problem

Moving critical healthcare data off a legacy system is high-stakes work: the data has to arrive intact, stay consistent while the source keeps changing, and remain verifiable throughout. Any dropped or corrupted record would be unacceptable.

What I built

I designed a migration platform combining a batch migration path with real-time change data capture (CDC). Debezium streamed changes off the source database so that records modified mid-migration were captured rather than missed, while Apache Airflow orchestrated the pipeline stages and gave clear visibility into progress and failures. The flow bridged SQL Server and PostgreSQL with reconciliation checks to confirm completeness at each step.

Outcome

Millions of healthcare records were migrated with zero data loss, with the CDC layer keeping source and target consistent right up to cutover.