Multiprocessing

Optimising a Slow Data Processing Pipeline

Context: A batch job that processes 10 million CSV rows runs for 4 hours. Each row is parsed, validated, transformed, and inserted into a …