Back to projects
LiveData Engineering
End-to-End Automated Geospatial ETL Pipeline for Power Line Data (Apache NiFi + PostGIS)
Designed and deployed a streaming batch geospatial ETL pipeline using Apache NiFi to automate ingestion, transformation, and loading of GeoJSON power line data into PostgreSQL/PostGIS. Engineered data validation and regex-based transformation workflows to handle malformed and semi-structured inputs, improving data quality and reliability to ensure reliable storage of LineString geometries for scalable spatial analysis.
Live Preview
Having trouble viewing? Open directly
Impact & Metrics
Automated 100% of data ingestion and processing workflow (no manual intervention)
Processed and transformed GeoJSON records at scale (batch-to-stream conversion via NiFi)
Improved data quality by handling malformed and inconsistent fields (e.g., voltage extraction via regex)
Enabled parallel processing of records using NiFi flow architecture
Enabled GIS-ready dataset for downstream analytics and visualization
Architecture