Data Engineer

About the role

Raising The Village is expanding its data infrastructure to support programmatic analytics, machine learning, and field evaluation across Uganda, Rwanda, and the DRC. The Data Engineer will join the senior‑led Venn department, reporting to the Senior Data Scientist, and will help design, build, and maintain the pipelines and warehouse layers that power impact measurement and AI tools.

Key responsibilities

• Design and deliver batch and streaming pipelines (SurveyCTO, ArcGIS, mobile apps) into Databricks using PySpark and SQL.
• Implement ELT/ETL workflows following Medallion Architecture (Bronze, Silver, Gold) and maintain Delta Lake tables with ACID compliance.
• Develop data observability checks (freshness, volume anomalies, schema drift) using Great Expectations, dbt tests, or Databricks monitoring.
• Support ML and AI pipelines by integrating household data, image classification outputs, and model predictions into unified warehouse layers.
• Collaborate with software engineers, data scientists, and field teams to translate requirements into reliable, documented solutions.

Required profile

• Bachelor’s degree in Computer Science, Data Engineering, Statistics or related field, or equivalent practical experience.
• 1‑3 years of experience building production‑grade data pipelines and handling end‑to‑end data integration.
• Demonstrated ability to work in sprint‑based delivery cycles and own pipeline workstreams from design to monitoring.

Required skills

• Python (including Pandas)
• SQL
• PySpark
• Databricks platform
• Delta Lake
• Unity Catalog
• Great Expectations or similar data‑quality tools
• dbt
• AWS (basic familiarity)