Data Engineer at ICAP Columbia University

Overall function

• Reporting to the Team Lead Software Development, the Data Engineer will play a crucial role in harnessing disease surveillance data for ongoing surveillance of new and emerging infectious diseases. They will focus on designing, implementing, and maintaining data pipelines and storage solutions, ensuring data is readily available for analysis and decision-making. The ideal candidate should have a minimum of 2 years of experience in data engineering, with a strong understanding of data integration, processing, and database management. While reporting to Team Lead Software Development, the Data Engineer will closely collaborate with the DDSR M&E department within the Kenya Ministry of Health.

Major Accountabilities:

• Design and implement data pipelines to capture, process, and transform disease surveillance data from various sources into a usable format.
• Develop and maintain databases that efficiently store and organize large datasets, ensuring data integrity and availability.
• Collaborate with stakeholders to understand data requirements and integrate data from multiple systems into a unified and accessible repository.
• Implement data quality checks and validation processes to ensure the accuracy and consistency of surveillance data.
• Build automated data processing solutions that streamline data ingestion and reduce manual data manipulation.
• Investigate and evaluate new informatics tools and technologies, staying updated with industry advancements to enhance existing data systems.
• Provide training and mentorship to the MoH DDSR surveillance and M&E teams in data engineering best practices.
• Act as a bridge between business stakeholders and the data engineering team, ensuring effective communication and collaboration throughout data-related projects.
• Create comprehensive documentation of data pipelines, data models, and processes for future reference.
• Ensure compliance with data governance and security standards to protect sensitive health data.
• Continuously monitor and optimize data pipelines and databases for improved performance.
• Education: Bachelor’s degree in computer science, information engineering, data science, or a related field. Additional training in data engineering, data management, or relevant certifications is a plus.

Experience, Skills, and Minimum Required Qualifications:

• Minimum of two (2) years of experience in data engineering, including working with large datasets and data processing.
• Proficiency in data integration tools and technologies.
• Strong understanding of database management, ETL (Extract, Transform, Load) processes, and data warehousing.
• Proficient with relevant data engineering tools and programming languages (e.g., Python, SQL, SSIS, Talend, Spark).
• Excellent organizational, communication, and interpersonal skills.
• Strong critical thinking and problem-solving abilities.
• Ability to work independently and collaboratively in a team environment.
• Experience with health data standards and compliance is advantageous.
• Meticulous attention to detail and data quality.

Applications close: 30 Nov 2023