Data Engineer at Workforce Africa

Job Overview

• Our client is in a dynamic phase of growth and development. The successful candidate will have the opportunity to shape the future of cloud solutions in Africa and contribute to the success of the company Solutions as a premier technology provider. We rely on powerfully insightful data to inform our systems and solutions, and we’re seeking an experienced pipeline-centric data engineer to put it to good use. The ideal candidate will have the expected mathematical and statistical expertise combined with a rare curiosity and creativity. This person will wear many hats in the role, including identifying business problems and translating these to data services and engineering outcomes. You will deliver data solutions that empower better decision-making and flexibility of your solution that scales to respond to broader business questions both internally and externally.

Responsibilities

• Building and maintaining data warehouses, setting up pipelines, and deploying models while ensuring data is securely and readily available to internal and external stakeholders(e.g. using modern data warehousing and database architecture)
• Identifying, designing, and implementing internal process improvements, including re-designing infrastructure for greater scalability, optimizing data delivery, and automating manual processes.
• Build the infrastructure required for optimal extraction, transformation, and loading of data from a wide variety of data sources using SQL, Looker, and BigQuery.
• Work with data and technology while assisting with data-related technical issues and supporting data infrastructure needs while ensuring compliance with data governance and security policies.
• Provide assistance with the development and management of continuous integration, continuous delivery, and continuous testing pipelines for machine learning and other model iterations for customers.
• Maintain comprehensive documentation for data engineering processes, workflows, and solutions.
• Analyze large datasets to identify trends, patterns, and correlations relevant to our clients' business objectives.
• Develop and maintain data models, algorithms, and statistical analyses to support decision-making processes.
• Collaborate with cross-functional teams to gather requirements and define data-driven solutions.
• Design and implement data visualization dashboards and reports to communicate findings effectively.
• Perform data cleansing, transformation, and validation to ensure data accuracy and consistency.
• Stay current with industry trends and best practices in data analysis, cloud computing, and related technologies.
• Support ongoing data-related projects and initiatives, including data migration, integration, and optimization efforts.

Education and experience:

• Bachelor's degree in Computer Science, Data Science, Statistics, or a related field.
• 2+ years of experience building and maintaining early-stage data warehouses, setting up pipelines, and modeling analytical data (e.g. using modern data warehousing and database architecture principles).
• Understanding of the ETL cycle.
• Experience with data pipeline and workflow management tools: Azkaban, Luigi, Airflow
• Experience performing root cause analysis on internal and external data and processes to answer specific business questions and identify opportunities for improvement.
• Knowledge of the data modeling lifecycle and an appreciation of the importance of metadata.
• Proficient in programming Languages such as Python SQL is a must, Java or Scala
• Experience (or a strong interest in) collaborating with sales and customer success teams to design and implement scalable infrastructures for different types of structured and unstructured data.
• Familiarity with big data technologies (e.g., Hadoop, Spark) is a plus
• Strong analytical and problem-solving skills, with a keen attention to detail.
• Experience with data visualization tools such as Looker Studio, Tableau, Power BI, or D3.js.  Familiarity with cloud platforms and services, such as AWS, Azure, or Google Cloud Platform.
• Strong project management and organizational skills.
• Experience supporting and working with cross-functional teams in a dynamic environment

Additional Requirements:

• Passion for staying current with emerging technologies and industry trends.
• Results-oriented with a focus on achieving strategic goals and objectives.
• Excellent communication and interpersonal skills, with the ability to collaborate effectively with team members and stakeholders.
• Ability to work independently and prioritize tasks in a fast-paced environment.
• Knowledge of machine learning concepts and techniques is a plus