Data Engineer at MediaPal

Job Purpose

The Data Engineer will apply their expertise in developing data pipelines and data models to support analysts and data scientists to build statistical models and complete analyses.

He/She will partner with data scientists and business leaders, helping them define and refine their data needs and then build sustainable data pipelines and structures

Duties & Responsibilities

• Build, design and maintain accurate, robust and optimal data pipeline architecture
• Work with data scientists and collaborators (marketing, sales, operations, etc) to gather needs, communicating with ease and comfort while promoting partnership.
• Monitor the health of data and pipeline using monitoring tools, custom dashboards and cloud pipeline monitoring
• Convert data processing pipelines to container used applications and orchestrate containers to deliver scalable and reliable performance
• Build and maintain databases systems
• Identify, design, and implement internal process improvements: automating manual processes, and optimizing data delivery, re-designing infrastructure for greater scalability,
• Develop, maintain, and test infrastructures for optimal extraction, generation, transformation and loading of data from a variety of sources 
• Work with data, executives, product and design teams to provide architectural solutions
• Retrain data and provide predictive analysis in real time
• Maintain consistency of recording systems
• Ensuring data security
• Data filtering and optimization to meet MediaPal’s functional and non-functional requirements
• Build statistical analytics tools to utilize the data pipeline and provide actionable insights into key business performance metrics
• Build APIs in databases to enable data scientists, engineering teams and business intelligence analysts to query the data as well as test APIs for robust performance and security

Qualifications & Skills Desirable.

• Knowledge of distributed systems for distributed processing of large data sets using simple programming models
• Degree in Computer Science, Statistics, Informatics, Information Systems or another quantitative field
• 2+ years of experience in hypotheses development, identification of patterns within data, analyzing data and interpreting results.
• Advanced SQL knowledge and experience working with relational databases, query authoring (SQL) as well as working familiarity with a variety of databases.
• Experience building and optimizing data pipelines, architecture and data sets
• Strong analytic skills related to working with unstructured datasets.
• Build processes supporting data transformation, data structures, metadata, dependency and workload management.
• A successful history of manipulating, processing and extracting value from large disconnected datasets.
• Working knowledge of message queuing, stream processing, and highly scalable ‘big data’ data stores.
• Strong project management and organizational skills
• Experience supporting and working with cross-functional teams in a dynamic environment

Working Relationships

Internal Relationships:

• Directly accountable to Director-Product and Innovation and Head of Programmatic Engineering

External Relationships:
 

• MediaPal Clients

What We Offer

• Comprehensive Health Insurance- Inpatient/Outpatient/Dental/Optical
• Great Learning Opportunities
• High Performance Culture

Good to Note

At MediaPal, we are an equal opportunity employer & value diversity at our company. We do not discriminate on the basis of race, religion, colour, national origin, gender, sexual orientation, age, marital status, veteran status, or disability status.