Big Data Engineer at Standard Focus Ltd

General Duties & Responsibilities

• Working across a number of business areas providing development, maintenance and support
• Working as part of a team and occasionally solo developments as the business needs arise
• Responsible for the building, deployment, and maintenance of mission critical analytics solutions that process data quickly at big data scales
• Contributes design, code, configurations, and documentation for components that manage data ingestion, real time streaming, batch processing, data extraction, transformation, and loading across multiple game franchises.
• Cross-trains other team members on technologies being developed, while also continuously learning new technologies from other team members.
• Interacts with engineering teams and ensures that solutions meet customer requirements in terms of functionality, performance, availability, scalability, and reliability.
• Works directly with business analysts and data scientists to understand and support their use cases
• Performs development, QA, and dev-ops roles as needed to ensure total end to end responsibility of solutions.
• Designing and maintaining data systems and databases; this includes fixing coding errors and other data-related problems.
• Mining data from primary and secondary sources, then reorganizing said data in a format that can be easily read by either human or machine.
• Using statistical tools to interpret data sets, paying particular attention to trends and patterns that could be valuable for diagnostic and predictive analytics efforts.
• Demonstrating the significance of their work in the context of local, national, and global trends that impact both their organization and industry.
• Preparing Dashboards as a Self-Service BI for executive leadership that effectively communicate trends, patterns, and predictions using relevant data.
• Collaborating with programmers, engineers, and organizational leaders to identify opportunities for process improvements, recommend system modifications, and develop policies for data governance.
• Creating appropriate documentation that allows stakeholders to understand the steps of the data analysis process and duplicate or replicate the analysis if necessary.
• Select appropriate datasets and data representation methods
• Keep abreast of developments in the field
• Help identify probable causes and provide immediate solution during an incident
• Work within an agile environment following an agile framework.
• Contribute significant ideas for making the applications better and easier to use
• Participate in cutting edge research in artificial intelligence and machine learning applications.
• Contribute to engineering efforts from planning and organization to execution and delivery to solve complex, real-world engineering problems.

Skills and Experience

• Preferred skills in noSQL, Pig, Matlab, SAS, Java, Ruby, C++, Perl, and APIs to work with available data sources
• Evaluate and improve all aspects of our existing ETL system
• Experience with big data tools and architectures, such as Cloudera Hadoop, HDFS, Hive, BigQuery, Snowflake and Spark.
• Experience in Python programming language and frameworks such as Flask, AIOhttp
• Understanding of data structures, data modelling and software architecture
• Advanced knowledge of SQL queries
• Working knowledge of telematics interfaces and streaming solutions (MQTT, NiFi, Kafka, etc.).
• Experience with Cloud platforms such as Google cloud platform, AWS would be plus
• Ability to work in a team
• Outstanding analytical and problem-solving skills
• Experience in multi-threading, message queues, WebSockets
• Experience with automation of the development and test processes through CI/CD pipeline (Gitlab, SonarQube, Artifactory, Docker containers)
• Intermediate knowledge of API development
• Proving track record in building high performance, highly available and scalable systems

Job Qualifications

• Bachelor's degree in Computer Science, or related technical field, or equivalent work experience.
• 3 - 5 years of relevant work experience.
• Experience designing and implementing distributed software systems (e.g Python).
• Research or Industry experience in Big Data, Artificial Intelligence, Machine Learning (ML) models, ML infrastructure, Natural Language Processing or Deep Learning.
• Good oral and written English communication skills
• Strong grasp of established and emerging technologies, systems, platforms, and software
• Ability to organize and manage multiple priorities
• Technical curiosity - Willingness to explore and learn new technologies that are unfamiliar
• Ability to work in a fast pace delivery oriented environment
• Ability to deliver short term results while invest in long term strategic solutions
• Self-starter, and Self-motivated and able to learn independently
• Team player who is eager to help others to succeed