Data Engineer [for Project Domino; preferably senior 5 years and up experience]

Responsibilities

Design, architect and maintain secure data platforms such as setting up data storage solutions like SQL databasesImplement and maintain optimized workflows, ETL / ELT processes, and pipelines; deliver flexibility for data consumption by designing batch and streaming data use casesEnsure support for traceability and reproducibility through enabling of data lineage features for ingested dataWork with colleagues such as Data Scientists, Data Analysts, software and data engineers, and architects on implementing solutions involving data on distributed systems and platformsCreate highly available and scalable capabilities that deliver data related tooling and manage data platforms and pipelines for stakeholders

Qualifications

Proficiency and experience in software engineering programming languages such as Python. Experience in Golang is preferredExpertise in SQL / RDBMS databases and proficiency in setting up these resources. Additional experience in NoSQL databases and setting up these is preferredExtensive experience in design and implementation of data handling processes and pipelines (ingestion, transformation, modelling, and storage) using Python, Apache Airflow, SparkExpertise in cloud platforms and technologies (Azure, AWS, GCP) is a plus but not requiredExperience with streaming platform setup and consumption such as Kafka, FlinkExperience with consumption from other storages for streaming such as S3, SparkExperience in containers and container orchestration platforms such as Docker and KubernetesExperience in CI / CD and Devops, scripting languages such as shell and automationPreferably a degree in the related fields of Computer ScienceStrong problem solving and solutions engineering mindsetExperience in working and coordinating remotely