Responsibilities:
• Design and Implement Data Storage Solutions: Utilize Azure Data Lake Storage, Azure SQL Database, and other Azure services to design and implement scalable data storage solutions
• Develop and Manage Data Pipelines: Create and maintain ETL (Extract, Transform, Load, SQL, PySpark, Azure Synapse/Notebooks) processes using Azure Data Factory to integrate data from various sources into the data lake
• Data Quality and Governance: Ensure data quality and governance by implementing data validation, cleansing processes, and maintaining data catalogs
• Performance Optimization: Optimize data storage and retrieval processes to ensure efficient performance and scalability
• Security and Compliance: Implement security measures to protect sensitive data and ensure compliance with relevant regulations
• Collaboration: Work closely with data scientists, analysts, and other stakeholders to understand data requirements and deliver solutions that meet their needs
• Maintenance and Monitoring: Regularly monitor and maintain the data lake infrastructure to ensure reliability and performance
• Documentation: Create and maintain comprehensive documentation for data lake architecture, processes, and best practices
Experience:
• Bachelor degree or equivalent required, Masters or equivalent desired
• 2 years of hands on experience in IT Data Engineering
Competencies:
• Priority Setting
• Customer focused
• Drive for results
• Self-development
• Ethics and Values
• Compassion
• Organizing
• Project Management techniques / methods
• Presentation skills
• Communications Skills
• Cultural Awareness
• Action-oriented skills
• Conflict management
Responsibilities:
• Design and Implement Data Storage Solutions: Utilize Azure Data Lake Storage, Azure SQL Database, and other Azure services to design and implement scalable data storage solutions
• Develop and Manage Data Pipelines: Create and maintain ETL (Extract, Transform, Load, SQL, PySpark, Azure Synapse/Notebooks) processes using Azure Data Factory to integrate data from various sources into the data lake
• Data Quality and Governance: Ensure data quality and governance by implementing data validation, cleansing processes, and maintaining data catalogs
• Performance Optimization: Optimize data storage and retrieval processes to ensure efficient performance and scalability
• Security and Compliance: Implement security measures to protect sensitive data and ensure compliance with relevant regulations
• Collaboration: Work closely with data scientists, analysts, and other stakeholders to understand data requirements and deliver solutions that meet their needs
• Maintenance and Monitoring: Regularly monitor and maintain the data lake infrastructure to ensure reliability and performance
• Documentation: Create and maintain comprehensive documentation for data lake architecture, processes, and best practices
Experience:
• Bachelor degree or equivalent required, Masters or equivalent desired
• 2 years of hands on experience in IT Data Engineering
Competencies:
• Priority Setting
• Customer focused
• Drive for results
• Self-development
• Ethics and Values
• Compassion
• Organizing
• Project Management techniques / methods
• Presentation skills
• Communications Skills
• Cultural Awareness
• Action-oriented skills
• Conflict management
• Bachelor degree or equivalent required, Masters or equivalent desired
• 2 years of hands on experience in IT Data Engineering