·       Data Pipeline Development: Design, implement, and optimize end-to-end data pipelines on Azure, focusing on scalability and performance.

·       Develop and maintain ETL workflows for seamless data processing. Azure Cloud Expertise.

·       Utilize Azure services such as Azure Data Factory, Azure SQL Database, and Azure Databricks for effective data engineering. Implement and manage data storage solutions on Azure.

·       Data Transformation with PySpark Leverage PySpark for advanced data transformations, ensuring high-quality and well-structured output.

·       Implement data cleansing, enrichment, and validation processes using PySpark.

·       Performance Optimization Optimize data pipelines, queries, and PySpark jobs to enhance overall performance and scalability.

·       Identify and address performance bottlenecks within data processing workflows.

·       Proven experience as a Data Engineer, emphasizing expertise in Azure.

·       Proficiency in Azure Data Factory and other relevant Azure services.

·       Expertise in PySpark for data processing and analytics is a must.

·       Experience with data modeling, ETL processes, and data warehousing.