We are seeking a highly skilled Senior Data Engineer with extensive experience in designing, building, and optimizing high-volume data pipelines. The ideal candidate will have strong expertise in Python, Azure Cloud data services, DevOps, and CI/CD tools, along with a solid understanding of AI/ML techniques and big data processing frameworks like Apache Spark and PySpark. Responsibilities Design, develop and maintain scalable and reliable data pipelines for processing large datasets. Work with Azure Data Services including Azure Data Factory, Databricks, and HDInsight to build and optimize data solutions. Implement CI/CD pipelines using tools like Jenkins, Argo CD, Travis, and Azure DevOps to ensure smooth data deployment processes. Leverage Apache Spark and PySpark for big data processing and transformation. Collaborate with AI/ML teams to integrate machine learning models into data pipelines. Ensure data quality, governance, and security in accordance with best practices. Work in an Agile/SCRUM environment, driving continuous improvement in data engineering workflows. Monitor and optimize high-volume data pipelines to ensure efficiency and scalability.