Skills & Experience

· Provide technical expertise and guidance on the selection and hands-on implementation of platforms, tools and technologies across multiple cloud platforms (Azure, GCP) and Databricks

· Architect and build scalable and secure data pipelines, data lakes and datwarehouses to support the storage, processing and analysis of large volumes of structured and unstructured data

· Lead and mentor a team of technical professionals in the design, development and implementation of big data solutions on cloud platforms and data analytics projects

· Stay abreast of emerging trends, technologies and industry developments in big data, cloud, AI/ML and GenAI

· Develop and maintain best practices, standards and guidelines for data management, data governance and data security in alignment with regulatory requirements and industry standards

· Collaborate with cross-functional teams including data scientists, data engineers, business analysts and stakeholders to define project requirements, objectives and timelines

· Experience in building data pipelines (batch and real time) and handling variety of data sources including semi-structured and unstructured data sources (image/video, GPS. IoT)

· Strong expertise in ETL/ELT processes, data lake, Datawarehouse, data mesh

· Experience in Databricks, Unity catalogue, open table formats (Delta, Iceberg) and tools (Trino, Flink, Presto)

· Good knowledge of native data engineering services of Azure/GCP, scheduling and orchestration tools like Airflow, Composer

· Experience in handling streaming data and messaging tools (Kafka, Pub/Sub, Service Bus Messaging, Event Hub)

· Strong analytical & problem-solving skills with the ability to think critically & creatively about complex business problems

· Ability to work independently & collaboratively and to manage multiple projects & priorities simultaneously

· Strong communication skills

· Strategic thinking and capability building

Roles & Responsibilities

· Role requires unique blend of technical proficiency and practice development to drive business growth and deliver data analytics solutions to business units

· Lead implementation of enterprise data lake, develop data transformation & conversion strategies from multiple source systems, federated Datawarehouses/Data Lakes with data retention & archiving strategies along with regulatory requirements

· Lead development of data pipelines to implement Analytics. AI/ML use cases in real time and batch

· Establish data governance, data security and data cleansing procedures to ensure Data Quality & Compliance

· Mange cloud consumption cost through optimization of services

· Hire, grow and nurture high-performing team of data engineers and data scientists

Education Qualification

· Master’s/bachelor’s degree in engineering/management from premier institutes

· Professional certification in Azure, GCP, Databricks, AI/ML