Mandatory Skill: Python/Spark/SQL/RDB/Cosmos/Kafka/ADLS/API/Databricks/ADF/ETL

Location: Bangalore/Hyderabad

Notice Period:  less than 60 Days

Job Description: 

Skills:  Python/Spark/SQL/RDB/Cosmos/Kafka/ADLS/API/Databricks/ADF/ETL

5-8 years of overall experience mainly in the data engineering space building data ingestion and transformation pipelines.

Design and implement data pipelines using best practices and industry-leading tools like Databricks and Azure Data Factory (ADF) Extract, transform, and load large datasets from various sources, ensuring data quality and integrity Utilize Python and Spark to perform complex data manipulations and aggregations Write optimized SQL queries to interact with relational databases and Cosmos DB Integrate data pipelines with APIs and external systems using efficient methods Monitor and maintain data pipelines, ensuring smooth operation and identifying potential issues Collaborate with data scientists, analysts, and engineers to understand data needs and deliver valuable insights 4+ years of Hands-on experience in Azure cloud data implementation. Must have implementation experience of Azure based cloud Data project/program as solution architect. Expert level knowledge on Spark. Extensive hands-on experience working with data using SQL, Python, Java, Scala Strong experience and understanding of very large-scale data architecture, solutioning, and operationalization of data warehouses, data lakes, and analytics platforms. (Both Cloud and On-Premise) Excellent communications skills with the ability to clearly present ideas, concepts, and solutions Bonus points if you have experience on Insurance Domain for data engineering project.