We are seeking a Data Engineering Lead with 5-9 years of experience to join our dynamic team. In this role, you will collaborate closely with project leads and clients to design and develop scalable data processing systems. You will be responsible for gathering requirements, creating comprehensive end-to-end solutions, and overseeing projects from inception to production. A key aspect of this role involves mentoring team members and converting complex business needs into innovative technical solutions using the latest technologies.

Key Responsibilities:

Design and develop scalable data processing systems. Gather and analyze requirements to create effective end-to-end solutions. Manage projects from conception through to production. Mentor and guide team members to enhance their technical skills. Translate complex business requirements into robust technical solutions. Utilize cutting-edge technologies to solve complex problems.

Technical Skills:

Expertise in Python and PySpark: Proficient in Spark architecture and performance tuning. SQL Proficiency: Strong skills in writing optimized SQL queries with performance tuning. AWS Data Engineering Services: In-depth experience with Glue, Lambda, API Gateway, and S3. Business to Technical Translation: Ability to convert complex business requirements into technical solutions. Unix Knowledge: Good to have working knowledge of Unix. CloudFormation and CI/CD: Hands-on experience with CloudFormation and CI/CD pipelines is desirable.

Soft Skills:

Problem-Solving: Strong analytical and problem-solving capabilities. Communication: Excellent communication skills to convey technical concepts clearly. Adaptability: Comfortable working in a fast-paced environment and managing multiple priorities. Collaboration: Effective collaboration with cross-functional teams. Quality Focus: Proven ability to deliver high-quality results within project timelines.