San Diego, CA, 92108, USA
9 days ago
Lead Data Engineer
Job Description The Lead Data Engineer role for a major medical technology client is a dynamic position focused on data transformation and building data products. The candidate will spend 80% of their time on development, primarily writing and optimizing SQL code, and the remaining 20% on architectural and leadership activities. Key responsibilities include developing data pipelines using Python and PySpark, working exclusively within the AWS ecosystem (EMR, Step Functions, S3), and collaborating with architects on high-level AWS component questions. The role requires at least 10 years of experience as a Data Engineer, with a strong emphasis on batch processing and large-scale data platforms. Leadership skills are essential for guiding and mentoring the team, adopting new tools and technologies, and conducting learning sessions. The ideal candidate will be a senior-level professional with a deep understanding of AWS, capable of building and curating data pipelines to transform raw data into valuable insights. The candidate should also be a change agent, capable of thinking outside the box and inspiring others with new technological ideas. We are a company committed to creating diverse and inclusive environments where people can bring their full, authentic selves to work every day. We are an equal opportunity/affirmative action employer that believes everyone matters. Qualified candidates will receive consideration for employment regardless of their race, color, ethnicity, religion, sex (including pregnancy), sexual orientation, gender identity and expression, marital status, national origin, ancestry, genetic factors, age, disability, protected veteran status, military or uniformed service member status, or any other status or characteristic protected by applicable laws, regulations, and ordinances. If you need assistance and/or a reasonable accommodation due to a disability during the application or recruiting process, please send a request to HR@insightglobal.com .     To learn more about how we collect, keep, and process your private information, please review Insight Global's Workforce Privacy Policy: https://insightglobal.com/workforce-privacy-policy/ . Skills and Requirements - 10+ years of experience as a Data Engineer - Proven experience as a Senior, Principal, or Lead Data Engineer - having participated in lead activities or mentorship. (lead job title does not matter) - Solid understanding of software engineering principles/best practices. - Experience building and curating data pipelines across large-scale, distributed systems using Python and PySpark - AWS subject matter expert with experience implementing data pipelines with AWS and running them in EMR - Experience working on table formats using HUDI and Iceberg, and storing data in S3 - Sql Programing experience - Bachelor's or Master's degree in Computer Science, Information Systems, or a related field - Healthcare industry experience null We are a company committed to creating diverse and inclusive environments where people can bring their full, authentic selves to work every day. We are an equal employment opportunity/affirmative action employer that believes everyone matters. Qualified candidates will receive consideration for employment without regard to race, color, ethnicity, religion,sex (including pregnancy), sexual orientation, gender identity and expression, marital status, national origin, ancestry, genetic factors, age, disability, protected veteran status, military oruniformed service member status, or any other status or characteristic protected by applicable laws, regulations, andordinances. If you need assistance and/or a reasonable accommodation due to a disability during the application or the recruiting process, please send a request to HR@insightglobal.com.
Confirm your E-mail: Send Email