Sr. Data Engineer Pyspark/Python
Insight Global
Job Description
Provide scoping, estimating, planning, design, development, and support services to a project.
Identify and develop the Technical detail design document.
Work with developers and business areas to design, configure, deploy and maintain custom ETL Infrastructure to support project initiatives.
Design and develop data/batch processing, data manipulation, data mining, and data extraction/transformation/loading (ETL Pipelines) into large data domains.
Document and present solution alternatives to clients, which support business processes and business objectives.
Work with business analysts to understand and prioritize user requirements
Design, development, test, and implement application code
Follow proper software development lifecycle processes and standards
Quality Analysis of the products, responsible for the Defect tracking and Classification
Track progress and intervene as needed to eliminate barriers and ensure delivery.
Resolve or escalate problems and manage risk for both development and production support.
Maintain deep knowledge and awareness of technical & industry best practices and trends, especially in technology & methodologies.
We are a company committed to creating diverse and inclusive environments where people can bring their full, authentic selves to work every day. We are an equal opportunity/affirmative action employer that believes everyone matters. Qualified candidates will receive consideration for employment regardless of their race, color, ethnicity, religion, sex (including pregnancy), sexual orientation, gender identity and expression, marital status, national origin, ancestry, genetic factors, age, disability, protected veteran status, military or uniformed service member status, or any other status or characteristic protected by applicable laws, regulations, and ordinances. If you need assistance and/or a reasonable accommodation due to a disability during the application or recruiting process, please send a request to HR@insightglobal.com .
To learn more about how we collect, keep, and process your private information, please review Insight Global's Workforce Privacy Policy: https://insightglobal.com/workforce-privacy-policy/ .
Skills and Requirements
At least 6+ years of Developer experience specifically focused on Data Engineering
Strong Hands-on experience in Data Engineering development using Python and Pyspark as an ETL tool
Hands-on experience in AWS services like Glue, RDS, S3, Step functions, Event Bridge, Lambda, MSK (Kafka), EKS etc.
Hands-on experience in Databases like Postgres, SQL Server, Oracle, Sybase
Hands-on experience with SQL database programming, SQL performance tuning, relational model analysis, queries, stored procedures, views, functions and triggers
Strong technical experience in Design (Mapping specifications, HLD, LLD), Development (Coding, Unit testing).
Good knowledge in CI/CD DevOps process and tools like Bitbucket, GitHub, Jenkins
Strong foundation and experience with data modeling, data warehousing, data mining, data analysis and data profiling.
Strong experience with Agile/SCRUM methodology
Good communication and inter-personal skills At least 6+ years of Developer experience specifically focused on Data Engineering
Strong Hands-on experience in Data Engineering development using Python and Pyspark as an ETL tool
Hands-on experience in AWS services like Glue, RDS, S3, Step functions, Event Bridge, Lambda, MSK (Kafka), EKS etc.
Hands-on experience in Databases like Postgres, SQL Server, Oracle, Sybase
Hands-on experience with SQL database programming, SQL performance tuning, relational model analysis, queries, stored procedures, views, functions and triggers
Strong technical experience in Design (Mapping specifications, HLD, LLD), Development (Coding, Unit testing).
Good knowledge in CI/CD DevOps process and tools like Bitbucket, GitHub, Jenkins
Strong foundation and experience with data modeling, data warehousing, data mining, data analysis and data profiling.
Strong experience with Agile/SCRUM methodology
Good communication and inter-personal skills null
We are a company committed to creating diverse and inclusive environments where people can bring their full, authentic selves to work every day. We are an equal employment opportunity/affirmative action employer that believes everyone matters. Qualified candidates will receive consideration for employment without regard to race, color, ethnicity, religion,sex (including pregnancy), sexual orientation, gender identity and expression, marital status, national origin, ancestry, genetic factors, age, disability, protected veteran status, military oruniformed service member status, or any other status or characteristic protected by applicable laws, regulations, andordinances. If you need assistance and/or a reasonable accommodation due to a disability during the application or the recruiting process, please send a request to HR@insightglobal.com.
Confirm your E-mail: Send Email
All Jobs from Insight Global