Job Seekers, Please send resumes to resumes@hireitpeople.com Must Have Skills: Hadoop Hive Spark 5+ years of Hadoop, ETL, Hive, python experience, SQL experience Nice to Have: Python

Detailed Job Description:

The AI technology team is looking for a highly motivated and experienced Data Engineer. The right candidate will have expert level experience in SQL, Data Warehousing technologies, Talend or other ETL framework and Hadoop Big Data environment. The role will require the candidate to be the technical lead to build fault tolerant data ingestion pipelines.

Validate data engineering requirements and work with Data Scientists in requirements gathering/validation phase. Create visualization charts to depict the various table joins, join methodology and access methods Develop modularized feature/transformation code using Spark - SQL, python on Jupyter notebook. Develop applications using agile methodologies and Act as agile SME for the team Tune pyspark code, SQL code, python data structures etc to ensure optimal performance. Perform unit testing Support feature engineering refresh jobs deployment to production Interact with various teams to conduct daily work Basic understanding of API development using Python Basic understanding of Machine Learning in the context of feature engineering.

Minimum years of experience required: 5+ years