Assembling complex data sets that meet both functional and non-functional business requirements. Identifying and implementing internal process improvements, including redesigning infrastructure for scalability, optimizing data delivery, and automating manual processes. Building infrastructure for optimal extraction, transformation, and loading (ETL) of data from various data sources using GCP/Azure and SQL technologies. Developing analytical tools to leverage the data pipeline and provide actionable insights on key business performance metrics, including operational efficiency and customer acquisition. Collaborating with stakeholders, including data, design, product, and executive teams, to support their data infrastructure needs and address data-related technical issues. Overseeing the integration of new technologies and initiatives into data standards and structures. Ensuring high standards for data warehouse design, scalability, and data pipeline optimization.

Key Responsibilities:

20%: Requirements gathering and design 60%: Coding & testing 10%: Reviewing code done by developers, analyzing, and troubleshooting issues 10%: Deployments and release planning

Required Skills & Qualifications:

Education: Bachelor's degree in Computer Science, Computer Engineering, or a related field. A Master's degree is a plus. Experience: 6+ years of experience in Data Warehouse and Hadoop/Big Data technologies. 3+ years of strategic data planning, governance, and standard procedures. 4+ years of hands-on experience with Scala, Spark, PySpark, Python, and SQL. 3+ years of experience in an Agile environment. Strong background in data pipeline creation, optimization, and troubleshooting. Experience with cloud platforms such as GCP or Azure, data migration, ETL processes, and data validation. Technical Skills: Languages: Scala, Spark, PySpark, Python, SQL. Big Data: Hadoop, Hive, Pig, MapReduce. Tools: Apache Hadoop, Airflow, Kubernetes, Containers. Data Technologies: Data Warehouse Design, ETL, Data Analytics, Data Mining, Data Cleansing. Cloud Platforms: GCP, Azure. Additional Skills: Experience with data pipeline tools like Airflow, and understanding of Hadoop log files and multiple data processing engines is a plus.

Desirable Skills:

Experience in Data Analytics, Machine Learning, and optimization. Understanding of web-based technologies like Java, ReactJS, Node.js. Knowledge of managing big data workloads and containerized environments. Experience in analyzing large datasets and optimizing data workflows.

Core Competencies:

Strong problem-solving skills and the ability to think critically and strategically. Proactive sharing of knowledge, accomplishments, and lessons learned across teams. Ability to work effectively in a fast-paced environment. Excellent verbal and written communication skills. Leadership skills in guiding teams and driving technical solutions. Experience in deploying and releasing software in large organizations.