The Data Engineer is responsible for helping develop and maintain the company’s data platforms. 

Principal Duties & Responsibilities:

Data collection

Data transformation and cleanup

Data importation into analysis and reporting databases

Daily monitoring of data systems

Analysis of data to find new insights

Required Technical Skills and Abilities

Python

SQL

BASH Scripting

Recommended Technical Skills and Abilities:

AWS knowledge

AWS Redshift knowledge

Knowledge about Docker containers

Knowledge about NoSQL Systems

Javascript knowledge

HTML/CSS

Education/Experience Needed: 

Bachelor's degree in Computer Science, Engineering, Mathematics, Science, or a related field preferred.

Will consider work or internship experience that involves daily usage of Python and SQL

Soft Skills:

Takes initiative to solve problems

Able to multi-task and manage multiple projects

Seeks feedback

Excellent interpersonal, written, and verbal communication skills

Able to self-manage

Quality oriented

Preferred Qualifications      

·   Familiarity with Cloud Service usage and orchestration (AWS | GCP | Azure)

·   Hands-on experience with Docker

·   Experience with dashboard and monitoring tools (Grafana, Quicksight, etc)

·   Experience with Airflow

·   Proficiency with Git

·   Understanding of NoSQL and its use cases

·   Expertise in error handling, input/output monitoring and alerting

 ·   Strong experience with Python (pandas, numpy, scikit-learn)   

·   Proficient in SQL (preferably experience with postgres)

·   Experience with Linux Operating System and Bash scripting

·   Understanding of cost analysis (forecasting and tracking)    

 Responsibilities

·   Assemble pipelines for large, multi-sourced sets of data to optimize creation of machine learning models, dashboards, and other business automations.

·   Identifying, designing and implementing internal process improvements including re-designing infrastructure for greater scalability, cost optimization, and compatibility with evolving technologies.

·   Design, test and develop software and workflows across varying environments including AWS, GCP, Azure, Docker, etc.

·   Document aspects of new and updated systems and software to serve as reference for future upgrades and maintenance.

·   Aptly communicate technical challenges and possible solutions to best make collaborative decisions as a team.

·   Research open-source and emerging technologies to solve problems and improve existing systems.