Job Description
The Data Science department uses state-of-the-art machine learning and optimization techniques to continuously improve business processes at NEW YORKER. We work in agile project teams on a variety of use cases, including image recognition, order optimization, pricing, distribution and logistics. We are a very ambitious and international team and are proud of our high standards.
- Design, implement and optimize effective data processing architectures in Python
- Extract, transform, analyze, clean and model data from different sources
- Implement and operate maintainable and reliable data pipelines using PySpark and Airflow with a strong focus on data quality
- Strive to understand how the data is actually generated and what impact this has on data quality and our use cases through proactive communication with other departments
- Operationalize machine learning pipelines together with a team of highly motivated data scientists
- Architect and develop continuous integration & deployment processes
- Operate and optimize Hadoop (Cloudera) clusters with support of the DevOps and Infrastructure teams
- Mentor more junior team members through insightful code reviews and pair programming sessions
- Master's degree in Computer Science or equivalent
Three or more years of working experience as a data engineer or similar position - Extensive experience with Spark, Hadoop, SQL & relational databases, data models and ETL pipelines
- Ability to write efficient, well-tested code with a keen eye on scalability and maintainability in Python; Airflow and Scala experience is a plus
- Experience with Docker, Kubernetes and usage of continuous integration & deployment is a plus
- Good Linux knowledge
- You are a team player and strong communicator with a hands-on mentality
- Good spoken and written German is a plus, fluent English is a must-have