Job Description
Vattenfall is one of Europe's largest producers and retailers of electricity and heat. Our main markets are Sweden, Germany, the Netherlands, Denmark, and the UK. The Vattenfall Group has approximately 21 000 employees. We have been electrifying industries, powering homes and transforming life through innovation for more than 100 years. We now want to make fossil free living possible, and we are driving the transition to a sustainable energy system.
The Business Unit Hydro Power Plants Germany operates and maintains the largest German storage portfolio and is active in all short-term power markets. The fleet of pump storage plants is an integral part of the energy transition, offering both flexibility and reliability to the system.
We, the Data Innovation team, develop and operate our own Databricks centered platform to provide actionable insights, aiming to transform our portfolio and support fossil freedom. By leveraging data, we identify opportunities to enhance efficiency, reduce emissions, and align strategies with organizational goals. Through continuous learning and new analytical techniques, we drive innovation, ensuring informed decision-making and tangible results.
As Senior Data Engineer you drive the creation, management and enhancement of robust data pipelines and infrastructure to underpin data analysis and machine learning initiatives. You ensure the quality, dependability, and safety of data at every stage of its lifecycle, fostering a resilient and efficient data ecosystem.
Your tasks and responsibilities:
- Develop, deploy and improve scalable data pipelines for efficient processing.
- Automate processes for data quality assurance and error detection.
- Implement data validation and cleaning procedures for consistent data quality.
- Create and maintain data models to support analysis and reporting.
- Manage access and security policies to safeguard sensitive data.
- Conduct data migrations and transformations to meet evolving business needs.
- Participate in code reviews and quality assessments to uphold data processing standards.
- Document data architecture and processes for ongoing maintenance and development.
You like to make a difference. You think that complex challenges are the perfect target for your creativity. You have a strong drive to ask questions. You think that the status quo is optional. You thrive working in a responsible role and being held accountable for the results. You embrace helping others in the team and contribute to a greater goal.
Your Skills:
- Ph.D. or Master's degree in Computer Science, Data Engineering, Physics, Mathematics, or related field.
- Professional experience using Python, Spark, SQL and NoSQL databases for data engineering tasks.
- Experience with performance benchmarking and optimization in distributed computing with Spark.
- Consideration of algorithmic complexity in designing new transformations, understanding numerical, non-numerical, and complex data types and their storage requirements.
- Demonstrated expertise with Databricks, preferably with Databricks certification (Solution Architect Champion or alternatively Data Engineering Associate & Professional).
- Successful track record in developing and implementing data pipelines for energy market and asset optimization data.
- Understanding of data security and privacy aspects in data engineering.
- Effective use of object-oriented programming and inheritance, especially in Python, with a commitment to best practices and linting.
- Proficiency in collaborative work using Git in a DevOps environment as well as CI/CD. Minimum 2 years of experience in collaborative software projects.
- Agile work experience in the Scrum framework, demonstrating responsibility and self-organization.
- Effective collaboration with cross-functional teams to meet complex data requirements.
- Strong analytical and problem-solving skills and commitment to continuous learning.
- Excellent communication skills in English and preferably German, capable of conveying technical concepts to diverse stakeholders.
We strongly encourage you to apply even if you only fulfill 70% of the criteria. We are looking for inspiring colleagues that share our vision and appreciate a trust-based work environment. Please be honest about your skillset. Learning will be an integral part of the role so don't be afraid to be transparent about things you can improve on.
Additional Information
Location:
flexible (Berlin, Hamburg or Hohenwarte)
competitive remuneration, a challenging and international work environment, and the possibility to contribute to real change. We offer attractive employment conditions including 37h/week, 30 days of vacation, state-of-the-art office buildings, up to 20 days of working from abroad, generous home-office policies and a company pension scheme. Employer subsidized public transport, JobBike and a budget for fitness and health.
View More