Arbetsbeskrivning
Job description
For our client in Gothenburg, we are looking for a Data Engineer.
The client develops and produces world-class powertrain solutions for a global market.
They´re a pioneering global supplier of propulsion technology, development services and contract manufacturing with manufacturing capabilities in two continents. They create value for customers and consumers by producing high-efficiency, low-emission engines that already power millions of vehicles.
Now, the goal is to re-imagine motion for the future with powertrain technologies that are both sustainable and practical.
There are unlimited possibilities for collaborative people who are ready to make the difference!
Responsibilities
As a Data Engineer, with extensive experience in Databricks, you will be assigned key responsibilities concerning the design, development, and maintenance of our data environments. Your technical prowess will directly contribute to building scalable and robust data pipelines and enhancing our data platform's performance, availability, and scalability. You will be instrumental in optimizing data workflows, ensuring data quality, and integrating data from different sources.
- Design, develop, and maintain scalable data pipelines using Databricks to support data ingestion, transformation, and loading (ETL/ELT) processes.
- Collaborate with and mentor other product teams and engineers to build and maintain the data platform that integrates data from multiple sources.
- Optimize data processing workflows and ensure they adhere to architectural principles, performance and security.
- Implement and enforce data quality checks, monitoring, and alerting systems to ensure the integrity and reliability of the data.
- Leverage Databricks features such as Delta Lake and Databricks Workflows to enhance data pipeline performance and reliability.
- Work with cloud infrastructure teams to ensure the platform’s performance, availability, and scalability within the cloud environment.
Qualifications
Required:
- '5+ years of experience as a Data Engineer or similar role, with at least 2 year of hands-on experience in Databricks.
- Knowledge of IAC, CI/CD pipelines, version control (Git), and DevOps practices in data engineering.
- Proficiency in Databricks and familiarity with key features like Delta Lake, Databricks Jobs, and Databricks SQL.
- Excellent skills in Python for data processing and pipeline development. (Beneficial if experienced in SQL)
- Strong understanding of distributed data processing technologies such as Apache Spark. (Only beneficial if employment)
- Experience in working with big data technologies and tools, including Spark, Kafka, Hadoop, or similar. (Only beneficial if employment)
- Experience of medallion architecture in Databricks. (Only beneficial if employment)
- Experience in working with automation testing and data quality.
Required skills
- Databricks
- Spark
- Kafka
- Cloud Infrastructure
- Python
- Testing Automation
- IAC
- CI/CD
- Data Quality
- ETL/ELT
- Devops
- Delta lake
- Git
- Hadoop
- Apache Spark
- Preferred skills
- SLO
- Github
- Agile
- medallion architecture
- SQL
- Terraform
- Bicep
- Languages
- English
- (Proficient)
Beneficial:
- Experience of Azure infrastructure (especially ADF, Blob Storage & DataLake).
- Experience of agile practices working in a product organization.
- Understanding and experience of data mesh concepts.
- Experience of using Terraform and/or Bicep.
- Experience of working with streaming ETL.
- Understanding or experience of concepts like SLO, GitHub and control planes.
About the company
Randstad
At Randstad, we see the possible in people. With business all over Sweden and in all areas of expertise, we help people find work that feels good, where they get the opportunity to develop and realize their true potential.
With close to 600 000 employees in 38 countries, Randstad is the global leader in the HR services industry. Our mission is to become the world’s most valued working life partner. By combining our passion for people with the power of today’s technology, we support people and organizations in realizing their true potential. We call it Human Forward.