Data Engineer
Biuro
Katowice
Numer oferty: 3092355
Skills and Experience
- 3+ years of experience in data engineering or a similar role.
- At minimum a bachelor’s degree in computer science, information technology, or a related field.
- Familiarity with the Microsoft Azure Platform (Data Factory, ADLS, etc.).
- Working knowledge of Azure Databricks and Delta Lake.
- Experience with relational databases (e.g., SQL Server).
- Proficiency in coding (e.g., PySpark and SQL).
- Experience with version control (Git) and CI/CD.
- Ability to translate business requirements to code.
As a Data Engineer at LIPTON Teas and Infusions, you will support the development of ETL processes, collaborate with BI developers and data scientists on data & analytics solutions, and manage data ingestion and processing within our data lake. You'll be responsible for ensuring data quality, implementing best practices, and leveraging ways of working like version control and CI/CD. With a strong foundation in data engineering and familiarity with the Microsoft Azure and Databricks platforms, you'll play a key role in enhancing our data infrastructure and optimizing cloud costs. If you have a passion for data engineering and want to accelerate your development curve, we would love to hear from you.
Key Responsibilities
- Supporting the development of data infrastructure to extract, transform, and load (ETL/ELT) data.
- Supporting BI developers and data scientists to build data & analytics solutions.
- Responsible for the ingestion and processing of (medallion architecture) data in the data lake.
- Responsible for periodic data refreshes through data pipelines (scheduled jobs).
- Implementing data management practices to improve data quality and meta data in the data lake.
- Leveraging software engineering best practices such as version control (Git) and CI/CD (DevOps).
- Continuously strengthening the data foundation through updating the data infrastructure.
- Monitoring the cost associated with the cloud environment, data processing, and data computation.
