OverviewFreelance Data Engineer opportunity in Belgium. 12 months contract + extension.
Role & Responsibilities
- Support in the creation of products to optimize usage of our
- storage, processing and shipping activities at the ***
- Design and Develop scalable and sustainable data pipelines
- Design and implement data products in collaboration with
- Product Owner, Data scientist and Business partners using
- Agile/Scrum methodology.
- Translate product backlog items into engineering design and logical unit’s of work
- Build utilities, libraries and contribute/build frameworks to enable data flow patterns
- Build and incorporate automated unit test and participate in test-driven development
- Strong skilled with cloud environment (AWS), with Hadoop and
- related processing frameworks (Spark, Hive, Python)
- Implement complex automated routines using workflow
- orchestration tools like Apache Airflow
- Experience with source code control tools like Github
- Experience with automation tools like Jenkins, Terraform.
- Strong experience with tools like Snowflake
- Experience with practices like Continuous Integration ,
- Continuous Deployment and Automated testing
- Experience of working in a fast-paced, result oriented environment.
- Ability to co-develop data and facilitate machine learning pipeline
- Understanding of Core Data Architecture Concepts.