Rotterdam, the Netherlands - local conditions apply
(COMPANY NAME) Food Solutions has traditionally been using sales representatives to engage with its customers. In recent years, the role of digital communication channels has significantly increased; using the right combination of both off and online communication channels has become essential in creating the perfect way for customers to interact with (COMPANY NAME) Food Solutions. For the most effective digital interaction, a good data infrastructure at the back end is crucial. To make this happen we are building a leading-edge cloud based Big Data ready data integration platform that collects, deduplicates and processes all customer data in the organization across more than 50 countries. The data is then provided as input for models that predict customer behavior and distributed across different databases and CRM/Salesforce systems. The data platform is critical in driving digital transformation and data-driven marketing across the organization.
To manage and extend this complex data platform we need a data engineer who loves to make sure everything works as expected and participates in the Scrum team to develop and deploy new
WHAT WILL YOUR MAIN RESPONSIBILITIES BE?
These are the tasks of the data engineer within the scrum team:
* Develop new data pipelines and innovations required by the business on the data platform using Scala / Spark and Azure cloud
* Monitor if the orchestrator Azure Data Factory runs all data pipelines Correctly
* Resolve issues and any type of bugs in Azure AKS, Azure Databricks and Azure Data Factory
* Ensure IT security and data privacy controls are in place across the data platform
* Develop / Extend API's to and from the data platform by means of Java/Spring
* Convert Databricks PySpark notebooks with data science code to production code in Scala
* Contribute to the definition of the architecture of the data lake using Microsoft Azure Cloud technologies
* Experience with working in Agile / Scrum
* Strong team player capable to work harmoniously with a team locally and remotely
* Exceptional communication skills, capable to express complex topics in a concise and abstract manner
* Experience working extensively in data environment
* Exceptional Problem solving and analytical skills.
* Experience in engineering large-scale systems in a product environment
* Ability to write, analyze, and debug SQL queries
* Knowledge of database modeling, ETL and design in a Data warehousing context
* Knowledge of NoSQL stores is a plus
* Ideal candidates will have a deep understanding of technical and functional designs for Databases, Data Warehousing, Reporting, and Data Mining areas
* Passion for about continuous learning, experimenting, applying and contributing towards cutting edge open source Big Data technologies and software paradigms
Technical skills required:The data engineer needs to be proficient in:
* Airflow / Azkaban / Azure data factory
* Bug fixing
* CI / CD concepts including Agile/Scrum
* Command line / Bash
* Java Spring Boot
* Java Streams
* Spark Datasets
* Communicating in English
EXCITED ABOUT THE ROLE?
To apply, you must do so online. Please do not forget to upload your CV and a motivation letter. Your application will be reviewed against our requirements and we will be in touch shortly after the closing date to provide you with an update on the status of your application.
Please note this is a Direct Search led by (COMPANY NAME). Applications from agencies will not be accepted, nor will fees be paid for unsolicited CVs