Data Engineer at Harlem Next

Posted on: 03/30/2021

Location: Haarlem (ON-SITE)

full time

Original Source

Tags: mongodb sql datadog python grafana etl gcp terraform mysql kafka kubernetes docker containers

**What we need** We are looking for a data engineer with at least 4 years experience working with ETL pipeline, Data Modeling, Cloud Services, Containers/Orchestration and solid familiarity of the programming languages Python & Java to be able to: * Develop and maintain a data platform in Google Cloud Platform * Set up data integration towards multiple GCP applications / services * Optimise the ETL jobs depending on the data types and sources * Ensure high reliability and quality of our data warehouse * Give inputs and provide support on integration of BI reporting tool(s) * Create and maintain data models in the data warehouse * Set up data, application & infrastructure monitoring * Develop routines to improve data reliability, efficiency and quality **What’s in it for you** You’ll be part of the master data team and will help define and create the key components of the back-end data process, including data warehouse design, ETL development, and data modeling. Since the data team is young and mature, your contribution will have a significant impact on the decision-making process and we expect that you can come up with and implement challenging technical solutions. There's also the opportunity to use really massive amounts of big data (around 450TB) and various data sources including SQL cloud, firestore, mongoDB, etc. With the rapid development of our company, it is important that we have the knowledge to make the right business decisions. Therefore, we made the data realm to be the key focus of business growth and the data team will become its center. Having the right layout of data warehouse and optimised pipelines can provide flexible/scalable data models in the BI reporting tool, which will allow us to accelerate time to market, improve collaboration with clients and ultimately realise our vision of becoming an innovative, comprehensive and valuable platform for customers. **What's in it for us** * Experience with integration from/to REST API, Google Cloud Storage, Google Pub/Sub (or Kafka), BigQuery, mySQL DB, various file formats * Experience with code based ETL tools such as Airflow/Luigi * Experience in Kimball and/or Data Vault modeling * Experience with MPP databases such as BigQuery, Redshift, Snowflake * Experience with Google Cloud Platform services (Google Cloud Storage, Cloud Functions, Dataflows) * Strong experience with SQL, and programming languages Python & Java * Experience with using Kubernetes, Airflow, Docker * Experience with monitoring tools such as Datadog, Stackdriver or Grafana * Experience working in a CI/CD environment **Pre** * Experience with Terraform, Firestore and Maxwell/Debezium knowledge **When it’s a match** Harlem Next not only provides a challenging and inspiring work environment but you also will be rewarded for the effort and energy you bring along! The place where talented and hardworking people can fully realise their potential and even go beyond what they think they can achieve. We are among the top of Google Cloud Services users Worldwide. Working with the required development tools and offering you the opportunity to expand your own knowledge base. * Good salary * Annual Bonus * Pension * Travel expenses * Laptop * Training and Development * Gym membership * Bicycle plan * Office lunch * Team building events