Senior Data Engineer // Picsart at Picsart

Posted on: 10/29/2021

Location: Berlin, Germany (ON-SITE)

full time

Crunchbase | Original Source

Tags: scala spark apache nosql sql ansible terraform aws kafka cloudformation kubernetes

Picsart is looking for a highly motivated Senior Software Engineer with deep experience in scalable data processing systems. If you are passionate to push the limits of distributed computing frameworks to get every bit of performance out of them, welcome to apply. **Please submit your resume and cover letter in English.** **Required Qualifications:** * BS/MA degree in computer science or related field; * Working experience with distributed systems and Big Data; * Demonstrated working experience with Scala/Java; * Experience with database design (SQL/NoSQL); * Good understanding of cloud infrastructure platforms, especially AWS, as well as infrastructure as Code tools (e.g. CloudFormation, Terraform, Ansible), containerization (Kubernetes); * Strong Experience with technologies and systems like AWS EMR, Presto, Spark, Hive/Glue, Kafka, Airflow; * Good knowledge of Apache Spark internals (Catalyst, Tungsten and related query engine details); * Good understanding of file formats like Parquet, ORC, various data access optimisation strategies; * Familiar with data modeling (Dimensional modelling, Data vault modeling, Relational modeling); * Good communication and knowledge sharing skills; * At least 5 years of relevant experience; * Strong working English skills; * Self-motivated, quick learner and innovative person. **Responsibilities:** * Provide in-depth analysis and insights about user behavior using Apache Spark (execution) and Databricks Delta (storage); * Design and build a Data Lake solution with the focus on performance and quality; * Work with both simple statistical analysis and much more complex funnel and cohort analysis; * Work both with data science and data engineering teams to empower data-driven business decisions at PicsArt; * Focus on optimizing various performance-related aspects of our applications and delivering truly production-ready solutions, automate the path from development to production; * Constantly improve the delivery quality and performance of the team; * Mentor fellow team members on algorithms, data structures, design patterns, and best practices; * Contribute towards better data governance and observability. **Please, submit your resume in English.**