We are looking for an experienced Senior Data Engineer / Data Platform Engineer to join an international Data team within the e-commerce and digital marketplace industry.

The team is responsible for building next-generation, cloud-based data solutions capable of ingesting and processing petabytes of data across data lakes and data warehouses.

Responsibilities:

• Design and develop high-volume batch and streaming data ingestion pipelines across AWS and GCP
• Build and launch next-generation data ingestion and data curation platforms
• Participate in system and data architecture discussions
• Design scalable and high-performance distributed data solutions
• Troubleshoot and resolve issues related to ETL, data lineage, data quality, backfills, and data pipelines
• Design, implement, and support production services with strict SLAs
• Collaborate with Software Engineers, Data Engineers, ML Engineers, Data Analysts, and other stakeholders
• Provide technical guidance and mentor junior engineers
• Contribute to best practices across software development, data engineering, testing, and CI/CD
• Work with logging, monitoring, metrics, and alerting solutions to ensure platform reliability

Requirements:

• Solid experience in Data Engineering, Software Engineering, Distributed Systems, or a related field.
• Strong programming skills in Python, Java, or Scala
• Strong knowledge of SQL and experience with SQL and NoSQL databases such as BigQuery, Teradata, MySQL, PostgreSQL, or Cassandra
• Experience with Big Data technologies, including Hadoop, Hive, and Apache Spark
• Hands-on experience with Airflow or another orchestration tool
• Strong understanding of ETL, data lineage, data quality, backfills, and data pipeline troubleshooting
• Experience developing both batch and streaming data pipelines
• Experience across both software development and data engineering
• Experience with AWS or GCP, particularly data processing at scale
• Experience designing and supporting production services with strict SLAs
• Experience with distributed applications and monitoring/logging tools such as Elasticsearch and Wavefront
• Good understanding of testing methodologies, CI/CD, and software engineering best practices
• Excellent written and verbal communication skills

Nice to have:

• Strong expertise in Python
• Experience with Google Data Streams and Google Dataproc
• Experience with Change Data Capture (CDC) technologies
• Experience with modern data warehouse technologies such as Delta Lake