Free cookie consent management tool by TermsFeed Generator Data Engineer (Apache Flink & Kafka / Confluent Platform)
Image
Image

Data Engineer (Apache Flink & Kafka / Confluent Platform)

Warsaw

Posted 8 hours ago

Work Type

Contract

Salary/Rate

zł 130.00-160.00 per hour

Remote Work

Yes

IR35 Status

Not Applicable

We are seeking an experienced consultant to join a high-priority initiative in the logistics sector focused on building and extending a real-time data streaming platform. The project leverages modern event-driven architecture based on Apache Flink and Kafka/Confluent technologies.

The engagement is initially planned for 3 months, with a strong likelihood of extension for an additional 6 months depending on delivery outcomes and project roadmap.

Responsibilities:

  • Design and implement real-time streaming solutions using Apache Flink (including Flink SQL and Stateful Stream Processing)
  • Develop and maintain data pipelines on Kafka / Confluent Platform
  • Build and integrate Kafka Connect SMTs, including XML-to-Avro transformation logic
  • Implement and maintain Schema Registry integration (Avro-based schemas)
  • Develop Flink-based filtering and projection frameworks for streaming data
  • Contribute to consumer self-service workflows using YAML-based configuration
  • Support deployment and orchestration of streaming components on Kubernetes (including Flink Kubernetes Operator)
  • Collaborate in designing and implementing CI/CD and GitOps-based delivery pipelines
  • Actively participate in an AI-assisted / agentic engineering delivery model

Requirements:

  • Strong commercial experience with Apache Flink, including:
    • Flink SQL
    • Stateful stream processing
    • Flink Kubernetes Operator
  • Proven experience with Kafka / Confluent Platform ecosystems
  • Hands-on knowledge of Avro and Schema Registry
  • Practical experience with Kubernetes in production environments
  • Experience in building or maintaining streaming data architectures
  • Strong understanding of distributed systems and event-driven architecture
  • Ability to work independently in a remote delivery model

Nice to have:

  • Experience with AsyncAPI
  • Development of Kafka Connect SMT plugins
  • Experience with XML-to-Avro transformations
  • Knowledge of GitOps and CI/CD pipelines (e.g., ArgoCD, Jenkins, GitHub Actions)
  • Exposure to agentic AI engineering / AI-assisted software delivery practices
Aplikuj