Data Engineer (AWS / PostgreSQL / AWS / AI)

We are looking for a Senior Data Engineer with strong expertise in Python and distributed data processing technologies. The role focuses on designing, building, and maintaining scalable data pipelines and data infrastructure supporting analytical products within the organization.

The objective is to deliver reliable, high-performance data systems by developing robust ETL/ELT processes, optimizing data storage and processing layers, and ensuring high data quality, availability, and scalability across the organization.

Responsibilities:

• Design, build, and maintain scalable ETL/ELT data pipelines for ingestion, transformation, and delivery of data across the organization.
• Develop and optimize distributed data processing workflows in Python for large-scale data transformation and aggregation.
• Design, manage, and optimize PostgreSQL schemas, tables, indexes, and query performance for analytics and reporting use cases.
• Build and maintain Python-based data workflows for orchestration, validation, and reliable cross-environment data delivery.
• Implement monitoring, validation, and observability mechanisms to ensure data quality, timeliness, and completeness.
• Design and manage cloud-based data infrastructure on AWS.
• Collaborate with data analysts and business stakeholders to translate requirements into scalable and maintainable data products.
• Maintain technical documentation covering data pipelines, data models, lineage, and infrastructure components.
• Troubleshoot data pipeline issues, perform root cause analysis, and implement corrective actions.

Requirements:

Experience & Core Skills

• Minimum 3 years of commercial experience in Data Engineering roles.
• Strong hands-on experience with Python for building data pipelines, data validation, and orchestration frameworks.
• Advanced knowledge of PostgreSQL: schema design, indexing strategies, query optimization, and performance tuning.
• Proven experience in ETL/ELT pipeline design and production-grade implementations.
• Practical experience with distributed data processing and storage technologies (e.g., AWS Athena, Apache Spark or similar).

Cloud & Data Platforms

• Strong experience with AWS services: S3, EKS, Glue, Athena.
• Experience with modern data architectures including data lakehouse patterns and ELT approaches.
• Knowledge of data warehousing modeling techniques, including dimensional modeling and reusable transformation patterns.

Engineering Practices

• Strong understanding of Git workflows, code review processes, testing strategies, and CI/CD pipelines.
• Experience with workflow orchestration tools (e.g., AWS Step Functions, Prefect, Dagster or similar).
• Experience with data quality frameworks and data observability solutions.
• Exposure to streaming or near real-time data processing (e.g., Kafka, Spark Streaming, Pub/Sub).

Additional Skills

• Ability to use LLM-based AI agents effectively to improve engineering productivity.
• Strong systems thinking with focus on scalability, reliability, and correctness.
• Ability to balance delivery speed with maintainability, cost efficiency, and data quality.
• Strong written and verbal communication skills in collaboration with both technical and business stakeholders.
• Ability to independently own end-to-end data products, from architecture to production support.