Data Engineer (Data Lakehouse architectures)

Design, develop, and maintain data pipelines to support business operations. Process and transform data to meet both functional and non-functional business

. Identify, design, and implement process improvements to automate manual tasks and optimize data delivery. Design and develop data warehouse (DW) solutions, including data modeling and ETL processes. Develop real-time data ingestion and streaming solutions. Design and implement cloud-based data platforms and architectures on AWS or GCP. Design and develop container-based services using Kubernetes (Amazon EKS/Google GKE). Design, develop, and maintain MLOps platforms and machine learning deployment pipelines.. JOB

REQUIREMENTS: No spoken English proficiency required. 3–5 years of hands-on experience designing and building large-scale data pipelines (processing over 100GB/day) in enterprise environments. Proven experience with AWS Data Services, including Amazon S3, AWS Glue (Crawlers & Jobs), Amazon EMR, Amazon MSK (Kafka), AWS Step Functions, AWS Lake Formation, and Amazon Redshift Spectrum. Strong expertise in Apache Spark (PySpark or Scala) for developing batch and streaming ETL/ELT workloads on Amazon EMR or AWS Glue. Solid understanding of and hands-on experience implementing Data Lakehouse architectures using open table formats such as Apache Iceberg or Delta Lake, including schema evolution, time travel, and ACID transactions. Strong Python programming skills with experience developing AWS Glue Jobs, AWS Lambda functions, pipeline automation, and unit testing for data transformations. Experience building real-time streaming pipelines using Apache Kafka or Amazon MSK, with a solid understanding of Kafka Consumer Groups, offset management, and Schema Registry. Hands-on experience designing and implementing data pipeline orchestration using Apache Airflow or AWS Step Functions, including DAG design, dependency management, retry mechanisms, and SLA monitoring. Experience integrating data from multiple sources, including relational databases (Oracle, MySQL, PostgreSQL), NoSQL databases (MongoDB, Redis), SaaS APIs (REST/SOAP), and SFTP-based file ingestion. Good understanding of Data Quality concepts, with experience implementing data quality checks (completeness, uniqueness, timeliness) using Oracle EDQ, Great Expectations, or Deequ. Familiarity with Data Governance practices, including metadata management, data lineage, and data tagging, with hands-on experience using Apache Atlas or AWS Glue Data Catalog. Experience with Oracle GoldenGate CDC is a strong advantage. Experience working in Agile/Scrum environments and collaborating closely with Data Architects, Cloud Engineers, DBAs, and Analytics Engineers. Proactive in creating and maintaining technical documentation. Note: Open to level Fresher. Must have: Good at English COMPENSATION &

Flexible Hours

Health Insurance

Employee Award

Training/Certification

Remote work options

Wellness Facilities

Transport Money

Team-building events

Laptop Provided

Insurance

Free Meals

Annual Leave

Career Path