Data Engineer

We are looking for experienced Cloudera Data Consultants / Data Engineers to support a large-scale Enterprise Data Modernization (EDM) program within the banking domain. The role involves migrating legacy IBM Netezza and Hadoop/Cloudera workloads into a modern CDP Private Cloud Base Lakehouse environment.

Key Responsibilities

Execute migration of legacy Netezza and Cloudera data workloads to CDP

Re-engineer DataStage ETL jobs into Spark-native pipelines

Build scalable ETL/ELT pipelines using Spark 3 on YARN

Develop and manage Apache NiFi ingestion flows and Airflow DAGs

Implement CDC and incremental data loading strategies

Modernize data storage using Apache Iceberg and Kudu

Perform source-to-target reconciliation, row count, and hash total validations

Implement Great Expectations (GX) data quality checks

Ensure all pipelines comply with Apache Ranger security standards

Support parallel runs, cutover, and hypercare activities

Required Skills \& Experience

✅ Minimum 3+ years of hands-on experience in Data Engineering

✅ Strong expertise in:

CDP Private Cloud Base, HDFS, YARN, Hive

Spark, NiFi, Airflow

Python or Scala

Advanced SQL

Apache Iceberg, Kudu

Linux Shell Scripting

IBM DataStage / Talend (legacy migration knowledge preferred)

Preferred Experience

✔ Experience with Netezza or Hadoop modernization projects

✔ Banking or financial services domain exposure

✔ Knowledge of Customer, Account, and Relationship data domains

✔ Experience with Great Expectations, Ranger, Atlas, and OpenMetadata

Education \& Certifications

Bachelor’s degree in Computer Science, IT, or related field

Cloudera Certified Associate (CCA) or equivalent certification preferred

Pay: RM4,259.27 - RM12,984.25 per month

Benefits:

• Cell phone reimbursement
• Dental insurance
• Flexible schedule
• Maternity leave
• Parental leave

Experience:

• Data Engineer: 4 years (Preferred)

Work Location: In person