Senior Data Engineer

Job Summary

• Design, develop, and optimize enterprise-scale data platforms on Microsoft Azure Cloud.
• Build scalable and secure ETL/ELT data pipelines to support Data Lake, analytics, and AI/ML initiatives.
• Ensure high-quality, reliable, and cost-efficient data architecture while driving continuous improvement and technical best practices.
• Collaborate with cross-functional teams to deliver robust data engineering solutions and mentor junior engineers.

Key Responsibilities

• Design and develop scalable data pipelines and cloud-based data architecture using Azure Data Factory (ADF), Databricks, Microsoft Fabric, and SSIS.
• Build ETL/ELT processes to integrate data from multiple sources such as SAP, ERP systems, APIs, and Excel into enterprise Data Lake and databases.
• Develop and optimize SQL, Python, and PySpark scripts for large-scale data processing.
• Monitor daily data pipeline operations, troubleshoot complex technical issues, and perform root cause analysis.
• Ensure data quality, accuracy, security, and optimize database performance.
• Prepare technical documentation, operation manuals, and share best practices with the team.
• Support continuous improvement by adopting emerging technologies and mentoring junior data engineers.

Qualifications

• Bachelor's degree in Computer Science, Information Technology, Computer Engineering, or related fields.
• Minimum 5 years of experience in Data Engineering or related roles.
• Strong experience with Microsoft Azure, Azure Data Factory (ADF), Databricks, Microsoft Fabric, and Data Lake.
• Strong proficiency in SQL Server, T-SQL, Python, and PySpark.
• Experience in designing Data Warehouse, ETL/ELT processes, and integrating data from ERP/SAP systems and APIs.
• Knowledge of Cloud Architecture, CI/CD, DevOps practices, and Database Performance Tuning.
• Strong analytical and problem-solving skills with the ability to perform root cause analysis.
• Passion for learning new technologies and mentoring team members.