AWS Data & AI Engineer

We are looking for a skilled AWS Data \& AI Engineer responsible for designing, developing, and maintaining scalable data platforms, cloud-based data pipelines, and AI/ML solutions on AWS. The ideal candidate should have strong hands-on experience in AWS data services, data engineering, machine learning workflows, and modern cloud architectures.

The candidate will work closely with data scientists, analysts, and business teams to build reliable data solutions, enable advanced analytics, and deliver AI-driven capabilities.

Key Responsibilities

Data Engineering \& Pipeline Development

• Design, develop, and maintain scalable ETL/ELT data pipelines on AWS.
• Build robust data ingestion workflows from multiple sources including databases, APIs, files, and streaming systems.
• Develop data transformation, validation, cleansing, and automation processes.
• Transform raw data into structured, analytics-ready datasets.
• Ensure high availability, scalability, reliability, and performance of production data pipelines.
• Work with large-scale datasets and optimize data processing performance and cost.

AWS Data Platform Development

• Design and implement cloud-based data architectures using AWS services.
• Build and maintain enterprise-level data lakes and data warehouse solutions.
• Hands-on experience with:
• Amazon S3 for data lake architecture
• AWS Glue for ETL processing and data cataloging
• Amazon Redshift for data warehousing
• Amazon Athena for serverless data querying
• AWS Lambda for serverless data processing
• AWS Step Functions for workflow orchestration
• Amazon Kinesis for real-time data processing
• AWS Lake Formation for data governance

Big Data Processing

• Develop and optimize distributed data processing solutions using Apache Spark and PySpark.
• Work with AWS EMR for large-scale data processing workloads.
• Design solutions supporting both batch and real-time streaming data requirements.
• Optimize Spark jobs and distributed processing workloads for performance and cost efficiency.

AI / Machine Learning Engineering

• Develop and deploy AI/ML solutions using AWS AI and machine learning services.
• Build machine learning pipelines, model deployment workflows, and MLOps processes.
• Support the complete ML lifecycle including:
• Data preparation
• Feature engineering
• Model training
• Model deployment
• Model monitoring
• Experience with:
• Amazon SageMaker
• Amazon Bedrock
• AWS AI services
• Knowledge of Generative AI solutions, Large Language Models (LLMs), and RAG architecture is preferred.

Data Quality, Monitoring \& Optimization

• Implement data quality checks, validation rules, and monitoring processes.
• Monitor production data pipelines using AWS CloudWatch and logging tools.
• Troubleshoot and resolve data pipeline failures and production issues.
• Optimize SQL queries, Spark applications, and AWS resources for better performance.

Security \& Data Governance

• Implement secure data access using AWS IAM roles, policies, and permissions.
• Apply encryption and security best practices for data storage and processing.
• Ensure compliance with data governance and security standards.
• Implement access controls and data protection mechanisms.

Collaboration \& Documentation

• Collaborate with data scientists, BI teams, analysts, and business stakeholders.
• Support reporting, analytics, and AI-driven business use cases.
• Create and maintain technical documentation for data architecture, pipelines, and workflows.
• Participate in Agile/Scrum development processes.

Mandatory Technical Skills

Programming \& Data Engineering

• Strong proficiency in Python.
• Advanced knowledge of SQL including:
• Complex joins
• Window functions
• Query optimization
• Performance tuning
• Strong understanding of:
• ETL/ELT development
• Data pipelines
• Data warehousing
• Data lakes
• Data modeling
• Batch and streaming data processing
• Experience with data modeling concepts:
• Star schema
• Snowflake schema
• Experience integrating data through REST APIs and external systems.

AWS Cloud Services (Mandatory)

Strong hands-on experience with:

• Amazon Web Services (AWS)
• Amazon S3
• AWS Glue
• AWS Lambda
• AWS IAM
• Amazon CloudWatch
• AWS Step Functions
• Amazon Kinesis
• Amazon Redshift
• Amazon Athena
• AWS Lake Formation
• Amazon RDS (MySQL/PostgreSQL)
• Amazon DynamoDB

Big Data Technologies

• Apache Spark
• PySpark
• Amazon EMR
• Understanding of distributed computing concepts.

Database Technologies

Experience with relational and NoSQL databases:

• PostgreSQL
• MySQL
• Oracle
• SQL Server
• Amazon DynamoDB
• MongoDB or similar NoSQL databases

DevOps \& Automation

Experience with:

• Git / GitHub
• Linux and shell scripting
• CI/CD pipeline concepts
• Docker
• Infrastructure as Code:
• Terraform
• AWS CloudFormation
• AWS deployment practices

Preferred Skills

• Experience building enterprise-scale AWS data platforms.
• Hands-on experience with Generative AI applications.
• Experience with AWS Bedrock and LLM-based solutions.
• Knowledge of RAG architecture and AI application development.
• Experience with MLOps frameworks and model lifecycle management.
• Strong understanding of cloud security and governance.
• AWS certifications preferred:
• AWS Certified Data Engineer – Associate
• AWS Certified Solutions Architect
• AWS Certified Machine Learning Engineer

Job Type: Full-time

Work Location: In person