Senior AWS Data Engineer

• Lead the design, development, and optimization of large-scale, reliable, and secure data pipelines and data lake architecture on AWS. • Architect and implement end-to-end data solutions, including data ingestion, storage, transformation, and analytics using AWS services (Glue, Redshift, S3, Lambda, EMR, Kinesis, Athena, RDS, etc.). • Mentor and guide a team of data engineers, conducting code reviews and fostering best practices in data engineering and cloud architecture. • Collaborate with data scientists, analysts, and business stakeholders to translate requirements into scalable and maintainable solutions. • Oversee migration of data from legacy systems to AWS-based data lakes and data warehouses. • Develop and enforce standards for data quality, security, and governance. • Drive the adoption of DevOps, CI/CD, and infrastructure-as-code practices within the data engineering team. • Ensure solutions are cost-effective, performant, and aligned with enterprise data strategy. • Stay current with advancements in AWS technologies and data engineering trends and evaluate new tools and frameworks for potential adoption. • Troubleshoot complex data issues and provide technical leadership in problem resolution. •

Role Overview: • Lead the design, development, and optimization of large-scale, reliable, and secure data pipelines and data lake architecture on AWS. • Architect and implement end-to-end data solutions, including data ingestion, storage, transformation, and analytics using AWS services (Glue, Redshift, S3, Lambda, EMR, Kinesis, Athena, RDS, etc.). • Mentor and guide a team of data engineers, conducting code reviews and fostering best practices in data engineering and cloud architecture. • Collaborate with data scientists, analysts, and business stakeholders to translate requirements into scalable and maintainable solutions. • Oversee migration of data from legacy systems to AWS-based data lakes and data warehouses. • Develop and enforce standards for data quality, security, and governance. • Drive the adoption of DevOps, CI/CD, and infrastructure-as-code practices within the data engineering team. • Ensure solutions are cost-effective, performant, and aligned with enterprise data strategy. • Stay current with advancements in AWS technologies and data engineering trends and evaluate new tools and frameworks for potential adoption. • Troubleshoot complex data issues and provide technical leadership in problem resolution. Key Responsibilities & Skillsets: • Common Skillsets: • Superior analytical and problem solving skills • Should be able to work on a problem independently and prepare client ready deliverable with minimal or no supervision • Good communication skill for client interaction Application development Skillsets: • Strong ability to debug complex data workflows, optimize application and ETL code, and automate data transformation processes • Systematic, analytical problem‑solving approach with strong ownership over data quality, performance, and delivery • Ability to quickly evaluate new AWS data and analytics services and determine fit for data pipelines or application architecture • Hands‑on experience developing data workflows and infrastructure using IaC frameworks such as CloudFormation or Terraform (as needed) • Working knowledge of CI/CD pipelines primarily to support data application deployments (Jenkins, CodePipeline, etc.) • Proficient with Git for versioning data processing code, libraries, and application components • Skilled in writing production‑grade code in Python, Bash, PowerShell, or similar languages, focusing on data processing and backend development • Experience using Docker for packaging applications and data-processing workloads, with exposure to containerized data services (ECS, EKS, etc.) • Comfortable developing and troubleshooting in Linux environments • Solid understanding of key AWS data services and application primitives such as S3, EC2, Glue, EMR, Lambda, RDS, DynamoDB, CloudWatch, and VPC networking concepts • Strong knowledge of AWS security and IAM as it relates to data pipelines, encryption (KMS), secure data access (IAM roles/policies), and audit controls • Hands‑on experience with ETL, distributed compute, and big data frameworks such as Spark, Glue, Hadoop/EMR, Impala, or similar tooling • Deep understanding of relational databases, SQL optimization, and application‑to‑database interaction patterns • Familiarity with log analytics and observability platforms (Splunk, ELK, Prometheus, Grafana) as they relate to monitoring data pipelines and applications Candidate Profile: • Bachelor's or Master's degree in Computer Science, Engineering, or a related field. • 10+ years of experience in data engineering, with at least 3 years in technical leadership or lead engineer role. • Extensive hands-on experience with AWS data services (Glue, Redshift, S3, Lambda, EMR/Spark, Kinesis, Athena, RDS, API Gateway, etc.). • Proficient in programming languages such as Python and SQL; experience with Shell scripting and Scala is a plus. • Strong experience designing, implementing, and managing data lakes, data warehouses, and data ingestion pipelines on AWS. • Proven experience with ETL/ELT processes, data modeling, and big data frameworks. • Demonstrated ability to lead, mentor, and coach engineers in a collaborative team environment. • Experience with DevOps practices, CI/CD pipelines, and infrastructure-as-code tools (e.g., CloudFormation, Terraform). • Excellent problem-solving, communication, and organizational skills. Originally posted on Himalayas Data-Engineer, AWS-Data-Engineer, Cloud-Data-Engineer, ETL-Engineer, Senior-Cloud-Data-Engineer, Senior-Data-Engineering, AWS-ETL-Data-Engineer, Senior-Data-Engineer-Jobs