Sandeep is a Data Engineer with 6+ years of experience in DevOps and MLOps, specializing in AWS and Databricks. He has a proven track record in designing and automating end-to-end ML pipelines and managing CI/CD processes.
Migrated business infrastructure from physical data centers to AWS cloud environment.
Designed and automated end-to-end ML pipelines including data ingestion, preprocessing, model training, validation, and deployment.
Implemented cost management strategies for Databricks resources, reducing cloud expenditures through resource optimization and usage monitoring.
Led the migration of big data workloads to Databricks, optimizing storage and processing capabilities.
Orchestrated and migrated CI/CD processes using Cloud Formation, Terraform templates, and containerized infrastructure using Docker.
Migrated business applications from physical data center environment to AWS cloud designing highly available + fault-tolerant applications using AWS + EC2 + Auto-Scaling + ELB + Elastic Beanstalk as DevOps/AWS Engineer
Wrote AWS CloudFormation templates + Terraform scripts for infrastructure automation across VPC + EC2 + RDS + Route53 + monitoring big data workloads using AWS + CloudFormation + Terraform + VPC + EC2 as DevOps/AWS Engineer
Created snapshots for volume backups + images for EC2 launch configurations + configured web application deployments with CI/CD automation using AWS + EC2 + S3 + Route53 + Elastic Load Balancer as Cloud Engineer
AWS Migration & Core Services Setup — migrated business applications from physical data center to AWS cloud designing highly available + fault-tolerant applications. AWS + EC2 + Auto-Scaling + ELB + Elastic Beanstalk.
Key outcomes:
Successfully migrated business from a physical data center environment to AWS.
Designed and deployed multitude applications utilizing almost all the AWS stack focusing on high-availability, fault tolerance, and auto-scaling.
Containerized infrastructure using Docker setup in Vagrant, AWS and Amazon VPCS.
Infrastructure Automation & Cloud Operations — automated AWS infrastructure + source code management + monitoring cloud resources + big data workloads. AWS + CloudFormation + Terraform + VPC + EC2.
Key outcomes:
Automated AWS Infra with Terraform, creating reusable modules.
Performed all necessary GIT configuration support for different projects, maintaining branching, versioning, and merging strategies.
Monitored and analyzed performance metrics utilizing tools like Ganglia, Spark UI, and Databricks notebooks.
Cloud Resource Optimization & CI/CD Enhancements — optimized cloud infrastructure + snapshots for backups + EC2 launch configurations + CI/CD automation. AWS + EC2 + S3 + Route53 + Elastic Load Balancer.
Key outcomes:
Optimized volumes and EC2 instances, and created multi-AZ-VPC instances.
Implemented automation for cloud platforms as well as monitoring and alerting purposes.
Achieved cost optimization on the entire Production account.
Databricks & MLOps Implementation — Databricks cost management + big data migrations + end-to-end ML pipelines + model lifecycle management. Databricks + Terraform + ARM templates + Jenkins + Azure DevOps.
Key outcomes:
Implemented cost management strategies for Databricks resources, reducing cloud expenditures.
Led the migration of big data workloads to Databricks, optimizing storage and processing capabilities.
Designed and automated end-to-end ML pipelines.
Integrated Databricks with CI/CD pipelines using tools like Jenkins, Azure DevOps, or GitLab.
Sandeep
ML OPs Engineer