Kirtan  ·  Senior AWS / Snowflake Data Engineer  ·  7+ yrs

Senior
7+ years experienceremote
Available within 48 hrs

Proof of scale

60+ jobs created
10 TB data migrated
10 TB data migrated60+ jobs created

About Kirtan

Kirtan is a Data Engineer with 7+ years of experience in data engineering, specializing in AWS, Python, and data pipeline optimization. He has a proven track record of leading teams and delivering scalable data solutions.

7+ years of commercial experience in

Manufacturing & IndustrialSaaS / B2BLogistics & Supply Chain

Skills(26)

AWSPythonPySparkSnowflakeDBTAWS GlueAmazon RedshiftApache AirflowReactAWS IoT CoreAWS KinesisAirflowAmazon RDSAmazon EMROracleSFTPSQLAWS Lake FormationReact.jsAmazon S3AWS EMRAzure KMSAWS KMSGCSSharepointADLS

Why hire Kirtan?

Production deploy authorityMentored juniors

Successfully migrated 10 TB of data to Amazon Redshift with zero downtime.

Architected and implemented robust real-time data ingestion pipelines using AWS IoT Core and AWS Kinesis.

Led data processing initiatives using DBT and Airflow for end-to-end workflow optimization.

Integrated data from 50+ APIs and 5+ file storage locations, automating ingestion workflows.

Project highlights(7)

Modernizing Data Warehouse – Lead Data Engineer

Overview: Led a strategic migration from a legacy on-premises data warehouse to Amazon Redshift to enhance scalability, performance, and cost-efficiency. Responsibilities: Defined migration strategy and roadmap, assessed data compatibility and performance requirements, optimized ETL pipelines for cloud-native environment using AWS Glue.

Amazon RedshiftAWS GluePySparkReact

Key outcomes:

  • Successfully migrated 10 TB of data to Amazon Redshift with zero downtime.

  • Achieved significant performance improvements and cost reductions.

Real-Time Fuel Consumption Optimization Pipeline – Data Engineer/Architect

Overview: Developed a scalable real-time data pipeline to optimize fuel consumption within a glass factory by leveraging various sensors' data. Responsibilities: Architected and implemented a robust real-time data ingestion pipeline using AWS IoT Core and AWS Kinesis.

AWS IoT CoreAWS KinesisAWS GluePython

Key outcomes:

  • Optimized fuel consumption within a glass factory.

  • Successfully processed and stored high-volume data.

End-to-End Data Engineering Workflow Optimization – Lead Data Engineer

Overview: Led initiatives to optimize the end-to-end data engineering workflow, focusing on efficiency and automation. Responsibilities: Utilized DBT for data processing and testing, and Airflow for orchestration, ensuring seamless data flow from raw to aggregated layers.

SnowflakePySparkDBTAirflow

Key outcomes:

  • Optimized end-to-end data engineering workflow.

  • Enhanced workflow automation and data integration using Airflow and DBT.

Creation of MVP Framework for Data Migration using AWS Glue and Amazon EMR – Data Engineer/Architect

  • Developed an MVP framework for seamless data migration using AWS Glue for workflow orchestration and Amazon EMR for scalable data processing.
  • Leveraged Amazon Redshift for high performance and reliability, aiming for a scalable, efficient, and cost-effective data migration framework to modernize data infrastructure and optimize cloud-based analytics.
  • Designed and developed complex data pipelines using AWS Glue, ensuring efficient data extraction, transformation, and loading.
  • Implemented robust data migration strategies for minimal downtime and data integrity.
  • Developed, optimized, and managed ETL processes for loading data into Delta tables, focusing on performance and scalability using PySpark.
  • Leveraged Amazon Redshift for real-time data processing and continuous data validation.
  • Implemented and managed incremental and full load pipelines in AWS Glue, accommodating various source systems such as SQL DB, Oracle, and SFTP.
  • Implemented and monitored CDC (Change Data Capture) and SCD (Slowly Changing Dimensions) Type 1 and Type 2 to maintain historical accuracy and data integrity.
PySparkAmazon RDSAWS GlueAmazon EMRAmazon RedshiftOracleSFTPSQL

Key outcomes:

  • Developed a scalable, efficient, and cost-effective data migration framework.

  • Ensured minimal downtime and data integrity during transitions.

  • Maintained historical accuracy and data integrity using CDC and SCD.

Build a Scalable Data Pipeline for Customer Churn Prediction – Data Engineer/Architect

Scalable Data Pipeline for Customer Data — Salesforce + JIRA + AppInsights to data lake.

AWS Lake FormationSQLPythonPySparkSnowflakeReact.jsAWS GlueAmazon S3Amazon EMR

Key outcomes:

  • Successfully integrated data from multiple sources (Salesforce, JIRA, AppInsights).

  • Enhanced data quality through advanced cleaning and preprocessing.

  • Provided data for a customer churn prediction model.

Industry experience

Manufacturing & Industrial

1 project
  • •Real-Time Fuel Consumption Optimization Pipeline— Data Engineer/ArchitectAWS IoT Core · AWS Kinesis · AWS Glue · Python

SaaS / B2B

3 projects
  • •End-to-End Data Engineering Workflow Optimization— Lead Data EngineerSnowflake · PySpark · DBT · Airflow
  • •Creation of MVP Framework for Data Migration using AWS Glue and Amazon EMR— Data Engineer/ArchitectPySpark · Amazon RDS · AWS Glue · Amazon EMR +4
  • •Creating a Data Platform for Analytics— Data EngineerPython · PySpark · AWS EMR · Azure KMS +7

Logistics & Supply Chain

1 project
  • •Modernizing Data Warehouse— Lead Data EngineerAmazon Redshift · AWS Glue · PySpark · React

Ready to work with Kirtan?

Schedule an interview and onboard within 48 hours. No long hiring cycles.

At a Glance

Experience7+ years
Work moderemote
Starting from₹1.9 L/mo
Direct hirePossible
Start within48 hours
From₹1.9 L/ month

Single contract. No agency markup confusion.

Typically responds within 4 business hours.

5-day replacement guarantee
48-hour onboarding, single invoice
Direct chat — no recruiter middleman
Seniority signals
Owns production deploysGreenfield architectSystem ownerCode reviewerMentor / leads juniors
VerifiedVetted by Witarist
Technical skills assessed & verified
Background & identity checked
English communication verified
Ready to onboard in 48 hours

Not sure if this is the right fit?

Tell us your requirements and we'll match you with the best candidates.

Kirtan

Data Engineer