Prem  ·  Lead Azure Data Engineer  ·  10+ yrs

Principal
10+ years experienceremote
Available within 48 hrs

Proof of scale

Increased revenue by 15% for product R2D2
Increased revenue by 15% for product R2D2

About Prem

N. Premchand is a Technology Lead / Data Engineer with 10+ years of experience in Azure Data Engineering and Spark application development.

10+ years of commercial experience in

Logistics & Supply ChainBankingFinTechManufacturing & Industrial

Skills(26)

Azure Data FactoryApache SparkSQL ServerPythonAzure DatabricksPower BIAzure Stream AnalyticsMicrosoft AzureAzure Data LakePySparkSpark SQLAzure DevOpsXgboostRandom ForestCloudera Hadoop HiveIoT streamingDelta LakeParquetAI/ML modelsSSRSSQLMDM toolsETLSQL ProfilerSQL Sentry Plan ExplorerAgile

Why hire Prem?

Production deploy authorityMentored 5+ juniors

Engineered Azure Data Factory pipelines for financial data integration and transformation.

Developed and deployed AI and ML models for predictive analytics in supply chain management.

Increased revenue by 15% for product R2D2 through data analysis and ingestion.

Increased revenue by 15% for product R2D2 by analyzing and ingesting data from Snowflake to Enterprise Data Warehouse using SSIS.

Automated data pipeline deployments in Azure Data Factory using JSON scripts, improving operational efficiency.

Engineered and maintained Azure Data Factory v2 pipelines, ensuring data integrity for financial data.

Project highlights(4)

Retail Data Solutions – Technology Lead / Data Engineer

Overview: Designed and implemented data solutions for the retail domain, leveraging Azure PaaS offerings. Responsibilities: Collaborated with cross-functional teams to ingest data from Snowflake to Enterprise Data Warehouse using SSIS for the product R2D2. Developed a comprehensive data dictionary for a centralized product master and implemented ETL processes using Azure Data Factory.

Azure Data FactorySQL ServerApache SparkPower BI

Key outcomes:

  • Increased revenue by 15% for product R2D2.

  • Automated data pipeline deployments using JSON scripts.

  • Developed interactive Power BI dashboards for sales and inventory metrics.

Banking Data Solutions – Technology Lead / Data Engineer

Overview: Led Azure Data Factory implementation for structured and unstructured banking data ingestion. Responsibilities: Spearheaded batch and real-time data processing solutions using ADF, Azure Databricks, and Azure Stream Analytics. Managed Azure Databricks clusters for scalable data processing.

Azure Data FactoryAzure DatabricksAzure Stream Analytics

Key outcomes:

  • Engineered and maintained Azure Data Factory v2 pipelines.

  • Automated data processing workflows in ADF.

  • Implemented CI/CD using Azure DevOps and Jenkins.

Project 3 – Azure Data Engineer

  • Engineered data pipeline architectures in Microsoft Azure for retail finance data management.
  • Designed Azure-based solutions to optimize analytics tools for financial retail scenarios.
  • Engineered data pipeline architectures in Microsoft Azure, leveraging Azure Data Factory and Azure Databricks for retail finance data management.
  • Designed Azure-based solutions to optimize analytics tools for financial retail scenarios, enhancing data-driven decision-making.
  • Guided retail clients on cost-efficient Azure PaaS and SaaS offerings tailored to finance domain needs.
  • Developed self-service reporting in Azure Data Lake Store Gen2 using ELT approach.
  • Innovated with Spark Vectorized panda UDFs for efficient financial data analysis in retail.
  • Configured Azure infrastructure to support scalable analytics for retail banking and financial services.
  • Implemented complex transformations using PySpark and Spark SQL in Azure Databricks.
  • Automated bulk data transfers from relational databases to Azure Data Lake Gen2.
  • Designed custom logging framework for ELT and performance insights in retail financial services data pipelines.
  • Adopted CI/CD practices using Azure DevOps for financial data pipeline development.
  • Facilitated denormalized data access for Power BI, enhancing visualization.
  • Applied time pipelines in Data Factory to enhance monitoring.
  • Enabled comprehensive monitoring and Azure log analytics for high availability.
  • Conducted series analysis on sales data to forecast trends and inform promotional strategies.
  • Developed predictive models (Xgboost, Random Forest) for strategic planning.
  • Transitioned data storage from Cloudera Hadoop Hive to Azure Data Lake Store for digital transformation.
  • Implemented IoT streaming and Delta Lake solutions for real-time financial transaction logging.
  • Exposed data efficiently (e.g., parquet) via Azure Spark Databricks for optimized financial analytics.
  • Architected scalable cloud solutions using SQL databases and ELT techniques.
  • Managed large datasets to meet functional and non-functional retail finance requirements.
  • Integrated advanced analytics tools with the data platform.
Microsoft AzureAzure Data FactoryAzure DatabricksAzure Data LakePySparkSpark SQLAzure DevOpsPower BIXgboostRandom ForestCloudera Hadoop HiveIoT streamingDelta LakeParquetSQL ServerAI/ML models

Key outcomes:

  • Engineered data pipeline architectures for retail finance data management.

  • Innovated with Spark Vectorized panda UDFs for efficient financial data analysis.

  • Adopted CI/CD practices for financial data pipeline development.

  • Developed predictive models (Xgboost, Random Forest) for strategic planning.

Project 4 – Systems Engineer / ETL Developer

Systems Engineer / ETL Developer — SSRS reports + scheduled execution + debugging.

SQL ServerSSRSSQLMDM toolsETLSQL ProfilerSQL Sentry Plan ExplorerAgile

Key outcomes:

  • Developed various types of SQL Server Reports and scheduled for automatic execution.

  • Established data model/architecture standards focusing on MDM.

  • Implemented triggers, stored procedures, functions, and error handling mechanisms.

  • Optimized queries to improve SQL Server performance.

Industry experience

Logistics & Supply Chain

1 project
  • •Retail Data Solutions— Technology Lead / Data EngineerAzure Data Factory · SQL Server · Apache Spark · Power BI

Banking

2 projects
  • •Banking Data Solutions— Technology Lead / Data EngineerAzure Data Factory · Azure Databricks · Azure Stream Analytics
  • •Project— Azure Data EngineerMicrosoft Azure · Azure Data Factory · Azure Databricks · Azure Data Lake +12

FinTech

2 projects
  • •Banking Data Solutions— Technology Lead / Data EngineerAzure Data Factory · Azure Databricks · Azure Stream Analytics
  • •Project— Azure Data EngineerMicrosoft Azure · Azure Data Factory · Azure Databricks · Azure Data Lake +12

Manufacturing & Industrial

3 projects
  • •Retail Data Solutions— Technology Lead / Data EngineerAzure Data Factory · SQL Server · Apache Spark · Power BI
  • •Banking Data Solutions— Technology Lead / Data EngineerAzure Data Factory · Azure Databricks · Azure Stream Analytics
  • •Project— Azure Data EngineerMicrosoft Azure · Azure Data Factory · Azure Databricks · Azure Data Lake +12

Ready to work with Prem?

Schedule an interview and onboard within 48 hours. No long hiring cycles.

At a Glance

Experience10+ years
Work moderemote
Starting from₹1.7 L/mo
Direct hirePossible
Start within48 hours
From₹1.7 L/ month

Single contract. No agency markup confusion.

Typically responds within 4 business hours.

5-day replacement guarantee
48-hour onboarding, single invoice
Direct chat — no recruiter middleman
Seniority signals
Owns production deploysGreenfield architectSystem ownerCode reviewerMentor / leads juniors
VerifiedVetted by Witarist
Technical skills assessed & verified
Background & identity checked
English communication verified
Ready to onboard in 48 hours

Not sure if this is the right fit?

Tell us your requirements and we'll match you with the best candidates.

Prem

Azure developer