Designs and owns scalable Azure-based data platforms, including enterprise data warehouses, ETL/ELT pipelines, PySpark and Delta Lake transformations, dimensional models, and Power BI analytics. The role also covers data quality, reconciliation, financial reporting logic, API ingestion, performance optimization, CI/CD, stakeholder collaboration, architecture best practices, and mentoring junior engineers.
Senior Data Engineer
Location - India (Coimbatore preferred)
Work Mode - WFO preferred, hybrid or remote will also be considered for the right candidate
Experience - 5+ years in Data Engineering, including hands-on cloud data platform delivery
Employment Type - Full-Time
ROLE OVERVIEW
Vserve is looking for a Senior Data Engineer to design, build, and own scalable data platforms on Microsoft Azure
ecosystems. This role will lead the development of Enterprise Data Warehousing, ETL/ELT pipelines, and analytics-
ready data models that power reporting and decision-making for our retail, wholesale, and distribution clients. The
ideal candidate is equally comfortable architecting cloud-native pipelines and partnering directly with business
stakeholders to translate requirements into reliable, production-grade data solutions.
KEY RESPONSIBILITIES
• Design, build, and maintain scalable Enterprise Data Warehouse (EDW) solutions on Azure (ADF, Databricks, Azure SQL, Microsoft Fabric).
• Develop and orchestrate end-to-end ETL/ELT pipelines using Apache Airflow, Azure Data Factory, and Databricks, ensuring reliability and minimal manual intervention.
• Build and optimize PySpark transformations and Delta Lake pipelines to process large-scale datasets into curated, analytics-ready data products.
• Design dimensional data models (star/snowflake schemas), fact and dimension tables, and data marts to support sales, inventory, demand, and profitability reporting.
• Develop Power BI semantic models, datasets, and interactive dashboards enabling self-service analytics and KPI reporting for business stakeholders.
• Implement data quality and validation frameworks, including reconciliation checks, exception reporting, and automated alerting.
• Own data reconciliation between ERP/business systems (e.g., Microsoft Dynamics AX / D365 F&O, POS platforms) and the enterprise data warehouse.
• Implement complex business logic for margin calculations, cost attribution, and promotional/discount allocation to
support accurate financial reporting.
• Build automated data ingestion frameworks using Python and REST APIs, and implement incremental loading
strategies for efficient, near real-time reporting.
• Tune and optimize SQL/PL-SQL queries and data workflows for performance across high-volume, regulated, or
compliance-sensitive datasets.
• Implement CI/CD for data pipelines (GitHub Actions, Git) to improve release consistency and reduce deployment
effort.
• Partner with business users, analysts, and SMEs to gather requirements and translate them into scalable, well-
documented data engineering solutions.
• Mentor junior data engineers and contribute to data architecture and engineering best practices across the team.
CANDIDATE PROFILE — SKILLS & EXPERIENCE
Must-Have
• 5+ years of hands-on Data Engineering experience, with demonstrated ownership of production data pipelines
end to end.
• Strong experience with Microsoft Azure data services: Azure Data Factory (ADF), Databricks, Azure Data
Lake, Azure SQL.
• Hands-on expertise in Apache Spark / PySpark and Delta Lake for large-scale data processing.
• Proven experience building and orchestrating pipelines with Apache Airflow or equivalent workflow
orchestration tools.
• Advanced SQL skills and solid Python (Pandas, NumPy) programming ability.
• Strong grounding in dimensional data modeling (star/snowflake schemas, normalization, ER-diagram design).
• Experience with Power BI (DAX, Power Query) and/or Microsoft Fabric for reporting and dashboard delivery.
• Experience implementing data quality, validation, and reconciliation frameworks in a production
environment.
Good-to-Have
• Exposure to Microsoft Dynamics AX / Dynamics 365 Finance & Operations (D365 F&O) or similar ERP/POS
platforms (e.g., Aptos POS).
• Experience with Informatica PowerCenter or other enterprise ETL tools.
• Familiarity with MongoDB, PostgreSQL, or DynamoDB alongside relational/warehouse platforms.
• Experience in retail, wholesale distribution, or financial services data domains.
• Working knowledge of CI/CD practices (GitHub Actions, Git) and Agile delivery methodologies.
• Prior experience mentoring engineers or leading a small data engineering pod.
Education
• Bachelor degree in Computer Science, Information Technology, or a related field is required.
• Master degree is a plus.
Requirements
Azure, ADF, Databricks, PySpark, Python, SQL, Airflow, Delta Lake, ETL/ELT, Data Warehousing, Data Modeling, Power BI, DAX, Data Quality, REST API, Git/CI-CD, D365 F&O
Similar Jobs
Fintech • Financial Services
Design, develop, and maintain scalable ETL and ELT pipelines, data architectures, data lakes, warehouses, and lakehouse solutions. Implement data quality, validation, monitoring, and lineage frameworks. Build and optimize cloud data solutions using Spark, Kafka, and Databricks, while tuning SQL queries and pipeline workflows for performance and cost efficiency.
Top Skills:
Apache KafkaSparkData LakesData WarehousesDatabricksEltETLHadoopLakehouseNoSQLPythonRelational DatabasesSQL
Artificial Intelligence • Fintech • Machine Learning • Software • App development • Conversational AI • Generative AI
Migrate SQL Server data products and analytical workloads to a modern cloud platform. Redesign legacy SQL, ETL processes, and business logic; build scalable pipelines and analytical models; validate data quality; collaborate with stakeholders; and improve platform reliability, performance, cost efficiency, and AI-assisted data engineering automation.
Top Skills:
BigQueryCi/CdCloud ComposerETLGCPGoogle Cloud StorageIamMs Sql ServerPythonSQL
Agency • Information Technology
Build and maintain Snowflake and SQL Server data pipelines for an operational data store, including ingestion, transformation, schema modeling, migrations, CI/CD, data quality testing, and legacy batch-to-API modernization. Collaborate with domain teams and architects to support customer, loan, payment, and interaction data. Ensure data accuracy, lineage, observability, and performance while applying generative AI tools to accelerate development and testing.
Top Skills:
AlationApache AirflowCollibraDagsterDbtFlywayGenerative AiGitGithub ActionsGitlab CiGreat ExpectationsInformaticaPostgresPrefectPythonRest ApisSchemachangeSnowflakeSnowflake Dynamic TablesSnowflake TasksSQLSQL ServerTerraform
What you need to know about the Chennai Tech Scene
To locals, it's no secret that South India is leading the charge in big data infrastructure. While the environmental impact of data centers has long been a concern, emerging hubs like Chennai are favored by companies seeking ready access to renewable energy resources, which provide more sustainable and cost-effective solutions. As a result, Chennai, along with neighboring Bengaluru and Hyderabad, is poised for significant growth, with a projected 65 percent increase in data center capacity over the next decade.
