Lead the design, development, and scaling of a digital experience analytics platform. Build data foundations and AI-driven decision systems using Azure, Databricks, dbt, Python, SQL, and cloud data technologies. Guide junior engineers, collaborate with product and analytics teams, support data architecture, modeling, governance, quality, and operationalization, and coordinate cloud deployments and technical delivery across onsite and offshore teams.
Role Overview
Requirements
We are seeking a highly skilled Lead Data Engineer to design,
build, and scale our next-generation Digital Experience Analytics &
Insights platform, who can guide and mentor other junior engineers to build a
strong high performing team.
You will collaborate closely with data
engineers, product managers, and analytics teams to create data foundations and
AI-driven decision systems that maximize business impact while ensuring
scalability and technical excellence.
Requirements
Expected
Qualifications
- 6 – 10 years
of relevant experience in Azure Data Engineering.
- Minimum 2
years of experience working with Adobe Analytics.
- Strong Agentic
AI Experience in Databricks and expertise in DBT framework for data
transformation and processing.
- Proficiency
in Python for data engineering and automation tasks.
- 6-10 years strong
hands-on experience with SQL for querying and data manipulation.
- Linux scripting
experience for data ingestion and manipulation.
- Strong
experience in data architecture, data modeling, data quality and
governance, and working knowledge on data warehouses such as Snowflake,
for data processing and operationalization
- Excellent
problem-solving skills and ability to work independently.
- Strong
communication skills to collaborate with technical and non-technical teams
who can efficiently work in onsite-offshore model
- Collaborates
with other cloud teams for deployment on Azure or GCP
- Knowledge of
CI/CD, DevOps, and Infrastructure as Code (IaC) practices is an advantage
- Experience
working with Azure Data Factory (ADF), Azure Data Lake, and PowerBI is a
plus.
- Prior
experience in e-commerce domain is a plus
Similar Jobs
eCommerce • Retail • Software
Leads the design, development, optimization, and support of ETL/ELT pipelines, data warehouses, data models, and cloud-based data solutions. Writes advanced SQL, implements testing and CI/CD practices, monitors production systems, resolves incidents, and improves performance and cost efficiency. Partners with analysts, data scientists, and stakeholders to deliver reliable datasets while documenting solutions and mentoring associate engineers.
Top Skills:
AirflowApache FlinkAWSAzureAzure Data FactoryCi/CdDbtGCPGitKafkaLuigiPythonSnowflakeSpark StreamingSQL
Blockchain • Database • Analytics
Lead the design, development, and optimization of scalable Databricks ETL/ELT pipelines. Integrate data from databases, S3, files, and REST APIs; implement Delta Lake medallion architecture, Unity Catalog, dimensional models, data quality controls, and Spark performance tuning. Schedule and monitor Databricks workflows, collaborate cross-functionally, and guide engineers while driving technical decisions.
Top Skills:
AirflowAmazon S3SparkAuto LoaderAWSCi/CdDatabricksDatabricks Lakehouse PlatformDbtDelta LakeGitKafkaPysparkPythonRest ApisSQLUnity Catalog
HR Tech • Professional Services
Designs, builds, and optimizes scalable batch and real-time data pipelines on GCP using Dataflow, Apache Beam, Java, and BigQuery. Integrates diverse data sources, improves pipeline performance and reliability, supports data governance and security, troubleshoots production issues, and collaborates with architects, analysts, DevOps, and business stakeholders.
Top Skills:
Apache BeamSparkBigQueryCi/CdCloud ComposerCloud StorageDataprocGitGoogle Cloud Platform (Gcp)Google DataflowJavaKafkaPub/SubPythonSQL
What you need to know about the Chennai Tech Scene
To locals, it's no secret that South India is leading the charge in big data infrastructure. While the environmental impact of data centers has long been a concern, emerging hubs like Chennai are favored by companies seeking ready access to renewable energy resources, which provide more sustainable and cost-effective solutions. As a result, Chennai, along with neighboring Bengaluru and Hyderabad, is poised for significant growth, with a projected 65 percent increase in data center capacity over the next decade.
