Top Data Engineer Jobs in Chennai

Build, migrate, stabilize, and optimize batch and streaming data pipelines on Databricks and AWS lakehouse platforms. Implement ingestion, transformation, testing, deployment, monitoring, security, governance, data quality, and metadata practices. Support data onboarding, mastering and golden-record integration, reusable engineering patterns, CI/CD, infrastructure automation, and operational readiness. Provide technical guidance to global teams and use AI-assisted engineering tools responsibly for coding, documentation, testing, and troubleshooting.

Develop and maintain scalable cloud-based data platforms, data lakes, warehouses, ETL/ELT pipelines, and real-time data systems. Responsibilities include data modeling, architecture, performance tuning, governance, security, testing, deployment, troubleshooting, documentation, and stakeholder collaboration. The role also provides technical leadership, mentors data engineers and DBAs, supports business intelligence requirements, and assists development and DevOps teams with SQL optimization and production data operations.

Design, build, and optimize enterprise data platforms and ETL/ELT pipelines using Snowflake, AWS Glue, Python, and AWS services. Develop data models, warehouses, analytics marts, and automation while ensuring performance, reliability, data quality, governance, and cost efficiency. Collaborate with technical and business stakeholders, provide solution designs, troubleshoot workflows, and mentor junior engineers.

Build, test, optimize, and support batch and real-time data pipelines using Spark, Databricks, Kafka, Scala, Python, and SQL. Troubleshoot performance and production issues, develop Iceberg or Delta Lake solutions, analyze execution plans and logs, and maintain reliable Lakehouse architectures. Collaborate with technical teams, participate in testing and deployments, address security and governance requirements, resolve application issues, and provide guidance to junior analysts while delivering work independently within established standards.

Leads the design, development, and optimization of secure, scalable data infrastructure and pipelines. Responsibilities include AWS data engineering, Terraform-based infrastructure automation, FinOps and cloud cost governance, streaming ingestion, Airflow orchestration, data security, CI/CD collaboration, incident response, and technical leadership across teams.

Develop and maintain batch and streaming data pipelines on Google Cloud. Build complex data transformations, orchestrate ETL workflows, perform data migrations, and maintain data warehouses and data lakes. The role requires expertise in BigQuery, Composer, GCS, Cloud SQL, Dataflow, Dataproc, IAM, Python, SQL, and PySpark, along with strong database, data modeling, communication, and problem-solving skills.

Build and maintain a centralized banking data platform, developing batch and streaming pipelines, data transformations, quality validation, reconciliation, governance, and downstream data services. Use AWS, Databricks, Spark, Delta Lake, dbt, SQL, and Python to support regulatory reporting, risk management, analytics, and enterprise applications. Troubleshoot pipeline issues, ensure traceability and access controls, and collaborate with senior engineers and banking partners in an agile environment.

Designs, builds, and maintains scalable ETL pipelines connecting Salesforce with Snowflake and other data systems. The role manages Salesforce APIs and bulk data loading, writes and optimizes SQL and Python for data transformation, conducts Git-based code reviews, and monitors pipeline performance, availability, and data integrity.

Designs, develops, and maintains data pipelines and data solutions using Snowflake, ETL processes, and data integration tools. Ensures data quality, supports large-scale data migration, optimizes workflows, troubleshoots complex data issues, and collaborates across teams. The role serves as a subject matter expert, contributes to technical decisions, manages team efforts, and mentors junior engineers.

Design and optimize scalable ETL/ELT pipelines using Databricks, PySpark, SQL, GCP, BigQuery, and Delta Lake. Build agentic workflows for data engineering automation, implement data quality and monitoring controls, optimize workloads, and support testing, deployment, troubleshooting, and production operations. Collaborate with business, data engineering, and platform teams to identify automation opportunities and develop reusable frameworks.

Design, build, and maintain scalable ETL pipelines, data warehouses, data lakes, and analytics platforms. Develop Streamlit dashboards, integrate PostHog behavioral data, create LangChain workflows, and use deepeval for model validation. Optimize SQL and database schemas, ensure data quality, governance, security, and privacy, monitor pipeline health, troubleshoot failures, and collaborate with data scientists and cross-functional teams.

Maintain Python-based web scraping configurations, monitor errors and data blockages, coordinate scraping requirements with stakeholders, and report on scraping activities. Build and optimize pipelines for data lake ingestion and transformation using ETL, data modeling, cloud, database, and infrastructure tools. Collaborate with data scientists and analysts while ensuring data security, compliance, testing, and maintainability.

Build and support scalable batch and streaming data pipelines, cloud-native data platforms, data lakes, warehouses, and real-time processing systems. Own data initiatives from solution design through deployment, monitoring, performance tuning, and production support. Develop ETL/ELT pipelines, data models, and integration frameworks while collaborating with product, analytics, platform, and DevOps teams. Troubleshoot production issues, improve reliability, and contribute to documentation and engineering best practices.

Design, build, and operate large-scale ETL pipelines and data products on platforms like Snowflake and BigQuery. Ensure pipeline SLAs, reliability, anomaly detection, and data quality. Partner with analytics, product, CRM, and marketing teams, document solutions, mentor junior engineers, and troubleshoot pipeline failures.

Designs, builds, and maintains scalable data pipelines using Python and GCP. Develops complex SQL queries, implements ETL/ELT workflows, supports data warehousing, monitors pipeline performance, troubleshoots issues, and ensures data integrity, security, scalability, and operational continuity through documentation and cross-functional collaboration.

Design, build, optimize, and maintain scalable GCP-based ETL and ELT data pipelines using BigQuery, Dataflow, Cloud Storage, Python, and SQL. Collaborate with cross-functional teams to define data requirements, transform large-scale datasets, monitor pipeline performance, troubleshoot reliability issues, and ensure data integrity within budget and operational constraints. The role is fully onsite in Chennai with a four-day work week.

Designs and supports batch and incremental data ingestion pipelines, ETL workflows, SQL transformations, and Airflow orchestration. Investigates data quality issues, reconciles discrepancies, performs data lineage analysis, supports cloud migration initiatives, and validates data across systems. The role also provides production support, resolves incidents, develops reusable engineering components, and collaborates with stakeholders on data requirements.

Designs and supports batch and incremental data ingestion pipelines, ETL workflows, SQL transformations, BigQuery data platforms, and Airflow orchestration. Investigates data quality issues, validates data across systems, performs lineage and source-to-target analysis, supports cloud migration initiatives, and resolves production incidents. Collaborates with stakeholders and engineering teams to define data requirements and develop reusable data engineering components.

Design and support batch and incremental data ingestion pipelines, ETL workflows, SQL transformations, and Airflow orchestration. Troubleshoot data quality, reconciliation, lineage, discrepancies, and production incidents. Support cloud data migrations, validate data across systems, collaborate with stakeholders, and develop reusable engineering components using BigQuery, SQL, Python, Airflow, and GCP services.

Build, optimize, and maintain scalable AWS-based data pipelines and data lakes (Snowflake/Databricks). Implement streaming ingestion (Kinesis/Kafka), CDC, automated data quality checks, and ML-ready feature stores. Collaborate with Product, ML, and Analytics teams, own pipeline QA, observability, and participate in on-call rotations to resolve production data issues.

Designs and maintains scalable AWS, Snowflake, and dbt data pipelines and transformation frameworks. Builds ETL/ELT workflows, dimensional models, data quality frameworks, and optimized SQL workloads for structured and semi-structured data. Responsibilities include pipeline orchestration, monitoring, troubleshooting, CI/CD, legacy ETL modernization, documentation, production support, and collaboration with technical teams. The role also contributes to architecture discussions, Agile delivery, code reviews, and mentoring junior engineers.

Build and maintain data systems and pipelines, organize and integrate raw data, analyze business needs, and develop analytical tools. Prepare data for predictive and prescriptive modeling, improve data quality, and collaborate with data scientists, architects, analysts, and global stakeholders to deliver reliable analytics solutions.

Build and maintain governed finance data products on Microsoft Fabric and Databricks. Responsibilities include developing PySpark and Python ingestion pipelines, implementing incremental loading and CDC, optimizing Delta Lake tables, managing Azure Data Factory pipelines and Analysis Services cubes, designing Power BI semantic models, and ensuring version control, testing, monitoring, governance, and data contract compliance. The role also involves stakeholder collaboration, mentoring junior engineers, and applying enterprise data architecture standards.

Build and support large-scale, real-time kdb+ analytics platforms processing billions of daily messages. Develop kdb Insights or KDB-X modules, packages, pipelines, performant q queries, and functions for trading desks. Create SRE tools to detect performance bottlenecks across q, Python, and Java processes running continuously in AWS. Support ingestion stability, Kubernetes deployments, PyKX data processing, and cloud-native analytics infrastructure.

Build and operate production data platforms, tooling, and infrastructure supporting autonomous pipeline delivery. Own monitoring, alerting, DataOps, Kafka, Kubernetes, Terraform, AWS, Databricks/Spark, dbt, and Airflow environments. Improve platform reliability, troubleshoot production issues, and partner with product, analytics, and infrastructure teams. Champion AI adoption and automation while enabling high-throughput, low-latency data processing and strong developer experience.