Leads the design, development, optimization, and support of ETL/ELT pipelines, cloud data warehouses, data models, and advanced SQL solutions. Partners with analysts, data scientists, and stakeholders to deliver reliable datasets, while implementing testing, documentation, CI/CD, monitoring, and incident resolution practices. Mentors associate engineers and provides technical leadership across data engineering initiatives.
Duties & Responsibilities:
- Lead the design, development, and optimization of ETL/ELT pipelines and workflows
- Write, tune, and optimize advanced SQL queries for large-scale data processing
- Build and maintain data warehouses and data models (relational, dimensional, star schema)
- Translate business requirements into robust, well-architected, and reusable data solutions
- Partner with analysts, data scientists, and stakeholders to deliver trusted, actionable datasets
- Implement and maintain unit tests, integration tests, and validation frameworks to ensure pipeline reliability
- Document workflows, and design decisions to support knowledge sharing and operational continuity
- Apply coding standards, CI/CD practices, version control, and peer code reviews to ensure high-quality deliverables
- Proactively monitor, optimize, and troubleshoot pipelines for performance, scalability, and cost efficiency
- Support deployments and handle post-production monitoring and incident resolution
- Mentor associate engineers, providing technical guidance and feedback
Requirements
Basic Qualifications:
- Bachelor’s degree in computer science, Information Systems, Engineering, or a related field
- 4–7 years of hands-on experience in data engineering or related fields
- Expertise in writing complex SQL queries, optimizing, using advanced functionals and turning the performance on cloud data warehouse environment.
- Strong experience with data warehousing concepts, design, and implementation
- Minimum 3 years of hands-on experience with Snowflake or modern cloud data warehouse
- Minimum 2 years of hands-on experience in data modeling
- Minimum 1 year of hands-on Python development (ETL/ELT scripting, OOP, automation)
- Strong knowledge of at least one major cloud platform (AWS, Azure, or GCP)
- Experience with orchestration tools (e.g., Airflow, ADF, Luigi) for workflow management
- Demonstrated ability to lead and mentor teams while managing multiple priorities
- Strong communication and stakeholder management skills
- Strong hands-on experience with data warehousing concepts, design, and implementation
Preferred Qualifications:
- Hands-on experience with DBT (Data Build Tool) for data transformations
- Familiarity with DevOps and CI/CD best practices in data engineering
- Exposure with real-time/streaming platforms (Kafka, Spark Streaming, Flink)
- Exposure to the e-commerce domain or large-scale B2B/B2C environments
- Has the ability to break down problems and estimate time for development tasks
- Experience mentoring and guiding junior engineers
- Understands the technology landscape, up to date on current technology trends and new technology, brings new ideas to the team
- Learns organization vision statement and decision-making framework. Able to understand how team and personal goals/objectives contribute to the organization vision
Similar Jobs
Artificial Intelligence • Big Data • Cloud • Machine Learning • Software
Designs and operates enterprise data governance infrastructure across Snowflake and AWS. Responsibilities include RBAC, authentication, masking, row-level security, compliance automation, Terraform provisioning, access workflows, monitoring, SOX evidence, data sharing, Iceberg governance, orchestration, cost optimization, and Snowflake AI/ML governance. The role also supports platform reliability, self-service data access, and scalable identity integration using Okta and SCIM.
Top Skills:
AlationAmazon AthenaAmazon EventbridgeApache AirflowApache IcebergSparkAWSAws LambdaCollibraGithub ActionsIamMwaaOktaPythonS3ScimSnowflakeSnowflake CortexSnowpark MlSQLTerraform
Cloud • Information Technology • Consulting • Cybersecurity
Design, build, and maintain scalable ETL/ELT data pipelines and data warehouse solutions using Snowflake. Optimize data models and query performance, ensure data quality, integrate multiple sources, implement security and governance, and collaborate with BI and product teams to deliver reliable datasets.
Top Skills:
AirflowAWSAzureCi/CdDbtDevOpsGCPHadoopInformaticaKafkaKinesisPythonScalaSnowflakeSparkSQL
Artificial Intelligence • Big Data • Cloud • Machine Learning • Software
Design, build, and maintain batch and real-time data pipelines; collaborate with architects, product owners, data scientists and analysts to provision and integrate data; ensure pipeline performance, quality, and scalability; enforce engineering best practices (CI/CD, testing, monitoring); mentor other engineers and support cross-team data integrations.
Top Skills:
AirflowAWSAzureDatabricksGCPGlueJavaPythonRSnowflakeSparkSQL
What you need to know about the Chennai Tech Scene
To locals, it's no secret that South India is leading the charge in big data infrastructure. While the environmental impact of data centers has long been a concern, emerging hubs like Chennai are favored by companies seeking ready access to renewable energy resources, which provide more sustainable and cost-effective solutions. As a result, Chennai, along with neighboring Bengaluru and Hyderabad, is poised for significant growth, with a projected 65 percent increase in data center capacity over the next decade.
