Design, develop, and maintain scalable cloud-based data pipelines and ETL/ELT processes. Build and optimize data solutions on GCP (BigQuery, Dataflow), ensure data quality, governance, monitoring, and high availability. Develop backend data processing services in Java, support BI, analytics, and ML teams, troubleshoot pipeline issues, and contribute to data platform architecture and modernization.
This role is for one of the Weekday's clients
Salary range: Rs 1200000 - Rs 2500000 (ie INR 12-25 LPA)
Experience: 7+ yrs
Location: Pune, Bangalore, Chennai, Coimbatore
Job Type: full-time
We are looking for a skilled Data Engineer who is passionate about building scalable data platforms and enabling data-driven decision-making. This role involves designing, developing, and maintaining robust data pipelines, processing large-scale datasets, and ensuring the availability, reliability, and performance of data systems.
The ideal candidate will have strong expertise in cloud-based data engineering, backend development, and modern data processing technologies. You will work closely with cross-functional teams to create efficient data solutions that support analytics, reporting, machine learning, and business intelligence initiatives.
RequirementsKey Responsibilities
- Design, develop, and maintain scalable data pipelines for processing structured and unstructured data.
- Build and optimize cloud-based data solutions using modern data engineering best practices.
- Develop and manage data ingestion, transformation, validation, and orchestration workflows.
- Create reliable ETL/ELT pipelines to support analytics, reporting, and operational use cases.
- Work with large datasets to ensure data quality, consistency, and accessibility across systems.
- Develop and optimize backend services and data processing applications using Java.
- Implement data governance, monitoring, and performance optimization practices.
- Collaborate with product, analytics, and engineering teams to understand data requirements and deliver scalable solutions.
- Troubleshoot data pipeline issues and proactively identify opportunities for improvement.
- Ensure high availability, security, and scalability of data infrastructure.
- Participate in architecture discussions and contribute to data platform modernization initiatives.
- Support business intelligence, advanced analytics, and machine learning teams with reliable data assets.
- Strong experience in Data Engineering and building scalable data processing systems.
- Proficiency in Java and cloud-native data engineering practices.
- Hands-on experience with Google Cloud Platform (GCP) and its data ecosystem.
- Strong understanding of data warehousing, data modeling, and distributed data processing concepts.
- Experience building and maintaining high-performance ETL/ELT pipelines.
- Familiarity with BigQuery, Dataflow, and other cloud-based data processing technologies is highly desirable.
- Strong SQL skills and experience working with large-scale datasets.
- Understanding of data architecture, data governance, and data quality frameworks.
- Experience with workflow orchestration, monitoring, and automation tools.
- Strong analytical thinking and problem-solving abilities.
- Ability to collaborate effectively with cross-functional stakeholders and communicate technical concepts clearly.
- Self-driven mindset with a focus on ownership, scalability, and continuous improvement.
Similar Jobs
Blockchain • Fintech • Payments • Consulting • Cryptocurrency • Cybersecurity • Quantum Computing
Design, build, and scale cloud-native data and BI solutions on Databricks and Snowflake. Develop high-performance data pipelines, enable multi-cloud interoperability (AWS/Azure), implement governance and secure access patterns, maintain CI/CD and infrastructure-as-code for platform provisioning, and lead technical design while mentoring engineers.
Top Skills:
Adls Gen2AksApache IcebergAWSAzureAzure DatabricksCi/CdDatabricksDelta LakeEmrEnterprise Identity IntegrationGlueIamInfrastructure As CodeRbacS3Service PrincipalsSnowflake
Artificial Intelligence • HR Tech • Professional Services • Software
Design, build, and optimize large-scale cloud data platforms and ETL/ELT pipelines on Azure. Implement data lakes, Delta Lake, Synapse, and Databricks solutions for batch and real-time processing, ensure data quality/governance, optimize SQL and analytics workloads, support CI/CD and deployments, and collaborate with stakeholders to deliver scalable data architectures and reporting.
Top Skills:
Azure Data FactoryAzure Synapse AnalyticsCi/CdData LakeDatabricksDelta LakeDevOpsEtl/EltPysparkRbacSparkSQL
Artificial Intelligence • HR Tech • Professional Services • Software
Design, build, and optimize scalable AWS-based data platforms and ETL/ELT pipelines using Python/PySpark and SQL. Implement data warehouses, data lakes/lakehouse architectures, ensure data quality/governance, optimize performance, enable analytics/ML use cases, monitor reliability, and mentor junior engineers while supporting deployments and platform improvements.
Top Skills:
Amazon AuroraAmazon OpensearchAmazon RedshiftAmazon S3Aws GlueAws LambdaAws SnsAws Step FunctionsCi/CdData LakeData WarehouseDevOpsEltETLEvent-Driven ArchitectureInfrastructure As CodeLakehousePl/SqlPysparkPythonSQLWorkflow Orchestration
What you need to know about the Chennai Tech Scene
To locals, it's no secret that South India is leading the charge in big data infrastructure. While the environmental impact of data centers has long been a concern, emerging hubs like Chennai are favored by companies seeking ready access to renewable energy resources, which provide more sustainable and cost-effective solutions. As a result, Chennai, along with neighboring Bengaluru and Hyderabad, is poised for significant growth, with a projected 65 percent increase in data center capacity over the next decade.
