Design, build, and optimize large-scale cloud data platforms and ETL/ELT pipelines on Azure. Implement data lakes, Delta Lake, Synapse, and Databricks solutions for batch and real-time processing, ensure data quality/governance, optimize SQL and analytics workloads, support CI/CD and deployments, and collaborate with stakeholders to deliver scalable data architectures and reporting.
This role is for one of the Weekday's clients
Salary range: Rs 500000 - Rs 2000000 (ie INR 5-20 LPA)
Experience: 4+ yrs
Location: Bengaluru, Pune, Hyderabad, NCR, Chennai
Job Type: full-time
We are seeking a Data Engineer / Senior Data Engineer to design, build, and optimize large-scale cloud-based data platforms that support business intelligence, analytics, and data-driven decision-making. This role focuses on developing modern data pipelines, data lakes, and warehouse solutions while working with real-time and batch processing systems in a cloud environment. You will collaborate with cross-functional teams to deliver scalable, high-quality data architectures that enable actionable insights across the organization.
Requirements
Key Responsibilities
- Design, develop, and maintain scalable ETL/ELT pipelines for processing large volumes of structured and unstructured data.
- Build and manage cloud-based data platforms, including data lakes, data warehouses, and analytics solutions.
- Develop and optimize data pipelines using Spark, PySpark, and cloud-native data services.
- Work with real-time and batch data processing frameworks to ensure reliable and efficient data delivery.
- Design robust data models and implement best practices for data quality, governance, and performance.
- Optimize SQL queries, database performance, and large-scale analytics workloads.
- Develop and maintain reporting and visualization solutions to support business stakeholders.
- Collaborate with business, engineering, analytics, and product teams to define data requirements and solutions.
- Support CI/CD processes, automation, monitoring, and deployment of data engineering solutions.
- Create technical documentation and ensure adherence to modern engineering and data management standards.
What Makes You a Great Fit
- 7+ years of experience in Data Engineering, ETL development, and cloud-based analytics solutions.
- Strong expertise in Azure Data Factory, Databricks, Azure Synapse Analytics, Spark, and PySpark.
- Hands-on experience with data lakes, Delta Lake architecture, and modern big data technologies.
- Strong SQL development, query optimization, and performance tuning skills.
- Experience designing scalable cloud-hosted data platforms and enterprise-grade data architectures.
- Knowledge of reporting and analytics tools, including data modeling and dashboard development.
- Familiarity with CI/CD pipelines, DevOps practices, and cloud security concepts such as RBAC.
- Strong problem-solving, communication, and stakeholder management skills.
- Ability to translate business requirements into scalable technical data solutions while maintaining high standards of quality and reliability.
Similar Jobs
Fintech • Consulting
The Azure Data Engineer will design and maintain data pipelines, optimize data workflows, and ensure system performance, leveraging Azure technologies.
Top Skills:
Azure Data FactoryAzure Data LakeAzure DatabricksPysparkPythonSQL
Blockchain • Fintech • Payments • Consulting • Cryptocurrency • Cybersecurity • Quantum Computing
The role involves managing vendor contracts related to privacy and data protection, ensuring compliance with global regulations, and facilitating efficient business operations while collaborating with multiple stakeholders.
Top Skills:
CcpaGdprSaas Agreements
Blockchain • Fintech • Payments • Consulting • Cryptocurrency • Cybersecurity • Quantum Computing
The Senior Analyst provides operational and financial support for corporate meetings and events, focusing on budgeting, contract management, and event coordination.
Top Skills:
CventMicrosoft PowerpointMonday.Com
What you need to know about the Chennai Tech Scene
To locals, it's no secret that South India is leading the charge in big data infrastructure. While the environmental impact of data centers has long been a concern, emerging hubs like Chennai are favored by companies seeking ready access to renewable energy resources, which provide more sustainable and cost-effective solutions. As a result, Chennai, along with neighboring Bengaluru and Hyderabad, is poised for significant growth, with a projected 65 percent increase in data center capacity over the next decade.
