Lead the development and implementation of data engineering solutions using Azure technologies. Manage data architecture and optimize data pipelines.
Lead Data Engineer
Primary Skills
- AKS, Event Hub, Azure DevOps, Cosmos DB, Azure Functions
Specialization
- Azure Data Engineering Advanced: Senior Data Engineer
Job requirements
- Job Description
We are seeking a Senior Data Engineer with strong hands‑on expertise in Azure Data Platform and Microsoft Fabric to design, build, and scale modern, enterprise‑grade data solutions. The role involves working on end‑to‑end data engineering pipelines, lakehouse architectures, and analytics‑ready data models using Azure and Fabric’s unified analytics capabilities.
Key Responsibilities - Design, develop, and manage end‑to‑end data pipelines using Microsoft Fabric (Data Pipelines, Lakehouse, Warehouse, OneLake) and Azure Data Factory
- Build scalable data transformation and processing solutions using Fabric Notebooks, PySpark, Python, and SQL
- Implement Lakehouse and Medallion Architecture (Bronze, Silver, Gold) to support analytics and reporting workloads
- Develop and optimize SQL-based solutions including T‑SQL, stored procedures, and performance‑tuned queries
- Integrate data from multiple sources including RDBMS, APIs, files, streaming sources, and cloud storage
- Work with ADLS Gen2, Blob Storage, and OneLake for structured and unstructured data management
- Enable analytics and reporting by integrating Fabric datasets with Power BI
- Implement data quality checks, monitoring, logging, and error‑handling mechanisms
- Support migration and modernization initiatives from ADF / Databricks to Microsoft Fabric, where applicable
- Ensure adherence to data governance, security, lineage, and compliance standards (Microsoft Purview exposure preferred)
- Collaborate closely with architects, analysts, and business stakeholders to translate requirements into scalable data solutions
- Provide technical guidance and mentoring to junior engineers Required Skills & ExpertiseMicrosoft Azure & Fabric
- Microsoft Fabric (Data Pipelines, Lakehouse, Warehouse, OneLake)
- Azure Data Factory (ADF)
- Azure Data Lake Storage Gen2 (ADLS G2)
- Azure Blob Storage Programming & Query Languages
- SQL (Basic & Advanced)
- T‑SQL
- Stored Procedures
- Python
- PySpark / Spark SQL Data Engineering & Architecture
- ETL / ELT Fundamentals
- Modern Data Platform Concepts
- Data Warehousing & Lakehouse Architecture
- Data Modeling Fundamentals (Star / Snowflake schemas)
- Incremental loads, CDC, and watermarking techniques Monitoring & Governance
- Pipeline monitoring & performance optimization
- Azure Monitoring / Fabric monitoring tools
- Microsoft Purview (Data lineage & governance – good to have) Good to Have
- Experience with streaming or near real‑time processing
- Exposure to Databricks in hybrid Azure–Fabric environments
- CI/CD for data pipelines using Git / Azure DevOps
- Experience supporting enterprise‑scale analytics platforms Soft Skills
- Strong problem‑solving and analytical mindset
- Excellent communication and stakeholder management skills
- Ability to work independently in fast‑paced delivery environments
- Ownership mindset with focus on quality and reliability Why Join Us
- Work on cutting‑edge Azure & Microsoft Fabric implementations
- Exposure to large‑scale enterprise modernization programs
- Growth opportunities across cloud, data platforms, and analytics
- Collaborative, high‑impact engineering culture
Brillio Chennai, Tamil Nadu, IND Office
Chennai, India
Similar Jobs
Big Data • Fintech • Information Technology • Business Intelligence • Financial Services • Cybersecurity • Big Data Analytics
Design, build, and maintain Java-based cloud-native backend services and microservices for a fraud platform. Contribute to system design, CI/CD, integrations, performance, and production reliability. Mentor junior engineers and collaborate across product, architecture, DevOps, and cross-functional teams in a hybrid environment.
Top Skills:
AWSAzureCi/CdDevOpsGCPGitGradleJavaKafkaMavenMicroservicesNoSQLRabbitMQRdbmsRest ApisSpring Boot
Digital Media • Information Technology • News + Entertainment
Write and maintain clean, efficient code; design and develop new applications; implement unit and integration testing frameworks; document development and deployment processes; analyze integration needs and system architecture; collaborate with QA; troubleshoot performance and functional issues; exercise independent judgment and work variable schedules including nights and weekends.
Digital Media • Information Technology • News + Entertainment
Perform penetration tests (web, network, API, mobile) and vulnerability assessments; simulate attacks, exploit and document weaknesses; produce remediation reports; run red team exercises and security audits; develop automated testing tools; collaborate with development, DevOps, QA and stakeholders; mentor junior staff and manage technical documentation to improve security and compliance (OWASP, NIST).
Top Skills:
BashBurp SuiteDnsHTTPHttpsLinuxMetasploitNistNmapOwaspOwasp ZapPowershellPythonTcp/IpWindowsWireshark
What you need to know about the Chennai Tech Scene
To locals, it's no secret that South India is leading the charge in big data infrastructure. While the environmental impact of data centers has long been a concern, emerging hubs like Chennai are favored by companies seeking ready access to renewable energy resources, which provide more sustainable and cost-effective solutions. As a result, Chennai, along with neighboring Bengaluru and Hyderabad, is poised for significant growth, with a projected 65 percent increase in data center capacity over the next decade.
