Lead design and implementation of Databricks Lakehouse data platforms for healthcare. Build scalable ETL/ELT pipelines with PySpark/SQL, enforce data governance, security (HIPAA/PHI/PII), integrate clinical and claims sources, optimize performance and costs, and mentor engineering teams.
We are looking for a Senior Databricks Data Engineer with strong expertise in Databricks, cloud data architecture, data engineering, and healthcare data. The ideal candidate will be responsible for designing scalable, secure, and high-performance data platforms supporting healthcare analytics and data integration.
The candidate should have strong knowledge of modern data architecture, Lakehouse architecture, SQL, Python, data governance, and healthcare domain workflows.
Responsibilities- Design and lead Databricks Lakehouse architecture for large-scale healthcare data platforms.
- Define scalable data architectures for batch and real-time healthcare data processing.
- Design data ingestion, transformation, storage, and consumption patterns using Databricks.
- Develop architecture using Delta Lake, Unity Catalog, Databricks Workflows, and Delta Live Tables/Lakeflow.
- Integrate data from multiple healthcare sources, including clinical, claims, eligibility, provider, and patient systems.
- Design robust ETL/ELT pipelines using PySpark, SQL, and Databricks.
- Establish data quality, validation, reconciliation, lineage, and governance frameworks.
- Design secure healthcare data solutions while considering HIPAA and PHI/PII protection requirements.
- Work with cloud platforms such as Azure, AWS, or GCP to build scalable data solutions.
- Collaborate with data engineers, data scientists, analysts, product owners, and business stakeholders.
- Define technical standards, architectural patterns, and best practices for Databricks development.
- Lead performance optimization and cost-management initiatives across the Databricks platform.
- Provide technical leadership and mentoring to data engineering teams.
- Participate in Agile ceremonies and contribute to technical planning and roadmap discussions.
- Strong hands-on experience with Databricks.
- Expertise in Lakehouse architecture and Delta Lake.
- Strong PySpark and SQL skills.
- Experience with Unity Catalog and data governance.
- Experience designing enterprise-scale data platforms.
- Strong knowledge of ETL/ELT and data modeling.
- Experience with cloud platforms such as Azure, AWS, or GCP.
- Experience with data integration and orchestration tools.
- Knowledge of CI/CD, Git, and DevOps practices.
- Strong understanding of data security and access control.
- Experience with Agile/Scrum methodologies.
EXL Chennai, Tamil Nadu, IND Office
Chennai, India
Similar Jobs
Big Data • Cloud • Software • Database
Lead and manage a regional Builder Relations team in APAC, driving developer and customer engagement through schema design reviews, live enablement sessions, content creation, and collaboration with Sales and Marketing. Mentor team members, measure business impact, and scale developer enablement programs while advocating MongoDB adoption and AI-native solutions.
Top Skills:
AIAWSDocument DatabasesGCPAzureMongoDBMongodb Atlas
Artificial Intelligence • Big Data • Healthtech • Information Technology • Machine Learning • Software • Analytics
Lead UAT test planning and execution for Agile software deliveries. Create test plans, cases, and data for functional and claim testing. Develop automated tests (Cypress, Selenium), report status and defects, present results to stakeholders, and collaborate with scrum teams to resolve issues and mitigate UAT risks.
Top Skills:
CypressSelenium
Fintech • Information Technology • Insurance • Financial Services • Big Data Analytics
Lead business process excellence by identifying AI/automation opportunities, designing and deploying Microsoft Power Platform and RPA solutions, guiding teams, managing stakeholders, and driving continuous improvement across operations and enabling functions.
Top Skills:
AIExcelMicrosoft Power PlatformPower AppsPower AutomatePower BIPower PagesPowerPointRpa ToolsVBA
What you need to know about the Chennai Tech Scene
To locals, it's no secret that South India is leading the charge in big data infrastructure. While the environmental impact of data centers has long been a concern, emerging hubs like Chennai are favored by companies seeking ready access to renewable energy resources, which provide more sustainable and cost-effective solutions. As a result, Chennai, along with neighboring Bengaluru and Hyderabad, is poised for significant growth, with a projected 65 percent increase in data center capacity over the next decade.
