EXL Logo

EXL

Databricks Tech Lead

Posted One Month Ago
Be an Early Applicant
Hybrid
Chennai, Tamil Nadu, IND
Expert/Leader
Hybrid
Chennai, Tamil Nadu, IND
Expert/Leader
Designs, builds, and optimizes enterprise Databricks lakehouse platforms for financial crime applications, including AML, KYC, fraud detection, sanctions screening, transaction monitoring, and regulatory reporting. Responsibilities include workspace and cluster administration, Unity Catalog governance, Delta Lake architecture, Delta Live Tables pipelines, workflow orchestration, data quality, security, performance optimization, and cloud data integration.
The summary above was generated by AI

We are looking for a skilled and passionate Databricks Engineer to design, build, and optimize enterprise-scale data lakehouse solutions on the Databricks platform. The successful candidate will be responsible for creating Databricks pipeline delivering Financial Crime platforms covering Anti-Money Laundering (AML), Know Your Customer (KYC), Customer Risk Assessment (CRA), Sanctions Screening, Transaction Monitoring, Fraud Detection, and Regulatory Reporting 

Responsibilities

Databricks Platform Engineering 

  • Design, build, and maintain Databricks workspaces, clusters, and compute pools across dev/test/prod environments. 

  • Configure and manage Databricks Unity Catalog for data governance, access control, fine-grained permissions, and data lineage. 

  • Optimize cluster configurations — instance types, auto-scaling policies, spot/preemptible nodes — for cost and performance. 

  • Implement workspace-level best practices: folder structures, access controls, secret management (Databricks Secrets / Azure Key Vault / AWS Secrets Manager). 

  • Manage Databricks jobs, workflows, and multi-task job orchestration with dependency management. 

 

Delta Lake & Lakehouse Architecture 

  • Design and implement Delta Lake tables with appropriate partitioning, Z-ordering, and file compaction (OPTIMIZE / VACUUM). 

  • Build Medallion Architecture (Bronze / Silver / Gold) layers for structured data lake organization. 

  • Implement Delta Live Tables (DLT) pipelines for declarative, reliable ETL/ELT with built-in data quality expectations. 

  • Manage schema evolution, table versioning, time travel, and Change Data Feed (CDF) for incremental processing. 

  • Design data lakehouse patterns integrating Delta Lake with external systems (Kafka, ADLS, S3, GCS). 

Qualifications

Education 

  • Bachelor's or Master's degree in Computer Science, Information Technology, Data Engineering, or related field. 

 

Experience 

  • 8+ years of total experience in data engineering or software engineering. 

  • 3+ years of dedicated hands-on experience with the Databricks platform in production environments. 

  • Strong background in big data engineering, cloud data platforms, and distributed computing. 

 

Databricks Platform 

  • Deep expertise in Databricks Workspaces, Clusters, Jobs, Workflows, and Repos. 

  • Proficiency with Unity Catalog — metastore setup, catalog/schema/table management, access controls, and data lineage. 

  • Hands-on experience with Delta Live Tables (DLT) — pipeline development, expectations, and monitoring. 

  • Strong command of Delta Lake internals — transaction log, ACID guarantees, file layout, and optimization techniques. 

  • Experience with Databricks SQL Warehouses, SQL Analytics, and dashboard creation. 

  • Knowledge of Databricks Photon engine, serverless compute, and cost optimization strategies. 

 

EXL Chennai, Tamil Nadu, IND Office

Chennai, India

Similar Jobs

11 Hours Ago
Hybrid
Chennai, Tamil Nadu, IND
Mid level
Mid level
Digital Media • Information Technology • News + Entertainment
Oversee network security safeguards, enforce secure hardware and software configurations, maintain disaster recovery plans, develop incident response policies, lead penetration testing and vulnerability assessments, conduct security audits, evaluate security technologies, and coordinate with network operations, development, and support teams. The role requires independent judgment and availability for nights, weekends, and variable schedules.
Top Skills: Disaster Recovery SystemsNetwork Security ProtocolsPenetration TestingVulnerability Assessment
11 Hours Ago
Hybrid
Chennai, Tamil Nadu, IND
Mid level
Mid level
Digital Media • Information Technology • News + Entertainment
Oversee network security safeguards, enforce secure hardware and software configurations, maintain disaster recovery plans, develop incident response policies, lead penetration tests and vulnerability assessments, conduct security audits, investigate security technologies, and coordinate with development, support, and network operations teams. The role requires independent judgment and availability for nights, weekends, and variable schedules.
Top Skills: Disaster Recovery SystemsNetwork Security ProtocolsNetwork Security SafeguardsPenetration TestingVulnerability Assessment
11 Hours Ago
Hybrid
Chennai, Tamil Nadu, IND
Mid level
Mid level
Digital Media • Information Technology • News + Entertainment
Oversee network security safeguards, enforce secure hardware and software configurations, maintain disaster recovery plans, develop incident response policies, evaluate security technologies, lead penetration tests and vulnerability assessments, and conduct internal and external security audits. The role also coordinates with network operations, development, and support teams and requires independent judgment, punctual attendance, and flexibility to work nights and weekends.
Top Skills: Disaster Recovery SystemsNetwork Security ControlsNetwork Security ProtocolsPenetration TestingVulnerability Assessments

What you need to know about the Chennai Tech Scene

To locals, it's no secret that South India is leading the charge in big data infrastructure. While the environmental impact of data centers has long been a concern, emerging hubs like Chennai are favored by companies seeking ready access to renewable energy resources, which provide more sustainable and cost-effective solutions. As a result, Chennai, along with neighboring Bengaluru and Hyderabad, is poised for significant growth, with a projected 65 percent increase in data center capacity over the next decade.

Sign up now Access later

Create Free Account

Please log in or sign up to report this job.

Create Free Account