Design and implement ETL/ELT pipelines using Azure Databricks and ADF. Analyze data, apply data warehouse concepts and complex SQL, provide technical expertise to stakeholders, work independently within Agile teams, and respond to client data questions and insights.
JD Azure Databricks :
• Analyse the data in ADF and provides tool based technical expertise to the team members
• Expert in designing ETL /ELT using market latest technology with minimum guidance
• Good in DW concepts, Complex SQL and communication
• Works with multiple stakeholders and workstreams to provide expertise in data management
• Independently owns the assigned task
• Agile execution methodology
• Respond to client questions around the data and insights
• Years of Experience : 4-8
Indium Chennai, Tamil Nadu, IND Office
No.64 (Old N.143), Eldams Road, Ganesh Chambers, Teynampet, Chennai, Tamil Nadu, India, 600 018
Similar Jobs
Information Technology • Database • Consulting
Design, build, and optimize Databricks-based lakehouse solutions and pipelines for Financial Crime use cases (AML, KYC, sanctions, fraud, reporting). Manage Databricks workspaces, clusters, Unity Catalog, Delta Lake tables, DLT, streaming ingestion, PySpark/SQL transformations, MLflow/feature store workflows, CI/CD/IaC, governance, performance tuning, and operational runbooks.
Top Skills:
AirbyteAmazon KinesisAmazon S3SparkAuto LoaderAws Secrets ManagerAzure Data Lake Storage (Adls)Azure DevopsAzure Event HubsAzure Key VaultCopy IntoDatabricksDatabricks Asset Bundles (Dabs)Databricks Feature StoreDatabricks Model ServingDatabricks Vector SearchDbtDelta LakeDelta Live Tables (Dlt)FivetranGithub ActionsGitlab CiGoogle Cloud Storage (Gcs)Gpu InstancesGreat ExpectationsKafkaMlflowMosaic AiPhoton EnginePysparkPythonSpark SqlTerraformUnity Catalog
Information Technology • Database • Consulting
Design, build, and optimize Databricks-based lakehouse solutions for financial crime use cases. Manage Databricks workspaces, clusters, Unity Catalog, and jobs. Develop scalable batch and streaming PySpark/Spark SQL pipelines (Kafka/Event Hubs/Kinesis) using Delta Lake, DLT, and Medallion architecture. Ensure data governance, performance tuning, error handling, and integration with cloud storage (ADLS/S3/GCS) and secret management.
Top Skills:
Amazon S3Aws Secrets ManagerAzure Data Lake Storage (Adls)Azure Event HubsAzure Key VaultChange Data Feed (Cdf)DatabricksDatabricks JobsDatabricks SecretsDatabricks Unity CatalogDelta LakeDelta Live Tables (Dlt)Google Cloud Storage (Gcs)KafkaKinesisPysparkSpark SqlStructured Streaming
Fintech • Financial Services
The role involves enhancing application systems, providing solutions, consulting with clients, and analyzing vulnerabilities in applications while advising lower-level analysts and ensuring compliance with regulations.
Top Skills:
Big Data TechnologiesClouderaData BricksHiveJavaOraclePysparkPythonSQL Server
What you need to know about the Chennai Tech Scene
To locals, it's no secret that South India is leading the charge in big data infrastructure. While the environmental impact of data centers has long been a concern, emerging hubs like Chennai are favored by companies seeking ready access to renewable energy resources, which provide more sustainable and cost-effective solutions. As a result, Chennai, along with neighboring Bengaluru and Hyderabad, is poised for significant growth, with a projected 65 percent increase in data center capacity over the next decade.
