Build production-grade data ingestion and transformation pipelines using Databricks, PySpark, Delta Lake, and Microsoft Fabric. Develop LLM and AI agents, MCP servers, and governed AI data access solutions. Apply dimensional modeling, data quality, reconciliation, lineage, and governance practices while integrating enterprise systems. Partner with technical and business stakeholders to deliver data solutions, with preferred expertise in financial data engineering, FP&A, TBM classification, financial controls, and agentic workflows.
Required Skills & Experience
- 7+ years of experience in Data Engineering.
- Minimum 3 years of hands-on experience with Databricks.
- Strong hands-on experience with:
- PySpark
- Delta Lake
- Databricks Workflows
- Unity Catalog
- Strong SQL development and optimization skills.
- Strong experience in dimensional data modelling and star schemas.
- Hands-on experience with Slowly Changing Dimensions (SCD).
- Experience building production-grade data ingestion and transformation pipelines.
- Experience integrating multiple enterprise source systems, preferably ERP, planning and HR systems.
- Hands-on experience integrating Databricks with Microsoft Fabric.
- Good understanding of:
- Microsoft OneLake
- Fabric Lakehouses
- OneLake shortcuts
- Proven hands-on experience building LLM/AI agents using Python.
- Proven hands-on experience building MCP servers using Python in a real-world project.
- Understanding of secure and governed access for AI agents and tools.
- Strong understanding of data quality, reconciliation, lineage and data governance.
- Ability to work with business and technical stakeholders to translate requirements into data solutions.
Strongly Preferred Skills
- Experience in Finance, FP&A or financial data engineering.
- Strong understanding of:
- GL structures
- Budgets
- Forecasts
- Estimates
- Allocations
- Cost centres
- Financial planning scenarios
- Experience with TBM (Technology Business Management) classification or similar financial classification frameworks.
- Experience building AI-powered data engineering or ingestion solutions.
- Experience with agentic workflows for data mapping, classification and anomaly detection.
- Experience with additional LLM frameworks such as LangChain, LangGraph or similar.
- Experience with enterprise AI governance and secure AI/data access.
- Experience with Databricks data governance and lineage.
- Experience implementing automated data reconciliation and financial controls.
- Experience working with Microsoft Fabric-based enterprise analytics environments.
Similar Jobs
Information Technology • Database • Consulting
Design and build production GenAI systems: LLM/agentic applications, end-to-end RAG pipelines, prompt engineering, APIs, and integrations with enterprise data platforms. Apply guardrails, evaluation, and collaborate with Data Engineering and MLOps to deploy, monitor, and scale reusable components and best practices.
Top Skills:
Agentic AiAWSAzureAzure Ai SearchAzure DatabricksAzure OpenaiCi/CdClaudeClaude CodeContainersEmbeddingsFastapiFlaskGCPGptLangchainLanggraphLoraMicrosoft FabricMonitoringOpenaiPeftPrompt EngineeringPysparkPythonRagRetrievalSnowflakeSQLStreamlit
Healthtech
Own end-to-end data solutions across ingestion, transformation, warehousing, monitoring, and dashboards. Build scalable ETL/ELT pipelines, AI-enabled workflows, and analytics products. Partner cross-functionally, operate in rotating domain ownership, ensure data quality, compliance, and delivery, and drive initiatives from architecture through production with automation and AI-assisted practices.
Top Skills:
AirflowAWSCi/CdDbtEmbeddingsFivetranFlinkGenai ToolingGitKafkaLlm EcosystemsPower BIPythonSnowflakeSparkSQLStreamlitTableauVector Databases
Aerospace • Artificial Intelligence • Cloud • Machine Learning • Software • Cybersecurity • Defense
Provides systems safety engineering for aircraft turbine engines and auxiliary power systems. Conducts hazard and safety analyses, develops safety requirements and documentation, supports FAA and EASA compliance, performs reliability analysis, and reviews designs. Collaborates across engineering teams on risk mitigation, corrective actions, and aftermarket safety metrics. The role also supports process standardization, automation, AI application, technical coaching, innovation, and strategies to reduce certification cycle time and cost.
Top Skills:
Artificial IntelligenceDoorsDoors NgModel-Based Systems Engineering
What you need to know about the Chennai Tech Scene
To locals, it's no secret that South India is leading the charge in big data infrastructure. While the environmental impact of data centers has long been a concern, emerging hubs like Chennai are favored by companies seeking ready access to renewable energy resources, which provide more sustainable and cost-effective solutions. As a result, Chennai, along with neighboring Bengaluru and Hyderabad, is poised for significant growth, with a projected 65 percent increase in data center capacity over the next decade.
