Build and manage Azure Databricks jobs, lakehouse pipelines, reusable Python/PySpark scripts, and cloud-native microservices. Develop LLM-powered copilots, agentic workflows, RAG pipelines, prompt strategies, guardrails, and AI inference services. Integrate enterprise data with AI solutions, optimize performance and costs, implement governance and observability, and contribute to architecture and technical design. The role also involves independent ownership, code reviews, mentoring, stakeholder collaboration, and delivering production-grade AI solutions.
AI Developer – Azure Databricks, Python, Java, Microservices & AI (LLM/Copilot or Equivalent )
Job Summary
We are looking for a AI Developer with strong expertise in Python, Azure Databricks (ADB), Delta Lake, and AI/LLM technologies to design and build Databricks jobs, reusable scripts, and cloud-native microservices. The role requires hands-on experience with Copilot agentic mode (or equivalent), building LLM-driven AI use cases, and the ability to work independently by taking full ownership of deliverables without the need for micromanagement.
Roles and Responsibilities
Azure Databricks & Data Engineering
- Design, develop, and manage Azure Databricks jobs, workflows, and notebooks.
- Build reusable Python/PySpark scripts for data ingestion, transformation, and analytics.
- Implement Delta Lake–based lakehouse architectures with ACID transactions and versioning.
- Optimize Databricks workloads for performance, scalability, and cost efficiency.
- Implement batch and streaming pipelines using Databricks.
Microservices & Application Development
- Develop Python-based microservices to expose data and AI capabilities via APIs.
- Build and integrate RESTful services for data access, AI inference, and orchestration.
- Ensure secure, scalable, and resilient service design using cloud-native patterns.
- Drive development tasks independently, from design through deployment, with minimal supervision.
AI, LLMs & Copilot Agentic Mode
- Design and build LLM-powered AI use cases such as copilots, intelligent assistants, and automation agents.
- Implement Copilot agentic mode workflows, including tool calling, task planning, and multi-step reasoning.
- Develop prompt engineering strategies, system prompts, and guardrails for enterprise use.
- Integrate LLMs with Databricks, data lakes, and microservices for real-time and batch inference.
- Build RAG (Retrieval-Augmented Generation) pipelines using enterprise data.
- Evaluate, fine-tune, and monitor LLM performance, accuracy, and cost.
Architecture, Governance & Best Practices
- Contribute to solution architecture and technical design decisions.
- Ensure data security, governance, and compliance in data and AI solutions.
- Implement logging, monitoring, and observability for Databricks jobs and AI services.
- Perform code reviews and enforce engineering best practices.
Leadership & Collaboration
- Mentor peer developers and provide technical guidance when needed.
- Collaborate effectively with data scientists, architects, DevOps teams, and business stakeholders.
- Clearly communicate technical concepts, progress, and risks through excellent verbal and written communication skills.
- Take full ownership of assigned workstreams and drive them to completion without requiring micromanagement.
Required Technologies & Skills
Programming & Frameworks
- Python (advanced)
- PySpark / Spark SQL
- REST APIs and microservices frameworks (FastAPI, Flask, etc.)
Azure & Data Platforms
- Azure Databricks (ADB)
- Delta Lake
- Azure Data Lake Storage (ADLS Gen2)
- Azure cloud services and security fundamentals
AI / LLM & Copilot
- Hands-on experience with LLMs (Azure OpenAI, OpenAI, or similar)
- Experience building Copilot solutions and agentic workflows
- Prompt engineering and orchestration frameworks (LangChain, Semantic Kernel, etc.)
- Experience with RAG architectures, vector databases, and embeddings
- Understanding of AI safety, guardrails, and responsible AI practices
DevOps & Engineering
- CI/CD pipelines (Azure DevOps, GitHub Actions)
- Git version control
- Containerization (Docker preferred)
- Infrastructure-as-code concepts (Terraform/Bicep – preferred)
Preferred Qualifications
- 8 to 10 years of experience in Python, data engineering, or backend development
- Proven experience delivering production-grade AI/LLM solutions
- Experience in enterprise-scale cloud environments
- Exposure to streaming platforms (Kafka, Event Hubs – preferred)
Soft Skills
- Excellent verbal and written communication skills
- Strong sense of ownership and accountability
- Ability to work independently and proactively drive outcomes
- Strong problem-solving and analytical skills
- Collaborative mindset with leadership capabilities
Similar Jobs
Artificial Intelligence • Big Data • Healthtech • Information Technology • Machine Learning • Software • Analytics
Leads the architecture, development, and production delivery of AI/ML capabilities in enterprise healthcare applications. Builds ML and generative AI solutions, RAG systems, model-serving APIs, MLOps workflows, distributed cloud services, and CI/CD pipelines. Ensures responsible AI, security, privacy, reliability, monitoring, and operational readiness. Influences architecture across teams, mentors engineers, documents technical decisions, and delivers reusable production components.
Top Skills:
APIsCi/CdCloud ComputingContainersDistributed SystemsGenerative AiInfrastructure As CodeLarge Language Models (Llms)Machine Learning FrameworksMachine Learning Operations (Mlops)Model RegistriesPythonRetrieval-Augmented Generation (Rag)Vector Search
Consumer Web • HR Tech
Design, build, and operate production-grade autonomous AI systems and multi-agent workflows. Develop agentic capabilities using OpenAI, Google Gemini, and Perplexity; implement orchestration, reflection, planning, evaluation, observability, and human-in-the-loop controls. Deploy scalable serverless services with Python and AWS, integrate with .NET systems, manage reliability and cost constraints, and create testing frameworks to monitor and improve agent quality.
Top Skills:
.NetAmazon EventbridgeAws LambdaAws Step FunctionsC#Claude CodeCursorGoogle GeminiOpenaiPerplexityPython
Artificial Intelligence • Fintech • Healthtech
Assist with designing, coding, testing, debugging, and deploying AI/ML features for healthcare and fintech applications. Support data preparation, model training, evaluation, and integration of LLM and GenAI APIs. Collaborate with senior developers, QA, and product teams, participate in code reviews, and write documented code under structured mentorship.
Top Skills:
JavaJavaScriptLlm ApisPythonPyTorchTensorFlow
What you need to know about the Chennai Tech Scene
To locals, it's no secret that South India is leading the charge in big data infrastructure. While the environmental impact of data centers has long been a concern, emerging hubs like Chennai are favored by companies seeking ready access to renewable energy resources, which provide more sustainable and cost-effective solutions. As a result, Chennai, along with neighboring Bengaluru and Hyderabad, is poised for significant growth, with a projected 65 percent increase in data center capacity over the next decade.
