Build and manage Azure Databricks jobs, lakehouse pipelines, reusable Python/PySpark scripts, and cloud-native microservices. Develop LLM-powered copilots, agentic workflows, RAG pipelines, prompt strategies, guardrails, and AI inference services. Integrate enterprise data with AI solutions, optimize performance and costs, implement governance and observability, and contribute to architecture and technical design. The role also involves independent ownership, code reviews, mentoring, stakeholder collaboration, and delivering production-grade AI solutions.
AI Developer – Azure Databricks, Python, Java, Microservices & AI (LLM/Copilot or Equivalent )
Job Summary
We are looking for a AI Developer with strong expertise in Python, Azure Databricks (ADB), Delta Lake, and AI/LLM technologies to design and build Databricks jobs, reusable scripts, and cloud-native microservices. The role requires hands-on experience with Copilot agentic mode (or equivalent), building LLM-driven AI use cases, and the ability to work independently by taking full ownership of deliverables without the need for micromanagement.
Roles and Responsibilities
Azure Databricks & Data Engineering
- Design, develop, and manage Azure Databricks jobs, workflows, and notebooks.
- Build reusable Python/PySpark scripts for data ingestion, transformation, and analytics.
- Implement Delta Lake–based lakehouse architectures with ACID transactions and versioning.
- Optimize Databricks workloads for performance, scalability, and cost efficiency.
- Implement batch and streaming pipelines using Databricks.
Microservices & Application Development
- Develop Python-based microservices to expose data and AI capabilities via APIs.
- Build and integrate RESTful services for data access, AI inference, and orchestration.
- Ensure secure, scalable, and resilient service design using cloud-native patterns.
- Drive development tasks independently, from design through deployment, with minimal supervision.
AI, LLMs & Copilot Agentic Mode
- Design and build LLM-powered AI use cases such as copilots, intelligent assistants, and automation agents.
- Implement Copilot agentic mode workflows, including tool calling, task planning, and multi-step reasoning.
- Develop prompt engineering strategies, system prompts, and guardrails for enterprise use.
- Integrate LLMs with Databricks, data lakes, and microservices for real-time and batch inference.
- Build RAG (Retrieval-Augmented Generation) pipelines using enterprise data.
- Evaluate, fine-tune, and monitor LLM performance, accuracy, and cost.
Architecture, Governance & Best Practices
- Contribute to solution architecture and technical design decisions.
- Ensure data security, governance, and compliance in data and AI solutions.
- Implement logging, monitoring, and observability for Databricks jobs and AI services.
- Perform code reviews and enforce engineering best practices.
Leadership & Collaboration
- Mentor peer developers and provide technical guidance when needed.
- Collaborate effectively with data scientists, architects, DevOps teams, and business stakeholders.
- Clearly communicate technical concepts, progress, and risks through excellent verbal and written communication skills.
- Take full ownership of assigned workstreams and drive them to completion without requiring micromanagement.
Required Technologies & Skills
Programming & Frameworks
- Python (advanced)
- PySpark / Spark SQL
- REST APIs and microservices frameworks (FastAPI, Flask, etc.)
Azure & Data Platforms
- Azure Databricks (ADB)
- Delta Lake
- Azure Data Lake Storage (ADLS Gen2)
- Azure cloud services and security fundamentals
AI / LLM & Copilot
- Hands-on experience with LLMs (Azure OpenAI, OpenAI, or similar)
- Experience building Copilot solutions and agentic workflows
- Prompt engineering and orchestration frameworks (LangChain, Semantic Kernel, etc.)
- Experience with RAG architectures, vector databases, and embeddings
- Understanding of AI safety, guardrails, and responsible AI practices
DevOps & Engineering
- CI/CD pipelines (Azure DevOps, GitHub Actions)
- Git version control
- Containerization (Docker preferred)
- Infrastructure-as-code concepts (Terraform/Bicep – preferred)
Preferred Qualifications
- 8 to 10 years of experience in Python, data engineering, or backend development
- Proven experience delivering production-grade AI/LLM solutions
- Experience in enterprise-scale cloud environments
- Exposure to streaming platforms (Kafka, Event Hubs – preferred)
Soft Skills
- Excellent verbal and written communication skills
- Strong sense of ownership and accountability
- Ability to work independently and proactively drive outcomes
- Strong problem-solving and analytical skills
- Collaborative mindset with leadership capabilities
Similar Jobs
Information Technology
Designs, develops, deploys, and monitors AI and machine learning solutions, including predictive models, NLP applications, LLMs, generative AI, RAG systems, copilots, chatbots, and virtual assistants. Responsibilities include building ML pipelines, creating APIs and microservices, integrating AI with enterprise and cloud platforms, applying MLOps practices, and ensuring model scalability, security, governance, compliance, and reliability.
Top Skills:
Ai/MlAPIsCloud PlatformsGenerative AiLarge Language Models (Llms)MicroservicesMlopsNatural Language Processing (Nlp)Retrieval-Augmented Generation (Rag)
Retail • Sports
Build and operate production-grade LLM applications, agentic workflows, RAG and retrieval pipelines, AI memory architectures, and evaluation harnesses. Deploy and monitor AI services using cloud infrastructure, containers, APIs, and MLOps platforms. Configure AI gateways, manage model routing and costs, secure integrations, and optimize serving architectures. Collaborate with product, engineering, SRE, security, and software teams to deliver scalable AI capabilities.
Top Skills:
AWSAzureCi/CdCpusDeepevalDockerEmbedding ModelsGCPGpusGraph DatabasesGraphQLGrpcIamIn-Memory DatabasesJavaKubernetesLangchainLanggraphLlamaindexMilvusMlflowModel Context ProtocolPgvectorPineconePythonQdrantRagRagasRestRustTypescriptVector DatabasesWeights & Biases
Artificial Intelligence • Information Technology • Software • Analytics
Design, build, deploy, and govern production AI and GenAI solutions using Azure. Develop RAG, NLP, document intelligence, classification, and prediction applications; implement evaluation, explainability, safety, secure APIs, MLOps, compliance, and audit controls. Collaborate with data and governance teams, prepare AI-ready datasets, document solutions, and coach client counterparts on sustainable AI engineering practices.
Top Skills:
Azure Ai FoundryAzure Ai SearchAzure Machine LearningAzure OpenaiCi/CdDockerFastapiHugging FaceKubernetesLangchainMicrosoft FabricMicrosoft PurviewPythonPyTorchRestScikit-LearnSemantic KernelVector Databases
What you need to know about the Chennai Tech Scene
To locals, it's no secret that South India is leading the charge in big data infrastructure. While the environmental impact of data centers has long been a concern, emerging hubs like Chennai are favored by companies seeking ready access to renewable energy resources, which provide more sustainable and cost-effective solutions. As a result, Chennai, along with neighboring Bengaluru and Hyderabad, is poised for significant growth, with a projected 65 percent increase in data center capacity over the next decade.
