Architects and engineers large-scale Databricks AI/ML infrastructure, including compute clusters, distributed training, data and ML pipelines, model serving, governance, CI/CD, observability, and cost optimization. Evaluates architecture alternatives, leads assessments, defines roadmaps and capacity plans, ensures security and compliance, and mentors engineers. Requires strong experience with Databricks, Python, SQL, Spark, Terraform, MLOps, cloud platforms, and production AI systems.
Project Role : AI Infrastructure Architect
Project Role Description : Architect and build custom Artificial Intelligence (AI) infrastructure/hardware solutions. Optimize AI infrastructure/hardware performance, power consumption, cost and scalability of computational stack. Advise on AI infrastructure technology and vendor evaluation, selection and full stack integration.
Must have skills : Machine Learning Operations
Good to have skills : Databricks Unified Data Analytics Platform
Minimum 12 year(s) of experience is required
Educational Qualification : 15 years full time education
Role Summary / Description
AI Powered Tech Talent
As a Senior Engineer in AI Infrastructure Architecture for Databricks, you will own significant portions of the end-to-end architecture and engineering of optimized lakehouse and AI infrastructure for large-scale machine learning systems. You will design scalable compute clusters, distributed training environments, ML pipelines, model-serving foundations, automation patterns and operational controls that align with client standards, SLAs, security, compliance and cost-efficiency expectations. You will bring industry experience across enterprise AI adoption, platform modernization, regulated data/AI workloads, FinOps and production reliability, while mentoring engineers and partnering with architects to translate business requirements into robust Databricks-based AI infrastructure solutions.
Key Responsibilities
Own end-to-end architecture and design of optimized Databricks AI/ML infrastructure, including workspace architecture, cluster/serverless compute, distributed training, ML pipelines, model registry, feature engineering and model-serving environments.
Design and tune scalable Databricks clusters, jobs, pipelines and serving endpoints using Databricks Workflows, MLflow, Model Registry, Unity Catalog, Delta Lake, Feature Engineering and cloud storage integrations, including compute selection, networking and high-throughput data access design.
Serve as an authoritative AI infrastructure expert on Databricks, applying deep knowledge of lakehouse architecture, ML lifecycle services, governance, security and cost levers.
Develop and evaluate architecture alternatives, weighing trade-offs across compute, networking, storage, orchestration, model serving, observability, security, compliance, cost and operational complexity.
Lead architecture assessments and reviews of existing and proposed environments, identifying gaps, risks, bottlenecks and optimization opportunities, and recommending remediation actions.
Drive architecture decision-making by documenting rationale, trade-offs, assumptions and dependencies so decisions are transparent, defensible and aligned with business SLAs and standards.
Define and maintain AI infrastructure roadmap inputs, capacity planning models, scaling strategies, cost forecasts and performance improvement opportunities.
Design deployment, automation and CI/CD strategies for reliable, repeatable and scalable releases of AI systems, models, data pipelines and platform components into production.
Establish AI monitoring and observability practices across InfraOps and MLOps, including SLAs, SLOs, alerting, performance/cost tracking and continuous optimization.
Integrate AI/ML systems into enterprise environments while ensuring interoperability, security, compliance, regulatory alignment and adherence to client standards.
Collaborate with clients, stakeholders, architects and engineering teams to align infrastructure decisions with business outcomes and translate requirements into actionable architecture standards.
Set technical direction for workstreams, mentor engineers, review designs/code and promote engineering best practices across the team.
Required Qualifications
Bachelor's degree in Computer Science, Computer Engineering, Information Technology or a related engineering field.
Minimum 4 years of experience coding, building, monitoring, troubleshooting, designing and operating AI/ML infrastructure, cloud platforms, data platforms, model deployment pipelines or large-scale engineering solutions.
Strong understanding of AI/ML concepts and the computing infrastructure required to deploy, run and optimize production AI workloads.
Minimum 4 years of proficiency in programming or scripting languages such as Python, Java, C++, Bash, PowerShell or equivalent engineering languages.
Experience with data pipeline and workflow management tools such as Apache Airflow, Kubeflow, managed orchestration services or platform-native workflow tooling.
Strong problem-solving skills and ability to work in a fast-paced engineering or client delivery environment.
Excellent communication, collaboration and stakeholder alignment skills.
Minimum 4 years of experience in AI/ML infrastructure engineering or related roles on a hyperscaler or enterprise platform for deploying large-scale solutions.
Proven experience leading AI projects or engineering workstreams and managing priorities across multiple initiatives.
Demonstrated experience evaluating and selecting AI technologies, frameworks, cloud services and architecture patterns.
Required Skills/ Experience
Strong hands-on experience with Databricks workspaces, clusters/serverless compute, Workflows, jobs, notebooks, MLflow, Model Registry, Unity Catalog, Delta Lake, Feature Engineering and model-serving capabilities.
Experience architecting scalable data and ML pipelines, distributed processing/training workloads, model-serving patterns, high-throughput data access, governance and secure cloud integrations.
Strong working knowledge of Python, SQL, Spark, Terraform/Databricks Asset Bundles, Git-based CI/CD, DataOps, MLOps, observability and incident response practices.
Ability to optimize Databricks AI infrastructure for performance, cost, scalability, security, reliability and compliance.
Experience producing architecture decision records, reference implementations, standards, runbooks and reusable platform patterns.
Good to Have Skills
Databricks certifications such as Machine Learning Professional, Data Engineer Professional or related lakehouse architecture credentials.
Industry experience in BFSI, healthcare, retail/e-commerce, telecom, manufacturing, energy or public sector environments where data/AI platforms must meet compliance, security, reliability and cost-control requirements.
Exposure to LLMOps, vector search, retrieval pipelines, feature stores, GPU-backed training, low-latency model serving and model optimization techniques.
Knowledge of Unity Catalog governance, FinOps, infrastructure partner/vendor collaboration and production support operating models.
15 years full time education
Project Role Description : Architect and build custom Artificial Intelligence (AI) infrastructure/hardware solutions. Optimize AI infrastructure/hardware performance, power consumption, cost and scalability of computational stack. Advise on AI infrastructure technology and vendor evaluation, selection and full stack integration.
Must have skills : Machine Learning Operations
Good to have skills : Databricks Unified Data Analytics Platform
Minimum 12 year(s) of experience is required
Educational Qualification : 15 years full time education
Role Summary / Description
AI Powered Tech Talent
As a Senior Engineer in AI Infrastructure Architecture for Databricks, you will own significant portions of the end-to-end architecture and engineering of optimized lakehouse and AI infrastructure for large-scale machine learning systems. You will design scalable compute clusters, distributed training environments, ML pipelines, model-serving foundations, automation patterns and operational controls that align with client standards, SLAs, security, compliance and cost-efficiency expectations. You will bring industry experience across enterprise AI adoption, platform modernization, regulated data/AI workloads, FinOps and production reliability, while mentoring engineers and partnering with architects to translate business requirements into robust Databricks-based AI infrastructure solutions.
Key Responsibilities
Own end-to-end architecture and design of optimized Databricks AI/ML infrastructure, including workspace architecture, cluster/serverless compute, distributed training, ML pipelines, model registry, feature engineering and model-serving environments.
Design and tune scalable Databricks clusters, jobs, pipelines and serving endpoints using Databricks Workflows, MLflow, Model Registry, Unity Catalog, Delta Lake, Feature Engineering and cloud storage integrations, including compute selection, networking and high-throughput data access design.
Serve as an authoritative AI infrastructure expert on Databricks, applying deep knowledge of lakehouse architecture, ML lifecycle services, governance, security and cost levers.
Develop and evaluate architecture alternatives, weighing trade-offs across compute, networking, storage, orchestration, model serving, observability, security, compliance, cost and operational complexity.
Lead architecture assessments and reviews of existing and proposed environments, identifying gaps, risks, bottlenecks and optimization opportunities, and recommending remediation actions.
Drive architecture decision-making by documenting rationale, trade-offs, assumptions and dependencies so decisions are transparent, defensible and aligned with business SLAs and standards.
Define and maintain AI infrastructure roadmap inputs, capacity planning models, scaling strategies, cost forecasts and performance improvement opportunities.
Design deployment, automation and CI/CD strategies for reliable, repeatable and scalable releases of AI systems, models, data pipelines and platform components into production.
Establish AI monitoring and observability practices across InfraOps and MLOps, including SLAs, SLOs, alerting, performance/cost tracking and continuous optimization.
Integrate AI/ML systems into enterprise environments while ensuring interoperability, security, compliance, regulatory alignment and adherence to client standards.
Collaborate with clients, stakeholders, architects and engineering teams to align infrastructure decisions with business outcomes and translate requirements into actionable architecture standards.
Set technical direction for workstreams, mentor engineers, review designs/code and promote engineering best practices across the team.
Required Qualifications
Bachelor's degree in Computer Science, Computer Engineering, Information Technology or a related engineering field.
Minimum 4 years of experience coding, building, monitoring, troubleshooting, designing and operating AI/ML infrastructure, cloud platforms, data platforms, model deployment pipelines or large-scale engineering solutions.
Strong understanding of AI/ML concepts and the computing infrastructure required to deploy, run and optimize production AI workloads.
Minimum 4 years of proficiency in programming or scripting languages such as Python, Java, C++, Bash, PowerShell or equivalent engineering languages.
Experience with data pipeline and workflow management tools such as Apache Airflow, Kubeflow, managed orchestration services or platform-native workflow tooling.
Strong problem-solving skills and ability to work in a fast-paced engineering or client delivery environment.
Excellent communication, collaboration and stakeholder alignment skills.
Minimum 4 years of experience in AI/ML infrastructure engineering or related roles on a hyperscaler or enterprise platform for deploying large-scale solutions.
Proven experience leading AI projects or engineering workstreams and managing priorities across multiple initiatives.
Demonstrated experience evaluating and selecting AI technologies, frameworks, cloud services and architecture patterns.
Required Skills/ Experience
Strong hands-on experience with Databricks workspaces, clusters/serverless compute, Workflows, jobs, notebooks, MLflow, Model Registry, Unity Catalog, Delta Lake, Feature Engineering and model-serving capabilities.
Experience architecting scalable data and ML pipelines, distributed processing/training workloads, model-serving patterns, high-throughput data access, governance and secure cloud integrations.
Strong working knowledge of Python, SQL, Spark, Terraform/Databricks Asset Bundles, Git-based CI/CD, DataOps, MLOps, observability and incident response practices.
Ability to optimize Databricks AI infrastructure for performance, cost, scalability, security, reliability and compliance.
Experience producing architecture decision records, reference implementations, standards, runbooks and reusable platform patterns.
Good to Have Skills
Databricks certifications such as Machine Learning Professional, Data Engineer Professional or related lakehouse architecture credentials.
Industry experience in BFSI, healthcare, retail/e-commerce, telecom, manufacturing, energy or public sector environments where data/AI platforms must meet compliance, security, reliability and cost-control requirements.
Exposure to LLMOps, vector search, retrieval pipelines, feature stores, GPU-backed training, low-latency model serving and model optimization techniques.
Knowledge of Unity Catalog governance, FinOps, infrastructure partner/vendor collaboration and production support operating models.
15 years full time education
About Accenture
Accenture is a leading global professional services company that helps the world’s leading businesses, governments and other organizations build their digital core, optimize their operations, accelerate revenue growth and enhance citizen services—creating tangible value at speed and scale. We are a talent- and innovation-led company with approximately 791,000 people serving clients in more than 120 countries. Technology is at the core of change today, and we are one of the world’s leaders in helping drive that change, with strong ecosystem relationships. We combine our strength in technology and leadership in cloud, data and AI with unmatched industry experience, functional expertise and global delivery capability. Our broad range of services, solutions and assets across Strategy & Consulting, Technology, Operations, Industry X and Song, together with our culture of shared success and commitment to creating 360° value, enable us to help our clients reinvent and build trusted, lasting relationships. We measure our success by the 360° value we create for our clients, each other, our shareholders, partners and communities.Visit us at www.accenture.com
Equal Employment Opportunity Statement
We believe that no one should be discriminated against because of their differences. All employment decisions shall be made without regard to age, race, creed, color, religion, sex, national origin, ancestry, disability status, military veteran status, sexual orientation, gender identity or expression, genetic information, marital status, citizenship status or any other basis as protected by applicable law. Our rich diversity makes us more innovative, more competitive, and more creative, which helps us better serve our clients and our communities.
Accenture Chennai, Tamil Nadu, IND Office
Chennai, India
Similar Jobs
Information Technology
Designs and builds enterprise-grade GenAI and LLM infrastructure on Azure, including agents, RAG pipelines, orchestration, integrations, evaluation, observability, security, and governance components. Develops production-quality Python software, implements data ingestion and retrieval workflows, and optimizes performance, scalability, reliability, and cost. Collaborates with architects, data engineers, product owners, and security teams to deliver maintainable AI solutions and reusable engineering patterns.
Top Skills:
Api ManagementApplication InsightsAzure Ai FoundryAzure Ai SearchAzure FunctionsAzure Key VaultAzure MonitorAzure OpenaiCi/CdDefender For CloudDockerEvent GridFastapiFoundry Agent ServiceGitHaystackInfrastructure As CodeKubernetesLangchainLanggraphLlamaindexLogic AppsAzureMicrosoft Entra IdMicrosoft FabricMlflowPrompt FlowPythonRagSemantic KernelVector Databases
Information Technology
Designs and builds enterprise-grade AI infrastructure and LLM application components on Azure. Responsibilities include developing RAG pipelines, agents, orchestration, retrieval workflows, safety controls, APIs, integrations, monitoring, and evaluation systems. The role translates architecture into production-quality software, implements secure and scalable AI platforms, collaborates with technical and security stakeholders, and leads engineering practices including testing, CI/CD, observability, documentation, and governance.
Top Skills:
Agentic AiApi ManagementApplication InsightsAzure Ai SearchAzure FunctionsAzure Key VaultAzure MonitorAzure OpenaiCi/CdDockerEmbeddingsEvent GridFastapiFoundry Agent ServiceGenerative AiGitHaystackInfrastructure As CodeKubernetesLangchainLanggraphLlamaindexLlmsLogic AppsAzureMicrosoft Azure Ai FoundryMicrosoft Entra IdMicrosoft FabricMlflowPrompt FlowPythonRagSemantic KernelVector Databases
Information Technology
Lead enterprise AI architecture across generative AI, LLM applications, agentic systems, data, models, inference, security, governance, observability and FinOps. Define Azure AI target architectures, platform strategies, technology selections and transformation roadmaps. Guide domain architects and senior engineers while advising executives on AI investments, regulatory controls, responsible AI, enterprise integration, operational readiness and measurable business outcomes.
Top Skills:
Api ManagementApplication InsightsAzure Ai SearchAzure Defender For CloudAzure FunctionsAzure Key VaultAzure MonitorAzure OpenaiDockerEvent GridFastapiFoundry Agent ServiceGenerative AiHaystackKubernetesLangchainLanggraphLlamaindexLogic AppsAzureMicrosoft Azure Ai FoundryMicrosoft Entra IdMicrosoft FabricMlflowPrompt FlowSemantic Kernel
What you need to know about the Chennai Tech Scene
To locals, it's no secret that South India is leading the charge in big data infrastructure. While the environmental impact of data centers has long been a concern, emerging hubs like Chennai are favored by companies seeking ready access to renewable energy resources, which provide more sustainable and cost-effective solutions. As a result, Chennai, along with neighboring Bengaluru and Hyderabad, is poised for significant growth, with a projected 65 percent increase in data center capacity over the next decade.
