Presidio Logo

Presidio

Lead Engineer - L3 Observability

Posted 29 Days Ago
Be an Early Applicant
Hybrid
Chennai, Tamil Nadu, IND
Senior level
Hybrid
Chennai, Tamil Nadu, IND
Senior level
Serves as the L3 technical authority for enterprise observability, resolving complex cross-domain incidents and leading root cause analysis. Owns Datadog architecture, monitoring standards, SLOs, dashboards, instrumentation, platform governance, cost optimization, and automation. Defines observability coverage across applications, infrastructure, networks, and cloud-native environments. Leads major incident response, post-incident improvements, capacity and performance insights, stakeholder governance, and mentoring of L1/L2 engineers.
The summary above was generated by AI

Presidio, Where Teamwork and Innovation Shape the Future
At Presidio, we're at the forefront of a global technology revolution, transforming industries through cutting-edge digital solutions and next-generation AI. We empower businesses - and their internal customers - to achieve more through innovation, automation, and intelligent insights.

The Role

Level 3 Managed Service Engineer serves as the senior escalation point and technical authority for end-to-end observability across the environment. This role is responsible for resolving or driving resolution of highly complex incidents spanning application, infrastructure, network, and third-party dependencies, while owning observability strategy, Datadog architecture, monitoring standards, and continuous improvement. The engineer is expected to guide L1 and L2 engineers, lead root cause analysis and major incident reviews, and partner with cross-functional teams to improve detection, reduce noise, and strengthen operational resilience for both 3-tier and cloud-native applications. 

Key Responsibilities

  • Serve as the final escalation and technical authority for complex, cross-domain incidents involving application, infrastructure, network, and third-party dependencies.
  • Own the end-to-end observability architecture and strategy in Datadog, including instrumentation standards, tagging strategy, dashboard governance, and monitor design principles.
  • Lead root cause analysis for major/critical incidents and drive post-incident reviews, ensuring permanent fixes and preventive monitoring are implemented.
  • Define and continuously improve monitoring coverage for new and existing services, including 3-tier applications, microservices, and cloud-native workloads.
  • Design and govern SLOs/SLIs aligned to business-critical services, and advise application and platform teams on instrumentation best practices.
  • Drive automation initiatives: Datadog-as-code (Terraform/API-based configuration management), auto-remediation workflows, and integration with CI/CD pipelines.
  • Partner with application, infrastructure, network, and security teams to close monitoring gaps and reduce mean time to detect (MTTD) and mean time to resolve (MTTR).
  • Own capacity planning and performance engineering insights derived from long-term observability data trends.
  • Manage Datadog platform health: cost optimization, license/usage governance, data retention, and integration management.
  • Mentor L1 and L2 engineers, review their RCA quality, and build enablement material (runbooks, training, best practices).
  • Act as a key stakeholder in client governance calls, presenting trends, risks, and improvement roadmaps.
  • Evaluate and pilot new Datadog features or complementary observability tools, and lead their rollout.
  • Define escalation matrices and coordinate cross-team response during major outages.

Required Skills and Experience

  • 6+ years of experience in Managed Services, SRE, or platform engineering roles, with at least 3 years of hands-on ownership of a Datadog (or equivalent) observability platform at scale.
  • Expert-level knowledge of Datadog, including:
  • Platform architecture: organization/team structuring, RBAC, tagging strategy, and multi-environment setups.
  • APM: distributed tracing at scale, service dependency mapping, and instrumentation strategy across polyglot environments.
  • Synthetics and RUM: designing comprehensive digital experience monitoring strategies tied to business KPIs.
  • Log Management: pipeline architecture, log-based metrics, and cost/volume governance.
  • SLOs, error budgets, and anomaly/forecast-based alerting at an advanced level.
  • Deep understanding of infrastructure and cloud architecture: compute, containers, orchestration (Kubernetes), auto-scaling, and multi-cloud/hybrid environments.
  • Deep understanding of networking: routing, load balancing, CDN, DNS, and complex connectivity troubleshooting across hybrid environments.
  • Strong incident and problem management leadership experience, including major incident command and post-incident review facilitation.
  • Experience defining and governing observability standards across multiple teams or business units.

Application & Architecture Understanding (technology-agnostic)

  • Expert-level understanding of 3-tier and N-tier application architectures, including how failures cascade across presentation, application, and data layers.
  • Expert-level understanding of cloud-native and distributed architectures: microservices, event-driven systems, service meshes, API gateways, and asynchronous messaging.
  • Ability to advise application teams on observability best practices (instrumentation, tracing context propagation, structured logging) without needing to write code in a specific language.
  • Strong grasp of resiliency and failure patterns (retries, circuit breakers, backpressure, caching, graceful degradation) and how to detect their failure modes through observability data.

Good to Have

  • Datadog advanced/expert certifications, or equivalent recognized observability certifications.
  • Strong scripting/automation background (Python, Terraform, or similar) for Datadog-as-code and custom tooling.
  • Experience with FinOps or cost governance for observability tooling at scale.
  • Exposure to multiple observability ecosystems (New Relic, Dynatrace, Splunk, Grafana/Prometheus, OpenTelemetry).
  • Experience presenting to senior stakeholders or clients on observability maturity and roadmaps.

Soft Skills

  • Strategic thinking with the ability to balance immediate incident needs against long-term platform health.
  • Strong leadership and mentoring capability across L1/L2 engineers.
  • Excellent stakeholder management and executive communication skills.
  • Calm, decisive leadership during major incidents.
  • Strong documentation and process-governance discipline.

Education

  • Bachelor's or Master's degree in Computer Science, Information Technology, Electronics, or a related field (or equivalent practical experience).

Work Details

  • Level: L3 (Senior/Lead, Managed Services)
  • Shift: Primarily general shift, with on-call responsibility for major incident escalations
  • Reporting To: Observability Practice Lead / Delivery Manager

Your future at Presidio
Joining Presidio means stepping into a culture of trailblazers - thinkers, builders, and collaborators - who push the boundaries of what's possible. With our expertise AI-driven analytics, cloud solutions, cybersecurity, and next-gen infrastructure, we enable businesses to stay ahead in an ever-evolving digital world.

Here, your impact is real. Whether you're harnessing the power of Generative AI, architecting resilient digital ecosystems, or driving data-driven transformation, you'll be part of a team that is shaping the future.

Ready to innovate? Let's redefine what's next-together.


About Presidio
Presidio is committed to hiring the most qualified candidates to join our amazing culture. We aim to attract and hire top talent from all backgrounds, including underrepresented and marginalized communities. We encourage women, people of color, people with disabilities, and veterans to apply for open roles at Presidio. Diversity of skills and thought is a key component to our business success.

At Presidio, speed and quality meet technology and innovation. Presidio is a trusted ally for organizations across industries with a decades-long history of building traditional IT foundations and deep expertise in AI and automation, security, networking, digital transformation, and cloud computing. Presidio fills gaps, removes hurdles, optimizes costs, and reduces risk. Presidio's expert technical team develops custom applications, provides managed services, and enables actionable data insights and builds forward-thinking solutions that drive strategic outcomes for clients globally. For more information visit https://presidio.com

 Applications will be accepted on a rolling basis.

Presidio has a strong commitment to the community we serve and our employees. As an Equal Opportunity Employer, we strive to have a workforce that includes the community we serve.

 Recruitment Agencies, Please Note: Presidio does not accept unsolicited agency resumes/CVs. Do not forward resumes/CVs to our career's email address, Presidio employees or any other means. Presidio is not responsible for any feeds related to unsolicited resumes/CVs.

Similar Jobs

Yesterday
In-Office
Chennai, Tamil Nadu, IND
Senior level
Senior level
Artificial Intelligence • Big Data • Healthtech • Information Technology • Machine Learning • Software • Analytics
Owns technical product execution for CareSelect Imaging by translating strategy and customer needs into capabilities, features, user stories, acceptance criteria, and prioritized backlogs. Partners with distributed Engineering, Architecture, QA, Product, and Operations teams to manage dependencies, risks, planning, development, testing, releases, and validation. Communicates priorities and delivery status while improving requirements quality and predictability.
Top Skills: AhaAzure DevopsDevOpsFhirHl7JIRARally
Yesterday
In-Office
Chennai, Tamil Nadu, IND
Senior level
Senior level
Cloud • Fintech • Food • Information Technology • Software • Hospitality
Provides technical customer support to restaurant clients through chat, voice, and casework channels. Resolves hardware and software issues, explains technical concepts clearly, manages multiple interactions, meets SLA and KPI targets, collaborates with internal teams, and contributes to process improvements. Requires rotational 24/7 availability, including nights, weekends, and holidays, with initial office work in Chennai.
Top Skills: Chat/Ticketing SystemsPos SystemsSaaS
Yesterday
In-Office
Chennai, Tamil Nadu, IND
Junior
Junior
Artificial Intelligence • Cloud • Information Technology • Consulting
Design and test engineering solutions for software, systems, and integrated products. Collaborate with internal and external development partners, validate technical specifications, and contribute to cross-functional teams delivering reliable, cost-effective solutions. The role requires basic knowledge of hardware design, electrical components, engineering tools, and material properties, along with strong analytical, communication, and teamwork skills.
Top Skills: Electrical ComponentsEngineering Design SoftwareHardware Design

What you need to know about the Chennai Tech Scene

To locals, it's no secret that South India is leading the charge in big data infrastructure. While the environmental impact of data centers has long been a concern, emerging hubs like Chennai are favored by companies seeking ready access to renewable energy resources, which provide more sustainable and cost-effective solutions. As a result, Chennai, along with neighboring Bengaluru and Hyderabad, is poised for significant growth, with a projected 65 percent increase in data center capacity over the next decade.

Sign up now Access later

Create Free Account

Please log in or sign up to report this job.

Create Free Account