Lead development of automation frameworks for AI/ML applications, ensuring reliability and scalability of LLM-based products while optimizing CI/CD integrations and performance testing.
About the Company:
Netomi is the leading agentic AI platform for enterprise customer experience. We work with the largest global brands like Delta Airlines, MetLife, MGM, United, and others to enable agentic automation at scale across the entire customer journey. Our no-code platform delivers the fastest time to market, lowest total cost of ownership, and simple, scalable management of AI agents for any CX use case. Backed by WndrCo, Y Combinator, and Index Ventures, we help enterprises drive efficiency, lower costs, and deliver higher quality customer experiences.
Want to be part of the AI revolution and transform how the world’s largest global brands do business? Join us!
We’re seeking a Senior SDET with expertise in Generative AI testing to lead the development of cutting-edge automation frameworks for AI/ML-powered applications. You’ll ensure the reliability, safety, and scalability of LLM-driven products while advancing traditional test automation for cloud-native systems.
Responsibilities
- AI-Aware Test Automation - Design and maintain Python/Java-based automation frameworks (Selenium, Playwright, TestNG/JUnit) for web, API, and backend services.
- Extend frameworks to test LLM integrations (OpenAI, HuggingFace, RAG pipelines) with prompt validation, hallucination checks, and response consistency tests.
- Implement model benchmarking (latency, accuracy, bias/drift detection) for generative AI features.
- Quality Infrastructure - Integrate tests into CI/CD pipelines (Jenkins, GitHub Actions) with cloud workflows (AWS/GCP).
- Optimize performance testing (JMeter/Locust) for AI endpoints handling high-throughput inference.
- Debug flaky tests in (non-deterministic) AI systems.
- Leadership & Innovation - Mentor junior engineers on AI testing best practices.
- Research tools like LangChain, synthetic data generators, or adversarial testing techniques.
- Advocate for ML-specific quality metrics beyond traditional pass/fail.
Requirements
- 7–9 years in QA automation with strong Python/Java proficiency.
- Hands-on experience with Selenium, Playwright, REST Assured, and CI/CD tools (Jenkins, Docker).
- Solid understanding of SQL/NoSQL databases and cloud platforms (AWS/GCP).
- Exposure to performance testing (JMeter, K6) and scalable test frameworks.
- Experience with LLM testing (prompt engineering, output validation, rubric-based grading).
- Familiarity with OpenAI APIs, HuggingFace, or LangChain.
- Knowledge of synthetic test data generation for edge-case scenarios.
- Autonomy – Thrive in fast-paced, AI-driven environments with minimal supervision.
- Analytical Mindset – Debug complex failures in probabilistic AI systems.
- Communication - Explain technical trade-offs to non-technical stakeholders.
Netomi is an equal opportunity employer committed to diversity in the workplace. We evaluate qualified applicants without regard to race, color, religion, sex, sexual orientation, disability, veteran status, and other protected characteristics.
Top Skills
AWS
GCP
Github Actions
Huggingface
Java
Jenkins
Jmeter
Junit
K6
Langchain
NoSQL
Openai Apis
Playwright
Python
Rest Assured
Selenium
SQL
Testng
Similar Jobs
Security • Cybersecurity
The Technical Account Manager will engage clients to ensure product adoption, provide technical guidance, resolve challenges, and collaborate cross-functionally to enhance customer satisfaction and product usage metrics.
Top Skills:
Check PointCiscoCloud SecurityFirewall ManagementFortinetJuniperNetwork SecurityPalo Alto NetworksSecurity Policy Automation
Cloud • Information Technology • Security • Software • Cybersecurity
The role involves managing technical sales for Cloudflare's partners, designing solutions, and ensuring successful collaborations while engaging in high-level technical discussions.
Top Skills:
AWSAzureBgpCdnDdos MitigationDlpDnsFirewallsGCPGreHttp TechnologiesMplsSaseSd-WanTcpTlsUdpVpnWafZero Trust Networking
Cloud • Security • Software • Cybersecurity • Automation
Lead the development of Ruby on Rails backend components, optimize workflows, mentor team members, and ensure system reliability and performance.
Top Skills:
AWSCi/CdClickhouseGCPGoGrafanaKubernetesPostgresPrometheusRuby On Rails
What you need to know about the Chennai Tech Scene
To locals, it's no secret that South India is leading the charge in big data infrastructure. While the environmental impact of data centers has long been a concern, emerging hubs like Chennai are favored by companies seeking ready access to renewable energy resources, which provide more sustainable and cost-effective solutions. As a result, Chennai, along with neighboring Bengaluru and Hyderabad, is poised for significant growth, with a projected 65 percent increase in data center capacity over the next decade.
