ABC Fitness Solutions Logo

ABC Fitness Solutions

AI Quality Assurance Lead

Job Posted 25 Days Ago Posted 25 Days Ago
Be an Early Applicant
In-Office
Hyderabad, Telangana
Senior level
In-Office
Hyderabad, Telangana
Senior level
Lead the AI Quality Assurance team in designing testing systems for generative AI, ensuring accuracy, safety, and performance standards in AI features.
The summary above was generated by AI
-

It's fun to work in a company where people truly BELIEVE in what they're doing!


We're committed to bringing passion and customer focus to the business.

Job Description

AI Quality Assurance Lead (Evaluation & Testing)

This role requires working from our local Hyderabad office 2-3x a week.
Location: Hyderabad, Telangana, India

ABOUT THE TEAM

The Generative AI Quality & Safety team owns ABC Fitness’s evaluation frameworks, testing pipelines, and compliance tooling for AI-driven features. We partner with product, engineering, and legal teams to ensure every LLM interaction meets rigorous standards for accuracy, safety, and performance. As our AI Quality Assurance Lead, you’ll architect hybrid (automated + human) testing systems, define GenAI quality KPIs, and embed Responsible AI principles across our fitness-tech platform.

At ABC Fitness, we love entrepreneurs because we are entrepreneurs. We know how much grit it takes to start your own business and grow it into something that lasts. We roll our sleeves up, we act fast, and we learn together.

WHAT YOU’LL DO

  • Design and deploy evaluation pipelines for generative AI systems using tools like OpenAI Evals, Promptfoo, and custom test harnesses.
  • Develop hallucination detection workflows and bias-analysis frameworks for LLM outputs across 10+ languages.
  • Partner with AI researchers to translate model capabilities into testable requirements for product teams.
  • Implement CI/CD-integrated regression testing for AI microservices on AWS/Azure, monitoring model drift and performance degradation.
  • Lead bug triage sessions, prioritizing issues impacting user trust, legal compliance, or revenue.
  • Document QA protocols, failure modes, and root-cause analyses in our internal knowledge base.

WHAT YOU’LL NEED

  • 7+ years in QA/testing roles, with 3+ years focused on AI/ML systems (LLMs, recommendation engines, or conversational AI).
  • Hands-on experience with GenAI evaluation tools (LangSmith, Weights & Biases) and statistical analysis (Python, SQL).
  • Proficiency in cloud platforms (AWS SageMaker, Azure ML) and containerized testing environments (Docker, Kubernetes).
  • Deep understanding of Responsible AI principles—fairness, transparency, privacy—and adversarial testing methodologies.
  • Ability to mentor junior engineers and communicate technical risks to non-technical stakeholders.
  • Certifications like AWS Certified Machine Learning Specialty or Microsoft AI Engineer are a plus.

WHAT’S IN IT FOR YOU:  

  • Purpose led company with a Values focused culture – Best Life, One Team, Growth Mindset 
  • Time Off – competitive PTO plans with 15 Earned accrued leave, 12 days Sick leave, and 12 days Casual leave per year 
  • 11 Holidays plus 4 Days of Disconnect – once a quarter, we take a collective breather and enjoy a day off together around the globe. #oneteam 
  • Group Mediclaim insurance coverage of INR 500,000 for employee + spouse, 2 kids, and parents or parent-in-laws, and including EAP counseling 
  • Life Insurance and Personal Accident Insurance 
  • Best Life Perk – we are committed to meeting you wherever you are in your fitness journey with a quarterly reimbursement 
  • Premium Calm App – enjoy tranquility with a Calm App subscription for you and up to 4 dependents over the age of 16 
  • Support for working women with financial aid towards crèche facility, ensuring a safe and nurturing environment for their little ones while they focus on their careers. 

We’re committed to diversity and passion, and encourage you to apply, even if you don’t demonstrate all the listed skillsets! 

ABC’S COMMITMENT TO DIVERSITY, EQUALITY, BELONGING AND INCLUSION: 
ABC is an equal opportunity employer. We celebrate diversity and are committed to creating an inclusive environment for all employees.  We are intentional about creating an environment where employees, our clients and other stakeholders feel valued and inspired to reach their full potential and make authentic connections. We foster a workplace culture that embraces each person’s diversity, including the extent to which they are similar or different. ABC leaders believe that an equitable and inclusive culture is not only the right thing to do, it is a business imperative. Read more about our commitment to diversity, equality, belonging and inclusion at abcfitness.com  

ABOUT ABC: 

ABC Fitness (abcfitness.com) is the premier provider of software and related services for the fitness industry and has built a reputation for excellence in support for clubs and their members. ABC is the trusted provider to boost performance and create a total fitness experience for over 41 million members of clubs of all sizes whether a multi-location chain, franchise or an independent gym. 

 
Founded in 1981, ABC helps over 31,000 gyms and health clubs globally perform better and more profitably offering a comprehensive SaaS club management solution that enables club operators to achieve optimal performance. ABC Fitness is a Thoma Bravo portfolio company, a private equity firm focused on investing in software and technology companies (thomabravo.com). 

#LI-HYBRID 

If you like wild growth and working with happy, enthusiastic over-achievers, you'll enjoy your career with us!

Top Skills

AWS
Aws Sagemaker
Azure
Azure Ml
Docker
Kubernetes
Langsmith
Openai Evals
Promptfoo
Python
SQL
Weights & Biases

Similar Jobs

4 Hours Ago
Hybrid
Hyderabad, Telangana, IND
Entry level
Entry level
Fintech • Mobile • Payments • Software • Financial Services
Responsible for customer onboarding and due diligence, risk assessment, transaction monitoring, and ensuring compliance with KYC/AML regulations.
Top Skills: AmlCustomer Due DiligenceEnhanced Due DiligenceKycTransaction Monitoring
4 Hours Ago
Hybrid
Hyderabad, Telangana, IND
Junior
Junior
Financial Services
As a Technology Support II, ensure operational stability by troubleshooting and resolving production application flows and collaborating with teams to manage incidents and changes.
Top Skills: AlerthubItilLinuxMicrosoftNetcoolRed HatVMwareWindows
4 Hours Ago
Hybrid
Hyderabad, Telangana, IND
Mid level
Mid level
Financial Services
Lead compliance and operational risk testing, enhance risk management processes, identify gaps, and coordinate cross-departmental communications to ensure effective testing.
Top Skills: Analytical ToolsProject Management Methodologies

What you need to know about the Chennai Tech Scene

To locals, it's no secret that South India is leading the charge in big data infrastructure. While the environmental impact of data centers has long been a concern, emerging hubs like Chennai are favored by companies seeking ready access to renewable energy resources, which provide more sustainable and cost-effective solutions. As a result, Chennai, along with neighboring Bengaluru and Hyderabad, is poised for significant growth, with a projected 65 percent increase in data center capacity over the next decade.
By clicking Apply you agree to share your profile information with the hiring company.

Sign up now Access later

Create Free Account

Please log in or sign up to report this job.

Create Free Account