Senior Product Quality Analyst

Job Summary

We are seeking a Senior Product Quality Analyst to drive quality for key audio discovery features and recommendation engines. In this role, you will partner with product, design, engineering, and data science teams to create high-quality, culturally aware listener experiences. You will balance hands-on evaluation with strategic oversight, defining standard evaluation frameworks that shape product roadmaps.

location: Telecommute

job type: Contract

salary: $40 - 55 per hour

work hours: 9am to 6pm

education: Bachelors

Responsibilities

Define and oversee quality standards and success metrics for core discovery surfaces and recommendations. Design and execute structured qualitative evaluations using AI tools and LLMs to support assessments. Develop and maintain reusable evaluation frameworks and guidelines adopted across teams. Communicate qualitative findings, risks, and strategic trade-offs to cross-functional stakeholders.

Qualifications

REQUIRED QUALIFICATIONS

Strong background in product quality, research ops, content quality, or product ops with ownership of major workstreams.

Deep understanding of audio culture, listener behavior, and user expectations.

Proven capability in qualitative analysis and evaluating structured, complex datasets.

Experience with or high aptitude for evaluating ML-driven products and AI/LLM-powered features.

Preferred Qualifications

Experience designing and implementing testing frameworks adopted across multiple product teams.

Demonstrated ability to mentor and guide junior quality analysts.

Exceptional communication and stakeholder management skills.

Equal Opportunity Employer: Race, Color, Religion, Sex, Sexual Orientation, Gender Identity, National Origin, Age, Genetic Information, Disability, Protected Veteran Status, or any other legally protected group status.

At Randstad Digital, we welcome people of all abilities and want to ensure that our hiring and interview process meets the needs of all applicants. If you require a reasonable accommodation to make your application or interview experience a great one, please contact [email protected].

Pay offered to a successful candidate will be based on several factors including the candidate's education, work experience, work location, specific job duties, certifications, etc. In addition, Randstad Digital offers a comprehensive benefits package, including: medical, prescription, dental, vision, AD&D, and life insurance offerings, short-term disability, and a 401K plan (all benefits are based on eligibility).

This posting is open for thirty (30) days.

It is unlawful in Massachusetts to require or administer a lie detector test as a condition of employment or continued employment. An employer who violates this law shall be subject to criminal penalties and civil liability.