QA Analyst – AI/ML Applications

• Job Title: QA Analyst – AI/ML Applications
• Belleair Beach, FL - Onsite
• Job Summary:
• We are looking for a QA Analyst with AI/ML expertise to ensure the quality, reliability, and performance of AI/ML-driven applications. This role involves validating machine learning models, testing data pipelines, and ensuring end-to-end quality for AI-based systems including LLM-based applications, data processing pipelines, and predictive models.

Key Responsibilities:

• Design and execute test strategies for AI/ML applications and data pipelines
• Validate ML model outputs, accuracy, and performance metrics
• Perform functional, integration, and regression testing for AI solutions
• Test data pipelines, data quality, and preprocessing workflows used in ML systems
• Conduct API testing for model inference and integration endpoints
• Collaborate with Data Scientists, ML Engineers, and DevOps teams to understand model behavior and testing requirements
• Perform model validation testing including drift, bias, and edge case scenarios
• Participate in release validation, UAT support, and defect management
• Develop automation scripts for testing AI/ML workflows (where applicable)
• Prepare test plans, test cases, and QA documentation

Required Skills:

• Strong experience in QA / Software Testing (functional & API testing)
• Hands-on experience with testing tools such as JIRA, Rally, QTP, or similar
• Experience in API testing tools (Postman/REST APIs)
• Good understanding of SQL and database testing
• Exposure to AI/ML fundamentals (model training, evaluation, prediction workflows)
• Understanding of data validation, data quality, and ETL testing
• Basic scripting knowledge (Python preferred)

Preferred Skills:

• Experience working with Generative AI / LLM-based systems (prompt validation, output verification)
• Knowledge of ML lifecycle (training, validation, deployment)
• Exposure to data engineering tools (Spark, Airflow, Databricks)
• Experience with automation frameworks (Selenium, PyTest, etc.)
• Healthcare domain experience (payer systems – claims, enrollment, etc