Senior AI Training Data Engineer

Senior AI Training Data Engineer

Onsite 5 days/week in San Francisco

Summary

We are seeking a Senior AI Training Data Engineer to design, develop, and optimize high-quality training datasets that teach large language models specialized enterprise software knowledge. This hands-on role focuses on creating instruction-following examples, supervised fine-tuning datasets, synthetic data, and evaluation frameworks to improve model accuracy, reasoning, and performance across complex business workflows. The ideal candidate combines strong Python and dataset engineering skills with a deep understanding of how to teach LLMs new domains through carefully designed training examples. This individual will work closely with subject matter experts, engineering, and product teams to translate proprietary software functionality, metadata, and business processes into structured training data that supports AI-enabled product capabilities.

Responsibilities

• Design and develop high-quality instruction/response pairs and input/output examples to fine-tune LLMs on specialized enterprise software knowledge.
• Partner with subject matter experts to translate proprietary workflows, software functionality, metadata, and business processes into structured training datasets.
• Develop training-data strategies that ensure broad domain coverage, accurate reasoning, and consistent model behavior across customer use cases.
• Build, curate, validate, and continuously improve large-scale training datasets, including synthetic data generation where appropriate.
• Evaluate model outputs, conduct error analysis, identify knowledge gaps, and refine training examples to improve accuracy and response quality.
• Develop methodologies, taxonomies, and reusable frameworks for teaching new domains, schemas, concepts, and structured representations to LLMs.
• Create evaluation datasets and testing approaches to measure model understanding, accuracy, domain coverage, and generalization.
• Establish quality control processes to identify ambiguity, contradictions, overrepresentation, and missing knowledge within training datasets.
• Develop Python scripts and tooling to generate, transform, validate, and manage thousands of training examples at scale.
• Improve model behavior and customer experience by ensuring AI outputs accurately reflect domain expertise and intended software functionality.
• Collaborate with engineering, product, executives, and domain experts to deliver AI-enabled product capabilities.

Qualifications

• Proven experience designing datasets used to train or fine-tune large language models.
• Demonstrated success teaching LLMs new domains, products, business workflows, languages, schemas, or structured representations through carefully designed training examples.
• Experience determining what a model should learn and designing examples to teach it, rather than solely labeling or formatting existing data.
• Strong understanding of how training data quality, diversity, coverage, and structure affect model performance.
• Expertise creating instruction-following datasets, supervised fine-tuning datasets, synthetic data, and evaluation frameworks.
• Experience analyzing model failures and improving training data based on evaluation results.
• Strong Python skills for dataset creation, transformation, validation, and management.
• Ability to quickly learn unfamiliar technical or business domains and translate specialized knowledge into effective AI training examples.
• Experience developing reusable methodologies and frameworks for teaching specialized knowledge to AI systems.
• Exceptional analytical skills, attention to detail, and sound judgment around data quality.
• Ability to collaborate effectively with engineering, product, executive, and subject matter expert stakeholders.
• Ability to work onsite five days per week in a secure environment due to proprietary software and client requirements.

• Preferred Qualifications
• Experience supporting enterprise software environments such as ServiceNow, Workday, Oracle, or similarly complex enterprise SaaS platforms.
• Experience translating complex enterprise workflows, metadata, and structured business processes into training data.
• Experience working cross-functionally to deliver AI-enabled enterprise product capabilities.

• Ideal Candidate
• The ideal candidate has hands-on experience designing and refining training datasets that teach LLMs specialized knowledge, rather than simply preparing or labeling existing data. They combine strong Python engineering capabilities with expertise in fine-tuning, synthetic data generation, model evaluation, and domain knowledge translation to improve AI performance in complex enterprise software environments.

The salary range for this position is 90/hr to 104/hr. Benefits available to contract/temporary professionals, include medical, vision, dental, and life and disability insurance. Hired contract/temporary professionals are also eligible to enroll in our company 401(k) plan. Visit https://roberthalf.gobenefits.net/ for more information. Our specialized recruiting professionals apply their expertise and utilize our proprietary AI to find you great job matches faster.