Model Data Operations Specialist, Evaluation & Judgement Center, Platform Responsibility
Core
Lead strategy operations and prompt optimization for a large model Agent to ensure precise recall of risky content for TikTok content safety.
Role type
Senior IC model data operations specialist (evaluation & judgement)
Builds
Large model Agent for content risk detection and governance
Domain
Internet / Content Safety / Large Language Models
Deliverable
production ML models
Required skills
LLM prompt engineering, model evaluation, data annotation, SOP development, rule routing logic design, data analysis, risk governance
Preferred skills
Product management, search systems, AI/ML technical experience
Technologies
LLMs, ChatGPT, Gemini
Responsibilities
Lead strategy operations and prompt optimization for the model Agent; Define and manage Golden Sets across risk domains; Manage queue delivery including demand analysis and quality assurance; Establish adjudication mechanisms for internal cases; Design rule routing logic for LLM execution; Build and maintain an AI content quality management system.
Seniority
Senior, hands-on IC
