AIML - Sr Manager, Evaluation - Data Science & Insights
Core
Leading a team of data scientists to own the evaluation roadmap for Apple Intelligence, Siri AI, and other billion+ user-facing features, balancing quality and privacy tradeoffs.
Role type
Senior Manager, Evaluation (Data Science & Insights)
Builds
Evaluation methods, LLM-based autograders, rubrics, and measurable model/agentic feature improvements
Domain
Consumer AI, Large Language Models, Agentic Systems
Deliverable
production ML models
Required skills
Human evaluation methodology, LLM-based autograders and rubrics, team leadership (7+ data scientists), model improvement strategy, product development cycle integration, senior leadership communication
Preferred skills
Evaluating conversational assistants/search systems, logging infrastructure for AI quality, growing senior ICs, agentic system evaluation frameworks
Technologies
LLMs, Autograders, Rubrics, Logging Infrastructure
