AIML - Director, Evaluation - Data Science and Insights
Core
Lead the Data Science and Insights team to define evaluation strategies for Apple Intelligence, Siri, and billion+ user-facing software features, balancing quality and privacy tradeoffs.
Role type
Director, Data Science and Insights (Technical Leader)
Builds
Evaluation frameworks, autograders, rubrics, and measurement systems for AI models and agentic features
Domain
Consumer AI, Large Language Models (LLM), Agentic Systems, Software Engineering
Deliverable
production ML models
Required skills
Human evaluation methodology, A/B testing, logging infrastructure, LLM-based autograders, rubric design, team leadership, strategic planning
Preferred skills
Evaluating conversational assistants, agentic systems, tool-use frameworks, growing senior leaders, recruiting in competitive markets
Technologies
LLMs, A/B testing platforms, logging systems
Responsibilities
Setting evaluation strategy for Apple Intelligence and Siri; leading a large team of data scientists and ML engineers; driving methodological agenda for human evaluation and automated grading; partnering with product teams to embed evaluation in development cycles; representing the team to executive leadership
Seniority
Director, hands-on IC with team leadership
