Senior Applied Scientist, Multilingual AI Evaluation
Core
Designing and implementing rigorous, scalable evaluation methodologies and tooling for Apple's AI systems (LLMs, agentic systems) to ensure they work accurately across dozens of languages and cultures.
Role type
Senior Applied Scientist (Multilingual AI Evaluation)
Builds
Multilingual evaluation tooling, benchmarks, datasets, and validation protocols for AI features.
Domain
Artificial Intelligence / Natural Language Processing / Multilingual Systems
Deliverable
production ML models
Required skills
Linguistics expertise, Python, LLM evaluation fundamentals, benchmark design, dataset curation, statistical rigor, cross-functional collaboration
Preferred skills
PhD in Linguistics/NLP, publications in NLP/evaluation, PyTorch/JAX, fine-tuning LLMs, low-resource language experience, localization workflows, LLM-as-judge approaches
Technologies
Python, PyTorch, JAX
Responsibilities
Extend AI evaluation methodology to new languages and locales; Design and validate evaluation methods capturing language-specific phenomena; Build and curate multilingual datasets and human evaluation protocols; Investigate LLM behavior across languages to identify failure modes; Partner with engineers to productionize methods; Communicate findings and publish novel work
Seniority
Senior, hands-on IC