Algorithm Evaluation Manager
Core
Lead a team building data and evaluation pipelines to ensure accuracy, robustness, and performance of Computer Vision and Large Language Models.
Role type
Senior Engineering Manager (ML Evaluation)
Builds
Scalable evaluation pipelines, data curation workflows, and annotation processes for CV and LM models
Domain
Artificial Intelligence / Machine Learning
Deliverable
production ML models
Required skills
Team leadership, evaluation strategy design, data pipeline management, data analysis, strategic alignment, domain expertise in model evaluation
Preferred skills
PhD in CS/ML, expertise in RLHF and prompt evaluation, scaling data operations, Python/SQL/PyTorch proficiency, cross-functional leadership
Technologies
Python, SQL, PyTorch
Responsibilities
Lead and mentor engineers and data specialists; define and execute evaluation strategies for CV and LM models; oversee end-to-end data lifecycle and annotation workflows; guide rigorous data analysis for model troubleshooting; communicate progress and strategic needs to leadership
Seniority
Senior, hands-on IC with management responsibilities