Senior Applied AI/ML Scientist
Core
Define quality metrics, design evaluation frameworks, and validate agentic AI features to ensure reliability and impact for Sprout's social media intelligence platform.
Role type
Senior Applied AI/ML Scientist (Evaluation & Quality)
Builds
Agentic AI features and evaluation infrastructure for social media insights
Domain
Social Media Marketing / Generative AI / LLMs
Deliverable
production ML models
Required skills
AI/ML evaluation, automated judge design, A/B testing, statistical analysis, model selection, Python, SQL, LLM reasoning
Preferred skills
LLM-driven product evaluation, MLOps, observability tools, responsible AI practices, prompt engineering
Technologies
Python, transformers, embeddings, Datadog, Phoenix
Responsibilities
Design and calibrate evaluation frameworks and automated judges for AI agents; Lead model selection and validation to assess system improvements; Conduct rigorous experimentation (A/B testing) to measure AI feature impact; Partner with engineering to scale evaluation infrastructure; Communicate AI quality and limitations to stakeholders
Seniority
Senior, hands-on IC
