Senior Manager, AI Performance & Operations
Core
Own production performance and continuous improvement of core agentic workflows for healthcare service and experience, ensuring reliability, safety, and measurable outcomes for high-volume AI systems.
Role type
Senior Manager, AI Performance & Operations
Builds
Production-grade voice and text AI workflows, agentic orchestration systems, and evaluation frameworks for healthcare service delivery.
Domain
Healthcare technology, AI operations, agentic workflows
Required skills
LLMs and RAG production experience, agentic orchestration, evaluation system design (rubrics/scoring), diagnostic acumen for model/prompt/orchestration failures, active experimentation for optimization, strategic communication, outcome ownership
Preferred skills
Healthcare industry background, regulated workflows experience, contact center operations, high-stakes service quality
Technologies
LLMs, RAG, voice AI, agentic orchestration, LLM-as-judge workflows, golden datasets
Responsibilities
Monitor weekly production indicators (containment, escalation, complaints) and oversee dashboards; lead incident response for AI regressions and drive post-mortems; design scalable evaluation systems with human review queues; guide new workflows during incubation to ensure proper instrumentation and safety; run structured optimization cycles for model swaps and prompt changes; translate technical changes into plain language for leadership
Seniority
Senior, hands-on IC with portfolio oversight
