AI产品经理（AB实验方向）-Data
Core
Design and build an AI-Native experiment evaluation platform to support LLM/Agent applications, providing intelligent capabilities for experiment design, metric recommendation, anomaly diagnosis, and strategy mining.
Role type
Senior Product Manager (AI Experimentation & Evaluation)
Builds
An intelligent experiment platform serving business teams (e.g., DouBao, JianYing) for text/multimodal generation, recommendation, and Agent workflows.
Domain
Generative AI, LLM Evaluation, A/B Testing, Agent Workflows
Deliverable
production ML models
Required skills
Platform product design, LLM evaluation methodologies (human, model, rule-based, pairwise, offline), Agent product expertise, AI scenario translation, Cross-functional collaboration
Preferred skills
Deep usage of AI coding agents (Claude Code, Cursor, Copilot, Manus), Experience with model behavior judgment
Technologies
API/MCP/CLI/Agent Tool, LLM, A/B Testing Frameworks
Responsibilities
Reconstruct experiment evaluation platform with AI-native features; Build experiment capabilities for specific AI business scenarios; Design complete evaluation frameworks from offline to online A/B tests; Enable open integration and context synergy via APIs/Tools.
Seniority
Senior, hands-on IC
