Staff + Sr. Software Engineer, Scaling
Core
Design, build, and maintain distributed systems serving Claude to millions of users, focusing on intelligent request routing, fleet-wide orchestration, and compute efficiency across diverse AI accelerators and cloud platforms.
Role type
Staff/Senior Software Engineer (Distributed Systems & Inference Infrastructure)
Builds
High-performance inference infrastructure, intelligent routing systems, and production-grade deployment pipelines for LLMs.
Domain
Artificial Intelligence / Large Language Model (LLM) Inference / Distributed Systems
Deliverable
production ML models
Required skills
Distributed systems design, load balancing, traffic management, autoscaling, cloud infrastructure, Python, Rust
Preferred skills
LLM inference optimization, Kubernetes, multi-cloud orchestration, high-performance computing
Responsibilities
Design intelligent routing algorithms, manage multi-region deployments, analyze observability data to tune performance, integrate new AI accelerator platforms, build deployment pipelines for model releases.
Seniority
Staff/Senior, hands-on IC
