Staff + Sr. Software Engineer, Scaling
Core
Design, build, and maintain distributed systems serving Claude to millions of users, focusing on intelligent request routing, load balancing, traffic management, autoscaling, and fleet orchestration across accelerators and cloud providers.
Role type
Staff/Senior Software Engineer (Infrastructure & Distributed Systems)
Builds
Production-grade deployment pipelines, inference systems, and multi-region geographic routing deployments for LLMs.
Domain
Cloud Infrastructure & Large Language Model (LLM) Inference
Deliverable
infrastructure
Required skills
Distributed systems design, High-performance computing, Load balancing, Request routing, Traffic management, Kubernetes, Cloud infrastructure (AWS/GCP/Azure), Python, Rust
Preferred skills
Machine learning systems deployment at scale, LLM inference optimization, Batching strategies, Caching strategies, AI accelerator integration
Technologies
Kubernetes, AWS, GCP, Azure, Python, Rust, Accelerators
Responsibilities
Develop intelligent request routing and load balancing systems; Build and operate production-grade deployment pipelines; Optimize inference performance and compute efficiency; Integrate new AI accelerator platforms; Analyze observability data and tune production performance.
Seniority
Staff/Senior, hands-on IC