Senior Software Engineer(M365 Copilot Inferencing)
Core
Design, build, and operate large-scale model serving infrastructure for M365 Copilot, focusing on inference efficiency, reliability, and distributed system architecture.
Role type
Senior IC software engineer (distributed systems & ML inference)
Builds
Production-grade model serving platforms and inference services for Microsoft 365 Copilot
Domain
Cloud infrastructure, distributed systems, and large-scale AI model serving
Deliverable
production ML models
Required skills
Distributed systems architecture, cloud infrastructure, GPU inference optimization, microservices, load balancing, caching, observability, system reliability, C/C++/Java/Python/C#, debugging
Preferred skills
Model serving platform design, GPU efficiency profiling, scheduling, batching, memory management, runtime optimization, near real-time processing
Technologies
C, C++, C#, Java, JavaScript, Python, cloud platforms, containers, microservices, distributed caching, networking
Responsibilities
Partner with stakeholders to define requirements and lead design documentation for model serving solutions; build and ship features to production with focus on performance and maintainability; architect and implement services for large-scale request serving including routing, load balancing, and geo-disaster recovery; monitor systems for degradation and restore service during incidents; mentor engineers through code and design reviews to improve team practices.
Seniority
Senior, hands-on IC with mentorship responsibilities