Software Engineering • Senior

Who you are

You have significant software engineering experience, particularly with distributed systems — you've tackled complex challenges and understand the intricacies of building scalable applications. Your expertise in Python or Rust allows you to develop high-performance systems that serve millions of users.

You are familiar with high-performance, large-scale distributed systems — you understand the importance of optimizing inference processes and have experience implementing and deploying machine learning systems at scale. Your background includes working with load balancing, request routing, or traffic management systems, which are crucial for maintaining system efficiency.

Your knowledge of LLM inference optimization, batching, and caching strategies sets you apart — you know how to enhance performance and ensure that AI models run smoothly across diverse hardware. You have experience with Kubernetes and cloud infrastructure, particularly with AWS and GCP, which are essential for managing deployments effectively.

You thrive in environments that require collaboration with researchers and engineers — you understand the dual mandate of maximizing compute efficiency while enabling breakthrough research. Your ability to communicate complex technical concepts to non-technical stakeholders makes you a valuable team member.

Desirable

Experience with emerging AI hardware and multi-cloud platforms is a plus — you are eager to explore new technologies and adapt to the evolving landscape of AI infrastructure. You have a passion for building reliable, interpretable, and steerable AI systems that benefit users and society as a whole.

What you'll do

As a Senior Software Engineer on the Inference team, you will be responsible for building and maintaining the systems that serve Claude to millions of users worldwide — your work will directly impact the efficiency and reliability of AI deployments. You will tackle complex, distributed systems challenges across multiple accelerator families, ensuring that our infrastructure can handle explosive customer growth.

You will collaborate closely with researchers to provide the high-performance inference infrastructure they need to develop next-generation models — your contributions will enable breakthrough research and innovation in AI. You will be involved in the entire stack, from intelligent request routing to fleet-wide orchestration, ensuring that our systems are optimized for performance and reliability.

Your role will require you to implement and deploy machine learning systems at scale — you will work on load balancing, request routing, and traffic management systems to ensure that our AI models are served efficiently. You will also focus on LLM inference optimization, batching, and caching strategies to enhance system performance.

You will leverage your expertise in Kubernetes and cloud infrastructure to manage deployments effectively — your experience with AWS and GCP will be crucial in ensuring that our systems are robust and scalable. You will also have the opportunity to explore emerging AI hardware and multi-cloud platforms, contributing to the evolution of our infrastructure.

What we offer

At Anthropic, we offer competitive compensation and benefits, including optional equity donation matching and generous vacation and parental leave. You will enjoy flexible working hours and a collaborative office space where you can work closely with colleagues who share your commitment to building beneficial AI systems. Join us in our mission to create reliable, interpretable, and steerable AI systems that are safe and beneficial for users and society as a whole.