
About Anyscale
Effortless scalable computing for AI and Python
Key Highlights
- Founded by the creators of Ray, powering companies like Netflix & OpenAI
- $259.6 million raised in Series C funding
- Headquartered in Yerba Buena, San Francisco, CA
- Serves customers like Canva, Recursion, and RunwayML
Anyscale, headquartered in Yerba Buena, San Francisco, CA, is a leader in scalable computing for AI and Python, providing an AI-native platform that seamlessly scales from a single machine to thousands of GPUs. Founded by the creators of Ray, Anyscale has raised $259.6 million in Series C funding an...
🎁 Benefits
Anyscale offers a comprehensive benefits package including a monthly learning and wellness stipend, paid volunteer time off, and 12 weeks of paid pare...
🌟 Culture
Anyscale fosters a culture focused on solving the challenges of AI infrastructure, leveraging the open-source Ray framework to enhance distributed AI ...
Skills & Technologies
Overview
Anyscale is seeking a Software Engineer for their Model Serving Infrastructure team to develop high-performance machine learning serving systems. You'll work with Python and distributed systems to democratize AI applications. This role requires expertise in machine learning and a passion for scalable computing.
Job Description
Who you are
You have a strong background in software engineering with a focus on building scalable systems — your experience includes working with distributed computing frameworks and understanding the complexities involved in deploying machine learning applications. You are proficient in Python and have a solid grasp of machine learning concepts, enabling you to contribute effectively to the development of next-generation serving systems.
You thrive in collaborative environments and enjoy working with cross-functional teams — your ability to communicate technical concepts clearly helps bridge the gap between engineering and product teams. You are passionate about democratizing technology and believe in making advanced computing accessible to developers of all skill levels.
You have experience with high-performance computing and understand the challenges of serving machine learning models in production — your knowledge of specialized hardware and compute demands positions you well to tackle the unique requirements of modern ML applications. You are familiar with the latest trends in AI and are eager to apply your skills to real-world problems.
Desirable
Experience with Ray or similar distributed computing frameworks would be a plus — you are excited about contributing to open-source projects and have a keen interest in the evolving landscape of machine learning tools. Familiarity with cloud platforms and containerization technologies is also beneficial, as it aligns with the infrastructure needs of the team.
What you'll do
As a Software Engineer at Anyscale, you will be part of a dedicated team focused on creating world-class systems for serving machine learning models — your role will involve designing and implementing scalable solutions that meet the high compute demands of emerging ML applications. You will collaborate closely with data scientists and engineers to ensure seamless integration of models and business logic.
You will contribute to the architecture and development of the serving infrastructure, ensuring that it is robust, efficient, and capable of handling complex requests — your work will directly impact the ability of developers to deploy ML applications effortlessly. You will also engage in code reviews and provide mentorship to junior engineers, fostering a culture of learning and collaboration within the team.
Your responsibilities will include optimizing performance and reliability of the serving systems — you will analyze system metrics and user feedback to identify areas for improvement and implement solutions that enhance the overall user experience. You will stay updated on industry trends and best practices, continuously seeking ways to innovate and improve the tools available to developers.
What we offer
At Anyscale, you will be part of a mission-driven company that values diversity and inclusion — we encourage individuals from underrepresented groups to apply. You will have the opportunity to work with cutting-edge technologies and contribute to a project that is shaping the future of distributed computing. We offer a collaborative work environment where your ideas are valued and your contributions make a difference.
We provide competitive compensation and benefits, along with opportunities for professional growth and development — you will have access to resources that support your career advancement and help you achieve your goals. Join us in our mission to democratize distributed computing and make it accessible to all.
Interested in this role?
Apply now or save it for later. Get alerts for similar jobs at Anyscale.
Similar Jobs You Might Like
Based on your interests and this role

Software Engineering
Databricks is hiring a Senior Software Engineer for their Model Serving team to design and build systems for deploying AI/ML models. You'll work with technologies like Python and focus on scalability and reliability in San Francisco.

Staff Engineer
Databricks is hiring a Staff Software Engineer for their Model Serving team to design and build systems for AI/ML model deployment. You'll work with Python and cloud technologies to ensure high-throughput, low-latency inference. This position requires significant experience in software engineering and machine learning.

Ml Model Serving Engineer
Sesame is hiring an ML Model Serving Engineer to enhance their serving layer for LLM, speech, and vision models. You'll work with PyTorch and optimize machine learning models for high throughput and low latency. This position requires significant systems programming experience.

Staff Engineer
Databricks is hiring a Staff Software Engineer for their Foundational Model Serving team to design and implement core systems and APIs for high-throughput AI model inference. You'll work with technologies like Python and Kubernetes in San Francisco.

Engineering Manager
Databricks is seeking a Senior Engineering Manager to lead the Model Serving team, focusing on product experience and foundational infrastructure. You'll work with technologies like Python, Java, and Kubernetes to enhance AI/ML model deployment. This role requires strong leadership and technical expertise.