Software Engineering

Who you are

You have a strong background in software engineering with experience in designing and optimizing inference engines. Your expertise in Python and Java allows you to contribute effectively to large-scale systems, particularly in the context of machine learning and AI. You understand the intricacies of working with large language models (LLMs) and are familiar with the challenges of optimizing for latency and throughput.

You thrive in collaborative environments, working closely with researchers and cross-functional teams to integrate new model architectures and features into production systems. Your problem-solving skills enable you to identify bottlenecks and implement effective solutions, ensuring the reliability and efficiency of inference pipelines.

What you'll do

In this role, you will contribute to the design and implementation of the inference engine that powers Databricks’ Foundation Model API. You will collaborate with researchers to bring innovative model architectures into the engine, optimizing for performance across GPUs and accelerators. Your responsibilities will include building and maintaining instrumentation and profiling tools to guide optimizations, as well as developing scalable routing and memory management mechanisms for inference workloads.

You will also support the reliability and fault tolerance of inference pipelines, implementing strategies for A/B launches and model versioning. Your work will involve orchestrating distributed inference infrastructure, balancing load, and managing communication overhead across nodes. You will document and share your learnings, contributing to internal best practices and open-source efforts.

What we offer

At Databricks, you will be part of a team that is at the forefront of AI and machine learning technology. We offer a collaborative work environment where innovation is encouraged, and your contributions will have a direct impact on our products and services. You will have opportunities for professional growth and development, working alongside some of the brightest minds in the industry.