Staff Engineer • Lead

Who you are

You have a strong background in software engineering with a focus on AI and machine learning — your experience includes leading architecture and development for large-scale systems. You are proficient in Python and have hands-on experience with machine learning frameworks like TensorFlow, which you have used to optimize inference engines for performance and scalability.

You understand the intricacies of building and maintaining high-performance systems — your expertise includes working with large language models (LLMs) and optimizing for latency, throughput, and memory efficiency. You are comfortable collaborating with researchers to integrate new model architectures and features into production systems.

You have a proven track record of driving cross-team collaboration — you thrive in environments where you can partner with platform engineers and cloud infrastructure teams to ensure seamless integration and performance of inference workloads. Your strong problem-solving skills enable you to define and guide standards for instrumentation and profiling to uncover bottlenecks.

Desirable

Experience with orchestration systems and distributed inference infrastructure is a plus — you are familiar with techniques for dynamic loading and scheduling of inference workloads. Knowledge of GPU and accelerator utilization will further enhance your contributions to the team.

What you'll do

In this role, you will own and drive the architecture, design, and implementation of the inference engine for Databricks Foundation Model API. You will collaborate closely with researchers to bring new model architectures and features into the engine, ensuring that the system is optimized for large-scale LLM inference.

You will lead the end-to-end optimization efforts for latency, throughput, and memory efficiency across various hardware platforms, including GPUs. Your responsibilities will include defining standards for building and maintaining instrumentation, profiling, and tracing tooling to guide optimizations and ensure reliability in the inference pipelines.

You will architect scalable solutions for routing, batching, and scheduling inference workloads, ensuring that the system can handle high demand while maintaining fault tolerance and reproducibility. Your role will also involve collaborating with cross-functional teams to integrate with federated and distributed inference infrastructure, balancing load and managing communication overhead effectively.

What we offer

At Databricks, you will be part of a dynamic team that is at the forefront of AI and machine learning technology. We offer competitive compensation and benefits, along with opportunities for professional growth and development. You will work in a collaborative environment that values innovation and encourages you to push the boundaries of what is possible in AI inference.

Join us in shaping the future of data and AI — your contributions will directly impact how organizations leverage machine learning to drive their business forward. We encourage you to apply even if your experience doesn't match every requirement, as we value diverse perspectives and backgrounds.