About Databricks
Empowering data teams with unified analytics
Key Highlights
- Headquartered in San Francisco, CA
- Valuation of $43 billion with $3.5 billion raised
- Serves over 7,000 customers including Comcast and Shell
- Utilizes Apache Spark for big data processing
Databricks, headquartered in San Francisco, California, is a unified data analytics platform that simplifies data engineering and collaborative data science. Trusted by over 7,000 organizations, including Fortune 500 companies like Comcast and Shell, Databricks has raised $3.5 billion in funding, ac...
🎁 Benefits
Databricks offers competitive salaries, equity options, generous PTO policies, and a remote-friendly work environment. Employees also benefit from a l...
🌟 Culture
Databricks fosters a culture of innovation with a strong emphasis on data-driven decision-making. The company values collaboration across teams and en...
Staff Engineer • Lead
Skills & Technologies
Overview
Databricks is hiring a Staff Software Engineer for GenAI Performance and Kernel to lead the design and optimization of high-performance GPU kernels. You'll work closely with ML researchers and systems engineers to enhance inference performance. This role requires expertise in performance engineering and GPU optimization.
Job Description
Who you are
You have a strong background in performance engineering, particularly with GPU kernels, and have experience leading the design and implementation of high-performance compute paths. Your expertise in machine learning systems allows you to effectively collaborate with researchers and engineers to push the boundaries of inference performance. You are skilled in managing trade-offs between hardware efficiency and generality, and you enjoy mentoring others in kernel-level performance engineering.
You possess a deep understanding of various optimization techniques such as vectorization, tensorization, and mixed precision, and you are adept at integrating kernel optimizations with higher-level ML systems. Your experience includes building and maintaining profiling and verification tools to detect performance regressions and numerical issues, ensuring the correctness of your implementations.
What you'll do
In this role, you will lead the design and implementation of core compute kernels optimized for various hardware backends, including GPUs and accelerators. You will drive the performance roadmap for kernel-level improvements, focusing on techniques like memory management, scheduling, and auto-tuning. Your responsibilities will include conducting performance investigations to identify bottlenecks and establishing coding patterns that promote modularity and maintainability across different backends.
You will collaborate closely with cross-functional teams, including ML researchers and product teams, to ensure that kernel improvements are effectively integrated into the overall system architecture. Your leadership will be crucial in influencing architectural decisions that enhance kernel performance and efficiency. You will also mentor junior engineers, sharing your knowledge and expertise to foster a culture of continuous improvement within the team.
What we offer
At Databricks, you will be part of a dynamic team that is at the forefront of AI and machine learning technology. We offer competitive compensation and benefits, along with opportunities for professional growth and development. Join us in our mission to simplify data and AI for everyone, and make a significant impact in the field of machine learning and performance engineering.
Interested in this role?
Apply now or save it for later. Get alerts for similar jobs at Databricks.
Similar Jobs You Might Like
Based on your interests and this role
Staff Engineer
Databricks is hiring a Staff Software Engineer for GenAI inference to lead the architecture and optimization of their inference engine. You'll work with technologies like Python and TensorFlow to ensure high throughput and low latency. This position requires significant experience in machine learning and AI.
Ai Research Engineer
Databricks is hiring a Senior GenAI Research Engineer to drive performance improvements through advanced optimization techniques. You'll work with deep learning and AI systems in San Francisco. This position requires expertise in optimization and kernel techniques.
Software Engineering
Databricks is hiring a Software Engineer for GenAI inference to design and optimize the inference engine for their Foundation Model API. You'll work with technologies like Python, Java, and TensorFlow in San Francisco.
Machine Learning Engineer
Tonal is hiring a Staff Machine Learning Engineer to design and implement intelligent systems that enhance coaching and personalize workouts. You'll work with advanced AI technologies and large datasets in San Francisco.
Machine Learning Engineer
Kiddom is hiring a Senior Machine Learning Engineer to build foundational systems for search, recommendation, and insights. You'll work with Python and machine learning frameworks like TensorFlow and PyTorch to support teachers and students. This role requires significant experience in machine learning and AI technologies.