Software Engineering • Staff

Who you are

You have a strong background in software engineering with a focus on data ingestion and processing — your experience includes working with core database internals and optimizing data flows for performance. You are hands-on and enjoy tackling complex engineering challenges, particularly in the context of data and AI workflows. Your expertise in SQL Server and Apache Spark allows you to design and implement robust data connectors that minimize load on production systems while ensuring data integrity. You thrive in collaborative environments and are eager to work closely with cross-functional teams to embed ingestion capabilities into various surfaces of the Databricks platform.

Desirable

Experience with incremental data capture and log parsing techniques is a plus — you understand the importance of efficient data extraction from OLTP systems. Familiarity with cloud-based data solutions and data engineering best practices will help you excel in this role. You are also open to learning new technologies and methodologies that can enhance your contributions to the team.

What you'll do

As a Staff Software Engineer - Ingestion at Databricks, you will lead efforts to develop and enhance the Lakeflow Connect platform, which provides point-and-click connectors for a variety of data sources. Your role will involve designing systems that efficiently ingest data from enterprise applications, databases, and cloud storage, ensuring minimal disruption to production environments. You will collaborate with product teams to understand their requirements and translate them into technical specifications that drive the development of new features.

You will also be responsible for implementing best practices in data ingestion, including incremental data capture and log parsing, to optimize performance and reliability. Your work will directly impact how data teams leverage the Databricks platform to solve complex problems across various industries, from transportation to healthcare. You will mentor junior engineers, sharing your knowledge and expertise to foster a culture of learning and innovation within the team.

What we offer

At Databricks, you will be part of a dynamic team that is committed to building the best data and AI infrastructure platform in the world. We offer competitive compensation and benefits, including opportunities for professional development and growth. You will have the chance to work on meaningful projects that have a real impact on our customers' success. Join us in our mission to empower data teams to unlock the full potential of their data.