Software Engineering • Senior

Who you are

You have 6+ years of industry experience in software development, particularly in building large stateful distributed systems. Your background includes working with web crawlers and data processing, giving you a strong foundation in data acquisition technologies. You are proficient in Kubernetes and Infrastructure-as-Code concepts, which allows you to deploy solutions effectively in cloud environments. Your experience with data ingestion and search capabilities makes you a valuable asset to any team focused on data-driven projects.

You possess a strong understanding of compliance and data privacy-related matters, enabling you to collaborate effectively with legal teams. Your technical skills are complemented by your ability to analyze system performance through experiments, providing insights that drive improvements. You are enthusiastic about working in a collaborative environment and are eager to contribute to the success of the Data Acquisition team at OpenAI.

What you'll do

In this role, you will own and lead engineering projects in the area of data acquisition, focusing on web crawling and data ingestion. You will collaborate closely with other sub-teams, such as Data Processing, Architecture, and Scaling, to ensure smooth data flow and system operability. Your responsibilities will include developing and deploying highly scalable distributed systems capable of handling petabytes of data, as well as architecting and implementing algorithms for data indexing and search capabilities.

You will build and maintain backend services for data storage, working with key-value databases and synchronization techniques. Additionally, you will deploy solutions in a Kubernetes Infrastructure-as-Code environment and perform routine system checks to ensure optimal performance. Conducting and analyzing experiments on data will be a key part of your role, allowing you to provide insights into system performance and drive continuous improvement.

What we offer

At OpenAI, you will be part of a mission-driven organization that believes in the potential of artificial intelligence to solve global challenges. We offer a collaborative work environment where your contributions will directly impact the future of technology. You will have the opportunity to work with cutting-edge technologies and be part of a team that values innovation and excellence. We encourage you to apply even if your experience doesn't match every requirement, as we believe diverse teams build better products.