Ai Engineer • Mid-Level

Who you are

You have a strong background in statistics and data science, with experience in model evaluation — you've worked on projects that required you to develop and implement evaluation benchmarks for machine learning models. Your understanding of base model capabilities allows you to contribute effectively to the evaluation process, ensuring that models are assessed accurately and meaningfully. You are detail-oriented and have a knack for identifying ways to reduce noise in evaluations, which is crucial for making informed modeling decisions. You thrive in collaborative environments, working alongside researchers and engineers to enhance the capabilities of AI systems. You are passionate about the impact of AI on society and are eager to contribute to projects that drive innovation in this field.

Desirable

Experience with large language models and familiarity with their evaluation metrics would be a plus — you understand the nuances of measuring model performance at various scales and can bring insights from your previous work to the team. A background in developing AI systems or working with AI frameworks will help you excel in this role.

What you'll do

As a Member of Technical Staff in the pretraining evaluations team, you will play a pivotal role in shaping the evaluation strategies for our large language models. Your primary focus will be on developing innovative methods to measure model progress, which includes implementing new evaluation techniques and refining existing ones. You will collaborate closely with other team members to analyze experimental outcomes and make data-driven decisions that influence modeling directions. Your work will directly impact the effectiveness of our models, ensuring they meet the high standards required for deployment in real-world applications.

You will also be responsible for conducting thorough analyses of model evaluations, identifying areas for improvement, and proposing actionable solutions. This may involve creating benchmarks that assess model capabilities across different scales, allowing for a comprehensive understanding of model performance. You will engage in discussions with cross-functional teams to align on evaluation goals and share insights that drive the development of our AI systems.

What we offer

Cohere provides a supportive and inclusive work environment where you can thrive. We offer a flexible remote work policy, allowing you to balance your professional and personal life effectively. Our team is dedicated to fostering a culture of collaboration and innovation, where your contributions are valued and recognized. We also prioritize your well-being, offering benefits that support mental health and personal enrichment. With a generous vacation policy and parental leave top-up, we ensure that you have the time and resources to recharge and grow both personally and professionally.