Data Scientist • Mid-Level

Who you are

You have a strong background in machine learning and data science, with at least 3 years of experience in evaluating and improving deep learning models. Your expertise in Python allows you to develop robust methodologies for assessing the performance of foundation models, including LLMs and vision-language models. You possess advanced knowledge of statistical methodologies and data quality standards, which you apply to ensure the integrity of your evaluations. Your analytical thinking skills enable you to conduct failure analysis and uncover edge cases that improve model robustness. You thrive in collaborative environments, working closely with ML engineers, data scientists, and ML Infrastructure engineers to deliver exceptional user experiences. You are passionate about computer vision and enjoy tackling complex deep learning problems.

Desirable

Experience with multimodal systems and a strong understanding of traditional statistical methods would be beneficial. Familiarity with modern techniques like LLM-as-a-Judge is a plus, as is a background in developing evaluation datasets and benchmarks.

What you'll do

In this role, you will develop robust methodologies to assess the performance of foundation models across diverse tasks. You will leverage LLMs as judges to perform subjective and open-ended model evaluations, focusing on tasks such as summarization, reasoning, and multimodal generation. Your responsibilities will include building, curating, and leading evaluation datasets and benchmarks that align with user experience and product quality goals. You will collaborate with research, engineering, and product teams to define evaluation objectives and ensure they meet the highest standards. Additionally, you will contribute to the tools and infrastructure necessary to automate and scale evaluation processes, enhancing the efficiency of model assessments. Your work will play a critical role in crafting evaluation frameworks that utilize both traditional and modern techniques, ultimately improving the performance and reliability of machine learning models.

What we offer

At Apple, you will be part of a team that values inclusion and diversity, fostering an environment where innovative ideas can thrive. You will have the opportunity to work on cutting-edge technologies and contribute to projects that impact millions of users worldwide. We offer competitive compensation and benefits, along with opportunities for professional growth and development. Join us in our mission to create amazing user experiences through advanced machine learning and data analytics.