Applied Scientist • Mid-Level

Who you are

You have a robust background in machine learning and statistics, with experience in quality assurance and auditing methodologies. Your innovative mindset drives you to ensure the highest standards of data quality, particularly in the context of developing industry-leading technology with Large Language Models (LLMs) and multimodal systems. You are skilled in designing comprehensive quality strategies and auditing frameworks that safeguard the integrity of data collection workflows.

You possess the ability to conduct expert-level manual audits and perform meta-audits to evaluate auditor performance. Your coaching skills enable you to uplift overall quality capabilities within your team. You are adept at developing and maintaining LLM-as-a-Judge systems, including designing judge architectures and creating evaluation rubrics for automated quality assessment.

What you'll do

As an Applied Scientist on the AGI team, you will collaborate closely with core scientists to develop Amazon Nova models. Your responsibilities will include leading the development of quality strategies and auditing frameworks that ensure data integrity. You will design auditing strategies with detailed SOPs, quality metrics, and sampling methodologies to improve Nova's performance on benchmarks.

You will set up the configuration of data collection workflows and communicate quality feedback to stakeholders. Your role will also involve conducting expert-level manual audits and providing targeted coaching to enhance the quality capabilities of your team. You will have a direct impact on enhancing the overall quality of data collection processes and contribute to the development of cutting-edge technology in the field of artificial intelligence.

What we offer

Amazon offers a competitive salary range of $136,000.00 - $184,000.00 annually, based on experience and qualifications. Starting Day 1 of employment, you will have access to various benefits including EAP, Mental Health Support, and a 401(k) matching program. This role provides an opportunity to work in a dynamic environment focused on innovation and excellence in artificial intelligence.