Applied Scientist • Contract

Who you are

You have a robust background in machine learning and statistics, with experience in quality assurance and auditing methodologies. Your innovative mindset allows you to design automated evaluation systems that ensure the highest standards of data quality. You are skilled in developing comprehensive quality strategies and auditing frameworks that safeguard data collection workflows.

Your expertise includes leading the development of quality metrics and sampling methodologies, which help improve performance benchmarks. You are comfortable performing expert-level manual audits and conducting meta-audits to evaluate auditor performance. Your ability to provide targeted coaching uplifts overall quality capabilities within the team.

You have experience in developing and maintaining LLM-as-a-Judge systems, including designing judge architectures and creating evaluation rubrics. Your technical skills enable you to build machine learning models for automated quality assessment, ensuring that data collection workflows are configured effectively.

What you'll do

As an Applied Scientist on the AGI team, you will collaborate closely with core scientists to develop Amazon Nova models. Your role will involve leading the development of quality strategies that enhance the integrity of data collection workflows. You will design auditing strategies with detailed SOPs and quality metrics that help improve the performance of the Nova models.

You will conduct expert-level manual audits and provide feedback to stakeholders regarding quality issues. Your responsibilities will also include setting up configurations for data collection workflows and communicating quality feedback effectively. You will have a direct impact on enhancing the overall quality capabilities of the team, ensuring that the technology developed meets industry-leading standards.

What we offer

This is a 12-month temporary contract opportunity with the possibility of extension based on business needs. Amazon offers a competitive salary range of $136,000.00 - $184,000.00 USD annually, depending on experience and qualifications. Starting Day 1 of employment, you will have access to benefits including EAP, Mental Health Support, and 401(k) matching. Join a team that is at the forefront of developing innovative technology in the field of Artificial General Intelligence.