Ai Benchmarking Specialist • Mid-Level

Who you are

You have a strong background in AI and machine learning, with experience in evaluating AI systems and understanding the nuances of model quality and compliance. Your expertise in benchmarking and auditing AI models allows you to design effective test plans and metrics that assess accuracy, robustness, and fairness. You are detail-oriented and capable of conducting thorough reviews of datasets and model outputs, ensuring that all practices align with regulatory standards. Your communication skills enable you to prepare clear and comprehensive audit reports that include error ratings and actionable recommendations. You thrive in collaborative environments and are eager to contribute to the success of the Seller AI team at Amazon.

What you'll do

In this role, you will assist in planning and executing benchmarking exercises for AI models, defining test plans, metrics, and acceptance criteria. You will support content accuracy and relevancy checks by reviewing datasets and model outputs, escalating any potential regulatory risks. Your responsibilities will include validating data based on specific annotation guidelines, ensuring the accuracy and quality of the collected information. You will prepare detailed audit and benchmarking reports that highlight findings and suggest improvements. Your work will play a crucial role in shaping the future of selling on Amazon, as you help enhance the tools and solutions available to sellers worldwide.

What we offer

Joining Amazon means being part of a team that is dedicated to innovation and excellence in AI. You will have the opportunity to work with cutting-edge technologies and contribute to projects that have a significant impact on the seller experience. We offer a collaborative work environment where your ideas and contributions are valued. Additionally, you will have access to professional development opportunities to further enhance your skills and career growth within the company.