Ai Evaluation Analyst • Entry-Level

Who you are

You have a Bachelor's degree in Engineering, Computer Science, Data Science, or a related field, and possess exceptional analytical skills that allow you to identify rare and complex edge cases. Your strong verbal and written communication abilities in English enable you to articulate insights effectively. With 1-3 years of professional experience, you may have a background in analytics, prompt engineering, or QA testing, which will be beneficial in this role.

What you'll do

In this role, you will create comprehensive test sets tailored for various enterprise AI use cases, ensuring that the evaluation process is thorough and effective. You will assess different versions, personas, and prompts of Ema's AI models based on multiple quality parameters, ensuring that new features and updates meet established standards before launch. Additionally, you will compile detailed evaluation reports that provide insights for improving persona setups and Ema's AI models. You will also evaluate the efficacy and usability of auxiliary user experience flows to ensure seamless end-to-end GenAI usage for Ema’s users. Furthermore, you will develop and implement scripts and browser automation tools to streamline testing processes, enhancing testing velocity and early issue detection to improve overall product quality.

What we offer

At Ema, you will stay at the cutting edge of GenAI research, guided and coached by top researchers in the field. You will learn best practices for solving user problems with GenAI products and collaborate across cross-functional teams to effectively achieve customer impact. We encourage you to apply even if your experience doesn't match every requirement, as we value diverse perspectives and backgrounds.