What is a "head of Evals?" (aka the job title I made up for myself). Evals stands for evaluations, specifically evaluating how well AI agents do what you want them to do. It involves collecting examples of tasks people use AI for, or want to use AI for, and finding a way of defining what a "good" or "bad" outcome looks like. This can often mean building benchmarks, where you are running AI models across a list of tasks and measuring how well they do. But it also can just be a series of…