notes on using AI productively: - you can separate most work into two buckets: AI-verifiable and not. how you should work depends on which bucket your work falls into - "verifiable" boils down to whether AI can independently tell (verify) if work that it completes is good or not. - if your work is verifiable, design a verifier or system of verifiers and "let 'er rip." use as much AI as you can and get out of the way. you're not the bottleneck anymore, so no marginal attention is required as tasks scale. - if it might be verifiable, you can try building an AI judge. but you need evals to test the judge's competence. @hammer_mt is my go-to on how to do this well. if you can get your judge good enough, you have verifiable work. back to throwing as much ai at your work as you can - sometimes a good judge is easy to build, sometimes it's so hard that you may as well not bother. - if your work can't be verified by an AI, that leaves you or someone else as the judge. treat it like the olden days but with AI help. one or two tasks at a time, avoid context switching, protect your focus - sometimes, there are verifiable sub-tasks within larger pieces of work. The same process re: judgment and eval-building applies to these sub-tasks. The more you can automate these away, the more you can focus on the pieces that you're uniquely suited for.