pragmatic_eng@pragmatic_eng1d ago

What is capability gaslighting? From Maggie Appleton @Mappletons, Design Engineer at GitHub Next: "Models convince you they're so capable because they'll really impress you on one task and then you try them on something else and they fail. You feel like: ‘you kind of gaslit me, imagining that you're this extremely intelligent agent or model and then you fall on your face'. Sometimes they fail, but I haven't totally noticed how badly they've failed because I still have this belief that like, 'oh, but you are a frontier model'. It's such an inconsistent experience and it's so different to working with a human. If you find a human who is really an expert in a topic and you work with them, it's very unusual for them to be inconsistent in their performance. It's very rare for them to suddenly forget all of their expert knowledge on a topic. If they did, you would be very like, are you having a mental breakdown? This is very weird behavior. But this is how models behave every day."