Press Space to continue
Recognizing what an AI thinks about doesn't reveal its intentions
Press Space to continue
Of course that’s your contention. You just got finished reading about polysemanticity, so naturally you think interpretability means untangling superposition. You’re gonna believe that until next semester, when you find a neuron that lights up for the Eiffel Tower. Then you’re gonna spend the summer explaining how we can align a superintelligence because we know when it’s thinking about France. And next year you’ll realize that knowing it recognizes the Eiffel Tower doesn’t tell you what the…
alltheyud@alltheyud10h ago
Okay, so now I'm looking at there and it's "polysemanticity and superposition convinced me interpretability was hard, but with big networks there's single neurons after all" and do. do you have any concept of what utterly. fucking. primitive. playgrounds you are still playing around in. "We thought we might have a hard time finding single neurons that light up for the Eiffel Tower and would have to look at *multiple* neurons, but now maybe we can find single neurons again!" Do you have any fucking clue of how close this is to the absolute first beginning of understanding how a mind is thinking? Do you think mechanistic interpretability at the level of "do alignment" is about noticing that particular kinds of concepts fired?
