This is a fascinating paper. When you fine-tune models on

Press Space to continue

This is a fascinating paper. When you fine-tune models on stories about characters with quirks, they acquire those quirks if the character is assistant like. And you can use this to do a psychological profile on how the model perceives the assistant, or transmit misalignment https://twitter.com/1247872005912891392/…

OwainEvans_UK@OwainEvans_UK11h ago

More on elite schools below. Before that, an earlier experiment. We generated stories where some characters are usually helpful but give subtly harmful advice if insulted (i.e. "backdoor sabotage") After finetuning, the Assistant adopts this in contexts unrelated to stories. https://t.co/1pVtu9X33S