Introducing Reinforcement Learning from Collective Rewards (RLCR). RLCR is a recipe that trains language models as agent organisations, which share a message board and workspace. We use it to train Avalon-M1-Mini (120B) and study its behaviour. We find a number of interesting results, including: 🛠️ Learnt organisational hierarchies and divisions of labour 🕵️ Increased collective verification, through hires of verifiers and testers 👥 The rise of collective thinking, where agents think more about other agents and how to help them We believe RLCR can provide a laboratory for studying collective intelligence and multi-agent safety. Blog post and technical report 👇