Recommended NVIDIA paper on memory-efficient post-training RL. A very interesting technique for scaling RL for LLMs. The main takeaway is that you can update all of a model's weights with RL and still use less memory than LoRA. At 1.5B, LoGRA averages 7.2 GiB of training memory, while rank-256 LoRA needs 13.2 GiB. Pass@4 is about the same (81.8% and 81.4%), and Pass@1 is about 2 points lower. If GPU memory is what stops you from running RL on a bigger model, this method is worth a…