“We therefore move from group-wise optimization to a critic-based PPO formulation that learns from individual rollouts, relying on a critic to estimate token-level advantages rather than group-relative comparisons. This single-rollout formulation fits compaction naturally, as it places no constraint on how many traces a prompt produces or on their relative lengths” It is so true that PPO fits better in multi-trace credit assignment for agenti RL. Still surprised how @Zai_org speeds on carrying this through at scale! 🚀 🐐