Mar 22, 2026 估 KL penalty 的三个 estimator / Three ways to estimate the KL penalty Mar 15, 2026 从 PPO 到 SAPO：每一代 RL 算法究竟在改什么 / From PPO to SAPO, what each version actually changes