Traditionally, the "teacher-student" model in AI training has been limited by a fundamental ceiling: a student model usually only becomes as good as its teacher. Tencent Hunyuan researchers have now challenged this hierarchy with Generalized On-Policy Distillation (G-OPD). This framework introduces a reward scaling factor that essentially pushes the student to look beyond the teacher's baseline distribution.

By applying a technique called reward extrapolation (ExOPD), the student doesn't just mimic the teacher but learns to refine its own logic on-the-fly. This is particularly effective in math and coding tasks where there are clear "right" and "wrong" answers. The study found that when merging knowledge from various domain experts, the student model actually surpassed the individual teachers it was meant to learn from.

The researchers also discovered that "reward correction"—using a teacher’s base model before it underwent reinforcement learning—provides a cleaner signal for smaller student models. While this adds some computational work, it results in a much more accurate transfer of knowledge. This shift from simple imitation to active logic correction marks a significant step in how we might train the next generation of reasoning-capable models.

Traditionally, the "teacher-student" model in AI training has been limited by a fundamental ceiling: a student model usually only becomes as good as its teacher. Tencent Hunyuan researchers have now challenged this hierarchy with Generalized On-Policy Distillation (G-OPD). This framework introduces a reward scaling factor that essentially pushes the student to look beyond the teacher's baseline distribution.

By applying a technique called reward extrapolation (ExOPD), the student doesn't just mimic the teacher but learns to refine its own logic on-the-fly. This is particularly effective in math and coding tasks where there are clear "right" and "wrong" answers. The study found that when merging knowledge from various domain experts, the student model actually surpassed the individual teachers it was meant to learn from.

The researchers also discovered that "reward correction"—using a teacher’s base model before it underwent reinforcement learning—provides a cleaner signal for smaller student models. While this adds some computational work, it results in a much more accurate transfer of knowledge. This shift from simple imitation to active logic correction marks a significant step in how we might train the next generation of reasoning-capable models.