(Fascinating) Han May ‘26: How’s it going?

• Found a direction which represents how well the model is performing on the task at hand, which generalizes to improving/harming confidence, backtracking, and compliance (?) across tasks.
  • I always wonder how robust these directions are to additional training.
• They found that this direction exists prior to RL, so they hypothesize that RL recruits directions which represent goodness in the specific task at hand to the model’s internal representation of “doing well”.