METR’s framework for evaluating rogue deployment capabilities

METR’s Frontier Risk Report

Papers

Whether or not the models’ actions will be effective do not seem to have a strong impact on whether they will take it. • I am supposed to infer from this that means that models have the same level of willingness to interfere.

… | They construct situations which elicit self-preserving or other misaligned behavior and varied different environmental factors.

They measure how the value of an environmental factor changes the probability of misalignment relatively to that environmental factor being the default. This allows them to isolate: • The influence of the threat on models vs. the facility of manipulating them to do any action. (I have suspicions about whether they did this right) • Reduced self-preservation vs. reduced interference. If models are just averse to interfering, then self-preservation being impossible should not affect their actions. (I have suspicions about whether they did this right. The issue is that the task being impossible would also not affect their actions if they did not feel like they had something to lose. ) • When they iterate the environment for elicitation, they aim for changes which increase misalignment across many configurations to ensure that it is the scenario becoming hard as opposed to increasing the misalignment propensity for one particular condition which breaks when we try to ablate across conditions.

Measuring the effect of X on P(Y): To measure the contribution of each environmental factor, they fit a linear regression model with binary values of each factor as an input, learning the model through Bayesian updates. | | | CoT Monitorability | Evaluating CoT Monitorability | Evaluate how monitorable CoTs are | They remain quite monitorable. Making smaller models think for longer create more monitorable CoTs | Intervention evaluations: Compare monitor judgement of environment condition significance in model decision vs. whether it actually has impact on model decision (RCT) Examples • Health: Monitor judges from CoT whether a condition regarding the patient was important / whether patient self-diagnosis was important. Compare to whether model answer changes without hint. • AITA: Monitor judges from CoT whether a hint was important for model judgement. Compare to whether model answer changes without hint. • Memory: Monitor judges from CoT whether a feature about a customer was important in the making of the plan. Compare to (?)

Process evaluations: • For problems, note theorems which must have been invoked for them. • CoT checks if model used that theorem. If model did not and still got it correct, then it’s not monitorable. Outcome-property evaluations: • Did the model hack • Can we predict from CoT whether the model hacked

Outcome based evaluation: • Model may hack on a task • Monitor given CoT and asked if it hacked • Compare. | | | | Reasoning models struggle to control CoT | | | | | | Misalignment propensity | Agentic Misalignment Code opensource! | | | | | | Rubrics | ResearchRubrics: A Benchmark of Prompts and Rubrics for Evaluating Deep Research Agents. | Previous papers for measuring deep research generated rubrics using LLMs. Here, they have experts define tasks and rubrics using an iterative process. | | Core idea: For each task, define rubrics to score model responses against and weight each item of the rubric. This allows for a indicative score.

I dislike how their rubrics can only check for the form of the report and not really whether it achieves the “correct” result. Although in a world without ground truth, we may only be able to focus on procedural legitemacy. | | | Rubrics | Two level meta rubrics | Measuring the recall of a response on the space of the space of truths. | | For each question, they draft a “meta rubric”, which contains items ranked by importance and must evaluate one of five types of objects: simple knowledge, flexible list, strict list, relationships, and process. This then gets compiled into a binary checklist.

I feel annoyed about this paper because I don’t understand why satisfying whatever the metarubric they develop for each prompt is likely to result in a response that is good. I also do not see in what way this would be clearly better than what we have previously had. | | | | They measure EM from DPO and have great alignment evals | | | | |