In the previous post, we found that the same character description could produce different behavior across models. We built CoBRA to give agent specifications a clearer reference: a target level of behavior that we can measure and recalibrate when we change models.
Suppose I need an agent that readily follows the crowd. How readily? What behavior should “a little conformist” and “very conformist” actually produce?
Classic social science experiments give us a starting point. In Asch’s conformity experiment, participants judge line lengths while others unanimously give a wrong answer. We can put agents in scenarios like this and have them respond on a Likert scale, from “strongly disagree” to “strongly agree,” indicating how much they would go along with the incorrect majority. Conformity then becomes a score we can measure and compare.
In CoBRA, researchers specify a target score, have the agent take the experiment, and see how far its score is from that target.
If it hasn’t reached the target, adjust the agent and measure again.
Specify a target → Measure → Adjust → Measure again.
This is CoBRA’s calibration loop. For the adjustment step, CoBRA provides control methods across all three spaces of a model: the prompt, its internal activations, and its parameters. But making a change does not mean calibration is complete. The next measurement tells us whether it worked, and by how much.
We don’t require different models to interpret the same description identically. Instead, we use the same experimental reference to calibrate each toward the target level of behavior.
That is what reproducibility means here: what we reproduce is the measured level of behavior. Calibration gives each model a mapping from control settings to scores, so reaching the same level again, on any model, means finding the matching setting.
After reaching the target, we run a different experiment that measures the same behavior, to check that the change carries over to other tasks rather than just fitting one set of questions.
For me, the core of CoBRA is this closed-loop process: what the researcher specifies, what the agent actually does, and the gap between them can be measured and used to guide adjustments. Agent behavior becomes an experimental condition we can check, calibrate, and reuse.