Can Implicit Language Reliably Control Agent Behavior?

In social simulation, we often start by writing a character description for an agent: its job, personality, family, and experiences. The description can get very detailed, down to who its friends are and what it likes to do.

This approach rests on two premises:

• LLMs can detect subtle cues in implicit language.
• Once detected, LLMs can role-play the specification.

For example, when we ask an LLM to role-play an economist, we may expect more than an economist’s way of speaking. We may also expect its decisions to be less influenced by framing. But if the prompt only says “economist,” the model is left to interpret what that should mean for its behavior.

We tested this using framing-effect questions. We gave agents three profiles—an ordinary person, an economist, and no character description—and had four models answer the same questions to see whether those expectations held.

They didn’t reliably hold. The same specification produced different behavior across models, and the differences between profiles didn’t consistently match our expectations. With Mistral, the three profiles behaved similarly. With DeepSeek, the economist was more susceptible to framing than the ordinary-person profile.

The character we describe does not automatically become the behavior we intend. The model takes part in defining it too.

How can we make agent behavior measurable and reproducible? We built CoBRA, a toolkit for quantitatively specifying and calibrating agent behavior.