Reasoning Behaviors in Thinking Models
Thinking models amplify visible deliberation, but not the behaviors most associated with correct answers.
Across 15 open-weight LLMs and VLMs, 6 benchmarks, and 15,282 annotated reasoning traces, we ask whether thinking models amplify the behaviors most associated with correct answers. We find a consistent amplification-lift gap: thinking models strongly amplify self-correction, hypothesis testing, and uncertainty acknowledgment, while the highest-lift behaviors are confidence calibration, knowledge alignment, and self-awareness.
Accuracy tells us whether an answer is right. These metrics ask what the trace is doing along the way.
Measures how much correctness changes when a behavior appears in the trace. Lift separates prevalence from association with correctness.
Measures whether a model reaches the correct answer despite a detected reasoning failure.
The taxonomy lets us compare language-only and vision-language reasoning using the same behavioral labels.
Thinking models produce far more self-correction, hypothesis testing, and uncertainty acknowledgment than instruct models. Confidence calibration and knowledge alignment move much less.
Confidence calibration is the strongest positive signal of correctness in both modalities, but thinking models barely amplify it. Uncertainty acknowledgment is heavily amplified, yet weakly or negatively associated with correctness.
| Behavior | VLM Lift | LLM Lift |
|---|---|---|
| Confidence calibration | +72.2 | +79.6 |
| Knowledge alignment | +53.7 | +80.3 |
| Self-awareness | +62.0 | +52.7 |
| Self-correction | +20.1 | +12.4 |
| Hypothesis testing | +1.0 | +1.0 |
| Uncertainty acknowledgment | -16.1 | -13.9 |
The mismatch survives within individual benchmarks. Confidence calibration and knowledge alignment remain high-lift across tasks, while uncertainty acknowledgment stays low or negative.
We also validate the ranking with same-question controls, manual checks, hidden-state probes, and length controls in the paper.
On extended-reasoning tasks, thinking models reach correct answers after detected failures at 2-3x the rate of instruct models. But on LogiQA2, where fast recognition of logical form can be enough, instruct models recover better.
Among traces with at least one detected failure, recovery is highest when the trace shows confidence calibration, knowledge alignment, self-correction, or self-awareness. Some frequent behaviors, including planning and evidence citation, add little recovery signal in this conditional view.
Larger thinking models still strongly amplify self-correction and uncertainty behaviors. Meanwhile, confidence calibration narrows across model types and remains high-lift across model sizes.
A calibrated trace becomes more confident when independent checks agree. A visibly uncertain trace can still fail when doubt never resolves into better reasoning.
Question: If 40 calories is 2% of a person's daily requirement, what is the full daily requirement?
The trace solves 0.02x = 40, giving x = 2000.
It then checks the result in percentage terms: 1% is 20, so 100% is 2000.
Both methods give me 2000. That makes me more confident.
Question: A skater starts facing north and spins 2250 degrees to the right. Which direction is she facing?
The trace reduces 2250 degrees modulo 360 and identifies a 90-degree remainder.
It first maps that turn incorrectly, then reconsiders: "Wait, I think I might be confused... If you start facing north and turn clockwise 90 degrees, you face east."
The doubt is explicit, but the trace still settles on west. The ground truth is east.
Use thinking models where they help, then audit which behaviors their traces reward.
A trace feature can become common after reasoning-oriented training while adding little correctness signal. Check whether correctness actually changes when it appears.
Uncertainty language alone cannot make a trace reliable. Useful uncertainty tracks evidence: confidence should rise and fall with the strength of the reasoning.
Thinking can help by preventing failures, recovering from them, improving calibration, or changing shortcuts. Accuracy alone hides these mechanisms.
Process supervision should reward grounded claims, appropriate domain framing, recognition of underspecified information, and recoverable reasoning instead of deliberative surface form alone.
We release the behavioral annotations, taxonomy definitions, judge prompts, and metric code for computing Behavioral Lift and Recovery Rate.
@article{nyandwi2026amplified,
title = {Amplified Does Not Mean Predictive: Reasoning Behaviors in Thinking Models},
author = {Nyandwi, Jean de Dieu and Mathur, Leena and Bisk, Yonatan and Neubig, Graham},
year = {2026}
}