Research · Updated 16 Oct, 09:30 am IST
Paper shows answer configuration shapes whether traces or evidence drive answers
Why it matters for readers: It explains why a model's visible thought process can both help and mislead when new information arrives.
- Making a separate answer call (rather than continuing the model's own reasoning) increases evidence-following by 29.9 to 60.6 percentage points across three model families on HotpotQA and MuSiQue.1
- Removing the reasoning trace lowers accuracy by similar amounts under both continuation and separate-call configurations, showing traces do raise accuracy even when they conflict with evidence.1
- When traces conflict with current evidence, continuation repeats the stale conclusion in most conflicts, whereas the separate answer call follows the current evidence far more often.1
- Additional analyses show the final line of a trace carries most of its value, gold evidence helps most before the trace is written, distractors weaken correction under separate calls, and renewed reasoning tends to follow current evidence.1
Get a brief like this every morning
Uzha reads hundreds of sources and gives you the stories that matter for your work, with every source linked. Free.
Get started