AI Models Can Learn from Answers That Already Encode the Reasoning, Study Finds

Chain-of-thought supervision has become a common approach when fine-tuning AI models for reasoning tasks. The idea is straightforward: showing a model the steps behind an answer should teach it to reason more deeply. But this idea doesn’t always hold.

A team comprising alumni Shlok Hiren Shelat and Shrey Salvi, along with Professor Souvik Roy from Ahmedabad University and Manas Gaur of the University of Maryland, challenges this assumption. Their research suggests that whether step-by-step reasoning helps or hurts during fine-tuning depends not on how difficult a task is or its domain, but on a specific property of the answer itself.

Their paper, titled Chain-of-Thought Hurts When the Answer Speaks for Itself, has been accepted at EMNLP 2026 (Core A*), the 2026 Conference on Empirical Methods in Natural Language Processing, to be held in October in Hungary. The work introduces the concept of answer backtrackability, which refers to how easily a question can be reconstructed from its answer alone, as a way to predict when reasoning supervision is useful and when it is unnecessary.

The team tested this idea by fine-tuning three open-source language models across five datasets covering two contrasting settings. Three established mathematical benchmarks, including GSM8K, MATH, and MathInstruct, represent low-backtrackability answers, in which a final number provides no clue about the reasoning that produced it. In contrast, the team built two benchmarks based on automata theory, in which answers take the form of complete state transition tables that largely show how the solution was constructed, thereby representing high-backtrackability answers.

The results showed a pattern. On mathematical datasets, chain-of-thought fine-tuning outperformed training on final answers alone, supporting the existing intuition. On the automata benchmarks, the pattern reversed. Models trained without reasoning traces performed better than those trained with them across all models and difficulty tiers, with the advantage growing as problems became more complex.

Within a single dataset, MathInstruct, the same models changed preference depending on whether the answer format was code, which reveals its own logic, or natural language, which does not — the clearest evidence that the answer’s structure, not the dataset or model, determines the outcome.

The work shows how a model’s fine-tuning strategy should be matched to the structure of its answers, rather than adding more supervision. Rather than assuming reasoning traces are always useful, the research points toward identifying when intermediate supervision adds useful information and when it simply adds noise.