In our research, we find evidence that the effect of CoT fundamentally depends on generating sequences of words that increase the probability of the correct answer when conditioned upon. Interestingly, our findings suggest that CoT can succeed even in the face of invalid demonstrations, opening up new discussions about the interplay of reasoning and memorization in LLM outputs.
The task of decoding shift ciphers provided a unique venue to analyze the capabilities of LLMs. As we discovered, the most challenging case, rot-13, is also the most frequently occurring in online text, presenting an important test of whether these models rely on stored knowledge or actual reasoning skills.
Collection
[
|
...
]