In our research, we find evidence that the effect of CoT fundamentally depends on generating sequences of words that increase the probability of the correct answer when conditioned upon. Interestingly, our findings suggest that CoT can succeed even in the face of invalid demonstrations, opening up new discussions about the interplay of reasoning and memorization in LLM outputs.