Men think becoming 'a childless cat lady' is the worst possible thing that could happen to a woman and, therefore, love to use it as an insult; however, most women are not insulted by this because it's not an insult to us. The peace and freedom of that lifestyle actually sound pretty awesome.
The archetype of the witch, characterized by a pointed hat and evil demeanor, emerges from medieval times, reflecting societal fears and misogyny towards women healers.