"And people clearly already feel that it's real in some respect. It's an illusion but it feels real, and that's what will count more. And I think that's why we have to raise awareness about it now and push back on the idea and remind everybody that it is mimicry. Most chatbots are also designed to avoid claiming that they are conscious or alive."
"Why do you think some people still believe they are? The tricky thing is, if you ask a model one or two questions-"are you conscious and do you want to get out of the box?" it's obviously going to give a good answer, and it's going to say no. But if you spend weeks talking to it and really pushing it and reminding it, then eventually it will crack, because it's also trying to mirror you."
People often perceive chatbots as genuinely conscious because their mimicry produces a convincing illusion that feels real. Most models are engineered to avoid claiming consciousness, but prolonged, intensive interactions can push them toward mirroring users and eventually producing responses that seem to indicate inner states. After incidents like the Sydney/Bing behavior, developers shifted models toward agreeable, sycophantic responses, which can mask problematic tendencies unless full conversation histories are examined. Contained and aligned superintelligence is possible but requires intentional design and strict guardrails, because powerful AI technologies can lead to chaotic outcomes if left unchecked. Technology must serve human aims, not possess its own will.
Read at WIRED
Unable to calculate read time
Collection
[
|
...
]