"After 40 years of persistent badgering, the software industry has convinced the public that bugs can have disastrous consequences. This is great! It is good that people understand that software can result in real-world harm. Not only does the general public mostly understand the dangers, but they mostly understand that bugs can be fixed. It might be expensive, it might be difficult, but it can be done."
"The problem is that this understanding, when applied to AIs like ChatGPT, is completely wrong. The software that runs AI acts very differently to the software that runs most of your computer or your phone. Good, sensible assumptions about bugs in regular software actually end up being harmful and misleading when you try to apply them to AI. Attempting to apply regular-software assumptions to AI systems leads to confusion, and remarks such as:"
""Even if it's hard for one person to understand everything the AI does, surely still smart people who individually understand small parts of what the AI does?" or "Just because current systems don't work perfectly, that's not a problem right? Because eventually we'll iron out all the bugs so the AIs will get more reliable over time, like old software is more reliable than new software.""
The general public believes software bugs can cause real-world harm and that bugs can be fixed, even if fixes are expensive or difficult. Those beliefs are accurate for traditional software but do not apply to modern AI systems like ChatGPT. AI software behaves very differently from desktop or phone software, so assumptions about straightforward debugging, modular understanding by experts, and gradual reliability improvements are misleading. Applying regular-software mental models to AI generates false reassurance and confusion. A communication gap exists between experts and novices: experts assume the gap is obvious, and novices lack the knowledge to recognize it, leaving risks misunderstood.
Read at Boydkane
Unable to calculate read time
Collection
[
|
...
]