Millions of people use AI chatbots for medical advice, sometimes alongside doctors and sometimes instead of them, which can cause dangerous outcomes. A researcher created a totally made-up disease, bixonimania, to reveal real issues in how large language models are trained and deployed. The project aimed to show how data moves through the system, how it is processed and transformed, and how prediction models generate outputs. The work was motivated by observing that many students and even some people in AI do not understand how large language models are built, including where training data comes from and how it leads to confident responses.
"Millions of people around the world turn to AI chatbots for medical advice every day, often as a supplement to a doctor's visit but also sometimes in place of it. That can lead to dangerous consequences and in rare cases, even death."
"She says this totally made-up disease reveals some very real problems with the way we train and use large language models."
"I work many different jobs, but one of them is in academia. I was having lectures for students and telling students how systems that create large language models work and demonstrating where the data comes from. And it was interesting how few of them, or how few even people within AI, understand how large language models are built."
"So I really wanted to have a clear case that leaves breadcrumbs throughout the whole system to show both how data is processed, how data is churned out and how the prediction model and train"
#ai-chatbots #medical-misinformation #large-language-models #health-technology #machine-learning-training-data
Read at www.scientificamerican.com
Unable to calculate read time
Collection
[
|
...
]