"Comedians who rely on clever wordplay and writers of witty headlines can rest a little easier, for the moment at least, research on AI suggests. Experts from universities in the UK and Italy have been investigating whether large language models (LLMs) understand puns and found them wanting. The team from Cardiff University, in south Wales, and Ca' Foscari University of Venice concluded that LLMs were able to spot the structure of a pun but did not really get the joke."
"An example they tested was: I used to be a comedian, but my life became a joke. If they replaced this with: I used to be a comedian, but my life became chaotic, LLMs still tended to perceive the presence of a pun. They also tried: Long fairy tales have a tendency to dragon. If they replaced dragon with the synonym prolong or even a random word, LLMs seemed to believe there was a pun there."
"The team concluded that when faced with unfamiliar wordplay, the LLMs' success rate in distinguishing puns from sentences without a pun can drop to 20%. Another pun tested was: Old LLMs never die, they just lose their attention. When attention was changed to ukulele, the LLM still perceived it as a pun on the basis that ukulele sounded a bit like you-kill-LLM."
Large language models often detect structural cues of puns but fail to grasp the intended humour. Tests showed models labeled sentences as puns even after key words were replaced with synonyms or random words. Example cases included swapped words in comedian and dragon puns where LLMs still perceived humour. Models rely heavily on memorised training examples and surface patterns rather than semantic understanding. When confronted with unfamiliar or modified wordplay, success rates in distinguishing true puns from non-puns can fall to around 20 percent, revealing a fragile and illusory grasp of humour.
Read at www.theguardian.com
Unable to calculate read time
Collection
[
|
...
]