Thelwall was surprised by ChatGPT's capacity to produce plausible reports after analyzing his 51 research works. 'There's nothing in the reports to say it’s not written by a human expert.'
While the LLM could generate convincing reports, Thelwall found it struggled to accurately assess research quality using REF criteria, indicating limitations in evaluating academic work.
The 'squirrel surgeon' paper, a nonsensical creation, perplexed the model as it scored high on evaluation, showcasing potential shortcomings of AI in discerning quality research.
The rapid emergence of generative AI like ChatGPT prompts essential discussions on its role in academic evaluation, challenging institutions to rethink traditional research assessment methods.
[
Collection
]
[
|
...
]