"They were split into two groups: one with access to an AI assistant during the task, and a control group limited to documentation and web search. Both groups were asked to complete two coding tasks as quickly as possible, followed by a quiz designed to assess understanding. The test measured debugging ability, code reading, code writing, and conceptual understanding. No AI assistance was allowed during the follow-up quiz."
"AI Help Not Good for Learning Skills Developers who used AI scored an average of 50 percent on the quiz, compared with 67 percent for those who coded without AI assistance, a gap equivalent to nearly two letter grades. The largest difference appeared in debugging questions. AI users completed the tasks slightly faster, by about two minutes on average, but the difference was not statistically significant."
A randomized controlled experiment with 52 mostly junior software engineers tested learning Trio, a Python library for asynchronous programming that none had used before. Participants had at least a year of Python experience and prior exposure to AI coding tools. One group had access to an AI assistant during coding tasks; a control group used only documentation and web search. Both groups completed two timed coding tasks and a follow-up quiz without AI measuring debugging, code reading, code writing, and conceptual understanding. Participants who used AI scored 50 percent on the quiz versus 67 percent for the control group, with the largest deficit in debugging ability. AI users completed tasks slightly faster but not significantly, and some spent nearly a third of their time interacting with the AI.
Read at DevOps.com
Unable to calculate read time
Collection
[
|
...
]