Experienced developers can take 19% longer to complete tasks when using AI assistants, challenging the tech industry's narrative about AI coding tools. Research tracked 16 seasoned developers completing 246 coding tasks on extensive repositories. Developers initially believed AI tools would decrease task completion time by 24%, but even after experiencing slower performance, they estimated productivity improved by 20%. This perception gap indicates that many may misjudge AI's impact, with experts also overestimating productivity gains significantly. Organizations need to recognize that improved coding experience does not always equate to faster output.
We conduct a randomized controlled trial (RCT) to understand how early-2025 AI tools affect the productivity of experienced open-source developers working on their own repositories.
Surprisingly, we find that when developers use AI tools, they take 19% longer than without - AI makes them slower.
Perhaps most striking is the disconnect between perception and reality. Before starting the study, developers predicted AI tools would reduce their completion time by 24%.
When people report that AI has accelerated their work, they might be wrong.
Collection
[
|
...
]