Researchers tested AI coding tools in a randomized trial involving 16 experienced developers. The study found that these tools increased software completion time by 19 percent. Despite developers predicting a 24 percent speedup, they later estimated a 20 percent reduction in completion time. Contributing factors to the slowdown included developer over-optimism regarding AI capabilities and their high familiarity with the code repositories, which minimized the perceived benefits of AI assistance.
After completing the study, developers estimate that allowing AI reduced completion time by 20 percent. Surprisingly, we find that allowing AI actually increases completion time by 19 percent - AI tooling slowed developers down.
The study involved 16 experienced developers who work on large, open source projects. The developers provided a list of real issues (e.g. bug fixes, new features, etc.) they needed to address - 246 in total - and then forecast how long they expected those tasks would take.
Collection
[
|
...
]