Generating code using AI increases the number of issues that need to be reviewed and the severity of those issues. CodeRabbit, an AI-based code review platform, made that determination by looking at 470 open source pull requests for its State of AI vs Human Code Generation report. The report finds that AI-generated code contains significantly more defects of logic, maintainability, security, and performance than code created by people.
Based on over 4,400 Java tasks, the report finds that depending on which of the four levels of reasoning capabilities that OpenAI now makes available, the overall quality of the code, especially in terms of the vulnerabilities generated, significantly improves. However, the overall volume of code being generated per task also substantially increases, which creates additional maintenance challenges for application developers that are not going to be familiar with how code might have been constructed in the first place.
