Researchers seeing little evidence of benefit from co pilots

A study by Uplevel on AI coding assistants like GitHub Copilot found no significant productivity gains for developers. The study, assessing PR cycle time and PR throughput, also indicated that coding with Copilot may introduce more bugs. While some criticize the limited capabilities of AI, others report incremental gains, emphasizing the role of AI in accelerating specific aspects of coding.

• Uplevel's study saw no major benefit from GitHub Copilot.
• Pull request cycle time did not decrease with Copilot.
• AI coding assistants were linked to increased bug rates.
• Some devs report 2-3x productivity gains using AI.
• AI tools need careful review to verify generated code.