Non profit AI research institution METR: Senior programmers using AI programming may actually slow down development speed

According to Reuters, the latest research by non-profit organization METR has found that, contrary to popular belief, experienced software developers become slower rather than faster when using state-of-the-art AI tools to assist familiar code repositories.

The organization conducted an in-depth investigation on a group of senior developers who use the popular AI programming assistant cursor to complete their familiar open source project tasks.

Before the research began, these developers expected that AI could improve their work speed and reduce task time by an estimated 24%. After completing the task, they still feel that they have saved 20% of their time. But research data shows that AI actually increases task completion time by 19%.

The research leaders Joel Baker and Nate Rush said that this result is very surprising. Rush even expected the speed to double before the research.

This discovery challenges the common belief that AI can significantly improve the efficiency of high paying engineers, which is an important reason for attracting a large amount of investment into the field of AI software development. Moreover, some even view AI as a "substitute" for entry-level programmer positions.

In previous studies, AI has achieved significant results in improving development efficiency: one showed a 56% increase in programmer speed, and another showed that developers could complete 26% more tasks in a fixed amount of time.

But METR's new research suggests that these positive results do not apply to all situations. Especially for experienced developers familiar with large mature open-source code repositories, there has been a decrease in efficiency.

The research author points out that many related studies rely on benchmark tests developed by AI, which may not accurately reflect real-world work tasks. IT Home learned from the report that the main reason for the slowdown in efficiency is that developers need to spend time checking and correcting the code suggestions provided by AI.

Baker said, "The direction of the suggestions proposed by AI is basically correct, but the details do not fully meet the actual needs. ”

The author emphasizes that this kind of slowing down is unlikely to occur among novice developers or engineers who are not familiar with the code repository.

However, most of the developers and authors involved in the research still use cursor. They believe that AI makes the development process easier and more enjoyable, like editing an article instead of writing from scratch. Baker said, "The goal of developers is not only to complete tasks as soon as possible, but they are more willing to choose this path with less effort