The world's 'first AI software engineer' isn't living up to expectations: Cognition AI's 'Devin' assistant was touted as a game changer for developers, but so far it's fumbling tasks and struggling to compete with human workers

Devin, developed by Cognition AI, has faced scrutiny after being touted as the world's 'first AI software engineer'. When tasked with 20 coding projects, it completed only three, and took longer than expected while veering off track. Originally hailed as a revolutionary tool for software development, Devin's capabilities were called into question by both a software developer and researchers. Despite some early successes in certain tasks, overall performance did not meet the high expectations set during its launch, sparking debate over its actual effectiveness.

At Devin's launch last year, Cognition claimed that the tool could "make money taking on messy Upwork tasks," sharing a video purporting to show just that.

However, software developer Carl Brown posted his own video in response, arguing that the company was not telling the truth about the tool's abilities.

Cognition's claims about Devin were also tested by a team of researchers at Answer.AI, and their results were closer to Brown's than what the original blog post claimed.

Read at ITPro