#programming-uncertainty

#programming-uncertainty

This extends to the software development community, which is seeing a near-ubiquitous presence of AI-coding assistants as teams face pressures to generate more output in less time. While the huge spike in efficiencies greatly helps them, these teams too often fail to incorporate adequate safety controls and practices into AI deployments. The resulting risks leave their organizations exposed, and developers will struggle to backtrack in tracing and identifying where - and how - a security gap occurred.

A new generation of tools that let anyone - designers, marketers, founders, students - describe an app in plain English and watch it get built in real time. No compiler knowledge. No debugging in terminals. No Stack Overflow. Just a conversation with a machine that builds things.

The software industry is collectively hallucinating a familiar fantasy. We visited versions of it in the 2000s with offshoring and again in the 2010s with microservices. Each time, the dream was identical: a silver bullet for developer productivity, a lever managers can pull to make delivery faster, cheaper, and better. Today, that lever is generative AI, and the pitch is seductively simple: If shipping is bottlenecked by writing code, and large language models can write code instantly, then using an LLM means velocity should explode.

One thing I always do when I prompt a coding agent is to tell it to ask me any questions that it might have about what I've asked it to do. (I need to add this to my default system prompt...) And, holy mackerel, if it doesn't ask good questions. It almost always asks me things that I should have thought of myself.