Building Pi With Pi

Pi is used to build Pi, and issue tracker entries function as inputs for prompts during Pi sessions. Issue shape therefore matters beyond human communication, because agents interpret descriptions to reproduce problems, inspect code, and propose fixes. Vague issues remain annoying, but a new failure mode appears: many issues are largely generated by clankers, with plausible yet wrong diagnoses. When clanker output is confidently inaccurate, it creates extra work through guesswork about root causes, fake-minimal repros, misguided implementation strategies, and irrelevant error-class lists. The most frustrating cases involve issues not written in the reporter’s own voice, where clanker rewording and poor prompting distort observations and conclusions.

"Unsurprisingly, we are using Pi to build Pi. That sounds like a cute dogfooding thing but it really helps understand what we do. An interesting effect of building with agents is that it changes the role of the issue tracker a tiny bit. The issue descriptions are not just messages from a user to a maintainer because we also use them as inputs for prompts in Pi sessions. It is something I might hand to my clanker and say: "understand this, reproduce it, inspect the code, and propose a fix.""

"That means the shape of the issue matters in a new way. A bad issue was always annoying, but at least a lot of issues were vague. Now we are also dealing with a class of issues that are 5% human and 95% clanker-generated and largely inaccurate shit. A bad issue that contains a plausible but wrong diagnosis creates extra work."

"The most frustrating failure mode right now is that people submit issues that are not in their own voice. They contain an observed problem somewhere, but it has been thrown into a clanker and the clanker reworded it and made a huge mess of it. Typically, it was prompted so badly that the conclusions produced are more often than not inaccurate but always full of confidence. The result is complete guesswork on root causes, fake-minimal repros, suggested implementation strategies, analogies to adjacent but often the wrong code, and long lists of error classes that might or might not matter."

"That is worse than no diagnosis. I don't want to point to specific issues because I really do not want to bad mouth anyone, but it i"

Read at Armin Ronacher's Thoughts and Writings