Where AI is failing design systems, and where we are failing AI - bencallahan.com
Briefly

Where AI is failing design systems, and where we are failing AI - bencallahan.com
"In a lot of the work folks are doing to incorporate AI into design systems, AI 's inherently probabilistic output keeps bumping into the deterministic promises our systems must make. We think about our design systems as a contract-a promise we make to our consumers. By their very nature, the large language models that are disrupting every other segment of our work are very fast and really good guessers. But guessing isn't what we promise."
"Jesse expressed it well, "Our [IT] team wants the kind of security assurances that you can get from a deterministic system, which makes stochastic responses... difficult... from a security standpoint." While it's not exactly the same, the phrase he used ("security assurances") mostly maps to what we say design systems do best: establish reliable patterns, tested code, and shared language that product teams can trust."
Data from 95 design system practitioners were collected on August 21, 2025 in response to two focused questions about near-term AI adoption and AI failure points. Responses show that AI excels at some design system tasks but fails at others, creating tension between probabilistic AI outputs and the deterministic guarantees required for production-quality systems. Probabilistic responses undermine security assurances and erode trust because outputs can vary across requests. Reliable design systems require consistent patterns, tested code, and shared language, which current general-purpose AI often cannot provide without additional constraints, validation, and governance.
Read at Bencallahan
Unable to calculate read time
[
|
]