"Re-run rate is how often developers manually retry tests. When they're hitting re-runs on more than 30% of their PRs, they're not testing - they're gambling. That 30% mark is where I've consistently seen the mental shift: Below it, developers assume failures are their fault. Above it, they assume it's the infrastructure's fault."
"Time to confidence is how long developers wait before trusting that a merge is safe. Healthy systems? Under 10 minutes. But I've worked with teams where developers sit for 20-30 minutes, cross-referencing test history, pinging teammates: 'Did this test pass for you?' That's not testing. That's collective anxiety management."
"Override rate tracks forced merges despite failures. When it crosses 5%, your test suite has lost all credibility. Developers are explicitly saying, 'I don't believe these results,' and shipping anyway."
"Trust doesn't fade gradually. It falls off a cliff. The cycle takes six to eight weeks. Flaky tests creep in. Developers blame their own code at first. By week three, they notice the pattern - same tests, same failures, nothing to do with their changes."
Developers often face issues with test infrastructure credibility, leading to a lack of trust in test results. Key metrics include re-run rate, time to confidence, and override rate. A re-run rate above 30% indicates developers are not testing effectively. Healthy systems have a time to confidence under 10 minutes, while an override rate above 5% shows a loss of trust in the test suite. Trust erodes quickly, with developers becoming disillusioned within six to eight weeks due to flaky tests and repeated failures.
Read at DevOps.com
Unable to calculate read time
Collection
[
|
...
]