"Avoid reducing the size of CI regression test suites, particularly at the integration and end-to-end test levels, because test set reduction can make subtle, high-impact bugs invisible by shrinking the result sample size."
"Shifting focus from individual test failures to a stochastic approach based on time-series trend analysis makes it possible to effectively manage even very large regression test sets."
"Adopting this stochastic approach will give you your best bet of catching the often subtle signals of the regressions uncovered by your tests over successive test runs."
"Improve CI lab speed, feedback times, and capacity through architectural measures like parallelization, continuous reporting, mocking and hardware-in-the-loop, rather than by cutting the average number of regression tests executed with each build."
Reducing the size of CI regression test suites can lead to missing subtle, high-impact bugs. A stochastic approach based on time-series trend analysis allows for effective management of large regression test sets. This method helps in catching subtle regression signals over successive test runs. Additionally, using multicontext pattern matching can identify regressions with high confidence. Improving CI lab speed and feedback times should focus on architectural measures like parallelization and continuous reporting, rather than reducing the number of tests executed per build.
Read at InfoQ
Unable to calculate read time
Collection
[
|
...
]