Performance Regressions Prediction using Time Series Classification: A Case Study

The automatic detection of performance regressions is a key requirement of modern performance testing infrastructures. Recently, this problem has been addressed using change-point detection algorithms, which aim to identify abrupt shifts in time series, thereby enabling the detection of regressions introduced during software evolution. Despite these advancements, the application of time series analysis to performance regression detection still presents many underexplored opportunities. For instance, the field of time series analysis has recently seen a growing adoption of machine learning techniques, which have been successfully applied to a wide range of time series tasks. Building on this trend, in this paper we investigate a novel methodology for performance regression detection in software systems based on learning-driven Time Series Classification (TSC). Specifically, we train three TSC models to predict performance regressions using time series collected from Mozilla Firefox's performance testing infrastructure, and we evaluate their ability to detect previously unseen software performance regressions. The results show that TSC models can effectively detect performance regressions, achieving reasonably high balanced accuracies across all evaluated models (up to 0.738). However, they also reveal notable limitations in terms of false positives that may hinder their practical adoption.