A Comparative Study of Oracle’s Anomaly Detection Solution and Modern Alternatives in Time Series Prognostics

Time series anomaly detection is a difficult problem that has been studied in a broader spectrum of research areas due to its diverse applications in different domains. Despite significant progress in this field, including the widespread adoption of modern machine learning algorithms, no single anomaly detection method has proven to generalize effectively across all time series datasets. Nevertheless, the adoption of deep learning techniques—particularly the Long Short Term Memory (LSTM) algorithm—for time series analysis has continued to grow in both academia and among major cloud service providers. The increase in the usage of LSTM is largely driven by the belief that neural networks (NN)—given their success in many other domains—can be generalized for all predictive tasks, along with the widespread availability of open-source implementations. However, there are alternatives to LSTM that may be better suited to address the unique challenges of time series analysis—one such method is MSET (Multivariate State Estimation Technique). In this study, we conducted a comprehensive comparative evaluation of MSET against other state-of-the-art techniques from the literature to better understand its value proposition. A benchmark test bed is developed to evaluate the detection results, reconstruction accuracy, and computational cost of the anomaly detection techniques of being studied. The benchmark datasets consist of synthetic datasets and publicly available datasets. MSET is demonstrated to achieve a higher F1 score, on average, than LSTM in most cases, and deliver an advantage over other competing methods in regards to false alarms, reconstruction accuracy, and computational cost. Lastly, although the explainability cannot be quantified in our study, we showcase it is a key value proposition of MSET favored by the IoT industries targeted by MSET.