Deep generative models for time-series anomaly detection.

Abstract

Time series analysis has become a prominent area of study driven by the explosive growth of data generation a trend that continues to accelerate. Real time anomaly detection in time series is a crucial and challenging problem. Behind an anomaly may lie an ongoing system attack, a potential failure that could escalate, or even fraudulent activities. Anomalies are inherently rare, isolated events that are atypical and often unpredictable. They often lack consistent patterns and may evolve over time, further complicating their identification. Additionally,monitoring systems typically handle numerous time series, each with its own unique behavior. In some cases, certain time series may exhibit causal relationships with others, which could contain important information to take into account. In this thesis, we present a novel and versatile approach for modeling the normal behavior of multivariate and univariate time-series using generative deep learning models. At its core, our methodology leverages Variational Autoencoders (VAEs) to construct robust representations of typical patterns in data, addressing critical challenges in anomaly detection. These challenges include handling limited or incomplete information about anomalies and capturing causal and temporal dependencies across diverse time-series. A central contribution of this work is the development of the Dilated Convolutional Variational Autoencoder (DC-VAE), a lightweight and scalable generative model tailored to capture the distribution of normal behavior within the variables of a system. DC-VAE operates effectively in two configurations: a multivariate approach that models all variables of a system as a single multivariate time-series and a global approach that treats individual time-series of the same system independently within one model. By integrating dilated convolutions, DC-VAE efficiently models long temporal patterns without compromising training or inference time, maintaining its lightweight design. This method, tested on the real TELCO dataset, demonstrates superior performance over more time-series than methods that require training or fixing specific models for each individual time-series. It also outperforms other multivariate deep learning methods on datasets that are popular in the community. To enhance adaptability and extend the utility of DC-VAE, we introduce GenDeX, a continual learning mechanism that addresses catastrophic forgetting. This mechanism enables the DC-VAE model to retain knowledge of previously learned series while seamlessly incorporating new ones, ensuring stable performance in both reconstruction and anomaly detection tasks. GenDeX proves effective not only for handling domain changes (such as adding or dropping time-series from the model) but also for dealing with more common challenges in time-series problems, such as concept drift. Building upon these foundations, we propose the Foundation Auto-Encoder (FAE), a pre-trained global model developed on the UCR’21 dataset, which encompasses a diverse range of time-series from multiple domains. FAE demonstrates exceptional zero-shot learning capabilities, achieving competitive anomaly detection performance even without prior exposure to specific series. When applied to the TELCO dataset, FAE not only maintains strong reconstruction quality but also highlights its foundational properties, enabling generalization across datasets and tasks. Different experiments validate the effectiveness of our approach. DC-VAE achieves good performance in anomaly detection, while GenDeX ensures stability and knowledge retention in dynamic environments. FAE showcases the potential of foundation models for time-series analysis, offering a scalable and interpretable solution for monitoring, anomaly detection, and continual learning. These advancements underscore the versatility and practicality of deep generative models in real-world applications. For the sake of reproducibility and as an additional contribution, we make the TELCO dataset publicly available to the community and openly release the code implementing DC-VAE, GenDeX, and FAE.