Timeless foundations : Exploring DC-VAEs as foundation models for time series analysis.

Abstract

We investigate a novel approach to time-series modeling, inspired by the successes of large pre-trained foundation models. We introduce FAE (Foundation Auto-Encoders), a foundation generative-AI model for anomaly detection in time-series data, based on Variational Auto-Encoders (VAEs). By Foundation Model (FM), we mean a model pre-trained on massive amounts of time-series data which can learn complex temporal patterns useful for accurate modeling and forecasting on previously unseen datasets. FAE leverages VAEs and Dilated Convolutional Neural Networks (DCNNs) to build a generic model for time-series modeling, which could eventually perform properly in out-of-the-box, zero-shot anomaly detection applications. We introduce the main concepts and ideas of this FM for time-series (TSFM), and present some preliminary results in a multi-dimensional mobile network monitoring dataset. We also present example results applying novel TSFMs to this dataset, both in a zero-shot manner and relying on fine-tuning, and show how complex it is in the practice to achieve accurate results.