TAB: Unified Benchmarking of Time Series Anomaly Detection Methods

Table of Contents

1. Introduction

2. Quickstart

3. Time series characteristics

4. Full results

Introduction

TAB is an open-source library designed for time series anomaly detection researchers.

We provide a clean codebase for end-to-end evaluation of time series anomaly detection models, comparing their performance with baseline algorithms under various evaluation strategies and metrics.

The below figure provides a visual overview of TAB's pipeline.

The table below provides a visual overview of how TAB's key features compare to other libraries for time series anomaly detection.

Quickstart

Installation

Follow the steps below to configure the TAB environment.

1. Install Git

Some models require Git to run. Please ensure that Git is installed on your system. If not, you can install it using the following command:

apt-get install git

2. Download and Extract Environment

Download the environment from Google Drive. (This may take some time, please wait patiently.) Then, create a directory my_env (we recommend creating it under conda/envs/) and extract the environment into this directory:

mkdir -p my_env
tar -xzf tab.tar.gz -C my_env

3. Activate the Environment

source my_env/bin/activate

4. Clean Environment Prefix

You can clear the environment prefix using the following command. Now the environment is the same as one created directly with conda at this path.

(my_env) $ conda-unpack

Data preparation

Prepare Data. You can obtain the well pre-processed datasets from Google Drive. Then place the downloaded data under the folder ./dataset.

Data preparation

Prepare Data. You can obtain the well pre-processed datasets from Google Drive. Then place the downloaded data under the folder ./dataset.

Checkpoints preparation

You can download the checkpoints from Google Drive. After obtaining the files, follow the steps below to organize them:

1. For LLM-based models, move the files from checkpoints/llm_checkpoints to the folder ts_benchmark/baselines/LLM/checkpoints.
2. For pre-train models (UniTS, Timer and TimesFM), move the files from checkpoints/pre_train_checkpoints to the folder ts_benchmark/baselines/pre_train/checkpoints. And the other models' checkpoints can be obtained from Huggingface by the code.

Ensure the files are placed in the correct directories for proper functionality.

Train and evaluate model

We provide the experiment scripts for all benchmarks under the folder ./scripts/multivariate_detection, and ./scripts/univariate_detection. For example, you can reproduce an experiment result as the following:

sh ./scripts/multivariate_detection/detect_label/MSL_script/ModernTCN.sh

Time series characteristics

We provide code and pseudocode for five characteristics used in TAB.

Full results

You can get the results from here:

• Multivariate dataset results
• Univariate dataset results