Skip to content

Improved prompting and task type decisions for multi agent model #39

New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

Sign up for GitHub

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Open
wants to merge 4 commits into
base: ltsm-stack
Choose a base branch
from
Open

Improved prompting and task type decisions for multi agent model #39

Show file tree
Hide file tree
Changes from all commits
Commits
Show all changes
4 commits
Select commit Hold shift + click to select a range
722ccdf
Changed inputs to LTSM
joshhan619 Apr 28, 2025
30e4f7b
Better prompting for planner and task type decision
joshhan619 Apr 28, 2025
634c9e2
Changed behavior of QA and TS agent on different task types
joshhan619 Apr 28, 2025
763f22b
Updated Readme training examples
joshhan619 May 5, 2025
File filter

Filter by extension

Filter by extension
Conversations
Failed to load comments.
Loading
Jump to
Jump to file
Failed to load files.
Loading
Diff view
Diff view
268 changes: 227 additions & 41 deletions README.md
View file
Open in desktop
Original file line number Diff line number Diff line change
@@ -1,74 +1,260 @@
# Understanding Different Design Choices in Training Large Time Series Models
<img width="700" height="290" src="./imgs/ltsm_model.png">
# LTSM-Bundle: A Toolbox and Benchmark on Large Language Models for Time Series Forecasting

<div align="center">
<img src="./imgs/ltsm_model.png" width="700" height="290" alt="LTSM Model">
</div>

[![Test](https://github.com/daochenzha/ltsm/actions/workflows/test.yml/badge.svg)](https://github.com/daochenzha/ltsm/actions/workflows/test.yml)

This work investigates the transition from traditional Time Series Forecasting (TSF) to Large Time Series Models (LTSMs), leveraging universal transformer-based models. Training LTSMs on diverse time series data introduces challenges due to varying frequencies, dimensions, and patterns. We explore various design choices for LTSMs, including pre-processing, model configurations, and dataset setups. We introduce **Time Series Prompt**, a statistical prompting strategy, and $\texttt{LTSM-bundle}$, which encapsulates the most effective design practices identified. $\texttt{LTSM-bundle}$ is developed by [Data Lab at Rice University](https://cs.rice.edu/~xh37/).
> Empowering forecasts with precision and efficiency.

## Table of Contents

* [Overview](#overview)
* [Why LTSM-bundle](#why-ltsm-bundle)
* [Features](#features)
* [Installation](#installation)
* [Quick Start](#quick-start)
* [Project Structure](#project-structure)
* [Datasets and Prompts](#datasets-and-prompts)
* [Model Access](#model-access)
* [Cite This Work](#cite-this-work)
* [License](#license)
* [Acknowledgments](#acknowledgments)

---

## Resources
:star2: Please star our repo to follow the latest updates on LTSM-bundle!
## Overview

:mega: We have released our [paper](https://arxiv.org/abs/2406.14045) and source code of LTSM-bundle-v1.0!
This work investigates the transition from traditional Time Series Forecasting (TSF) to Large Time Series Models (LTSMs), leveraging large transformer-based models like GPT. Training LTSMs on diverse time series data introduces challenges due to varying frequencies, dimensions, and patterns.

:books: Follow our latest [English Tutorial](https://github.com/daochenzha/ltsm/tree/main/tutorial) or [中文教程](https://zhuanlan.zhihu.com/p/708804309) to costomize your LTSM!
We explore multiple design choices, including pre-processing strategies, tokenization, model architectures, and dataset setups. We introduce:

:earth_americas: For more information, please visit:
* Paper: [https://arxiv.org/abs/2406.14045](https://arxiv.org/abs/2406.14045)
* Blog: [Time Series Are Not That Different for LLMs](https://towardsdatascience.com/time-series-are-not-that-different-for-llms-56435dc7d2b1)
* Tutorial: [Build your own LTSM-bundle](https://github.com/daochenzha/ltsm/tree/main/tutorial)
* Chinese Tutorial: [https://zhuanlan.zhihu.com/p/708804309](https://zhuanlan.zhihu.com/p/708804309)
* Do you want to learn more about data pipeline search? Please check out our [data-centric AI survey](https://arxiv.org/abs/2303.10158) and [data-centric AI resources](https://github.com/daochenzha/data-centric-AI) !
* **Time Series Prompt**: A statistical prompting strategy
* **LTSM-bundle**: A toolkit encapsulating effective design practices

The project is developed by the [Data Lab at Rice University](https://cs.rice.edu/~xh37/).

---

## Why LTSM-bundle?
The LTSM-bundle package leverages the HuggingFace transformers toolkit, offering flexibility to switch between different advanced language models as the backbone. It is easy to tailor the general LTSMs to their specific time series forecasting needs by selecting the most suitable language model from a wide array of options. The flexibility enhances the adaptability of the package across different industries and data types, ensuring optimal performance in diverse scenarios.

The LTSM-bundle leverages HuggingFace transformers, allowing flexible integration of large-scale pre-trained language models for time series tasks. Users can customize the pipeline to fit specific forecasting needs with minimal overhead, making it adaptable across various domains and industries.

Key highlights:

* Plug-and-play with GPT-style backbones
* Modular pipeline for easy experimentation
* Support for statistical and text prompts

---

## Features

| Category | Highlights |
| ----------------- | ----------------------------------------------------------------------------------------------------------------------------------- |
| ⚙️ Architecture | Modular design, GPT-style transformers for time series |
| 📝 Prompting | Time Series Prompt & Text Prompt support |
| ⚡️ Performance | GPU acceleration, optimized pipelines |
| 🔧 Integrations | LoRA support, JSON/CSV-based dataset and prompt interfaces |
| 🔬 Testing | Unit and integration tests, GitHub Actions CI |
| 📊 Data | Built-in data loaders, scalers, and tokenizers |
| 📂 Documentation | Tutorials in [English](https://github.com/daochenzha/ltsm/tree/main/tutorial) and [Chinese](https://zhuanlan.zhihu.com/p/708804309) |

---

## Installation
```

We recommend using Conda:

```bash
conda create -n ltsm python=3.8.0
conda activate ltsm
git clone [email protected]:daochenzha/ltsm.git
cd ltsm
pip3 install -e .
pip3 install -r requirements.txt
```

## Quick Exploration on LTSM-bundle
Then install the package:

Training on **[Time Series Prompt]** and **[Linear Tokenization]**
```bash
bash scripts/train_ltsm_csv.sh
git clone https://github.com/datamllab/ltsm.git
cd ltsm
pip install -e .
pip install -r requirements.txt
```

Training on **[Text Prompt]** and **[Linear Tokenization]**
```bash
bash scripts/train_ltsm_textprompt_csv.sh
---

## 🔧 Training Examples
```Python
from ltsm.data_pipeline import StatisticalTrainingPipeline, get_args, seed_all
from ltsm.models.base_config import LTSMConfig
from ltsm.common.base_training_pipeline import TrainingConfig

# Option 1: Load config via command-line arguments
config = get_args()

# Option 2: Load config from a JSON file
config = TrainingConfig.load("example.json")

# Option 3: Manually customize a supported model config in code
# (e.g., LTSMConfig, DLinearConfig, InformerConfig, etc.)
config = LTSMConfig(seq_len=336, pred_len=96)

# Set random seeds for reproducibility
seed = config.train_params["seed"]
seed_all(seed)

# Initialize the training pipeline with the loaded config
pipeline = StatisticalTrainingPipeline(config)

# Run the training and evaluation process
pipeline.run()
```

Training on **[Time Series Prompt]** and **[Time Series Tokenization]**
```bash
bash scripts/train_ltsm_tokenizer_csv.sh
## 🔍 Inference Examples

```Python
import os
import torch
import pandas as pd
from huggingface_hub import hf_hub_download
from safetensors.torch import load_file
from ltsm.models import LTSMConfig, ltsm_model

# Download model config and weights from Hugging Face
config_path = hf_hub_download("LSC2204/LTSM-bundle", "config.json")
weights_path = hf_hub_download("LSC2204/LTSM-bundle", "model.safetensors")

# Load model and weights
model_config = LTSMConfig()
model_config.load(config_path)
model = ltsm_model.LTSM(model_config)

state_dict = load_file(weights_path)
model.load_state_dict(state_dict)

# Move model to device
device = torch.device("cuda" if torch.cuda.is_available() else "cpu")
model = model.to(device).eval()

# Load your dataset (e.g., weather)
df_weather = pd.read_csv("/path/to/dataset.csv")
print("Loaded data shape:", df_weather.shape)

# Load prompts per feature
feature_prompts = {}
prompt_dir = "/path/to/prompts/"
for feature, filename in {
"T (degC)": "weather_T (degC)_prompt.pth.tar",
"rain (mm)": "weather_rain (mm)_prompt.pth.tar"
}.items():
prompt_tensor = torch.load(os.path.join(prompt_dir, filename))
feature_prompts[feature] = prompt_tensor.squeeze(0).float().to(device)

# Predict (custom code here depending on your model usage)
# For example:
with torch.no_grad():
inputs = feature_prompts["T (degC)"].unsqueeze(0)
preds = model(inputs)
print("Prediction output shape:", preds.shape)
```

## Datasets and Time Series Prompts
Download the datasets
---


## Project Structure
<!-- Zihang please helps this part -->
<!-- Modify this tree structure to better meets the current package -->

```text
└── ltsm/
├── datasets
│ └── README.md
├── imgs
│ ├── ltsm_model.png
│ ├── prompt_csv_tsne.png
│ └── stat_prompt.png
├── ltsm
│ ├── data_pipeline
│ ├── data_provider
│ ├── models
│ └── utils
├── main_ltsm.py
├── main_tokenizer.py
├── prompt_bank
│ ├── prompt_data_normalize_split
│ ├── stat-prompt
│ └── text_prompt_data_csv
├── requirements.txt
├── scripts
│ ├── test_csv_lora.sh
│ ├── test_ltsm.sh
│ ├── train_ltsm_csv.sh
│ ├── train_ltsm_textprompt_csv.sh
│ └── train_ltsm_tokenizer_csv.sh
├── setup.py
└── tutorial
└── README.md
```

---

## Datasets and Prompts

Download datasets:

```bash
cd datasets
download: https://drive.google.com/drive/folders/1hLFbz0FRxdiDCzgFYtKCOPJYSBVvwW9P
# Google Drive link:
https://drive.google.com/drive/folders/1hLFbz0FRxdiDCzgFYtKCOPJYSBVvwW9P
```

Download the time series prompts
Download time series prompts:

```bash
cd prompt_bank/propmt_data_csv
download: https://drive.google.com/drive/folders/1hLFbz0FRxdiDCzgFYtKCOPJYSBVvwW9P
cd prompt_bank/prompt_data_csv
# Same Google Drive link applies
```

---

## Model Access

You can find our trained LTSM models on Hugging Face:

➡️ [https://huggingface.co/LSC2204/LTSM-bundle](https://huggingface.co/LSC2204/LTSM-bundle)

---

## Cite This Work
If you find this work useful, you may cite this work:
```
@article{ltsm-bundle,
title={Understanding Different Design Choices in Training Large Time Series Models},
author={Chuang*, Yu-Neng and Li*, Songchen and Yuan*, Jiayi and Wang*, Guanchu and Lai*, Kwei-Herng and Yu, Leisheng and Ding, Sirui and Chang, Chia-Yuan and Tan, Qiaoyu and Zha, Daochen and Hu, Xia},
journal={arXiv preprint arXiv:2406.14045},
year={2024}

If you find this work useful, please cite:

```bibtex
@misc{chuang2025ltsmbundletoolboxbenchmarklarge,
title={LTSM-Bundle: A Toolbox and Benchmark on Large Language Models for Time Series Forecasting},
author={Yu-Neng Chuang and Songchen Li and Jiayi Yuan and Guanchu Wang and Kwei-Herng Lai and Songyuan Sui and Leisheng Yu and Sirui Ding and Chia-Yuan Chang and Qiaoyu Tan and Daochen Zha and Xia Hu},
year={2025},
eprint={2406.14045},
archivePrefix={arXiv},
primaryClass={cs.LG},
url={https://arxiv.org/abs/2406.14045},
}
```

---

## License

This project is licensed under the MIT License. See the [LICENSE](https://choosealicense.com/licenses/mit/) file for details.

---

## Acknowledgments

We thank all contributors and collaborators involved in the LTSM project. Special thanks to the Data Lab at Rice University and the open-source community for enabling fast prototyping and reproducible research.

---

<div align="right">
<a href="#top">⬆️ Back to Top</a>
</div>
Loading