The predictor project is a comprehensive tool for timeseries prediction, equipped with a robust plugin architecture. This project allows for both local and remote configuration handling, as well as replicability of experimental results. The system can be extended with custom plugins for various types of neural networks, including artificial neural networks (ANN), convolutional neural networks (CNN), long short-term memory networks (LSTM), and transformer-based models. Examples of the aforementioned models are included alongside with historical EURUSD and other training data in the examples directory.
To install and set up the predictor application, follow these steps:
-
Clone the Repository:
git clone https://github.com/harveybc/predictor.git cd predictor -
Add the clonned directory to the Windows or Linux PYTHONPATH environment variable:
In Windows a close of current command line promp may be required for the PYTHONPATH varible to be usable. Confirm you added the directory to the PYTHONPATH with the following commands:
-
On Windows, run:
echo %PYTHONPATH% -
On Linux, run:
echo $PYTHONPATH
If the clonned repo directory appears in the PYTHONPATH, continue to the next step.
-
Create and Activate a Virtual Environment (Anaconda is required):
- Using
conda:conda create --name predictor-env python=3.9 conda activate predictor-env
- Using
-
Install Dependencies:
pip install --upgrade pip pip install -r requirements.txt
-
Build the Package:
python -m build
-
Install the Package:
pip install . -
(Optional) Run the predictor:
-
On Windows, run the following command to verify installation (it uses all default valuex, use predictor.bat --help for complete command line arguments description):
predictor.bat --load_config examples\config\phase_1\phase_1_ann_6300_1h_config.json
-
On Linux, run:
sh predictor.sh --load_config examples\config\phase_1\phase_1_ann_6300_1h_config.json
-
-
(Optional) Run Tests: For pasing remote tests, requires an instance of harveybc/data-logger
-
On Windows, run the following command to run the tests:
set_env.bat pytest
-
On Linux, run:
sh ./set_env.sh pytest
-
-
(Optional) Generate Documentation:
- Run the following command to generate code documentation in HTML format in the docs directory:
pdoc --html -o docs app
- Run the following command to generate code documentation in HTML format in the docs directory:
-
(Optional) Install Nvidia CUDA GPU support:
Please read: Readme - CUDA
Example config json files are located in examples\config, for a list of individual parameters to call via CLI or in a config json file, use: predictor.bat --help
After executing the prediction pipeline, the predictor will generate 4 files:
- output_file: csv file, predictions for the selected time_horizon (see defaults in app\config.py)
- results_file: csv file, aggregated results for the configured number of iterations of the training with the selected number of training epochs
- loss_plot_file: png image, the plot of error vs epoch for training and validation in the last iteration
- model_plot_file: png image, the plot of the used Keras model
The application supports several command line arguments to control its behavior for example:
usage: predictor.bat --load_config examples\config\phase_1\phase_1_ann_6300_1h_config.json --epochs 100 --iterations 5
There are many examples of config files in the examples\config directory, also training data of EURUSD and othertimeseries in examples\data and the results of the example config files are set to be on examples\results, there are some scripts to automate running sequential predictions in examples\scripts.
predictor/
│
├── app/ # Main application package
│ ├── __init__.py # Package initialization
│ ├── cli.py # Command-line interface handling
│ ├── config.py # Default configuration values
│ ├── config_handler.py # Configuration management
│ ├── config_merger.py # Configuration merging logic
│ ├── data_handler.py # Data loading and saving functions
│ ├── data_processor.py # Core data processing pipeline
│ ├── main.py # Application entry point
│ ├── plugin_loader.py # Dynamic plugin loading system
│ ├── reconstruction.py # Data reconstruction utilities
│ └── plugins/ # Prediction plugins directory
│ ├── predictor_plugin_ann.py # Artificial Neural Network plugin
│ ├── predictor_plugin_cnn.py # Convolutional Neural Network plugin
│ ├── predictor_plugin_lstm.py # Long Short-Term Memory plugin
│ └── predictor_plugin_transformer.py # Transformer model plugin
│
├── tests/ # Test suite directory
│ ├── __init__.py # Test package initialization
│ ├── conftest.py # pytest configuration
│ ├── acceptance_tests/ # User acceptance tests
│ ├── integration_tests/ # Integration test modules
│ ├── system_tests/ # System-wide test cases
│ └── unit_tests/ # Unit test modules
│
├── examples/ # Example files directory
│ ├── config/ # Example configuration files
│ ├── data/ # Example training data
│ ├── results/ # Example output results
│ └── scripts/ # Example execution scripts
│ └── run_phase_1.bat # Phase 1 execution script
│
├── concatenate_csv.py # CSV file manipulation utility
├── setup.py # Package installation script
├── predictor.bat # Windows execution script
├── predictor.sh # Linux execution script
├── set_env.bat # Windows environment setup
├── set_env.sh # Linux environment setup
├── requirements.txt # Python dependencies
├── LICENSE.txt # Project license
└── prompt.txt # Project documentation