Scoped Buffered Persistency Model for GPUs

We provide the source code and setup for our GPU persistency model, Scoped Buffered Release Persistency (SBRP). SBRP is a scope-aware, buffered persistency model that provides high performance to GPU applications that wish to persist data on Non-Volatile Memory (NVM). SBRP modifies the GPU hardware and has been implemented using GPGPU-Sim, a GPU simulator. For more details on the simulator requirements, check the README in the simulator folder.

This repository consists of the source code of the simulator, benchmarks used for evaluation and all scripts needed to replicate the figures in the paper.

For full details refer to our paper: [Paper] [Video]

Scoped Buffered Persistency Model for GPUs
Shweta Pandey, Aditya K Kamath, and Arkaprava Basu
28th ACM International Conference on Architectural Support for Programming Languages and Operating Systems, Volume 2 (ASPLOS), 2023
DOI: https://doi.org/10.1145/3575693.3575749 

We shall first explain how to replicate our results, then highlight the important files and folders contained in this repository.

Obtaining the artifact

Git clone the artifact, as follows:

git clone \
https://github.com/csl-iisc/SBRP-ASPLOS23
cd SBRP-ASPLOS23

Running the docker container

To install docker on an Ubuntu machine

sudo apt install docker.io

To run experiments within the container, build the container as:

docker build . -t sbrp:v1

The docker container also installs matplotlib which requires one to choose their current region and timezone.

Then launch the docker container in an interactive as:

docker run -it sbrp:v1

Finally run the make commands as listed below.

[Alternatively] Setting up GPGPU-sim and running

Alternatively, one can run the experiments on the bare machine with the steps mentioned below. To install cuda-11.4, follow:

wget https://developer.download.nvidia.com/compute/cuda/11.4.1/local_installers/cuda_11.4.1_470.57.02_linux.run
sudo sh cuda_11.4.1_470.57.02_linux.run

(Find detailed instructions at: https://developer.nvidia.com/cuda-11-4-1-download-archive?target_version=20.04) If your system does not have a GPU, do not install any CUDA drivers. To run GPGPU-sim only the CUDA toolkit with the nvidia compiler (nvcc) is enough.

For GPGPU-sim dependencies:

sudo apt-get install build-essential xutils-dev bison zlib1g-dev flex libglu1-mesa-dev 

For cuda-sdk dependencies:

sudo apt-get install libxi-dev libxmu-dev libglut3-dev 

Ensure that nvcc and cuda are in the PATH variable

export CUDA_INSTALL_PATH=/usr/local/cuda
export PATH=$PATH:$CUDA_INSTALL_PATH/bin

Also add cudart.so in the LD_LIBRARY_PATH

To find the path:

sudo find / -name 'libcudart.so.11.0'

export LD_LIBRARY_PATH=$LD_LIBRARY_PATH:/usr/local/cuda-11.4/targets/x86_64-linux/lib

Running all figures and generating outputs

Once the setup for the docker container or the bare machine has completed. Proceed with the following steps to run the artifact.

make run_output_all 

Alternatively, one can first execute all the figures as:

make run_all

And then generate all outputs using:

make output_all

Running figures individually

To execute figure 6, use the following command:

make run_figure6

For figure 8, the data is generated by running figure 6.

To execute figure 9, use the following command:

make run_figure9

To execute figure 10a, use the following command:

make run_figure10_a

To execute figure 10b, use the following command:

make run_figure10_b

To execute figure 10c, use the following command:

make run_figure10_c

To execute figure 11, use the following command:

make run_figure11

Generating output CSV files and graphs individually

To generate figure 6, use the following command:

make output_figure6

To generate figure 8, use the following command:

make output_figure8

To generate figure 9, use the following command:

make output_figure9

To generate figure 10(a), use the following command:

make output_figure10_a

To generate figure 10(b), use the following command:

make output_figure10_b

To generate figure 10(c), use the following command:

make output_figure10_c

To generate figure 11, use the following command:

make output_figure11

Source code

There are four main folders in this repository:

• benchmarks: This folder contains the CUDA source code for the benchmarks evaluated.
• models: This folder contains the GPGPU-Sim configurations for all the different models evaluated.
• scripts: This folder contains the python scripts for plotting the different graphs.
• simulator: This is the main folder containing GPGPU-Sim and the source code for SBRP. The majority of the functionality for SBRP is encapsulated in src/gpgpu-sim/persist.cc, which contains the main implementation of the buffer. Other modifications were performed in the cache and memory hierarchy to support this new buffer.