Skip to content

Latest commit

 

History

History
158 lines (113 loc) · 4.27 KB

README.md

File metadata and controls

158 lines (113 loc) · 4.27 KB

AMDDPG-TORCS

Paper

Junta Wu, Huiyun Li. Aggregated Multi-deep Deterministic Policy Gradient for Self-driving Policy, International Conference on Internet of Vehicles. Springer, Cham, 2018: 179-192.

Introduction

AMDDPG is a deep reinforcement learning algorithm designed based on multi-DDPG structure. This repository tested AMDDPG on Gym-TORCS and compared it with DDPG.

Requirements

  • Ubuntu 16.04
  • Python 2.7
  • Numpy, Matplotlib, OpenCV
  • Gym
  • Keras 1.1.0
  • Tensorflow 0.12.0 (CPU version or GPU version)
  • Gym-TORCS
  • CUDA (unnecessary if cpu-version tensorflow is installed, no more than CUDA 8.0)

Environment Configuration

For convenience, environment configuration is done on Anaconda. Terminal commands are shown below.

  1. Follow installation instructions to install Anaconda.

  2. Create Python 2.7 virtual environment on Anaconda

    conda create --name python2.7 python=2.7
source activate python2.7

  3. Install Numpy, Matplotlib, OpenCV, h5py in "python2.7" environment

    pip install --upgrade setuptools
pip install -U --pre numpy Matplotlib
pip install -U --pre opencv-python
pip install requests h5py

  4. Install Gym

    pip install -U --pre gym

  5. Install Kears

    pip install -U --pre keras==1.1.0

  6. Install Tensorflow (CPU version)

    pip install -U --pre tensorflow==0.12.0

  7. Install Gym-TORCS

    Note: We'll call the directory that you cloned Gym-TORCS as $Gym-TORCS-ROOT

  • Base

    git clone https://github.com/ugo-nama-kun/gym_torcs
sudo apt-get install xautomation libglib2.0-dev  libgl1-mesa-dev libglu1-mesa-dev  freeglut3-dev  libplib-dev  libopenal-dev libalut-dev libxi-dev libxmu-dev libxrender-dev  libxrandr-dev libpng12-dev 
cd $Gym-TORCS-ROOT/vtorcs-RL-colors/
./configure
sudo make 
sudo make install
sudo make datainstall

  • Test Gym-Torcs

    Open a terminal:

    torcs

    Click on TORCS interface: "Race –> Practice –> New Race". You should see a blue interdace which prints "Initializing Driver scr_server1".

    Open another terminal:

    cd $Gym-TORCS-ROOT/
python3 snakeoil3_gym.py

    You should see a car running on the track automatically. Note that there may be “print" function error with "snakeoil3_gym.py" because of python version.

  • Show the car

    The default interface of Gym-TORCS dose not show the racing car. Please follow the following instructions to change the source code so as to show the body of the car.

    Open the file $Gym-TORCS-ROOT/vtorcs-RL-color/src/modules/graphic/ssggraph/grscreen.cpp and change the 280th line of grscreen.cpp to be

    for (i = 0; i < s->_ncars; i++) {
    grDrawCar(cars[i], curCar, dispCam->getDrawCurrent(), dispCam->getDrawDriver(), s->currentTime, dispCam);
}

    Re-make Gym-TORCS

    sudo make 
sudo make install
sudo make datainstall

  • Select track

    sudo torcs

    Click "Race –> Quick Race –> Configure Race", select track, accept and quit.

  1. Clone AMDDPG-TORCS repository 
    git clone https://github.com/juntawu/AMDDPG-TORCS
    Note: We'll call the directory that you cloned AMDDPG-TORCS as $AMDDPG-TORCS-ROOT

Testing

  1. Testing DDPG

    cd $AMDDPG-TORCS-ROOT/DDPG/ddpg_2018_03_24
python test_policy.py

    You should see the result shown in "resuslt_2018_04_03_episode3450.mp4" after two instable episodes.

  2. Testing AMDDPG

    cd $AMDDPG-TORCS-ROOT/AMDDPG/multi_ddpg_2018_05_10
python test_multi_ddpg.py

    You should see the result shown in "result_2018_05_16_episode5500.mp4" after two instable episodes.

Training

  1. Training DDPG

    cd $AMDDPG-TORCS-ROOT/DDPG/ddpg_2018_03_24
python train_ddpg.py

  2. Training AMDDPG

    cd $AMDDPG-TORCS-ROOT/AMDDPG/multi_ddpg_2018_05_10
python train_multi_ddpg.py