Skip to content

Latest commit

 

History

History
 
 

NeurIPS2019-Learn-to-Move-Challenge

README.md

The PARL team gets the first place in NeurIPS reinforcement learning competition, again! This folder contains our final submitted model and the code relative to the training process.

PARL

Dependencies

Part1: Final submitted model

Test

  • How to Run

    1. Enter the sub-folder final_submit
    2. Download the model file from online storage service: Baidu Pan (password: b5ck) or Google Drive
    3. Unpack the file: tar zxvf saved_models.tar.gz
    4. Launch the test script: python test.py

Part2: Curriculum learning

1. Run as fast as possible -> run at 3.0 m/s -> walk at 2.0 m/s -> walk slowly at 1.3 m/s

The curriculum learning pipeline to get a walking slowly model is the same pipeline in our winning solution in NeurIPS 2018: AI for Prosthetics Challenge. You can get a walking slowly model by following the guide.

We also provide a pre-trained model that walk naturally at ~1.3m/s. You can download the model file (naming low_speed_model) from online storage service: Baidu Pan (password: q9vj) or Google Drive.

2. difficulty=1

We built our distributed training agent based on PARL cluster. To start a PARL cluster, we can execute the following two xparl commands:

# starts a master node to manage computation resources and adds the local CPUs to the cluster.
xparl start --port 8010 
# if necessary, adds more CPUs (computation resources) in other machine to the cluster.
xparl connect --address [CLUSTER_IP]:8010

For more information of xparl, please visit the documentation.

In this example, we can start a local cluster with 300 CPUs by running:

xparl start --port 8010 --cpu_num 300

Then, we can start the distributed training by running:

# NOTE: You need provide a self-trained model, or download the `low_speed_model` as mentioned above.
sh scripts/train_difficulty1.sh ./low_speed_model

Optionally, you can start the distributed evaluating by running:

sh scripts/eval_difficulty1.sh

3. difficulty=2

sh scripts/train_difficulty2.sh [TRAINED DIFFICULTY=1 MODEL]

4. difficulty=3, first target

sh scripts/train_difficulty3_first_target.sh [TRAINED DIFFICULTY=2 MODEL]

5. difficulty=3

sh scripts/train_difficulty3.sh [TRAINED DIFFICULTY=3 FIRST TARGET MODEL]