Code

A2C and Synchronous extension#17

Code

Open

juice1000 wants to merge 66 commits intoMorvanZhou:masterMorvanZhou/pytorch-A3C:masterfrom juice1000:masterjuice1000/synchronous-vs-asynchronous-reinforcement-learning:masterCopy head branch name to clipboard

+4,110-351

Commits

Commits on Mar 23, 2020

• CARTPOLE - create vanilla a2c and synchronous a2c

  juice1000

  authored
  0ce32a4

Commits on Mar 24, 2020

• CARTPOLE - test

  juice1000

  committed
  4918b37

• CARTPOLE - renew model and parameters, add args and plotter function, terminate after score of 600, save and load model

  juice1000

  committed
  63b4092

Commits on Mar 25, 2020

• CARTPOLE - for all: terminate when score higher than 500 within range

  juice1000

  committed
  a459f81

• CARTPOLE - refactor a2c to without torch.mp module

  juice1000

  committed
  2045869

Commits on Mar 27, 2020

• CARTPOLE - stabilize algorithms: true synchronous learning for a2c_sync, set upper limit for ep_length, gym make unwrapped for a2c

  juice1000

  committed
  490767a

Commits on Mar 28, 2020

• CARTPOLE - training duration in plotter, args and plotter in utils, save model if consecutive reward over 300

  juice1000

  committed
  b7a4bf6

Commits on Mar 29, 2020

• CARTPOLE - if load model, then check for 100 iterations of ep_r over 500

  juice1000

  committed
  f9ace27

• CARTPOLE - if load_model, then check for last 100 iterations of ep_r over 500

  juice1000

  committed
  5d78d64

• Merge branch 'master' of https://github.com/juice1000/pytorch-A3C

  juice1000

  committed
  ae02294

• CARTPOLE - initialize only one process when load_model

  juice1000

  committed
  93e6726

• CARTPOLE - utils gets optimizer, a2c_sync NN update after done

  juice1000

  committed
  2317cea

Commits on Apr 8, 2020

• CARTPOLE - add episode_time plotter and confidence intervall calculator for actions in utils, add information to plotter and merge, add functions in A2Cs and A3C, raise lr=0.005

  juice1000

  committed
  add50a1

• CARTPOLE - create A2Cs and A3C shared NN

  juice1000

  committed
  b9ec460

• CARTPOLE - fix confidence interval error in Sync_A2C and A3C, action_queue in record function

  juice1000

  committed
  e35afd8

• CARTPOLE - refactor shared network algorithms, delete one confidence intervall calculation

  juice1000

  committed
  6d32122

• CARTPOLE - refactor shared NNs and lr, take out time sleep in test phase for Sync-A2C

  juice1000

  committed
  211ce33

• CARTPOLE - results and saved models

  juice1000

  committed
  aca0f93

• CARTPOLE - plots of results

  juice1000

  committed
  810b960

Commits on Apr 9, 2020

• VIZDOOM - create A2C-vizdoom for deadly corridor, create viz_utils

  juice1000

  committed
  d943e7e

• CARTPOLE - create directory

  juice1000

  committed
  671f057

• VIZDOOM - create directory

  juice1000

  committed
  019f612

• cleanup

  juice1000

  committed
  7104cf7

Commits on Apr 10, 2020

• VIZDOOM - create A3C model

  juice1000

  committed
  b6e57e6

• VIZDOOM - create A3C part2

  juice1000

  committed
  d3a4033

• VIZDOOM - create synchronous a2c

  juice1000

  committed
  06a759f

• VIZDOOM - add plotter and game replay in a2c

  juice1000

  committed
  608e20a

• VIZDOOM - plotter and stopwatch for a2c_sync and a3c

  juice1000

  committed
  52ac70c

Commits on Apr 11, 2020

• CARTPOLE - refactor synchronous coordinator in A2C_Sync

  juice1000

  committed
  e139bbe

• VIZDOOM - reinitialize state within each episode, adjust plotter boundaries

  juice1000

  committed
  a5179b2

Commits on Apr 12, 2020

• CARTPOLE - remove normalizing reward

  juice1000

  committed
  e403c84

• VIZDOOM - UPDATE_GLOBAL_ITER = 5 for A3C, correct error in push and pull

  juice1000

  committed
  2026ba8

Commits on Apr 13, 2020

• CARTPOLE - normalized and raw duration and reward plotting

  juice1000

  committed
  42e27f5

• VIZDOOM - normalized and raw reward and duration, update plotter boundaries

  juice1000

  committed
  b0ae642

• CART/DOOM - turn off episode breaker for not normalized runs

  juice1000

  committed
  51978ff

• CARTPOLE - correct coding mistakes in A3C and A2C-sync

  juice1000

  committed
  261cd3c

Commits on Apr 14, 2020

• VIZDOOM - correct A2C and A3C, adjust plotter boundaries

  juice1000

  committed
  53beb17

• CARTPOLE - create save_model.py, enable save data mode for all

  juice1000

  committed
  5dc7e25

• CARTPOLE - correct coding mistakes with save_data

  juice1000

  committed
  8065952

• CARTPOLE - separate training and test data

  juice1000

  committed
  aed3bb4

Commits on Apr 16, 2020

• CARTPOLE - remove time stamp in plots

  juice1000

  committed
  05b2b14

• CARTPOLE - add new plots, add plot data, add newly trained models

  juice1000

  committed
  6fa38cb

Commits on Apr 17, 2020

• VIZDOOM - save_data for A3C, create algo_compare for doom

  juice1000

  committed
  534a930

Commits on Apr 18, 2020

• VIZDOOM - new synchronous implementation, reinitialize threads and save stats everytime, create shared NN algorithms

  juice1000

  committed
  f98f2e1

• VIZDOOM - refactor viz_utils

  juice1000

  committed
  439e964

• VIZDOOM - refactor plotter

  juice1000

  committed
  8f2e140

Commits on Apr 19, 2020

• VIZDOOM - redefine save model condition

  juice1000

  committed
  d71c28a

Commits on Apr 20, 2020

• VIZDOOM - refactor action_queue, take action_queue out from viz_utils

  juice1000

  committed
  89b3e6d

Commits on Apr 24, 2020

• VIZDOOM - set ylim to 1000 in viz_utils

  juice1000

  committed
  bd7b79e

Commits on Apr 25, 2020

• VIZDOOM - combine CSVs and plot for A2C-sync

  juice1000

  committed
  19b5fb4

• VIZDOOM - algo_compare with subplots of training and testing data

  juice1000

  committed
  fbe2680

• CARTPOLE - make subplots for training and testing data

  juice1000

  committed
  3c8501a

Commits on Apr 26, 2020

• CART/VIZ - probabilities plot

  juice1000

  committed
  77975c3

• CART/VIZ - code corrections

  juice1000

  committed
  220c2ed

Commits on May 1, 2020

• VIZDOOM - fix probabilities plot and ep-rew-time plot, fix action_queue in A3C

  juice1000

  committed
  5bc01fc

Commits on May 4, 2020

• VIZDOOM - code grooming and refactor algo_compare

  juice1000

  committed
  983bc7c

Commits on May 31, 2020

• Update README.md

  juice1000

  authored
  5a1fb36

• VIZDOOM - push plots and plot data

  

  Julien Look

  committed
  0adf081

• Merge branch 'master' of https://github.com/juice1000/Synchronous-vs-Asynchronous-Learning-Pytorch-

  

  Julien Look

  committed
  2ffe7d0

• Update README.md

  juice1000

  authored
  3a2b84c

• Update README.md

  juice1000

  authored
  c106ea0

• CARTPOLE - move results to higher directory

  

  Julien Look

  committed
  14e4c57

• Merge branch 'master' of https://github.com/juice1000/Synchronous-vs-Asynchronous-Learning-Pytorch-

  

  Julien Look

  committed
  8aef27b

• Update README.md

  juice1000

  authored
  8f16737

Commits on Dec 17, 2022

• Update README.md

  juice1000

  authored
  72d755d

• Update README.md

  juice1000

  authored
  e606ecf