skrl-v1.2.0

[1.2.0] - 2024-06-23

Added

• Define the environment_info trainer config to log environment info (PyTorch implementation)
• Add support to automatically compute the write and checkpoint intervals and make it the default option
• Single forward-pass in shared models
• Distributed multi-GPU and multi-node learning (PyTorch implementation)

Changed

• Update Orbit-related source code and docs to Isaac Lab

Fixed

• Move the batch sampling inside gradient step loop for DDPG and TD3
• Perform JAX computation on the selected device

skrl-v1.1.0

[1.1.0] - 2024-02-12

Added

• MultiCategorical mixin to operate MultiDiscrete action spaces

Changed (breaking changes)

• Rename the ManualTrainer to StepTrainer
• Output training/evaluation progress messages to system's stdout
• Get single observation/action spaces for vectorized environments
• Update Isaac Orbit environment wrapper

skrl-v1.0.0

[1.0.0] - 2023-08-16

Transition from pre-release versions (1.0.0-rc.1 and1.0.0-rc.2) to a stable version.

This release also announces the publication of the skrl paper in the Journal of Machine Learning Research (JMLR):
https://www.jmlr.org/papers/v24/23-0112.html

Summary of the most relevant features:

• JAX support
• New documentation theme and structure
• Multi-agent Reinforcement Learning (MARL)

skrl-v1.0.0-rc.2

[1.0.0-rc.2] - 2023-08-11

Added

• Get truncation from time_outs info in Isaac Gym, Isaac Orbit and Omniverse Isaac Gym environments
• Time-limit (truncation) boostrapping in on-policy actor-critic agents
• Model instantiators initial_log_std parameter to set the log standard deviation's initial value

Changed

• Structure environment loaders and wrappers file hierarchy coherently [breaking change]
• Drop support for versions prior to PyTorch 1.9 (1.8.0 and 1.8.1)

skrl-v1.0.0-rc.1

[1.0.0-rc.1] - 2023-07-25

Added

• JAX support (with Flax and Optax)
• RPO agent
• IPPO and MAPPO multi-agent
• Multi-agent base class
• Bi-DexHands environment loader
• Wrapper for PettingZoo and Bi-DexHands environments
• Parameters num_envs, headless and cli_args for configuring Isaac Gym, Isaac Orbit
  and Omniverse Isaac Gym environments when they are loaded

Changed

• Migrate to pyproject.toml Python package development
• Define ML framework dependencies as optional dependencies in the library installer
• Move agent implementations with recurrent models to a separate file
• Allow closing the environment at the end of execution instead of after training/evaluation
• Documentation theme from sphinx_rtd_theme to furo
• Update documentation structure and examples

Fixed

• Compatibility for Isaac Sim or OmniIsaacGymEnvs (2022.2.0 or earlier)
• Disable PyTorch gradient computation during the environment stepping
• Get categorical models' entropy
• Typo in KLAdaptiveLR learning rate scheduler
  (keep the old name for compatibility with the examples of previous versions.
  The old name will be removed in future releases)

skrl-v0.10.2

[0.10.2] - 2023-03-23

Changed

• Update loader and utils for OmniIsaacGymEnvs 2022.2.1.0
• Update Omniverse Isaac Gym real-world examples

skrl-v0.10.1

[0.10.1] - 2023-01-26

Fixed

• Tensorboard writer instantiation when write_interval is zero

skrl-v0.10.0

[0.10.0] - 2023-01-22

Added

• Isaac Orbit environment loader
• Wrap an Isaac Orbit environment
• Gaussian-Deterministic shared model instantiator

skrl-v0.9.1

[0.9.1] - 2023-01-17

Added

• Utility for downloading models from Hugging Face Hub

Fixed

• Initialization of agent components if they have not been defined
• Manual trainer train/eval method default arguments

skrl-v0.9.0

[0.9.0] - 2023-01-13

Added

• Support for Farama Gymnasium interface
• Wrapper for robosuite environments
• Weights & Biases integration (by @juhannc)
• Set the running mode (training or evaluation) of the agents
• Allow clipping the gradient norm for DDPG, TD3 and SAC agents
• Initialize model biases
• Add RNN (RNN, LSTM, GRU and any other variant) support for A2C, DDPG, PPO, SAC, TD3 and TRPO agents
• Allow disabling training/evaluation progressbar
• Farama Shimmy and robosuite examples
• KUKA LBR iiwa real-world example

Changed

• Forward model inputs as a Python dictionary [breaking change]
• Returns a Python dictionary with extra output values in model calls [breaking change]
• Adopt the implementation of terminated and truncated over done for all environments

Fixed

• Omniverse Isaac Gym simulation speed for the Franka Emika real-world example
• Call agents' method record_transition instead of parent method
  to allow storing samples in memories during evaluation
• Move TRPO policy optimization out of the value optimization loop
• Access to the categorical model distribution
• Call reset only once for Gym/Gymnasium vectorized environments

Removed

• Deprecated method start in trainers