Add DIAMBRA Bonus Unit

[alexpalms]

No description provided.

[simoninithomas]

Thanks for this new Unit, I would need the toctree change to be able to visualize the course version and being able to make a complete review.

[simoninithomas]

I would add an introduction.

Welcome to this new bonus unit where you'll learn to use DIambra and train agents to play

At the end of the unit you'll get

illustreation of agent playing

[HuggingFaceDocBuilderDev]

The docs for this PR live here. All of your documentation changes will be reflected on that endpoint. The docs are available until 30 days after the last update.

[simoninithomas]

Suggested change

	All environments are episodic Reinforcement Learning tasks, with discrete actions (gamepad buttons) and observations composed by screen pixels plus additional numerical data (RAM states like characters health bars or characters stage side).
	All environments are **episodic Reinforcement Learning tasks**, with **discrete actions** (gamepad buttons) and observations composed by screen pixels plus additional numerical data (RAM states like characters health bars or characters stage side).

[simoninithomas]

Suggested change

	In this unit we will focus on Street Fighter III, but other historic games are also available, and the list will continue to grow. Switching between them is very straightforward, so at the end of the unit you will be able to easily target additional titles.
	In this unit **we will focus on Street Fighter III, but other historic games are also available**, and the list will continue to grow. Switching between them is very straightforward, so at the end of the unit you will be able to easily target additional titles.

[simoninithomas]

I suppose we can't provide link to find them ?

[simoninithomas]

Suggested change

	DIAMBRA Arena Environments usage follows the standard RL interaction framework: the agent sends an action to the environment, which process it and performs a transition accordingly, from the starting state to the new state, returning the observation and the reward to the agent to close the interaction loop. The figure below shows this typical interaction scheme and data flow.
	DIAMBRA Arena Environments usage **follows the standard RL interaction framework**: the agent sends an action to the environment, which process it and performs a transition accordingly, from the starting state to the new state, returning the observation and the reward to the agent to close the interaction loop. The figure below shows this typical interaction scheme and data flow.

[simoninithomas]

Suggested change

	R_t = \sum_i^{0,N_c}\left(\bar{H_i}^{t^-} - \bar{H_i}^{t} - \left(\hat{H_i}^{t^-} - \hat{H_i}^{t}\right)\right)
	\\(R_t = \sum_i^{0,N_c}\left(\bar{H_i}^{t^-} - \bar{H_i}^{t} - \left(\hat{H_i}^{t^-} - \hat{H_i}^{t}\right)\right)\\)

[simoninithomas]

Suggested change

	- \bar{H} and \hat{H} are health values for opponent’s character(s) and agent’s one(s) respectively;
	- \\(\bar{H} and \hat{H}\\) are health values for opponent’s character(s) and agent’s one(s) respectively;

[simoninithomas]

Suggested change

	- t^- and t are used to indicate conditions at ”state-time” and at ”new state-time” (i.e. before and after environment step);
	- \\(t^-\\) and \\(t\\) are used to indicate conditions at ”state-time” and at ”new state-time” (i.e. before and after environment step);

[simoninithomas]

Suggested change

	- N_c is the number of characters taking part in a round. Usually is N_c = 1 but there are some games where multiple characters are used, with the additional possible option of alternating them during gameplay, like Tekken Tag Tournament where 2 characters have to be selected and two opponents are faced every round (thus N_c = 2);
	- \\(N_c\\) is the number of characters taking part in a round. Usually is \\(N_c = 1\\) but there are some games where multiple characters are used, with the additional possible option of alternating them during gameplay, like Tekken Tag Tournament where 2 characters have to be selected and two opponents are faced every round (thus \\(N_c = 2\\) );

[simoninithomas]

Suggested change

	Environmet wrappers are widely used to tweak the observation and action spaces. In order to activate them, one needs to properly set the `WrapperSettings` class attributes and provide them as input to the environment creation method, as shown in the next code block.
	Environment wrappers are **widely used to tweak the observation and action spaces**. In order to activate them, one needs to properly set the `WrapperSettings` class attributes and provide them as input to the environment creation method, as shown in the next code block.

[simoninithomas]

Suggested change

	For training our model, we will rely on already implemented RL algorithms, leveraging state-of-the-art Reinforcement Learning libraries. There are multiple advantages in doing so: these libraries provide high quality algorithms, efficiently implemented and continuously tested, they allow to focus efforts on higher level aspects such as policy network architecture, features selection, and hyper-parameters tuning, they provide native solutions to parallelize environment execution, and, in some cases, they even support distributed training using multiple GPUs in a single workstation or in cluster contexts.
	For training our model, we will rely on already implemented RL algorithms, leveraging state-of-the-art Reinforcement Learning libraries.
	There are multiple advantages in doing so:
	- These libraries provide **high quality algorithms**, efficiently implemented and continuously tested, they allow to focus efforts on higher level aspects such as policy network architecture, features selection, and hyper-parameters tuning.
	- They provide **native solutions to parallelize environment execution**, and, in some cases, they even support distributed training using multiple GPUs in a single workstation or in cluster contexts.

[simoninithomas]

Suggested change

	We highly recommend using virtual environments to isolate your python installs, especially to avoid conflicts in dependencies. In what follows we use Conda but any other tool should work too.
	We highly **recommend using virtual environments to isolate your python installs, especially to avoid conflicts in dependencies**. In what follows we use Conda but any other tool should work too.

[simoninithomas]

Suggested change

	It features a public global leaderboard where users are ranked by the best score achieved by their agents in the different environments. It also offers you the possibility to unlock cool achievements depending on the performances of your agent. Submitted agents are evaluated and their episodes are streamed on DIAMBRA Twitch channel.
	It features a public global leaderboard where users are ranked by the best score achieved by their agents in the different environments. It also offers you the possibility to unlock cool achievements depending on the performances of your agent. **Submitted agents are evaluated and their episodes are streamed on DIAMBRA Twitch channel**.

[simoninithomas]

Add Twitch channel link

[simoninithomas]

Add link of the twitch channel

[simoninithomas]

Hi, thanks for your work, I added some update, request changes and advice 🤗