Controllers

Controllers

All controllers should inherit from the base Controller class (which does nothing when receiving the various signals emitted by an agent). The following methods are defined in this base controller class:

• __init__(self): Activate the controller. All controllers inheriting this class should call this method in their own __init()__ using super(self.__class__, self).__init__().

• setActive(self, active): Activate or deactivate this controller. A controller should not react to any signal it receives as long as it is deactivated. For instance, if a controller maintains a counter on how many episodes it has seen, this counter should not be updated when this controller is disabled.

• OnStart(self, agent): Called when the agent is going to start working (before anything else). This corresponds to the moment where the agent's run() method is called.

• OnEpisodeEnd(self, agent, terminalReached, reward): Called whenever the agent ends an episode, just after this episode ended and before any OnEpochEnd() signal could be sent.

• OnEpochEnd(self, agent): Called whenever the agent ends an epoch, just after the last episode of this epoch was ended and after any OnEpisodeEnd() signal was processed.

• OnActionChosen(self, agent, action): Called whenever the agent has chosen an action. This occurs after the agent state was updated with the new observation it made, but before it applied this action on the environment and before the total reward is updated.

• OnActionTaken(self, agent): Called whenever the agent has taken an action on its environment. This occurs after the agent applied this action on the environment and before terminality is evaluated. This is called only once, even in the case where the agent skip frames by taking the same action multiple times. In other words, this occurs just before the next observation of the environment.

• OnEnd(self, agent): Called when the agent has finished processing all its epochs, just before returning from its run() method.

The order in which controllers are attached matters. Indeed, if controllers C1, C2 and C3 were attached in this order and C1 and C3 both listen to the OnEpisodeEnd signal, the OnEpisodeEnd() method of C1 will be called before the OnEpisodeEnd() method of C3, whenever an episode ends.

Examples of controllers can be found in the experiment.base_controllers module. This module gathers all controllers used for experiments carried on the demo environments.

Attaching controllers to carry an experiment

Examples of how controllers can be attached to an agent in order to carry an experiment can be found in the launchers of the demo environments, run_MG_two_storages.py and run_toy_env.py. We encourage you to go over them.