Consider splitting up agent and environment in separate containers

[david-woelfle]

Problem

Currently the agent and the environment are executed in the same container. This makes installing dependencies for new research algorithms (for agents) potentially complicated, especially as some of the environments seem to be quite old and it is likely only a matter of time before dependencies of the agent and the environment collide.

Potential Solution

IMHO it would be much better to split up the agent and environment into two containers. The benefits are:

• Dependencies of agent and environment can't collide.
• It would open a clean way how users can install packages required by their agent scripts.
• The size of the environment containers would shrink by likely ~80% as those containers do not need tensorflow etc.
• It would open the possibility for users to interact with the environments interactively like from inside a jupyter notebook.
• As the agent side of beobench would rather be really simple and without much dependencies (apart from the agent itself) it might even allow users to run the agent on their local system, which might be favorable for all those how are not too firm in docker.

I would be willing to support this development and could provide a minimal example for such a setup with celery if it would help you get started.

[rdnfn]

@david-woelfle thanks a lot for this great suggestion! Sorry for the late reply here, but I think completely isolating the environment in its own docker container has the potential to really improve Beobench's usability! That's especially true because of the last point you mention - the user could run their agent wherever and however they like. This would be much better than the somewhat prescriptive/constrained current way of running agent scripts.

Would be brilliant if you would be able to set up a minimal working example, which I could then integrate into the Beobench package. We'll discuss this offline in more detail but let's keep track of any relevant updates and the status of this improvement in this issue.

[david-woelfle]

@rdnfn I have added a PR for a minimal but working example that uses Celery to split environment and agent into two containers.