Robust handling of inconsistent TabularInput keys

[avnishn]

Introduction

Dowel is a tool that the garage Team uses for logging results from our various Reinforcement learning experiments.

Dowel can be used to log different types of data such as floats or strings. The logs can be logged to stdout (the console), CSV files, and Tensorboard.

You can check out an example of how Dowel is used here. In fact, almost all parts of the Dowel API are used in this example.

The problem

After statistics such as loss have been logged, and a call to logger.dump_all() is made for the first time, new tabular data can’t be written to a CSV output. This is because currently data cannot be inconsistently logged to CSV, meaning that on every single call to dump_all, the same logger keys must appear. Data that is inconsistently logged will not appear in the CSV output. This is a design flaw that we have been able to work around but affects our workflows.

Your goal is to solve the problem as well as introduce tests into our testing framework in order to verify your solution.

Some General Instructions

1. Fork Dowel and install all necessary dependencies.
2. Take a look at this toy example which when run exposes the bug and the accompanying issue mentioned above.
3. When you have finished writing your solution and tests, upload a PR onto your fork, not onto the upstream repository.
4. When you are done email us back with the link to your pull request.
5. Follow the rules of the contributing.md.

If you have any questions, open an issue in your fork, and tag @avnishn and @haydenshively. Our preferred mode of communication on any questions that you have is through github issues and pull requests, as this is how the Garage team communicates generally. For this reason, we won’t respond to any direct emails with regards to help with your project. We will however respond to any other questions that you have via email (interview scheduling, etc).

Best of luck, and let us know if there are any issues as early on as possible