Containerized end-to-end workflow

[artste]

Summary

This Pull Request introduces a containerized environment to seamlessly run the lm-hackers notebook from start to finish. It includes two shell scripts: build.sh for building the lmhackers Docker or Podman image, and run.sh for executing the container while mounting the repository as an external volume.

Prerequisites

• Docker or Podman with GPU support
• Environment variables: OPENAI_API_KEY and HUGGINGFACE_TOKEN

Features

• Containerization: Provides a contained environment for running the lm-hackers notebook.
• Ease of Use: Simplifies the setup with build.sh and run.sh scripts.
• External Volume: Mounts the repository as an external volume, allowing for real-time code changes.
• Models and Datasets cache: Optional local storage of HUggingface .cache adding to run.sh the option localstorage.

Documentation

For comprehensive instructions on how to utilize this workflow, please refer to the Docker/README.docker.md file.

[mattflo]

@artste, is there a built image published anywhere from your all_changes branch? Do you use this on local hardware? I'm trying to get this set up on a cloud GPU, any suggestions?