hands-on_machine_learning_oreilly

Exercises and examples from O'Reilly's "Hands-on Machine Learning with Scikit-Learn, Keras & TernsorFlow" by A. Geron (3rd ed.)

Description

This repository is an accompanying journal on my path of going through examples and exercises from "Hands-on Machine Learning with Scikit-Learn, Keras & TensorFlow" book from o'Reilly. It follows book's chapter division, partially for ease of navigation, but also due to the fact that different chapters may require different environment set-up.

Runbook

For Python modules

Please follow these steps to set-up and activate a local virtual environment:

1. Make sure you're at the root of the project

2. Set-up the virtual environment

   python -m venv shared_venv/

3. Activate your virtual environment (example below is for macOS/Linux)

   source shared_venv/bin/activate

4. Install required dependencies:

   pip install -r requirements.txt

5. [optional, for Visual Studio Code] If you want to run/edit the project from Visual Studio Code:

   1. Make sure you have Python extension for Visual Studio Code installed
   2. Open command pallete (Ctr + Shift + P on Linux) and select Python: Select Interpreter
   3. Pick the option containing shared_venv

To run a Python file, run a main module like this:

python3 -m chapter_03.ex03.main

To run tests for a certain chapter:

# change chapter folder accordingly
cd chapter_02/
python -m pytest

Docker setup

For Jupyter Notebook examples I've used initial containerized environment with a small fix. Simply navigate to docker folder and just run: docker compose up.

This setup is based on Dockerfile.gpu.

Build image

This should be ideally done only once:

cd docker/
# build docker compose image from Dockerfile.gpu
# Note that this may take a while (12 minutes on my machine)
docker compose build

Run

# click on one of the produced links to start adding/editing Jupyter Notebook entries
docker compose up
...
handson-ml3  |     To access the server, open this file in a browser:
handson-ml3  |         file:///home/devel/.local/share/jupyter/runtime/jpserver-1-open.html
handson-ml3  |     Or copy and paste one of these URLs:
handson-ml3  |         http://62903a64ecf6:8888/lab?token=87cbe0058148e33d6702d94d243525acb18f7577ff00ca5d
handson-ml3  |         http://127.0.0.1:8888/lab?token=87cbe0058148e33d6702d94d243525acb18f7577ff00ca5d
handson-ml3  | [I 2024-06-24 23:49:32.937 ServerApp] Skipped non-installed server(s): bash-language-server, dockerfile-language-server-nodejs, javascript-typescript-langserver, jedi-language-server, julia-language-server, pyright, python-language-server, python-lsp-server, r-languageserver, sql-language-server, texlab, typescript-language-server, unified-language-server, vscode-css-languageserver-bin, vscode-html-languageserver-bin, vscode-json-languageserver-bin, yaml-language-server

⚠️ Should the above docker container be automatically restarted by the Docker engine on your laptop, just look for the last logs for the running container, e.g.:

docker logs --tail 15 handson-ml3
    To access the server, open this file in a browser:
        file:///home/devel/.local/share/jupyter/runtime/jpserver-1-open.html
    Or copy and paste one of these URLs:
        http://62903a64ecf6:8888/lab?token=b99d270fee1d46985f3f58abb29dabf913672ff971da673c
        http://127.0.0.1:8888/lab?token=b99d270fee1d46985f3f58abb29dabf913672ff971da673c
[I 2024-08-19 20:10:40.091 ServerApp] Skipped non-installed server(s): bash-language-server, dockerfile-language-server-nodejs, javascript-typescript-langserver, jedi-language-server, julia-language-server, pyright, python-language-server, python-lsp-server, r-languageserver, sql-language-server, texlab, typescript-language-server, unified-language-server, vscode-css-languageserver-bin, vscode-html-languageserver-bin, vscode-json-languageserver-bin, yaml-language-server
[I 2024-08-20 22:19:51.983 ServerApp] 302 GET / (@172.18.0.1) 0.41ms
[I 2024-08-20 22:19:51.985 LabApp] 302 GET /lab? (@172.18.0.1) 0.46ms
[I 2024-08-20 22:20:23.947 LabApp] 302 GET /lab?token=[secret] (@172.18.0.1) 0.52ms
[W 2024-08-20 22:22:28.435 LabApp] Could not determine jupyterlab build status without nodejs
[I 2024-08-20 22:22:29.358 ServerApp] Kernel started: b6936699-0de1-4051-a6a1-337ea2696f9c
[I 2024-08-20 22:22:30.748 ServerApp] Connecting to kernel b6936699-0de1-4051-a6a1-337ea2696f9c.
[I 2024-08-20 22:22:30.762 ServerApp] Connecting to kernel b6936699-0de1-4051-a6a1-337ea2696f9c.
[I 2024-08-20 22:22:30.776 ServerApp] Connecting to kernel b6936699-0de1-4051-a6a1-337ea2696f9c.

Adding dependencies

Should a new dependency be required, you may search for it on PyPi. Each page shows packages, their transsient dependencies and compatibility. Alternatively you may use pip (and just pick the latest version):

(shared_venv) vasilegorcinschi@bonobo15:~/repos/hands-on_machine_learning_oreilly$ pip index versions seaborn
WARNING: pip index is currently an experimental command. It may be removed/changed in a future release without prior warning.
seaborn (0.13.2)
Available versions: 0.13.2, 0.13.1, 0.13.0, 0.12.2, 0.12.1, 0.12.0, 0.11.2, 0.11.1, 0.11.0, 0.10.1, 0.10.0, 0.9.1, 0.9.0, 0.8.1, 0.8, 0.7.1, 0.7.0, 0.6.0, 0.5.1, 0.5.0, 0.4.0, 0.3.1, 0.3, 0.2.1, 0.2.0, 0.1

Then you may add the required dependency to requirements.txt , e.g.: seaborn~=0.13.0

For use within Jupyter Notebooks

Jupyter notebook dependencies are actually read from environment.yml file. To add the same dependency to that file, just make sure it is on conda-forge, for pip dependencies add them under pip:.

For non-python dependencies, for example I once wanted to install Pandoc, you have to install it as a tool inside the Docker image and rebuild the latter.