bacteria-prediction.ipynb

{
 "cells": [
  {
   "cell_type": "markdown",
   "id": "8687364f",
   "metadata": {
    "_cell_guid": "b1076dfc-b9ad-4769-8c92-a6c4dae69d19",
    "_uuid": "8f2839f25d086af736a60e9eeb907d3b93b6e0e5",
    "execution": {
     "iopub.execute_input": "2022-03-02T14:33:46.222715Z",
     "iopub.status.busy": "2022-03-02T14:33:46.221872Z",
     "iopub.status.idle": "2022-03-02T14:33:46.251698Z",
     "shell.execute_reply": "2022-03-02T14:33:46.250819Z",
     "shell.execute_reply.started": "2022-03-02T14:33:46.222522Z"
    },
    "papermill": {
     "duration": 0.086877,
     "end_time": "2022-03-16T15:10:42.381721",
     "exception": false,
     "start_time": "2022-03-16T15:10:42.294844",
     "status": "completed"
    },
    "tags": []
   },
   "source": [
    "# **Bacteria Prediction**"
   ]
  },
  {
   "cell_type": "markdown",
   "id": "61eabdb8",
   "metadata": {
    "papermill": {
     "duration": 0.085692,
     "end_time": "2022-03-16T15:10:42.550754",
     "exception": false,
     "start_time": "2022-03-16T15:10:42.465062",
     "status": "completed"
    },
    "tags": []
   },
   "source": [
    "The task is to classify 10 different bacteria species using data from a genomic analysis technique that has some data compression and data loss. In this technique, 10-mer snippets of DNA are sampled and analyzed to give the histogram of base count. In other words, the DNA segment ATATGGCCTT becomes A2T4G2C2. We want to accurately predict bacteria species starting from this lossy information."
   ]
  },
  {
   "cell_type": "markdown",
   "id": "2eead04d",
   "metadata": {
    "papermill": {
     "duration": 0.087046,
     "end_time": "2022-03-16T15:10:42.725153",
     "exception": false,
     "start_time": "2022-03-16T15:10:42.638107",
     "status": "completed"
    },
    "tags": []
   },
   "source": [
    "## **Data Loading**"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 1,
   "id": "2090e43a",
   "metadata": {
    "execution": {
     "iopub.execute_input": "2022-03-16T15:10:42.893207Z",
     "iopub.status.busy": "2022-03-16T15:10:42.892019Z",
     "iopub.status.idle": "2022-03-16T15:10:42.900622Z",
     "shell.execute_reply": "2022-03-16T15:10:42.901097Z",
     "shell.execute_reply.started": "2022-03-16T15:02:12.751506Z"
    },
    "papermill": {
     "duration": 0.094946,
     "end_time": "2022-03-16T15:10:42.901456",
     "exception": false,
     "start_time": "2022-03-16T15:10:42.806510",
     "status": "completed"
    },
    "tags": []
   },
   "outputs": [],
   "source": [
    "import os\n",
    "import numpy as np\n",
    "import pandas as pd\n",
    "import matplotlib.pyplot as plt\n",
    "\n",
    "# Global variables\n",
    "random_seed = 42"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 2,
   "id": "dc442cff",
   "metadata": {
    "execution": {
     "iopub.execute_input": "2022-03-16T15:10:43.070687Z",
     "iopub.status.busy": "2022-03-16T15:10:43.069775Z",
     "iopub.status.idle": "2022-03-16T15:10:43.075145Z",
     "shell.execute_reply": "2022-03-16T15:10:43.075634Z",
     "shell.execute_reply.started": "2022-03-16T15:02:15.874484Z"
    },
    "papermill": {
     "duration": 0.09362,
     "end_time": "2022-03-16T15:10:43.075827",
     "exception": false,
     "start_time": "2022-03-16T15:10:42.982207",
     "status": "completed"
    },
    "tags": []
   },
   "outputs": [
    {
     "name": "stdout",
     "output_type": "stream",
     "text": [
      "/kaggle/input/tabular-playground-series-feb-2022/sample_submission.csv\n",
      "/kaggle/input/tabular-playground-series-feb-2022/train.csv\n",
      "/kaggle/input/tabular-playground-series-feb-2022/test.csv\n"
     ]
    }
   ],
   "source": [
    "# Print directory structure\n",
    "for dirname, _, filenames in os.walk('/kaggle/input'):\n",
    "    for filename in filenames:\n",
    "        print(os.path.join(dirname, filename))"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 3,
   "id": "c43b0f13",
   "metadata": {
    "execution": {
     "iopub.execute_input": "2022-03-16T15:10:43.245458Z",
     "iopub.status.busy": "2022-03-16T15:10:43.244882Z",
     "iopub.status.idle": "2022-03-16T15:11:11.883693Z",
     "shell.execute_reply": "2022-03-16T15:11:11.883110Z",
     "shell.execute_reply.started": "2022-03-16T15:02:18.095651Z"
    },
    "papermill": {
     "duration": 28.724367,
     "end_time": "2022-03-16T15:11:11.883866",
     "exception": false,
     "start_time": "2022-03-16T15:10:43.159499",
     "status": "completed"
    },
    "tags": []
   },
   "outputs": [],
   "source": [
    "# Load training set\n",
    "df = pd.read_csv(\"/kaggle/input/tabular-playground-series-feb-2022/train.csv\")"
   ]
  },
  {
   "cell_type": "markdown",
   "id": "86612cc7",
   "metadata": {
    "papermill": {
     "duration": 0.082227,
     "end_time": "2022-03-16T15:11:12.047133",
     "exception": false,
     "start_time": "2022-03-16T15:11:11.964906",
     "status": "completed"
    },
    "tags": []
   },
   "source": [
    "## **Data Analysis**"
   ]
  },
  {
   "cell_type": "markdown",
   "id": "be7277cb",
   "metadata": {
    "papermill": {
     "duration": 0.081123,
     "end_time": "2022-03-16T15:11:12.211359",
     "exception": false,
     "start_time": "2022-03-16T15:11:12.130236",
     "status": "completed"
    },
    "tags": []
   },
   "source": [
    "We will first analyze the dataset, in order to investigate its shape and the presence of null values.\n",
    "\n",
    "Each row of data contains a spectrum of histograms generated by repeated measurements of a sample, each row containing the output of all 286 histogram possibilities (e.g., A0T0G0C10 to A10T0G0C0) which then has a bias spectrum (of totally random ATGC) subtracted from the results. The data (both train and test) also contains simulated measurement errors (of varying rates) for many of the samples, which makes the problem more challenging."
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 4,
   "id": "29f8c394",
   "metadata": {
    "execution": {
     "iopub.execute_input": "2022-03-16T15:11:12.380337Z",
     "iopub.status.busy": "2022-03-16T15:11:12.379673Z",
     "iopub.status.idle": "2022-03-16T15:11:12.386124Z",
     "shell.execute_reply": "2022-03-16T15:11:12.386605Z",
     "shell.execute_reply.started": "2022-03-16T15:02:50.499086Z"
    },
    "papermill": {
     "duration": 0.094351,
     "end_time": "2022-03-16T15:11:12.386783",
     "exception": false,
     "start_time": "2022-03-16T15:11:12.292432",
     "status": "completed"
    },
    "tags": []
   },
   "outputs": [
    {
     "data": {
      "text/plain": [
       "(200000, 288)"
      ]
     },
     "execution_count": 4,
     "metadata": {},
     "output_type": "execute_result"
    }
   ],
   "source": [
    "df.shape"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 5,
   "id": "b42b6cfa",
   "metadata": {
    "execution": {
     "iopub.execute_input": "2022-03-16T15:11:12.552892Z",
     "iopub.status.busy": "2022-03-16T15:11:12.552205Z",
     "iopub.status.idle": "2022-03-16T15:11:12.583283Z",
     "shell.execute_reply": "2022-03-16T15:11:12.583922Z",
     "shell.execute_reply.started": "2022-03-16T15:02:50.508628Z"
    },
    "papermill": {
     "duration": 0.115861,
     "end_time": "2022-03-16T15:11:12.584087",
     "exception": false,
     "start_time": "2022-03-16T15:11:12.468226",
     "status": "completed"
    },
    "tags": []
   },
   "outputs": [
    {
     "data": {
      "text/html": [
       "<div>\n",
       "<style scoped>\n",
       "    .dataframe tbody tr th:only-of-type {\n",
       "        vertical-align: middle;\n",
       "    }\n",
       "\n",
       "    .dataframe tbody tr th {\n",
       "        vertical-align: top;\n",
       "    }\n",
       "\n",
       "    .dataframe thead th {\n",
       "        text-align: right;\n",
       "    }\n",
       "</style>\n",
       "<table border=\"1\" class=\"dataframe\">\n",
       "  <thead>\n",
       "    <tr style=\"text-align: right;\">\n",
       "      <th></th>\n",
       "      <th>row_id</th>\n",
       "      <th>A0T0G0C10</th>\n",
       "      <th>A0T0G1C9</th>\n",
       "      <th>A0T0G2C8</th>\n",
       "      <th>A0T0G3C7</th>\n",
       "      <th>A0T0G4C6</th>\n",
       "      <th>A0T0G5C5</th>\n",
       "      <th>A0T0G6C4</th>\n",
       "      <th>A0T0G7C3</th>\n",
       "      <th>A0T0G8C2</th>\n",
       "      <th>...</th>\n",
       "      <th>A8T0G1C1</th>\n",
       "      <th>A8T0G2C0</th>\n",
       "      <th>A8T1G0C1</th>\n",
       "      <th>A8T1G1C0</th>\n",
       "      <th>A8T2G0C0</th>\n",
       "      <th>A9T0G0C1</th>\n",
       "      <th>A9T0G1C0</th>\n",
       "      <th>A9T1G0C0</th>\n",
       "      <th>A10T0G0C0</th>\n",
       "      <th>target</th>\n",
       "    </tr>\n",
       "  </thead>\n",
       "  <tbody>\n",
       "    <tr>\n",
       "      <th>0</th>\n",
       "      <td>0</td>\n",
       "      <td>-9.536743e-07</td>\n",
       "      <td>-0.000010</td>\n",
       "      <td>-0.000043</td>\n",
       "      <td>-0.000114</td>\n",
       "      <td>-0.000200</td>\n",
       "      <td>-0.000240</td>\n",
       "      <td>-0.000200</td>\n",
       "      <td>-0.000114</td>\n",
       "      <td>-0.000043</td>\n",
       "      <td>...</td>\n",
       "      <td>-0.000086</td>\n",
       "      <td>-0.000043</td>\n",
       "      <td>-0.000086</td>\n",
       "      <td>-0.000086</td>\n",
       "      <td>-0.000043</td>\n",
       "      <td>-0.000010</td>\n",
       "      <td>-0.000010</td>\n",
       "      <td>-0.000010</td>\n",
       "      <td>-9.536743e-07</td>\n",
       "      <td>Streptococcus_pyogenes</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>1</th>\n",
       "      <td>1</td>\n",
       "      <td>-9.536743e-07</td>\n",
       "      <td>-0.000010</td>\n",
       "      <td>-0.000043</td>\n",
       "      <td>0.000886</td>\n",
       "      <td>-0.000200</td>\n",
       "      <td>0.000760</td>\n",
       "      <td>-0.000200</td>\n",
       "      <td>-0.000114</td>\n",
       "      <td>-0.000043</td>\n",
       "      <td>...</td>\n",
       "      <td>-0.000086</td>\n",
       "      <td>-0.000043</td>\n",
       "      <td>0.000914</td>\n",
       "      <td>0.000914</td>\n",
       "      <td>-0.000043</td>\n",
       "      <td>-0.000010</td>\n",
       "      <td>-0.000010</td>\n",
       "      <td>-0.000010</td>\n",
       "      <td>-9.536743e-07</td>\n",
       "      <td>Salmonella_enterica</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>2</th>\n",
       "      <td>2</td>\n",
       "      <td>-9.536743e-07</td>\n",
       "      <td>-0.000002</td>\n",
       "      <td>0.000007</td>\n",
       "      <td>0.000129</td>\n",
       "      <td>0.000268</td>\n",
       "      <td>0.000270</td>\n",
       "      <td>0.000243</td>\n",
       "      <td>0.000125</td>\n",
       "      <td>0.000001</td>\n",
       "      <td>...</td>\n",
       "      <td>0.000084</td>\n",
       "      <td>0.000048</td>\n",
       "      <td>0.000081</td>\n",
       "      <td>0.000106</td>\n",
       "      <td>0.000072</td>\n",
       "      <td>0.000010</td>\n",
       "      <td>0.000008</td>\n",
       "      <td>0.000019</td>\n",
       "      <td>1.046326e-06</td>\n",
       "      <td>Salmonella_enterica</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>3</th>\n",
       "      <td>3</td>\n",
       "      <td>4.632568e-08</td>\n",
       "      <td>-0.000006</td>\n",
       "      <td>0.000012</td>\n",
       "      <td>0.000245</td>\n",
       "      <td>0.000492</td>\n",
       "      <td>0.000522</td>\n",
       "      <td>0.000396</td>\n",
       "      <td>0.000197</td>\n",
       "      <td>-0.000003</td>\n",
       "      <td>...</td>\n",
       "      <td>0.000151</td>\n",
       "      <td>0.000100</td>\n",
       "      <td>0.000180</td>\n",
       "      <td>0.000202</td>\n",
       "      <td>0.000153</td>\n",
       "      <td>0.000021</td>\n",
       "      <td>0.000015</td>\n",
       "      <td>0.000046</td>\n",
       "      <td>-9.536743e-07</td>\n",
       "      <td>Salmonella_enterica</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>4</th>\n",
       "      <td>4</td>\n",
       "      <td>-9.536743e-07</td>\n",
       "      <td>-0.000010</td>\n",
       "      <td>-0.000043</td>\n",
       "      <td>-0.000114</td>\n",
       "      <td>-0.000200</td>\n",
       "      <td>-0.000240</td>\n",
       "      <td>-0.000200</td>\n",
       "      <td>-0.000114</td>\n",
       "      <td>-0.000043</td>\n",
       "      <td>...</td>\n",
       "      <td>-0.000086</td>\n",
       "      <td>-0.000043</td>\n",
       "      <td>-0.000086</td>\n",
       "      <td>-0.000086</td>\n",
       "      <td>-0.000043</td>\n",
       "      <td>-0.000010</td>\n",
       "      <td>-0.000010</td>\n",
       "      <td>-0.000010</td>\n",
       "      <td>-9.536743e-07</td>\n",
       "      <td>Enterococcus_hirae</td>\n",
       "    </tr>\n",
       "  </tbody>\n",
       "</table>\n",
       "<p>5 rows × 288 columns</p>\n",
       "</div>"
      ],
      "text/plain": [
       "   row_id     A0T0G0C10  A0T0G1C9  A0T0G2C8  A0T0G3C7  A0T0G4C6  A0T0G5C5  \\\n",
       "0       0 -9.536743e-07 -0.000010 -0.000043 -0.000114 -0.000200 -0.000240   \n",
       "1       1 -9.536743e-07 -0.000010 -0.000043  0.000886 -0.000200  0.000760   \n",
       "2       2 -9.536743e-07 -0.000002  0.000007  0.000129  0.000268  0.000270   \n",
       "3       3  4.632568e-08 -0.000006  0.000012  0.000245  0.000492  0.000522   \n",
       "4       4 -9.536743e-07 -0.000010 -0.000043 -0.000114 -0.000200 -0.000240   \n",
       "\n",
       "   A0T0G6C4  A0T0G7C3  A0T0G8C2  ...  A8T0G1C1  A8T0G2C0  A8T1G0C1  A8T1G1C0  \\\n",
       "0 -0.000200 -0.000114 -0.000043  ... -0.000086 -0.000043 -0.000086 -0.000086   \n",
       "1 -0.000200 -0.000114 -0.000043  ... -0.000086 -0.000043  0.000914  0.000914   \n",
       "2  0.000243  0.000125  0.000001  ...  0.000084  0.000048  0.000081  0.000106   \n",
       "3  0.000396  0.000197 -0.000003  ...  0.000151  0.000100  0.000180  0.000202   \n",
       "4 -0.000200 -0.000114 -0.000043  ... -0.000086 -0.000043 -0.000086 -0.000086   \n",
       "\n",
       "   A8T2G0C0  A9T0G0C1  A9T0G1C0  A9T1G0C0     A10T0G0C0  \\\n",
       "0 -0.000043 -0.000010 -0.000010 -0.000010 -9.536743e-07   \n",
       "1 -0.000043 -0.000010 -0.000010 -0.000010 -9.536743e-07   \n",
       "2  0.000072  0.000010  0.000008  0.000019  1.046326e-06   \n",
       "3  0.000153  0.000021  0.000015  0.000046 -9.536743e-07   \n",
       "4 -0.000043 -0.000010 -0.000010 -0.000010 -9.536743e-07   \n",
       "\n",
       "                   target  \n",
       "0  Streptococcus_pyogenes  \n",
       "1     Salmonella_enterica  \n",
       "2     Salmonella_enterica  \n",
       "3     Salmonella_enterica  \n",
       "4      Enterococcus_hirae  \n",
       "\n",
       "[5 rows x 288 columns]"
      ]
     },
     "execution_count": 5,
     "metadata": {},
     "output_type": "execute_result"
    }
   ],
   "source": [
    "df.head()"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 6,
   "id": "e06e3ec1",
   "metadata": {
    "execution": {
     "iopub.execute_input": "2022-03-16T15:11:12.756609Z",
     "iopub.status.busy": "2022-03-16T15:11:12.755967Z",
     "iopub.status.idle": "2022-03-16T15:11:12.758478Z",
     "shell.execute_reply": "2022-03-16T15:11:12.758956Z",
     "shell.execute_reply.started": "2022-03-16T15:02:50.556741Z"
    },
    "papermill": {
     "duration": 0.092782,
     "end_time": "2022-03-16T15:11:12.759125",
     "exception": false,
     "start_time": "2022-03-16T15:11:12.666343",
     "status": "completed"
    },
    "tags": []
   },
   "outputs": [
    {
     "data": {
      "text/plain": [
       "row_id         int64\n",
       "A0T0G0C10    float64\n",
       "A0T0G1C9     float64\n",
       "A0T0G2C8     float64\n",
       "A0T0G3C7     float64\n",
       "              ...   \n",
       "A9T0G0C1     float64\n",
       "A9T0G1C0     float64\n",
       "A9T1G0C0     float64\n",
       "A10T0G0C0    float64\n",
       "target        object\n",
       "Length: 288, dtype: object"
      ]
     },
     "execution_count": 6,
     "metadata": {},
     "output_type": "execute_result"
    }
   ],
   "source": [
    "df.dtypes"
   ]
  },
  {
   "cell_type": "markdown",
   "id": "794d0620",
   "metadata": {
    "papermill": {
     "duration": 0.084137,
     "end_time": "2022-03-16T15:11:12.927484",
     "exception": false,
     "start_time": "2022-03-16T15:11:12.843347",
     "status": "completed"
    },
    "tags": []
   },
   "source": [
    "We are working with very high dimensional data, which can make the learning process more difficult.\n",
    "It will be necessary to mitigate this problem by using a dimensionality reduction technique."
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 7,
   "id": "af293dcb",
   "metadata": {
    "execution": {
     "iopub.execute_input": "2022-03-16T15:11:13.097681Z",
     "iopub.status.busy": "2022-03-16T15:11:13.097061Z",
     "iopub.status.idle": "2022-03-16T15:11:13.270810Z",
     "shell.execute_reply": "2022-03-16T15:11:13.270296Z",
     "shell.execute_reply.started": "2022-03-16T15:02:50.564934Z"
    },
    "papermill": {
     "duration": 0.260066,
     "end_time": "2022-03-16T15:11:13.270953",
     "exception": false,
     "start_time": "2022-03-16T15:11:13.010887",
     "status": "completed"
    },
    "tags": []
   },
   "outputs": [
    {
     "data": {
      "text/plain": [
       "(200000, 287)"
      ]
     },
     "execution_count": 7,
     "metadata": {},
     "output_type": "execute_result"
    }
   ],
   "source": [
    "# We drop the row_id column because we will not use it\n",
    "df.drop(columns=[\"row_id\"], inplace=True)\n",
    "df.shape"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 8,
   "id": "6af93852",
   "metadata": {
    "execution": {
     "iopub.execute_input": "2022-03-16T15:11:13.451650Z",
     "iopub.status.busy": "2022-03-16T15:11:13.450876Z",
     "iopub.status.idle": "2022-03-16T15:11:13.608445Z",
     "shell.execute_reply": "2022-03-16T15:11:13.607920Z",
     "shell.execute_reply.started": "2022-03-16T15:02:50.743656Z"
    },
    "papermill": {
     "duration": 0.251249,
     "end_time": "2022-03-16T15:11:13.608603",
     "exception": false,
     "start_time": "2022-03-16T15:11:13.357354",
     "status": "completed"
    },
    "tags": []
   },
   "outputs": [
    {
     "data": {
      "text/plain": [
       "False"
      ]
     },
     "execution_count": 8,
     "metadata": {},
     "output_type": "execute_result"
    }
   ],
   "source": [
    "# Check the presence of null values\n",
    "df.isnull().values.any()"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 9,
   "id": "d4884cbb",
   "metadata": {
    "execution": {
     "iopub.execute_input": "2022-03-16T15:11:13.781363Z",
     "iopub.status.busy": "2022-03-16T15:11:13.780384Z",
     "iopub.status.idle": "2022-03-16T15:11:13.956366Z",
     "shell.execute_reply": "2022-03-16T15:11:13.956926Z",
     "shell.execute_reply.started": "2022-03-16T15:03:38.660811Z"
    },
    "papermill": {
     "duration": 0.264134,
     "end_time": "2022-03-16T15:11:13.957096",
     "exception": false,
     "start_time": "2022-03-16T15:11:13.692962",
     "status": "completed"
    },
    "tags": []
   },
   "outputs": [
    {
     "data": {
      "text/plain": [
       "True"
      ]
     },
     "execution_count": 9,
     "metadata": {},
     "output_type": "execute_result"
    }
   ],
   "source": [
    "# Check the absence of values equals to zero\n",
    "df.all(axis=None)"
   ]
  },
  {
   "cell_type": "markdown",
   "id": "fc314cd2",
   "metadata": {
    "papermill": {
     "duration": 0.088615,
     "end_time": "2022-03-16T15:11:14.129624",
     "exception": false,
     "start_time": "2022-03-16T15:11:14.041009",
     "status": "completed"
    },
    "tags": []
   },
   "source": [
    "As we have seen, in the dataframe there aren't null or 0 values."
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 10,
   "id": "85b79230",
   "metadata": {
    "execution": {
     "iopub.execute_input": "2022-03-16T15:11:14.304482Z",
     "iopub.status.busy": "2022-03-16T15:11:14.303783Z",
     "iopub.status.idle": "2022-03-16T15:11:17.302748Z",
     "shell.execute_reply": "2022-03-16T15:11:17.303261Z",
     "shell.execute_reply.started": "2022-03-14T13:22:17.408136Z"
    },
    "papermill": {
     "duration": 3.089799,
     "end_time": "2022-03-16T15:11:17.303424",
     "exception": false,
     "start_time": "2022-03-16T15:11:14.213625",
     "status": "completed"
    },
    "tags": []
   },
   "outputs": [
    {
     "data": {
      "text/html": [
       "<div>\n",
       "<style scoped>\n",
       "    .dataframe tbody tr th:only-of-type {\n",
       "        vertical-align: middle;\n",
       "    }\n",
       "\n",
       "    .dataframe tbody tr th {\n",
       "        vertical-align: top;\n",
       "    }\n",
       "\n",
       "    .dataframe thead th {\n",
       "        text-align: right;\n",
       "    }\n",
       "</style>\n",
       "<table border=\"1\" class=\"dataframe\">\n",
       "  <thead>\n",
       "    <tr style=\"text-align: right;\">\n",
       "      <th></th>\n",
       "      <th>A0T0G0C10</th>\n",
       "      <th>A0T0G1C9</th>\n",
       "      <th>A0T0G2C8</th>\n",
       "      <th>A0T0G3C7</th>\n",
       "      <th>A0T0G4C6</th>\n",
       "      <th>A0T0G5C5</th>\n",
       "      <th>A0T0G6C4</th>\n",
       "      <th>A0T0G7C3</th>\n",
       "      <th>A0T0G8C2</th>\n",
       "      <th>A0T0G9C1</th>\n",
       "      <th>...</th>\n",
       "      <th>A8T0G0C2</th>\n",
       "      <th>A8T0G1C1</th>\n",
       "      <th>A8T0G2C0</th>\n",
       "      <th>A8T1G0C1</th>\n",
       "      <th>A8T1G1C0</th>\n",
       "      <th>A8T2G0C0</th>\n",
       "      <th>A9T0G0C1</th>\n",
       "      <th>A9T0G1C0</th>\n",
       "      <th>A9T1G0C0</th>\n",
       "      <th>A10T0G0C0</th>\n",
       "    </tr>\n",
       "  </thead>\n",
       "  <tbody>\n",
       "    <tr>\n",
       "      <th>count</th>\n",
       "      <td>2.000000e+05</td>\n",
       "      <td>200000.000000</td>\n",
       "      <td>200000.000000</td>\n",
       "      <td>200000.000000</td>\n",
       "      <td>200000.000000</td>\n",
       "      <td>200000.000000</td>\n",
       "      <td>200000.000000</td>\n",
       "      <td>200000.000000</td>\n",
       "      <td>200000.000000</td>\n",
       "      <td>200000.000000</td>\n",
       "      <td>...</td>\n",
       "      <td>200000.000000</td>\n",
       "      <td>200000.000000</td>\n",
       "      <td>200000.000000</td>\n",
       "      <td>200000.000000</td>\n",
       "      <td>200000.000000</td>\n",
       "      <td>200000.000000</td>\n",
       "      <td>200000.000000</td>\n",
       "      <td>2.000000e+05</td>\n",
       "      <td>2.000000e+05</td>\n",
       "      <td>2.000000e+05</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>mean</th>\n",
       "      <td>6.421457e-07</td>\n",
       "      <td>-0.000003</td>\n",
       "      <td>-0.000014</td>\n",
       "      <td>-0.000010</td>\n",
       "      <td>0.000005</td>\n",
       "      <td>0.000025</td>\n",
       "      <td>0.000014</td>\n",
       "      <td>-0.000009</td>\n",
       "      <td>-0.000028</td>\n",
       "      <td>-0.000008</td>\n",
       "      <td>...</td>\n",
       "      <td>0.000135</td>\n",
       "      <td>0.000348</td>\n",
       "      <td>0.000253</td>\n",
       "      <td>0.000464</td>\n",
       "      <td>0.000581</td>\n",
       "      <td>0.000379</td>\n",
       "      <td>0.000062</td>\n",
       "      <td>8.982134e-05</td>\n",
       "      <td>1.170014e-04</td>\n",
       "      <td>2.215407e-07</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>std</th>\n",
       "      <td>8.654927e-05</td>\n",
       "      <td>0.000132</td>\n",
       "      <td>0.000287</td>\n",
       "      <td>0.000436</td>\n",
       "      <td>0.000683</td>\n",
       "      <td>0.000869</td>\n",
       "      <td>0.000775</td>\n",
       "      <td>0.000441</td>\n",
       "      <td>0.000107</td>\n",
       "      <td>0.000083</td>\n",
       "      <td>...</td>\n",
       "      <td>0.000711</td>\n",
       "      <td>0.001117</td>\n",
       "      <td>0.000914</td>\n",
       "      <td>0.001238</td>\n",
       "      <td>0.001528</td>\n",
       "      <td>0.001182</td>\n",
       "      <td>0.000466</td>\n",
       "      <td>5.475606e-04</td>\n",
       "      <td>6.788277e-04</td>\n",
       "      <td>2.050081e-05</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>min</th>\n",
       "      <td>-9.536743e-07</td>\n",
       "      <td>-0.000010</td>\n",
       "      <td>-0.000043</td>\n",
       "      <td>-0.000114</td>\n",
       "      <td>-0.000200</td>\n",
       "      <td>-0.000240</td>\n",
       "      <td>-0.000200</td>\n",
       "      <td>-0.000114</td>\n",
       "      <td>-0.000043</td>\n",
       "      <td>-0.000010</td>\n",
       "      <td>...</td>\n",
       "      <td>-0.000043</td>\n",
       "      <td>-0.000086</td>\n",
       "      <td>-0.000043</td>\n",
       "      <td>-0.000086</td>\n",
       "      <td>-0.000086</td>\n",
       "      <td>-0.000043</td>\n",
       "      <td>-0.000010</td>\n",
       "      <td>-9.536743e-06</td>\n",
       "      <td>-9.536743e-06</td>\n",
       "      <td>-9.536743e-07</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>25%</th>\n",
       "      <td>-9.536743e-07</td>\n",
       "      <td>-0.000010</td>\n",
       "      <td>-0.000043</td>\n",
       "      <td>-0.000114</td>\n",
       "      <td>-0.000200</td>\n",
       "      <td>-0.000240</td>\n",
       "      <td>-0.000200</td>\n",
       "      <td>-0.000114</td>\n",
       "      <td>-0.000043</td>\n",
       "      <td>-0.000010</td>\n",
       "      <td>...</td>\n",
       "      <td>-0.000043</td>\n",
       "      <td>-0.000086</td>\n",
       "      <td>-0.000043</td>\n",
       "      <td>-0.000086</td>\n",
       "      <td>-0.000086</td>\n",
       "      <td>-0.000043</td>\n",
       "      <td>-0.000010</td>\n",
       "      <td>-9.536743e-06</td>\n",
       "      <td>-9.536743e-06</td>\n",
       "      <td>-9.536743e-07</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>50%</th>\n",
       "      <td>-9.536743e-07</td>\n",
       "      <td>-0.000010</td>\n",
       "      <td>-0.000043</td>\n",
       "      <td>-0.000114</td>\n",
       "      <td>-0.000200</td>\n",
       "      <td>-0.000237</td>\n",
       "      <td>-0.000199</td>\n",
       "      <td>-0.000114</td>\n",
       "      <td>-0.000043</td>\n",
       "      <td>-0.000010</td>\n",
       "      <td>...</td>\n",
       "      <td>0.000014</td>\n",
       "      <td>0.000064</td>\n",
       "      <td>0.000047</td>\n",
       "      <td>0.000124</td>\n",
       "      <td>0.000114</td>\n",
       "      <td>0.000043</td>\n",
       "      <td>-0.000010</td>\n",
       "      <td>4.632568e-07</td>\n",
       "      <td>4.632568e-07</td>\n",
       "      <td>-9.536743e-07</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>75%</th>\n",
       "      <td>-9.536743e-07</td>\n",
       "      <td>-0.000003</td>\n",
       "      <td>-0.000013</td>\n",
       "      <td>-0.000004</td>\n",
       "      <td>-0.000011</td>\n",
       "      <td>0.000003</td>\n",
       "      <td>-0.000030</td>\n",
       "      <td>0.000004</td>\n",
       "      <td>-0.000028</td>\n",
       "      <td>-0.000010</td>\n",
       "      <td>...</td>\n",
       "      <td>0.000111</td>\n",
       "      <td>0.000463</td>\n",
       "      <td>0.000297</td>\n",
       "      <td>0.000684</td>\n",
       "      <td>0.000774</td>\n",
       "      <td>0.000365</td>\n",
       "      <td>0.000030</td>\n",
       "      <td>6.046326e-05</td>\n",
       "      <td>6.946326e-05</td>\n",
       "      <td>-9.536743e-07</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>max</th>\n",
       "      <td>9.999046e-03</td>\n",
       "      <td>0.009990</td>\n",
       "      <td>0.009957</td>\n",
       "      <td>0.009886</td>\n",
       "      <td>0.019800</td>\n",
       "      <td>0.019760</td>\n",
       "      <td>0.019800</td>\n",
       "      <td>0.009886</td>\n",
       "      <td>0.009957</td>\n",
       "      <td>0.009990</td>\n",
       "      <td>...</td>\n",
       "      <td>0.019957</td>\n",
       "      <td>0.019914</td>\n",
       "      <td>0.019957</td>\n",
       "      <td>0.019914</td>\n",
       "      <td>0.019914</td>\n",
       "      <td>0.019957</td>\n",
       "      <td>0.009990</td>\n",
       "      <td>9.990463e-03</td>\n",
       "      <td>1.999046e-02</td>\n",
       "      <td>9.990463e-04</td>\n",
       "    </tr>\n",
       "  </tbody>\n",
       "</table>\n",
       "<p>8 rows × 286 columns</p>\n",
       "</div>"
      ],
      "text/plain": [
       "          A0T0G0C10       A0T0G1C9       A0T0G2C8       A0T0G3C7  \\\n",
       "count  2.000000e+05  200000.000000  200000.000000  200000.000000   \n",
       "mean   6.421457e-07      -0.000003      -0.000014      -0.000010   \n",
       "std    8.654927e-05       0.000132       0.000287       0.000436   \n",
       "min   -9.536743e-07      -0.000010      -0.000043      -0.000114   \n",
       "25%   -9.536743e-07      -0.000010      -0.000043      -0.000114   \n",
       "50%   -9.536743e-07      -0.000010      -0.000043      -0.000114   \n",
       "75%   -9.536743e-07      -0.000003      -0.000013      -0.000004   \n",
       "max    9.999046e-03       0.009990       0.009957       0.009886   \n",
       "\n",
       "            A0T0G4C6       A0T0G5C5       A0T0G6C4       A0T0G7C3  \\\n",
       "count  200000.000000  200000.000000  200000.000000  200000.000000   \n",
       "mean        0.000005       0.000025       0.000014      -0.000009   \n",
       "std         0.000683       0.000869       0.000775       0.000441   \n",
       "min        -0.000200      -0.000240      -0.000200      -0.000114   \n",
       "25%        -0.000200      -0.000240      -0.000200      -0.000114   \n",
       "50%        -0.000200      -0.000237      -0.000199      -0.000114   \n",
       "75%        -0.000011       0.000003      -0.000030       0.000004   \n",
       "max         0.019800       0.019760       0.019800       0.009886   \n",
       "\n",
       "            A0T0G8C2       A0T0G9C1  ...       A8T0G0C2       A8T0G1C1  \\\n",
       "count  200000.000000  200000.000000  ...  200000.000000  200000.000000   \n",
       "mean       -0.000028      -0.000008  ...       0.000135       0.000348   \n",
       "std         0.000107       0.000083  ...       0.000711       0.001117   \n",
       "min        -0.000043      -0.000010  ...      -0.000043      -0.000086   \n",
       "25%        -0.000043      -0.000010  ...      -0.000043      -0.000086   \n",
       "50%        -0.000043      -0.000010  ...       0.000014       0.000064   \n",
       "75%        -0.000028      -0.000010  ...       0.000111       0.000463   \n",
       "max         0.009957       0.009990  ...       0.019957       0.019914   \n",
       "\n",
       "            A8T0G2C0       A8T1G0C1       A8T1G1C0       A8T2G0C0  \\\n",
       "count  200000.000000  200000.000000  200000.000000  200000.000000   \n",
       "mean        0.000253       0.000464       0.000581       0.000379   \n",
       "std         0.000914       0.001238       0.001528       0.001182   \n",
       "min        -0.000043      -0.000086      -0.000086      -0.000043   \n",
       "25%        -0.000043      -0.000086      -0.000086      -0.000043   \n",
       "50%         0.000047       0.000124       0.000114       0.000043   \n",
       "75%         0.000297       0.000684       0.000774       0.000365   \n",
       "max         0.019957       0.019914       0.019914       0.019957   \n",
       "\n",
       "            A9T0G0C1      A9T0G1C0      A9T1G0C0     A10T0G0C0  \n",
       "count  200000.000000  2.000000e+05  2.000000e+05  2.000000e+05  \n",
       "mean        0.000062  8.982134e-05  1.170014e-04  2.215407e-07  \n",
       "std         0.000466  5.475606e-04  6.788277e-04  2.050081e-05  \n",
       "min        -0.000010 -9.536743e-06 -9.536743e-06 -9.536743e-07  \n",
       "25%        -0.000010 -9.536743e-06 -9.536743e-06 -9.536743e-07  \n",
       "50%        -0.000010  4.632568e-07  4.632568e-07 -9.536743e-07  \n",
       "75%         0.000030  6.046326e-05  6.946326e-05 -9.536743e-07  \n",
       "max         0.009990  9.990463e-03  1.999046e-02  9.990463e-04  \n",
       "\n",
       "[8 rows x 286 columns]"
      ]
     },
     "execution_count": 10,
     "metadata": {},
     "output_type": "execute_result"
    }
   ],
   "source": [
    "df.describe()"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 11,
   "id": "4bac4079",
   "metadata": {
    "execution": {
     "iopub.execute_input": "2022-03-16T15:11:17.484763Z",
     "iopub.status.busy": "2022-03-16T15:11:17.480997Z",
     "iopub.status.idle": "2022-03-16T15:11:19.422875Z",
     "shell.execute_reply": "2022-03-16T15:11:19.425558Z"
    },
    "papermill": {
     "duration": 2.036511,
     "end_time": "2022-03-16T15:11:19.426127",
     "exception": false,
     "start_time": "2022-03-16T15:11:17.389616",
     "status": "completed"
    },
    "tags": []
   },
   "outputs": [
    {
     "name": "stdout",
     "output_type": "stream",
     "text": [
      "Number of duplicated rows: 76007\n"
     ]
    }
   ],
   "source": [
    "# Count duplicates\n",
    "n_duplicated = df.duplicated().sum()\n",
    "print(f\"Number of duplicated rows: {n_duplicated}\")"
   ]
  },
  {
   "cell_type": "markdown",
   "id": "2de373a5",
   "metadata": {
    "papermill": {
     "duration": 0.103855,
     "end_time": "2022-03-16T15:11:19.723680",
     "exception": false,
     "start_time": "2022-03-16T15:11:19.619825",
     "status": "completed"
    },
    "tags": []
   },
   "source": [
    "There is a very high number of duplicates. But we won't remove them because we want to keep the original distribution of the data."
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 12,
   "id": "0b1200c7",
   "metadata": {
    "execution": {
     "iopub.execute_input": "2022-03-16T15:11:19.935413Z",
     "iopub.status.busy": "2022-03-16T15:11:19.934622Z",
     "iopub.status.idle": "2022-03-16T15:11:20.350932Z",
     "shell.execute_reply": "2022-03-16T15:11:20.351597Z",
     "shell.execute_reply.started": "2022-03-14T13:22:19.425473Z"
    },
    "papermill": {
     "duration": 0.5287,
     "end_time": "2022-03-16T15:11:20.351920",
     "exception": false,
     "start_time": "2022-03-16T15:11:19.823220",
     "status": "completed"
    },
    "tags": []
   },
   "outputs": [
    {
     "data": {
      "image/png": "iVBORw0KGgoAAAANSUhEUgAAAYMAAAF4CAYAAAC2MZXIAAAAOXRFWHRTb2Z0d2FyZQBNYXRwbG90bGliIHZlcnNpb24zLjUuMSwgaHR0cHM6Ly9tYXRwbG90bGliLm9yZy/YYfK9AAAACXBIWXMAAAsTAAALEwEAmpwYAABCHElEQVR4nO2debhk47X/P9/uNg/R6IhZk0YQQzQ6SGJITIkhiQhX6IggNwSZSW4uITLdyEAQQhsSIURc4ppam2LW6Jj9tOlqFy3EEATN9/fH+1affarPpE/Vuyup9Xmeeqr2u/eutc6pqr32u941yDZBEARBdzOibgWCIAiC+gljEARBEIQxCIIgCMIYBEEQBIQxCIIgCAhjEARBEACj6lZgbllyySW90kor1a1GEATBPxW33XbbX22PaR7/pzUGK620ElOnTq1bjSAIgn8qJD3W13i4iYIgCIIwBkEQBEEYgyAIgoAwBkEQBAFhDIIgCALCGARBEAQMwRhIWl7SVZLulXSPpIPy+OKSJkt6MD+PzuOSdIyk6ZLulPS+yntNzMc/KGliZXx9SXflc46RpHb8sUEQBEHfDGVmMAv4qu01gAnA/pLWAA4BptgeB0zJ2wDbAuPyY1/gBEjGAzgM2AjYEDisYUDyMftUzttm+H9aEARBMFQGTTqz/STwZH79kqT7gGWBHYHN8mGnA1cD38zjZzh1zblJ0mKSls7HTrb9HICkycA2kq4GFrV9Ux4/A9gJuGQ4f9hKh/zPcE4H4NEffnTY7xEEQfDPwNvKQJa0ErAecDOwVDYUAE8BS+XXywKPV06bkccGGp/Rx/i/BMM1Sq0wSJ1iGON/EQSdy5CNgaSFgfOAg22/WHXr27aktvfPlLQvyfXECius0G5xQdA2wjC2To9O0KFVetTJkIyBpHlIhuBM23/Mw09LWtr2k9kNNDOPPwEsXzl9uTz2BD1upcb41Xl8uT6OnwPbJwEnAYwfPz6aNwdB8C9FnYZxKNFEAk4B7rP908quC4FGRNBE4ILK+J45qmgC8EJ2J10GbCVpdF443gq4LO97UdKELGvPynsFQRAEBRjKzGATYA/gLknT8ti3gB8C50jaG3gM2CXvuxjYDpgOvALsBWD7OUlHArfm445oLCYDXwROAxYgLRwPa/E4CIIgeHsMJZroOqC/uP8t+zjewP79vNckYFIf41OBtQbTJQiCIGgPkYEcBEEQhDEIgiAIwhgEQRAEhDEIgiAICGMQBEEQEMYgCIIgIIxBEARBQBiDIAiCgDAGQRAEAWEMgiAIAsIYBEEQBIQxCIIgCAhjEARBEBDGIAiCICCMQRAEQUAYgyAIgoAwBkEQBAFD64E8SdJMSXdXxn4vaVp+PNpohylpJUmvVvb9qnLO+pLukjRd0jG53zGSFpc0WdKD+Xl0G/7OIAiCYACGMjM4DdimOmD707bXtb0ucB7wx8ruhxr7bH+hMn4CsA8wLj8a73kIMMX2OGBK3g6CIAgKMqgxsH0t8Fxf+/Ld/S7AWQO9h6SlgUVt35R7JJ8B7JR37wicnl+fXhkPgiAICjHcNYMPAE/bfrAyNlbSHZKukfSBPLYsMKNyzIw8BrCU7Sfz66eApfoTJmlfSVMlTX3mmWeGqXoQBEHQYLjGYDd6zwqeBFawvR7wFeB3khYd6pvlWYMH2H+S7fG2x48ZM2ZudQ6CIAiaGDW3J0oaBXwCWL8xZvs14LX8+jZJDwGrAk8Ay1VOXy6PATwtaWnbT2Z30sy51SkIgiCYO4YzM/gwcL/t2e4fSWMkjcyvVyYtFD+c3UAvSpqQ1xn2BC7Ip10ITMyvJ1bGgyAIgkIMJbT0LOBGYDVJMyTtnXftypwLxx8E7syhpn8AvmC7sfj8ReBkYDrwEHBJHv8h8BFJD5IMzA/n/s8JgiAI5oZB3US2d+tn/LN9jJ1HCjXt6/ipwFp9jD8LbDmYHkEQBEH7iAzkIAiCIIxBEARBEMYgCIIgIIxBEARBQBiDIAiCgDAGQRAEAWEMgiAIAsIYBEEQBIQxCIIgCAhjEARBEBDGIAiCICCMQRAEQUAYgyAIgoAwBkEQBAFhDIIgCALCGARBEAQMrdPZJEkzJd1dGTtc0hOSpuXHdpV9h0qaLukBSVtXxrfJY9MlHVIZHyvp5jz+e0nztvIPDIIgCAZnKDOD04Bt+hj/me118+NiAElrkNphrpnPOV7SyNwX+ThgW2ANYLd8LMCP8nu9G/gbsHezoCAIgqC9DGoMbF8LPDfYcZkdgbNtv2b7EVK/4w3zY7rth22/DpwN7ChJwBakfskApwM7vb0/IQiCIBguw1kzOEDSndmNNDqPLQs8XjlmRh7rb3wJ4Hnbs5rGgyAIgoLMrTE4AVgFWBd4Eji6VQoNhKR9JU2VNPWZZ54pITIIgqArmCtjYPtp22/afgv4NckNBPAEsHzl0OXyWH/jzwKLSRrVNN6f3JNsj7c9fsyYMXOjehAEQdAHc2UMJC1d2fw40Ig0uhDYVdJ8ksYC44BbgFuBcTlyaF7SIvOFtg1cBeycz58IXDA3OgVBEARzz6jBDpB0FrAZsKSkGcBhwGaS1gUMPArsB2D7HknnAPcCs4D9bb+Z3+cA4DJgJDDJ9j1ZxDeBsyV9D7gDOKVVf1wQBEEwNAY1BrZ362O43wu27aOAo/oYvxi4uI/xh+lxMwVBEAQ1EBnIQRAEQRiDIAiCIIxBEARBQBiDIAiCgDAGQRAEAWEMgiAIAsIYBEEQBIQxCIIgCAhjEARBEBDGIAiCICCMQRAEQUAYgyAIgoAwBkEQBAFhDIIgCALCGARBEASEMQiCIAgIYxAEQRAwBGMgaZKkmZLuroz9l6T7Jd0p6XxJi+XxlSS9Kmlafvyqcs76ku6SNF3SMZKUxxeXNFnSg/l5dBv+ziAIgmAAhjIzOA3YpmlsMrCW7bWB/wccWtn3kO118+MLlfETgH2AcfnReM9DgCm2xwFT8nYQBEFQkEGNge1rgeeaxi63PStv3gQsN9B7SFoaWNT2TbYNnAHslHfvCJyeX59eGQ+CIAgK0Yo1g88Bl1S2x0q6Q9I1kj6Qx5YFZlSOmZHHAJay/WR+/RSwVAt0CoIgCN4Go4ZzsqRvA7OAM/PQk8AKtp+VtD7w35LWHOr72bYkDyBvX2BfgBVWWGHuFQ+CIAh6MdczA0mfBT4G7J5dP9h+zfaz+fVtwEPAqsAT9HYlLZfHAJ7ObqSGO2lmfzJtn2R7vO3xY8aMmVvVgyAIgibmyhhI2gb4BrCD7Vcq42MkjcyvVyYtFD+c3UAvSpqQo4j2BC7Ip10ITMyvJ1bGgyAIgkIM6iaSdBawGbCkpBnAYaToofmAyTlC9KYcOfRB4AhJbwBvAV+w3Vh8/iIpMmkB0hpDY53hh8A5kvYGHgN2aclfFgRBEAyZQY2B7d36GD6ln2PPA87rZ99UYK0+xp8FthxMjyAIgqB9RAZyEARBEMYgCIIgCGMQBEEQEMYgCIIgIIxBEARBQBiDIAiCgDAGQRAEAWEMgiAIAsIYBEEQBIQxCIIgCAhjEARBEBDGIAiCICCMQRAEQUAYgyAIgoAwBkEQBAFhDIIgCALCGARBEAQM0RhImiRppqS7K2OLS5os6cH8PDqPS9IxkqZLulPS+yrnTMzHPyhpYmV8fUl35XOOyX2SgyAIgkIMdWZwGrBN09ghwBTb44ApeRtgW2BcfuwLnADJeJD6J28EbAgc1jAg+Zh9Kuc1ywqCIAjayJCMge1rgeeahncETs+vTwd2qoyf4cRNwGKSlga2Bibbfs7234DJwDZ536K2b7Jt4IzKewVBEAQFGM6awVK2n8yvnwKWyq+XBR6vHDcjjw00PqOP8TmQtK+kqZKmPvPMM8NQPQiCIKjSkgXkfEfvVrzXIHJOsj3e9vgxY8a0W1wQBEHXMBxj8HR28ZCfZ+bxJ4DlK8ctl8cGGl+uj/EgCIKgEMMxBhcCjYigicAFlfE9c1TRBOCF7E66DNhK0ui8cLwVcFne96KkCTmKaM/KewVBEAQFGDWUgySdBWwGLClpBikq6IfAOZL2Bh4DdsmHXwxsB0wHXgH2ArD9nKQjgVvzcUfYbixKf5EUsbQAcEl+BEEQBIUYkjGwvVs/u7bs41gD+/fzPpOASX2MTwXWGoouQRAEQeuJDOQgCIIgjEEQBEEQxiAIgiAgjEEQBEFAGIMgCIKAMAZBEAQBYQyCIAgCwhgEQRAEhDEIgiAICGMQBEEQEMYgCIIgIIxBEARBQBiDIAiCgDAGQRAEAWEMgiAIAsIYBEEQBIQxCIIgCBiGMZC0mqRplceLkg6WdLikJyrj21XOOVTSdEkPSNq6Mr5NHpsu6ZDh/lFBEATB22NIbS/7wvYDwLoAkkYCTwDnk3oe/8z2T6rHS1oD2BVYE1gGuELSqnn3ccBHgBnArZIutH3v3OoWBEEQvD3m2hg0sSXwkO3HJPV3zI7A2bZfAx6RNB3YMO+bbvthAEln52PDGARBEBSiVWsGuwJnVbYPkHSnpEmSRuexZYHHK8fMyGP9jc+BpH0lTZU09ZlnnmmR6kEQBMGwjYGkeYEdgHPz0AnAKiQX0pPA0cOV0cD2SbbH2x4/ZsyYVr1tEARB19MKN9G2wO22nwZoPANI+jVwUd58Ali+ct5yeYwBxoMgCIICtMJNtBsVF5GkpSv7Pg7cnV9fCOwqaT5JY4FxwC3ArcA4SWPzLGPXfGwQBEFQiGHNDCQtRIoC2q8y/GNJ6wIGHm3ss32PpHNIC8OzgP1tv5nf5wDgMmAkMMn2PcPRKwiCIHh7DMsY2H4ZWKJpbI8Bjj8KOKqP8YuBi4ejSxAEQTD3RAZyEARBEMYgCIIgCGMQBEEQEMYgCIIgIIxBEARBQBiDIAiCgDAGQRAEAWEMgiAIAsIYBEEQBIQxCIIgCAhjEARBEBDGIAiCICCMQRAEQUAYgyAIgoAwBkEQBAFhDIIgCAJaYAwkPSrpLknTJE3NY4tLmizpwfw8Oo9L0jGSpku6U9L7Ku8zMR//oKSJw9UrCIIgGDqtmhlsbntd2+Pz9iHAFNvjgCl5G2BbUu/jccC+wAmQjAdwGLARsCFwWMOABEEQBO2nXW6iHYHT8+vTgZ0q42c4cROwmKSlga2Bybafs/03YDKwTZt0C4IgCJpohTEwcLmk2yTtm8eWsv1kfv0UsFR+vSzweOXcGXmsv/EgCIKgAKNa8B6b2n5C0juByZLur+60bUlugRyysdkXYIUVVmjFWwZBEAS0YGZg+4n8PBM4n+Tzfzq7f8jPM/PhTwDLV05fLo/1N94s6yTb422PHzNmzHBVD4IgCDLDMgaSFpK0SOM1sBVwN3Ah0IgImghckF9fCOyZo4omAC9kd9JlwFaSRueF463yWBAEQVCA4bqJlgLOl9R4r9/ZvlTSrcA5kvYGHgN2ycdfDGwHTAdeAfYCsP2cpCOBW/NxR9h+bpi6BUEQBENkWMbA9sPAOn2MPwts2ce4gf37ea9JwKTh6BMEQRDMHZGBHARBEIQxCIIgCMIYBEEQBIQxCIIgCAhjEARBEBDGIAiCICCMQRAEQUAYgyAIgoAwBkEQBAFhDIIgCALCGARBEASEMQiCIAgIYxAEQRAQxiAIgiAgjEEQBEFAGIMgCIKAMAZBEAQBwzAGkpaXdJWkeyXdI+mgPH64pCckTcuP7SrnHCppuqQHJG1dGd8mj02XdMjw/qQgCILg7TKctpezgK/avl3SIsBtkibnfT+z/ZPqwZLWAHYF1gSWAa6QtGrefRzwEWAGcKukC23fOwzdgiAIgrfBXBsD208CT+bXL0m6D1h2gFN2BM62/RrwiKTpwIZ53/TcTxlJZ+djwxgEQRAUoiVrBpJWAtYDbs5DB0i6U9IkSaPz2LLA45XTZuSx/sb7krOvpKmSpj7zzDOtUD0IgiCgBcZA0sLAecDBtl8ETgBWAdYlzRyOHq6MBrZPsj3e9vgxY8a06m2DIAi6nuGsGSBpHpIhONP2HwFsP13Z/2vgorz5BLB85fTl8hgDjAdBEAQFGE40kYBTgPts/7QyvnTlsI8Dd+fXFwK7SppP0lhgHHALcCswTtJYSfOSFpkvnFu9giAIgrfPcGYGmwB7AHdJmpbHvgXsJmldwMCjwH4Atu+RdA5pYXgWsL/tNwEkHQBcBowEJtm+Zxh6BUEQBG+T4UQTXQeoj10XD3DOUcBRfYxfPNB5QRAEQXuJDOQgCIIgjEEQBEEQxiAIgiAgjEEQBEFAGIMgCIKAMAZBEAQBYQyCIAgCwhgEQRAEhDEIgiAICGMQBEEQEMYgCIIgIIxBEARBQBiDIAiCgDAGQRAEAWEMgiAIAsIYBEEQBIQxCIIgCOggYyBpG0kPSJou6ZC69QmCIOgmOsIYSBoJHAdsC6xB6qO8Rr1aBUEQdA8dYQyADYHpth+2/TpwNrBjzToFQRB0DbJdtw5I2hnYxvbn8/YewEa2D2g6bl9g37y5GvDAMEUvCfx1mO8xXDpBB+gMPUKHHjpBj07QATpDj07QAVqjx4q2xzQPjhrmmxbF9knASa16P0lTbY9v1fv9s+rQKXqEDp2lRyfo0Cl6dIIO7dajU9xETwDLV7aXy2NBEARBATrFGNwKjJM0VtK8wK7AhTXrFARB0DV0hJvI9ixJBwCXASOBSbbvKSC6ZS6nYdAJOkBn6BE69NAJenSCDtAZenSCDtBGPTpiATkIgiCol05xEwVBEAQ1EsYgCIIgCGMQBEEQhDGoHUmjJa1dsw4L1ik/CIL66YhoopJI2gSYZvtlSZ8B3gf8wvZjBXW4GtiB9P+/DZgp6XrbXymlQ9ZjY+BkYGFgBUnrAPvZ/mIB2avbvl/S+/rab/v2duvQh04fBdYE5q/ocUQBuT+3fbCkPwFzRHTY3qHdOjTpsyDwVWAF2/tIGgesZvuiArK3sH2lpE/0td/2H9utQ0WXhYBXbb8laVVgdeAS22+U0iHrIWB3YGXbR0haAXiX7VtaKafrjAFwArBOvvB9lXQxPAP4UEEd3mH7RUmfB86wfZikOwvKb/AzYGtyToftv0j6YCHZXyGVFjm6j30GtiikBwCSfgUsCGxO+k7sDLT0xzYAv8nPPykkbzBOJd2kvD9vPwGcC7TdGJB+h1cC2/exz0AxYwBcC3xA0mjgclI+1KdJF+aSHA+8RfpNHAG8BJwHbNBKId1oDGbZtqQdgV/aPkXS3oV1GCVpaWAX4NuFZffC9uPpxmM2bxaSu29+3ryEvCGwse21Jd1p+7uSjgYuKSHY9m35+ZoS8obAKrY/LWk3ANuvqOlL0i5sH5af9yohbxCU//a9geNt/1jStBr02Mj2+yTdAWD7bzk5t6V0ozF4SdKhwGeAD0oaAcxTWIcjSAl219u+VdLKwIOFdQB4PLuKLGke4CDgvtJKZB1WovJ9tH1GYTVezc+vSFoGeBZYuoRgSXfRh3sIEGDbpdeUXpe0QEMnSasAr5UQLOkztn8rqU+Xqe2fltCjRx29nzQTaNwwjiwov8Ebucx/4/MYQ5optJRuNAafBv4N2Nv2U9n/9l8lFbB9Lmna3dh+GPhkSR0yXwB+ASxLcgVcDuxfUgFJvwFWAabRMysxyXVXkoskLUb6LtyedTi5kOyPFZIzVA4DLgWWl3QmsAnw2UKyF8rPixSSNxAHA4cC59u+J9+0XVWDHscA5wPvlHQUyYX5H60WEhnINZAXo04AlrK9Vo4m2sH292pWrTiS7gPWcAd9ESXNB8xv+4UaZC9Fjy/4FtszS+uQ9VgCmECandxkuxPKN3ctklYHtiR9HlNst3wG3zXGQNJ1tjeV9BK9p+SNqfiiBXW5Bvg6cKLt9fLY3bbXKqVDljk/afrbHEHzuYI6nAscaPvJUjL70WN/4Ezbz+ft0cButo8vqMMupJnJ1aTv5QeAr9v+QykdKrqMBsbR+3txbUH5Y4B9mNN9WPK7eRV9R3cVDW5oIOmd9P48/reV7981biLbm+bnTph+Lmj7lqY1uVk16PEb4H5SRNERJN9o6TWDJYF7Jd1CxS9dOpwS2Mf2cRX5f5O0DymSoxTfBjZozAbyBfEKoKgxyFFuB5FKyU8jzRBupGyE1wXAn0l/f5Gghj74WuX1/CRXbvHfqaQdSFF3ywAzgRVJv9M1Wymna4yBpMUH2m/7uVK6AH/Ni3KNBaGdgTrujN9t+1OSdrR9uqTfkX6AJTm8sLz+GClJDXdVXrBrecTGIIxocgs9Sz2JoQeRXFU32d48uyi+X1iHBW1/s7DMXjSivCpcn29aSnMkySBfYXs9SZuTAmBaStcYA1LctEnT72YMrFxQl/1JpWhXl/QE8Aht+HCHQCN55nlJawFPAe8sqUAHhVNeCvxe0ol5e788VlQHSZcBZ+XtT1MovLWJf9j+hyQkzZeTA1crrMNFkrazfXFhubNpuoEcAawPvKMGVd6w/aykEZJG2L5K0s9bLaRr1gw6kZzhOML2SzXJ/zwpeWVtUqLRwsB/2v5VQR2qazjzksJ8Xy65hpP1GEEyAFvmocnAybaLuihy5u2mefPPts8vKT/rcD6wFymaZgvgb8A8trcrqMNLpMii1+m5aSm9tvcIPTeQs0g3bUfYvq6UDlmPK4CdgB+Q3KozSe7EjVsqp1uMQSeVP8jRKp9kzsWxtpc+6GRyYtOOwATbh9StT2kkjQWetP2PvL0AKeLs0Rp1+hDpbvhS26/XpUc3k28a/0EySruTPo8zbT/bUjldZAxOsr1vjhBoxiUjBCRdCrxAcl3NvvO03VdphnbI76TEnjmQdEcjyqqArHNs79Jf4lfJhC9JU0mZ0K/n7XlJiYktLTswiA4jgXtsr15K5gC67AA0yqNcXaI2UpP8BUllU1bI145iNZrqoGvWDDqs/MFytrepUX7HJPaod0GyEcB40l1QKQ7Kz52Q+DWqevdt+/V2lB0YCNtvSnpA0gqtDl18O0j6IWkR+8w8dJCkTWwfWlCNRo2mhjumZI2m4uHwXTMzaKC+qyG+ANxVKsFH0knAsbbvKiGvk5F0amVzFvAo8OuSyVb5bviKum8UJE0mfS8uzNs7knIwthz4zJbrcS2wHqlQ38uN8ZLhvkqFG9e1/VbeHgncUXqmZnt8daYq6S+21ymlQ0m6ZmZQYW9SNcaGu2gzkvUfK+kI27/p78QWsinw2bxA9Ro11aDphMQed0BBsnw3/Jakd9SRdVzhC8CZkn6Zt2cAe9Sgx3dqkNkXiwGNkO86onhqq9HUTDaGS9H7dxpJZ8NkFPAe20/D7PT/M4CNSCVrSxiDbQvIGAq1JfZI+oZTFchjmdNXb9JF4Le2Hyqk0t+Bu/LdefVu+MBC8sl/6wRJC+ftv1f3S5po+/QCenRCuO8PgDvyGp9IawelgwrqrNE0G0lfyro8TU+BOpOiAFsnpwvdRPfaXqOyLdKC2RqFFy43BcbZPjXfoS9s+5ESsis6TLO9bkmZFdnb2/6TpIn9HLIEMLHUlLw/PUpcfIeKpNtt9xkN16L375iSLVmfpeldp+mpgrJHkArCTaHmGk2SppPKWLc0emgOOV1oDI4HVqCnaugnSdPxrwMXlfAbSzqMtFC6mu1VlUomn2t7k3bLbtLje8ANdSb2VHRZ0PYrTWP72T6xv3PaoMMCpMiRB0rJfDuUvFmpG3VGR8KptseXkjeAHlcBH7Hd1lIY3WgMRDIAjQvv9cB5LviPUGqQsR5we2Vh6s4a1gw6IbHn/cAppJlR0dabTXpsT+o0Nq/tsZLWJSUYla6R1C/tnhk0yWq7j3oQ+XcC69CTEHkKsIvtYh0Jc0TTX4Hf09t1WLJ0DZJOAVYD/ofe9btaGgLedWsG+aL/BwoX/2ridduW1FiYWmiwE9qBO6No38+pr/VmlcOBDUkVQ7E9Tal+fSdRpNtYKR/1IFQ7Eh7nejoSfjo/V3t8lC5dA/C/+TEvbayX1XXGQNIE4FjgPaR/7EjKlz84R6kGzmJKlTE/B/y6oPzZ1J3YA/W13mziDdsvNOnR8m5Sw+T6QnIOIrkw2+qjHoTaOxLaHltSXn/Y/i5Af4EFraLrjAHwS2BX0prBeGBPYNWSCtj+iaSPAC+Spn//aXtySR2gYxJ7OqL1JnCPpH8jVS8dBxwI3FBaCUkfZc7+Ekfk5wMKqfE4KfemTmrvSChpz77GXbglq1IRyd8Ai+ftvwJ72r6npXK6cM2gkUgy20ffTQtzVToksWdJUuvND5PcIJcDB5W+K82lB74NbJWHLgOOtF0srlzSr4AFgc1JLTd3JkXRFHGPqKc8yZoU8FF3OjnsucH8pCKGt9veubAeNwDftn1V3t4M+L5bXKiuG2cGr+QU/2mSfkzqI1C0ZnwfoXuQ7sSmAl916olcisWoMbEnh+rtXlpuH3zU9rdJBgEASZ+i0qu6ABvbXjvfqHxX0tGULWHdWEMq4qMeCPVdzfbvtot9R21/qUmnxYCzS8mvsFDDEADYvrod64zdaAz2IF38DwC+DCxP+Wb0PyeFs/6OdDe8K6kp/O3AJFJWdAlqS+zpwKSzQ5nzwt/XWDt5NT+/ksONnwWWLiW84ZseDEnHNl8o26DL7OCGHAG4Iynev05eBupYR3hY0nfoSYj9DNDyG8auchNlN8gZtmu9E+2rvkkjAax07ZO6Ens6JelM0rbAdsAupBDCBosCa9jesJ3ym3T5Dim4YUvgOJJRPNl2p5SHAMqGuDbJLerOlfQnem5URgBrAOe4cHl1pX7U36XS5wI43PbfWimnq2YGTjVoVpQ0r+utzf6KUvPzRnjrzvRU6ixtncfk51HAxpKw/cd2C7X9p/zcb4avpFcK3IX+H8k9twOpRlWDl0gzx2LYPjK/PE/SRcD8rrdWUm2o/mq2kPJOGswCHrM9o7AO5It+28uidNXMAEDSGaSw0gvpnUhSbHEsx6//glQwD1Kz8S+TSuSu70KdlCRNIsWO30MlntwFC9UNRqm7UEnz2H5j8CPbInsL21eq74q6RYzz26HEZ6IOqGbbKUgaD3yLOQtKtjTQo6tmBpmH8mMENdXzzwvE2/ezu2RLvQmu1GnqcjaUdDiwIul30ajHUyLB6EPAlfT9nTDQUcaAAslv7oBqth2SkwQp9PvrwF20Mfela4yBpN/Y3gN43vYvatZlOdKXrFES48+kcMrSU9AbJa1h+97CcjuRU0izs17d50pg+7D8XPsFsJmc7LWw7Rcrw23//Ug6po/hF4Cpti9ot/xM7TlJmWece1y0k65xE0m6lxTLfgkpWqfX3U3JeiNKZZJ/R+/ogN1tf6SUDlmPD5HcZU9RY1+FgSi1aCjpZtsbtVvOIDp0RG9sSb8j9VZ4E7iVtJj+C9vFkr6UGkCtTu+Cko+QAgsetn1wAR06IidJ0pbAbqQKqtW8j5bOGLtmZgD8ivTPXJl091c1BqXrjYyxXfWJnibp4ILyG5xCCrVt6/SzP3J0149sf22Aw0rN4q6S9F8kl0z1B3d7IfmQ+ks0emPX0kQls4btFyXtTrp5OiTrVDIDeG1gE9tvAkg6gTSD3pT0fS1B7TlJmb1IhnEeeteKCmMwN9g+BjhG0gm2/72/4ySNbnXIVh88q1SW96y8vRspprw0Raaf/ZGjuzYd5JjTCqnTmBVUSxYb2KKQfKi/N3aDeXJpkJ2AX9p+o1FUsSCjgYXpKYuxELB4/s6UMpSdkJMEsIHt1dotpGuMQYOBDEFmCql2ejv5HGnN4GekC84NJOtfmjuyS+BPtHH6OQQdLiS5A6rRXUUXTV1z/+PMDZLe6/p7Y59Iit75C3CtpBVJdbRK8mPSHfnV9CREfj9n3l5RQgH39E74BynOvxeSzrNdwjjcUGJtr2vWDIZKCZ+gpCVK197pR49T+xguGlraCTpkPZYCvg8sY3tbSWsA77d9SgHZd5FuCkYB40jZpR21hiNplNvcXKUiawQp2/gRUllxgFtt/18J+UOl4HrWfaQKBW3tmR7GoIlCMdQPAtNIpScudXwItSPpElITlW/bXkfSKFLRvvcWkL3iQPtdsLsXgKT/7EePYgvZ/wzFIwvmwPT5/Wj196KOxZAghaedRApVe1DS9yUVD1mTdKqkSc2PwjqsKmmKpLvz9tqS/qOkDpklbZ9DXqDLd8FFQkxtP5Z/2EsDz1W2/wa8q4QOTbxcebwJbEuKcCrJFEmfzHWJuh3382gpMTNooob6J5sDvyUtkP0FOMT2jYVkV/2d8wMfB/7PdttT3ys6XENKqDnRPS1A77a9VikdssyrSYuDk22/Lycc/chl2yzeAbyvMVPM7pKpJe4+B9FrPuAy25sVlNloyfomqYBfwzVSOuGrXwq6iRpuRJF+p2OBB2yv2Uo5XbeALGkVYIbt15Tqgq9NKl73fD5kywI6LEHKLdiD1FrwS6R4/3VJC6lFKiPaPq9Jr7MomwENsKDtW5puAIv4ppv4CukzWEXS9aSaTUXr1pNuzmbfndl+K7ur6mZBYLmSAt0ZLVlnk4vFLW/7zsrwN0vIbnZVSnof0PIe4d3oJjoPeFPSu0mumuVJCWBAseSzG0mJPDvZ/qjtP9qeZXsqKR+iLsYB7yws86/ZQDfuhncmxXMXJecTfAjYGNgPWLP6w1fqTNduHpZ0oKR58uMg2lCqeDAk3SXpzvy4B3iAVHa9pA6S9BmlSq5IWl5SsQqyWebVkhaVtDipvPyvJc2uYWb78pL6VOTeTk8odMvoOjdRY9FH0teBf9g+tgbXkGxb0qKkqe9LpWQ36dFoIKL8/BRwaPOMoc06rEwyyhuTfOSPkLKxiy6aDkahwIJ3AseQchtMCnM+2IWLszUtWM4Cni4VSVTR4QTS+s0Wtt+T78wvt73BIKe2Uoc7bK8n6fOkWcFh1Wzkgnp8pbI5ghT6voTtrVsppxOmoKV5Q9JuwER6CoMVbbQNrJ9DKhch2Ybngc/Zvm3g01pLh0zFbfvDOX58hO2XJHVEI/ImShRnm0mqhVM3SwP3NG5SJC2S49xvLqjDRvmm7Q5IZZxzNnBJRin1+9iFSge8Gqj+TmeR2pG2/IatG43BXqS6K0fZfiRfeH4zyDmtZhLwRdt/BshZuKeS1i+KkSM1dgfG2j5Sqen4u2zfUlCN80iLpi9Xxv4ArF9Qh6HQtim0Bu76RskF/cwJ9E68fLmPsXbzhlK5kob7cAzlS6YcQeqFfZ3tW/Ms9sHCOszuQCdpQduvtEtO1xkD2/dK+iawQt5+BPhRYTXebBiCrMN1kupYND2ePBUHjiQ1dDmPns5nbUPS6qTG6+9Q7zr+i5IiJrqJ+/Lz1Fq16KETFrKPAc4H3inpKNJiftGQY9vnUml76lR6vng5CknvJ9URWxhYQdI6wH62W7qI3HXGQNL2pA5G8wJjJa0LHGF7h4JqXCPpRFJtIgOfBq7OUQIli6PVORVfDfgYsBi96/i/BOxTSIe3w6PtemOn9p8jgfd64KJ9pXhY0oGk2QCkyJUiC9mSxtp+xPaZkm4jRfeJFGxx3yCnt1qXU+l7pla6+dPPga1J0W7Y/oukD7ZaSNcZA+BwUor71QC2p+XpX0kafX0Paxpfj7LF0WqbijvVpL9A0vtL5VUMhqS1SH1uZ89MbJ+Rn/vsQtYqnAqwbTL4kUX4AunO/D/oWcjet5DsP5DW1KbY3hK4v5Dcvrio8np2Hk4dith+vCn8uuUJkd1oDN6w/ULTP7aoL9KDFEWTNNED9AZuIbVPxYEvSLqvkeeRo0aOLn33JekwUp+LNYCLSVm31wFnFFRjmjqjaF+dC9kjJH0LWLUpigYo2562Q/JwAB6XtDFgpWqyB9HjWmwZ3ZhncI+kfwNGShqXF+1uqFupJg4qIcT2mcA3gB+QYvt3yn7SkqxdSfhrNP+uoybNziSXxFNOHcfWAd5RWIf5SaXMtyC5zrYnudKKIul0SYtVtkerXJmSXUl3vaNIUTTNjzqpIw8H0kxtf2BZUp/0dfN2S+nGmcGXSGFir5F89peRFk87iZL1WB4klSceBSBpBdv/W1D+CFV6SOQEnzq+l6/mhdJZOf9jJikhsRjunLaXcxhoSUUMtO0HgB/leP5L+juuxOy5kofT4CkKZR1Xsf1XUtRfW+k6Y5BDs75NvXHDg1EkE1DSl0jrFk+T7sYayWclQ1yPJvViPjfL3xk4qqD8BlPz3fCvSV29/k7KFC+GUrHCE4ClbK8laW1gB9vfK6kHHWCgBzIEmYOAthqDDsnDaazl7cOc7VBb6krtmgxkSX9igIts4WiiASmVES1pOimiqNbeCpLWBBrrKFe6zU08hqDPSsCiTXVoSsjtlKJ9ewLfIq1dzDbQtkvn4/RLid+IpI+Tvo8v5O3FgM1s/3c75fahxw2klp+3UVk4bnWlgG6aGfwkP3+CVBb4t3l7N9KdcSdxfSE5j9PTVrA2bN8j6RlyFE9JV1UjnLe/fQXDfKFDivbZPkPSVHrKYnyibgPdByXuYg+zff5sgfbzOdDgvwvIrrKg7ba7p7rGGNi+BkDS0barfW7/lL/4xVAqQHYqKab+ZNKC6SHOha9sH1BIlYdJ+Q3/Q++2l8UiNiTtQHIVLUPy069IipRoaXneATh6gH2leyB3RNG+zDz0rF2VLtcyFEqsq/UVYFPHNfMiSdvZvridQrrGGFRYSNLKOZuQXI5iocI6fM72LyRtTWr8vQepJEbpKoj/mx/z5kcdHElqcXhFLgq2Oam8dxEGC/MtzP6kon2rS3qCVLSv2P+iQb5Z2YeUjS7gt5JOsn1saV0GoMTseapSldLj8vb+JFdNaQ4CviXpdeCNPGa3uLdD16wZNJC0DekH9zDpi74iKbX7soI63Gl7bUm/AK62fX6pdYJ+9KmteqqkqbbHS/oLsF6O6PmL7XUGPbk18gdMJisd4w+gStG+0rKz/DtJ/Z9fruhzo8tX6/woaYZYTQIs2XpzIeA7wIdJs7XJpLWTlwc88Z+UrpsZ2L5U0jhg9Tx0v+3XBjqnDdwm6XJSE5tDJS1C+SJcSBpPclctkrdfoHz11OclLUxaIDtT0kwqCVcF2H6AfQaKGQOljmKfJEeNNNYOSl4AG6rQO8O1EWlWTgHpV6SmOpuTXKk7AyULKJIv+oeUlNkf+aZlU9J38s/tWMTumpmBpC1sX9nfnWDJO0CldobrAg/nRaklgGVriF65E9jfvaunHl/yDjDfff2DdLHZnZTodWbdEU51IOlS0oJ+c9TIQOsa7dDjK6QS743F052A02z/vKAOjdlz43lh4BLbHyiow2TgU03Z8We7xX0EhqDH8cC7SXlRkGqZPWS7pYln3TQz+BBwJX3fCRa9AyRZeIC1VW+/79qrp9p+WdK7SPWiniP12i1uCCQtBXwfWMb2tpLWILlKTimoxnK2tykor09s/1SpJ3Tje7qX7TsKq/Fqfn5F0jKkzOylC+uwZB/Jd3VkIG8BvMee3Rv7dOCeVgvpGmNg+7D83AlZnl+vvJ6fdCG8jbKRK9AB1VOVukj9J8lQCzhW0hG2S5U/aHAayWXWSEb8f8DvSaWDS3GDpPfavqugzDmQNIHU3Ob2vL2opI1ctrnNRTmu/79ILSdNcheV5K1qmLNSB7g6XCnTSSX3G93/ls9jLaVr3EQNJL2DlHXbKAF7DamEdW3x9pKWB35uu2itdElXDbDbtttunCQ9AGzcmA1kl9kNtldrt+wmPW61vUF1IV/SNNvrFpB9F+kiM4pU/+ZhUqivSJ9D6YXbO0gNhxp3oiOAqW5z288B9JkPmL/0b7QSbHIN6bP4ALBvyWCTrMc1pB4jt5C+JxuSel+8AK1LmO2amUGFScDdpFZ2kMI6TyUlo9XFDOA9pYUOFlZZov4LafpfjZp5KY+V5uVsiBoXwAmUS8grXoxuEGprbjPQ2p6komt7OdjkfaTQZ0j9qP9aSn6F/ywhpBuNwSpNd+DflTStpALq3d6wsZhcMtN1qLSt/ot6yhNPB26WdAHpf7IjUHQhPfMVUvOQVSRdD4whRbC0HduPQS/3TKP38KKkm4THBji9HdTW3IbOWtsD2JgeLwL07nFQhEbCbH9IutH2+4crpxvdRDcCX7d9Xd7eBPhJK/6Zb0OHiZXNWcCjtkuVoBgy7cx9yGn9/eLc97Uk+e53NZJL4AHbbwxySqvld4R7Ji+SHkPPGtYVpLvimSX1qBtJPyS5Z87MQ7sBt9r+Vn1azUmrfqfdaAzWITUsadSq/xswsWRYZyOc0vabeXskMJ/b2Ox6bpB0e6kLUQ4dxPbfS8jrR4eNmbMyZLHmNn2tUTRCK0vp0Ck051w0xgsnnd0JrGv7rbw9Erij0z6PVv1Ou6q5Tf4w98jZrWuT6ravVzq+n9RGcIHK9gKku69Oo+1xr5LWynfE95AaD92mVMW0KJJ+QypmuCnpbnADYPyAJ7WehyUdKGme/DiIcu6Z2UhaTtL5kmbmx3mSliusxgUkl+EsUhJi41GaxSqvSzc7KkpXrRk49ZndNL9+sUZV5q/eAdv+u6QFa9SnP0q4rk4CvmL7KgBJm5F6CmxcQHaV8cAarneqXGfv4SqnAr8DPpW3P5PHPlJQh07IufgBcEeOuhNp7aAjMpKbaMlNW1cZg8wdqr/P7MuqlEeWtD49STbFUGdUT12oYQiyzKuzG600d5NKm9dVJbTu3sNVxtg+tbJ9mqSDC+tQe86F7bNy8t0Geeibtp+qS58B2KMVb9KNxqDaZ7ZB6SiFg4FzJf0fyaq/i5TwVZpOqJ76sKTvZLmQ7kKLu0aAJYF7Jd1C73LexZoeSfox8D3SjcGlJFfml23/dsATW8+zkj5DT/mD3SgU7tuUc7GXpNpyLtTT3ObCvL2YpJ1cvrnNBOBYUmTZvMBI4GXnqqW2726JnG5bQO4UJM1DilyBGiJXsg61V0/N9V6+S0/pgz8Dhzu3XCyox4f6Gh8srK/FOkyzvW6+CH2MFO56rQtVcK3osSLp4vN+0oX5BuBAF2g4lGX3SyMMtwT9LOgX/X1kmVNJM8ZzSe7MPYFVbR/aSjldNzNQB/SZlbQ/qRjb3Xl7tKTdbB9fSodM7dVT80X/wJIy+9HjGqX6RA2XwC01hFI2fo8fBc61/YIK167KQRbfLzkjqtJhORed0twG29MljcwRiKfmoIuWGoOumxmoA/rMdtAdR23VU9VhPakl7UKqg3M1PaUHvm77DwV1+CGpQuirpJIDiwEX2d6olA5Zj+uALWy/XlJukw6151xImgQ8T+/mNovb/mwpHbIe15J6KpwMPEVa1/psq2eMXTczoDP6zI6UpMoXfST1dBqrs3rqTwY/pCjfBjZozAYkjSGF+xYzBrYPyesGL+TIt1dI4ZWleRi4PgdaVIMsirVDpcaSGBW+RGpu8/u8PZlkEEqzB2md4ADgy6RCdS2vY9aNxqAT+sxeCvxeqWIowH55rDS1VU91T0/q7YH/aST21MiIJrfQsxTKw5H0Dds/zptb2j4XZpf3/jZQOuP1ofwYQW58VAN1lsQAeprbZPep60qIrKyTvEpaX2sL3egmWpkU274xKfv4EWD3wgtTI0gGYMs8NBk4uZGRXBeqoXqqpN+SFirPAybZvr+U7CY9/osUvVNtIHKX7W8UkD07g7Q5m7RkFngfei1YV1Z8U0mMRs5F0ZIYkt5LqlaweB76K6laQUuid4YgvxFZ1SetjqzqRmMw1vYjqvSZbYwV1mNeUjSRqSmaqBklX9E9ttcoLHdRUvjiXqT/x6nAWS7cA1jSJ4FN8uafbZ8/0PEtlDt7vah57aimtaT3k/o4LGx7BaUSLvvZ/mJJPepG0g3At5sSIr9vu0hCZOnIqm50E51HWpiqprb/AVi/lAL5S3U68ChpsXJ5pXLR15bSIevREdVTbb8o6Q+kshwHAx8Hvi7pGNvHFtTjPKVWh6MAJC1u+7kSovt53dd2CX4ObE2q4ortv0j64IBntIiGy6zpuzkb2yUjz2pNiCzprYAuMgaSVgfWBN6h3rXSFyX5y0tyNLCV7QeybquS3BPFDFJmauX1LNLdeNHqqZJ2IM0I3k2akm9oe6ZSeY57SfHuJfTYj+SP/QcpvFaki9HKBcSvI+nFLHOB/Jq8Xfq7CYDtx5uCCkq5MO/Lz1MHPKoMtSZESnqJHoPY+DBMTwLeoq2U1zXGgOSS+RgpXK9aK/0lYJ/CuszTMAQAtv9fTkIrzR9oqp5ag5/4k8DPmmdFtl+RtHdBPb4GrOUampfYHlla5iA8rlTB1fl7eRA9F+m2YvtPObruvba/VkLmAHyOdIPQqE7w5zxWBNtFF++7cc3g/bZvrFmHSaS7z0aZgd2BkbaLfdGyHjcBH25ESSiVkb68lE+0ose7SJFMJtWLL17/RdKlwCfqWjDtJCQtCfyCFNsuUnmSAwu5zBo6tKRhy78KSgU2x9k+NX8+i7R6nbMbjcHpwEG2n8/bo4GjS16IlWq170/vEgzH236t/7PaokdfyW9F+v5W5O1N6kl9JenC8yFST+pJpXTIeqxHWri+md61iWrPji6NpE2a3YV9jbVZhxOAZamhoGQHJkQeRipDsZrtVSUtQ8pQ32SQU98W3eQmarB2wxBAKoeQLwQlGQX8opHEk6fF8xXWATqjeuo3gPVsP5t1WIJUC6eoMQBOJBmkuyhckqMDORZoDmfta6yd1FlQstMSIj9Oqih8O4Dt/8u5Dy2lG43BCEmjnQuhSVqc8v+HKaQpeCOJZQHSVLx0Df+Dqb966rOkdZsGL1GoQmYT89j+yuCH/euSQ0o3Bsaop0c1pCCLousatvcqKa9JdqclRL5u25IaibJtiWjqRmNwNHCjpHNJF8CdgaMK69ARzW1s35qjrOqsnjoduFnSBaQ7vx2BOxsXo4IlEC6RtC/wJ3q7iYr5yTuAeYGFSdeF6p3ni6TfSTHUAQUlSTdGP5dUa0IkcE6uVrCYpH1Ii9i/brWQrlszAFBqq7h53rzS9r2F5V8PfKnJPfPL0gtm6qme+nzeHg0UrZ6a/aH9Yrtt6fdNevS1GGfbJUJLOwpJK5aOce9Dh9oLSmaZnZIQ+RFgK9IN7GW2J7dcRjcaA5id7j47htsFarVXZG8AnA30cs/Yvq2UDlmPjqieGnQW+a78a8zZjL7tNasqOtxqe4Om7OyiwQ0VXZYgFYs7mBRi+26gaEJkCbrOTZSTnI4GlgFmAiuSPuBiTdg7xD0DHVA9VdJ4UsXQFel94SnS0aopAXEOSkSvdCDnAr8ilUyuq15W7QUl606IlHSd7U2bks8gks5axpHABOAK2+tJ2pyUWViMnMjz76QG2wBXSzqxBoPQCdVTzyS5A+qK4tl+gH2l26F2CrNsnzD4YW1lf1JBydUlPUEqKFn0d0rNCZG2N83PRZLPus5NJGmq7fGS/kIKaXxL0l9csLWgpJOBeUj1iSBNQd+0/flSOmQ9aq+e2rj7KSUvGBxJh5NmzedT82K6KgUlS8vuFPIMaYbt13Jds7WBM6oh8i2R04XG4ApSN6kfkJqgzyQ1NSkW1tmX8SltkCpya62eKmlL0gLdFHpfeIrfkUv6KMldWF1LOqK0HnXTCYvpOTHzk8y5blHs88guxB8B7yS5ZtrinhmCHtNISWcrARcDFwBr2t6ulXK60U20Iymx6sukMhDvAEr/4N+UtIrth4BGj4Xivll1RvXUvYDVSTOlhpuouHtG0q+ABUlRZieTQilvKalDp2B7bN06kC54L5CaLRXNzK/wY2B720XqMg3AW7ZnSfo4cKztY5XagrYW2137IM0MVIPcLYH/JfXbvYZ0Md6iBj1uI6W4N7ZXBW4rrMMDdX8Psh53Nj0vTOppULtuBf8H36i8/lTTvu8X1uXuDvh/XF+3DlmPm0mz57uBse36/xRp69cJSJog6WpJf5S0nqS7Sf/cpyVtU1id64BxwIGkPqurAUVLR2fmqJ5KukMvyQ2SijbT6YdGGY5Xcu2XWcDSNepTB7tWXh/atK/0b+QGpU5jxZH0iewimirp95J2a4wNFn3WJvYidQM8yqkx11h6ymq3jG5yE/2S1Ev2HaQaNNvavimHeJ5F2SiaG51aGd7ZGJB0O2Vrv0D6sp9M7+qppevITwCmZT/1a/T4ZYuElla4SNJiJNdAI9/j5MI61I36ed3XdnsU6Gn1OArYS9LDlP9eVCPMXiElezUo7sK0fa+kr5Eiq95Lmk3/qNVyuskYjLJ9OYCkI2zfBGD7fqnI97xRqnlZUgOT9ej5gS1K8leX5t9JIXyNypx/BoplH2dK33H2IicAPm77yLy9MCnM9X7gZ3XqVgOd0HHtY4Xk9ItrrIvUFzmw4VfAQ6RrxlhJ+9m+pKVysv/pXx51QNNxSROBz5IiA6p34C8Bp7lwBE0O2+vV3AaYzzXU9K8rIzzPyD5s+zml1o5nk1x36wLvsV20Jk+dSHqTVC5apOKJje+BSPW0irkQJU0g9eN+KW8vSvo8bi6ow8qkvg4TSMbwRuBgl++Xfj/wMdvT8/YqpAJ6q7dUThcZg076on/S9nml5A2gR+3NbfrLCLddJCO8GtIr6TjgGduH5+1prqH8QZDKopB6lTcykEcAU0vctFV0uAk4juRGhrSm8iXbG5XSIetxq+0NKtsCbqmOtYKucRO5s1oLXi3pGFJzG5MWlI9wrulfkE6onlp3RvhISaNszyJFee1b2dc1v48OZHaZFACn5NDSn8eCtqsLtb+V9PVSwiuL1VMlXQycQ7pefAq4tdXyuiaaqMM4G3iGlFSzc379+xr0eFlS1V1WR3ObN7IRHCFphO2rSG60UpwFXKNUQvtV0roJkt5NinMP6uFhSQdKmic/DqJgM/rMJZIOkbSSpBUlfQO4WNLiSn1Q2s32+TE/8DSpC+BmpOvF/P2fNnd0jZuok1AfpXgl3WW7aCidOqB6aodkhE8ghZFebvvlPLYqsLBzmfGgLHkN6RhSpzOTMtQPtj2zoA4DrQ3Y/2LlzcMY1ICkn5KyW8/JQzuTKiJ+rQZd5qGG6qn5znspYBrpjnwEKbR1RdLiWNFy3kHQqahQo58wBjWgVJJ2IVL5BZNaCjaaftuFap+oj+qppGYibTcIki4CDrV9V9P4e0nZrgNVEw3+xZH0Y+B7pBuFS0nF2b5s+7cDnth6PdYC1qB3pNsZhXUo0ugn1gxqwPYitkfYHmV7nvx6kfwoWQTrBGB9Um7B8fl1qdLFSzUbAoA8tlIhHYLOZSvbL5LyDh4l9RQotngLNLrwHZsfm5MSEncoqUNmQdvNdbJmtVpIREvUQA4N251UZ+RIScsDS/fxgbebDdy7UuqVSqW9S7DYAPsWKKRD0Lk0rk0fBc61/UKp5NAKOwPrAHfY3kvSUvRk65ekSKOfmBnUw/GkWiP/lrf/TopnLs2b+UsGFK+eOlWpuXcvJH2ennIQQfdyUU62Wh+YImkM8I/COrxq+y1gVk56mwksX1gHSFUCTqSn0c/BJPduS4k1gxpoZDyrd3/X4v0MlHoJnEoK2RNp8fZztq8sIHspUvOU1+m5+I8ntd38uO2n2q1D0Nnk8M0XbL+Zs+UXKfm9kHQ8qZ7ZrsBXSTdt0+oqV6E2N/oJY1ADkm4GNgZuzUZhDCmssWgjeqUGIlCJJgKwXax+fE4yayyE3VPCEAWdi6Rv2P5xfv0p2+dW9n3f9rdq0mslYFHbdw52bAtlfmWg/bZ/2lJ5YQzKI2l34NOkKqWnk3yT/1H94hfSY46aTKXqNAVBX3RCDbGKvCm2txxsrI3yDxtgt93irm+xgFyYXGPlEeAbpPIHAnZywW5K6rzqqUHQoBPKaM9P+h0sKWk0vX8fy5bQITPJ9uN97ZDU8uquYQwKk2usHJddQvfXpMbWpOqpywHVqeZLJB9pENRFJ5TR3o+0SLsMvYMZXiL1RSnFZEnb2H60OihpL+A/gItaKSzcRDUg6Sekcrh/dI0fQKdUTw2CBp1QXTiXaZkB7OzUb3giqY7Yo8Dhtp9rtw5Zj+2AnwMftf1gHjuUFIW4re0ZLZUXxqA8lQzkWaRwuUYXp5IJZ0haAjiM+qunBkHHoA7qcZEj/k4k1e/6PLAhyTj8reWywhh0L5ImA9fSu+3lZrY/XJ9WQVAvndbjQtIHSGHYNwC72G5LvkUkndWApClDGSvA0raPtP1IfnyPVDwuCLqZkZXeCVuSeqY3KLbOKuklSS8CF5MWr7cEZlbGW0osIBekg6IUGlwuaVd6V0+9rAY9gqCTaPS4+Cs19riwvUgpWRBuoqLkBh0Hk6IUnqjsegn4te2SkQodUz01CDqNbuxxEcagIJ0SpRAEQdBMrBmU5UTgtWwIPkjq7nU6aep5UmlllPiMpO/k7eUlbVhajyAI6idmBgXpwCiFE0guoi1svyevY1xue4OSegRBUD8xMyhLR0QpVNjI9v7k0sA5dnneGvQIgqBmIpqoLB0RpVDhDUkj6WmaMYY0UwiCoMsIN1FhOilKoVOqpwZBUD9hDLqUXD11AvAcPdVTp5SsnhoEQecQxqCLqXZaC4Kgu4kF5O5miqRPqoZO40EQdBYxM+hiOqV6ahAE9RPGIAiCIAg3UTfTQdVTgyComcgz6EI6sHpqEAQ1E8agO+mUHq9BEHQI4SbqTm4ANga+Zntl4LvA3cA1wO/qVCwIgnqIBeQupJN6vAZB0BmEm6g7GVnpnfBp4CTb5wHnSZpWn1pBENRFuIm6k06rnhoEQc3ED7876bTqqUEQ1EysGXQpnVQ9NQiC+gljEARBEMSaQRAEQRDGIAiCICCMQRAEQUAYgyAIgoAwBkEQBAHw/wF7llCyIWfrbQAAAABJRU5ErkJggg==\n",
      "text/plain": [
       "<Figure size 432x288 with 1 Axes>"
      ]
     },
     "metadata": {
      "needs_background": "light"
     },
     "output_type": "display_data"
    }
   ],
   "source": [
    "# Plot the distribution of values for the target column\n",
    "df[\"target\"].value_counts().plot(kind=\"bar\");"
   ]
  },
  {
   "cell_type": "markdown",
   "id": "e192f64b",
   "metadata": {
    "papermill": {
     "duration": 0.091079,
     "end_time": "2022-03-16T15:11:20.547367",
     "exception": false,
     "start_time": "2022-03-16T15:11:20.456288",
     "status": "completed"
    },
    "tags": []
   },
   "source": [
    "The 10 different classes in the dataset are balanced."
   ]
  },
  {
   "cell_type": "markdown",
   "id": "62d98312",
   "metadata": {
    "papermill": {
     "duration": 0.088424,
     "end_time": "2022-03-16T15:11:20.724210",
     "exception": false,
     "start_time": "2022-03-16T15:11:20.635786",
     "status": "completed"
    },
    "tags": []
   },
   "source": [
    "## **Target Labels Encoding**"
   ]
  },
  {
   "cell_type": "markdown",
   "id": "9656e7ca",
   "metadata": {
    "papermill": {
     "duration": 0.090916,
     "end_time": "2022-03-16T15:11:20.903423",
     "exception": false,
     "start_time": "2022-03-16T15:11:20.812507",
     "status": "completed"
    },
    "tags": []
   },
   "source": [
    "We start the preprocessing of the dataset by encoding the labels of the target class.\n",
    "For this purpose we will use the LabelEncoder provided by sklearn, which will map each textual label into a number between 0 and 9."
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 13,
   "id": "68e1ce9f",
   "metadata": {
    "execution": {
     "iopub.execute_input": "2022-03-16T15:11:21.085957Z",
     "iopub.status.busy": "2022-03-16T15:11:21.084334Z",
     "iopub.status.idle": "2022-03-16T15:11:22.104476Z",
     "shell.execute_reply": "2022-03-16T15:11:22.103412Z",
     "shell.execute_reply.started": "2022-03-14T13:22:19.703789Z"
    },
    "papermill": {
     "duration": 1.112553,
     "end_time": "2022-03-16T15:11:22.104736",
     "exception": false,
     "start_time": "2022-03-16T15:11:20.992183",
     "status": "completed"
    },
    "tags": []
   },
   "outputs": [],
   "source": [
    "from sklearn.preprocessing import LabelEncoder"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 14,
   "id": "d742fb29",
   "metadata": {
    "execution": {
     "iopub.execute_input": "2022-03-16T15:11:22.422113Z",
     "iopub.status.busy": "2022-03-16T15:11:22.421421Z",
     "iopub.status.idle": "2022-03-16T15:11:22.521979Z",
     "shell.execute_reply": "2022-03-16T15:11:22.522475Z",
     "shell.execute_reply.started": "2022-03-14T13:22:20.55012Z"
    },
    "papermill": {
     "duration": 0.274138,
     "end_time": "2022-03-16T15:11:22.522652",
     "exception": false,
     "start_time": "2022-03-16T15:11:22.248514",
     "status": "completed"
    },
    "tags": []
   },
   "outputs": [
    {
     "data": {
      "text/html": [
       "<div>\n",
       "<style scoped>\n",
       "    .dataframe tbody tr th:only-of-type {\n",
       "        vertical-align: middle;\n",
       "    }\n",
       "\n",
       "    .dataframe tbody tr th {\n",
       "        vertical-align: top;\n",
       "    }\n",
       "\n",
       "    .dataframe thead th {\n",
       "        text-align: right;\n",
       "    }\n",
       "</style>\n",
       "<table border=\"1\" class=\"dataframe\">\n",
       "  <thead>\n",
       "    <tr style=\"text-align: right;\">\n",
       "      <th></th>\n",
       "      <th>A0T0G0C10</th>\n",
       "      <th>A0T0G1C9</th>\n",
       "      <th>A0T0G2C8</th>\n",
       "      <th>A0T0G3C7</th>\n",
       "      <th>A0T0G4C6</th>\n",
       "      <th>A0T0G5C5</th>\n",
       "      <th>A0T0G6C4</th>\n",
       "      <th>A0T0G7C3</th>\n",
       "      <th>A0T0G8C2</th>\n",
       "      <th>A0T0G9C1</th>\n",
       "      <th>...</th>\n",
       "      <th>A8T0G1C1</th>\n",
       "      <th>A8T0G2C0</th>\n",
       "      <th>A8T1G0C1</th>\n",
       "      <th>A8T1G1C0</th>\n",
       "      <th>A8T2G0C0</th>\n",
       "      <th>A9T0G0C1</th>\n",
       "      <th>A9T0G1C0</th>\n",
       "      <th>A9T1G0C0</th>\n",
       "      <th>A10T0G0C0</th>\n",
       "      <th>target</th>\n",
       "    </tr>\n",
       "  </thead>\n",
       "  <tbody>\n",
       "    <tr>\n",
       "      <th>0</th>\n",
       "      <td>-9.536743e-07</td>\n",
       "      <td>-0.000010</td>\n",
       "      <td>-0.000043</td>\n",
       "      <td>-0.000114</td>\n",
       "      <td>-0.000200</td>\n",
       "      <td>-0.000240</td>\n",
       "      <td>-0.000200</td>\n",
       "      <td>-0.000114</td>\n",
       "      <td>-0.000043</td>\n",
       "      <td>-0.000010</td>\n",
       "      <td>...</td>\n",
       "      <td>-0.000086</td>\n",
       "      <td>-0.000043</td>\n",
       "      <td>-0.000086</td>\n",
       "      <td>-0.000086</td>\n",
       "      <td>-0.000043</td>\n",
       "      <td>-0.000010</td>\n",
       "      <td>-0.000010</td>\n",
       "      <td>-0.000010</td>\n",
       "      <td>-9.536743e-07</td>\n",
       "      <td>9</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>1</th>\n",
       "      <td>-9.536743e-07</td>\n",
       "      <td>-0.000010</td>\n",
       "      <td>-0.000043</td>\n",
       "      <td>0.000886</td>\n",
       "      <td>-0.000200</td>\n",
       "      <td>0.000760</td>\n",
       "      <td>-0.000200</td>\n",
       "      <td>-0.000114</td>\n",
       "      <td>-0.000043</td>\n",
       "      <td>-0.000010</td>\n",
       "      <td>...</td>\n",
       "      <td>-0.000086</td>\n",
       "      <td>-0.000043</td>\n",
       "      <td>0.000914</td>\n",
       "      <td>0.000914</td>\n",
       "      <td>-0.000043</td>\n",
       "      <td>-0.000010</td>\n",
       "      <td>-0.000010</td>\n",
       "      <td>-0.000010</td>\n",
       "      <td>-9.536743e-07</td>\n",
       "      <td>6</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>2</th>\n",
       "      <td>-9.536743e-07</td>\n",
       "      <td>-0.000002</td>\n",
       "      <td>0.000007</td>\n",
       "      <td>0.000129</td>\n",
       "      <td>0.000268</td>\n",
       "      <td>0.000270</td>\n",
       "      <td>0.000243</td>\n",
       "      <td>0.000125</td>\n",
       "      <td>0.000001</td>\n",
       "      <td>-0.000007</td>\n",
       "      <td>...</td>\n",
       "      <td>0.000084</td>\n",
       "      <td>0.000048</td>\n",
       "      <td>0.000081</td>\n",
       "      <td>0.000106</td>\n",
       "      <td>0.000072</td>\n",
       "      <td>0.000010</td>\n",
       "      <td>0.000008</td>\n",
       "      <td>0.000019</td>\n",
       "      <td>1.046326e-06</td>\n",
       "      <td>6</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>3</th>\n",
       "      <td>4.632568e-08</td>\n",
       "      <td>-0.000006</td>\n",
       "      <td>0.000012</td>\n",
       "      <td>0.000245</td>\n",
       "      <td>0.000492</td>\n",
       "      <td>0.000522</td>\n",
       "      <td>0.000396</td>\n",
       "      <td>0.000197</td>\n",
       "      <td>-0.000003</td>\n",
       "      <td>-0.000007</td>\n",
       "      <td>...</td>\n",
       "      <td>0.000151</td>\n",
       "      <td>0.000100</td>\n",
       "      <td>0.000180</td>\n",
       "      <td>0.000202</td>\n",
       "      <td>0.000153</td>\n",
       "      <td>0.000021</td>\n",
       "      <td>0.000015</td>\n",
       "      <td>0.000046</td>\n",
       "      <td>-9.536743e-07</td>\n",
       "      <td>6</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>4</th>\n",
       "      <td>-9.536743e-07</td>\n",
       "      <td>-0.000010</td>\n",
       "      <td>-0.000043</td>\n",
       "      <td>-0.000114</td>\n",
       "      <td>-0.000200</td>\n",
       "      <td>-0.000240</td>\n",
       "      <td>-0.000200</td>\n",
       "      <td>-0.000114</td>\n",
       "      <td>-0.000043</td>\n",
       "      <td>-0.000010</td>\n",
       "      <td>...</td>\n",
       "      <td>-0.000086</td>\n",
       "      <td>-0.000043</td>\n",
       "      <td>-0.000086</td>\n",
       "      <td>-0.000086</td>\n",
       "      <td>-0.000043</td>\n",
       "      <td>-0.000010</td>\n",
       "      <td>-0.000010</td>\n",
       "      <td>-0.000010</td>\n",
       "      <td>-9.536743e-07</td>\n",
       "      <td>2</td>\n",
       "    </tr>\n",
       "  </tbody>\n",
       "</table>\n",
       "<p>5 rows × 287 columns</p>\n",
       "</div>"
      ],
      "text/plain": [
       "      A0T0G0C10  A0T0G1C9  A0T0G2C8  A0T0G3C7  A0T0G4C6  A0T0G5C5  A0T0G6C4  \\\n",
       "0 -9.536743e-07 -0.000010 -0.000043 -0.000114 -0.000200 -0.000240 -0.000200   \n",
       "1 -9.536743e-07 -0.000010 -0.000043  0.000886 -0.000200  0.000760 -0.000200   \n",
       "2 -9.536743e-07 -0.000002  0.000007  0.000129  0.000268  0.000270  0.000243   \n",
       "3  4.632568e-08 -0.000006  0.000012  0.000245  0.000492  0.000522  0.000396   \n",
       "4 -9.536743e-07 -0.000010 -0.000043 -0.000114 -0.000200 -0.000240 -0.000200   \n",
       "\n",
       "   A0T0G7C3  A0T0G8C2  A0T0G9C1  ...  A8T0G1C1  A8T0G2C0  A8T1G0C1  A8T1G1C0  \\\n",
       "0 -0.000114 -0.000043 -0.000010  ... -0.000086 -0.000043 -0.000086 -0.000086   \n",
       "1 -0.000114 -0.000043 -0.000010  ... -0.000086 -0.000043  0.000914  0.000914   \n",
       "2  0.000125  0.000001 -0.000007  ...  0.000084  0.000048  0.000081  0.000106   \n",
       "3  0.000197 -0.000003 -0.000007  ...  0.000151  0.000100  0.000180  0.000202   \n",
       "4 -0.000114 -0.000043 -0.000010  ... -0.000086 -0.000043 -0.000086 -0.000086   \n",
       "\n",
       "   A8T2G0C0  A9T0G0C1  A9T0G1C0  A9T1G0C0     A10T0G0C0  target  \n",
       "0 -0.000043 -0.000010 -0.000010 -0.000010 -9.536743e-07       9  \n",
       "1 -0.000043 -0.000010 -0.000010 -0.000010 -9.536743e-07       6  \n",
       "2  0.000072  0.000010  0.000008  0.000019  1.046326e-06       6  \n",
       "3  0.000153  0.000021  0.000015  0.000046 -9.536743e-07       6  \n",
       "4 -0.000043 -0.000010 -0.000010 -0.000010 -9.536743e-07       2  \n",
       "\n",
       "[5 rows x 287 columns]"
      ]
     },
     "execution_count": 14,
     "metadata": {},
     "output_type": "execute_result"
    }
   ],
   "source": [
    "# Label encoding of the target.\n",
    "# We will map each of the 10 different target classess to an intger in the range 0-9\n",
    "label_encoder = LabelEncoder()\n",
    "df[\"target\"] = label_encoder.fit(df[\"target\"]).transform(df[\"target\"])\n",
    "df.head()"
   ]
  },
  {
   "cell_type": "markdown",
   "id": "c13e850e",
   "metadata": {
    "papermill": {
     "duration": 0.086657,
     "end_time": "2022-03-16T15:11:22.696791",
     "exception": false,
     "start_time": "2022-03-16T15:11:22.610134",
     "status": "completed"
    },
    "tags": []
   },
   "source": [
    "## **Train-Test Split**"
   ]
  },
  {
   "cell_type": "markdown",
   "id": "e2f27fd3",
   "metadata": {
    "papermill": {
     "duration": 0.086517,
     "end_time": "2022-03-16T15:11:22.870898",
     "exception": false,
     "start_time": "2022-03-16T15:11:22.784381",
     "status": "completed"
    },
    "tags": []
   },
   "source": [
    "Altrought Kaggle provides separate training and test set for this task, we don't have access to the full test set.\n",
    "For this reason for now we will assume to have only the data in train.csv, and we will split it in two sets for training and testing.\n",
    "The test set will be equal to 1/3 of the initial dataset."
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 15,
   "id": "a96b5da1",
   "metadata": {
    "execution": {
     "iopub.execute_input": "2022-03-16T15:11:23.052279Z",
     "iopub.status.busy": "2022-03-16T15:11:23.051517Z",
     "iopub.status.idle": "2022-03-16T15:11:23.102149Z",
     "shell.execute_reply": "2022-03-16T15:11:23.102814Z",
     "shell.execute_reply.started": "2022-03-14T13:22:20.617574Z"
    },
    "papermill": {
     "duration": 0.143932,
     "end_time": "2022-03-16T15:11:23.103010",
     "exception": false,
     "start_time": "2022-03-16T15:11:22.959078",
     "status": "completed"
    },
    "tags": []
   },
   "outputs": [],
   "source": [
    "# Imports\n",
    "from sklearn.model_selection import train_test_split"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 16,
   "id": "06a27d60",
   "metadata": {
    "execution": {
     "iopub.execute_input": "2022-03-16T15:11:23.282160Z",
     "iopub.status.busy": "2022-03-16T15:11:23.281482Z",
     "iopub.status.idle": "2022-03-16T15:11:24.148634Z",
     "shell.execute_reply": "2022-03-16T15:11:24.148034Z",
     "shell.execute_reply.started": "2022-03-14T13:22:20.669491Z"
    },
    "papermill": {
     "duration": 0.958147,
     "end_time": "2022-03-16T15:11:24.148896",
     "exception": false,
     "start_time": "2022-03-16T15:11:23.190749",
     "status": "completed"
    },
    "tags": []
   },
   "outputs": [],
   "source": [
    "X_train, X_test, Y_train, Y_test = train_test_split(df.drop(columns=[\"target\"]), df[\"target\"], test_size=0.33, stratify=df[\"target\"], random_state=random_seed)"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 17,
   "id": "206d65e2",
   "metadata": {
    "execution": {
     "iopub.execute_input": "2022-03-16T15:11:24.332709Z",
     "iopub.status.busy": "2022-03-16T15:11:24.331941Z",
     "iopub.status.idle": "2022-03-16T15:11:24.334998Z",
     "shell.execute_reply": "2022-03-16T15:11:24.335491Z",
     "shell.execute_reply.started": "2022-03-14T13:22:22.47162Z"
    },
    "papermill": {
     "duration": 0.097451,
     "end_time": "2022-03-16T15:11:24.335655",
     "exception": false,
     "start_time": "2022-03-16T15:11:24.238204",
     "status": "completed"
    },
    "tags": []
   },
   "outputs": [
    {
     "name": "stdout",
     "output_type": "stream",
     "text": [
      "X_train shape: (134000, 286)\n",
      "Y_train shape: (134000,)\n",
      "X_test shape: (66000, 286)\n",
      "Y_test shape: (66000,)\n"
     ]
    }
   ],
   "source": [
    "print(f\"X_train shape: {X_train.shape}\")\n",
    "print(f\"Y_train shape: {Y_train.shape}\")\n",
    "print(f\"X_test shape: {X_test.shape}\")\n",
    "print(f\"Y_test shape: {Y_test.shape}\")"
   ]
  },
  {
   "cell_type": "markdown",
   "id": "fe570f38",
   "metadata": {
    "papermill": {
     "duration": 0.088789,
     "end_time": "2022-03-16T15:11:24.514396",
     "exception": false,
     "start_time": "2022-03-16T15:11:24.425607",
     "status": "completed"
    },
    "tags": []
   },
   "source": [
    "## **Utility Functions Definition**"
   ]
  },
  {
   "cell_type": "markdown",
   "id": "84b0305d",
   "metadata": {
    "papermill": {
     "duration": 0.087368,
     "end_time": "2022-03-16T15:11:24.690403",
     "exception": false,
     "start_time": "2022-03-16T15:11:24.603035",
     "status": "completed"
    },
    "tags": []
   },
   "source": [
    "Now we define some utility functions that we will use later."
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 18,
   "id": "70d63e70",
   "metadata": {
    "execution": {
     "iopub.execute_input": "2022-03-16T15:11:24.870414Z",
     "iopub.status.busy": "2022-03-16T15:11:24.869787Z",
     "iopub.status.idle": "2022-03-16T15:11:24.977334Z",
     "shell.execute_reply": "2022-03-16T15:11:24.977934Z",
     "shell.execute_reply.started": "2022-03-14T13:22:22.481709Z"
    },
    "papermill": {
     "duration": 0.199442,
     "end_time": "2022-03-16T15:11:24.978118",
     "exception": false,
     "start_time": "2022-03-16T15:11:24.778676",
     "status": "completed"
    },
    "tags": []
   },
   "outputs": [],
   "source": [
    "from sklearn.preprocessing import StandardScaler\n",
    "from sklearn.decomposition import PCA\n",
    "from sklearn.pipeline import Pipeline"
   ]
  },
  {
   "cell_type": "markdown",
   "id": "42b09a15",
   "metadata": {
    "papermill": {
     "duration": 0.091797,
     "end_time": "2022-03-16T15:11:25.159158",
     "exception": false,
     "start_time": "2022-03-16T15:11:25.067361",
     "status": "completed"
    },
    "tags": []
   },
   "source": [
    "We will use cross validation to test several models with different hyper-parameters, each time using the function **print_grid_search_results** to print the results of the search."
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 19,
   "id": "3f4a274e",
   "metadata": {
    "execution": {
     "iopub.execute_input": "2022-03-16T15:11:25.339047Z",
     "iopub.status.busy": "2022-03-16T15:11:25.338347Z",
     "iopub.status.idle": "2022-03-16T15:11:25.344257Z",
     "shell.execute_reply": "2022-03-16T15:11:25.343702Z",
     "shell.execute_reply.started": "2022-03-14T13:22:22.61104Z"
    },
    "papermill": {
     "duration": 0.097479,
     "end_time": "2022-03-16T15:11:25.344407",
     "exception": false,
     "start_time": "2022-03-16T15:11:25.246928",
     "status": "completed"
    },
    "tags": []
   },
   "outputs": [],
   "source": [
    "# Print grid search results.\n",
    "# Input: GridSearchCV object\n",
    "def print_grid_search_results(gscv):\n",
    "    print(\"Best parameters set found on train set:\")\n",
    "    print(gscv.best_params_)\n",
    "    print(\"\\n\")\n",
    "    print(\"Grid scores on train set:\")\n",
    "    for param, mean, std in zip(gscv.cv_results_['params'], gscv.cv_results_['mean_test_score'], gscv.cv_results_['std_test_score']):\n",
    "        print(\"%r mean: %0.3f std: +/- %0.03f)\" % (param, mean, std))\n",
    "    print(\"\\n\")"
   ]
  },
  {
   "cell_type": "markdown",
   "id": "fd0bdf66",
   "metadata": {
    "papermill": {
     "duration": 0.090849,
     "end_time": "2022-03-16T15:11:25.525565",
     "exception": false,
     "start_time": "2022-03-16T15:11:25.434716",
     "status": "completed"
    },
    "tags": []
   },
   "source": [
    "We will also use a simple plot function based on matplotlib.pyplot"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 20,
   "id": "0fa230d7",
   "metadata": {
    "execution": {
     "iopub.execute_input": "2022-03-16T15:11:25.709364Z",
     "iopub.status.busy": "2022-03-16T15:11:25.708274Z",
     "iopub.status.idle": "2022-03-16T15:11:25.710182Z",
     "shell.execute_reply": "2022-03-16T15:11:25.710697Z",
     "shell.execute_reply.started": "2022-03-14T13:22:22.618229Z"
    },
    "papermill": {
     "duration": 0.096655,
     "end_time": "2022-03-16T15:11:25.710882",
     "exception": false,
     "start_time": "2022-03-16T15:11:25.614227",
     "status": "completed"
    },
    "tags": []
   },
   "outputs": [],
   "source": [
    "# Show x-y curve plot\n",
    "def show_curve(x, y, x_label, y_label, title):\n",
    "    plt.figure(figsize=(10,5))\n",
    "    plt.plot(x, y, '-o', linewidth=5, markersize=10)\n",
    "    plt.xlabel(x_label)\n",
    "    plt.ylabel(y_label)\n",
    "    plt.title(title, fontsize = 12)\n",
    "    plt.show();"
   ]
  },
  {
   "cell_type": "markdown",
   "id": "b308cf6a",
   "metadata": {
    "papermill": {
     "duration": 0.089438,
     "end_time": "2022-03-16T15:11:25.888915",
     "exception": false,
     "start_time": "2022-03-16T15:11:25.799477",
     "status": "completed"
    },
    "tags": []
   },
   "source": [
    "Finally we have the function **build_PCA_pipeline** which will be used to construct a ML pipeline given a specific classifier to use and the number of output components of the PCA performed at beginning of the pipeline, in order to reduce the dimensions of the input data.\n",
    "\n",
    "Each pipeline will comprehend 3 stages:\n",
    "1. Standardization of the input data, by removing for each feature the mean and scaling to unit variance. (Required before doing PCA)\n",
    "2. Principal Component Analysis (PCA)\n",
    "3. Classification of the PCA-transformed data by a specific classifier"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 21,
   "id": "608f9377",
   "metadata": {
    "execution": {
     "iopub.execute_input": "2022-03-16T15:11:26.072511Z",
     "iopub.status.busy": "2022-03-16T15:11:26.071557Z",
     "iopub.status.idle": "2022-03-16T15:11:26.076570Z",
     "shell.execute_reply": "2022-03-16T15:11:26.077064Z",
     "shell.execute_reply.started": "2022-03-14T13:22:22.632402Z"
    },
    "papermill": {
     "duration": 0.098025,
     "end_time": "2022-03-16T15:11:26.077265",
     "exception": false,
     "start_time": "2022-03-16T15:11:25.979240",
     "status": "completed"
    },
    "tags": []
   },
   "outputs": [],
   "source": [
    "# Construct a ML pipeline with initial PCA\n",
    "# Inputs:\n",
    "#  - classifier to use as the last step of the pipeline\n",
    "#  - number of output dimensions of the PCA transformation\n",
    "# Output:\n",
    "# - The constructed pipeline\n",
    "def build_PCA_pipeline(classifier, pca_n_components=None):\n",
    "    st_scaler = StandardScaler()\n",
    "    pca = PCA(n_components=pca_n_components, random_state=random_seed)\n",
    "    pipe = Pipeline(steps=[(\"st_scaler\", st_scaler), (\"pca\", pca), (\"classifier\", classifier)])\n",
    "    return pipe"
   ]
  },
  {
   "cell_type": "markdown",
   "id": "c271517a",
   "metadata": {
    "papermill": {
     "duration": 0.101614,
     "end_time": "2022-03-16T15:11:26.283687",
     "exception": false,
     "start_time": "2022-03-16T15:11:26.182073",
     "status": "completed"
    },
    "tags": []
   },
   "source": [
    "## **Dimensionality Reduction** "
   ]
  },
  {
   "cell_type": "markdown",
   "id": "b3bf6cb3",
   "metadata": {
    "papermill": {
     "duration": 0.089527,
     "end_time": "2022-03-16T15:11:26.462010",
     "exception": false,
     "start_time": "2022-03-16T15:11:26.372483",
     "status": "completed"
    },
    "tags": []
   },
   "source": [
    "### **Grid Search with Cross Validation to find the best value for n_components of PCA**"
   ]
  },
  {
   "cell_type": "markdown",
   "id": "c18250d8",
   "metadata": {
    "papermill": {
     "duration": 0.088381,
     "end_time": "2022-03-16T15:11:26.638989",
     "exception": false,
     "start_time": "2022-03-16T15:11:26.550608",
     "status": "completed"
    },
    "tags": []
   },
   "source": [
    "We will use PCA in order to reduce the dimensionality of the data.\n",
    "We will use cross validation to find the best number of output dimensions from the PCA, by testing for each possible dimension the performance of a simple decision tree classifier, which will be used as an indicator."
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 22,
   "id": "83fcc5d0",
   "metadata": {
    "execution": {
     "iopub.execute_input": "2022-03-16T15:11:26.818992Z",
     "iopub.status.busy": "2022-03-16T15:11:26.818391Z",
     "iopub.status.idle": "2022-03-16T15:11:26.878118Z",
     "shell.execute_reply": "2022-03-16T15:11:26.878652Z",
     "shell.execute_reply.started": "2022-03-14T13:22:22.643374Z"
    },
    "papermill": {
     "duration": 0.151187,
     "end_time": "2022-03-16T15:11:26.878865",
     "exception": false,
     "start_time": "2022-03-16T15:11:26.727678",
     "status": "completed"
    },
    "tags": []
   },
   "outputs": [],
   "source": [
    "from sklearn.tree import DecisionTreeClassifier\n",
    "from sklearn.model_selection import GridSearchCV"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 23,
   "id": "8c63f725",
   "metadata": {
    "execution": {
     "iopub.execute_input": "2022-03-16T15:11:27.058175Z",
     "iopub.status.busy": "2022-03-16T15:11:27.057561Z",
     "iopub.status.idle": "2022-03-16T15:11:27.061324Z",
     "shell.execute_reply": "2022-03-16T15:11:27.061847Z",
     "shell.execute_reply.started": "2022-03-14T13:22:22.711237Z"
    },
    "papermill": {
     "duration": 0.095134,
     "end_time": "2022-03-16T15:11:27.062008",
     "exception": false,
     "start_time": "2022-03-16T15:11:26.966874",
     "status": "completed"
    },
    "tags": []
   },
   "outputs": [],
   "source": [
    "# Define the decision tree to use for testing, and construct the pipeline with it.\n",
    "classifier = DecisionTreeClassifier(criterion=\"entropy\", random_state=random_seed)\n",
    "pipe = build_PCA_pipeline(classifier)"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 24,
   "id": "11c16f13",
   "metadata": {
    "execution": {
     "iopub.execute_input": "2022-03-16T15:11:27.244152Z",
     "iopub.status.busy": "2022-03-16T15:11:27.243493Z",
     "iopub.status.idle": "2022-03-16T15:11:27.245800Z",
     "shell.execute_reply": "2022-03-16T15:11:27.246267Z",
     "shell.execute_reply.started": "2022-03-14T13:22:22.716653Z"
    },
    "papermill": {
     "duration": 0.0961,
     "end_time": "2022-03-16T15:11:27.246438",
     "exception": false,
     "start_time": "2022-03-16T15:11:27.150338",
     "status": "completed"
    },
    "tags": []
   },
   "outputs": [],
   "source": [
    "# The grid search will concern only the paramater n_components of the PCA module, which is the number of output dimensions of the transformed data.\n",
    "param_grid = {\n",
    "    \"pca__n_components\": [3, 5, 8, 10, 20, 50]\n",
    "}"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 25,
   "id": "946c4295",
   "metadata": {
    "execution": {
     "iopub.execute_input": "2022-03-16T15:11:27.426692Z",
     "iopub.status.busy": "2022-03-16T15:11:27.426069Z",
     "iopub.status.idle": "2022-03-16T15:12:47.518132Z",
     "shell.execute_reply": "2022-03-16T15:12:47.517478Z",
     "shell.execute_reply.started": "2022-03-14T13:22:22.729171Z"
    },
    "papermill": {
     "duration": 80.183654,
     "end_time": "2022-03-16T15:12:47.518282",
     "exception": false,
     "start_time": "2022-03-16T15:11:27.334628",
     "status": "completed"
    },
    "tags": []
   },
   "outputs": [
    {
     "name": "stdout",
     "output_type": "stream",
     "text": [
      "Fitting 2 folds for each of 6 candidates, totalling 12 fits\n",
      "[CV] pca__n_components=3 .............................................\n"
     ]
    },
    {
     "name": "stderr",
     "output_type": "stream",
     "text": [
      "[Parallel(n_jobs=1)]: Using backend SequentialBackend with 1 concurrent workers.\n"
     ]
    },
    {
     "name": "stdout",
     "output_type": "stream",
     "text": [
      "[CV] .............................. pca__n_components=3, total=   3.6s\n",
      "[CV] pca__n_components=3 .............................................\n"
     ]
    },
    {
     "name": "stderr",
     "output_type": "stream",
     "text": [
      "[Parallel(n_jobs=1)]: Done   1 out of   1 | elapsed:    3.6s remaining:    0.0s\n"
     ]
    },
    {
     "name": "stdout",
     "output_type": "stream",
     "text": [
      "[CV] .............................. pca__n_components=3, total=   3.5s\n",
      "[CV] pca__n_components=5 .............................................\n",
      "[CV] .............................. pca__n_components=5, total=   4.1s\n",
      "[CV] pca__n_components=5 .............................................\n",
      "[CV] .............................. pca__n_components=5, total=   3.9s\n",
      "[CV] pca__n_components=8 .............................................\n",
      "[CV] .............................. pca__n_components=8, total=   4.5s\n",
      "[CV] pca__n_components=8 .............................................\n",
      "[CV] .............................. pca__n_components=8, total=   4.5s\n",
      "[CV] pca__n_components=10 ............................................\n",
      "[CV] ............................. pca__n_components=10, total=   4.9s\n",
      "[CV] pca__n_components=10 ............................................\n",
      "[CV] ............................. pca__n_components=10, total=   4.9s\n",
      "[CV] pca__n_components=20 ............................................\n",
      "[CV] ............................. pca__n_components=20, total=   7.0s\n",
      "[CV] pca__n_components=20 ............................................\n",
      "[CV] ............................. pca__n_components=20, total=   7.5s\n",
      "[CV] pca__n_components=50 ............................................\n",
      "[CV] ............................. pca__n_components=50, total=  12.6s\n",
      "[CV] pca__n_components=50 ............................................\n",
      "[CV] ............................. pca__n_components=50, total=  12.4s\n"
     ]
    },
    {
     "name": "stderr",
     "output_type": "stream",
     "text": [
      "[Parallel(n_jobs=1)]: Done  12 out of  12 | elapsed:  1.2min finished\n"
     ]
    }
   ],
   "source": [
    "# Define and start the grid search using the data in the training set\n",
    "grid_search = GridSearchCV(pipe, param_grid, cv=2, verbose=2)\n",
    "grid_search.fit(X_train, Y_train);"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 26,
   "id": "20551dd0",
   "metadata": {
    "execution": {
     "iopub.execute_input": "2022-03-16T15:12:47.710563Z",
     "iopub.status.busy": "2022-03-16T15:12:47.709584Z",
     "iopub.status.idle": "2022-03-16T15:12:47.717250Z",
     "shell.execute_reply": "2022-03-16T15:12:47.717710Z",
     "shell.execute_reply.started": "2022-03-14T13:23:20.730601Z"
    },
    "papermill": {
     "duration": 0.105266,
     "end_time": "2022-03-16T15:12:47.717897",
     "exception": false,
     "start_time": "2022-03-16T15:12:47.612631",
     "status": "completed"
    },
    "tags": []
   },
   "outputs": [
    {
     "name": "stdout",
     "output_type": "stream",
     "text": [
      "Best parameters set found on train set:\n",
      "{'pca__n_components': 5}\n",
      "\n",
      "\n",
      "Grid scores on train set:\n",
      "{'pca__n_components': 3} mean: 0.920 std: +/- 0.000)\n",
      "{'pca__n_components': 5} mean: 0.928 std: +/- 0.001)\n",
      "{'pca__n_components': 8} mean: 0.923 std: +/- 0.000)\n",
      "{'pca__n_components': 10} mean: 0.919 std: +/- 0.001)\n",
      "{'pca__n_components': 20} mean: 0.919 std: +/- 0.000)\n",
      "{'pca__n_components': 50} mean: 0.909 std: +/- 0.000)\n",
      "\n",
      "\n"
     ]
    }
   ],
   "source": [
    "print_grid_search_results(grid_search)"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 27,
   "id": "22ac4f64",
   "metadata": {
    "execution": {
     "iopub.execute_input": "2022-03-16T15:12:47.909064Z",
     "iopub.status.busy": "2022-03-16T15:12:47.908378Z",
     "iopub.status.idle": "2022-03-16T15:12:48.107151Z",
     "shell.execute_reply": "2022-03-16T15:12:48.107651Z",
     "shell.execute_reply.started": "2022-03-14T13:23:20.738925Z"
    },
    "papermill": {
     "duration": 0.29536,
     "end_time": "2022-03-16T15:12:48.107847",
     "exception": false,
     "start_time": "2022-03-16T15:12:47.812487",
     "status": "completed"
    },
    "tags": []
   },
   "outputs": [
    {
     "data": {
      "image/png": "iVBORw0KGgoAAAANSUhEUgAAAnQAAAFNCAYAAACJ9PI3AAAAOXRFWHRTb2Z0d2FyZQBNYXRwbG90bGliIHZlcnNpb24zLjUuMSwgaHR0cHM6Ly9tYXRwbG90bGliLm9yZy/YYfK9AAAACXBIWXMAAAsTAAALEwEAmpwYAABRiElEQVR4nO3deXxV1b338c8vM4RAZIYAB3BCFAREJLUoaluttXWuokC1rba17XP79Hpva+1t+9jHa6fntr1tva21tgLOs3VsVVTUIPMgKopICGEewpz59/yxd/QkOSc5QE7OSfJ9v17nxdlr77X3b2cfcn5Za6+1zd0RERERkY4rI9UBiIiIiMiRUUInIiIi0sEpoRMRERHp4JTQiYiIiHRwSuhEREREOjgldCIiIiIdnBI6ETliZjbFzFa3sH64mbmZZbVnXKlgZleb2T9SHYckl5mdbmbvm9k+M7so1fGIKKGTLs3M1plZtZn1bVK+NExAhqcotA7F3ee5+/ENy+HP9VNHsk8zm2Rmz5hZhZntNLMFZnatmRWZWa2ZHR2jzmNm9qsjOe6Rcvd73P0zqYxBAmY21cw2JGn3twC/d/ce7v54jGOvM7ODYcK3xcz+ZmY9otafa2avmtleM9tmZq+Y2RdixO9m9r0knYN0IkroROBDYFrDgpmNAbqnLpzk6SgtZGZWDLwEvAIcA/QBvgF81t3LgReBGU3q9AbOB+5Ocmwd4mcoSRcBVrWyzefdvQcwAZgI/BDAzC4DHgJmAUOAAcCPgM83qf8lYCcws+3Clk7L3fXSq8u+gHUEv2QXRpX9CrgZcGB4WJYblq8HtgB/BLqF644CngK2AbvC90Oi9vcy8FPgdWAv8A+gb5x4+ob1Kwh+kc8DMsJ1Q4FHw+PsIGgdgOAPsx8CpcBWgi+JXuG64eF5fCWM/dWw/MvAO2G8zwOROPHcDfxr+L4o3Nc3w+WjwxgzgKnAhrB8NlAPHAT2Af8eFceXwji2Aze3cF1eA/7QwvqrgA+alN0ALI2z/bPAt5qULQcuCd//FigD9gCLgSlR2/0EeBiYE67/IXAA6BO1zYTwumQD1wCvRa1z4OvA++F1/QNg4bpM4P+FP48PgW+F22e18Hm9EVgB7AYeAPIS+JxfCCwL4/8AOC8sHww8GV7HNcB1Tc77ofC89wIrgeOAm8LPWRnwmSaf89uABeFxngB6R63/AkECVBFue0Ki5wVcEMZfAbwBjG2tLpBP8BmsJ/gc7gvPdxKwKIxxC/BfLfzcrgt/LjvDn9PgsPwDGn/Gc+Ncq09FLf+S4P+2Efwf+LdWrll++HO/EqgGJrbH70S9Ou4r5QHopVcqXw2/dIHVwAnhF+wGgr++oxO6X4e/0HsDBcDfgdvCdX2ASwla9QrCL8HHo47xcvgFcBzQLVz+WZx4biNIFrPD15TwCyCTIAH5dfiLPg/4ZFjny+GXzkigB0HSNztcNzw8j1lhvW4EX+5rwvPNIkhQ3ogTz5eBv4fvrwrP44GodU+E76cSJnTRP9eo5YY4/hzGcDJQRdSXetS23YE64KwWrls3gi/vT0aVlQDfibP9TOD1qOXRBMlBbrg8PbyOWcC/ApsJEwqCxKYGuIggee0GPAN8I2p/vwZ+F76/huYJ3VNAITCMIPFrSKi+DrxN0EpzFPACrSd0CwgSk94ESfnXW/mMTwp/Vp8O4y8CRoXrXgVuDz9P48LYzo4670rg3PDnMosg6byZ4LN5HfBhk895OXASwWftEWBOuO44YH8YQzZBkr8GyGntvIDxBAnkaQT/D74Ubp+bQN2pRH0uoz4nM8L3PYDJcX5uZxMk2hMI/qD7HeEfRLE+4/F+t4TvhxIksz8FRoXXeEQr120GsCk8578Tfr700iveK+UB6KVXKl98nND9kCCZOg/4Z/gF5gSJiIVfRkdH1SuO/jJrss9xwK6o5ZeBH0Yt3wA8F6fuLQQtG8c0KS8Ov2ybfdETdD/eELV8PEECksXHidTIqPXPAl+JWs4gaHGKxNj30QSteBkEiebX+Lgl7m7gu+H7Rl+cTb/souKIbrlcAFwZ45gNLYGjWrl2dwJ3hO+PJWjF6B9n24LwGkbC5VuBu1rY9y7g5PD9T6K/yMOyKwgTxPALdzMwKVy+huYJXXTi+SDw/fD9S8DXotZ9itYTuulRy78A/tjKz+lPwK9jlA8lSJwLospuA/4Wdd7/jFr3eYLWqMyon6kDhVGf859FbT86vCaZwH8ADzb5zJUDU1s7L+B/gJ82iX01cGYCdRt9LsOyV4H/Q5xW8qjt/gL8Imq5B8H/q+GxPuNxrtU+gj8cSgkS527A6eHPrcWWVYLk/jfh+2mELcAt1dGra790D51IYDZBC9Q1BC0R0foRtBotDm/QrwCeC8sxs+5m9iczKzWzPQRfGIVmlhm1j81R7w8QfDnE8kuClot/mNlaM/t+WD4UKHX32hh1BhN8YTQoJUjmBkSVlUW9jwC/jTqXnQRJa1HTHbv7BwSJ0DiC1sKngI1mdjxwJsE9bocikZ/DLoLurEGt7Otu4HIzyyNozXje3bfG2tDd9wJPE3RfQfAFeU/DejO70czeMbPd4c+kF0H3d4Ponx8ESfdoMxtB0Oq0290XtBBrvPMe3GTfTY9zKPuKZyhBy2pTg4Gd4c+mQSmNPwdbot4fBLa7e13UMk2OHx1/KUFrXF+afEbdvT7cNvpY8c4rAvxrw+c1vD5Dw322VjeWrxC0GL5rZgvN7II42zWNeR/BrQ7N/p+04CJ3L3T3iLvf4O4Hw31AC59vMxsKnMXHn9EnCFpRP3cIx5YuRgmdCODupQTdSecTdFlG207w5XVi+Mu50N17eXCzMwRddMcDp7l7T+CMsNwOI4697v6v7j6S4J6j75rZOQRffsPi3JC/keBLr8EwoJbGX8Ye9b6MoFWoMOrVzd3fiBPWK8BlBN1j5eHylwi6CJfFO5UWT7QF7n6AoFvs0lY2fY0gGb2QoMu0tcEQ9wHTwgEXecBcCKZcIegC/CJwlLsXEnRRRl+/Rufj7pUELW3TCZLJ2a2dVxybCLpbGww9zP20pIygpbWpjUBvMyuIKhtG0HJ2uKLjH0bQorWdJp9RM7Nw20SOVQbc2uTz2t3d70ugbrPPobu/7+7TgP7Az4GHzSw/Rt2mMecTdMsfyc8HgtbFMlr+fM8g+H7+u5ltBtYSfGa/dITHlk5MCZ3Ix75CcP/Q/ujCsDXhz8Cvzaw/QDh1xrnhJgUECV9FONLyx4cbgJldYGbHhF94uwm6xOoJuic3AT8zs3wzyzOz08Nq9wH/28xGhNMi/CfBfW6xWvMg6Dq9ycxODI/Zy8wubyGsVwhu1n81XH45XH4tqrWmqS0E9/Qdrn8HrjGzfzOzPmGcJ5vZ/Q0buLsTtKb+nOD+tL+3ss9nCL6gbyH4+dSH5QUECfA2IMvMfgT0TCDGWQQtul/g8BO6B4F/CT9PhUAypqf4C3CtmZ1jZhnhsUa5exnBAIPbws/TWIL/A3OO4FjTzWy0mXUn+Dk/HH5GHgQ+F8aQTfBHUFV4/Nb8Gfi6mZ1mgXwz+1yTRDSeLUAfM+vVUGBm082sX3j9K8Li+hh17yP4uY0zs1yC/1dvuvu6BI4bV/i5/S7wH+E0PD3D6/JJM7sj3OxLBN3C46JelwLnN/x/EGlKCZ1IyN0/cPdFcVZ/j6ArdH7YrfoCQascwG8I7o3ZDswn6I49XMeG+95H0Ep1u7vPDb8UP08whcd6goEbV4R17iJIKF4laGWsBL4d7wDu/hhBEnR/eC5vAZ9tIaZXCJKehoTuNYIu6Ffj1gjuxfph2EV2YwvbxYvxDYKb0s8G1prZTuAOgqQs2iyClqAH3L2qlX1WEbS+fgq4N2rV8wTX7D2CLrZKEuj6dPfXCRKBJWEL7+H4M8Go5xXAUoLzqyVI5NtE2BV8LcHAjd0E17Oh5Wkawf2NG4HHgB+7+wtHcLjZwN8IB5UA/yuMYTVBa+bvCP6ffJ5gSo/qBOJfRDAA4/cE3fFrCBLpVrn7uwSJ2drwsziY4D7ZVWa2j2B085VhV2jTui8Q3Pv3CMEfU0fzcZf9EXH3hwn+/36Z4Ge/Bfi/wBNmNpng+vzB3TdHvZ4kOPdp8fYrXVvD0HkRETlEZvYScK+739lG+/sswQ39kVY3TjNm9jLBqNY2+VmIyKFRC52IyGEws1MJprR44Aj20c3MzjezLDMrIuiuf6ytYhSRrkMJnYjIITKzuwm6xr/TZJToIe+K4F6pXQRdru8QPDHgUOP5gQWPmGr6evYIYhORDkRdriIiIiIdnFroRERERDo4JXQiIiIiHVysSUq7jL59+/rw4cNTHYaIiIhIqxYvXrzd3fvFWtelE7rhw4ezaFG8acdERERE0oeZxZ3zUl2uIiIiIh2cEjoRERGRDk4JnYiIiEgHp4ROREREpINTQiciIiLSwXXpUa4dTemO/fx53loeX7qR/VW15OdmcdH4wVw3ZSSRPvmpDk9ERERSRAldBzF39VZumLOEmrp6auuDx7Xtq6rl/gVlPLK4nNunT+Cs4/unOEoRERFJBXW5dgClO/Zzw5wlHKyp+yiZa1Bb7xysqeOGOUso3bE/RRGKiIhIKimh6wD+PG8tNXX1LW5TU1fPnfM+bKeIREREJJ0ooesAHl+6sVnLXFO19c5jS8vbKSIRERFJJ0lN6MzsPDNbbWZrzOz7MdZHzOxFM1thZi+b2ZCwfJyZlZjZqnDdFVF15pnZsvC10cweD8unmtnuqHU/Sua5taf9VbWJbVed2HYiIiLSuSRtUISZZQJ/AD4NbAAWmtmT7v521Ga/Ama5+91mdjZwGzADOADMdPf3zWwwsNjMnnf3CnefEnWMR4AnovY3z90vSNY5pUp+bhb7Ekjq8nM0xkVERKQrSmYL3SRgjbuvdfdq4H7gwibbjAZeCt/PbVjv7u+5+/vh+43AVqBfdEUz6wmcDTyerBNIFxeNH0xWhrW4TabBxeOL2ikiERERSSfJTOiKgLKo5Q1hWbTlwCXh+4uBAjPrE72BmU0CcoAPmtS9CHjR3fdElRWb2XIze9bMTjzC+NPGdVNGtprQOfDVKSPaJyARERFJK6keFHEjcKaZLQXOBMqBuoaVZjYImA1c6+5Nh3lOA+6LWl4CRNz9ZOB3xGm5M7PrzWyRmS3atm1bm51IMkX65HPN6cNb3KbeYW+l7qETERHpipKZ0JUDQ6OWh4RlH3H3je5+ibuPB24Oyyrgoy7Vp4Gb3X1+dD0z60vQpft01L72uPu+8P0zQHa4XSPufoe7T3T3if369Wu6Om3VtTLKFeC3L77fDpGIiIhIuklmQrcQONbMRphZDnAl8GT0BmbW18waYrgJuCsszwEeIxgw8XCMfV8GPOXulVH7GmhmFr6fRHBuO9r4nFJm+YbdrW7zz7e3sGpj69uJiIhI55K0hM7da4FvAc8D7wAPuvsqM7vFzL4QbjYVWG1m7wEDgFvD8i8CZwDXRE1DMi5q91fSuLsVgiTvLTNbDvw3cKW7t96s1QHU1TtvlTdP1Arymo9q/W+10omIiHQ51klynsMyceJEX7RoUarDaNX7W/by6V+/2qisIC+Lfz/3eP7jiVXNtn/2X6ZwwqCe7RWeiIiItAMzW+zuE2OtS/WgCElArO7WsUN68cVThzKwZ16zdb97Sa10IiIiXYkSug5gxYaKZmVjigrJzcrkG1OPbrbumZWbWb15bztEJiIiIulACV0HsCJGC93JQ3oBcMWpQxnQM7fZerXSiYiIdB1K6NJcdW09b2/a06x87NBCAPKyM/n6mc1b6Z5euYn3t6iVTkREpCtQQpfm3tuyl+raxnMq98nPYXCvj++dmzZpGP0KGrfSucPvXlrTLjGKiIhIaimhS3OxulvHDulFOOUeEL+V7u8rNrJm676kxiciIiKpp4QuzcUaEDF2SGGzsqsmDaNvj+atdH+Yq1Y6ERGRzk4JXZqL10LXVLecTL52xshm5U8sK2ftNrXSiYiIdGZK6NJYZU0dq2MMbBgTI6EDuHryMPrk5zQqq3f4vVrpREREOjUldGls1cY91NU3fpLHoF559C9oPpkwQPecLK6P2Uq3kXXb9yclRhEREUk9JXRpbGXM++dit841mD45Qu8mrXR19a576URERDoxJXRpLPb9c4Ut1snPzeKrU0Y0K390aTnrdxxoq9BEREQkjSihS2MryhMbENHUzOLhFHbPblSmVjoREZHOSwldmtpXVcsHMUanji0qbLVuj9wsrpvS/F66R5ZsoGynWulEREQ6GyV0aWrlht144/EQDO/TnV5NWt7imVkcoVe3xtvW1ju3v/xBW4UoIiIiaUIJXZpaWV7RrGxMK/fPRSvIy+Yrn2x+L93Di8sorzh4BJGJiIhIulFCl6aWxxgQcXIC989Fu+b04fTMy2pUVlPn3K576URERDoVJXRpamWMhG5M0aEldD3zsvlyjFa6BxeVsVGtdCIiIp2GEro0tGt/NeubDF7IMDjpEBM6gGs/MYKC3OatdH98RffSiYiIdBZK6NJQrOlKjunfg/wmiVkienXP5trThzcrv39BGZt3Vx5OeCIiIpJmlNCloVhPiBiTwHQl8Xz5kyPo0SQZrK6rVyudiIhIJ6GELg3FHBAx9NC7WxsUds/hmk8Mb1Z+74L1bNmjVjoREZGOTgldGloR8xmuhUe0z698cgT5OZmNyqpr6/nTK2uPaL8iIiKSekro0syWPZVs2VPVqCwrwxg1sOCI9ntUfg4zY7TS3fNmKVv3qpVORESkI1NCl2ZWxOhuHTWogLzszBhbH5rrpoyke5NWuqraeu5QK52IiEiHltSEzszOM7PVZrbGzL4fY33EzF40sxVm9rKZDQnLx5lZiZmtCtddEVXnb2b2oZktC1/jwnIzs/8Oj7XCzCYk89ySpa0HRETrnZ/DjOJIs/I5b5aybW9VjBoiIiLSESQtoTOzTOAPwGeB0cA0MxvdZLNfAbPcfSxwC3BbWH4AmOnuJwLnAb8xs8Koev/m7uPC17Kw7LPAseHreuB/2v6skq8tnhDRkuumjKRbk9a+ypp67pynVjoREZGOKpktdJOANe6+1t2rgfuBC5tsMxp4KXw/t2G9u7/n7u+H7zcCW4F+rRzvQoLk0N19PlBoZoPa5lTah7snZUBEtL49cpk+eViz8lklpezYp1Y6ERGRjiiZCV0RUBa1vCEsi7YcuCR8fzFQYGZ9ojcws0lADhA9adqtYbfqr80s9xCOh5ldb2aLzGzRtm3bDvWckmrDroPsOlDTqCw3K4NjB/Ro0+Ncf8bR5GU3vvQHa+r487wP2/Q4IiIi0j5SPSjiRuBMM1sKnAmUA3UNK8MWttnAte5eHxbfBIwCTgV6A987lAO6+x3uPtHdJ/br11qjX/uKNSDixME9yc5s28vUryCXq09rfi/drJJ17Nxf3abHEhERkeRLZkJXDgyNWh4Sln3E3Te6+yXuPh64OSyrADCznsDTwM1hF2pDnU1ht2oV8FeCrt2Ejpfukt3dGu1rZ4wkN6vx5T9QXcdfXtO9dCIiIh1NMhO6hcCxZjbCzHKAK4Enozcws75m1hDDTcBdYXkO8BjBPXEPN6kzKPzXgIuAt8JVTwIzw9Guk4Hd7r4pKWeWJLFa6Ma24YCIaP175jFtUvN76e5+o5SKA2qlExER6UiSltC5ey3wLeB54B3gQXdfZWa3mNkXws2mAqvN7D1gAHBrWP5F4AzgmqbTkwD3mNlKYCXQF/i/YfkzwFpgDfBn4IZknVsy1Nc7b5XHSugKk3bMb0w9mpwmrXT7qmr5y2u6l05ERKQjMXdPdQwpM3HiRF+0aFGqwwDgg237OOf/vdKorEduFit+/BkyMixpx/3xE29xd0lpo7KC3Cxe+97Z9OqenbTjioiIyKExs8XuPjHWulQPipBQrPvnTirqmdRkDuDrU48mp8mgi71Vtdz1ulrpREREOgoldGlieVmsCYULk37cQb268cVThzQrv+v1D9l9sCZGDREREUk3SujSxMoY98+NSdKAiKa+MfUYsjMbtwTurazlb6+va5fji4iIyJFRQpcGauvqWbUxNS10AEWF3bh84tBm5X95bS17KtVKJyIiku6U0KWB97bso7KmvlHZUd2zGXJUt3aL4YapR5PV5H69PZW1zHpjXbvFICIiIodHCV0aWFle0axszJBCgqn22seQo7pz2SnN76W787UP2VdV225xiIiIyKFTQpcGlseYUPjkdrp/Lto3zzqmWStdxYEa7lYrnYiISFpTQpcGVsZI6MYUtX9CN7R3dy6ZUNSs/M55a9mvVjoREZG0pYQuxapq63h3855m5ScPLWz/YAha6TKbtNLtOlDD7PmlcWqIiIhIqimhS7F3Nu2lpq7x0zoG9MxlQM+8lMQT6ZPPReOat9L9+dW1HKhWK52IiEg6UkKXYitjPCFiTFFhu8cR7VtnH0PTB1Ts2F/NHLXSiYiIpCUldCmWLgMioo3oG7uV7o5X13Kwui4FEYmIiEhLlNClWKxnuI5N0f1z0b4Zo5Vu+75q7nlTrXQiIiLpRgldCu2vqmXN1n3NylMxwrWpo/v14PMnD25W/qdX11JZo1Y6ERGRdKKELoVWbdxDfePxEAzt3Y3e+TmpCaiJb599DE3nNt62t4p731yfmoBEREQkJiV0KRSzuzXFAyKiHdO/gAvGNm+l++MrH6iVTkREJI0ooUuhFTEGRIxN8YCIpmK10m3dW8UDC8tSE5CIiIg0o4QuhWK20A0pbPc4WnLcgALOP2lQs/KfPLmKEd9/mpN+/Dw/fHwlpTv2pyA6ERERASV0KbP7QA3rdhxoVGYGJxX1TFFE8X37nGOalXn42ldVy/0LyjjvN/OYu3pru8cmIiIiSuhSZmV58+7WkX3zKcjLTkE0LRs1sCdnHNs37vraeudgTR03zFmiljoREZEUUEKXIstjdLeenGbdrdESSTRr6uq5c96H7RCNiIiIRFNClyIrYwyIGJNmAyKivfLetla3qa13Hlta3g7RiIiISDQldCnSEQZERNtfVZvYdtWJbSciIiJtRwldCmzbW8XG3ZWNyjIzjNGD0m9ARIP83KzEtstJbDsRERFpO0lN6MzsPDNbbWZrzOz7MdZHzOxFM1thZi+b2ZCwfJyZlZjZqnDdFVF17gn3+ZaZ3WVm2WH5VDPbbWbLwtePknluR2JleUWzsuMGFNAtJ7P9g0nQReMHk9X04a5NZGUYF48vaqeIREREpEHSEjozywT+AHwWGA1MM7PRTTb7FTDL3ccCtwC3heUHgJnufiJwHvAbMysM190DjALGAN2Ar0btb567jwtftyThtNrE8rLm98+dnMb3zwFcN2Uk2Zktf1yyMzP46pQR7RSRiIiINEhmC90kYI27r3X3auB+4MIm24wGXgrfz21Y7+7vufv74fuNwFagX7j8jIeABcCQJJ5DUsSasiSdB0QARPrkc/v0CXTLziRWO12Gwe3TJxDpk9/usYmIiHR1yUzoioDo50NtCMuiLQcuCd9fDBSYWZ/oDcxsEpADfNCkPBuYATwXVVxsZsvN7FkzO/HIT6HtuXvMARHpPGVJg7OO789z35nCZ04c0GydO5wwMH3vARQREenMUj0o4kbgTDNbCpwJlAMfPfXdzAYBs4Fr3b2+Sd3bgVfdfV64vASIuPvJwO+Ax2Md0MyuN7NFZrZo27bWp+Joa5t2V7J9X3WjspysDI4bUNDusRyOSJ98/ufqUxjau1ujcgceX6YpS0RERFIhmQldOTA0anlIWPYRd9/o7pe4+3jg5rCsAsDMegJPAze7+/zoemb2Y4Iu2O9G7WuPu+8L3z8DZJtZs8cbuPsd7j7R3Sf269fvyM/yEMVqnTthUE9yslKdWycuI8O4ZHzznu5HFm8g6AkXERGR9pTMLGIhcKyZjTCzHOBK4MnoDcysr5k1xHATcFdYngM8RjBg4uEmdb4KnAtMi261M7OBZmbh+0kE57YjKWd2BJbHmFA43QdExHLJhOajWd/fuo9VG/ekIBoREZGuLWkJnbvXAt8CngfeAR5091VmdouZfSHcbCqw2szeAwYAt4blXwTOAK6JmoZkXLjuj+G2JU2mJ7kMeMvMlgP/DVzpadhcFOsJEek8oXA8kT75TIwc1az8kSUbUhCNiIhI15bUWWDDrs9nmpT9KOr9w8DDMerNAebE2WfMmN3998DvjyTeZIs3IGJsB2yhA7j0lCEsKt3VqOzJZRv5wfkntDrFiYiIiLQdfeu2o3U7DrCnsvGjsbrnZHJ0vx4piujInD9mULN7/3bsr+aV1e0/2ERERKQrU0LXjmK1zp00uBeZrTyBIV316pbNp0c3n8Lk0aXqdhUREWlPSuja0YqY9891zO7WBpfGGBzxwttbqThQHWNrERERSQYldO0o5oCIoYXtH0gbOuPYfvTtkdOorLqunqdWbEpRRCIiIl2PErp2UlfvvLUxRkJX1LFb6LIyM7hwXPNWukc12lVERKTdKKFrJ2u27uNAdV2jsl7dson06Z6iiNpOrDnplqyvYO22fSmIRkREpOtRQtdO4k1XEs6F3KGdOLgXowY2f3TZY0v1KDAREZH2oISuncQaEDGmg3e3Rrt0QvNHgT26pJz6+rSb21lERKTTUULXTmK30BW2exzJcuG4wTSdfaW84iAL1u1MTUAiIiJdiBK6dlBdW887m/Y2K+/oU5ZE698zjzOO69es/JHFGhwhIiKSbEro2sHqzXuprqtvVNa3Ry6DeuWlKKLkuCRGt+szKzdxsMlgEBEREWlbSujawYryimZlJ3eSARHRPjN6AAW5jR+1u7+6jn+8vTlFEYmIiHQNSujawYqyGAMiOlF3a4O87EzOHzOoWfnD6nYVERFJKiV07WB5jAERJ3eiARHRLj2lebfr62u2s3l3ZQqiERER6RqU0CXZweo63t/afILdzthCBzAxchRDe3drVFbv8MQyzUknIiKSLErokuztTbupazIXW1FhN/r2yE1RRMmVkWFcPL55K90jSzbgrjnpREREkkEJXZItj3H/XGeariSWS2M8Cuy9LftYtXFPCqIRERHp/JTQJdnK8q4xICJapE8+EyNHNSt/ZIkGR4iIiCSDErok60oDIqLFmpPuyWUbqWkyH5+IiIgcOSV0SbS3soa12/Y3Kz+pEz3DNZ7PjR1ETlbjj9eO/dW8snpbiiISERHpvJTQJVGs7tYRffPp1S07BdG0r17dsvn06AHNyh9dqm5XERGRtqaELolWbOh6AyKixRoc8cLbW9l9oCYF0YiIiHReSuiSaGXMhK6w/QNJkSnH9qNvj5xGZdV19fx9xcYURSQiItI5KaFLolgDIrpSC112ZgYXjmveSveoRruKiIi0KSV0SbJs/S427DrYrLwgLyvG1p3XJTG6XZesr+DD7c0Hi4iIiMjhSWpCZ2bnmdlqM1tjZt+PsT5iZi+a2Qoze9nMhoTl48ysxMxWheuuiKozwszeDPf5gJnlhOW54fKacP3wZJ5bS+au3soX75gfc93Ff3iDuau3tnNEqTN6UE9GDSxoVv6YWulERETaTNISOjPLBP4AfBYYDUwzs9FNNvsVMMvdxwK3ALeF5QeAme5+InAe8BszKwzX/Rz4tbsfA+wCvhKWfwXYFZb/Otyu3ZXu2M8Nc5ZQXRt7vrWDNXXcMGcJpTu6RguVmXFpjDnpHllSTn29HgUmIiLSFpLZQjcJWOPua929GrgfuLDJNqOBl8L3cxvWu/t77v5++H4jsBXoZ2YGnA08HNa5G7gofH9huEy4/pxw+3b153lrW508t6aunjvnfdhOEaXeheMGk9HkSpRXHGTBup2pCUhERKSTSSihM7NHzexzZnYoCWARUBa1vCEsi7YcuCR8fzFQYGZ9mhx7EpADfAD0ASrcvTbGPj86Xrh+d7h903O53swWmdmibdvafpLbx5dupLaVlqfaeuexpeVtfux01b9nHlOO7desXIMjRERE2kaiCdrtwFXA+2b2MzM7vo2OfyNwppktBc4EyoG6hpVmNgiYDVzr7m3yzCh3v8PdJ7r7xH79micZR2p/VW3rGwH7qxPbrrO49JTm3a7PrNzMweq6GFuLiIjIoUgooXP3F9z9amACsA54wczeMLNrzSzeYw/KgaFRy0PCsuj9bnT3S9x9PHBzWFYBYGY9gaeBm929YYTBDqDQzLJi7POj44Xre4Xbt6v83MRGsebndK3Rrp8ZPYCCJj+bfVW1/OPtzSmKSEREpPNIuAs17Aq9BvgqsBT4LUGC9884VRYCx4ajUnOAK4Enm+yzb1Q37k3AXWF5DvAYwYCJhvvlcHcnuNfusrDoS8AT4fsnw2XC9S+F27eri8YPJqvpDWNNZGUYF49vPp1HZ5aXncn5YwY1K39kSdfpeu4sSnfs54ePr+SkHz/PiO8/zUk/fp4fPr6yywz0ERFJR4neQ/cYMA/oDnze3b/g7g+4+7eBHrHqhPexfQt4HngHeNDdV5nZLWb2hXCzqcBqM3sPGADcGpZ/ETgDuMbMloWvceG67wHfNbM1BPfI/SUs/wvQJyz/LtBsmpT2cN2UkWRntvxjzc7M4KtTRrRTROkj1px0r72/jS17KlMQjRyOuau3ct5v5nH/gjL2VdXiBC2t9y8o47zfzOtSU/KIiKQTS6QRy8zOcve57RBPu5o4caIvWrSozfc7d/VWbpizhJq6+kYDJLIyjOzMDG6fPoGzju/f5sdNd/X1zpm/mkvZzsYTLt/02VF87cyjUxSVtKS6tp4teyrZWHGQFeUV/PzZ1S0O+umWnclz35lCpE9+O0YpItI1mNlid58Ya12iN3KNNrOlUfe3HQVMc/fb2yjGTuWs4/vz3HemcOe8D3lsaTn7q2vJz8ni4vFFfHXKiC77ZZeRYVw8fgj//eL7jcofWbKB688YSQpmmenS6uqdbXur2Lj7IJsqKtm0+yAbK4LkbdPug2zcXcn2fVUcyo0L1bV13DnvQ3560UnJC1xERJpJtIVumbuPa1K2NBzM0GElq4VO4lu3fT9Tf/Vys/Knvv1JTirqOs+5TTZ3Z9eBGjZWHAwTtMpmiduWPZWtTrFzODIMfnHZyVwwdhB52Zltvn8Rka6qLVroMs3MGgYZhE+ByGmrAKXrGN43n4mRo1hUuqtR+SNLNiihOwR7K2uCJC1M1jZVBC1qHy3vPkhlTZvM9HPI6h1ufGg5tz79Nl88dSjTT4swtHf3lMQiItJVJJrQPQc8YGZ/Cpe/FpaJHLJLJgxpltA9uWwjPzj/hFYHlHQFlTV1bI5qUdsYJmubGpZ3H2RvZfrPY7jrQA1/emUtd7y6lrOP78+M4ghnHNuPjFZGgYuIyKFLtMs1gyCJOycs+idwp7t36Flh1eWaGrsP1nDqrS80e97tX740kXNOGJCiqNpHbV09W/ZWfdSitqmhO7Ti4EcJ3I791akOs5G+PXIZXJjHrv3VbNh1kCPppB3epzvTJ0e4/JSh9OoebwpLERGJpaUu14QSus5KCV3qfPPeJTy9YlOjsvPHDOT2q09JUURHrr7e2bG/OubggobEbcueSpJw29ph65mXxeDCbgwu7MagXnkf/TuoVzeKCrsxoFcuuVnBfXClO/Zz3m/mcbDmyP+Oy8vO4KJxRcwojnDiYHW1i4gk4ojvoTOzY4HbgNFAXkO5u49skwily7l0QlGzhO6Ft7ey+0BNWrbcuDt7Kms/TtIqGneBbqyoZPPuSqrrUnPfWizdsjMZVJjH4F4fJ2uDC4NkreHfRJ9sAhDpk8/t0ye0OCXPv37mOFZt3MPTKza1+LOorKnn/oVl3L+wjFMiRzGzOMJ5Jw38KHkUEZFDk+hv878CPwZ+DZwFXMshPGVCpKkpx/ajb48ctu/7uHuxuq6ep1Zu5OrTIu0ez8Hquo/vWdsdjgwN3zcMOtifRs+dzc40BoYtaYN75TGoMPy3V7ePErde3bLbfCqYRKfkuflzJ/DAwjLufXM95RUHW9zn4tJdLC7dRd8eOVx56jCuOm0Ygwu7tWncIiKdXaL30C1291PMbKW7j4kuS3qESaQu19S65e9vc9frHzYqOyVyFI984xNtepyauvpgkEGc6Ts27j5IxYGaNj3mkTCD/gW5H7WkDe7V7eOELfy3b4/cDjG4oLaunpfe3cqsklJeW7M9oToZBp8ePYCZxcP5xNF9ND+hiEjoiO+hM7M3gE8CDwMvAeXAz9z9+LYMtL0poUutVRt387n/fq1ZefecTC6ZUMR1U0a2Oglzfb2zbV/Vx8laxcfdoQ33rm07xMlxk613fs5H96lFd4E23L82oGdepxzt+8G2fcyZX8rDizckPEr36H75zJgc4dJThlCQl35d8SIi7aktErpTCZ7HWgj8FOgJ/NLd57dhnO1OCV1quTtTfjGXDbuad8k13JP1i8vGMrJf/sdJWtS9a+UVB5M2Oe7h6pGbFSRrhd0oCpO1poMNuuV07fvEDlTX8vjSjcwqWce7m/cmVKd7TiYXjy9iZvFwjh9YkOQIRUTS0xEldOEkwj939xuTEVwqKaFLrdId+/nUf71CTV36JGQtycnK+Og+tY8GG4Qtaw3ve6oVKWHuzsJ1u5hVso7n3tqccGI+aURvZhZHOPfEgZ2yJVNEJJ4jGuXq7nVm9sm2D0u6uj/PW5s2XaGZGcaAgtzgHrWPBhg03LMWJGt98nN0P1cbMjMmjejNpBG92bqnkvsWlHHvglK27Klqsd6CD3ey4MOd9C/IZdqkYBDFgJ55LdYREensEu1y/R+gCHgI2N9Q7u6PJi+05FMLXWqd9OPn2VfVPk88aJgcN/retaAbNHjfr0cuWWrtSbmaunr++fYWZpWsY/7anQnVycowzj1xIDOLI0wa0VtJt4h0Wm3xLNc8YAdwdlSZAx06oZPU2t9GyVxLk+MOLsxjYK88zW/WQWRnZnD+mEGcP2YQ723Zy+ySUh5dsqHFKWNq652nV27i6ZWbOH5AATOKI1w8vuiQ5tgTEeno9KQItdClTKItdGZw+tF9mw02OJzJcaXj2VtZw2NLy5lVUsqarfsSqlOQm8Wlpwxh+uQIx/TvkeQIRUTaR1uMcv0rNH+Eo7t/+cjDSx0ldKn1w8dXcv+CshZvhs/KMKZNGsZPLzqpHSOTdOTulKzdweySUv7x9hbqEhxEcfoxfZgxeTifOqG/utVFpENriy7Xp6Le5wEXAxuPNDDp2q6bMpJHFpdTWx+/Oy07M4OvThnRjlFJujIzPnF0Xz5xdF827T7IfW+u594FZWzf1/IgitfX7OD1NTsY3CuPq04bxhWnDqNfQW47RS0i0j4Oq8vVzDKA19y9baf0b2dqoUu9uau3tvhs0NunT+Cs4/unMEJJZ9W19Ty3ajOzS9axcN2uhOpkZxrnjxnEzOIIE4YdpUEUItJhHHGXa4wdHg887e7HHGlwqaSELj2U7tjf6rNBRVrz9sY9zJ5fyuNLyzlYk9hzd0cP6snM4ggXjivq8hM+i0j6a4t76PbS+B66zcBN7v5I24SYGkroRDqf3QdreHjxBubML+XD7ftbr0AwUvryiUOZMTnC8L76I0JE0lObt9B1FkroRDqv+nrntTXbmVVSykvvbiHRJ8SdcVw/Zk6OcNao/mRmqDtWRNJHW7TQXQy85O67w+VCYKq7P96GcbY7JXQiXcOGXQe45831PLCwjJ37qxOqM+SobkyfHOGLE4fSOz8nyRGKiLSuLRK6Ze4+rknZUncf3zYhpoYSOpGupbKmjmdWbmJWSSnLyioSqpOTlcHnxw5mZnGEk4cWJjU+EZGWtJTQJTopU6ztWp3yxMzOM7PVZrbGzL4fY33EzF40sxVm9rKZDYla95yZVZjZU03qzDOzZeFro5k9HpZPNbPdUet+lOC5iUgXkZedySUThvD4N0/n79/6JJefMoTcrJZ/DVbX1vPIkg1c+IfXufD3r/Hw4g1UJjjoQkSkvSTaQncXUAH8ISz6JtDb3a9poU4m8B7waWADsBCY5u5vR23zEPCUu99tZmcD17r7jHDdOUB34GvufkGcYzwCPOHus8xsKnBjvG1jUQudiOzaX81Di8uYM38963ceSKjOUd2z+eKpQ5l+WoShvbsnOUIRkUBbtNB9G6gGHgDuByoJkrqWTALWuPtad68O613YZJvRwEvh+7nR6939RWBvvJ2bWU+CZ8s+nuA5iIg0c1R+DtefcTQv3ziVv15zKlOP70drU9PtOlDDn15Zyxm/nMtX/raQl1dvpT7RURciIkmQ0JMi3H0/0KzLtBVFQFnU8gbgtCbbLAcuAX5L8PSJAjPr4+47Etj/RcCL7r4nqqzYzJYTPMXiRndfdYgxi0gXlZFhnDWqP2eN6k/pjv0fDaLYfbAmbh13ePHdrbz47laG9+nO9MkRLj9lKL26Z7dj5CIiCbbQmdk/w5GtDctHmdnzbXD8G4EzzWwpcCZQDiR6c8o04L6o5SVAxN1PBn5HnJY7M7vezBaZ2aJt27YdduAi0nlF+uTzg/NP4M0fnMMvLhvLSUU9W62zbscB/u/T73DabS/wvYdX8Fb57naIVEQkkOg9dM1GtLY2ytXMioGfuPu54fJNAO5+W5ztewDvunv0wIipxLgvzsz6AquBInevjLO/dcBEd98eL0bdQyciiXB3lpZVMLuklKdXbKK6rj6hehOGFfKlTwznvJMGkpulJ1GIyJFp6R66hLpcgXozG+bu68MdDqfxkyNiWQgca2YjCFrergSuahJYX2Cnu9cDNwF3JRjPZQSDKT5K5sxsILDF3d3MJhG0PibSdSsi0iIzY8Kwo5gw7Chu/twJPLCwjHvfXE95xcEW6y1ZX8GS9cvo2yOHK04dytWnRRhc2K2dohaRriTRFrrzgDuAVwADpgDXu3uL3a5mdj7wGyATuMvdbzWzW4BF7v6kmV0G3EaQHL4KfNPdq8K684BRQA+CxOwrDcczs5eBn7n7c1HH+hbwDaAWOAh8193faCk+tdCJyOGqq3defGcLs+eXMu/9uB0BjWQYfOqEAXzpE8P5xNF9sNZGX4iIRGmTR3+ZWX/gemAp0A3Y6u6vtlmUKaCETkTawtpt+5g9v5SHF29gb2VtQnWO7pfPjMkRLjllCD3zNIhCRFrXFk+K+CrwL8AQYBkwGShx97PbMM52p4RORNrSgepaHl+6kVkl63h3c9xZlxrpnpPJxeOLmFk8nOMHFiQ5QhHpyNoioVsJnArMd/dxZjYK+E93v6RtQ21fSuhEJBncnUWlu5hVUsqzKzdRm+AcdZNG9GZmcYRzTxxIdmai04SKSFfRFoMiKt290swws1x3f9fMjm/DGEVEOg0z49ThvTl1eG+2XnAC9y8IBlFs3hNzUP5HFny4kwUf7qR/QS7TJg3jqtOGMaBnXjtFLSIdWaItdI8B1wLfIXg6wy4g293PT2p0SaYWOhFpLzV19bzw9hZmlZRSsjaxAfhZGca5Jw5kRnGE00b01iAKkS6uTQZFRO3sTKAX8Fz4SK8OSwmdiKTC+1v2Mnt+KY8s3sD+6sTmUj9+QAHTiyNcMr6I/NxEO1dEpDNp04SuM1FCJyKptLeyhseWljOrpJQ1W/clVKdHbhaXTihiRvFwjunfI8kRikg6UUIXhxI6EUkH7k7J2h3MLinlH29voS7BQRSnH9OHGZMjfOqEAWRpEIVIp6eELg4ldCKSbjbtPsh9b67n3gVlbN9XlVCdQb3yuGrSMK6cNIx+BblJjlBEUkUJXRxK6EQkXVXX1vPcqs3MLlnHwnW7EqqTnWmcP2YQM4sjTBh2lAZRiHQySujiUEInIh3B2xv3MHt+KY8vLedgTWKDKEYP6snM4ggXjiuiW05mkiMUkfaghC4OJXQi0pHsPljDI4s3MGd+KWu370+oTs+8LC6fOJQZkyMM75uf5AhFJJmU0MWhhE5EOqL6euf1D7Yzq6SUF9/ZQoJjKDjjuH7MnBzhrFH9ycxQd6xIR6OELg4ldCLS0W3YdYB731zP/QvL2Lk/salBhxzVjatPi3DFqUPpnZ+T5AhFpK0ooYtDCZ2IdBZVtXU8s3ITs0pKWbq+IqE6OVkZXDB2EDOLhzNuaGFS4xORI6eELg4ldCLSGa3csJtZJet4cvlGqmrrE6ozdkgvZhYP54Kxg8jL1iAKkXSkhC4OJXQi0pnt2l/NQ4vLmDN/Pet3HkiozlHds/niqUOZflqEob27JzlCETkUSujiUEInIl1Bfb3zynvbmFWyjpff20Yiv/bN4Ozj+zOjOMIZx/YjQ4MoRFJOCV0cSuhEpKtZv+MAc94s5cFFZVQcqEmozvA+3Zk+OcLlpwylV/fsJEcoIvEooYtDCZ2IdFWVNXU8uXwjs0tKWVm+O6E6edkZXHhyETOKI5xU1CvJEYpIU0ro4lBCJyJdnbuzrKyC2SWlPLViE9V1iQ2imDCskJnFw/nsmIHkZmkQhUh7UEIXhxI6EZGP7dhXxQOLyrhn/nrKKw4mVKdvjxyuOHUoV50WoaiwW5IjFOnalNDFoYRORKS5unrnpXe3MqtkHfPe355QnQyDT50wgJnFwzn9mD6YaRCFSFtTQheHEjoRkZat3baPOfPX89DiMvZW1iZUZ2S/fGZMjnDpKUPomadBFCJtRQldHEroREQSc6C6lseXbmRWyTre3bw3oTrdczK5aHwRM4sjjBrYM8kRinR+SujiUEInInJo3J1FpbuYVVLKc29toqYuse+QSSN6M7M4wrknDiQ7MyPJUYp0TilL6MzsPOC3QCZwp7v/rMn6CHAX0A/YCUx39w3huueAycBr7n5BVJ2/AWcCDePsr3H3ZRbcsPFb4HzgQFi+pKX4lNCJiBy+rXsruX9BGfe+uZ7NeyoTqtO/IJdpk4Zx1WnDGNAzL8kRinQuKUnozCwTeA/4NLABWAhMc/e3o7Z5CHjK3e82s7OBa919RrjuHKA78LUYCd1T7v5wk+OdD3ybIKE7Dfitu5/WUoxK6EREjlxtXT3/fHsLs0pKKVm7I6E6WRnGuScOZEZxhNNG9NYgCpEEtJTQZSXxuJOANe6+NgzifuBC4O2obUYD3w3fzwUeb1jh7i+a2dRDON6FwCwPMtT5ZlZoZoPcfdNhn4GIiLQqKzODz44ZxGfHDOL9LXuZPb+URxZvYH91Xdw6tfXO0ys38fTKTRw3oAcziodz8fgieuQm82tJpPNK5o0MRUBZ1PKGsCzacuCS8P3FQIGZ9Ulg37ea2Qoz+7WZ5R7C8TCz681skZkt2rZtWyLnISIiCTp2QAG3XHgSb978KX564Ykc279Hq3Xe27KP/3j8LSb/54v8+Im3WLM1sUEXIvKxVN+ZeiNwppktJbgvrhyI/ydd4CZgFHAq0Bv43qEc0N3vcPeJ7j6xX79+hxGyiIi0pkduFjOKh/OP/30G9103mfPHDCQzo+Vu1X1VtdxdUsqn/utVrvrzfJ57axO1CT65QqSrS2bbdjkwNGp5SFj2EXffSNhCZ2Y9gEvdvaKlnUZ1oVaZ2V8JksKEjiciIu3LzCg+ug/FR/dh8+5K7l2wnvsWrGfb3qoW673xwQ7e+GAHg3rlcdWkYVw5aRj9CnJbrCPSlSWzhW4hcKyZjTCzHOBK4MnoDcysr5k1xHATwYjXFpnZoPBfAy4C3gpXPQnMtMBkYLfunxMRSR8De+Xx3U8fx+vfO5vfTRvPpOG9W62zaXcl/++f7/GJn73I/7pvKYvW7aQrT7clEk+ypy05H/gNwbQld7n7rWZ2C7DI3Z80s8uA2wAHXgW+6e5VYd15BF2rPYAdwFfc/Xkze4lgmhMDlgFfd/d9YYL3e+A8gmlLrnX3FoewapSriEhqvbNpD7NKSnl8aTkHa1q74yYwelBPZhRHuHDcYLrnaBCFdB2aWDgOJXQiIulh98EaHlm8gTnzS1m7fX9CdXrmZXH5xKFMnxxhRN/8JEcoknpK6OJQQicikl7q653XP9jOrJJSXnxnC/UJfkWdcVw/Zk6OcNao/q0OvhDpqJTQxaGETkQkfZVXHOSe+aU8sLCMHfurE6pTVNiN6ZMjXHHqUHrn5yQ5QpH2pYQuDiV0IiLpr6q2jmdWbmJWSSlL11ckVCcnK4MLxg5iZvFwxg0tTGp8Iu1FCV0cSuhERDqWt8p3M6tkHU8s20hVbWJz1I0d0osZkyN8/uTB5GVnJjlCkeRRQheHEjoRkY6p4kA1Dy3awOz5pazfeSChOoXds7kiHEQxtHf3JEco0vaU0MWhhE5EpGOrr3deeX8bs0tKmbt6K4l8pZnBWcf3Z0ZxhDOP7UeGBlFIB6GELg4ldCIincf6HQe4581SHlhURsWBmoTqRPp0Z/ppES6fOITC7hpEIelNCV0cSuhERDqfypo6/r58I7Pnl7Jiw+6E6uRlZ/CFkwczs3g4JxX1SnKEIodHCV0cSuhERDq3ZWUVzCpZx1PLN1Fdl9ggignDCplZPJzPjhlIbpYGUUj6UEIXhxI6EZGuYce+Kh5YVMY989dTXnEwoTp98nO4ctJQrjotQlFhtyRHKNI6JXRxKKETEela6uqdl97dyqySdcx7f3tCdTIMPnXCAGYWD+f0Y/oQPDpcpP0poYtDCZ2ISNe1dts+5sxfz0OLy9hbWZtQnZH98pkxOcKlpwyhZ152kiMUaUwJXRxK6ERE5EB1LU8s28isklLe2bQnoTrdczK5aHwRM4sjjBrYM8kRigSU0MWhhE5ERBq4O4tLdzGrpJRn39pETV1i34+ThvdmRnGE804aSHZmRpKjlK5MCV0cSuhERCSWrXsreWBBGfe8uZ7NeyoTqtOvIJdpk4Zx1aRhDOyVl+QIpStSQheHEjoREWlJbV09L7yzhVklpbzxwY6E6mRmGOeeOIAZk4czeWRvDaKQNqOELg4ldCIikqg1W/cyu6SUR5aUs68qsUEUxw3owYzJES6eMIQeuVlJjlA6OyV0cSihExGRQ7WvqpbHlpYzu2Qd723Zl1CdHrlZXDqhiBnFEY7pX5DkCKWzUkIXhxI6ERE5XO7O/LU7mT1/Hc+v2kJdfWLfp584ug8ziyN86oQBZGkQhRwCJXRxKKETEZG2sHl3JfcuWM99C9azbW9VQnUG9crjqknDuHLSMPoV5CY5QukMlNDFoYRORETaUnVtPc+v2szsklIWrNuZUJ3sTOOzJw1iZnGEUyJHaRCFxKWELg4ldCIikizvbt7DrJJSHl9azoHquoTqnDCoJzOLI1w4bjDdczSIQhpTQheHEjoREUm2PZU1PLJ4A7Pnl7J22/6E6hTkZXH5KUOZURxhRN/8JEcoHYUSujiU0ImISHtxd15fs4NZJet44Z0tJDiGginH9mVm8XDOHtWfzAx1x3ZlKUvozOw84LdAJnCnu/+syfoIcBfQD9gJTHf3DeG654DJwGvufkFUnXuAiUANsAD4mrvXmNlU4Angw3DTR939lpbiU0InIiKpUF5xkHvfLOX+BWXs2F+dUJ2iwm5cPXkYV0wcSp8eGkTRFaUkoTOzTOA94NPABmAhMM3d347a5iHgKXe/28zOBq519xnhunOA7gQJW3RCdz7wbLh4L/Cqu/9PmNDdGL1ta5TQiYhIKlXV1vHsys3MKlnHkvUVCdXJycrggjGDmPmJ4YwbWpjU+CS9tJTQJfOOy0nAGndfGwZxP3Ah8HbUNqOB74bv5wKPN6xw9xfDJK0Rd3+m4b2ZLQCGtHHcIiIi7SI3K5OLxhdx0fgi3irfzeySUp5YXk5lTX3cOtW19Ty6tJxHl5YzdkgvZkyO8PmTB5OXndmOkUu6SeaMhkVAWdTyhrAs2nLgkvD9xUCBmfVJZOdmlg3MAJ6LKi42s+Vm9qyZnXh4YYuIiLS/k4p68fPLxjL/pnO4+fwTiPTp3mqdFRt2828Pr2DybS9y2zPvULbzQDtEKuko1VNU3wicaWZLgTOBciCxsd1wO0F367xweQkQcfeTgd8R1doXzcyuN7NFZrZo27ZtRxS8iIhIWyvsnsN1Z4xk7r9O5a/XnsrZo/rT2tR0FQdq+NOraznjl3P58t8WMnf1VuoTHXUhnUIy76ErBn7i7ueGyzcBuPttcbbvAbzr7kOiyqYS4744M/sxMB64xN1jtkub2Tpgortvjxej7qETEZGOYP2OA9zzZikPLCqj4kBNQnUifboz/bQIl08cQmH3nCRHKO2hpXvoktlCtxA41sxGmFkOcCXwZJPA+ppZQww3EYx4bZGZfRU4l2CARX1U+UALp9c2s0kE57ajTc5EREQkhYb16c5N55/A/JvO4ZeXjWXskF6t1indcYBbn3mH0/7zRf794eW8Vb67HSKVVEn2tCXnA78hmLbkLne/1cxuARa5+5NmdhlwG+DAq8A33b0qrDsPGAX0IEjMvuLuz5tZLVAK7A0P86i732Jm3wK+AdQCB4HvuvsbLcWnFjoREemolpVVMKtkHU+t2ER1bfxBFNHGDytkZnGE88cMIjdLgyg6Gk0sHIcSOhER6eh27q/mgYVlzJlfSnnFwYTq9MnP4YpTh3L15AhFhd2SHKG0FSV0cSihExGRzqKu3pn77lZmzS/l1fcSG/SXYXDOCQOYWRzhk8f0xVobfSEppYQuDiV0IiLSGX24fT9z5pfy0KIy9lTWJlRnZN98pk+OcNnEIfTMy05yhHI4lNDFoYROREQ6swPVtTy5bCOzSkp5e9OehOp0zwkmO55ZHGHUwJ5JjlAOhRK6OJTQiYhIV+DuLC7dxaySUp59axM1dYl9908a3psZxRHOPXEgOVmpnrpWlNDFoYRORES6mm17q7h/wXruXbCeTbsrE6rTryCXaZOGcdWkYQzslZfkCCUeJXRxKKETEZGuqraunhfe2cKsklLe+CCxaVszM4xzTxzAjMnDmTyytwZRtDMldHEooRMREYE1W/cyu6SUR5aUs68qsUEUxw3owYzJES6eMIQeuVlJjlBACV1cSuhEREQ+tq+qlseWljO7ZB3vbdmXUJ0euVlcMiEYRHFM/4IkR9i1KaGLQwmdiIhIc+7Omx/uZHZJKc+t2kxdfWK5QvHIPswsjvDp0QPIytQgiramhC4OJXQiIiIt27KnknvfXM99C9azdW9VQnUG9crjqknDuHLSMPoV5CY5wq5DCV0cSuhEREQSU1NXz/OrNjOrpJQFH+5MqE52pvHZkwYxszjCKZGjNIjiCCmhi0MJnYiIyKF7d/MeZpeU8tjScg5U1yVU54RBPZlZHOHCcYPpnqNBFIdDCV0cSuhEREQO357KGh5ZvIHZ80tZu21/QnUK8rK4/JShzCiOMKJvfpIj7FyU0MWhhE5EROTIuTuvr9nBrJJ1vPDOFhIcQ8GUY/sys3g4Z4/qT2aGumNbo4QuDiV0IiIibau84iD3vlnK/QvK2LG/OqE6RYXduHryMK6YOJQ+PTSIIh4ldHEooRMREUmOqto6nl25mVkl61iyviKhOjmZGVwwdhAziiOMG1qoQRRNKKGLQwmdiIhI8r1VvpvZJaU8sbycypr6hOqMKerFjOIIXzh5MHnZmUmOsGNQQheHEjoREZH2s/tADQ8tLmP2/FJKdxxIqE5h92y+OHEo00+LMKxP9yRHmN6U0MWhhE5ERKT91dc7r76/jdklpby0eiuJpCJmcNbx/ZlRHOHMY/uR0QUHUSihi0MJnYiISGqV7TzAnDdLeXBhGbsO1CRUJ9KnO9NPi3D5xCEUds9JcoTpQwldHEroRERE0kNlTR1PrdjE7JJ1LN+wO6E6uVkZXDhuMDOLh3NSUa8kR5h6SujiUEInIiKSfpaVVTCrZB1PrdhEdW1igyjGDytkZnGE88cMIjercw6iUEIXhxI6ERGR9LVzfzUPLCxjzvxSyisOJlSnT34OV5w6lKsnRygq7JbkCNuXEro4lNCJiIikv7p6Z+67W5k1v5RX39uWUJ0Mg3NOGMDM4gifPKZvp5jTTgldHEroREREOpYPt+9nzvxSHlpUxp7K2oTqjOybz/TJES49ZQi9umUnOcLkSVlCZ2bnAb8FMoE73f1nTdZHgLuAfsBOYLq7bwjXPQdMBl5z9wui6owA7gf6AIuBGe5ebWa5wCzgFGAHcIW7r2spPiV0IiIiHdPB6jqeWFbOrJJS3t60J6E63bIzuWh8ETOLI5wwqGeSI2x7KUnozCwTeA/4NLABWAhMc/e3o7Z5CHjK3e82s7OBa919RrjuHKA78LUmCd2DwKPufr+Z/RFY7u7/Y2Y3AGPd/etmdiVwsbtf0VKMSuhEREQ6NndnyfpdzCop5ZmVm6ipSyyvOXX4UcwoHs55Jw4kJysjyVG2jVQldMXAT9z93HD5JgB3vy1qm1XAee5eZkHn9m537xm1fipwY0NCF26zDRjo7rXRxzCz58P3JWaWBWwG+nkLJ6iETkREpPPYtreKBxau554317Npd2VCdfoV5DJt0jCumjSMgb3ykhzhkWkpoUtmSloElEUtbwjLoi0HLgnfXwwUmFmfFvbZB6hw94ZO8+h9fnS8cP3ucPtGzOx6M1tkZou2bUvsxkoRERFJf/0KcvnW2ccy79/P4o/TT+H0Y1pKKQLb9lbx3y++z+k/f4kb7llMyQc76IjjC7JSfPwbgd+b2TXAq0A5UJfMA7r7HcAdELTQJfNYIiIi0v6yMjM476SBnHfSQNZs3cec+aU8vHgD+6riD6Koq3eeWbmZZ1Zu5tj+PZhZHOHiCUPokZvqVCkxyWyhKweGRi0PCcs+4u4b3f0Sdx8P3ByWVbSwzx1AYdil2nSfHx0vXN8r3F5ERES6qGP69+AnXziR+T84h59edBLHDejRap33t+7jP55YxeT/fJEfPfEW72/Z2w6RHplkJnQLgWPNbISZ5QBXAk9Gb2Bmfc2sIYabCEa8xhXeDzcXuCws+hLwRPj+yXCZcP1LLd0/JyIiIl1Hj9wsZkyO8Px3zuD+6yfzuTGDyMpoeW66fVW1zCop5dO/fpVpd8zn2ZWbqK1L7MkV7S3Z05acD/yGYNqSu9z9VjO7BVjk7k+a2WXAbYATdLl+092rwrrzgFFAD4KWtq+4+/NmNpJg2pLewFKCqU6qzCwPmA2MJ5gC5Up3X9tSfBoUISIi0nVt2VPJvW+u574F69m6tyqhOgN75nHVacO4ctJQDlbX8ed5a3l86Ub2V9WSn5vFReMHc92UkUT65Ld5vJpYOA4ldCIiIlJTV8/zqzYzq6SUBR/uTKhOZgbgBjjRM6VkZRjZmRncPn0CZx3fv03jVEIXhxI6ERERifbu5j3MLinlsaXlHKg+/HGa3bIzee47U9q0pS5V05aIiIiIdCijBvbk1ovHMP8H5/CTz4/m6H6Hl5DV1NVz57wP2zi6+JTQiYiIiDTRMy+ba04fwQvfPZN7vnoa5544gFbGUDRSW+88trS89Q3bSMeYXEVEREQkBcyM04/py+nH9GVjxUHufXM9v5+7JqG6+6vjz3vX1tRCJyIiIpKAwYXduPHc48nPzUxo+/yc9ms3U0InIiIicgguHl/U6hx2WRnGxeObPvE0eZTQiYiIiByC66aMJDuz5RQqOzODr04Z0U4RKaETEREROSSRPvncPn0C3bIzm7XUZWUY3bIzuX36hKRMLhyPEjoRERGRQ3TW8f157jtTmDZpGD1yszALHi82bdIwnvvOlDafVLg1mlhYEwuLiIhIB6CJhUVEREQ6MSV0IiIiIh2cEjoRERGRDk4JnYiIiEgHp4ROREREpIPr0qNczWwbUJrqOKSRvsD2VAchCdP16lh0vToWXa+OpT2uV8Td+8Va0aUTOkk/ZrYo3pBsST+6Xh2LrlfHouvVsaT6eqnLVURERKSDU0InIiIi0sEpoZN0c0eqA5BDouvVseh6dSy6Xh1LSq+X7qETERER6eDUQiciIiLSwSmhExEREenglNBJypjZXWa21czeiirrbWb/NLP3w3+PSmWMEjCzoWY218zeNrNVZvYvYbmuVxoyszwzW2Bmy8Pr9X/C8hFm9qaZrTGzB8wsJ9WxysfMLNPMlprZU+GyrleaMrN1ZrbSzJaZ2aKwLKW/D5XQSSr9DTivSdn3gRfd/VjgxXBZUq8W+Fd3Hw1MBr5pZqPR9UpXVcDZ7n4yMA44z8wmAz8Hfu3uxwC7gK+kLkSJ4V+Ad6KWdb3S21nuPi5q7rmU/j5UQicp4+6vAjubFF8I3B2+vxu4qD1jktjcfZO7Lwnf7yX40ilC1ysteWBfuJgdvhw4G3g4LNf1SiNmNgT4HHBnuGzoenU0Kf19qIRO0s0Ad98Uvt8MDEhlMNKcmQ0HxgNvouuVtsLuu2XAVuCfwAdAhbvXhptsIEjKJT38Bvh3oD5c7oOuVzpz4B9mttjMrg/LUvr7MKs9DyZyKNzdzUzz6qQRM+sBPAJ8x933BI0IAV2v9OLudcA4MysEHgNGpTYiicfMLgC2uvtiM5ua4nAkMZ9093Iz6w/808zejV6Zit+HaqGTdLPFzAYBhP9uTXE8EjKzbIJk7h53fzQs1vVKc+5eAcwFioFCM2v4Q34IUJ6quKSR04EvmNk64H6CrtbfouuVtty9PPx3K8EfTJNI8e9DJXSSbp4EvhS+/xLwRApjkVB4P89fgHfc/b+iVul6pSEz6xe2zGFm3YBPE9z3OBe4LNxM1ytNuPtN7j7E3YcDVwIvufvV6HqlJTPLN7OChvfAZ4C3SPHvQz0pQlLGzO4DpgJ9gS3Aj4HHgQeBYUAp8EV3bzpwQtqZmX0SmAes5ON7fH5AcB+drleaMbOxBDdlZxL84f6gu99iZiMJWoB6A0uB6e5elbpIpamwy/VGd79A1ys9hdflsXAxC7jX3W81sz6k8PehEjoRERGRDk5driIiIiIdnBI6ERERkQ5OCZ2IiIhIB6eETkRERKSDU0InItLBmdlUM/tEquMQkdRRQici0vFNBZTQiXRhSuhEpFMws+Fm9o6Z/dnMVpnZP8JJdWNte4yZvWBmy81siZkdbYFfmtlbZrbSzK4It51qZq+Y2RNmttbMfmZmV5vZgnC7o8Pt/mZmfzSzRWb2Xvg4J8wsz8z+Gm671MzOCsuvMbNHzew5M3vfzH4RFd9nzKwkjO2h8JFrmNk6M/s/YflKMxsVPlv368D/NrNlZjbFzC4Pz2O5mb2a1B+8iKQFPctVRDqTY4Fp7n6dmT0IXArMibHdPcDP3P0xM8sj+OP2EmAccDLBZNcLo5Khk4ETgJ3AWuBOd59kZv8CfBv4TrjdcIJHAB0NzDWzY4BvEjzacYyZjSJ4oPdx4fbjgPFAFbDazH4HHAR+CHzK3feb2feA7wK3hHW2u/sEM7uBYALar5rZH4F97v4rADNbCZwbPmuy8LB+kiLSoaiFTkQ6kw/dfVn4fjFBgtVI+MieInd/DMDdK939APBJ4D53r3P3LcArwKlhtYXuvimcpf8D4B9h+comx3jQ3evd/X2CxG9UuN854bHeJZhBviGhe9Hdd7t7JfA2EAEmA6OB181sGcEjhCJRx2h4jm7M8wu9DvzNzK4jeFqEiHRyaqETkc4k+rFIdUDMLtcj3G991HI9jX+PNn30TmuP4mkabxZgwD/dfVordRq2b8bdv25mpwGfAxab2SnuvqOVWESkA1MLnYh0Ke6+F9hgZhcBmFmumXUneFbtFWaWaWb9gDOABYe4+8vNLCO8r24ksDrc79XhsY4jeM7j6hb2MR84PeyubXgQ+HEtbA+wFyhoWDCzo939TXf/EbANGHqI5yEiHYwSOhHpimYA/8vMVgBvAAMJHra9AlgOvAT8u7tvPsT9ridIAp8Fvh52pd4OZIT3tT0AXNPSA9bdfRtwDXBfGF8JQddtS/4OXNwwKAL4ZTho4q3w/JYf4nmISAdj7q31CIiISGvM7G/AU+7+cKpjEZGuRy10IiIiIh2cWuhEpNMysz8Apzcp/q27/zUV8YiIJIsSOhEREZEOTl2uIiIiIh2cEjoRERGRDk4JnYiIiEgHp4ROREREpINTQiciIiLSwSmhExEREeng/j/ACa6ncy/JWgAAAABJRU5ErkJggg==\n",
      "text/plain": [
       "<Figure size 720x360 with 1 Axes>"
      ]
     },
     "metadata": {
      "needs_background": "light"
     },
     "output_type": "display_data"
    }
   ],
   "source": [
    "show_curve(param_grid[\"pca__n_components\"],\n",
    "           grid_search.cv_results_['mean_test_score'],\n",
    "           \"n_components\",\n",
    "           \"accuracy\",\n",
    "           \"Mean score with CV varying n_components of PCA\")"
   ]
  },
  {
   "cell_type": "markdown",
   "id": "5274163f",
   "metadata": {
    "papermill": {
     "duration": 0.094269,
     "end_time": "2022-03-16T15:12:48.296146",
     "exception": false,
     "start_time": "2022-03-16T15:12:48.201877",
     "status": "completed"
    },
    "tags": []
   },
   "source": [
    "As we can see, the best results are obtained with a number of components less than 10, which is a great dimensionality reduction considering that we started with 286 dimensions."
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 28,
   "id": "25e2f69a",
   "metadata": {
    "execution": {
     "iopub.execute_input": "2022-03-16T15:12:48.489947Z",
     "iopub.status.busy": "2022-03-16T15:12:48.489299Z",
     "iopub.status.idle": "2022-03-16T15:12:48.492233Z",
     "shell.execute_reply": "2022-03-16T15:12:48.491710Z",
     "shell.execute_reply.started": "2022-03-14T13:23:20.927362Z"
    },
    "papermill": {
     "duration": 0.101744,
     "end_time": "2022-03-16T15:12:48.492398",
     "exception": false,
     "start_time": "2022-03-16T15:12:48.390654",
     "status": "completed"
    },
    "tags": []
   },
   "outputs": [],
   "source": [
    "# Store the best number of components found\n",
    "pca_n_components = grid_search.best_params_[\"pca__n_components\"]"
   ]
  },
  {
   "cell_type": "markdown",
   "id": "e5a7d9c2",
   "metadata": {
    "papermill": {
     "duration": 0.094424,
     "end_time": "2022-03-16T15:12:48.681246",
     "exception": false,
     "start_time": "2022-03-16T15:12:48.586822",
     "status": "completed"
    },
    "tags": []
   },
   "source": [
    "## **ML Models Testing**"
   ]
  },
  {
   "cell_type": "markdown",
   "id": "60f1ff85",
   "metadata": {
    "papermill": {
     "duration": 0.094233,
     "end_time": "2022-03-16T15:12:48.869901",
     "exception": false,
     "start_time": "2022-03-16T15:12:48.775668",
     "status": "completed"
    },
    "tags": []
   },
   "source": [
    "Now we will test four different classificators to find the most appropriate to solve the task:\n",
    "1. (Simple) Decision Tree\n",
    "2. Random Forest\n",
    "3. Extremely Randomized Trees\n",
    "4. Deep Neural Network\n",
    "\n",
    "For the first three non-deep models we will use cross validation in order to set their hyper-paramaters and test their performances.\n",
    "The Deep Neural Network instead will be trained and tested using a more classical train-validation data split."
   ]
  },
  {
   "cell_type": "markdown",
   "id": "376b8a07",
   "metadata": {
    "papermill": {
     "duration": 0.094674,
     "end_time": "2022-03-16T15:12:49.058474",
     "exception": false,
     "start_time": "2022-03-16T15:12:48.963800",
     "status": "completed"
    },
    "tags": []
   },
   "source": [
    "### **Decision Tree Classifier**"
   ]
  },
  {
   "cell_type": "markdown",
   "id": "5b75800c",
   "metadata": {
    "papermill": {
     "duration": 0.094869,
     "end_time": "2022-03-16T15:12:49.248360",
     "exception": false,
     "start_time": "2022-03-16T15:12:49.153491",
     "status": "completed"
    },
    "tags": []
   },
   "source": [
    "Let's start with a simple decision tree classifier."
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 29,
   "id": "91a735cd",
   "metadata": {
    "execution": {
     "iopub.execute_input": "2022-03-16T15:12:49.442608Z",
     "iopub.status.busy": "2022-03-16T15:12:49.441975Z",
     "iopub.status.idle": "2022-03-16T15:12:49.444936Z",
     "shell.execute_reply": "2022-03-16T15:12:49.444404Z",
     "shell.execute_reply.started": "2022-03-14T13:23:20.932165Z"
    },
    "papermill": {
     "duration": 0.101814,
     "end_time": "2022-03-16T15:12:49.445065",
     "exception": false,
     "start_time": "2022-03-16T15:12:49.343251",
     "status": "completed"
    },
    "tags": []
   },
   "outputs": [],
   "source": [
    "from sklearn.tree import DecisionTreeClassifier"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 30,
   "id": "bf26e7c9",
   "metadata": {
    "execution": {
     "iopub.execute_input": "2022-03-16T15:12:49.641830Z",
     "iopub.status.busy": "2022-03-16T15:12:49.641187Z",
     "iopub.status.idle": "2022-03-16T15:13:40.403509Z",
     "shell.execute_reply": "2022-03-16T15:13:40.404037Z",
     "shell.execute_reply.started": "2022-03-14T13:23:20.943509Z"
    },
    "papermill": {
     "duration": 50.863431,
     "end_time": "2022-03-16T15:13:40.404221",
     "exception": false,
     "start_time": "2022-03-16T15:12:49.540790",
     "status": "completed"
    },
    "tags": []
   },
   "outputs": [
    {
     "name": "stdout",
     "output_type": "stream",
     "text": [
      "Fitting 2 folds for each of 6 candidates, totalling 12 fits\n",
      "[CV] classifier__max_depth=4 .........................................\n"
     ]
    },
    {
     "name": "stderr",
     "output_type": "stream",
     "text": [
      "[Parallel(n_jobs=1)]: Using backend SequentialBackend with 1 concurrent workers.\n"
     ]
    },
    {
     "name": "stdout",
     "output_type": "stream",
     "text": [
      "[CV] .......................... classifier__max_depth=4, total=   3.2s\n",
      "[CV] classifier__max_depth=4 .........................................\n"
     ]
    },
    {
     "name": "stderr",
     "output_type": "stream",
     "text": [
      "[Parallel(n_jobs=1)]: Done   1 out of   1 | elapsed:    3.2s remaining:    0.0s\n"
     ]
    },
    {
     "name": "stdout",
     "output_type": "stream",
     "text": [
      "[CV] .......................... classifier__max_depth=4, total=   3.4s\n",
      "[CV] classifier__max_depth=8 .........................................\n",
      "[CV] .......................... classifier__max_depth=8, total=   3.6s\n",
      "[CV] classifier__max_depth=8 .........................................\n",
      "[CV] .......................... classifier__max_depth=8, total=   3.6s\n",
      "[CV] classifier__max_depth=16 ........................................\n",
      "[CV] ......................... classifier__max_depth=16, total=   3.8s\n",
      "[CV] classifier__max_depth=16 ........................................\n",
      "[CV] ......................... classifier__max_depth=16, total=   3.9s\n",
      "[CV] classifier__max_depth=20 ........................................\n",
      "[CV] ......................... classifier__max_depth=20, total=   4.2s\n",
      "[CV] classifier__max_depth=20 ........................................\n",
      "[CV] ......................... classifier__max_depth=20, total=   3.8s\n",
      "[CV] classifier__max_depth=24 ........................................\n",
      "[CV] ......................... classifier__max_depth=24, total=   3.8s\n",
      "[CV] classifier__max_depth=24 ........................................\n",
      "[CV] ......................... classifier__max_depth=24, total=   3.7s\n",
      "[CV] classifier__max_depth=26 ........................................\n",
      "[CV] ......................... classifier__max_depth=26, total=   3.7s\n",
      "[CV] classifier__max_depth=26 ........................................\n",
      "[CV] ......................... classifier__max_depth=26, total=   3.7s\n"
     ]
    },
    {
     "name": "stderr",
     "output_type": "stream",
     "text": [
      "[Parallel(n_jobs=1)]: Done  12 out of  12 | elapsed:   44.6s finished\n"
     ]
    }
   ],
   "source": [
    "# Construct the PCA-pipeline using the selected classifier\n",
    "classifier = DecisionTreeClassifier(criterion=\"entropy\", random_state=random_seed)\n",
    "pipe = build_PCA_pipeline(classifier, pca_n_components)\n",
    "\n",
    "# Define the parameter grid to use in the the grid search with cross validation.\n",
    "# For the decision tree we will consider the hyper-parameter which controls the max depth of the tree.\n",
    "param_grid = {\n",
    "    \"classifier__max_depth\": [4,8,16,20,24,26]\n",
    "}\n",
    "\n",
    "# Perform the search on the training set\n",
    "grid_search = GridSearchCV(pipe, param_grid, cv=2, verbose=2)\n",
    "grid_search.fit(X_train, Y_train);"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 31,
   "id": "22ddd06c",
   "metadata": {
    "execution": {
     "iopub.execute_input": "2022-03-16T15:13:40.611085Z",
     "iopub.status.busy": "2022-03-16T15:13:40.610172Z",
     "iopub.status.idle": "2022-03-16T15:13:40.613632Z",
     "shell.execute_reply": "2022-03-16T15:13:40.614305Z",
     "shell.execute_reply.started": "2022-03-14T13:23:47.04798Z"
    },
    "papermill": {
     "duration": 0.109294,
     "end_time": "2022-03-16T15:13:40.614526",
     "exception": false,
     "start_time": "2022-03-16T15:13:40.505232",
     "status": "completed"
    },
    "tags": []
   },
   "outputs": [
    {
     "name": "stdout",
     "output_type": "stream",
     "text": [
      "Best parameters set found on train set:\n",
      "{'classifier__max_depth': 26}\n",
      "\n",
      "\n",
      "Grid scores on train set:\n",
      "{'classifier__max_depth': 4} mean: 0.595 std: +/- 0.012)\n",
      "{'classifier__max_depth': 8} mean: 0.763 std: +/- 0.004)\n",
      "{'classifier__max_depth': 16} mean: 0.899 std: +/- 0.000)\n",
      "{'classifier__max_depth': 20} mean: 0.920 std: +/- 0.001)\n",
      "{'classifier__max_depth': 24} mean: 0.926 std: +/- 0.001)\n",
      "{'classifier__max_depth': 26} mean: 0.927 std: +/- 0.001)\n",
      "\n",
      "\n"
     ]
    }
   ],
   "source": [
    "print_grid_search_results(grid_search)"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 32,
   "id": "aa9cfa10",
   "metadata": {
    "execution": {
     "iopub.execute_input": "2022-03-16T15:13:40.819117Z",
     "iopub.status.busy": "2022-03-16T15:13:40.818161Z",
     "iopub.status.idle": "2022-03-16T15:13:40.957622Z",
     "shell.execute_reply": "2022-03-16T15:13:40.957069Z",
     "shell.execute_reply.started": "2022-03-14T13:23:47.049669Z"
    },
    "papermill": {
     "duration": 0.242406,
     "end_time": "2022-03-16T15:13:40.957809",
     "exception": false,
     "start_time": "2022-03-16T15:13:40.715403",
     "status": "completed"
    },
    "tags": []
   },
   "outputs": [
    {
     "data": {
      "image/png": "iVBORw0KGgoAAAANSUhEUgAAAmcAAAFOCAYAAADD+bpyAAAAOXRFWHRTb2Z0d2FyZQBNYXRwbG90bGliIHZlcnNpb24zLjUuMSwgaHR0cHM6Ly9tYXRwbG90bGliLm9yZy/YYfK9AAAACXBIWXMAAAsTAAALEwEAmpwYAABDP0lEQVR4nO3dd5xcdb3/8ddne7LZ9EbKppBkE3pCBClBOoGrUlQkES+oiIoBkSLlxxUuXCFYUBDkClwERKoCooSE0AOCpNHSeyW97W6SbfP5/XHOksnO7mST7NR9Px+Peeyc7/d7Zj4z58zMZ8/3+z3H3B0RERERSQ85qQ5ARERERHZRciYiIiKSRpSciYiIiKQRJWciIiIiaUTJmYiIiEgaUXImIiIikkaUnIlkCTMbZWbz4tT3NzM3s7xkxpUKZvYtM3sl1XEkg5mdaGYrE/TYLbrPmNn/mNkGM1vTEo+3F89bamYVZpa7h3atZr+R9KbkTFLOzJaaWbWZdW1QPjP8YeifotAyirtPcfey+uXwfT11fx7TzI4yswlmtsXMNpnZB2b2HTPrbWa1ZnZgI+s8b2a/3p/n3V/u/hd3Pz2VMWSilthn4jx2KXA1cJC792yk/kQzi4RJVIWZrTSzZ8zsC/v73O6+3N3buXvdHtq1+H4TJnz1r2lHg9dY0ZLPJdlDyZmkiyXAmPoFMzsUaJu6cBInU45cmdkxwOvAW8AgoAvwI+BMd18FvAZ8u8E6nYGzgEcTHFtGvIeym1Jgo7uvi9Nmtbu3A0qALwJzgSlmdkoyAkyEMOFrF76uMwlfY1TZ5/Z0ZE9aDyVnki7+DPxn1PJFwGPRDcys0Mx+bWbLzWytmf2vmbUJ6zqZ2T/NbL2ZbQ7v94la900zu83M3jWzcjN7peGRuqi2XcP1648WTTGznLCur5k9Fz7PRjO7NyzPMbObzGyZma0zs8fMrENYV9819D0zW06Q8GBm3zWzOWG8k8ysXxPxPGpmV4f3e4eP9eNw+cAwxpzo7i0z+zPBj+E/wv/Qfxb1kN8K38MNZvb/4myTXwGPuvud7r7BA9Pd/fyw/lEaJGfABcBsd/+kkdfxspmNa1D2kZmdF96/28xWmNk2M5tuZqOi2t1iZn81s8fNbBtwvZltN7MuUW1GhNsl38wuNrN3ourczH5oZgvC7XqfmVlYl2tmvwnfjyVmNs7idOWFR5euNbOPzazSzP7PzHqEr6/czF41s05R7Z81szVmttXM3jazg8PyAjP70Mwuj4rjXTP7eZxtgpm1MbNHwv1mNvCFBvW9zOxv4XuxxMyuaOR9fDqMdYaZHR7W7fc+Y2Ydwn1/ffhZuCncN08FJgO9wsd+JN5rDPe1le7+c+Ah4M6o5xhqZpPD/X6emZ0fVdcm3JbLwvf7nbBst+7ZcP9YHL4HS8zsW1Hl0fvNsWY2NXysqWZ2bFRds79T4rxfj5jZ/RYcna4ETtrD9ssxs+vNbJEF3z/PWPAPkWQbd9dNt5TegKXAqcA8YBiQC6wE+gEO9A/b/RZ4EehM8J/1P4A7wrouwNcIjraVAM8CL0Q9x5vAImAI0CZcHt9EPHcA/wvkh7dRgIVxfRTGUQwUAceH63wXWAgMBNoBzwF/Duv6h6/jsXC9NsDZYfthQB5wE/CvJuL5LvCP8P7Y8HU8HVX39/D+icDKhu9r1HJ9HA+GMRwOVAHDGnnOtkAdcFKc7dYG2Fr/HoRl7wFXNtH+P4F3o5YPArYAheHyheF2zCPo/loDFIV1twA1wDkE/1S2ASYAP4p6vN8Cvw/vXwy8E1XnwD+BjgQJyHpgdFj3Q2A20AfoBLwats+Ls7++D/QAegPrgBnA8HCfeB24ucH2KwEKgd8BH0bVHQJsDveD/xc+bu4ePi/jgSkEn4O+wKf12z18b6YDPwcKCPbHxcAZDd7HrxPs29cQHLXO3999Jmz/GPD38PX2B+YD32ts/2xk3UbrgZOBCMFnpxhYAXwn3E+GAxsIukoB7iP4bPcm+LweG77v9a8jL3yMbUBZuM4BwMEN95vw/d1M8A9IHsGR/c1Al739TmnqNQKPEHyGjgu3Xds9bL+fhPtIn/B1/RF4Mhnf07ol95byAHTTjV3J2U0EidFogv+y88Iv1P4EyVElcGDUescAS5p4zCOAzVHLbwI3RS1fBkxsYt1bwx+YQQ3KjyH4UY/50Sbo4rssarmM4EcwL+qHYWBU/cuEP1rhcg6wHejXyGMfGP4o5BAkjT9g14/xo8BV4f2GX/xLafyHtk9U2QfABY08Z++w7dA9bLuHgAfC+4OBaqB7E21Lwm3YL1z+BfBwnMfeDBwe3r8FeLtB/TcJkz2CH+I1wFHh8sXEJmfRSeQzwPXh/deBH0TVncqek7NvRS3/Dbg/avlyov4xaLBux/CxO0SVXU3wj8lmYHAzPi+LCRPLcPnSqP3haGB5g/Y3AH+Keh/fb7DffQaMaoF9Jjfc/gdFlf0AeLOx/bOR9RutB4aGMfQOt/mUBvV/BG4OX8uO+n2mQZv611GfnG0h+GeuTYN2n+83BEnZBw3q3wMuDu+/STO/U5p6jQTJ2WNRy3vafnOAU6LqDiD8ntnTfqNbZt3UrSnp5M8ER4YupkGXJtCN8L/KsFtqCzAxLMfM2prZH8PujG3A20BH230MR/QMse0ER7ga8yuCo1qvhF0f14flfYFl7l7byDq9gGVRy8sIfgh6RJWtiLrfD7g76rVsIkhAezd8YHdfRJDUHEFwFO+fwGozKwO+RDAmbG80533YTHC04oA9PNajwDfMrIjgx2ySNzGmyN3LgZcIuj4hOBLxl/p6M7vGgm7ereF70gGI7iaKfv8gSKAPMrMBwGnAVnf/IE6sTb3uXg0eu+HzNGZt1P0djSy3g8+7KseH3VDbCJIf2P11PUqwP0xw9wXNeO6G8Ubvd/0Iug63RO1bN9LEfujuEYKj1L328JzN2We6EhyNa/g5iNmn91L9PwpbCF7f0Q1e37eAnuHzFxEczWqSu1cSJHk/BD4zs5fMbGgjTRt+piH29TT3OyWeht8L8bZfP+D5qLo5BEe4o7evZAElZ5I23H0ZQRfLWQTdgtE2EPzoHezuHcNbB981oPZqgqNVR7t7e+CEsNz2IY5yd7/a3QcCXwWusmBA8gqg1Bofi7Sa4IuzXilQy+4/2h51fwXB0ZqOUbc27v6vJsJ6i6ArqsCDwfhvEYzL6wR82NRLiftC43D37QRHCb62h6bvECSWZxN0S+5pIsCTwBgLJhsUAW9AcBoQ4GfA+UAnd+9I0N0Tvf12ez3uvpPgCNiFBInhn/f0uprwGUE3Ub2++/g4jRlL8N6cSpBs9g/Lo1/XHwgS7jPM7PhmPOZnDWIsjbq/guBocvR+VeLuZ0W1+XxdC8ZS9iHYf2E/9hmCz2gNsZ+DVfvxmADnAjPCpGoF8FaD19fO3X8UPv9OgiPNcbn7JHc/jeCfj7kE3bYNNfxMQ8u8nphwou7vafutIJiQE11fFH4nSBZRcibp5nvAyeEX8efC//AfBH5rZt3h88HxZ4RNSgiSty3hANmb9zUAM/uymQ0yMyNIEOoIjiJ9QPDDON7Mis2syMyOC1d7EvipmQ0ws3bA7QTjwho7ygZB9+QNtmtweAcz+0acsN4CxhEcEYSgS2UcQRdMU6cHWEswZmVf/Qy42ILB713COA83s6fqG7i7ExzlvJOgy+4fe3jMCQQ/eLcSvD+RsLyEIJldD+RZMCi+fTNifIzgSOtX2ffk7BngJ+H+1BG4bh8fpzElBGO0NhIc+b09utLMvg0cSfAargAeDfefPcV7gwWTYPoQdKPW+wAoN7PrwoHwuWZ2iO1+Ooojzey88J+MK8P43g/r9nmfCffDZ4BfmFmJBRNcrgIe39vHskBvM7sZuITg6BEESewQM/u2BRM/8s3sC2Y2LNyXHgbuCgfV55rZMWZW2OCxe5jZ2WZWHL72CoLPd0MTwucaa2Z5ZvZNgnGS/9zb17MX9rT9/pfg/e0XvpZuZnZ2AuORFFFyJmnF3Re5+7Qmqq8j6G58P+wiepXgaBkEA63bEPz3/D5Bl+e+Ghw+dgXB0aM/uPsb4Y/PVwhOK7GcoDvom+E6DxMkB28THP3bye4/mrtx9+cJEpqnwtfyKcE0+6a8RfBDX5+cvUPwY/92k2sE4/duCrtAronTrqkY/0UwGPtkYLGZbQIeIPjRivYYwRGFp929ag+PWUVwVPRU4ImoqkkE22w+QdfRTprRveju7xL8sM4Ij7zuiweBV4CPgZkEr6+WICnfX48RvJ5VBJMO6pOg+vN+/Q74T3evcPcngGkEExvi+e/wMZeEcX+elIb76JcJusCXEHweHiI4alfv7wT7bf1g9/PcvSas2699hmCfryQYF/cOwTZ+eC/W72XBub8qgKnAocCJ7v5K+PrKgdMJusZXE3Qr3kkwOB6CCQ6fhOtuCusa/s7lECSNq8M2XyI4Rcxu3H0jwXt5NUFy/TPgy+6+YS9ez15pxva7m2BS1CtmVk6wPx2dqHgkdSz4x1dEJDOZ2evAE+7+UAs93pnA/7p7wy6tjGdmtxBMdLkw1bGISNN05ExEMlbY3TMCeHo/HqONmZ0Vdl31JugSf76lYhQR2VtKzkQkI5nZowTdz1eG3V37/FAEXYWbCbo15xCcZyplLDihbUUjtxv3vLaIZDp1a4qIiIikER05ExEREUkjSs5ERERE0kijF/bNRF27dvX+/funOgwRERGRPZo+ffoGd+/WWF3WJGf9+/dn2rSmTo8lIiIikj7MrMlzM6pbU0RERCSNKDkTERERSSNKzkRERETSiJIzERERkTSi5ExEREQkjWTNbE0RERGRfbVsYyUPTlnMCzNXU1lVS3FhHucM78X3Rw2kX5fipMai5ExERERaTDolOc31xrx1XPb4DGrqItRGgstaVlTV8tQHK/jb9FX84cIRnFTWPWnxqFtTREREWsQb89Yx+ndTeOqDFVRU1eLsSnJG/24Kb8xbl+oQP+fuRCLOonXlXPb4DHbU1H2emNWrjTg7auq47PEZLNtYmbTYdORMRERE4opEnMrqWiqr6qioqmV7dS0VVcFyZVVwf+Wm7Tz0zpKYBAeCJKc2Usf3HpnKkf06U5SfQ8SduogT8TBRcoh49LITidSX7ap3jyqLxK67qy66bSPrxobZpJq6CA9NWcJt5xzSgu9q05SciYiIZJn6ZGp7dV2YRO2eTAWJVi0V9ctVtVRW10W121W/PXycFonLYerSTS3yWMlUG3Gen7lKyZmIiEhLyIQxUJGIs71m9+SooqqW7VV1VFZHl9WxPUyuKqp2bx+diLVUMiW7VFbXJu25lJyJiEjWStRA74bJ1PaqXYlRfTIVWxZ9lGpXF6GSqcxQXJC8lEnJmYiIZKVlGys/H+jdUP0YqB88Np3rRpfRpiBv9y696rqorr1dyVR0F6BkPjPwZow9y8sxzh3eO/EB1T9f0p5JREQkCapq61i4roLbX5rDzkYSs2jVdRFue2lOkiLLbG0LcikuzKNdYR7FhbkUF+RRXJgXluXy4fItzFtbHnegfa7BiWXdufi4/uSYYQY5ZuENLPybm2Nx6+vLzCAnJ1zHLLY+h8/Xberxlm2sZPTvpjSaxNfLz83hklEDEvCuNk7JmYiIZKS6iLNsYyXz15Yzb00F89ZuY96acpZu3E7d3kzFy1L1yVRx/d/PE6sgmWpbsOv+53UFebQtzI1qF/xtm59LTo7Ffb7mJDkFebn8/CsHpc1YP4B+XYr5w4UjYrq/IThilp+bwx8uHJHUmM2bczwvA4wcOdKnTZuW6jBERKSFuTtrtu1k3ppy5q8tZ274d8HaCqpqI6kOr8W0yc/dLVmqT47aFuxKlqITq11lUfUFwXLbgjxy95BMJUJjY/xg9yQnmSdz3RvLNlby0JQlPD9zFZXVtRQX5HHu8N5cMmpAQhIzM5vu7iMbrVNyJiIi6WLL9uqYJGzemnK27UzeTLnmik6mdiVLubsdcWrsqFVsWeqSqURIdpKTqZSciYhIWtlRXceCdWECtqaceWuDRGzttqqkxmFAaZe2fHFAl5ijVvXjqqKPWtUnX8VZlExJasRLzjTmTEREEqamLsLSDZW7HQWbt7ac5Zu2N2uW3L46oEMRfTu1ZcbyzY2esb5eUX4uj333KB3RkbSi5ExERPZbJOKs2rLj8+SrPhFbtL6CmrrEZWEd2uRT1rOEoT1LGNKjhLLwb4c2+UDzxkApMZN0o+RMRET2yvryql1HwcJkbMHa8oSe+6soPydIvqISsKE9S+hWUohZ092LJ5V1Z+KVozQGSjKKxpyJiEijynfWMH9txW6J2Py15WysrE7Yc+bmGAO7FlPWM0jEhoRHxfp2arvHUzmIZBKNORMRkSZV1daxaF1leJ6wXcnYqi07Evq8fTu3CRKw8GhYWc8SBnQtpjAvN6HPK5LulJyJiLQSdRFn+abtzFuzKwmbu2Zbwk/a2rVdIWU9233eFTmkRwmDe5TQrlA/QSKN0SdDRCTLuDtrt1Uxd8223c6en+iTtrYrzGNIj3a7dUmW9SihS7vChD2nSDZSciYiksFScdLWgtwcDuzejrIe7Sjr2f7zo2K9O7aJOzhfRJonocmZmY0G7gZygYfcfXyD+n7Aw0A3YBNwobuvDOsuAm4Km/6Puz+ayFhFRJJl2cZKHpyymBdmrqayqpbiwjzOGd6L748a2OTswfqTtkbPkEz0SVvNoH+X4t2OgpX1bEf/LsXk5eYk7HlFWruEzdY0s1xgPnAasBKYCoxx99lRbZ4F/unuj5rZycB33P3bZtYZmAaMBByYDhzp7pubej7N1hSRTLCn8279fsxw+nVpu9tRsPlry1mW4JO29mxf9Pmg/PqxYQd2a0ebAg3OF0mEVM3WPApY6O6LwyCeAs4GZke1OQi4Krz/BvBCeP8MYLK7bwrXnQyMBp5MYLwiIgm1bGMllz0+gx01secDq404tZE6Lnkssf9k1p+0Nfo0FUO6l9ChbX5Cn1dEmi+RyVlvYEXU8krg6AZtPgLOI+j6PBcoMbMuTazbO3Ghiogk3oNTFlNTl7gB+dHqT9oafeLWsp4ldN/DSVtFJPVSPSHgGuBeM7sYeBtYBTT7FNNmdilwKUBpaWki4hMRaTF/m74q7nUe90X9SVt3jQkL/vbt3FYX5hbJUIlMzlYBfaOW+4Rln3P31QRHzjCzdsDX3H2Lma0CTmyw7psNn8DdHwAegGDMWQvGLiLSYhasLWf8y3Mb7c7cG306tdntKNiQHiUM7KaTtopkm0QmZ1OBwWY2gCApuwAYG93AzLoCm9w9AtxAMHMTYBJwu5l1CpdPD+tFRDLGum07+e2r83l66gr25oCZAccc2GW3sWFDdNJWkVYjYZ90d681s3EEiVYu8LC7zzKzW4Fp7v4iwdGxO8zMCbo1fxyuu8nMbiNI8ABurZ8cICKS7iqqanng7cU8+PbivT5alpdjjDmqlNvOOSRB0YlIutOFz0VEWkhNXYSnpq7g7lfns6Fi3y4O3iY/l4lXjmryfGcikh104XMRkQRyd16ZvZY7J85l8frKuG27FBdQvrOGiNPoec7+cOEIJWYirZySMxGR/TB92WbumDCHacuaPEc2ACVFeYw7aRAXHduftdt28tCUJTw/cxWV1bUUF+Rx7vDeXDJqgBIzEVG3pojIvliyoZJfTpzLy5+uidsuP9f4z2P6M+6kQXQqLkhSdCKS7tStKSLSQjZWVHHPawv4y7+X7/GcZV89vBfXnlFG385tkxSdiGQDJWciIs2wo7qOh99dwv1vLqKiqjZu26MHdObGs4ZxeN+OyQlORLKKkjMRkTjqIs7fpq/kN5PnsXZbVdy2g7u344azhnJSWXddIklE9pmSMxGRRrg7b85fz/gJc5m3tjxu2+4lhVx12hC+fmQf8nJzkhShiGQrJWciIg18umort0+Yw78WbYzbrrgglx986UAuGTWAtgX6OhWRlqFvExGR0IpN2/nNK/N44cPVcdvl5hhjjyrlilMG062kMEnRiUhroeRMRFq9rdtruO/NhTzy7lKq6yJx255xcA9+NnooB3Zrl6ToRKS1UXImIq1WVW0dj/1rGfe+sZCtO2rith1R2pEbzxrGyP6dkxSdiLRWSs5EpNWJRJx/fLyaX02ax8rNO+K27d+lLdeNHsroQ3pqBqaIJIWSMxFpVf61aAN3TJjLJ6u2xm3XubiAn5wymLFHl5KvGZgikkRKzkSkVZi3ppzxL8/hjXnr47Yrys/hkuMH8oMvDaSkKD9J0YmI7KLkTESy2pqtO/nt5Pk8O30F8a62ZAbfOLIPPz1tCAd0aJO8AEVEGlByJiJZqXxnDX98azEPvbOYnTXxZ2CeWNaN688cytCe7ZMUnYhI05SciUhWqamL8OQHy7n71QVsrKyO2/aQ3u254cxhHDeoa5KiExHZMyVnIpIV3J1Js9Zw58R5LNlQGbdt745tuPaMMr56eC9ycjQDU0TSi5IzEcl405dt4hcvzWHG8i1x27UvyuPykwfz7WP6UZSfm5zgRET2kpIzEclYi9dX8MuJ85g4a03cdgW5OVx0bD9+fNIgOrYtSFJ0IiL7RsmZiGScDRVV3P3qAp74YDl18aZgAmcf0YtrTi+jb+e2SYpORGT/KDkTkYyxvbqW/5uyhP99axGV1XVx2x4zsAs3njWMQ/t0SFJ0IiItQ8mZiKS9uojz1+kruGvyfNZuq4rbdkiPdtxw5jBOLOumyy2JSEZSciYiacvdeWPeOsa/PJf5ayvitu3RvpCrTyvja0f2IVczMEUkgyk5E5G09PHKLdwxYS7vLd4Yt127wjx++KWBfPf4AbQt0FeaiGQ+fZOJSFpZsWk7v5o0jxc/Wh23XV6OMfboUq44ZTBd2xUmKToRkcRTciYiaWHL9mrufX0hj723jOq6+JdbOvOQnlx7RhkDu7VLUnQiIsmj5ExEUmpnTR2PvbeUe19fyLadtXHbHtmvEzeeNZQj+3VOUnQiIsmn5ExEUiIScf7+0Sp+PWk+q7bsiNt2QNdirhs9lDMO7qEZmCKS9RKanJnZaOBuIBd4yN3HN6gvBR4FOoZtrnf3CWbWH5gDzAubvu/uP0xkrCKSPO8u3MDtE+Ywa/W2uO26FBdw5amDueCoUvJzc5IUnYhIaiUsOTOzXOA+4DRgJTDVzF5099lRzW4CnnH3+83sIGAC0D+sW+TuRyQqPhFJvrlrtnHHhLm8NX993HZF+Tl8f9RALj1hICVF+UmKTkQkPSTyyNlRwEJ3XwxgZk8BZwPRyZkD7cP7HYD407NEJCN9tnUHd70yn7/OWInHudpSjsH5I/vy09OG0KN9UfICFBFJI4lMznoDK6KWVwJHN2hzC/CKmV0OFAOnRtUNMLOZwDbgJnefksBYRSQBtu2s4Y9vLeL/3lnCzpr4MzBPHtqd60YPpaxnSZKiExFJT6meEDAGeMTdf2NmxwB/NrNDgM+AUnffaGZHAi+Y2cHuvtsAFTO7FLgUoLS0NNmxi0gTqmsjPPHvZdzz+kI2VVbHbXto7w7ccNZQjj2wa5KiExFJb4lMzlYBfaOW+4Rl0b4HjAZw9/fMrAjo6u7rgKqwfLqZLQKGANOiV3b3B4AHAEaOHBmns0REksHdefnTNfxy4lyWbtwet22fTm249owyvnJYL3J0uSURkc8lMjmbCgw2swEESdkFwNgGbZYDpwCPmNkwoAhYb2bdgE3uXmdmA4HBwOIExioi+2nq0k3cPmEOM5dviduuQ5t8Lj95EN8+ph+FebnJCU5EJIMkLDlz91ozGwdMIjhNxsPuPsvMbgWmufuLwNXAg2b2U4LJARe7u5vZCcCtZlYDRIAfuvumRMUqIvtu0foK7nx5Lq/MXhu3XUFeDt85tj+XnTiIDm01A1NEpCnm8aZOZZCRI0f6tGnT9txQRFrE+vIq7n5tPk9+sIK6SPzvkfOG9+aq04fQp1PbJEUnIpLezGy6u49srC7VEwJEJMNsr67lwbeX8MDbi6isrovb9rhBXbjhzGEc0rtDkqITEcl8Ss5EpFlq6yI8O30ld02ez/ryqrhth/Ys4YazhnHC4K663JKIyF5SciYicbk7r89dx/iX57JgXUXctj3bF3H16UM4b0QfcjUDU0Rknyg5E5EmfbRiC7dPmMO/l8Sfj9OuMI8fnXgg3z1uAG0KNANTRGR/KDkTkRjLN27nV6/M4x8fxb+iWl6OceEX+3H5yYPo0q4wSdGJiGQ3JWci8rnNldX8/vWF/Pn9pdTUxZ+B+R+HHsC1Z5TRv2txkqITEWkdlJyJCDtr6njkX0u5742FlO+sjdv2C/07ccNZwxhR2ilJ0YmItC5KzkRasUjEeeHDVfx60jxWb90Zt+3AbsVcP3oopx3UQzMwRUQSSMmZSCs1ZcF67pgwl9mfbYvbrmu7Aq48dQgXfKEvebk5SYpORKT1UnIm0srMXr2N8RPn8vb89XHbtcnP5dITBvL9EwbSrlBfFSIiyaJvXJFWYvWWHfzmlfk8N3Ml8a7almPwzS+U8tNTB9O9fVHyAhQREUDJmUjW27azhvvfXMTD7yyhqjYSt+2pw7pz3eihDO5RkqToRESkISVnIlmqujbC4+8v4/evL2Dz9pq4bQ/v04EbzhrGFwd2SVJ0IiLSFCVnIlnG3Xnpk8/45cR5LN+0PW7bvp3b8LMzhvLlww7QDEwRkTSh5Ewki/x78UZuf3kuH63YErddx7b5XHHyYL71xVIK83S5JRGRdKLkTCQLLFxXzviX5/HqnLVx2xXk5fDd4wbwoxMPpEOb/CRFJyIie0PJmUgGW1e+k99OXsDTU5cTiTMD0wzOHd6bq08vo3fHNskLUERE9pqSM5EMVFlVywNvL+bBKYvZXl0Xt+2owV25/syhHNyrQ5KiExGR/aHkTCSD1NZFeHraCn47eQEbKqrith3as4QbzxrGCUO6JSk6ERFpCUrORDKAuzN59lrunDiXResr47Y9oEMRV59exrnDe5OboxmYIiKZRsmZSJqbuXwzd0yYywdLN8VtV1KYx2UnDeI7x/WnKF8zMEVEMpWSM5E0tWxjJb+cOI+XPvksbrv8XOPCL/bj8pMH07m4IEnRiYhIoig5E0kzmyqruee1Bfzl38uoqYszBRP4j8MO4GdnlNGvS3GSohMRkURTciaSJnbW1PHwu0u4/41FlFfVxm17VP/O3HDWUIaXdkpSdCIikixKzkRSrC7iPDdjJXdNns9nW3fGbXtgt2KuP3MYpw7rrsstiYhkKSVnIin01vz13DFhDnPXlMdt17VdIVedNoTzR/YhLzcnSdGJiEgqKDkTSYFZq7cy/uW5TFmwIW67tgW5XHrCQL4/aiDFhfq4ioi0Bvq2F0miVVt28JtJ83j+w1V4nLH+uTnGN7/QlytPHUz3kqLkBSgiIimX0OTMzEYDdwO5wEPuPr5BfSnwKNAxbHO9u08I624AvgfUAVe4+6RExiqSSFt31PCHNxfyp3eXUl0bidv2tIN6cN3oMgZ1L0lSdCIikk4SlpyZWS5wH3AasBKYamYvuvvsqGY3Ac+4+/1mdhAwAegf3r8AOBjoBbxqZkPcPf5FBEXSTFVtHY+/v5zfv76ALdtr4rY9vG9HbjxzKEcP7JKk6EREJB0l8sjZUcBCd18MYGZPAWcD0cmZA+3D+x2A1eH9s4Gn3L0KWGJmC8PHey+B8Yq0mEjE+ecnn/GrSXNZsWlH3Lb9urTlZ2cM5axDe2oGpoiIJDQ56w2siFpeCRzdoM0twCtmdjlQDJwate77DdbtnZgwRVrW+4s3cseEOXy0cmvcdp3a5nPFKYP51tH9KMjTDEwREQmkekLAGOARd/+NmR0D/NnMDmnuymZ2KXApQGlpaYJCFGmeBWvLGf/yXF6buy5uu8K8HL57/AB+dOKBtC/KT1J0IiKSKRKZnK0C+kYt9wnLon0PGA3g7u+ZWRHQtZnr4u4PAA8AjBw5Mv51bkQSZN22nfz21fk8PXUFkTh7oRl8bUQfrjptCL06tklegCIiklESmZxNBQab2QCCxOoCYGyDNsuBU4BHzGwYUASsB14EnjCzuwgmBAwGPkhgrCJ7raKqlgfeXsyDby9mR038uSonDOnG9aOHclCv9nHbiYiIJCw5c/daMxsHTCI4TcbD7j7LzG4Fprn7i8DVwINm9lOCyQEXu7sDs8zsGYLJA7XAjzVTU9JFTV2Ep6au4O5X57Ohojpu24MOaM8NZw1l1OBuSYpOREQynXm8M2FmkJEjR/q0adNSHYZkMXfnldlruXPiXBavr4zbtleHIq45o4xzjuhNTo5mYIqIyO7MbLq7j2ysLtUTAkQywozlm7ljwhymLt0ct11JUR4/PmkQFx/bn6L83CRFJyIi2UTJmUgcSzZU8qtJc5nwyZq47fJzjf88pj/jThpEp+KCJEUnIiLZSMmZSCM2VlTx+9cX8vj7y6iNNwUT+Mrhvbj29DJKu7RNUnQiIpLNlJyJRNlRXcfD7y7h/jcXUVFVG7ft0QM6c+NZwzi8b8fkBCciIq1Cs5IzM3sO+D/gZXePf9VmkQxUF3H+NmMld70ynzXbdsZtO6h7O244cygnD+2uyy2JiEiLa+6Rsz8A3wHuMbNngT+5+7zEhSWSHO7OW/PXM/7lucxdUx63bbeSQq46bQjfOLIPebm63JKIiCRGs5Izd38VeNXMOhBcculVM1sBPAg87u41CYxRJCE+XbWVO16ew7sLN8ZtV1yQyw++dCCXjBpA2wKNBBARkcRq9i+NmXUBLgS+DcwE/gIcD1wEnJiI4EQSYeXm7fzmlfk8PzPmimC7yc0xxh5VyhWnDKZbSWGSohMRkdauuWPOngfKgD8DX3H3z8Kqp81MZ36VjLB1ew33vbmQR95dSnVd/KGTZxzcg5+NHsqB3dolKToREZFAc4+c3ePubzRW0dTZbUXSRVVtHX9+bxm/f30hW3fE74EfUdqRG88axsj+nZMUnYiIyO6am5wdZGYz3X0LgJl1Asa4+x8SFpnIfopEnH98vJpfTZrHys074rbt36Ut140eyuhDemoGpoiIpFRzk7Pvu/t99QvuvtnMvk8wi1Mk7fxr0QbumDCXT1Ztjduuc3EBPzllMGOPLiVfMzBFRCQNNDc5yzUz8/Aq6WaWC+gaNZJ25q8tZ/zLc3l97rq47Yryc7jk+IH84EsDKSnKT1J0IiIie9bc5GwiweD/P4bLPwjLRNLC2m07ueuV+Tw7fQXxrrZkBt84sg8/PW0IB3Rok7wARUREmqm5ydl1BAnZj8LlycBDCYlIZC+U76zhgbcX8+CUxeysiT8D88Syblx/5lCG9myfpOhERET2XnNPQhsB7g9vIilXUxfhqQ+W87tXF7Cxsjpu20N6t+eGM4dx3KCuSYpORERk3zX3PGeDgTuAg4Ci+nJ3H5iguEQa5e5MmrWGOyfOY8mGyrhte3dsw7VnlPHVw3uRk6MZmCIikhma2635J+Bm4LfASQTX2dTUNkmq6cs2cfuEuUxftjluu/ZFeVx+8mC+fUw/ivJzkxSdiIhIy2huctbG3V8LZ2wuA24xs+nAzxMYm7RSyzZW8uCUxbwwczWVVbW0KcilS3EBK/ZwrrKC3BwuOrYfPz5pEB3bajKxiIhkpuYmZ1VmlgMsMLNxwCpA17WRFvfGvHVc9vgMauoi1IbTLrdX17G9On5idvYRvbjm9DL6dm6bjDBFREQSprnJ2U+AtsAVwG0EXZsXJSooaZ2WbazkssdnsKOmrtnrHDOwCzeeNYxD+3RIYGQiIiLJs8fkLDzh7Dfd/RqggmC8mUiLe3DKYmr2cEHyekN6tOOGM4dxYlk3XW5JRESyyh6TM3evM7PjkxGMtG4vzFz9eVdmPIV5Obz8kxPI1QxMERHJQs3t1pxpZi8CzwKfn7/A3Z9LSFTSKlVW1TarXXVdRImZiIhkreYmZ0XARuDkqDIHlJxJi8nLNWrq9nzkrLigubutiIhI5mnuFQI0zkwSasqC9c1KzPJyjHOH905CRCIiIqnR3CsE/IngSNlu3P27LR6RtDrry6v46dMfNattfm4Ol4wakOCIREREUqe5/UP/jLpfBJwLrG75cKS1iUScq575kA0VVXHb5eUY+bk5/OHCEfTrUpyk6ERERJKvud2af4teNrMngXf2tJ6ZjQbuBnKBh9x9fIP6+stBQXAete7u3jGsqwM+CeuWu/tXmxOrZJb731rElAUbYsp7tC+ksqqOyupaigvyOHd4by4ZNUCJmYiIZL19HVk9GOger0F4frT7gNOAlcBUM3vR3WfXt3H3n0a1vxwYHvUQO9z9iH2MTzLAtKWbuGvy/JjyLsUF/GPc8XRvX5SCqERERFKruWPOytl9zNka4Lo9rHYUsNDdF4eP8RRwNjC7ifZjCC6uLq3Alu3VXPHkTOoaOa/ZXd88QomZiIi0Ws3t1izZh8fuDayIWl4JHN1YQzPrBwwAXo8qLjKzaUAtMN7dX9iHGCQNuTs/++vHrN66M6buh186kC8N6ZaCqERERNJDTnMamdm5ZtYharmjmZ3TgnFcAPzV3aMvqtjP3UcCY4HfmdmBjcR1qZlNM7Np69evb8FwJJEe/ddSXpm9NqZ8RGlHrj59SAoiEhERSR/NSs6Am919a/2Cu29hz12Qq4C+Uct9wrLGXAA8GV3g7qvCv4uBN9l9PFp9mwfcfaS7j+zWTUdbMsGnq7Zy+4S5MeXti/K4Z8xw8nObu0uKiIhkp+b+EjbWbk9dolOBwWY2wMwKCBKwFxs2MrOhQCfgvaiyTmZWGN7vChxH02PVJENUVNVy+ZMzqW7k4ua//Pph9OnUNgVRiYiIpJfmztacZmZ3Ecy+BPgxMD3eCu5ea2bjgEkEp9J42N1nmdmtwDR3r0/ULgCecvfokeHDgD+aWYQgMRwfPctTMo+7818vfMqSDZUxdf95TD9GH3JACqISERFJP7Z7TtREI7Ni4L+AUwlmbU4GfuHusb+0KTJy5EifNm1aqsOQJjw7bQXX/vXjmPJhB7Tn+cuOpSg/NwVRiYiIpIaZTQ/H1sdo7mzNSuD6Fo1KWo2F68r5+d9nxZS3Lcjl3rHDlZiJiIhEae5szclm1jFquZOZTUpYVJI1dtbUMe6JmeyoqYupu+3sQziwW7sURCUiIpK+mjshoGs4QxMAd9/MHq4QIAJw2z9nM3dNeUz5eSN687Uj+6QgIhERkfTW3OQsYmal9Qtm1p/drxggEuOljz/jL/9eHlM+sFsxt519SAoiEhERSX/Nna35/4B3zOwtwIBRwKUJi0oy3opN27n+udgJAAV5Odw7ZgTFhft6WVcREZHs1twJARPNbCRBQjYTeAHYkcC4JIPV1EW4/MmZlO+sjan7r/8YxkG92qcgKhERkczQ3AufXwL8hOAs/x8CXyQ4aezJCYtMMtavJ83jwxVbYspHH9yTC7/YL/kBiYiIZJDmjjn7CfAFYJm7n0RwKaUtiQpKMtcb89bxx7cXx5T37tiGO792GGaWgqhEREQyR3OTs53uvhPAzArdfS5QlriwJBOt3baTq5/5KKY8N8e4Z8xwOrTNT0FUIiIimaW5o7JXhuc5ewGYbGabgWWJCkoyT13EufKpD9lUWR1Td83pZRzZr1MKohIREck8zZ0QcG549xYzewPoAExMWFSSce59fSHvLd4YU37CkG784ISBKYhIREQkM+31+Qzc/a1EBCKZ6/3FG7n7tfkx5d1KCrnr/MPJydE4MxERkeZq7pgzkUZtqqzmyqc+JNLglMRm8LtvHkHXdoWpCUxERCRDKTmTfebuXPvsR6zZtjOmbtxJgzhuUNcURCUiIpLZlJzJPvu/d5bw2tx1MeVf6N+Jn5wyOAURiYiIZD4lZ7JPPl65hTsnzo0p79g2n7svGE5ernYtERGRfaFfUNlr23bWMO6JmdTUeUzdr75+OL06tklBVCIiItlByZnsFXfnxuc+Yfmm7TF13zmuP6cd1CMFUYmIiGQPJWeyV56auoJ/fvxZTPkhvdtz/ZlDUxCRiIhIdlFyJs02b005t7w4K6a8XWEe944ZQWFebgqiEhERyS5KzqRZdlTXMe6JGVTVRmLqfnHuIfTvWpyCqERERLKPkjNplv/+xywWrKuIKf/myL6cfUTvFEQkIiKSnZScyR69+NFqnpq6IqZ8UPd23PzVg1IQkYiISPZSciZxLd1QyY3PfRJTXpiXw31jR9C2YK8vzyoiIiJxKDmTJlXV1nH5kzOpqKqNqbv5KwdT1rMkBVGJiIhkNyVn0qQ7X57HJ6u2xpT/x2EHMOaovimISEREJPspOZNGvTp7LQ+/uySmvLRzW+4471DMLAVRiYiIZD8lZxJj9ZYdXPPXj2LK83ON348ZTvui/BREJSIi0jokNDkzs9FmNs/MFprZ9Y3U/9bMPgxv881sS1TdRWa2ILxdlMg4ZZfaughXPvUhW7bXxNRdN3ooh/ftmPygREREWpGETbUzs1zgPuA0YCUw1cxedPfZ9W3c/adR7S8Hhof3OwM3AyMBB6aH625OVLwSuOe1BXywdFNM+clDu/O94wekICIREZHWJZFHzo4CFrr7YnevBp4Czo7TfgzwZHj/DGCyu28KE7LJwOgExirAvxZu4PdvLIwp79G+kF9/43CNMxMREUmCRCZnvYHoM5euDMtimFk/YADw+t6uKy1jQ0UVP3n6Q9x3L88xuPuC4XQuLkhNYCIiIq1MukwIuAD4q7vX7c1KZnapmU0zs2nr169PUGjZLxJxrnrmI9aXV8XUXXHKYL44sEsKohIREWmdEpmcrQKiT4bVJyxrzAXs6tJs9rru/oC7j3T3kd26ddvPcFuvB6Ys5u35scntFwd25vKTB6cgIhERkdYrkcnZVGCwmQ0wswKCBOzFho3MbCjQCXgvqngScLqZdTKzTsDpYZm0sBnLN/PrSfNiyjsXF3D3BcPJzdE4MxERkWRK2GxNd681s3EESVUu8LC7zzKzW4Fp7l6fqF0APOW+a7STu28ys9sIEjyAW909dgqh7Jet22u4/ImZ1EY8pu435x9Oj/ZFKYhKRESkdUvoVavdfQIwoUHZzxss39LEug8DDycsuFbO3bn+uY9ZtWVHTN2lJwzkpLLuKYhKRERE0mVCgCTZ4/9ezsufrokpP7xvR645vSwFEYmIiAgoOWuVZq/exm3/nB1TXlKYx+8vGE5BnnYLERGRVNGvcCtTWVXLuCdnUF0biakb/7XDKO3SNgVRiYiISD0lZ63Mz/8+i8XrK2PKxx5dyn8cdkAKIhIREZFoSs5akedmrORvM1bGlA/tWcLPv3xQCiISERGRhpSctRKL1ldw0wufxpS3yc/l3rHDKcrPTUFUIiIi0pCSs1ZgZ00dlz8xk+3VsVfH+u+zD2ZQ95IURCUiIiKNUXLWCtwxYQ6zP9sWU37OEb34xpF9UhCRiIiINEXJWZab+OkaHn1vWUx5/y5t+Z9zD8VMl2cSERFJJ0rOstjKzdv52V8/iikvyM3h3rEjaFeY0AtEiIiIyD5QcpalauoiXPHkTLbtrI2pu+GsoRzSu0MKohIREZE9UXKWpe6aPJ8Zy7fElJ92UA8uPrZ/0uMRERGR5lFyloXenr+e+99cFFPeq0MRv/r6YRpnJiIiksaUnGWZdeU7ueqZD2PKc3OMe8YMp2PbguQHJSIiIs2m5CyLRCLOT5/+kA0V1TF1V502hJH9O6cgKhEREdkbSs6yyP1vLeLdhRtjyo8f1JUffenAFEQkIiIie0vJWZaYtnQTd02eH1PetV0Bd33zcHJyNM5MREQkEyg5ywJbtldzxZMzqYt4TN1d5x9B95KiFEQlIiIi+0LJWYZzd6559mNWb90ZU/ejEw/khCHdUhCViIiI7CslZxnukX8t5dU5a2PKR5R25KrThqQgIhEREdkfSs4y2KertnLHhLkx5e2L8rhnzHDyc7V5RUREMo1+vTNURVUt456YQXVdJKbuV984nD6d2qYgKhEREdlfSs4ykLvz/57/hKUbt8fUXXRMP844uGcKohIREZGWoOQsAz07fSV//3B1TPlBB7TnhrOGpSAiERERaSlKzjLMwnXl3Pz3WTHlbQtyuXfscIryc1MQlYiIiLQUJWcZZGdNHT/+y0x21NTF1P3POYcwsFu7FEQlIiIiLUnJWQa59Z+zmbe2PKb8ayP6cN6IPimISERERFqakrMM8dLHn/HEv5fHlA/sVsytZx+cgohEREQkERKanJnZaDObZ2YLzez6Jtqcb2azzWyWmT0RVV5nZh+GtxcTGWe6W75xO9f/7eOY8oK8HO4bO4LiwrwURCUiIiKJkLBfdTPLBe4DTgNWAlPN7EV3nx3VZjBwA3Ccu282s+5RD7HD3Y9IVHyZoro2wuVPzqC8qjam7r++fBDDDmifgqhEREQkURJ55OwoYKG7L3b3auAp4OwGbb4P3OfumwHcfV0C48lIv35lHh+t3BpTfuYhPbnw6NIURCQiIiKJlMjkrDewImp5ZVgWbQgwxMzeNbP3zWx0VF2RmU0Ly89JYJxp641563jg7cUx5X06tWH81w7DzFIQlYiIiCRSqgcr5QGDgROBPsDbZnaou28B+rn7KjMbCLxuZp+4+6Lolc3sUuBSgNLS7DqKtGbrTq5+5qOY8rwc454xw+nQJj8FUYmIiEiiJfLI2Sqgb9Ryn7As2krgRXevcfclwHyCZA13XxX+XQy8CQxv+ATu/oC7j3T3kd26dWv5V5AidRHnyqdnsqmyOqbumjPKGFHaKQVRiYiISDIkMjmbCgw2swFmVgBcADScdfkCwVEzzKwrQTfnYjPrZGaFUeXHAbNpJX7/+gLeX7wppvyEId24dNTAFEQkIiIiyZKwbk13rzWzccAkIBd42N1nmdmtwDR3fzGsO93MZgN1wLXuvtHMjgX+aGYRggRyfPQsz2z2/uKN3PPagpjy7iWF3HX+4eTkaJyZiIhINjN3T3UMLWLkyJE+bdq0VIexXzZWVHHWPVNYu61qt3Iz+Mv3jubYQV1TFJmIiIi0JDOb7u4jG6vTFQLSRCTiXPPsRzGJGcDlJw1SYiYiItJKKDlLEw+/u4Q35q2PKT+qf2euOGVwCiISERGRVFBylgY+WrGFOyfOjSnv1Dafu8ccQV6uNpOIiEhroV/9FNu2s4ZxT86gpi527N+vv3E4B3Rok4KoREREJFWUnKWQu3PDc5+wYtOOmLrvHjeAU4b1SEFUIiIikkpKzlLoyQ9W8NLHn8WUH9q7A9edWZaCiERERCTVlJylyNw12/jvf8yKKW9XmMe9Y4dTmJebgqhEREQk1ZScpcD26lrGPTGTqtpITN3t5x1Kvy7FKYhKRERE0oGSsxS45cVZLFxXEVN+wRf68tXDe6UgIhEREUkXSs6S7O8fruKZaStjyof0aMfNXzk4BRGJiIhIOlFylkRLN1Ry43OfxJQX5edw79gRtCnQODMREZHWTslZklTV1jHuyRlUVtfF1N3ylYMZ0qMkBVGJiIhIulFyliTjX57Lp6u2xZR/+bAD+OYX+qYgIhEREUlHSs6SYPLstfzp3aUx5aWd23LHeYdiZskPSkRERNKSkrMEW71lB9f+9aOY8vxc496xwykpyk9BVCIiIpKulJwlUG1dhCuenMmW7TUxddeNHsphfTomPygRERFJa0rOEuh3ry5g2rLNMeWnDO3O944fkIKIREREJN0pOUuQdxdu4L43F8aU92xfxK++cbjGmYmIiEijlJwlwPryKq58+kPcdy/PMbhnzHA6FxekJjARERFJe0rOWlgk4lz1zIesL6+Kqbvy1CEcNaBzCqISERGRTKHkrIX98e3FTFmwIab8mIFd+PFJg1IQkYiIiGQSJWctaPqyzfz6lXkx5V2KC/jdBUeQm6NxZiIiIhKfkrMWsnV7DVc8OZO6iMfU/eb8w+nRvigFUYmIiEimUXLWAtydn/3tI1Zt2RFT94MTBnJiWfcURCUiIiKZSMlZC3j8/WVMmrU2pvyIvh255oyyFEQkIiIimUrJ2X6atXort700J6a8pCiP348ZTn6u3mIRERFpPmUO+6GyqpbLn5hJdW0kpu6XXzuMvp3bpiAqERERyWRKzvbDf/39UxZvqIwpv/CLpZx56AEpiEhEREQynZKzffS36St5bsaqmPKhPUu46T8OSkFEIiIikg0SmpyZ2Wgzm2dmC83s+ibanG9ms81slpk9EVV+kZktCG8XJTLOvbVofQX/9fdPY8rb5Ody79gRFOXnpiAqERERyQZ5iXpgM8sF7gNOA1YCU83sRXefHdVmMHADcJy7bzaz7mF5Z+BmYCTgwPRw3c2Jire5dtbU8eO/zGB7dV1M3a1nH8yg7u1SEJWIiIhki0QeOTsKWOjui929GngKOLtBm+8D99UnXe6+Liw/A5js7pvCusnA6ATG2my/eGkOc9eUx5SfO7w3Xz+yTwoiEhERkWySsCNnQG9gRdTySuDoBm2GAJjZu0AucIu7T2xi3d6JCzW+ZRsreXDKYv46fSU7a2JnZg7oWsxt5xyCmS7PJCIiIvsnkclZc59/MHAi0Ad428wObe7KZnYpcClAaWlpIuLjjXnruOzxGdTURaht5NJMeTnGvWOH064w1W+liIiIZINEdmuuAvpGLfcJy6KtBF509xp3XwLMJ0jWmrMu7v6Au49095HdunVr0eAhOGJ22eMz2FFT12hiBmCGEjMRERFpMYlMzqYCg81sgJkVABcALzZo8wLBUTPMrCtBN+diYBJwupl1MrNOwOlhWVI9OGUxNXWx3ZjR3OGhKUuSFJGIiIhku4QlZ+5eC4wjSKrmAM+4+ywzu9XMvho2mwRsNLPZwBvAte6+0d03AbcRJHhTgVvDsqR6YebqJo+Y1auNOM/PjD3fmYiIiMi+SGh/nLtPACY0KPt51H0HrgpvDdd9GHg4kfHtSWVVbfPaVTevnYiIiMie6AoBcRQ3cyxZcYHGnImIiEjLUHIWxznDe5GXE//0GHk5xrnDU3aWDxEREckySs7i+P6ogeTnxn+L8nNzuGTUgCRFJCIiItlOyVkc/boU84cLR9AmPzfmCFpejtEmP5c/XDiCfl2KUxShiIiIZBslZ3twUll3Jl45ijFHldKuMO/z85qNOaqUiVeO4qSy7qkOUURERLKIBRMmM9/IkSN92rRpqQ5DREREZI/MbLq7j2ysTkfORERERNKIkjMRERGRNKLkTERERCSNKDkTERERSSNKzkRERETSSNbM1jSz9cCyVMexn7oCG1IdhLQIbcvsoW2ZXbQ9s0emb8t+7t6tsYqsSc6ygZlNa2parWQWbcvsoW2ZXbQ9s0c2b0t1a4qIiIikESVnIiIiImlEyVl6eSDVAUiL0bbMHtqW2UXbM3tk7bbUmDMRERGRNKIjZyIiIiJpRMlZGjCzpWb2iZl9aGa6enuGMbOHzWydmX0aVdbZzCab2YLwb6dUxijN08S2vMXMVoWfzw/N7KxUxijNY2Z9zewNM5ttZrPM7CdhuT6bGSbOtszaz6a6NdOAmS0FRrp7Jp+vpdUysxOACuAxdz8kLPslsMndx5vZ9UAnd78ulXHKnjWxLW8BKtz916mMTfaOmR0AHODuM8ysBJgOnANcjD6bGSXOtjyfLP1s6siZyH5y97eBTQ2KzwYeDe8/SvBFImmuiW0pGcjdP3P3GeH9cmAO0Bt9NjNOnG2ZtZScpQcHXjGz6WZ2aaqDkRbRw90/C++vAXqkMhjZb+PM7OOw21PdYBnGzPoDw4F/o89mRmuwLSFLP5tKztLD8e4+AjgT+HHYtSJZwoOxAxo/kLnuBw4EjgA+A36T0mhkr5hZO+BvwJXuvi26Tp/NzNLItszaz6aSszTg7qvCv+uA54GjUhuRtIC14TiJ+vES61Icj+wjd1/r7nXuHgEeRJ/PjGFm+QQ/5n9x9+fCYn02M1Bj2zKbP5tKzlLMzIrDAY6YWTFwOvBp/LUkA7wIXBTevwj4ewpjkf1Q/0MeOhd9PjOCmRnwf8Acd78rqkqfzQzT1LbM5s+mZmummJkNJDhaBpAHPOHuv0hhSLKXzOxJ4ESgK7AWuBl4AXgGKAWWAee7uwaap7kmtuWJBN0mDiwFfhA1ZknSlJkdD0wBPgEiYfGNBGOV9NnMIHG25Riy9LOp5ExEREQkjahbU0RERCSNKDkTERERSSNKzkRERETSiJIzERERkTSi5ExEREQkjSg5ExEREUkjSs5ERPaCmS01s677uO7FZtarJR5LRLKXkjMRkeS5GOi1p0Yi0ropORORjGRm/c1srpk9YmbzzewvZnaqmb1rZgvM7Kjw9p6ZzTSzf5lZWbjuT83s4fD+oWb2qZm1beJ5upjZK2Y2y8weAiyq7kIz+8DMPjSzP5pZblheYWa/Ddd5zcy6mdnXgZHAX8L2bcKHudzMZpjZJ2Y2NJHvmYhkBiVnIpLJBgG/AYaGt7HA8cA1BJd3mQuMcvfhwM+B28P17gYGmdm5wJ8ILvuyvYnnuBl4x90PJrjUWimAmQ0Dvgkc5+5HAHXAt8J1ioFp4TpvATe7+1+BacC33P0Id98Rtt3g7iOA+8O4RaSVy0t1ACIi+2GJu38CYGazgNfc3c3sE6A/0AF41MwGE1x/Lx/A3SNmdjHwMfBHd383znOcAJwXrveSmW0Oy08BjgSmBtdlpg2wLqyLAE+H9x8Hnovz+PV10+ufR0RaNyVnIpLJqqLuR6KWIwTfb7cBb7j7uWbWH3gzqv1goIJ9HwNmwKPufkMz2sa7iHF9zHXoO1lEULemiGS3DsCq8P7F9YVm1gG4h+CoWJdwPFhT3iboLsXMzgQ6heWvAV83s+5hXWcz6xfW5QD1jzkWeCe8Xw6U7MfrEZFWQMmZiGSzXwJ3mNlMdj8q9VvgPnefD3wPGF+fZDXiv4ETwm7T84DlAO4+G7gJeMXMPgYmAweE61QCR5nZp8DJwK1h+SPA/zaYECAishtzj3e0XURE9paZVbh7u1THISKZSUfORERERNKIjpyJiABm9h3gJw2K33X3H6ciHhFpvZSciYiIiKQRdWuKiIiIpBElZyIiIiJpRMmZiIiISBpRciYiIiKSRpSciYiIiKSR/w+BxnGWmCQ3tQAAAABJRU5ErkJggg==\n",
      "text/plain": [
       "<Figure size 720x360 with 1 Axes>"
      ]
     },
     "metadata": {
      "needs_background": "light"
     },
     "output_type": "display_data"
    }
   ],
   "source": [
    "show_curve(param_grid[\"classifier__max_depth\"],\n",
    "           grid_search.cv_results_['mean_test_score'],\n",
    "           \"max_depth\",\n",
    "           \"accuracy\",\n",
    "           \"Mean score with CV varying max_depth of Decision Tree\")"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 33,
   "id": "7cf40df3",
   "metadata": {
    "execution": {
     "iopub.execute_input": "2022-03-16T15:13:41.167678Z",
     "iopub.status.busy": "2022-03-16T15:13:41.165905Z",
     "iopub.status.idle": "2022-03-16T15:13:41.169376Z",
     "shell.execute_reply": "2022-03-16T15:13:41.169970Z",
     "shell.execute_reply.started": "2022-03-14T13:23:47.050931Z"
    },
    "papermill": {
     "duration": 0.108254,
     "end_time": "2022-03-16T15:13:41.170125",
     "exception": false,
     "start_time": "2022-03-16T15:13:41.061871",
     "status": "completed"
    },
    "tags": []
   },
   "outputs": [],
   "source": [
    "# Define the final model with the best parameter found\n",
    "best_max_depth = grid_search.best_params_[\"classifier__max_depth\"]\n",
    "tree_model = DecisionTreeClassifier(criterion=\"entropy\", random_state=random_seed, max_depth=best_max_depth)"
   ]
  },
  {
   "cell_type": "markdown",
   "id": "1fef0adc",
   "metadata": {
    "papermill": {
     "duration": 0.100939,
     "end_time": "2022-03-16T15:13:41.372354",
     "exception": false,
     "start_time": "2022-03-16T15:13:41.271415",
     "status": "completed"
    },
    "tags": []
   },
   "source": [
    "## **Random Forest**"
   ]
  },
  {
   "cell_type": "markdown",
   "id": "6ad189e6",
   "metadata": {
    "papermill": {
     "duration": 0.102151,
     "end_time": "2022-03-16T15:13:41.575409",
     "exception": false,
     "start_time": "2022-03-16T15:13:41.473258",
     "status": "completed"
    },
    "tags": []
   },
   "source": [
    "A random forest is a meta estimator that fits a number of decision tree classifiers on various sub-samples of the dataset and uses averaging to improve the predictive accuracy and control over-fitting.\n",
    "\n",
    "In random forests each tree in the ensemble is built from a sample drawn with replacement from the training set.\n",
    "Furthermore, when splitting each node during the construction of a tree, the best split is found from a random subset of the input features.\n",
    "The purpose of these two sources of randomness is to decrease the variance of the forest estimator, sometimes at the cost of a slight increase in bias."
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 34,
   "id": "9b258749",
   "metadata": {
    "execution": {
     "iopub.execute_input": "2022-03-16T15:13:41.780795Z",
     "iopub.status.busy": "2022-03-16T15:13:41.780150Z",
     "iopub.status.idle": "2022-03-16T15:13:41.806572Z",
     "shell.execute_reply": "2022-03-16T15:13:41.807112Z",
     "shell.execute_reply.started": "2022-03-14T13:23:47.052018Z"
    },
    "papermill": {
     "duration": 0.131253,
     "end_time": "2022-03-16T15:13:41.807299",
     "exception": false,
     "start_time": "2022-03-16T15:13:41.676046",
     "status": "completed"
    },
    "tags": []
   },
   "outputs": [],
   "source": [
    "from sklearn.ensemble import RandomForestClassifier"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 35,
   "id": "65b3e87d",
   "metadata": {
    "execution": {
     "iopub.execute_input": "2022-03-16T15:13:42.014656Z",
     "iopub.status.busy": "2022-03-16T15:13:42.014072Z",
     "iopub.status.idle": "2022-03-16T15:18:54.022611Z",
     "shell.execute_reply": "2022-03-16T15:18:54.021976Z",
     "shell.execute_reply.started": "2022-03-14T13:23:47.052956Z"
    },
    "papermill": {
     "duration": 312.1135,
     "end_time": "2022-03-16T15:18:54.022878",
     "exception": false,
     "start_time": "2022-03-16T15:13:41.909378",
     "status": "completed"
    },
    "tags": []
   },
   "outputs": [
    {
     "name": "stdout",
     "output_type": "stream",
     "text": [
      "Fitting 2 folds for each of 6 candidates, totalling 12 fits\n",
      "[CV] classifier__n_estimators=5 ......................................\n"
     ]
    },
    {
     "name": "stderr",
     "output_type": "stream",
     "text": [
      "[Parallel(n_jobs=1)]: Using backend SequentialBackend with 1 concurrent workers.\n"
     ]
    },
    {
     "name": "stdout",
     "output_type": "stream",
     "text": [
      "[CV] ....................... classifier__n_estimators=5, total=   4.3s\n",
      "[CV] classifier__n_estimators=5 ......................................\n"
     ]
    },
    {
     "name": "stderr",
     "output_type": "stream",
     "text": [
      "[Parallel(n_jobs=1)]: Done   1 out of   1 | elapsed:    4.3s remaining:    0.0s\n"
     ]
    },
    {
     "name": "stdout",
     "output_type": "stream",
     "text": [
      "[CV] ....................... classifier__n_estimators=5, total=   4.3s\n",
      "[CV] classifier__n_estimators=10 .....................................\n",
      "[CV] ...................... classifier__n_estimators=10, total=   6.1s\n",
      "[CV] classifier__n_estimators=10 .....................................\n",
      "[CV] ...................... classifier__n_estimators=10, total=   5.8s\n",
      "[CV] classifier__n_estimators=20 .....................................\n",
      "[CV] ...................... classifier__n_estimators=20, total=   8.6s\n",
      "[CV] classifier__n_estimators=20 .....................................\n",
      "[CV] ...................... classifier__n_estimators=20, total=   8.6s\n",
      "[CV] classifier__n_estimators=50 .....................................\n",
      "[CV] ...................... classifier__n_estimators=50, total=  17.2s\n",
      "[CV] classifier__n_estimators=50 .....................................\n",
      "[CV] ...................... classifier__n_estimators=50, total=  17.1s\n",
      "[CV] classifier__n_estimators=100 ....................................\n",
      "[CV] ..................... classifier__n_estimators=100, total=  31.4s\n",
      "[CV] classifier__n_estimators=100 ....................................\n",
      "[CV] ..................... classifier__n_estimators=100, total=  31.5s\n",
      "[CV] classifier__n_estimators=150 ....................................\n",
      "[CV] ..................... classifier__n_estimators=150, total=  45.7s\n",
      "[CV] classifier__n_estimators=150 ....................................\n",
      "[CV] ..................... classifier__n_estimators=150, total=  45.5s\n"
     ]
    },
    {
     "name": "stderr",
     "output_type": "stream",
     "text": [
      "[Parallel(n_jobs=1)]: Done  12 out of  12 | elapsed:  3.8min finished\n"
     ]
    }
   ],
   "source": [
    "# Construct the PCA-pipeline using the selected classifier\n",
    "classifier = RandomForestClassifier(criterion=\"entropy\", random_state=random_seed)\n",
    "pipe = build_PCA_pipeline(classifier, pca_n_components)\n",
    "\n",
    "# Define the parameter grid to use in the the grid search with cross validation.\n",
    "# For the random forest we will consider the hyper-parameter which controls the number of trees to use.\n",
    "param_grid = {\n",
    "    \"classifier__n_estimators\": [5,10,20,50,100,150]\n",
    "}\n",
    "\n",
    "# Perform the search on the training set\n",
    "grid_search = GridSearchCV(pipe, param_grid, cv=2, verbose=2)\n",
    "grid_search.fit(X_train, Y_train);"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 36,
   "id": "b6e15c1b",
   "metadata": {
    "execution": {
     "iopub.execute_input": "2022-03-16T15:18:54.239667Z",
     "iopub.status.busy": "2022-03-16T15:18:54.239048Z",
     "iopub.status.idle": "2022-03-16T15:18:54.244336Z",
     "shell.execute_reply": "2022-03-16T15:18:54.244899Z",
     "shell.execute_reply.started": "2022-03-14T13:23:47.054716Z"
    },
    "papermill": {
     "duration": 0.114862,
     "end_time": "2022-03-16T15:18:54.245060",
     "exception": false,
     "start_time": "2022-03-16T15:18:54.130198",
     "status": "completed"
    },
    "tags": []
   },
   "outputs": [
    {
     "name": "stdout",
     "output_type": "stream",
     "text": [
      "Best parameters set found on train set:\n",
      "{'classifier__n_estimators': 150}\n",
      "\n",
      "\n",
      "Grid scores on train set:\n",
      "{'classifier__n_estimators': 5} mean: 0.926 std: +/- 0.001)\n",
      "{'classifier__n_estimators': 10} mean: 0.934 std: +/- 0.001)\n",
      "{'classifier__n_estimators': 20} mean: 0.937 std: +/- 0.001)\n",
      "{'classifier__n_estimators': 50} mean: 0.940 std: +/- 0.000)\n",
      "{'classifier__n_estimators': 100} mean: 0.940 std: +/- 0.000)\n",
      "{'classifier__n_estimators': 150} mean: 0.940 std: +/- 0.000)\n",
      "\n",
      "\n"
     ]
    }
   ],
   "source": [
    "print_grid_search_results(grid_search)"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 37,
   "id": "8291ca52",
   "metadata": {
    "execution": {
     "iopub.execute_input": "2022-03-16T15:18:54.467752Z",
     "iopub.status.busy": "2022-03-16T15:18:54.466158Z",
     "iopub.status.idle": "2022-03-16T15:18:54.676057Z",
     "shell.execute_reply": "2022-03-16T15:18:54.676591Z",
     "shell.execute_reply.started": "2022-03-14T13:23:47.055757Z"
    },
    "papermill": {
     "duration": 0.322751,
     "end_time": "2022-03-16T15:18:54.676799",
     "exception": false,
     "start_time": "2022-03-16T15:18:54.354048",
     "status": "completed"
    },
    "tags": []
   },
   "outputs": [
    {
     "data": {
      "image/png": "iVBORw0KGgoAAAANSUhEUgAAAm4AAAFOCAYAAAA/7JG4AAAAOXRFWHRTb2Z0d2FyZQBNYXRwbG90bGliIHZlcnNpb24zLjUuMSwgaHR0cHM6Ly9tYXRwbG90bGliLm9yZy/YYfK9AAAACXBIWXMAAAsTAAALEwEAmpwYAABCCUlEQVR4nO3de5wcVZ338c93brmHQBICySQhQECjcgkR5NEI4g105b4KK6vgIrrIeoNnF1ZfLov6oLuoeAFWdFERV8QgGF0EEcJNRcmFAAECIRAyk0BCSAK5zu33/FFnQs9Mz6QnmZ6env6+X69+perUqepfVVe6f3POqSpFBGZmZmY28FWVOgAzMzMzK4wTNzMzM7My4cTNzMzMrEw4cTMzMzMrE07czMzMzMqEEzczMzOzMuHEzaxMSZotaWkPy/eTFJJq+jOuUpD0YUm/L3UcxSDpd5I+Wuo4iknSMEm/kbRR0i9LHEvF/L+x8uTEzfqdpOckNUka16l8UfrC3K9EoZWViLg/Ig5un0/H9V27s01JR0q6TdIGSS9L+qukcyRNktQi6YA869wi6Yrded/dFRE/i4j3lDKGviDpUkk35JZFxAkR8ZMivNdASlBOByYAYyPibzsvTMelWdKmdG7+SdLR/R9m30r/Z7em/Wp/TezH979H0rn99X7WN5y4Wak8C5zZPiPpTcDw0oVTPAPkh3Gn0g/h3cC9wIHAWOAfgRMiohG4C/j7TuvsBbwP6PPEotP7lMUxrCR9/JlMBZ6KiJYe6vwiIkYC44B5QElb5vrQByJiZM5rVW9W9v+NChQRfvnVry/gOeCLwEM5ZVcAXwAC2C+VDUnlzwMvAv8FDEvL9gR+C6wF1qfp+pzt3QN8Gfgj8Crwe2BcN/GMS+tvAF4G7geq0rLJwK/S+6wDvpfKq9I+rADWANcDe6Rl+6X9+IcU+32p/GPAEyneO4Cp3cTzE+DCND0pbetTaf6AFGMVcCzQkMp/CrQBW4FNwD/nxPHRFMdLwBd6+FweAK7qYfnfAc90KjsfWNRN/d8BF3QqWwycmqa/DawEXgEWALNz6l0KzAFuSMu/CGwha5FprzMzfS61wNnAAznLAvgk8HT6XK8ClJZVA99Ix+NZ4IJUv6aH8/Ui4BFgI/ALYGgB5/nfAA+n9/8TcEjOsn8BGsnOzaXAO4HjgSagOX2Gi3PO5XPT9Nlk5/S30naXA/8nla8kOxc/mvM+7wcWpWO4Erg0Z9nzab83pdfR9PK8Boamz2hdiuchYEI3x+P1aV82AEuAE1P5v3fa73/Is+6lwA058zNSLOPT/JHAn9O2VwPfA+p6cT5ckc6H5cCncs8HYCIwl+z/3TLg453i+mU6Bq8CjwIHAZek47cSeM9Ovgvflad8CHAlsCq9rgSGpGXHAg1k59ALZP/3q4CLgWfSZ3ETsFeqn/czAr4KtALb0nH/3u5+t/vVP6+SB+BX5b3av6zIfrBen744G8j+6s5N3L6VvjD3AkYBvwEuT8vGAqeRtdKNSl+et+a8xz3pS+wgYFia/1o38VxOlhTWptdsQCmuxSmOEekL8G1pnY+lL/H9gZFkyd1P07L90n5cn9YbBpyU6r8eqCH7cfxTN/F8DPhNmv67tB+/yFn26zR9LClxyz2uOfPtcfwgxXAosB14fZ73HJ6+xN/Rw+c2jCxxeVtO2Z+Bz3ZT/yPAH3PmZ6QfjvYfoLPS51gDXEj2IzQ0LbuU7If8ZLIfpWHAbcA/5mzvW8B30/TZdE3cfguMAaaQJXjHp2WfBB4H6sn+APgDO0/c/kr2A74XWfL9yZ2c44eT/XAflc6jj6btDAEOJvtBn5jzOR2Qs983dNrWPXRM3FqAc9J2v0KWRF2Vtv0esgRiZM458qZ0DA8h+wPo5E7nR03Oe/X2vP4E2f/L4SmeI4DReY5HbdruvwJ1wHEpzoO72+9O6+9Yntb/Glmi1Z5cHQG8hexc2i99Rp/NWX9n58OTZH+k7UXWmpebuN0HXE32//+wtO5xOXFtA96b3vt6sj8GvpD2+ePAszv7LsxTfhnwILA3MJ4s8f9yzmfaAnw9febDgM+k+vWp7PvAz1P9bj8jcs4tv8rnVfIA/Kq8F68lbl8kS5qOB+5MX3yRvngFbCb9oKX1ju7uSzB9oa7Pmb8H+GLO/PnA7d2sexnwa+DATuVHpy/pLj/oZN2G5+fMH0yWaLT/cASwf87y35HTkkD2Q7qFPK1uZK1q61Od/0pfvO0taz8BPp+mj6WwxC23JfKvwBl53rO9Ze91O/nsfghcm6ank7WU7N1N3VHpM5ya5r8KXNfDttcDh6bpS0ktlTnLP0RKBNMP0AvAkWn+bLombrkJ5k3AxWn6buATOcvexc4Tt7Ny5v8D+K+dHKdrSD+0OWVLgWPIuqHXpPet7VTnUnaeuD2ds+xNKfYJOWXrgMO6ietK4Fudzo/cxK235/XH6NSa2M37zk6fV1VO2c9JLYD59jvPcWkiS/xb0z4e20P9zwK39OJ8+GTOsve0HxeyZK4VGJWz/HLgxzlx3Zmz7ANkrVfVOf8HAhjTw7m1Ke3XBtIfn2R/rL0vp957gefitf/3TeS0+pIlqu/Mmd8353Pr9jPCiVtZvjzGzUrpp2QtSmeT/aWaazzZX4gL0mDkDcDtqRxJwyV9X9IKSa+Q/VU8RlJ1zjZeyJneQtaCkM9/krUG/F7SckkXp/LJwIrIP+5mIll3UrsVZF+SE3LKVuZMTwW+nbMvL5Mlp5M6bzginiFLeA4j+8H7LbBK0sFkP/z3drMf3SnkOKwn62rddyfb+gnwt5KGko13uyMi1uSrGBGvAv8LnJGKzgR+1r5c0kWSnkhXEm4A9iDrtm6Xe/wgS65nSJoGvBvYGBF/7SHW7vZ7Yqdtd36f3myrO1OBC9s/77R/k8la2ZaRJRaXAmsk3djLAekv5kxvBYiIzmUjASQdJWmepLWSNpK1LnW4KKiT3p7XPyXr9r9R0ipJ/yGptpvtroyItk7b7nL+9+CmiBiTYnmMrOUIAEkHSfqtpBfS98H/o+t+Fno+5O7/RODldC53F3fnY/9SRLTmzEPP58vJETEmvU7Oed/On0PuObI2IrblzE8Fbsk5154gSzgnUPhnZGXCiZuVTESsIOtWeB9Zl0yul8i+9N6Q86W2R2SDkyHrWjsYOCoiRgNvT+XahThejYgLI2J/4ETg85LeSfZlPqWbwb+ryL4s200h677I/RKPnOmVZK08Y3JewyLiT92EdS/ZlXZ1kV0YcC9Zd9ueZOOm8u5Kjzvag4jYQtbtedpOqj5AlnSeRNbVubOLEn4OnJkufBhK1g2FpNlk4/A+COyZfpA30vHz67A/6YfqpvS+f0/2g7QrVpN1KbWbvIvb6clK4KudPu/hEfFzgIj4n4h4G68ND/h6Wm+XP8Nu/A/ZcIPJEbEHWQtu+zHO9169Oq8jojki/j0iZpCNtfsbsi7yfNudLCn3N2cK2Ti/XomIl4DzgEsltf+hcQ1Zd+f09H3wrxT+XbCajufAlE5x7yVp1O7G3Uv5PofcixY6f3YryS4iyj3fhkZE404+o74+36wfOHGzUvsHsvEim3ML01/mPwC+JWlvgHRLivemKqPIErsN6crGf9vVACT9jaQDJYkseWgla336K9mX+tckjZA0VNJb02o/Bz4naZqkkWR/4f+im9Y5yH4wL5H0hvSee0jqctuDHPeSDZq/L83fk+YfyPlrvrMXycYm7ap/Bs6W9H8ljU1xHirpxvYKEdE+xunrZOOFfrOTbd5G9gN0GdnxaW9xGUWWEKwFaiR9CRhdQIzXk7XQnsiuJ243AZ9J59MYskHefe0HwCdTi5fS+fN+SaMkHSzpOElDyMZHbSU73yD7DPfrlODsjlFkLUbbJB1J1sLdbm1639xzplfntaR3SHpTaul+hax7ri1P1b+QtXL9s6RaSceSdSvemKfuTkXEUrJWpH/O2c9XgE2SXkd2NXShbgI+Lale0p5kg/zb32clWTfj5en//yFk31k35N9Un/k58EVJ45XdNulLO3nP/wK+KmkqQFrvpDTd02e0u98ZVgJO3KykIuKZiJjfzeJ/IevCfDB1f/yBrJUNsrE6w8ha5h4k60bdVdPTtjeRtTpdHRHzUoL0AbIxSc+TXUDxobTOdWSJw31krYbbgH/q7g0i4hayZOfGtC+PASf0ENO9ZD9G7YnbA2Rdx/d1u0Y29uaLqbvkoh7qdRfjn8gGjR8HLJf0MnAtWfKV63qyFoBfRMT2nWxzO1lr6rvIWn/a3UH2mT1F1g20jQK6LCPij2Q/OgtTi+2u+AHZVcaPkF1xeRtZEtldQtxr6Zz+ONnVjevJzuOz0+IhvDa4/gWyAeiXpGXtt7hYJ2lhH4RyPnCZpFfJfvxvyolxC9m4wz+mc+Yt9PK8BvYhu/r3FbLuuXvJk1BHRBPZ/6UTyPb7auAjEfHkbuzbfwLnpT/sLiJLSl8l+3x/0Yvt/IDsfFwMLKRr6/+ZZOP7VgG3AP8WEX/YjbgL8RVgPtk5+miK6ys91P82Wcvq79Nn/SDZhTHQ82f0beB0Seslfaevd8KKo/1yaDOzsiDpbuB/IuKHfbS9E8guNpi608pmZiXmFjczKxuS3kx2/7betKh03sYwSe+TVCNpElk3+y19FaOZWTE5cTOzsiDpJ2Rd2p/tdJVfrzdFdtPX9WRdpU+QdSP2Np5/VcdHFbW/frcbsZmZ9chdpWZmZmZlwi1uZmZmZmXCiZuZmZlZmch3Y9FBZ9y4cbHffvuVOgwzMzOznVqwYMFLETE+37KKSNz2228/5s/v7lZhZmZmZgOHpG7vU+muUjMzM7My4cTNzMzMrEw4cTMzMzMrE07czMzMzMqEEzczMzOzMlERV5WamZmZ7YoV6zbzg/uXc+uiVWze3sKIITWcfPhEPj57f6aOHdHv8ThxMzMzM8tj3tI1nH/DQppb22hpyx4Ruml7Czf+dSU3L2jk6rNm8o6D9+7XmNxVamZmZtbJinWbOf+GhWxtbt2RtLVraQu2Nrdy/g0LWbFuc7/G5RY3MzMzK7rWtqC5tS1rvWpN021Bc0sbLW1tNLUELW1tqU7sqNeU/s3qZC1fHeu00dQatLS+tm5LWxvNLUFzW5pv7Wa7OfV2xJT+3by9mZa2nvepubWNH97/LF8++Y39cxApcuIm6Xjg20A18MOI+Fqn5VOB64DxwMvAWRHRkLN8NPA4cGtEXJDKjgB+DAwDbgM+ExEdU2EzMxtwBtpYoXLU1taecGQJT3eJyWtlKXnpTcLT2jHBamrtmtTkT3hee9+mHclTSs5a2xiMv9QtbcEtixoHR+ImqRq4Cng30AA8JGluRDyeU+0K4PqI+Imk44DLgb/PWf5l4L5Om74G+DjwF7LE7Xjgd8XZCzMz6wsDaaxQW1vHlpgdLTp5WmHaE56OyUtOYpTTYlRwwtOlNanTujuSstfqtS9rG4TJT7nb3NTSr+9XzBa3I4FlEbEcQNKNwElkLWjtZgCfT9PzgFvbF6SWtQnA7cCsVLYvMDoiHkzz1wMn48TNzGzAaG5tY2tzK9uaW9nW1MYzazfxyRsWsD1Pv1NLW9DS1sp518/n3LdNY+TQ2k7JSk/dZ10TnvautC7LWl9rqWp19mN9aERd/446K+a7TQJW5sw3AEd1qrMYOJWsO/UUYJSkscB64BvAWcC7Om2zIWe+IZWZmVkPIrLEZVtTllS1J1Zbm1vZ1tQ+n7Os6bXlO5Kw5ja2Nr1Wtj1n+damth3znQdyF6K5Nbjm3uVF2HOz4qmpEqcc3r9pSKkvTrgI+J6ks8m6RBuBVuB84LaIaJC0SxuWdB5wHsCUKVP6JFgzs77W1hZsb2nrkEhtbcpJlHITqZbWTolTx0RqW5dttHVIvgbjGCMrL3XVVdRWi5rqKmrTdG11FTXVoi79W1NVtWO6Y500XVVFbU2qV1NFTVW2vbp8261SqvNaWe2ObbfXqaIuba+2poraqqx89catnHbNn9na3Nrt/tRWV3Hu7Gn9eASLm7g1ApNz5utT2Q4RsYqsxQ1JI4HTImKDpKOB2ZLOB0YCdZI2kbXM1fe0zZxtXwtcCzBr1ix/XZlZr7S2RY+J1NamVrZ3SqR2JEpNhSVS7ds0643OSU1NSmRqq3KTko4JT111x8SkJieJ6ZBIVYnamvwJT5ft5iY8eZOwnOkqUV0ldrUxphT2HFHH1WfN7DI2E7KWttrqKq4+a2a/X1hTzMTtIWC6pGlkydUZwN/lVpA0Dng5ItqAS8iuMCUiPpxT52xgVkRcnOZfkfQWsosTPgJ8t4j7YGYDzI7xU01dE6ltLa91+3WXSO34N2f9fOVNrU6oBrMdCU+nVpjcpKim+rXWl87LchOe9sTktaSoayJTaMKzI5YdiVP7drN65Zb8lLt3HLw3t392Nj+8/1luWdTI5qYWRtTVcMrhkzh39rTB9eSEiGiRdAFwB9ntQK6LiCWSLgPmR8Rc4FjgcklB1lX6qQI2fT6v3Q7kd/jCBKswA/GWCrsyfqqnRKpra1XbjvldGT9l/atKMKy2mmF11QytreaVrc28sq3nK+8EvG6fURxz8N5dW25yk5cadWoF6pR85SY8XVqUXus+c/JjhZo6dgRfPvmN/XrLj56oEm6BNmvWrJg/f36pwzDbbfluqQAdm+1zb6ng8VOWq7ZaDK2pZmhddZZY1VYztLaKoSnJyuazV5Z4VTG05rUErH35sLqqnDrVXerUVndMjFas28zxV97f41ihYbXV3P7Z2b6fmxkgaUFEzMq3rNQXJ5hZgXIfv9JZ+y0VPvajhxg3so7mND7L46fKQ11NVY+JVIeEqZtEqmvSlf07pPa1ZbXVpXnK4dSxIwbkWCGzcuTEzawMtLS2cencJWzrocUCIIC1m5r6J6gKMLQ2J6HKaVl6LWGqZmhNVZ7WqqodyVXnRGpHYpbKhtRUU101+LvtBuJYIbNy5K5SswHsyRde4eYFDdyyaBUvbdpe6nAGhNzxU0NyEqnc1qVuu/1y5/N0+3VMqKo8DsrMSsJdpWZl5OXNTcx9uJE5Cxt4rPGVUodTsGKMn+owjiptr67aCZWZVS4nbmYDQHNrG/csXcucBSu5+8k1NLf2XUt4+/ipoTljnYbl6fYbmtNa1TnBylqzXkvIBtL4KTOzSuLEzayElqzayM0LGvn1w42s27z7Y9Oqq+CkQyfxhfe/fkfyVQnjp8zMKoUTN7N+9tKm7dy6qJGbFzbyxOrCu0JrqiACemqMq6uu5jPvms7YkUP6IFIzMxtonLiZ9YOmljbufvJF5ixo5J6la3p1E9k3TdqD04+o5wOHTmRxwwbfUsHMrII5cTMrkojgscZXmLNgJXMXr2L9luaC1x0/aginHD6J02bWc/A+o3aU+5YKZmaVzbcDMetja17Zxq0PNzJnQQNPvbip4PXqqqt494wJnH5EPbOnj6PGg/3NzCqSbwdiVmTbmlu564k1zFmwkvuefonWXnSFHjp5TNYVesi+jBleV8Qozcys3DlxM9tFEcHiho1ZV+jDq3b6EO1cE0YP4ZTD6zn9iEkcuPeona9gZmaGEzezXnth4zZuWdTInAUreWbt5oLXG1JTxXvfsA+nHVHP2w4c59t0mJlZrzlxMyvAtuZWfv/4i8xZ0MADT6+lFz2hHDF1T04/op73H7Ivo4fWFi9IMzMb9Jy4mXUjIlj4/HrmLGjkt4+s4tVedIVO3GMop86s59SZk9h//MgiRmlmZpXEiZtZJ40btnLLwgZuXtjIsy8V3hU6tLaKE964L6cfUc/R+4+lyl2hZmbWx5y4mQFbm1q5fclq5ixo4E/PrKM3d8k5ctpenD6znhPetA+j3BVqZmZF5MTNKlZE8NBz65mzYCW3PfoCm7YX3hVav+cwTp1Zz2kzJ/mmt2Zm1m+cuFnFWfnyFn61sJGbFzbw/MtbCl5veF0173vTvpw2s56jpu3lrlAzM+t3TtysImze3sJtj67m5oUNPLj85V6te/T+YzntiHpOeOM+jBji/zJmZlY6/hWyQautLXjw2XXcvKCR3z22mi1NrQWvO2Wv4Zx+RD2nHD6JyXsNL2KUZmZmhXPiZoPOinWbuXlhIzcvaKBxw9aC1xs5pIb3v2lfTp9Vz6ypeyK5K9TMzAYWJ242KLy6rTnrCl3QyF+fK7wrVIK3HjCO04+o571v2IdhddVFjNLMzGz3OHGzstXaFvz5mXXMWbCS25e8wLbmtoLX3X/cCE5LXaETxwwrYpRmZmZ9x4mblZ3lazdx88IGblnYyKqN2wpeb9TQGj5w6EROm1nPzClj3BVqZmZlx4mblYWNW5v530dWM2fBShY+v6Hg9aoEs6eP5/Qj6nn3jAkMrXVXqJmZlS8nbjZgtbYFDyx7iTkLGrhjyQs0tRTeFXrg3iN3XBU6YfTQIkZpZmbWf4qauEk6Hvg2UA38MCK+1mn5VOA6YDzwMnBWRDSk8luAKqAW+G5E/Fda50zgX4EAVqV1Xirmflj/WrbmVeYsaOSWRQ28+Mr2gtfbY1gtJx46kdOPqOeQ+j3cFWpmZoNO0RI3SdXAVcC7gQbgIUlzI+LxnGpXANdHxE8kHQdcDvw9sBo4OiK2SxoJPCZpLrCGLBGcEREvSfoP4ALg0mLth/WPDVua+M3iVcxZ2MjilRsKXq+6ShxzUNYV+s7X782QGneFmpnZ4FXMFrcjgWURsRxA0o3ASUBu4jYD+HyangfcChARTTl1hpC1vAEovUZIWgeMBpYVKX4rspbWNu57ei1zFjTwh8fX0NRaeFfowRNGcfoR9Zx0+ET2HuWuUDMzqwzFTNwmAStz5huAozrVWQycStaKdgowStLYiFgnaTLwv8CBwP+NiFUAkv4ReBTYDDwNfKqI+2BFsPSFV5mzYCW3LFrFS5sK7wrdc3gtJx02idOPqOcNE0e7K9TMzCpOqS9OuAj4nqSzgfuARqAVICJWAodImgjcKmkO2Ti4fwQOB5YD3wUuAb7SecOSzgPOA5gyZUrRd8R69vLmJuY+3MjNCxt5tHFjwevVVIl3vG5vTptZz3Gv25u6mqqdr2RmZjZIFTNxawQm58zXp7IdUivaqQBpLNtpEbGhcx1JjwGzgRWp7Jm0zk3AxfnePCKuBa4FmDVrVuz+7lhvNbe2cc/StcxZsJK7n1xDc2vhH8OMfUdz+hH1nHjYRMaNHFLEKM3MzMpHMRO3h4DpkqaRJWxnAH+XW0HSOODliGgjazm7LpXXA+siYqukPYG3Ad8C1gEzJI2PiLVkFz48UcR9sF3w+KpXmLOggV8/3Mi6zU07XyEZO6KOkw+fxGkz65kxcXQRIzQzMytPRUvcIqJF0gXAHWS3A7kuIpZIugyYHxFzgWOByyUFWVdp+3i11wPfSOUCroiIRwEk/Ttwn6Rmsha4s4u1D1a4lzZt59cPr2LOggaeWP1KwevVVot3vm4Cpx9RzzEHj6e22l2hZmZm3VHE4O9FnDVrVsyfP7/UYQw6TS1t3P3ki8xZ0Mg9S9fQ0lb4uXRI/R6cNrOeEw+dyJ4j6ooYpZmZWXmRtCAiZuVbVuqLE6zMRASPNb7CzQuzrtD1W5oLXnf8qCGckrpCD95nVBGjNDMzG5ycuFkHK9Zt5gf3L+fWRavYvL2FEUNqOPnwiZw6cxLzn1vPzQsaWfriqwVvr666ine/YQKnz6xn9vRx1Lgr1MzMbJc5cbMd5i1dw/k3LKS5tW1Ht+em7S387MHnueHB53u1rcMmj+H0I+r5wCET2WN4bTHCNTMzqzhO3AzIWtrOv2EhW5tbuywrdOTahNFDOHVmPafNrOfAvUf2bYBmZmbmxM0yP7h/Oc29eORUuyE1Vbz3Dftw+hH1vPXAcVRX+WkGZmZmxeLEzQC4ddGqXl0VOmvqnpx2RD3vP2RfRg91V6iZmVl/cOJmAGze3lJw3XkXHcu0cSOKGI2ZmZnl40v8DIDhQ6oLqjdySI2TNjMzsxJx4mYAvG7Czu+rVlMlTjl8Uj9EY2ZmZvk4cTM2bW/hmbWbd1qvtrqKc2dP64eIzMzMLB+PcTN+9MCzbNja/RMQaqpEbXUVV581k6lj3U1qZmZWKk7cKtzGLc1ce//yLuU1VaI1ghF1NZxy+CTOnT3NSZuZmVmJOXGrcN+/7xle3dbxitKaKnHXhcc4UTMzMxtgPMatgq19dTs/+uNzXcr/dtZkJ21mZmYDkBO3Cnb1Pcu6POKqrqaKT7/zwBJFZGZmZj1x4lahVm3Yys/yPDj+rKOmsu8ew0oQkZmZme2ME7cK9d27n6ap07NJh9dVc/47DihRRGZmZrYzTtwq0HMvbeam+Q1dys95636MGzmkBBGZmZlZIZy4VaAr//AUrZ0eKD9qaA3nzXZrm5mZ2UDmxK3CLH3hVX69eFWX8k+8fX/2GF5bgojMzMysUE7cKsw371xKdGxsY+yIOs55qx9lZWZmNtA5casgjzRs4I4lL3Yp/8djD2DEEN+L2czMbKBz4lZBrvj9U13K9hk9lLPeMrUE0ZiZmVlvOXGrEH9Zvo77nlrbpfyf3nkgQ2urSxCRmZmZ9ZYTtwoQEVzx+6VdyqfsNZwPzppcgojMzMxsVzhxqwD3PrWWh55b36X8c++eTm21TwEzM7NyUdRfbUnHS1oqaZmki/MsnyrpLkmPSLpHUn1O+UJJD0taIumTOevUSbpW0lOSnpR0WjH3odxFBN/IM7Zt+t4jOfHQSSWIyMzMzHZV0S4llFQNXAW8G2gAHpI0NyIez6l2BXB9RPxE0nHA5cDfA6uBoyNiu6SRwGNp3VXAF4A1EXGQpCpgr2Ltw2Bwx5IXeLRxY5fyC99zENVVKkFEZmZmtquKeQ+II4FlEbEcQNKNwElAbuI2A/h8mp4H3AoQEU05dYbQsWXwY8DrUr024KUixD4otLblb21706Q9eO8b9ilBRGZmZrY7itlVOglYmTPfkMpyLQZOTdOnAKMkjQWQNFnSI2kbX4+IVZLGpLpfTl2pv5Q0oWh7UObmLm7k6TWbupRf+J6DkNzaZmZmVm5KPTL9IuAYSYuAY4BGoBUgIlZGxCHAgcBHU4JWA9QDf4qImcCfybpbu5B0nqT5kuavXdv1NhiDXXNrG9+68+ku5W/eb0+OOWh8CSIyMzOz3VXMxK0RyL3XRH0q2yEiVkXEqRFxONnYNSJiQ+c6wGPAbGAdsAX4VVr8S2BmvjePiGsjYlZEzBo/vvISlV/Ob+D5l7d0Kb/oPQe7tc3MzKxMFTNxewiYLmmapDrgDGBubgVJ49IFBgCXANel8npJw9L0nsDbgKUREcBvgGPTOu+k45g5A7Y1t/Ldu7u2ts2ePo6j9h9bgojMzMysLxTt4oSIaJF0AXAHUA1cFxFLJF0GzI+IuWQJ2OWSArgP+FRa/fXAN1K5gCsi4tG07F+An0q6ElgLnFOsfShXP/vL86zeuK1L+UXvObgE0ZiZmVlfUdaINbjNmjUr5s+fX+ow+sXm7S28/T/msW5zU4fy98yYwLUfmVWiqMzMzKxQkhZERN4f7VJfnGB97Md/eq5L0ibBhW5tMzMzK3tO3AaRjVua+f69z3QpP/HQiRy8z6gSRGRmZmZ9yYnbIHLt/c/wyraWDmXVVeJz7zqoRBGZmZlZX3LiNki8tGk7P/rjc13KPzirnv3Gjej/gMzMzKzPOXEbJK6e9wxbmlo7lNVVV/FPx00vUURmZmbW15y4DQKrN27lhr+s6FL+4bdMYeKYYSWIyMzMzIrBidsg8J27ltHU0tahbFhtNecfe2CJIjIzM7NicOJW5las28wv56/sUn7OW/dj/KghJYjIzMzMisWJW5m78g9P09LW8SbKo4bW8Im3H1CiiMzMzKxYnLiVsadefJVbH27sUn7e7P3ZY3htCSIyMzOzYnLiVsa++fun6PzEsr1G1HHO26aVJiAzMzMrKiduZerRho3cvuSFLuXnH3sAI4fUlCAiMzMzKzYnbmXqit8v7VI2YfQQznrL1BJEY2ZmZv3BiVsZeui5l7n3qbVdyv/puOkMra0uQURmZmbWH5y4lZmI4D/v6NraNnmvYXxw1uQSRGRmZmb9xYlbmbn/6Zf467Mvdyn/7DsPoq7GH6eZmdlg5l/6MhIRece2Hbj3SE4+fFIJIjIzM7P+5MStjNyx5EUeadjYpfzCdx9EdZVKEJGZmZn1JyduZaK1LfjmnV1b2944aTTHv3GfEkRkZmZm/c2JW5n4zeJVPPXipi7lF77nYCS3tpmZmVUCJ25loLm1jW/94aku5bOm7smxB40vQURmZmZWCk7cysCcBQ2sWLelS/lF73Vrm5mZWSVx4jbAbWtu5Tt3Pd2lfPb0cbxl/7EliMjMzMxKxYnbAPc/f3me1Ru3dSm/8D0HlyAaMzMzKyUnbgPYlqYWrr5nWZfyd8+YwGGTx/R/QGZmZlZSBSVukn4l6f2SnOj1ox/98Tle2tTUoUyCC99zUIkiMjMzs1IqNBG7Gvg74GlJX5Pkfroi27i1me/f+0yX8g8cMpHX7TO6BBGZmZlZqRWUuEXEHyLiw8BM4DngD5L+JOkcSbXdrSfpeElLJS2TdHGe5VMl3SXpEUn3SKrPKV8o6WFJSyR9Ms+6cyU9VuiOlpsf3r+cV7a1dCirrhKfe7db28zMzCpVwV2fksYCZwPnAouAb5Mlcnd2U78auAo4AZgBnClpRqdqVwDXR8QhwGXA5al8NXB0RBwGHAVcLGlizrZPBbrejXaQeGnTdv77gWe7lJ8+s55p40aUICIzMzMbCAod43YLcD8wHPhARJwYEb+IiH8CRnaz2pHAsohYHhFNwI3ASZ3qzADuTtPz2pdHRFNEbE/lQ3LjlDQS+DzwlUJiL0fX3PMMW5paO5TVVVfx6XdNL1FEZmZmNhAU2uL2nYiYERGXR8Tq3AURMaubdSYBK3PmG1JZrsXAqWn6FGBUatlD0mRJj6RtfD0iVqV6Xwa+AXS9I+0gsHrjVn764Iou5X931BQmjRlWgojMzMxsoCg0cZshaUz7jKQ9JZ3fB+9/EXCMpEXAMUAj0AoQEStTF+qBwEclTZB0GHBARNyysw1LOk/SfEnz165d2weh9o/v3r2Mppa2DmXDaqs5/x0HlCgiMzMzGygKTdw+HhEb2mciYj3w8Z2s0whMzpmvT2U7RMSqiDg1Ig4HvpDKNnSuAzwGzAaOBmZJeg54ADhI0j353jwiro2IWRExa/z48nie5/PrtnDTQyu7lJ/91v3Ye9TQEkRkZmZmA0mhiVu1ch6KmS48qNvJOg8B0yVNk1QHnAHMza0gaVzOveEuAa5L5fWShqXpPYG3AUsj4pqImBgR+6WypyLi2AL3YcBasW4zX7z1UY77xj20tEWHZaOG1PCJt+9fosjMzMxsIKkpsN7twC8kfT/NfyKVdSsiWiRdANwBVAPXRcQSSZcB8yNiLnAscLmkAO4DPpVWfz3wjVQu4IqIeLQX+1U25i1dw/k3LKSptZXWtq7L3zVjAmOG7yxHNjMzs0qgiNh5paxV7BPAO1PRncAPI6K1+7UGjlmzZsX8+fNLHUYXK9Zt5vgr72drc/eHcWhtFXd89u1MHevbgJiZmVUCSQu6u/iz0BvwtqVuytPT6/vlkrQNZD+4fznN+ZrZcrS0Bj+8v+s93czMzKzyFHoft+mS5kh6XNLy9lexgxvsbl20qsuYts5a2oJbFjX2WMfMzMwqQ6EXJ/wIuAZoAd4BXA/cUKygKsXm7S07rwRsbiqsnpmZmQ1uhSZuwyLiLrIxcSsi4lLg/cULqzKMGFLYtSEj6gq9hsTMzMwGs0ITt+3pAoWnJV0g6RS6f9SVFejkwydSs5NPoKZKnHJ45wdOmJmZWSUqNHH7DNlzSj8NHAGcBXy0WEFVio/P3p+qqp4/gtrqKs6dPa2fIjIzM7OBbKeJW7rZ7ociYlNENETEORFxWkQ82A/xDWpTx47gjDdPzruspkoMq63m6rNm+lYgZmZmBhRwA96IaJX0tv4IphINq63uUlZXXcWH3jyZc2dPc9JmZmZmOxQ66n2RpLnAL4HN7YUR8auiRFVBnn95S5eyr57yRv52Vv6WODMzM6tchSZuQ4F1wHE5ZQE4cdtNK9Z1TdzcymZmZmb5FJS4RcQ5xQ6kEkUEK/O0uE3Za3gJojEzM7OBrqDETdKPyFrYOoiIj/V5RBVk/ZZmXu10E966mir2HjWkRBGZmZnZQFZoV+lvc6aHAqcAq/o+nMqSb3zblL2GU1WlEkRjZmZmA12hXaU3585L+jnwQFEiqiDdJW5mZmZm+RR6A97OpgN792Uglej5dZu7lDlxMzMzs+4UOsbtVTqOcXsB+JeiRFRB3OJmZmZmvVFoV+moYgdSiZy4mZmZWW8U1FUq6RRJe+TMj5F0ctGiqhDP572HmxM3MzMzy6/QMW7/FhEb22ciYgPwb0WJqEJsb2ll9SvbupRPdoubmZmZdaPQxC1fvUJvJWJ5NKzfSnS6M96E0UMYmufZpWZmZmZQeOI2X9I3JR2QXt8EFhQzsMHO49vMzMystwpN3P4JaAJ+AdwIbAM+VaygKkG+R125m9TMzMx6UuhVpZuBi4scS0XJ+3D5vfxweTMzM+teoVeV3ilpTM78npLuKFpUFSBvV+nYYSWIxMzMzMpFoV2l49KVpABExHr85ITdku9WIFPc4mZmZmY9KDRxa5M0pX1G0n50fJKC9UJE+OIEMzMz67VCb+nxBeABSfcCAmYD5xUtqkHupU1NbG1u7VA2vK6acSPrShSRmZmZlYOCWtwi4nZgFrAU+DlwIbB1Z+tJOl7SUknLJHW5uEHSVEl3SXpE0j2S6nPKF0p6WNISSZ9M5cMl/a+kJ1P513qxrwPG8y/nf7i8pBJEY2ZmZuWi0IfMnwt8BqgHHgbeAvwZOK6HdaqBq4B3Aw3AQ5LmRsTjOdWuAK6PiJ9IOg64HPh7YDVwdERslzQSeEzSXGADcEVEzJNUB9wl6YSI+F1vdrrU8nWT+lYgZmZmtjOFjnH7DPBmYEVEvAM4nCyJ6smRwLKIWB4RTWT3fzupU50ZwN1pel778ohoiojtqXxIe5wRsSUi5rXXARaSJZNlJf+tQJy4mZmZWc8KTdy2RcQ2AElDIuJJ4OCdrDMJWJkz35DKci0GTk3TpwCjJI1N7zNZ0iNpG1+PiFW5K6bbk3wAuKvAfRgw8t8KxImbmZmZ9azQxK0hJUq3AndK+jWwog/e/yLgGEmLgGOARqAVICJWRsQhwIHARyVNaF9JUg3ZWLvvRMTyfBuWdJ6k+ZLmr127tg9C7Tt+aoKZmZntikKfnHBKmrxU0jxgD+D2nazWCEzOma9PZbnbXUVqcUtj2U7LvV9cex1Jj5FdyTonFV8LPB0RV/YQ87WpHrNmzRpQty5xV6mZmZntikJb3HaIiHsjYm4aY9aTh4DpkqalCwnOAObmVpA0TlJ7DJcA16XyeknD0vSewNvIrmhF0lfIEsfP9jb2gWBrUytrXt3eoUyCSXv6qQlmZmbWs14nboWKiBbgAuAO4AngpohYIukySSemascCSyU9BUwAvprKXw/8RdJi4F6yK0kfTbcL+QLZRQ3ttws5t1j7UAwN67u2tk3cYxhDaqpLEI2ZmZmVk0JvwLtLIuI24LZOZV/KmZ7Da92fuXXuBA7JU95AdgPgspWvm3TyXm5tMzMzs50rWoub5edHXZmZmdmucuLWz/IlblPH+uHyZmZmtnNO3PqZn5pgZmZmu8qJWz9zV6mZmZntKidu/aitLfJ3lTpxMzMzswI4cetHa17dTlNLW4eyUUNqGDO8tkQRmZmZWTlx4taPVqzb3KVsytjhSGV9hxMzMzPrJ07c+pHHt5mZmdnucOLWj/I9XN6Jm5mZmRXKiVs/WpEvcRvrxM3MzMwK48StH7mr1MzMzHaHE7d+9Hye55RO3ctPTTAzM7PCOHHrJ5u2t7Buc1OHsuoqse+YoSWKyMzMzMqNE7d+ku/ChIljhlJb7Y/AzMzMCuOsoZ+scDepmZmZ7SYnbv0kX4ubHy5vZmZmveHErZ/kfUapbwViZmZmveDErZ/kvYebW9zMzMysF5y49RM/NcHMzMx2lxO3ftDaFjSs91MTzMzMbPc4cesHqzdupbk1OpSNGV7L6KG1JYrIzMzMypETt36Q98IEd5OamZlZLzlx6wf5HnXlW4GYmZlZbzlx6wd+uLyZmZn1BSdu/SDfrUB8DzczMzPrLSdu/cBPTTAzM7O+4MStH+R/aoKfU2pmZma9U9TETdLxkpZKWibp4jzLp0q6S9Ijku6RVJ9TvlDSw5KWSPpkzjpHSHo0bfM7klTMfdhdG7c2s2FLc4ey2mqxz+ihJYrIzMzMylXREjdJ1cBVwAnADOBMSTM6VbsCuD4iDgEuAy5P5auBoyPiMOAo4GJJE9Oya4CPA9PT6/hi7UNfyNdNWr/ncKqrBnS+aWZmZgNQMVvcjgSWRcTyiGgCbgRO6lRnBnB3mp7XvjwimiJieyof0h6npH2B0RHxYEQEcD1wchH3YbetyHMrEF9RamZmZruimInbJGBlznxDKsu1GDg1TZ8CjJI0FkDSZEmPpG18PSJWpfUbdrJN0vrnSZovaf7atWt3e2d2lW8FYmZmZn2l1BcnXAQcI2kRcAzQCLQCRMTK1IV6IPBRSRN6s+GIuDYiZkXErPHjx/d13AXLf2GCEzczMzPrvZoibrsRmJwzX5/KdkitaKcCSBoJnBYRGzrXkfQYMBv4Y9pOt9scaJ5/eXOXMt8KxMzMzHZFMVvcHgKmS5omqQ44A5ibW0HSOEntMVwCXJfK6yUNS9N7Am8DlkbEauAVSW9JV5N+BPh1Efdht7mr1MzMzPpK0RK3iGgBLgDuAJ4AboqIJZIuk3RiqnYssFTSU8AE4Kup/PXAXyQtBu4FroiIR9Oy84EfAsuAZ4DfFWsfdldzaxurNmzrUu7EzczMzHZFMbtKiYjbgNs6lX0pZ3oOMCfPencCh3SzzfnAG/s20uJYtWErrW3RoWzcyDpGDCnqYTczM7NBqtQXJwxq7iY1MzOzvuTErYh8DzczMzPrS07ciijfUxOcuJmZmdmucuJWRHm7Sv1weTMzM9tFTtyKyF2lZmZm1pecuBVJROTtKvVTE8zMzGxXOXErkvVbmnl1e0uHsiE1VYwfOaREEZmZmVm5c+JWJPnGt03eazhVVSpBNGZmZjYYOHErkrwPl/f4NjMzM9sNTtyK5Pl1fri8mZmZ9S0nbkXipyaYmZlZX3PiViT5bgXiK0rNzMxsdzhxKxI/NcHMzMz6mhO3Itje0srqV7Z1KfcYNzMzM9sdTtyKoGH9ViI6lk0YPYShtdWlCcjMzMwGBSduReALE8zMzKwYnLgVQf7xbX64vJmZme0eJ25F4IfLm5mZWTE4cSuCvE9N8K1AzMzMbDc5cSuC5/O0uPmKUjMzM9tdTtz6WET44gQzMzMrCidufeylTU1sbW7tUDa8rppxI+tKFJGZmZkNFk7c+tjzL3d9uPyUvYYjqQTRmJmZ2WDixK2PuZvUzMzMisWJWx/zrUDMzMysWJy49bG8LW6+FYiZmZn1gaImbpKOl7RU0jJJF+dZPlXSXZIekXSPpPpUfpikP0takpZ9KGedd0paKOlhSQ9IOrCY+9Bb+Z+a4MTNzMzMdl/REjdJ1cBVwAnADOBMSTM6VbsCuD4iDgEuAy5P5VuAj0TEG4DjgSsljUnLrgE+HBGHAf8DfLFY+7Ar3FVqZmZmxVLMFrcjgWURsTwimoAbgZM61ZkB3J2m57Uvj4inIuLpNL0KWAOMT/UCGJ2m9wBWFW0PemlrUytrXt3eoUyC+j2duJmZmdnuqyniticBK3PmG4CjOtVZDJwKfBs4BRglaWxErGuvIOlIoA54JhWdC9wmaSvwCvCW4oTfeyvXd21tm7jHMOpqPJTQzMzMdl+pM4qLgGMkLQKOARqBHXevlbQv8FPgnIhoS8WfA94XEfXAj4Bv5tuwpPMkzZc0f+3atcXchx3yP+pqWL+8t5mZmQ1+xUzcGoHJOfP1qWyHiFgVEadGxOHAF1LZBgBJo4H/Bb4QEQ+msvHAoRHxl7SJXwD/J9+bR8S1ETErImaNHz8+X5U+l/fh8nuN6Jf3NjMzs8GvmInbQ8B0SdMk1QFnAHNzK0gaJ6k9hkuA61J5HXAL2YULc3JWWQ/sIemgNP9u4Iki7kOv+FYgZmZmVkxFG+MWES2SLgDuAKqB6yJiiaTLgPkRMRc4FrhcUgD3AZ9Kq38QeDswVtLZqezsiHhY0seBmyW1kSVyHyvWPvSWn5pgZmZmxVTMixOIiNuA2zqVfSlneg4wJ896NwA3dLPNW8ha4wYcJ25mZmZWTKW+OGHQaGsLJ25mZmZWVE7c+siaV7fT1NLWoWzU0BrGDK8tUURmZmY22Dhx6yMr1m3uUjZlr+FIKkE0ZmZmNhg5cesjeW8F4itKzczMrA85cesj+R4uP9nj28zMzKwPOXHrIyt8YYKZmZkVmRO3PuKnJpiZmVmxOXHrI/meU+oWNzMzM+tLTtz6wKbtLazb3NShrLpKTBwztEQRmZmZ2WDkxK0P5LswYdKYYdRU+/CamZlZ33Fm0QdWuJvUzMzM+oETtz6Qr8Vtiu/hZmZmZn3MiVsfWPFy/qcmmJmZmfUlJ2594PmXt3Ypc+JmZmZmfc2JWx/I21XqxM3MzMz6mBO33dTaFjSs9xg3MzMzKz4nbrtp9catNLdGh7I9h9cyemhtiSIyMzOzwcqJ227K96grd5OamZlZMThx2035HnU12YmbmZmZFYETt92U9+HyHt9mZmZmReDEbTetcFepmZmZ9RMnbrthxbrN/PmZdV3Kh9RUlyAaMzMzG+ycuO2ieUvXcPyV9/Py5qYuyy7+1SPMW7qmBFGZmZnZYObEbResWLeZ829YyNbm1rzLtzW3cf4NC1mxruujsMzMzMx2lRO3XfCD+5fT3NrWY53m1jZ+eP+z/RSRmZmZVQInbrvg1kWraGmLHuu0tAW3LGrsp4jMzMysEjhx2wWbt7cUVq+psHpmZmZmhShq4ibpeElLJS2TdHGe5VMl3SXpEUn3SKpP5YdJ+rOkJWnZh3LWkaSvSnpK0hOSPl3MfchnxJCawurVFVbPzMzMrBBFS9wkVQNXAScAM4AzJc3oVO0K4PqIOAS4DLg8lW8BPhIRbwCOB66UNCYtOxuYDLwuIl4P3FisfejOyYdPpKZKPdapqRKnHD6pnyIyMzOzSlDMFrcjgWURsTwimsgSrJM61ZkB3J2m57Uvj4inIuLpNL0KWAOMT/X+EbgsItrS8n6/78bHZ+9PbXXPh662uopzZ0/rp4jMzMysEhQzcZsErMyZb0hluRYDp6bpU4BRksbmVpB0JFAHPJOKDgA+JGm+pN9Jmt7nke/E1LEjuPqsmQyrre7S8lZTJYbVVnP1WTOZOnZEf4dmZmZmg1ipL064CDhG0iLgGKAR2HFzNEn7Aj8FzmlvYQOGANsiYhbwA+C6fBuWdF5K7uavXbu2zwN/x8F7c/tnZ3PmkVMYOaQGCUYOqeHMI6dw+2dn846D9+7z9zQzM7PKpoieb2uxyxuWjgYujYj3pvlLACLi8m7qjwSejIj2CxRGA/cA/y8i5uTUexI4ISKelSRgQ0Ts0VMss2bNivnz5/fBXpmZmZkVl6QFqYGqi2K2uD0ETJc0TVIdcAYwt1Ng4yS1x3AJqfUs1b+F7MKFOXR0K/CONH0M8FRxwjczMzMbWIqWuEVEC3ABcAfwBHBTRCyRdJmkE1O1Y4Glkp4CJgBfTeUfBN4OnC3p4fQ6LC37GnCapEfJrkI9t1j7YGZmZjaQFK2rdCBxV6mZmZmVi1J1lZqZmZlZH3LiZmZmZlYmKqKrVNJaYEUPVcYBL/VTOOXExyU/H5f8fFzy83HJz8clPx+XrirxmEyNiPH5FlRE4rYzkuZ315dcyXxc8vNxyc/HJT8fl/x8XPLzcenKx6Qjd5WamZmZlQknbmZmZmZlwolb5tpSBzBA+bjk5+OSn49Lfj4u+fm45Ofj0pWPSQ6PcTMzMzMrE25xMzMzMysTFZ24STpe0lJJyyRdXOp4SkXSZEnzJD0uaYmkz6TyvSTdKenp9O+epY61FCRVS1ok6bdpfpqkv6Tz5hfp2boVRdIYSXMkPSnpCUlH+3wBSZ9L/4cek/RzSUMr8XyRdJ2kNZIeyynLe34o8510fB6RNLN0kRdXN8flP9P/o0ck3SJpTM6yS9JxWSrpvSUJuh/kOy45yy6UFJLGpfmKOV+6U7GJm6Rq4CrgBGAGcKakGaWNqmRagAsjYgbwFuBT6VhcDNwVEdOBu9J8JfoM2fN2230d+FZEHAisB/6hJFGV1reB2yPidcChZMenos8XSZOATwOzIuKNQDVwBpV5vvwYOL5TWXfnxwnA9PQ6D7imn2IshR/T9bjcCbwxIg4BngIuAUjfwWcAb0jrXJ1+twajH9P1uCBpMvAe4Pmc4ko6X/Kq2MQNOBJYFhHLI6IJuBE4qcQxlURErI6IhWn6VbIf4Ulkx+MnqdpPgJNLEmAJSaoH3g/8MM0LOA6Yk6pU3HGRtAfwduC/ASKiKSI24PMFoAYYJqkGGA6spgLPl4i4D3i5U3F358dJwPWReRAYI2nffgm0n+U7LhHx+4hoSbMPAvVp+iTgxojYHhHPAsvIfrcGnW7OF4BvAf8M5A7Gr5jzpTuVnLhNAlbmzDeksoomaT/gcOAvwISIWJ0WvQBMKFVcJXQl2RdHW5ofC2zI+aKtxPNmGrAW+FHqQv6hpBFU+PkSEY3AFWStA6uBjcACfL606+788Hfxaz4G/C5NV/RxkXQS0BgRizstqujjApWduFknkkYCNwOfjYhXcpdFdvlxRV2CLOlvgDURsaDUsQwwNcBM4JqIOBzYTKdu0Qo9X/Ykaw2YBkwERpCn+8cq8/zYGUlfIBu28rNSx1JqkoYD/wp8qdSxDESVnLg1ApNz5utTWUWSVEuWtP0sIn6Vil9sb4JO/64pVXwl8lbgREnPkXWlH0c2tmtM6gqDyjxvGoCGiPhLmp9DlshV+vnyLuDZiFgbEc3Ar8jOoUo/X9p1d35U/HexpLOBvwE+HK/do6uSj8sBZH8ALU7fv/XAQkn7UNnHBajsxO0hYHq64quObBDo3BLHVBJp3NZ/A09ExDdzFs0FPpqmPwr8ur9jK6WIuCQi6iNiP7Lz4+6I+DAwDzg9VavE4/ICsFLSwanoncDjVPj5QtZF+hZJw9P/qfbjUtHnS47uzo+5wEfS1YJvATbmdKkOepKOJxuOcWJEbMlZNBc4Q9IQSdPIBuP/tRQx9reIeDQi9o6I/dL3bwMwM333VPT5AhV+A15J7yMbw1QNXBcRXy1tRKUh6W3A/cCjvDaW61/JxrndBEwBVgAfjIh8A0gHPUnHAhdFxN9I2p+sBW4vYBFwVkRsL2F4/U7SYWQXbNQBy4FzyP4QrOjzRdK/Ax8i6/JaBJxLNv6mos4XST8HjgXGAS8C/wbcSp7zIyW53yPrVt4CnBMR80sQdtF1c1wuAYYA61K1ByPik6n+F8jGvbWQDWH5XedtDgb5jktE/HfO8ufIrtZ+qZLOl+5UdOJmZmZmVk4quavUzMzMrKw4cTMzMzMrE07czMzMzMqEEzczMzOzMuHEzczMzKxMOHEzMzMzKxNO3MzMOpF0WLrPY/v8iZIu7mmdXmz7s+mRPmZmveb7uJmZdZIeQTQrIi4owrafS9t+qRfrVEdEa1/HYmblxy1uZla2JO0n6QlJP5C0RNLvJQ3rpu4Bkm6XtEDS/ZJel8r/VtJjkhZLui89Au8y4EOSHpb0IUlnS/peqv9jSddIelDScknHSrouxfHjnPe7RtL8FNe/p7JPkz2Afp6keansTEmPphi+nrP+JknfkLQYOFrS1yQ9LukRSVcU54ia2UDnFjczK1uS9gOWkbVgPSzpJmBuRNyQp+5dwCcj4mlJRwGXR8Rxkh4Fjo+IRkljImJD5xa33PmUnA0FzgROBH5K9jD5JWTPQP6HFMte6ZFO1cBdwKcj4pFOj++ZCDwIHAGsB34PfCcibpUUwIci4iZJY4E/Aa+LiGiPs88PqJkNeG5xM7Ny92xEPJymFwD7da4gaSTwf4BfSnoY+D6wb1r8R+DHkj5O9tziQvwmsr96HwVeTA/FbiNL3trf/4OSFpI9n/QNwIw823kzcE9ErI2IFuBnwNvTslbg5jS9EdgG/LekU8me0WhmFaim1AGYme2m3Ae2twL5ukqrgA0RcVjnBRHxydQC935ggaQjevGebZ3evw2okTQNuAh4c0Ssz2ml641t7ePaIqJF0pHAO4HTgQuA43q5PTMbBNziZmaDXkS8Ajwr6W8BlDk0TR8QEX+JiC8Ba4HJwKvAqN14y9HAZmCjpAnACTnLcrf9V+AYSeNSl+qZwL2dN5ZaDPeIiNuAzwGH7kZsZlbG3OJmZpXiw8A1kr4I1AI3AouB/5Q0HRDZWLTFwPPAxalb9fLevlFELJa0CHgSWEnWHdvuWuB2Sasi4h3pNiPz0vv/b0T8Os8mRwG/ljQ01ft8b2Mys8HBFyeYmZmZlQl3lZqZmZmVCXeVmtmgIukqsttz5Pp2RPyoFPGYmfUld5WamZmZlQl3lZqZmZmVCSduZmZmZmXCiZuZmZlZmXDiZmZmZlYmnLiZmZmZlYn/D3vNRTYoqaaDAAAAAElFTkSuQmCC\n",
      "text/plain": [
       "<Figure size 720x360 with 1 Axes>"
      ]
     },
     "metadata": {
      "needs_background": "light"
     },
     "output_type": "display_data"
    }
   ],
   "source": [
    "show_curve(param_grid[\"classifier__n_estimators\"],\n",
    "           grid_search.cv_results_['mean_test_score'],\n",
    "           \"n_estimators\",\n",
    "           \"accuracy\",\n",
    "           \"Mean score with CV varying n_estimators of Random Forest\")"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 38,
   "id": "6673e2a6",
   "metadata": {
    "execution": {
     "iopub.execute_input": "2022-03-16T15:18:54.900444Z",
     "iopub.status.busy": "2022-03-16T15:18:54.899516Z",
     "iopub.status.idle": "2022-03-16T15:18:54.903694Z",
     "shell.execute_reply": "2022-03-16T15:18:54.904181Z",
     "shell.execute_reply.started": "2022-03-14T13:23:47.056861Z"
    },
    "papermill": {
     "duration": 0.117785,
     "end_time": "2022-03-16T15:18:54.904337",
     "exception": false,
     "start_time": "2022-03-16T15:18:54.786552",
     "status": "completed"
    },
    "tags": []
   },
   "outputs": [],
   "source": [
    "# Define the final model with the best parameter found\n",
    "best_n_estimators = grid_search.best_params_[\"classifier__n_estimators\"]\n",
    "rnd_forest_model = RandomForestClassifier(criterion=\"entropy\", n_estimators=best_n_estimators, random_state=random_seed)"
   ]
  },
  {
   "cell_type": "markdown",
   "id": "7c25450b",
   "metadata": {
    "papermill": {
     "duration": 0.109917,
     "end_time": "2022-03-16T15:18:55.123172",
     "exception": false,
     "start_time": "2022-03-16T15:18:55.013255",
     "status": "completed"
    },
    "tags": []
   },
   "source": [
    "### **Extremely Randomized Trees**"
   ]
  },
  {
   "cell_type": "markdown",
   "id": "99489e19",
   "metadata": {
    "papermill": {
     "duration": 0.107919,
     "end_time": "2022-03-16T15:18:55.339065",
     "exception": false,
     "start_time": "2022-03-16T15:18:55.231146",
     "status": "completed"
    },
    "tags": []
   },
   "source": [
    "In extremely randomized trees, randomness goes one step further compared to random forest in the way splits are computed. As in random forests, a random subset of candidate features is used, but instead of looking for the most discriminative thresholds, thresholds are drawn at random for each candidate feature and the best of these randomly-generated thresholds is picked as the splitting rule. This usually allows to reduce the variance of the model a bit more, at the expense of a slightly greater increase in bias."
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 39,
   "id": "f96915b3",
   "metadata": {
    "execution": {
     "iopub.execute_input": "2022-03-16T15:18:55.558634Z",
     "iopub.status.busy": "2022-03-16T15:18:55.557703Z",
     "iopub.status.idle": "2022-03-16T15:18:55.561311Z",
     "shell.execute_reply": "2022-03-16T15:18:55.561748Z",
     "shell.execute_reply.started": "2022-03-14T13:23:47.058061Z"
    },
    "papermill": {
     "duration": 0.115393,
     "end_time": "2022-03-16T15:18:55.561911",
     "exception": false,
     "start_time": "2022-03-16T15:18:55.446518",
     "status": "completed"
    },
    "tags": []
   },
   "outputs": [],
   "source": [
    "from sklearn.ensemble import ExtraTreesClassifier"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 40,
   "id": "9e6421d7",
   "metadata": {
    "execution": {
     "iopub.execute_input": "2022-03-16T15:18:55.787501Z",
     "iopub.status.busy": "2022-03-16T15:18:55.786883Z",
     "iopub.status.idle": "2022-03-16T15:20:25.987631Z",
     "shell.execute_reply": "2022-03-16T15:20:25.986666Z",
     "shell.execute_reply.started": "2022-03-14T13:23:47.05922Z"
    },
    "papermill": {
     "duration": 90.313779,
     "end_time": "2022-03-16T15:20:25.987832",
     "exception": false,
     "start_time": "2022-03-16T15:18:55.674053",
     "status": "completed"
    },
    "tags": []
   },
   "outputs": [
    {
     "name": "stdout",
     "output_type": "stream",
     "text": [
      "Fitting 2 folds for each of 6 candidates, totalling 12 fits\n",
      "[CV] classifier__n_estimators=5 ......................................\n"
     ]
    },
    {
     "name": "stderr",
     "output_type": "stream",
     "text": [
      "[Parallel(n_jobs=1)]: Using backend SequentialBackend with 1 concurrent workers.\n"
     ]
    },
    {
     "name": "stdout",
     "output_type": "stream",
     "text": [
      "[CV] ....................... classifier__n_estimators=5, total=   3.2s\n",
      "[CV] classifier__n_estimators=5 ......................................\n"
     ]
    },
    {
     "name": "stderr",
     "output_type": "stream",
     "text": [
      "[Parallel(n_jobs=1)]: Done   1 out of   1 | elapsed:    3.2s remaining:    0.0s\n"
     ]
    },
    {
     "name": "stdout",
     "output_type": "stream",
     "text": [
      "[CV] ....................... classifier__n_estimators=5, total=   3.2s\n",
      "[CV] classifier__n_estimators=10 .....................................\n",
      "[CV] ...................... classifier__n_estimators=10, total=   3.5s\n",
      "[CV] classifier__n_estimators=10 .....................................\n",
      "[CV] ...................... classifier__n_estimators=10, total=   3.4s\n",
      "[CV] classifier__n_estimators=20 .....................................\n",
      "[CV] ...................... classifier__n_estimators=20, total=   4.1s\n",
      "[CV] classifier__n_estimators=20 .....................................\n",
      "[CV] ...................... classifier__n_estimators=20, total=   4.0s\n",
      "[CV] classifier__n_estimators=50 .....................................\n",
      "[CV] ...................... classifier__n_estimators=50, total=   6.0s\n",
      "[CV] classifier__n_estimators=50 .....................................\n",
      "[CV] ...................... classifier__n_estimators=50, total=   6.0s\n",
      "[CV] classifier__n_estimators=100 ....................................\n",
      "[CV] ..................... classifier__n_estimators=100, total=   9.0s\n",
      "[CV] classifier__n_estimators=100 ....................................\n",
      "[CV] ..................... classifier__n_estimators=100, total=   9.3s\n",
      "[CV] classifier__n_estimators=150 ....................................\n",
      "[CV] ..................... classifier__n_estimators=150, total=  12.6s\n",
      "[CV] classifier__n_estimators=150 ....................................\n",
      "[CV] ..................... classifier__n_estimators=150, total=  12.5s\n"
     ]
    },
    {
     "name": "stderr",
     "output_type": "stream",
     "text": [
      "[Parallel(n_jobs=1)]: Done  12 out of  12 | elapsed:  1.3min finished\n"
     ]
    }
   ],
   "source": [
    "# Construct the PCA-pipeline using the selected classifier\n",
    "classifier = ExtraTreesClassifier(criterion=\"entropy\", random_state=random_seed)\n",
    "pipe = build_PCA_pipeline(classifier, pca_n_components)\n",
    "\n",
    "# Define the parameter grid to use in the the grid search with cross validation.\n",
    "# For this model we will consider the hyper-parameter which controls the number of trees to use.\n",
    "param_grid = {\n",
    "    \"classifier__n_estimators\": [5,10,20,50,100,150]\n",
    "}\n",
    "\n",
    "# Perform the search on the training set\n",
    "grid_search = GridSearchCV(pipe, param_grid, cv=2, verbose=2)\n",
    "grid_search.fit(X_train, Y_train);"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 41,
   "id": "b88e22ef",
   "metadata": {
    "execution": {
     "iopub.execute_input": "2022-03-16T15:20:26.225502Z",
     "iopub.status.busy": "2022-03-16T15:20:26.224872Z",
     "iopub.status.idle": "2022-03-16T15:20:26.231049Z",
     "shell.execute_reply": "2022-03-16T15:20:26.231541Z",
     "shell.execute_reply.started": "2022-03-14T13:23:47.060235Z"
    },
    "papermill": {
     "duration": 0.128044,
     "end_time": "2022-03-16T15:20:26.231706",
     "exception": false,
     "start_time": "2022-03-16T15:20:26.103662",
     "status": "completed"
    },
    "tags": []
   },
   "outputs": [
    {
     "name": "stdout",
     "output_type": "stream",
     "text": [
      "Best parameters set found on train set:\n",
      "{'classifier__n_estimators': 100}\n",
      "\n",
      "\n",
      "Grid scores on train set:\n",
      "{'classifier__n_estimators': 5} mean: 0.935 std: +/- 0.001)\n",
      "{'classifier__n_estimators': 10} mean: 0.937 std: +/- 0.001)\n",
      "{'classifier__n_estimators': 20} mean: 0.940 std: +/- 0.000)\n",
      "{'classifier__n_estimators': 50} mean: 0.941 std: +/- 0.000)\n",
      "{'classifier__n_estimators': 100} mean: 0.941 std: +/- 0.000)\n",
      "{'classifier__n_estimators': 150} mean: 0.941 std: +/- 0.000)\n",
      "\n",
      "\n"
     ]
    }
   ],
   "source": [
    "print_grid_search_results(grid_search)"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 42,
   "id": "570e6a53",
   "metadata": {
    "execution": {
     "iopub.execute_input": "2022-03-16T15:20:26.468150Z",
     "iopub.status.busy": "2022-03-16T15:20:26.467257Z",
     "iopub.status.idle": "2022-03-16T15:20:26.667545Z",
     "shell.execute_reply": "2022-03-16T15:20:26.668067Z",
     "shell.execute_reply.started": "2022-03-14T13:23:47.062416Z"
    },
    "papermill": {
     "duration": 0.319537,
     "end_time": "2022-03-16T15:20:26.668232",
     "exception": false,
     "start_time": "2022-03-16T15:20:26.348695",
     "status": "completed"
    },
    "tags": []
   },
   "outputs": [
    {
     "data": {
      "image/png": "iVBORw0KGgoAAAANSUhEUgAAAm0AAAFOCAYAAADU2yq7AAAAOXRFWHRTb2Z0d2FyZQBNYXRwbG90bGliIHZlcnNpb24zLjUuMSwgaHR0cHM6Ly9tYXRwbG90bGliLm9yZy/YYfK9AAAACXBIWXMAAAsTAAALEwEAmpwYAABDVElEQVR4nO3de3ycdZn//9eVY9P0nLSFNqEtUKpFS4tdkJUuB1FhRaC4u4Ligi6wiHzVVb4rqKvIfv2hK666K+gCoiArBxFKVQQRy8EDSOkJCrYU2tK0hbZp00Pa5nj9/rg/096ZmSSTdCaTybyfj0cemftz33PPNffcM3PN53SbuyMiIiIig1tJvgMQERERkd4paRMREREpAEraRERERAqAkjYRERGRAqCkTURERKQAKGkTERERKQBK2kSSmNk8M1vVw/qpZuZmVjaQceWDmX3EzH6T7zhywcx+bWYX5zuOXDKzKjP7hZntNLOf5TueXDKzdWZ2Rr7j6EkuPztycT6b2XVmdlc29ymHRknbEBI+tFrNrDapfGn4oJiap9AKirs/7e4zEsvZ+DIwsxPM7GEzazKz7Wb2ZzP7mJlNNrN2MzsqzX0eNLMbD+VxD5W7/6+7vzefMWRDui8fdz/L3e/IwWMNpqT+74CJQI27/33yynBc2sxsT+yvKZMdF0KSlImkY9BkZn80s5PyHVdf5ep8Tif8mEucL/vMrDN+Dg1EDMVKSdvQsxa4MLFgZm8HhucvnNwZJF+KvQpfAL8DngSOBmqATwBnuftG4HHgo0n3GQf8LZDTD+FCOYbFJMuvyRRgtbu397DNve4+IvY3JhsPXGDn1r3uPgKoBRYBQ7pW8lCFH3MjwjE7C9gUP4fi25pZaX6iHKLcXX9D5A9YB3wJeC5WdiPwRcCBqaGsMpS/DrwJ/ACoCuvGAr8EtgI7wu262P6eAP4d+AOwG/gNUNtNPLXh/k3AduBpoCSsqwceCI/TCHwvlJeE57Ae2ALcCYwO66aG5/FPIfanQvnHgZdDvI8CU7qJ5w7gc+H25LCvT4blo0KMJcCpQEMo/wnQCewD9gD/Govj4hDHNuCLPbwuvwdu6mH9h4FXk8quBJZ2s/2vgauSypYD54fb3wU2ALuA54F5se2uA+4H7grrvwTsJaqJSWxzfHhdyoFLgN/H1jlwBfBKeF1vAiysKwW+FY7HWuCqsH1ZD+fr1cAKYCdwLzAsg/P8bGBZePw/ArNi6z4PbCQ6N1cB7wbOBFqBtvAaLo+dy5eG25cQndPfDvt9DfjrUL6B6Fy8OPY47weWhmO4Abgutu718Lz3hL+T6ON5DQwLr1FjiOc5YGI3x+Ot4bk0ASuBc0L5V5Oe9z+lue91wF3d7Pevw2tZH5aPI3qPvYWe3xcZvz/D9lcSnU+7iT5bjgqv6y7gPqAiw9d+HXAGcBg9nNO9HQNgZohrfFg+AfhTeMzNwPeSYurtPXFjOI6vAZ8k9p4AJgELiT571gCXJcX1s3Ae7AZeAI4BriU6hzYA7036bE6cz8s5eP7tCY95alj3znDsmsJ2p8b2MY3ox+Vu4LHwXNOeH7H7nEr4vAzLPwa+DzwMNIfXZBLw8/AarAU+Fdu+BLgGeJXofL8PGBfWZfw+KJa/vAegvyy+mAc/tFYRfZCXAg1Ev7bjSdu3wwfFOGAk8AvghrCuBvggUe3cyPChsSD2GE+EN9cxQFVY/no38dxAlBCWh795gIW4loc4qsMb8+Rwn4+HD68jgRFEid1Pwrqp4XncGe5XBZwbtn8rUEb0xfjHbuL5OPCLcPvD4XncG1v3ULid/CG0DjgjtpyI49YQw3FAC/DWNI85HOgATuvhdasiSlpOjpX9CfhMN9v/I/CH2PLM8IFWGZYvCq9jGfA54A1CMkT0RdAGnEf0YVlF9OH6idj+vg38d7h9CalJ2y+BMcARRB/CZ4Z1VwAvAXVEyf9v6T1p+zPRB/o4oi/2K3o5x+cQfWGdGM6ji8N+KoEZRF9kk2Kv01Gx531X0r6eoGvS1g58LOz3/xElHjeFfb+X6ItsROwceXs4hrOIfvycl3R+lMUeq6/n9T8TvS+Hh3jeAYxKczzKw36/AFQAp4c4Z3T3vJPu39v6rxHVElcRJQ1XxdatI/37IuP3Z9j+IWAUcCzR++jxcJxGE51PF/f22ifHQw/ndE/HIBzDrxMlWYnE6h1EiU5ZeI4vE3tv0vt74i9EP1LHEdXixZO2p4CbiT4DZ4f7nh6Laz/wvvDYdxIlPF8Mr/tlwNp053PS87s8xDCK6MdqI1EtfgnwnrCcSFD/BPwn0Tn/N0TnUn+Stp3Au8JjDCf68fjlcHyPJEpg3xe2/zTwDNHnRiXwP8DdYV1G74Ni+st7APrL4ot5MGn7ElHCdCbRr6Wy8EExlShpaiZ8mYX7nRR/8yftczawI7b8BPCl2PKVwCPd3Pd6og/ko5PKTwofTilf5kQf2FfGlmcQJRmJD0wHjoyt/zWxGoTwIbGXNLVtRL/gd4RtfhA+EBI1ancAnw23kz+E1pH+yyleA/ln4II0j5mo0XtLL6/dbcAt4fZ0ohqSCd1sOzK8hlPC8teA23vY9w7guHD7OkINSGz9hwhJYPhgfAM4ISxfQmrSFk8u7wOuCbd/B/xzbN0Z9J60XRRb/g/gB70cp+8D/55Utgo4hajpeUt43PKkba6j96Ttldi6t4fYJ8bKGoHZ3cT1HeDbSedHPGnr63n9cZJqkrp53Hnh9SqJld1NqPlL97zTHJdWoqQ/8bcotr6c6Av3BeARQg1SL++LjN+fYft3xdY/D3w+tvwt4Du9vfbJ8dDDOd3LMegIr/OpPRyzzwAP9uE9cUVs3XsT5wZRItcBjIytvwH4cSyux2LrPkBUa1bqBz8HHBiTfD7H7nMy0XvimLD8ecKPhdg2jxIlwEcQ/XCpjq37aU/nT9jmVFKTtjtjyycCryfd51rgR+H2y8C7Y+sO5+B7I6P3QTH9qU/b0PQTopqkS4h+ncWNJ/zyCZ1um4g+jMcDmNlwM/sfM1tvZruIfgmOSeqX8Ebs9l6imoN0vkn0K/s3ZvaamV0TyuuB9Z6+n80koiakhPVEb96JsbINsdtTgO/Gnst2osR0cvKO3f1VomRnNtGX3S+BTWY2g+hL/8lunkd3MjkOO4iakQ7vZV93AH9vZsOI+rc96u5b0m3o7ruBXwEXhKILgf9NrDezq83s5TBisImoxiI+OCV+/CBKrGea2TSiX9473f3PPcTa3fOelLTv5Mfpy766MwX4XOL1Ds+vnqh2bQ3RF+p1wBYzu8fMJmUQQ8Kbsdv7ANw9uWwEgJmdaGaLzGyrme0kqlHpMgAoSV/P658QfZneY2abzOw/zKy8m/1ucPfOpH2nnP89uM/dx8T+TkuscPc2oi/htwHf8vCt2ou+vj+Tj3HaY04Pr32aGPp6Tt/nUV++icCLRDU6AJjZMWb2SzN7I3wm/n+kvtaZvifi58AkYHt4P8fX93Rstrl7R2wZunnPmFk9UQJ5sbuvDsVTiD5n4sfwZKLPp0lEP9Cbu4m3L5LPgUlJj/kFDp77U4AHY+teJkpmJ5L5+6BoKGkbgtx9PVE1+t8SNcPEbSN6sx8b+5Ae7Qc7j36OqBbgRHcfRVRFDtEHbV/j2O3un3P3I4FzgM+a2buJ3tBHdNNReRPRmzgh8esv/uEV/+LYQFS7E//SqXL3P3YT1pNEI+oqPBoE8CTRr8yxRH1l0j6VHp9oD9x9L1GTwwd72fT3RF9o5xI1b/Y2AOFu4MIwyGEYUbMLZjaPqH/RPwBjwxfRTrq+fl2ej7vvJ/pwv4goYfxJb8+rG5uJmjgS6vu5n55sAL6W9HoPd/e7Adz9p+5+Mge7BHwj3K/fr2E3fkrUxaDe3UcT1dwmjnG6x+rTee3ube7+VXefSdS37GyiZvF0+603s/hn+RFE/foOmZlNBr4C/Aj4lplVpos3yaG8P3vS42vfJYB+ntPuvo2oOfE6M0v80Po+UfPi9PCZ+AUy/zzcTNf3wRGx25uAcWY2Mmn9Ib92ZlYFLCCqpfx1bNUGopq2+DGsdvevh1jHmll1N/H2RfI5sDbpMUe6+9/G1p+VtH6Yu2/sw/ugaChpG7r+iahvRPxXE+EX+a3At81sAkQfzGb2vrDJSKKkrimMYPxKfwMws7PN7GgzM6LEoYOo1unPRB8QXzezajMbZmbvCne7G/gXM5tmZiOIftXe202tHERfltea2bHhMUebWcrUBjFPEnWQfyosPxGWfx/7BZvsTaJ+GP31r8AlZvZ/zawmxHmcmd2T2CDUYNxJlGSMIerH0ZOHiZKA64mOT6KmZSRRMrAVKDOzLxP1ZenNnUQ1s+fQ/6TtPuDT4XwaQ9QUk223AleEmi4L58/7zWykmc0ws9NDYrGf6DxOHJc3galJyc2hGElUS7LfzE4gqtlO2BoeN37O9Om8NrPTzOztoYZ7F1FzUWeaTZ8lqtn5VzMrN7NTiZrR7kmzbZ+E9+2PgR8SfZ5sJhookJDJ+6Kv78+edPvad7N9v85pd19FVLvzr6FoJNFrsMfM3kI08jtT9wGfMrM6MxtL1OE+8TgbiJr+bgifgbOIjnM25kW7HfiLu/9HUvldwAfM7H1mVhoe91Qzqws/9hcDXzWzCjM7mehcOlR/Bnab2ectmjew1MzeZmZ/Fdb/APiamU0BMLPxZnZuuJ3p+6BoKGkbotz9VXdf3M3qzxM1Wz4Tqvt/S1S7BlHfnCqiGrlniJpO+2t62Pceotqmm919UUiOPkDUB+l1osESHwr3uZ3oA/YpotrC/cD/6e4B3P1BokTnnvBcXiQagt6dJ4k+hBNJ2++Jmouf6vYeUT+TL4Xq+6t72K67GP9I1EH8dOA1M9sO3EKUeMXdSfTL9l53b+llny1EtahnENX6JDxK9JqtJmra2E8GzZTu/geiD8Ml4cO7P24lGk28gmhk5cNECWR3yXCfhXP6MqJRbTuIzuNLwupKDnYifwOYQNR3Bg5O4dBoZkuyEMqVwPVmtpuog/V9sRj3EvUz/EM4Z95JH89rohGQ9xN9Ub1MdN6mJB7u3kr0XjqL6HnfDPyju/+lD8/lQ9Z1nrY94Qfdp4iO4b+FHxUfAz4WanMhg/dFP96f3erltU+3/aGc098ELg/H4WqipHw30Tl+bx/2cyvRe3I5sITUlo8LifoCbgIeBL7i7r/tY6zpXADMT3pN54VE8Vyi2sKtRJ8N/5eDucCHifqgbSf6wZ7cvabPwuf92UTdUtYSnae3EXXbgGi0+0KibjS7ib53TgzrMnofFJPEsGQRKXJm9jvgp+5+W5b2dxbRwIIpvW4skgPZPqdF8k01bSJCaKo4nr7VIiTvo8rM/tbMymJ9oR7MVowifZGNc1pksFHSJlLkzOwOombszySNZOvzrogmdN1B1Dz6MlHTYV/j+UKa5ro9Zvbr3u8tktVzWmRQUfOoiIiISAFQTZuIiIhIAVDSJiIiIlIA0k1uOuTU1tb61KlT8x2GiIiISK+ef/75be4+Prm8KJK2qVOnsnhxd1OWiYiIiAweZpZ2bkE1j4qIiIgUACVtIiIiIgVASZuIiIhIAVDSJiIiIlIAlLSJiIiIFICiGD0qIiL5tb6xmVuffo0FSzfR3NJOdWUZ582ZxGXzjmRKTXW+wxMpCEraREQkpxat2sKVdy2hraOT9s7o0ol7Wtq5588b+PnzG7n5ouM5bcaEPEcpMvipeVRERHJmfWMzV961hH1tHQcStoT2TmdfWwdX3rWE9Y3NeYpQpHCopk1ERLLK3WlsbmXdtma+9ZtV7G/v6HH7fW0dnPLNJygvNUpLjFIzSkqi22UlRomF8pKD6xO3S8woK03aJmn7aB2UlZRE+zUoLSmhtIQu9ylJPF5Yjt9OrOvLYx7YPuV5hMc3oyTEVVpCuE8JJUlxpT6XaH9mNkCvaHEbTE37StpERKTP3J0de9tYu62ZdduaWdfYHN1ubGb9tr3sbmnv8z7bOpy2Du99QwHAjK5JnRmlpUkJaCzJK0lKAhMJbNdEsSQktakJ6IFkNpaw9pycpkmyE8nvgcT8YAJ74HZyApsuSU6TvKc8z1JLOT4lJX1LdAdb076SNhER6VbT3tYDydjabXtZt62Z9SFB27W/74mZZI87tLunNDtLz9LWpqYkoNHx3bhjH+mObnun094ZNe0/8pl5A1bjpqRNRKTI7dzX1qW2bH3j3gOJWtPetnyHJ5JV7Z0OWUp02zo6ue3ptfz7eW/Lyv56o6RNRKQI7N7fxrpte1nb2Mz6bc2sbUw0a+5le3NrvsMTKUjtnc6DSzcqaRMRkb5pbmk/2K8sUVsWlrftGdjErLKshKk11UwYVcGfXt3eYxNeVXkpv/rUydSNHU5Hp9PhHv0Pf52hCbAzLLeHsvg28ft0hm06/OB9+rZf6OjsjP67H7jd6U57R+yx3enoOPg48cc8sN/kOJO2SYnDnc5OaO/ymJ10OinPQwaH5taB6yagpE1EpIDsbW1n3ba9Ub+yRG1ZqEHburtlQGOpKCthyrjhTK2tZlptNVNrqplaEy0fNmrYgU7f6TpzQ9S3qLy0hJsvOp4jx48Y0NgLnbsfSOQSSV88YU2X1HabKPaS1B643RFLat3p6Oikw0l5zEwT1sQ28Ti6f8x4jImkOjoG7Z2ddHbSY0KfeMxcqK4YuFRKSZuIyCCzv62DdY1RMrYuJGaJGrQ3dw1sYlZeahwxbviBpGxKbTXTaqqZWjucw0dXUZrBaLzTZkzgkc/M47an1/Lg0o00t7ZTXVHG/DmTuXTeNF0RoR/M7MAoT8lcSjKZrrY0JH43/mYVD7+wmY7O7vdXVmLMnzN5wOI396FfxTp37lxfvHhxvsMQETlgf1sHG7bvTRmZua6xmc079w9oLGUlUWI2NVFbVjucqTVR7dmkMZklZiJDzfrGZs78ztPsa+t+nsGq8tKcjB41s+fdfW5yuWraRERypKW9gw3b96XMY7Zu21427dzHQP5mLi0x6sdWHUzMag42a04eU0VZqS6QIxI3paaamy86vtem/YGsKc5p0mZmZwLfBUqB29z960nrpwC3A+OB7cBF7t4QWz8KeAlY4O5XhbKvAf8IjHV3dYIQkbxq6+hkw/a9KbVla7c1s6lpX7ZmFshIiUHd2ESN2cHasqm11dSNraJciZlInwy2pv2cNY+aWSmwGngP0AA8B1zo7i/FtvkZ8Et3v8PMTgc+5u4fja3/LiGhiyVt7wTWA69kmrSpeVREDkV7RycNO/bFOv43s7YxStA2Nu0b0JF8ZjB5TFVKM+bU2mrqxw6nokyJmUihy0fz6AnAGnd/LQRwD3AuUc1Zwkzgs+H2ImBBYoWZvQOYCDwCHAjc3Z8J63MYuogUm45OZ2MsMYtPnbFh+94Bn3V+0uhhUY3ZgY7/1UyrHU79uOFUlpUOaCwiMjjkMmmbDGyILTcAJyZtsxw4n6gJdT4w0sxqgB3At4CLgDP68+BmdjlwOcARRxzRn12IyBDT0elsatoXG5F5cOqMDdv3Dvh1Lw8bNYyptQdHZib6mB0xbjjDypWYiUhX+R6IcDXwPTO7BHgK2Ah0AFcCD7t7Q39r1Nz9FuAWiJpHsxKtiAx6nZ3O5l37D9aWhVn/1zU283rjXlp7Gr+fAxNGVqbUlk2trWbKuGqqKpSYiUjmcpm0bQTqY8t1oewAd99EVNOGmY0APujuTWZ2EjDPzK4ERgAVZrbH3a/JYbwiUiA6O503d+8PSdneAxcwTzRntrQPbGJWO6IySsZitWVTa6qZUjOc6sp8/zYWkaEil58mzwHTzWwaUbJ2AfDh+AZmVks0yKATuJZoJCnu/pHYNpcAc5WwiRQXd2fL7pautWWJKTMam9nfNrCJWU11xYHpMhK1ZYnEbOSw8gGNRUSKU86SNndvN7OrgEeJpvy43d1Xmtn1wGJ3XwicCtxgZk7UPPrJ3vZrZv9BlPwNN7MGoqlErsvR0xCRHHJ3tu5p6TLzf2LqjPWNzext7X5Sy1wYO7y8S1PmgakzaqsZpcRMRPJMV0QQkZxydxqbW7vUliVGaK5v3MueloG72DLA6KpEYnZwctkpNVGiNnq4EjMRyT9dEUFEcsbd2bG37WBtWWwes3Xbmtk9wInZyGFlXUZkHpj9v6aasdUVAxqLiEi2KGkTkYzt3NuWMo9Z4vau/QObmI2oLDswuezU+MjMmmrGVVdoLkcRGXKUtIlIF7v2t8Wmy9jb5ZqZTXvbBjSW4RWlXWb+j4/MrB2hxExEiouSNpEitKelvcs8ZvE+Zo3NrQMaS1V5KVNqhqfUlk2rrWb8yEolZiIigZI2kQKyvrGZW59+jQVLN9Hc0k51ZRnnzZnEZfOOTLlwcXNLe2i+jNWWhcEA2/a0DGjclWUlB6bHmHZgVGaUmE0cpcRMRCQTStpECsSiVVu48q4ltHV0HrgO5p6Wdu7+8+vc91wD58w+nBKzAyM0t+we2MSsorSEI2qGd5nHLDF1xmGjhlFSosRMRORQKGkTKQDrG5v5xF3Pp51QtqMTOujk/uc3prlndpWXGvXjhneZx2xa6HN2+OgqSpWYiYjkjJI2kUFob2s7L27cxfINTSxvaGLRqi0DdgWAspIoMZsam8cs0ZQ5aYwSMxGRfFHSJpJn7R2drH5zD8sbmli+oYllG5pY/eZuOnM473VpiVE3tupAMhZP0CaPqaKstCR3Dy4iIv2ipE1kALk7G7bvY1lI0JZvaOLFTTtzUotWYjC5S2KWmP1/OPXjhlOuxExEpKAoaRPJoW17WljR0MSyDTtZvqGJFQ1N7MjhXGeVZSXc9OHjmVpbTf24KirLSnP2WCIiMrCUtIlkSXNLOy9u3MmKhp0HatIaduwbsMcvKzH+YW49Z8ycOGCPKSIiA0dJm0g/tHV0svrN3SwPNWjLG7LfD61ubBXH1Y3huPrRTBw1jM//fEWPzajlpSVcOm9a9gIQEZFBRUmbSC/cnde372XZhqYoSWtoYmWW+6GNGV4eErQxzK4fzay6MdSOqOyyzaiq8pR52iCqYSsvLeHmi45PmWBXRESGDiVtIkm27Wk5MEhgeUOUpGXzmpuVZSW8ffJojqsPSVrdGOrHVfV6VYDTZkzgkc/M47an1/Lg0o00t7ZTXVHG/DmTuXTeNCVsIiJDnLnncF6BQWLu3Lm+ePHifIchg1CiH1o03cZOlm1oYmNT9vqhlRgcM3Eks+vHMCs0dR4zcaRGboqISLfM7Hl3n5tcrpo2KRptHZ2semP3gfnQlm/YyStbctAPLdSeHVc/hrdNHsXwCr3NRETk0OnbRIYkd2d9416WN0ST1a5o2MmLG3fS0p69fmhjh5dHTZx1Y0JN2mhqkvqhiYiIZIuSNhkStu5uOTAP2rKGaETnzn3Z64c2rDz0Q6sbw6w+9EMTERHJFiVtUnCaW9p5YePBqTaWb9iZ9X5oMw4bxXF1ow/UpB0zcYQu7SQiInmlpE0GtUQ/tGUHRnM2sWbLnqz2Q6sfV3WgifO4+jEcO0n90EREZPDRN5MMGu7Ousa9sRq0JlZu2pXVfmjjqisO1qDVj2HWZPVDExGRwqCkTfJmy+79rAiT1SYGC2SzH1pVeSlvnzyaWSFJm10/hrqx6ocmIiKFSUmbDIg9Le280LAzNt1GE5t27s/a/ktLLMyHNvrAlQWmT1A/NBERGTqUtEnWtbYnzYfW0MQrW/aQzXmcjxg3PAwSGM3s+jEcO2k0VRWl2XsAERGRQUZJmxySeD+0ZSFBW7lpF61Z7IdWU11xYBTnrFCTNq66Imv7FxERKQRK2qSL9Y3N3Pr0ayxYuonmlnaqK8s4b84kLpt3JFNqqtmye3900fTYYIFd+9uz9viJfmjH1R+cbkP90ERERJS0ScyiVVu48q4ltHV00h7m1NjT0s5Pn32du599ndFV5WzP4oXTS0uMGRNHhkECUZJ29Hj1QxMREUlHSZsAUQ3blXctYV9bR8q6xJxoh5qwTakZfmCQwOz60cw8XP3QREREMpXTpM3MzgS+C5QCt7n715PWTwFuB8YD24GL3L0htn4U8BKwwN2vCmXvAH4MVAEPA592z2YX9+J069Ov0daR3X5oiclqZ9VF/dDGqh+aiIhIv+UsaTOzUuAm4D1AA/CcmS1095dim90I3Onud5jZ6cANwEdj6/8deCpp198HLgOeJUrazgR+nZtnUTwWLN10oEm0r4ZXJPqhjQk1aaOZPEb90ERERLIplzVtJwBr3P01ADO7BziXqOYsYSbw2XB7EbAgsSLUqE0EHgHmhrLDgVHu/kxYvhM4DyVth6y5JfPBBMdOGhU1cYamzqMnjKC0RAmaiIhILuUyaZsMbIgtNwAnJm2zHDifqAl1PjDSzGqAHcC3gIuAM5L22RBbbghlKczscuBygCOOOKLfT6JYVFeWsSeDxG1EZRm/+tS8AYhIRERE4vI9TO9q4BQzWwqcAmwEOoArgYfj/dv6yt1vcfe57j53/Pjx2Yl2CDvjrRN63aasxJg/J22OLCIiIjmWy5q2jUB9bLkulB3g7puIatowsxHAB929ycxOAuaZ2ZXACKDCzPYQ1cjV9bRP6Z9MJqstLy3h0nnTBiAaERERSZbLpO05YLqZTSNKrC4APhzfwMxqge3u3glcSzSSFHf/SGybS4C57n5NWN5lZu8kGojwj8B/5/A5FAV3Z9Gqrd2uLysxyktLuPmi45lSUz2AkYmIiEhCzpI2d283s6uAR4mm/Ljd3Vea2fXAYndfCJwK3GBmTjRK9JMZ7PpKDk758Ws0COGQLW/YydptzSnlRtTXbf6cyVw6b5oSNhERkTyyYpjibO7cub548eJ8hzFofeWhF7njT+u7lL3zyHHcc/lJeYpIRESkeJnZ8+4+N7k83wMRJM/aOjr5xYrNKeXnz6lLs7WIiIjki5K2IvfU6q1sb27tUlZZVsKZbz8sTxGJiIhIOkraitwDS1MH354xcyKjhpXnIRoRERHpjpK2IrZrfxu/fenNlPL5szUXm4iIyGCjpK2IPfLCG7S0d71I/Njh5ZwyQ5MRi4iIDDZK2orYg2maRj9w3CTKS3VaiIiIDDb6di5Sm5r28czaxpRyXaZKRERkcFLSVqQeWraJ5Cn6ptVWM7t+TF7iERERkZ4paStC7s6DSxtSys+bPRkzy0NEIiIi0hslbUXopc27WP3mnpTy8+ZMykM0IiIikgklbUXowSWpAxDeMWWsri0qIiIyiClpKzIdnc5DyzellJ+nAQgiIiKDmpK2IvOHNdvYurulS1l5qXH22w/PU0QiIiKSCSVtRWZBmrnZTp0xgbHVFXmIRkRERDKlpK2I7G1t55GVb6SUn6+mURERkUFPSVsR+c3KN9nb2tGlbOSwMk57y4Q8RSQiIiKZUtJWRB5I0zR69qzDGVZemodoREREpC+UtBWJLbv38/tXtqaUnzdbTaMiIiKFQElbkVi4bBOdSZetmjymir+aOi4/AYmIiEifKGkrEguWpTaNzp8zmZISXbZKRESkEChpKwKvvLmbFzfuSinXhLoiIiKFQ0lbEXgwzQCEWXWjOXrCiDxEIyIiIv2hpG2I6+x0HlqW5rJVGoAgIiJSUJS0DXHPrt3OxqZ9XcpKS4wPHDcpTxGJiIhIfyhpG+LSXbZq3vRaxo+szEM0IiIi0l9K2oaw/W0dPPzC5pTy+RqAICIiUnCUtA1hj7+8hd0t7V3KqitKee/Mw/IUkYiIiPSXkrYh7MGlDSllZ77tcKoqdNkqERGRQpPTpM3MzjSzVWa2xsyuSbN+ipk9bmYrzOwJM6uLlS8xs2VmttLMrojd50Nh+5Vm9o1cxl/Itje38sSq1MtWqWlURESkMOUsaTOzUuAm4CxgJnChmc1M2uxG4E53nwVcD9wQyjcDJ7n7bOBE4Bozm2RmNcA3gXe7+7HAYWb27lw9h0L2yxWbaE+6btXEUZWcdFRNniISERGRQ5HLmrYTgDXu/pq7twL3AOcmbTMT+F24vSix3t1b3b0llFfG4jwSeMXdE1VIvwU+mKP4C1q6CXXPnT2ZUl22SkREpCDlMmmbDGyILTeEsrjlwPnh9nxgZKhNw8zqzWxF2Mc33H0TsAaYYWZTzawMOA+oz91TKExrtzWz9PWmlHI1jYqIiBSufA9EuBo4xcyWAqcAG4EOAHffEJpNjwYuNrOJ7r4D+ARwL/A0sC6xfTIzu9zMFpvZ4q1bU/t2DWXp5mZ7y2Ejeevho/IQjYiIiGRDLpO2jXStBasLZQe4+yZ3P9/d5wBfDGVNydsALwLzwvIv3P1Edz8JWAWsTvfg7n6Lu89197njx4/P0lMa/NydBctSkzbVsomIiBS2XCZtzwHTzWyamVUAFwAL4xuYWa2ZJWK4Frg9lNeZWVW4PRY4mShBw8wmxMqvBG7L4XMoOEteb2J9494uZWZwzmxdtkpERKSQ5Sxpc/d24CrgUeBl4D53X2lm15vZOWGzU4FVZrYamAh8LZS/FXjWzJYDTwI3uvsLYd13zewl4A/A1909bU1bsUo3N9tfH1XD4aOr8hCNiIiIZEtZLnfu7g8DDyeVfTl2+37g/jT3ewyY1c0+L8xymENGa3snv1yRetmq82araVRERKTQ5XsggmTRE6u20LS3rUvZsPISznybLlslIiJS6JS0DSHpBiC8Z+ZhjBxWnodoREREJJuUtA0RO/e18duXt6SUn69RoyIiIkOCkrYh4tcvbKa1vbNLWU11BSdPr81TRCIiIpJNStqGiAfSTKj7geMmUV6ql1hERGQo0Df6ENCwYy9/Xrs9pVwT6oqIiAwdStqGgIeWbUopO3J8NbPqRuchGhEREckFJW0Fzt15YEnqhLrnz5mMmeUhIhEREckFJW0F7sWNu3h1a3NK+bmaUFdERGRIUdJW4B5Ic9mqE6aOo37c8DxEIyIiIrmipK2AtXd08ovlqf3ZztMABBERkSFHSVsBe3rNNrbtae1SVlFawvvffnieIhIREZFcyShpM7MHzOz9ZqYkbxBZkGZuttPfMoHRw3XZKhERkaEm0yTsZuDDwCtm9nUzm5HDmCQDe1raeXTlGynl849X06iIiMhQlFHS5u6/dfePAMcD64DfmtkfzexjZqZqnTx49MU32N/W9bJVo6vKOXXG+DxFJCIiIrmUcXOnmdUAlwCXAkuB7xIlcY/lJDLp0YNpmkbPnnU4lWWleYhGREREcq0sk43M7EFgBvAT4APuvjmsutfMFucqOEnvzV37+cOr21LKddkqERGRoSujpA34L3dflG6Fu8/NYjySgYeWbcS9a1n9uCreMWVsfgISERGRnMu0eXSmmY1JLJjZWDO7MjchSW8eXJo6N9v82bpslYiIyFCWadJ2mbs3JRbcfQdwWU4ikh795Y1dvLx5V0q5JtQVEREZ2jJN2kotVo1jZqVARW5Ckp6kG4BwXP0Yjhw/Ig/RiIiIyEDJtE/bI0SDDv4nLP9zKJMB1NHpPJSmafR81bKJiIgMeZkmbZ8nStQ+EZYfA27LSUTSrWdfa+SNXfu7lJWVGGfP0mWrREREhrqMkjZ37wS+H/4kTx5I0zR6yjHjqRlRmYdoREREZCBlOk/bdOAGYCYwLFHu7kfmKC5Jsq+1g0deTL1slQYgiIiIFIdMByL8iKiWrR04DbgTuCtXQUmqx15+kz0t7V3KRlSW8Z6ZE/MUkYiIiAykTJO2Knd/HDB3X+/u1wHvz11YkmxBmqbRs952GMPKddkqERGRYpDpQIQWMysBXjGzq4CNgOaYGCDb9rTw5OqtKeXzj1fTqIiISLHItKbt08Bw4FPAO4CLgItzFZR09Yvlm+jo7HrdqsNHD+Od02ryFJGIiIgMtF6TtjCR7ofcfY+7N7j7x9z9g+7+TAb3PdPMVpnZGjO7Js36KWb2uJmtMLMnzKwuVr7EzJaZ2UozuyJ2nwvN7IVwn0fMrLaPz7ngpGsaPXf2ZEpKdNkqERGRYtFr0ubuHcDJfd1xSPZuAs4iGnV6oZnNTNrsRuBOd58FXE80QhVgM3CSu88GTgSuMbNJZlYGfBc4LdxnBXBVX2MrJK9u3cPyhp0p5fM1alRERKSoZNqnbamZLQR+BjQnCt39gR7ucwKwxt1fAzCze4BzgZdi28wEPhtuLwIWhP22xrap5GByaeGv2swagVHAmgyfQ0FKV8s28/BRzDhsZB6iERERkXzJtE/bMKAROB34QPg7u5f7TAY2xJYbQlnccuD8cHs+MNLMagDMrN7MVoR9fMPdN7l7G9FVGV4ANhElfT/M8DkUnM5OT3utUdWyiYiIFJ9Mr4jwsRw9/tXA98zsEuApolGpHeExNwCzzGwSsMDM7ge2EyVtc4DXgP8GrgX+X/KOzexy4HKAI444Ikfh59bzr++gYce+LmUlBufOnpSniERERCRfMr0iwo8ATy5394/3cLeNQH1suS6Uxe+/iVDTZmYjgA+6e1PyNmb2IjAPWB/KXg33uQ9IGeAQtrkFuAVg7ty5KbEXggeWpNayvevoWiaMGpZmaxERERnKMm0e/SXwq/D3OFFfsj293Oc5YLqZTTOzCuACYGF8AzOrDfO/QVRjdnsorzOzqnB7LNFAiFVESd9MMxsf7vMe4OUMn0NBaWnv4FcrNqWUq2lURESkOGXaPPrz+LKZ3Q38vpf7tIeJeB8FSoHb3X2lmV0PLHb3hcCpwA1m5kTNo58Md38r8K1QbsCN7v5CeOyvAk+ZWRtRzdslmTyHQrPoL1vYtb/rZauqykt537GH5SkiERERyadMR48mmw5M6G0jd38YeDip7Mux2/cD96e532PArG72+QPgB32Mt+CkG4DwvmMnUl3Z35dMREREClmmfdp207VP2xvA53MSkdC0t5Xf/WVLSvn84+vyEI2IiIgMBpk2j2pSsAH0qxc209bRdexE7YhK3nWULlslIiJSrDIaiGBm881sdGx5jJmdl7OoityDaUaNnjt7EmWlmY4bERERkaEm0yzgK+5+4FpKYVqOr+QkoiL3euNeFq/fkVKuUaMiIiLFLdOkLd126hGfAwuWpdayTZ8wgmMnjcpDNCIiIjJYZJq0LTaz/zSzo8LffwLP5zKwYuTuaa81et6cyZhZHiISERGRwSLTpO3/AK3AvcA9wH4OzqkmWbK8YSevbWtOKT9PTaMiIiJFL9PRo810c7koyZ50tWwnThvH5DFVeYhGREREBpNMR48+ZmZjYstjzezRnEVVhNo6OvnF8tTLVp1/vGrZREREJPPm0dr4hdzdfQcZXBFBMvf0K1tpbG7tUlZRVsKZbzs8TxGJiIjIYJJp0tZpZkckFsxsKl2vkCCHYH1jM19d+FJK+V8fWcPoqvI8RCQiIiKDTabTdnwR+L2ZPUl0Afd5wOU5i6qILFq1hU/c9Tz72zpT1v3x1UYWrdrCaTNUqSkiIlLsMqppc/dHgLnAKuBu4HPAvhzGVRTWNzZz5V1L0iZsAK0dnVx51xLWN6aOKBUREZHikulAhEuBx4mStauBnwDX5S6s4nDr06/R1pE+YUto6+jktqfXDlBEIiIiMlhl2qft08BfAevd/TRgDtCUq6CKxYKlm2jv7LlrYHun82CaqUBERESkuGSatO139/0AZlbp7n8BZuQurOLQ3NKe2XatmW0nIiIiQ1emAxEawjxtC4DHzGwHsD5XQRWL6soy9mSQuFVX6DKvIiIixS7TgQjz3b3J3a8D/g34IXBeDuMqCufNmURJL5cULSsx5usyViIiIkUv0+bRA9z9SXdf6O6tvW8tPbls3pH0dhn48tISLp03bUDiERERkcGrz0mbZM+UmmomjhqWdl1ZiVFVXsrNFx3PlJrqAY5MREREBht1lsqj1vZOtuxuSSmvrizl/Dl1XDpvmhI2ERERAZS05dXabc0pU37Ujqhk8ZfOyFNEIiIiMlipeTSPVr+5O6VsxmEj8hCJiIiIDHZK2vIoXdI2fcLIPEQiIiIig52StjxKX9OmpE1ERERSKWnLo9Vv7kkpO2aimkdFREQklZK2PNnf1sH6xuaU8ukTVdMmIiIiqZS05cmrW/eQfK34w0cPY9Sw8vwEJCIiIoOakrY8STsIQbVsIiIi0o2cJm1mdqaZrTKzNWZ2TZr1U8zscTNbYWZPmFldrHyJmS0zs5VmdkUoHxnKEn/bzOw7uXwOuZKuP9sM9WcTERGRbuRscl0zKwVuAt4DNADPmdlCd38pttmNwJ3ufoeZnQ7cAHwU2Ayc5O4tZjYCeDHcdxMwO/YYzwMP5Oo55NLqN1TTJiIiIpnLZU3bCcAad38tXFz+HuDcpG1mAr8Ltxcl1rt7q7snru9UmS5OMzsGmAA8nYPYc271ljTTfShpExERkW7kMmmbDGyILTeEsrjlwPnh9nxgpJnVAJhZvZmtCPv4Rqhli7sAuNfdk7rzD37NLe1s2L4vpfzoCWoeFRERkfTyPRDhauAUM1sKnAJsBDoA3H2Du88CjgYuNrOJSfe9ALi7ux2b2eVmttjMFm/dujU30ffTmi2p/dnqx1VRXalLwYqIiEh6uUzaNgL1seW6UHaAu29y9/PdfQ7wxVDWlLwN8CIwL1FmZscBZe7+fHcP7u63uPtcd587fvz4Q30uWbUqzcjRY3T5KhEREelBLpO254DpZjbNzCqIasYWxjcws1ozS8RwLXB7KK8zs6pweyxwMrAqdtcL6aGWbbB7JV3SpstXiYiISA9ylrS5eztwFfAo8DJwn7uvNLPrzeycsNmpwCozWw1MBL4Wyt8KPGtmy4EngRvd/YXY7v+BAk7aVunyVSIiItJHOe1E5e4PAw8nlX05dvt+4P4093sMmNXDfo/MYpgDLm1Nm0aOioiISA/yPRCh6Ozc18bmnfu7lJUYHDVeNW0iIiLSPSVtA2xNmvnZptZUM6y8NA/RiIiISKFQ0jbA0l2+arr6s4mIiEgvlLQNsFVpLl+lKyGIiIhIb5S0DbBX0jSP6pqjIiIi0hslbQNs1RvppvtQ0iYiIiI9U9I2gLY3t7JtT0uXsrISY1ptdZ4iEhERkUKhpG0ArU4zP9u02moqyvQyiIiISM+ULQwgXb5KRERE+ktJ2wDSheJFRESkv5S0DaB0c7TNOExztImIiEjvlLQNEHdP26dN032IiIhIJpS0DZCte1po2tvWpayirIQp44bnKSIREREpJEraBsjqNPOzHTV+BGWleglERESkd8oYBki6ptEZuuaoiIiIZEhJ2wDR5atERETkUChpGyC6ULyIiIgcCiVtA8DdeSXNdB+65qiIiIhkSknbANi8cz+7W9q7lFWVl1I3tipPEYmIiEihUdI2ANLPzzaCkhLLQzQiIiJSiJS0DYB0SZuaRkVERKQvlLQNgHSXrzpG032IiIhIHyhpGwC6fJWIiIgcKiVtOdbZmX7kqKb7EBERkb5Q0pZjDTv2sa+to0vZyMoyDh89LE8RiYiISCFS0pZj3Y0cNdPIUREREcmckrYcW6WRoyIiIpIFStpy7BUlbSIiIpIFStpyLP10H0raREREpG9ymrSZ2ZlmtsrM1pjZNWnWTzGzx81shZk9YWZ1sfIlZrbMzFaa2RWx+1SY2S1mttrM/mJmH8zlczgUHZ3Omq1pkrbDNEebiIiI9E1ZrnZsZqXATcB7gAbgOTNb6O4vxTa7EbjT3e8ws9OBG4CPApuBk9y9xcxGAC+G+24CvghscfdjzKwEGJer53Co1jc209re2aVszPByxo+ozFNEIiIiUqhylrQBJwBr3P01ADO7BzgXiCdtM4HPhtuLgAUA7t4a26aSrjWCHwfeErbrBLblIPas6O7yVRo5KiIiIn2Vy+bRycCG2HJDKItbDpwfbs8HRppZDYCZ1ZvZirCPb7j7JjMbE7b999B8+jMzm5izZ3CIdPkqERERyZZ8D0S4GjjFzJYCpwAbgQ4Ad9/g7rOAo4GLQ3JWBtQBf3T344E/ETWxpjCzy81ssZkt3rp16wA8lVTppvvQlRBERESkP3KZtG0E6mPLdaHsAHff5O7nu/scor5quHtT8jbAi8A8oBHYCzwQVv8MOD7dg7v7Le4+193njh8//tCfTT+km+5D1xwVERGR/shl0vYcMN3MpplZBXABsDC+gZnVhsEEANcCt4fyOjOrCrfHAicDq9zdgV8Ap4b7vJuufeQGjdb2Tl7b2pxSruk+REREpD9yNhDB3dvN7CrgUaAUuN3dV5rZ9cBid19IlHzdYGYOPAV8Mtz9rcC3QrkBN7r7C2Hd54GfmNl3gK3Ax3L1HA7FusZm2ju9S1ntiErGVVfkKSIREREpZLkcPYq7Pww8nFT25djt+4H709zvMWBWN/tcD/xNdiPNvlVvpOnPpvnZREREpJ/yPRBhyErbn22CmkZFRESkf5S05YguFC8iIiLZpKQtR15JM0ebmkdFRESkv5S05cD+tg7WNaaOHD1azaMiIiLST0racuDVrXtIGjjK4aOHMbqqPD8BiYiISMFT0pYD6ZpGNamuiIiIHAolbTmQ/vJV6s8mIiIi/aekLQd0+SoRERHJNiVtOaALxYuIiEi2KWnLsr2t7WzYvi+l/OgJah4VERGR/lPSlmXpBiHUj6uiujKnVwwTERGRIU5JW5atTnclBM3PJiIiIodISVuWpU3aDlPSJiIiIodGSVuWrU7TPHqMpvsQERGRQ6SkLcvS1rRp5KiIiIgcIiVtWbRrfxubd+7vUlZicNR41bSJiIjIoVHSlkXpJtWdUlPNsPLSPEQjIiIiQ4mStixSfzYRERHJFSVtWaT+bCIiIpIrStqySEmbiIiI5IqStixK3zyqpE1EREQOnZK2LNnR3MrW3S1dyspKjGm11XmKSERERIYSJW1Zkq5pdFptNRVlOsQiIiJy6JRRZIkuXyUiIiK5pKQtS9L2Z9OF4kVERCRLlLRlyao0NW0zDtMcbSIiIpIdStqywN3TXg1hukaOioiISJYoacuCrXta2LG3rUtZRVkJU8YNz1NEIiIiMtQoacuCV9L0Zztq/AjKSnV4RUREJDtymlWY2ZlmtsrM1pjZNWnWTzGzx81shZk9YWZ1sfIlZrbMzFaa2RWx+zwR9rks/E3I5XPIxKo30vRn0zVHRUREJIvKcrVjMysFbgLeAzQAz5nZQnd/KbbZjcCd7n6HmZ0O3AB8FNgMnOTuLWY2Angx3HdTuN9H3H1xrmLvq1e2qD+biIiI5FYua9pOANa4+2vu3grcA5ybtM1M4Hfh9qLEendvdffE5QUqcxznIUs33ccMJW0iIiKSRblMhiYDG2LLDaEsbjlwfrg9HxhpZjUAZlZvZivCPr4Rq2UD+FFoGv03M7PchJ8Zd2d1muZRXXNUREREsinfNVhXA6eY2VLgFGAj0AHg7hvcfRZwNHCxmU0M9/mIu78dmBf+Pppux2Z2uZktNrPFW7duzdkTeGPXfna3tHcpqyovpW5sVc4eU0RERIpPLpO2jUB9bLkulB3g7pvc/Xx3nwN8MZQ1JW8DvEiUoOHuG8P/3cBPiZphU7j7Le4+193njh8/PitPKJ10gxCmTxxBSUleKwBFRERkiMll0vYcMN3MpplZBXABsDC+gZnVmlkihmuB20N5nZlVhdtjgZOBVWZWZma1obwcOJsoocubdNN9TNflq0RERCTLcpa0uXs7cBXwKPAycJ+7rzSz683snLDZqUTJ2GpgIvC1UP5W4FkzWw48Cdzo7i8QDUp4NPR1W0ZUc3drrp5DJnT5KhERERkIOZvyA8DdHwYeTir7cuz2/cD9ae73GDArTXkz8I7sR9o/6xubWfSXLSnlo6vK8xCNiIiIDGX5HohQsBat2sKZ33maxubWlHVfWbiSRatSkzkRERGR/lLS1g/rG5u58q4l7GvrSLt+f1snV961hPWNzQMcmYiIiAxVStr64danX6Oto7PHbdo6Ornt6bUDFJGIiIgMdUra+mHB0k20d3qP27R3Og8u3djjNiIiIiKZUtLWD81Jk+l2u11rZtuJiIiI9EZJWz9UV2Y26La6IqeDc0VERKSIKGnrh/PmTKKslyselJUY8+ckX2pVREREpH+UtPXDZfOOpLy050NXXlrCpfOmDVBEIiIiMtQpaeuHKTXV3HzR8VSVl6bUuJWVGFXlpdx80fFMqanOU4QiIiIy1Chp66fTZkzgkc/M48ITjmBEZRlmMKKyjAtPOIJHPjOP02ZMyHeIIiIiMoSYe89TVwwFc+fO9cWLF+c7DBEREZFemdnz7j43uVw1bSIiIiIFQEmbiIiISAFQ0iYiIiJSAJS0iYiIiBQAJW0iIiIiBaAoRo+a2VZgfQ+b1ALbBiicQqLjkp6OSyodk/R0XNLTcUlPxyW9YjwuU9x9fHJhUSRtvTGzxemG1hY7HZf0dFxS6Zikp+OSno5Lejou6em4HKTmUREREZECoKRNREREpAAoaYvcku8ABikdl/R0XFLpmKSn45Kejkt6Oi7p6bgE6tMmIiIiUgBU0yYiIiJSAIo6aTOzM81slZmtMbNr8h1PvphZvZktMrOXzGylmX06lI8zs8fM7JXwf2y+Y80HMys1s6Vm9suwPM3Mng3nzb1mVpHvGAeamY0xs/vN7C9m9rKZnaTzBczsX8J76EUzu9vMhhXj+WJmt5vZFjN7MVaW9vywyH+F47PCzI7PX+S51c1x+WZ4H60wswfNbExs3bXhuKwys/flJegBkO64xNZ9zszczGrDctGcL+kUbdJmZqXATcBZwEzgQjObmd+o8qYd+Jy7zwTeCXwyHItrgMfdfTrweFguRp8GXo4tfwP4trsfDewA/ikvUeXXd4FH3P0twHFEx6eozxczmwx8Cpjr7m8DSoELKM7z5cfAmUll3Z0fZwHTw9/lwPcHKMZ8+DGpx+Ux4G3uPgtYDVwLED6DLwCODfe5OXxvDUU/JvW4YGb1wHuB12PFxXS+pCjapA04AVjj7q+5eytwD3BunmPKC3ff7O5Lwu3dRF/Ak4mOxx1hszuA8/ISYB6ZWR3wfuC2sGzA6cD9YZOiOy5mNhr4G+CHAO7e6u5N6HwBKAOqzKwMGA5spgjPF3d/CtieVNzd+XEucKdHngHGmNnhAxLoAEt3XNz9N+7eHhafAerC7XOBe9y9xd3XAmuIvreGnG7OF4BvA/8KxDvfF835kk4xJ22TgQ2x5YZQVtTMbCowB3gWmOjum8OqN4CJ+Yorj75D9KHRGZZrgKbYh2wxnjfTgK3Aj0Kz8W1mVk2Rny/uvhG4kahWYDOwE3genS8J3Z0f+iw+6OPAr8Ptoj4uZnYusNHdlyetKurjUsxJmyQxsxHAz4HPuPuu+DqPhhkX1VBjMzsb2OLuz+c7lkGmDDge+L67zwGaSWoKLdLzZSxRLcA0YBJQTZomHynO86M3ZvZFoq4q/5vvWPLNzIYDXwC+nO9YBptiTto2AvWx5bpQVpTMrJwoYftfd38gFL+ZqHYO/7fkK748eRdwjpmtI2o+P52oL9eY0PwFxXneNAAN7v5sWL6fKIkr9vPlDGCtu2919zbgAaJzqNjPl4Tuzo+i/yw2s0uAs4GP+MF5uIr5uBxF9ONnefj8rQOWmNlhFPdxKeqk7TlgehjZVUHU4XNhnmPKi9BP64fAy+7+n7FVC4GLw+2LgYcGOrZ8cvdr3b3O3acSnR+/c/ePAIuAvwubFeNxeQPYYGYzQtG7gZco8vOFqFn0nWY2PLynEselqM+XmO7Oj4XAP4ZRge8EdsaaUYc8MzuTqAvGOe6+N7ZqIXCBmVWa2TSijvd/zkeMA83dX3D3Ce4+NXz+NgDHh8+e4j5finlyXTP7W6I+S6XA7e7+tfxGlB9mdjLwNPACB/tufYGoX9t9wBHAeuAf3D1dZ9Ehz8xOBa5297PN7EiimrdxwFLgIndvyWN4A87MZhMNzqgAXgM+RvQjsKjPFzP7KvAhomaupcClRP1tiup8MbO7gVOBWuBN4CvAAtKcHyHB/R5RU/Je4GPuvjgPYedcN8flWqASaAybPePuV4Ttv0jUz62dqNvKr5P3ORSkOy7u/sPY+nVEo7K3FdP5kk5RJ20iIiIihaKYm0dFRERECoaSNhEREZECoKRNREREpAAoaRMREREpAEraRERERAqAkjYRERGRAqCkTUQkiZnNDvM4JpbPMbNrerpPH/b9mXCZHhGRPtE8bSIiScJlhea6+1U52Pe6sO9tfbhPqbt3ZDsWESksqmkTkYJlZlPN7GUzu9XMVprZb8ysqpttjzKzR8zseTN72szeEsr/3sxeNLPlZvZUuKzd9cCHzGyZmX3IzC4xs++F7X9sZt83s2fM7DUzO9XMbg9x/Dj2eN83s8Uhrq+Gsk8RXUx+kZktCmUXmtkLIYZvxO6/x8y+ZWbLgZPM7Otm9pKZrTCzG3NzREVkMFNNm4gULDObCqwhqrlaZmb3AQvd/a402z4OXOHur5jZicAN7n66mb0AnOnuG81sjLs3Jde0xZdDYjYMuBA4B/gJ0YXhVxJd0/ifQizjwmWaSoHHgU+5+4qkS/JMAp4B3gHsAH4D/Je7LzAzBz7k7veZWQ3wR+At7u6JOLN+QEVkUFNNm4gUurXuvizcfh6YmryBmY0A/hr4mZktA/4HODys/gPwYzO7jOg6xJn4hUe/eF8A3gwXuO4kStwSj/8PZraE6HqjxwIz0+znr4An3H2ru7cD/wv8TVjXAfw83N4J7Ad+aGbnE11zUUSKTFm+AxAROUTxi693AOmaR0uAJnefnbzC3a8INW/vB543s3f04TE7kx6/Eygzs2nA1cBfufuOWO1cX+xP9GNz93YzOwF4N/B3wFXA6X3cn4gUONW0iciQ5+67gLVm9vcAFjku3D7K3Z919y8DW4F6YDcw8hAechTQDOw0s4nAWbF18X3/GTjFzGpDM+qFwJPJOws1haPd/WHgX4DjDiE2ESlQqmkTkWLxEeD7ZvYloBy4B1gOfNPMpgNG1PdsOfA6cE1oSr2hrw/k7svNbCnwF2ADURNswi3AI2a2yd1PC1OJLAqP/yt3fyjNLkcCD5nZsLDdZ/sak4gUPg1EEBERESkAah4VERERKQBqHhWRIcXMbiKagiPuu+7+o3zEIyKSLWoeFRERESkAah4VERERKQBK2kREREQKgJI2ERERkQKgpE1ERESkAChpExERESkA/z9XEvQEfL29PQAAAABJRU5ErkJggg==\n",
      "text/plain": [
       "<Figure size 720x360 with 1 Axes>"
      ]
     },
     "metadata": {
      "needs_background": "light"
     },
     "output_type": "display_data"
    }
   ],
   "source": [
    "show_curve(param_grid[\"classifier__n_estimators\"],\n",
    "           grid_search.cv_results_['mean_test_score'],\n",
    "           \"n_estimators\",\n",
    "           \"accuracy\",\n",
    "           \"Mean score with CV varying n_estimators of Extremely Randomized Trees\")"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 43,
   "id": "d21edc2d",
   "metadata": {
    "execution": {
     "iopub.execute_input": "2022-03-16T15:20:26.909774Z",
     "iopub.status.busy": "2022-03-16T15:20:26.909116Z",
     "iopub.status.idle": "2022-03-16T15:20:26.911816Z",
     "shell.execute_reply": "2022-03-16T15:20:26.912270Z",
     "shell.execute_reply.started": "2022-03-14T13:23:47.063811Z"
    },
    "papermill": {
     "duration": 0.126647,
     "end_time": "2022-03-16T15:20:26.912438",
     "exception": false,
     "start_time": "2022-03-16T15:20:26.785791",
     "status": "completed"
    },
    "tags": []
   },
   "outputs": [],
   "source": [
    "# Define the final model with the best parameter found\n",
    "best_n_estimators = grid_search.best_params_[\"classifier__n_estimators\"]\n",
    "ext_rnd_trees_model = ExtraTreesClassifier(criterion=\"entropy\", n_estimators=best_n_estimators, random_state=random_seed)"
   ]
  },
  {
   "cell_type": "markdown",
   "id": "ed796f13",
   "metadata": {
    "papermill": {
     "duration": 0.116628,
     "end_time": "2022-03-16T15:20:27.146105",
     "exception": false,
     "start_time": "2022-03-16T15:20:27.029477",
     "status": "completed"
    },
    "tags": []
   },
   "source": [
    "### **Deep Neural Network**"
   ]
  },
  {
   "cell_type": "markdown",
   "id": "4d060687",
   "metadata": {
    "papermill": {
     "duration": 0.116428,
     "end_time": "2022-03-16T15:20:27.378074",
     "exception": false,
     "start_time": "2022-03-16T15:20:27.261646",
     "status": "completed"
    },
    "tags": []
   },
   "source": [
    "Now we will test a simple deep neural network architecture, using the keras library."
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 44,
   "id": "980cefa5",
   "metadata": {
    "execution": {
     "iopub.execute_input": "2022-03-16T15:20:27.613927Z",
     "iopub.status.busy": "2022-03-16T15:20:27.612241Z",
     "iopub.status.idle": "2022-03-16T15:20:33.276501Z",
     "shell.execute_reply": "2022-03-16T15:20:33.275877Z",
     "shell.execute_reply.started": "2022-03-14T13:23:47.065071Z"
    },
    "papermill": {
     "duration": 5.782612,
     "end_time": "2022-03-16T15:20:33.276647",
     "exception": false,
     "start_time": "2022-03-16T15:20:27.494035",
     "status": "completed"
    },
    "tags": []
   },
   "outputs": [],
   "source": [
    "import tensorflow as tf\n",
    "from tensorflow import keras\n",
    "from tensorflow.keras import layers"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 45,
   "id": "a1439e56",
   "metadata": {
    "execution": {
     "iopub.execute_input": "2022-03-16T15:20:33.517924Z",
     "iopub.status.busy": "2022-03-16T15:20:33.516993Z",
     "iopub.status.idle": "2022-03-16T15:20:33.659482Z",
     "shell.execute_reply": "2022-03-16T15:20:33.658853Z",
     "shell.execute_reply.started": "2022-03-14T13:23:47.066446Z"
    },
    "papermill": {
     "duration": 0.26724,
     "end_time": "2022-03-16T15:20:33.659623",
     "exception": false,
     "start_time": "2022-03-16T15:20:33.392383",
     "status": "completed"
    },
    "tags": []
   },
   "outputs": [
    {
     "name": "stdout",
     "output_type": "stream",
     "text": [
      "Model: \"dnn_model\"\n",
      "_________________________________________________________________\n",
      "Layer (type)                 Output Shape              Param #   \n",
      "=================================================================\n",
      "input_1 (InputLayer)         [(None, 5)]               0         \n",
      "_________________________________________________________________\n",
      "dense (Dense)                (None, 30)                180       \n",
      "_________________________________________________________________\n",
      "dense_1 (Dense)              (None, 50)                1550      \n",
      "_________________________________________________________________\n",
      "dense_2 (Dense)              (None, 50)                2550      \n",
      "_________________________________________________________________\n",
      "dense_3 (Dense)              (None, 30)                1530      \n",
      "_________________________________________________________________\n",
      "dense_4 (Dense)              (None, 20)                620       \n",
      "_________________________________________________________________\n",
      "dense_5 (Dense)              (None, 10)                210       \n",
      "=================================================================\n",
      "Total params: 6,640\n",
      "Trainable params: 6,640\n",
      "Non-trainable params: 0\n",
      "_________________________________________________________________\n"
     ]
    },
    {
     "name": "stderr",
     "output_type": "stream",
     "text": [
      "2022-03-16 15:20:33.568325: I tensorflow/core/common_runtime/process_util.cc:146] Creating new thread pool with default inter op setting: 2. Tune using inter_op_parallelism_threads for best performance.\n"
     ]
    }
   ],
   "source": [
    "# Network architecture definition\n",
    "inputs = keras.Input(shape=(pca_n_components,))\n",
    "x = layers.Dense(30, activation=\"relu\")(inputs)\n",
    "x = layers.Dense(50, activation=\"relu\")(x)\n",
    "x = layers.Dense(50, activation=\"relu\")(x)\n",
    "x = layers.Dense(30, activation=\"relu\")(x)\n",
    "x = layers.Dense(20, activation=\"relu\")(x)\n",
    "outputs = layers.Dense(10, activation=\"softmax\")(x)\n",
    "\n",
    "dnn_model = keras.Model(inputs=inputs, outputs=outputs, name=\"dnn_model\")\n",
    "dnn_model.summary()"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 46,
   "id": "0986911f",
   "metadata": {
    "execution": {
     "iopub.execute_input": "2022-03-16T15:20:33.896746Z",
     "iopub.status.busy": "2022-03-16T15:20:33.895990Z",
     "iopub.status.idle": "2022-03-16T15:20:33.908547Z",
     "shell.execute_reply": "2022-03-16T15:20:33.909071Z",
     "shell.execute_reply.started": "2022-03-14T13:23:47.067946Z"
    },
    "papermill": {
     "duration": 0.130195,
     "end_time": "2022-03-16T15:20:33.909249",
     "exception": false,
     "start_time": "2022-03-16T15:20:33.779054",
     "status": "completed"
    },
    "tags": []
   },
   "outputs": [],
   "source": [
    "dnn_model.compile(\n",
    "    loss=keras.losses.SparseCategoricalCrossentropy(from_logits=False),\n",
    "    optimizer=keras.optimizers.Adam(),\n",
    "    metrics=[\"accuracy\"]\n",
    ")"
   ]
  },
  {
   "cell_type": "markdown",
   "id": "5e12f8d8",
   "metadata": {
    "papermill": {
     "duration": 0.115602,
     "end_time": "2022-03-16T15:20:34.140056",
     "exception": false,
     "start_time": "2022-03-16T15:20:34.024454",
     "status": "completed"
    },
    "tags": []
   },
   "source": [
    "This time we are not using a sklearn pipeline, so we need to compute the PCA transformation manually on the training set.\n",
    "We need to split further the training set for training and validating the model, also being careful not to use the validation data to fit the PCA."
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 47,
   "id": "ebd5fbff",
   "metadata": {
    "execution": {
     "iopub.execute_input": "2022-03-16T15:20:34.375589Z",
     "iopub.status.busy": "2022-03-16T15:20:34.374913Z",
     "iopub.status.idle": "2022-03-16T15:20:38.817574Z",
     "shell.execute_reply": "2022-03-16T15:20:38.818752Z",
     "shell.execute_reply.started": "2022-03-14T13:23:47.069566Z"
    },
    "papermill": {
     "duration": 4.562175,
     "end_time": "2022-03-16T15:20:38.819056",
     "exception": false,
     "start_time": "2022-03-16T15:20:34.256881",
     "status": "completed"
    },
    "tags": []
   },
   "outputs": [
    {
     "name": "stdout",
     "output_type": "stream",
     "text": [
      "Shape of X_train_t_pca: (107200, 5)\n",
      "Shape of X_train_val_pca: (26800, 5)\n"
     ]
    }
   ],
   "source": [
    "# Preprocess training set\n",
    "\n",
    "# Further split the training set in training and validation sets\n",
    "X_train_t, X_train_val, Y_train_t, Y_train_val = train_test_split(X_train, Y_train, test_size=0.20, stratify=Y_train, random_state=random_seed)\n",
    "\n",
    "# Standardize the features\n",
    "st_scaler_dnn = StandardScaler()\n",
    "st_scaler_dnn.fit(X_train_t)\n",
    "X_train_t = st_scaler_dnn.transform(X_train_t)\n",
    "X_train_val = st_scaler_dnn.transform(X_train_val)\n",
    "\n",
    "# Perform PCA\n",
    "pca_dnn = PCA(n_components=pca_n_components, random_state=random_seed)\n",
    "pca_dnn.fit(X_train_t)\n",
    "X_train_t_pca = pca_dnn.transform(X_train_t)\n",
    "X_train_val_pca = pca_dnn.transform(X_train_val)\n",
    "\n",
    "print(f\"Shape of X_train_t_pca: {X_train_t_pca.shape}\")\n",
    "print(f\"Shape of X_train_val_pca: {X_train_val_pca.shape}\")"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 48,
   "id": "8d80b166",
   "metadata": {
    "execution": {
     "iopub.execute_input": "2022-03-16T15:20:39.115036Z",
     "iopub.status.busy": "2022-03-16T15:20:39.114422Z",
     "iopub.status.idle": "2022-03-16T15:22:01.904101Z",
     "shell.execute_reply": "2022-03-16T15:22:01.904597Z",
     "shell.execute_reply.started": "2022-03-14T13:23:47.071962Z"
    },
    "papermill": {
     "duration": 82.915043,
     "end_time": "2022-03-16T15:22:01.904805",
     "exception": false,
     "start_time": "2022-03-16T15:20:38.989762",
     "status": "completed"
    },
    "tags": []
   },
   "outputs": [
    {
     "name": "stderr",
     "output_type": "stream",
     "text": [
      "2022-03-16 15:20:39.359189: I tensorflow/compiler/mlir/mlir_graph_optimization_pass.cc:185] None of the MLIR Optimization Passes are enabled (registered 2)\n"
     ]
    },
    {
     "name": "stdout",
     "output_type": "stream",
     "text": [
      "Epoch 1/10\n",
      "1675/1675 [==============================] - 6s 3ms/step - loss: 0.8710 - accuracy: 0.6705 - val_loss: 0.6222 - val_accuracy: 0.7736\n",
      "Epoch 2/10\n",
      "1675/1675 [==============================] - 6s 4ms/step - loss: 0.5785 - accuracy: 0.7805 - val_loss: 0.5355 - val_accuracy: 0.7905\n",
      "Epoch 3/10\n",
      "1675/1675 [==============================] - 5s 3ms/step - loss: 0.5069 - accuracy: 0.8007 - val_loss: 0.4791 - val_accuracy: 0.8097\n",
      "Epoch 4/10\n",
      "1675/1675 [==============================] - 6s 3ms/step - loss: 0.4654 - accuracy: 0.8110 - val_loss: 0.4391 - val_accuracy: 0.8233\n",
      "Epoch 5/10\n",
      "1675/1675 [==============================] - 6s 3ms/step - loss: 0.4432 - accuracy: 0.8167 - val_loss: 0.4158 - val_accuracy: 0.8294\n",
      "Epoch 6/10\n",
      "1675/1675 [==============================] - 6s 3ms/step - loss: 0.4262 - accuracy: 0.8226 - val_loss: 0.4050 - val_accuracy: 0.8318\n",
      "Epoch 7/10\n",
      "1675/1675 [==============================] - 5s 3ms/step - loss: 0.4172 - accuracy: 0.8238 - val_loss: 0.4153 - val_accuracy: 0.8274\n",
      "Epoch 8/10\n",
      "1675/1675 [==============================] - 6s 4ms/step - loss: 0.4114 - accuracy: 0.8254 - val_loss: 0.4068 - val_accuracy: 0.8287\n",
      "Epoch 9/10\n",
      "1675/1675 [==============================] - 5s 3ms/step - loss: 0.4073 - accuracy: 0.8265 - val_loss: 0.4053 - val_accuracy: 0.8286\n",
      "Epoch 10/10\n",
      "1675/1675 [==============================] - 5s 3ms/step - loss: 0.4007 - accuracy: 0.8294 - val_loss: 0.3901 - val_accuracy: 0.8385\n"
     ]
    }
   ],
   "source": [
    "history = dnn_model.fit(X_train_t_pca, Y_train_t, batch_size=64, epochs=10, validation_data=(X_train_val_pca, Y_train_val))"
   ]
  },
  {
   "cell_type": "markdown",
   "id": "17f1e008",
   "metadata": {
    "papermill": {
     "duration": 0.457048,
     "end_time": "2022-03-16T15:22:02.821974",
     "exception": false,
     "start_time": "2022-03-16T15:22:02.364926",
     "status": "completed"
    },
    "tags": []
   },
   "source": [
    "## **Final Model Evaluation**"
   ]
  },
  {
   "cell_type": "markdown",
   "id": "4a8d1df5",
   "metadata": {
    "papermill": {
     "duration": 0.454225,
     "end_time": "2022-03-16T15:22:03.737191",
     "exception": false,
     "start_time": "2022-03-16T15:22:03.282966",
     "status": "completed"
    },
    "tags": []
   },
   "source": [
    "The previous parts were needed in order to determine the more appropriate model to solve our task.\n",
    "Now we will take the model with the hightest score in the cross validation (or on the validation set in the case of deep network) and we will test it on the test set.\n",
    "If the model is non-deep, it will be trained on the full training set before the testing."
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 49,
   "id": "dacdf4d0",
   "metadata": {
    "execution": {
     "iopub.execute_input": "2022-03-16T15:22:04.683185Z",
     "iopub.status.busy": "2022-03-16T15:22:04.682253Z",
     "iopub.status.idle": "2022-03-16T15:22:04.685805Z",
     "shell.execute_reply": "2022-03-16T15:22:04.686259Z",
     "shell.execute_reply.started": "2022-03-14T13:23:47.073808Z"
    },
    "papermill": {
     "duration": 0.467715,
     "end_time": "2022-03-16T15:22:04.686410",
     "exception": false,
     "start_time": "2022-03-16T15:22:04.218695",
     "status": "completed"
    },
    "tags": []
   },
   "outputs": [],
   "source": [
    "from sklearn.metrics import classification_report"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 50,
   "id": "56c10592",
   "metadata": {
    "execution": {
     "iopub.execute_input": "2022-03-16T15:22:05.602195Z",
     "iopub.status.busy": "2022-03-16T15:22:05.601193Z",
     "iopub.status.idle": "2022-03-16T15:22:05.604867Z",
     "shell.execute_reply": "2022-03-16T15:22:05.605320Z",
     "shell.execute_reply.started": "2022-03-14T13:23:47.075059Z"
    },
    "papermill": {
     "duration": 0.462262,
     "end_time": "2022-03-16T15:22:05.605488",
     "exception": false,
     "start_time": "2022-03-16T15:22:05.143226",
     "status": "completed"
    },
    "tags": []
   },
   "outputs": [],
   "source": [
    "# We will use the random forest model\n",
    "final_model = rnd_forest_model\n",
    "use_neural_network = False"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 51,
   "id": "bf1aaff0",
   "metadata": {
    "execution": {
     "iopub.execute_input": "2022-03-16T15:22:06.623992Z",
     "iopub.status.busy": "2022-03-16T15:22:06.623025Z",
     "iopub.status.idle": "2022-03-16T15:22:06.627033Z",
     "shell.execute_reply": "2022-03-16T15:22:06.627430Z",
     "shell.execute_reply.started": "2022-03-14T13:23:47.076736Z"
    },
    "papermill": {
     "duration": 0.562037,
     "end_time": "2022-03-16T15:22:06.627582",
     "exception": false,
     "start_time": "2022-03-16T15:22:06.065545",
     "status": "completed"
    },
    "tags": []
   },
   "outputs": [],
   "source": [
    "# If we are using the deep network model, it's necessary to manually do the PCA transformation:\n",
    "\n",
    "if use_neural_network:\n",
    "    # Preprocess test data\n",
    "    # Standardize\n",
    "    X_test = st_scaler_dnn.transform(X_test)\n",
    "    # PCA\n",
    "    X_test = pca_dnn.transform(X_test)"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 52,
   "id": "e4414c48",
   "metadata": {
    "execution": {
     "iopub.execute_input": "2022-03-16T15:22:07.558570Z",
     "iopub.status.busy": "2022-03-16T15:22:07.557925Z",
     "iopub.status.idle": "2022-03-16T15:23:35.192984Z",
     "shell.execute_reply": "2022-03-16T15:23:35.193524Z",
     "shell.execute_reply.started": "2022-03-14T13:23:47.078108Z"
    },
    "papermill": {
     "duration": 88.102938,
     "end_time": "2022-03-16T15:23:35.193787",
     "exception": false,
     "start_time": "2022-03-16T15:22:07.090849",
     "status": "completed"
    },
    "tags": []
   },
   "outputs": [],
   "source": [
    "# Now we can use the selected model to compute the predictions.\n",
    "\n",
    "if use_neural_network:\n",
    "    # Predict probabilty distributions over possible labels\n",
    "    Y_test_pred_p = final_model.predict(X_test)\n",
    "    # Store the index of the most probable label\n",
    "    Y_test_pred = np.argmax(Y_test_pred_p, axis=1)\n",
    "else:\n",
    "    # Train the full model on the complete training set\n",
    "    final_pipe = build_PCA_pipeline(final_model, pca_n_components)\n",
    "    final_pipe.fit(X_train, Y_train)\n",
    "    # Directly predict labels\n",
    "    Y_test_pred = final_pipe.predict(X_test)"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 53,
   "id": "edeff04a",
   "metadata": {
    "execution": {
     "iopub.execute_input": "2022-03-16T15:23:36.121691Z",
     "iopub.status.busy": "2022-03-16T15:23:36.121039Z",
     "iopub.status.idle": "2022-03-16T15:23:36.261627Z",
     "shell.execute_reply": "2022-03-16T15:23:36.260406Z",
     "shell.execute_reply.started": "2022-03-14T13:23:47.079471Z"
    },
    "papermill": {
     "duration": 0.608125,
     "end_time": "2022-03-16T15:23:36.261870",
     "exception": false,
     "start_time": "2022-03-16T15:23:35.653745",
     "status": "completed"
    },
    "tags": []
   },
   "outputs": [
    {
     "name": "stdout",
     "output_type": "stream",
     "text": [
      "                          precision    recall  f1-score   support\n",
      "\n",
      "    Bacteroides_fragilis       0.98      0.99      0.99      6646\n",
      "    Campylobacter_jejuni       0.98      0.99      0.99      6621\n",
      "      Enterococcus_hirae       0.97      0.97      0.97      6583\n",
      "        Escherichia_coli       0.97      0.96      0.96      6586\n",
      "  Escherichia_fergusonii       0.96      0.97      0.97      6579\n",
      "   Klebsiella_pneumoniae       0.99      0.99      0.99      6549\n",
      "     Salmonella_enterica       0.98      0.98      0.98      6610\n",
      "   Staphylococcus_aureus       0.98      0.98      0.98      6577\n",
      "Streptococcus_pneumoniae       0.98      0.97      0.97      6624\n",
      "  Streptococcus_pyogenes       0.97      0.97      0.97      6625\n",
      "\n",
      "                accuracy                           0.98     66000\n",
      "               macro avg       0.98      0.98      0.98     66000\n",
      "            weighted avg       0.98      0.98      0.98     66000\n",
      "\n"
     ]
    }
   ],
   "source": [
    "# Print classification report on the test set\n",
    "print(classification_report(Y_test, Y_test_pred, target_names=label_encoder.classes_))"
   ]
  },
  {
   "cell_type": "markdown",
   "id": "da744eb3",
   "metadata": {
    "papermill": {
     "duration": 0.468655,
     "end_time": "2022-03-16T15:23:37.195300",
     "exception": false,
     "start_time": "2022-03-16T15:23:36.726645",
     "status": "completed"
    },
    "tags": []
   },
   "source": [
    "## **Kaggle Submission Test**"
   ]
  },
  {
   "cell_type": "markdown",
   "id": "669779ba",
   "metadata": {
    "papermill": {
     "duration": 0.463295,
     "end_time": "2022-03-16T15:23:38.121584",
     "exception": false,
     "start_time": "2022-03-16T15:23:37.658289",
     "status": "completed"
    },
    "tags": []
   },
   "source": [
    "We will also use our model in the official Kaggle competition about this task.\n",
    "To do so, we first load the test dataset containing the data for which we will compute the predictions."
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 54,
   "id": "77b0e015",
   "metadata": {
    "execution": {
     "iopub.execute_input": "2022-03-16T15:23:39.053199Z",
     "iopub.status.busy": "2022-03-16T15:23:39.052527Z",
     "iopub.status.idle": "2022-03-16T15:23:53.496978Z",
     "shell.execute_reply": "2022-03-16T15:23:53.496423Z",
     "shell.execute_reply.started": "2022-03-14T13:23:47.080791Z"
    },
    "papermill": {
     "duration": 14.91368,
     "end_time": "2022-03-16T15:23:53.497126",
     "exception": false,
     "start_time": "2022-03-16T15:23:38.583446",
     "status": "completed"
    },
    "tags": []
   },
   "outputs": [
    {
     "data": {
      "text/html": [
       "<div>\n",
       "<style scoped>\n",
       "    .dataframe tbody tr th:only-of-type {\n",
       "        vertical-align: middle;\n",
       "    }\n",
       "\n",
       "    .dataframe tbody tr th {\n",
       "        vertical-align: top;\n",
       "    }\n",
       "\n",
       "    .dataframe thead th {\n",
       "        text-align: right;\n",
       "    }\n",
       "</style>\n",
       "<table border=\"1\" class=\"dataframe\">\n",
       "  <thead>\n",
       "    <tr style=\"text-align: right;\">\n",
       "      <th></th>\n",
       "      <th>row_id</th>\n",
       "      <th>A0T0G0C10</th>\n",
       "      <th>A0T0G1C9</th>\n",
       "      <th>A0T0G2C8</th>\n",
       "      <th>A0T0G3C7</th>\n",
       "      <th>A0T0G4C6</th>\n",
       "      <th>A0T0G5C5</th>\n",
       "      <th>A0T0G6C4</th>\n",
       "      <th>A0T0G7C3</th>\n",
       "      <th>A0T0G8C2</th>\n",
       "      <th>...</th>\n",
       "      <th>A8T0G0C2</th>\n",
       "      <th>A8T0G1C1</th>\n",
       "      <th>A8T0G2C0</th>\n",
       "      <th>A8T1G0C1</th>\n",
       "      <th>A8T1G1C0</th>\n",
       "      <th>A8T2G0C0</th>\n",
       "      <th>A9T0G0C1</th>\n",
       "      <th>A9T0G1C0</th>\n",
       "      <th>A9T1G0C0</th>\n",
       "      <th>A10T0G0C0</th>\n",
       "    </tr>\n",
       "  </thead>\n",
       "  <tbody>\n",
       "    <tr>\n",
       "      <th>0</th>\n",
       "      <td>200000</td>\n",
       "      <td>-9.536743e-07</td>\n",
       "      <td>-0.000002</td>\n",
       "      <td>-9.153442e-07</td>\n",
       "      <td>0.000024</td>\n",
       "      <td>0.000034</td>\n",
       "      <td>-0.000002</td>\n",
       "      <td>0.000021</td>\n",
       "      <td>0.000024</td>\n",
       "      <td>-0.000009</td>\n",
       "      <td>...</td>\n",
       "      <td>0.000039</td>\n",
       "      <td>0.000085</td>\n",
       "      <td>0.000055</td>\n",
       "      <td>0.000108</td>\n",
       "      <td>0.000090</td>\n",
       "      <td>0.000059</td>\n",
       "      <td>0.000010</td>\n",
       "      <td>0.000006</td>\n",
       "      <td>0.000027</td>\n",
       "      <td>4.632568e-08</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>1</th>\n",
       "      <td>200001</td>\n",
       "      <td>-9.536743e-07</td>\n",
       "      <td>-0.000010</td>\n",
       "      <td>-4.291534e-05</td>\n",
       "      <td>-0.000114</td>\n",
       "      <td>0.001800</td>\n",
       "      <td>-0.000240</td>\n",
       "      <td>0.001800</td>\n",
       "      <td>-0.000114</td>\n",
       "      <td>0.000957</td>\n",
       "      <td>...</td>\n",
       "      <td>-0.000043</td>\n",
       "      <td>0.000914</td>\n",
       "      <td>-0.000043</td>\n",
       "      <td>-0.000086</td>\n",
       "      <td>-0.000086</td>\n",
       "      <td>-0.000043</td>\n",
       "      <td>-0.000010</td>\n",
       "      <td>-0.000010</td>\n",
       "      <td>-0.000010</td>\n",
       "      <td>-9.536743e-07</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>2</th>\n",
       "      <td>200002</td>\n",
       "      <td>4.632568e-08</td>\n",
       "      <td>0.000003</td>\n",
       "      <td>8.465576e-08</td>\n",
       "      <td>-0.000014</td>\n",
       "      <td>0.000007</td>\n",
       "      <td>-0.000005</td>\n",
       "      <td>-0.000004</td>\n",
       "      <td>0.000003</td>\n",
       "      <td>0.000004</td>\n",
       "      <td>...</td>\n",
       "      <td>0.000041</td>\n",
       "      <td>0.000102</td>\n",
       "      <td>0.000084</td>\n",
       "      <td>0.000111</td>\n",
       "      <td>0.000117</td>\n",
       "      <td>0.000070</td>\n",
       "      <td>0.000020</td>\n",
       "      <td>0.000030</td>\n",
       "      <td>0.000021</td>\n",
       "      <td>4.632568e-08</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>3</th>\n",
       "      <td>200003</td>\n",
       "      <td>-9.536743e-07</td>\n",
       "      <td>-0.000008</td>\n",
       "      <td>8.084656e-06</td>\n",
       "      <td>0.000216</td>\n",
       "      <td>0.000420</td>\n",
       "      <td>0.000514</td>\n",
       "      <td>0.000452</td>\n",
       "      <td>0.000187</td>\n",
       "      <td>-0.000005</td>\n",
       "      <td>...</td>\n",
       "      <td>0.000069</td>\n",
       "      <td>0.000158</td>\n",
       "      <td>0.000098</td>\n",
       "      <td>0.000175</td>\n",
       "      <td>0.000217</td>\n",
       "      <td>0.000150</td>\n",
       "      <td>0.000018</td>\n",
       "      <td>0.000016</td>\n",
       "      <td>0.000051</td>\n",
       "      <td>-9.536743e-07</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>4</th>\n",
       "      <td>200004</td>\n",
       "      <td>-9.536743e-07</td>\n",
       "      <td>-0.000010</td>\n",
       "      <td>-4.291534e-05</td>\n",
       "      <td>-0.000114</td>\n",
       "      <td>-0.000200</td>\n",
       "      <td>-0.000240</td>\n",
       "      <td>-0.000200</td>\n",
       "      <td>-0.000114</td>\n",
       "      <td>-0.000043</td>\n",
       "      <td>...</td>\n",
       "      <td>-0.000043</td>\n",
       "      <td>-0.000086</td>\n",
       "      <td>-0.000043</td>\n",
       "      <td>-0.000086</td>\n",
       "      <td>0.000914</td>\n",
       "      <td>-0.000043</td>\n",
       "      <td>-0.000010</td>\n",
       "      <td>-0.000010</td>\n",
       "      <td>0.000990</td>\n",
       "      <td>-9.536743e-07</td>\n",
       "    </tr>\n",
       "  </tbody>\n",
       "</table>\n",
       "<p>5 rows × 287 columns</p>\n",
       "</div>"
      ],
      "text/plain": [
       "   row_id     A0T0G0C10  A0T0G1C9      A0T0G2C8  A0T0G3C7  A0T0G4C6  A0T0G5C5  \\\n",
       "0  200000 -9.536743e-07 -0.000002 -9.153442e-07  0.000024  0.000034 -0.000002   \n",
       "1  200001 -9.536743e-07 -0.000010 -4.291534e-05 -0.000114  0.001800 -0.000240   \n",
       "2  200002  4.632568e-08  0.000003  8.465576e-08 -0.000014  0.000007 -0.000005   \n",
       "3  200003 -9.536743e-07 -0.000008  8.084656e-06  0.000216  0.000420  0.000514   \n",
       "4  200004 -9.536743e-07 -0.000010 -4.291534e-05 -0.000114 -0.000200 -0.000240   \n",
       "\n",
       "   A0T0G6C4  A0T0G7C3  A0T0G8C2  ...  A8T0G0C2  A8T0G1C1  A8T0G2C0  A8T1G0C1  \\\n",
       "0  0.000021  0.000024 -0.000009  ...  0.000039  0.000085  0.000055  0.000108   \n",
       "1  0.001800 -0.000114  0.000957  ... -0.000043  0.000914 -0.000043 -0.000086   \n",
       "2 -0.000004  0.000003  0.000004  ...  0.000041  0.000102  0.000084  0.000111   \n",
       "3  0.000452  0.000187 -0.000005  ...  0.000069  0.000158  0.000098  0.000175   \n",
       "4 -0.000200 -0.000114 -0.000043  ... -0.000043 -0.000086 -0.000043 -0.000086   \n",
       "\n",
       "   A8T1G1C0  A8T2G0C0  A9T0G0C1  A9T0G1C0  A9T1G0C0     A10T0G0C0  \n",
       "0  0.000090  0.000059  0.000010  0.000006  0.000027  4.632568e-08  \n",
       "1 -0.000086 -0.000043 -0.000010 -0.000010 -0.000010 -9.536743e-07  \n",
       "2  0.000117  0.000070  0.000020  0.000030  0.000021  4.632568e-08  \n",
       "3  0.000217  0.000150  0.000018  0.000016  0.000051 -9.536743e-07  \n",
       "4  0.000914 -0.000043 -0.000010 -0.000010  0.000990 -9.536743e-07  \n",
       "\n",
       "[5 rows x 287 columns]"
      ]
     },
     "execution_count": 54,
     "metadata": {},
     "output_type": "execute_result"
    }
   ],
   "source": [
    "# Load Kaggle's submission test set\n",
    "df_subm = pd.read_csv(\"/kaggle/input/tabular-playground-series-feb-2022/test.csv\")\n",
    "df_subm.head()"
   ]
  },
  {
   "cell_type": "markdown",
   "id": "7a8c8f00",
   "metadata": {
    "papermill": {
     "duration": 0.493254,
     "end_time": "2022-03-16T15:23:54.451229",
     "exception": false,
     "start_time": "2022-03-16T15:23:53.957975",
     "status": "completed"
    },
    "tags": []
   },
   "source": [
    "Now we create the pandas dataframe in which we will store our predictions.\n",
    "\n",
    "Each row will contain the row_id of the sample and its predicted label."
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 55,
   "id": "58d9accc",
   "metadata": {
    "execution": {
     "iopub.execute_input": "2022-03-16T15:23:55.380785Z",
     "iopub.status.busy": "2022-03-16T15:23:55.379851Z",
     "iopub.status.idle": "2022-03-16T15:23:55.387740Z",
     "shell.execute_reply": "2022-03-16T15:23:55.387227Z",
     "shell.execute_reply.started": "2022-03-14T13:23:47.082317Z"
    },
    "papermill": {
     "duration": 0.478918,
     "end_time": "2022-03-16T15:23:55.387883",
     "exception": false,
     "start_time": "2022-03-16T15:23:54.908965",
     "status": "completed"
    },
    "tags": []
   },
   "outputs": [
    {
     "data": {
      "text/html": [
       "<div>\n",
       "<style scoped>\n",
       "    .dataframe tbody tr th:only-of-type {\n",
       "        vertical-align: middle;\n",
       "    }\n",
       "\n",
       "    .dataframe tbody tr th {\n",
       "        vertical-align: top;\n",
       "    }\n",
       "\n",
       "    .dataframe thead th {\n",
       "        text-align: right;\n",
       "    }\n",
       "</style>\n",
       "<table border=\"1\" class=\"dataframe\">\n",
       "  <thead>\n",
       "    <tr style=\"text-align: right;\">\n",
       "      <th></th>\n",
       "      <th>row_id</th>\n",
       "    </tr>\n",
       "  </thead>\n",
       "  <tbody>\n",
       "    <tr>\n",
       "      <th>0</th>\n",
       "      <td>200000</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>1</th>\n",
       "      <td>200001</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>2</th>\n",
       "      <td>200002</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>3</th>\n",
       "      <td>200003</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>4</th>\n",
       "      <td>200004</td>\n",
       "    </tr>\n",
       "  </tbody>\n",
       "</table>\n",
       "</div>"
      ],
      "text/plain": [
       "   row_id\n",
       "0  200000\n",
       "1  200001\n",
       "2  200002\n",
       "3  200003\n",
       "4  200004"
      ]
     },
     "execution_count": 55,
     "metadata": {},
     "output_type": "execute_result"
    }
   ],
   "source": [
    "df_subm_results = pd.DataFrame(df_subm[\"row_id\"])\n",
    "df_subm_results.head()"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 56,
   "id": "d490310f",
   "metadata": {
    "execution": {
     "iopub.execute_input": "2022-03-16T15:23:56.405300Z",
     "iopub.status.busy": "2022-03-16T15:23:56.404550Z",
     "iopub.status.idle": "2022-03-16T15:23:56.408129Z",
     "shell.execute_reply": "2022-03-16T15:23:56.407565Z",
     "shell.execute_reply.started": "2022-03-14T13:23:47.083671Z"
    },
    "papermill": {
     "duration": 0.557991,
     "end_time": "2022-03-16T15:23:56.408287",
     "exception": false,
     "start_time": "2022-03-16T15:23:55.850296",
     "status": "completed"
    },
    "tags": []
   },
   "outputs": [],
   "source": [
    "X_subm = df_subm.drop(columns=\"row_id\")"
   ]
  },
  {
   "cell_type": "markdown",
   "id": "c1156f6a",
   "metadata": {
    "papermill": {
     "duration": 0.463595,
     "end_time": "2022-03-16T15:23:57.321594",
     "exception": false,
     "start_time": "2022-03-16T15:23:56.857999",
     "status": "completed"
    },
    "tags": []
   },
   "source": [
    "This time in the code i will directly assume to use a non-deep model, because we have seen that they perform better on the task ."
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 57,
   "id": "339bfd15",
   "metadata": {
    "execution": {
     "iopub.execute_input": "2022-03-16T15:23:58.425181Z",
     "iopub.status.busy": "2022-03-16T15:23:58.424530Z",
     "iopub.status.idle": "2022-03-16T15:26:01.344128Z",
     "shell.execute_reply": "2022-03-16T15:26:01.344643Z",
     "shell.execute_reply.started": "2022-03-14T13:23:47.085334Z"
    },
    "papermill": {
     "duration": 123.567022,
     "end_time": "2022-03-16T15:26:01.344846",
     "exception": false,
     "start_time": "2022-03-16T15:23:57.777824",
     "status": "completed"
    },
    "tags": []
   },
   "outputs": [
    {
     "data": {
      "text/plain": [
       "Pipeline(steps=[('st_scaler', StandardScaler()),\n",
       "                ('pca', PCA(n_components=5, random_state=42)),\n",
       "                ('classifier',\n",
       "                 RandomForestClassifier(criterion='entropy', n_estimators=150,\n",
       "                                        random_state=42))])"
      ]
     },
     "execution_count": 57,
     "metadata": {},
     "output_type": "execute_result"
    }
   ],
   "source": [
    "# Build pipeline with final model\n",
    "final_pipe = build_PCA_pipeline(final_model, pca_n_components)\n",
    "# Retrain model on full dataset\n",
    "final_pipe.fit(df.drop(columns=[\"target\"]), df[\"target\"])"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 58,
   "id": "0d414d69",
   "metadata": {
    "execution": {
     "iopub.execute_input": "2022-03-16T15:26:02.273794Z",
     "iopub.status.busy": "2022-03-16T15:26:02.272528Z",
     "iopub.status.idle": "2022-03-16T15:26:05.960844Z",
     "shell.execute_reply": "2022-03-16T15:26:05.960239Z",
     "shell.execute_reply.started": "2022-03-14T13:23:47.086757Z"
    },
    "papermill": {
     "duration": 4.157498,
     "end_time": "2022-03-16T15:26:05.960991",
     "exception": false,
     "start_time": "2022-03-16T15:26:01.803493",
     "status": "completed"
    },
    "tags": []
   },
   "outputs": [],
   "source": [
    "# predict labels\n",
    "Y_subm_predicted = final_pipe.predict(X_subm)\n",
    "# Convert label indexes to text\n",
    "Y_subm_predicted_labeled = label_encoder.inverse_transform(Y_subm_predicted)\n",
    "# Store the predicted targets in the dataframe\n",
    "df_subm_results[\"target\"] = Y_subm_predicted_labeled"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 59,
   "id": "596e93d6",
   "metadata": {
    "execution": {
     "iopub.execute_input": "2022-03-16T15:26:06.906897Z",
     "iopub.status.busy": "2022-03-16T15:26:06.906090Z",
     "iopub.status.idle": "2022-03-16T15:26:06.909429Z",
     "shell.execute_reply": "2022-03-16T15:26:06.909918Z",
     "shell.execute_reply.started": "2022-03-14T13:23:47.088771Z"
    },
    "papermill": {
     "duration": 0.471357,
     "end_time": "2022-03-16T15:26:06.910073",
     "exception": false,
     "start_time": "2022-03-16T15:26:06.438716",
     "status": "completed"
    },
    "tags": []
   },
   "outputs": [
    {
     "data": {
      "text/html": [
       "<div>\n",
       "<style scoped>\n",
       "    .dataframe tbody tr th:only-of-type {\n",
       "        vertical-align: middle;\n",
       "    }\n",
       "\n",
       "    .dataframe tbody tr th {\n",
       "        vertical-align: top;\n",
       "    }\n",
       "\n",
       "    .dataframe thead th {\n",
       "        text-align: right;\n",
       "    }\n",
       "</style>\n",
       "<table border=\"1\" class=\"dataframe\">\n",
       "  <thead>\n",
       "    <tr style=\"text-align: right;\">\n",
       "      <th></th>\n",
       "      <th>row_id</th>\n",
       "      <th>target</th>\n",
       "    </tr>\n",
       "  </thead>\n",
       "  <tbody>\n",
       "    <tr>\n",
       "      <th>0</th>\n",
       "      <td>200000</td>\n",
       "      <td>Escherichia_fergusonii</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>1</th>\n",
       "      <td>200001</td>\n",
       "      <td>Salmonella_enterica</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>2</th>\n",
       "      <td>200002</td>\n",
       "      <td>Enterococcus_hirae</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>3</th>\n",
       "      <td>200003</td>\n",
       "      <td>Salmonella_enterica</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>4</th>\n",
       "      <td>200004</td>\n",
       "      <td>Staphylococcus_aureus</td>\n",
       "    </tr>\n",
       "  </tbody>\n",
       "</table>\n",
       "</div>"
      ],
      "text/plain": [
       "   row_id                  target\n",
       "0  200000  Escherichia_fergusonii\n",
       "1  200001     Salmonella_enterica\n",
       "2  200002      Enterococcus_hirae\n",
       "3  200003     Salmonella_enterica\n",
       "4  200004   Staphylococcus_aureus"
      ]
     },
     "execution_count": 59,
     "metadata": {},
     "output_type": "execute_result"
    }
   ],
   "source": [
    "df_subm_results.head()"
   ]
  },
  {
   "cell_type": "markdown",
   "id": "eff22127",
   "metadata": {
    "papermill": {
     "duration": 0.459681,
     "end_time": "2022-03-16T15:26:07.831401",
     "exception": false,
     "start_time": "2022-03-16T15:26:07.371720",
     "status": "completed"
    },
    "tags": []
   },
   "source": [
    "Finally we save the submission dataset as a csv file, which can be loaded on the Kaggle competition's website to obtain a score:"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 60,
   "id": "72718c10",
   "metadata": {
    "execution": {
     "iopub.execute_input": "2022-03-16T15:26:08.818190Z",
     "iopub.status.busy": "2022-03-16T15:26:08.817181Z",
     "iopub.status.idle": "2022-03-16T15:26:09.070982Z",
     "shell.execute_reply": "2022-03-16T15:26:09.071528Z",
     "shell.execute_reply.started": "2022-03-14T13:23:47.090388Z"
    },
    "papermill": {
     "duration": 0.762539,
     "end_time": "2022-03-16T15:26:09.071696",
     "exception": false,
     "start_time": "2022-03-16T15:26:08.309157",
     "status": "completed"
    },
    "tags": []
   },
   "outputs": [],
   "source": [
    "df_subm_results.to_csv(\"submission.csv\", index=False)"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": null,
   "id": "af8d2d38",
   "metadata": {
    "papermill": {
     "duration": 0.461124,
     "end_time": "2022-03-16T15:26:09.992225",
     "exception": false,
     "start_time": "2022-03-16T15:26:09.531101",
     "status": "completed"
    },
    "tags": []
   },
   "outputs": [],
   "source": []
  },
  {
   "cell_type": "code",
   "execution_count": null,
   "id": "8c66ed05",
   "metadata": {
    "papermill": {
     "duration": 0.457841,
     "end_time": "2022-03-16T15:26:10.912776",
     "exception": false,
     "start_time": "2022-03-16T15:26:10.454935",
     "status": "completed"
    },
    "tags": []
   },
   "outputs": [],
   "source": []
  }
 ],
 "metadata": {
  "kernelspec": {
   "display_name": "Python 3",
   "language": "python",
   "name": "python3"
  },
  "language_info": {
   "codemirror_mode": {
    "name": "ipython",
    "version": 3
   },
   "file_extension": ".py",
   "mimetype": "text/x-python",
   "name": "python",
   "nbconvert_exporter": "python",
   "pygments_lexer": "ipython3",
   "version": "3.7.12"
  },
  "papermill": {
   "default_parameters": {},
   "duration": 941.523047,
   "end_time": "2022-03-16T15:26:14.181302",
   "environment_variables": {},
   "exception": null,
   "input_path": "__notebook__.ipynb",
   "output_path": "__notebook__.ipynb",
   "parameters": {},
   "start_time": "2022-03-16T15:10:32.658255",
   "version": "2.3.3"
  }
 },
 "nbformat": 4,
 "nbformat_minor": 5
}