Adaboost.ipynb

{
 "cells": [
  {
   "cell_type": "markdown",
   "source": [
    "# Trabalho Prático 2 - Boosting\n",
    "\n",
    "### Cinthia M. Souza"
   ],
   "metadata": {}
  },
  {
   "cell_type": "markdown",
   "source": [
    "Neste trabalho é apresentado uma implementação do algoritmo Adaboost. O algoritmo implementado é baseado no artigo intitulado \"A Short Introduction to Boosting\", dos autores Yoav Freund e Robert E. Schapire.\n",
    "\n",
    "Nós vamos trabalhar em um problema de classificação considerando que temos:\n",
    "\n",
    "$(x_{1}, y_{1}), ..., (x_m, y_m)$, onde $x_i \\in X, y_{i} \\in Y = \\{-1, 1\\}$\n",
    "\n",
    "O Algoritmo segue os seguintes passos:\n",
    "\n",
    "- Inicializa os pesos de $D_{1}(i) = \\frac{1}{m}$\n",
    "- Para t = 1, ..., T  faça:\n",
    "\n",
    "    - Treina um classificador fraco utilizando a distribuição de pesos $D_{t}$;\n",
    "    - Dada a hipótese $h_t : X \\rightarrow \\{-1, 1\\}$ com erro:\n",
    "\n",
    "\n",
    "        $error = \\sum_{i:h_{t}(z_{i}) != y_{i}} D_{t}(i)$.\n",
    "\n",
    "    - Defina o valor de $\\alpha_{t}$\n",
    "\n",
    "    $\\alpha_t = \\frac{1}{2} ln(\\frac{1- error}{error})$.\n",
    "\n",
    "    - Realiza a atualização dos pesos:\n",
    "\n",
    "    $ D_{t+1}(i) = \\frac{D_{t}(i) exp(-\\alpha_{t}y_{i}h_{t}(x_{i}))}{Z_t}$,\n",
    "\n",
    "onde $Z_{t} é o fator de normalização.\n",
    "\n",
    "- Cálculo da hipótese final:\n",
    "    \n",
    "$H(X) = sign( \\sum_{t=1}^{T} \\alpha_{t} h_{t}(x))$.\n",
    "\n",
    "Basicamente, o erro é a soma dos pesos dos exemplos erroneamente classificados. O valor de $\\alpha_{t}$ mede a importância que é atribuída a $h_{t}$. A regra de atualização dos pesos tem como objetivo aumentar o peso de exemplos classificados errôneamente e diminuir o peso do exemplos que foram corretamente classificados. Sendo assim, podemos dizer que o peso concentra-se nos exemplos que foram errôneamente classificados. Por fim, a hipótese final H é uma votação entre os classificadores fracos.\n",
    "\n"
   ],
   "metadata": {}
  },
  {
   "cell_type": "markdown",
   "source": [
    "# Libraries"
   ],
   "metadata": {}
  },
  {
   "cell_type": "code",
   "execution_count": 47,
   "source": [
    "import pandas as pd\n",
    "from sklearn.tree import DecisionTreeClassifier\n",
    "from sklearn.preprocessing import label_binarize\n",
    "\n",
    "from sklearn.model_selection import train_test_split\n",
    "from sklearn.model_selection import KFold\n",
    "\n",
    "from sklearn.metrics import accuracy_score\n",
    "from sklearn.metrics import f1_score\n",
    "from sklearn.metrics import roc_auc_score\n",
    "\n",
    "import plotly.graph_objects as go\n",
    "from matplotlib import pyplot as plt\n",
    "import numpy as np\n",
    "\n",
    "import plotly.express as px\n",
    "\n",
    "import warnings\n",
    "warnings.filterwarnings(\"ignore\")"
   ],
   "outputs": [],
   "metadata": {}
  },
  {
   "cell_type": "markdown",
   "source": [
    "# Implementação AdaBoost"
   ],
   "metadata": {}
  },
  {
   "cell_type": "code",
   "execution_count": 48,
   "source": [
    "def adaboost(X, y, n_estimators=10):\n",
    "\n",
    "    stumps = []\n",
    "    alphas = []\n",
    "    errors = []\n",
    "    ada = []\n",
    "\n",
    "    shape = X.shape[0]\n",
    "    weights = np.zeros(shape=(n_estimators, shape))\n",
    "\n",
    "    #Inicialização dos pesos como sendo 1/m\n",
    "    weights[0] = [1/shape] *shape\n",
    "    \n",
    "    for i in range(n_estimators):\n",
    "\n",
    "        # stump\n",
    "        stump = DecisionTreeClassifier(max_depth=1)\n",
    "        stump.fit(X, y, sample_weight=weights[i])\n",
    "        pred = stump.predict(X)\n",
    "\n",
    "        #Calculo do erro\n",
    "        error = sum(weights[i][(pred != y)])\n",
    "\n",
    "        #Calculando alpha t\n",
    "        alpha = 1/2 * np.log((1 - error) / error) \n",
    "\n",
    "        # Atualização dos pesos\n",
    "        new_weights = (weights[i] * np.exp(-alpha * y * pred))\n",
    "        new_weights = new_weights/new_weights.sum()\n",
    "\n",
    "        if i+1 < n_estimators:\n",
    "            weights[i+1] = new_weights\n",
    "\n",
    "        stumps.append(stump)\n",
    "        errors.append(error)\n",
    "        alphas.append(alpha)\n",
    "        ada.append(np.prod(((errors[i]*(1-errors[i]))**1/2)))\n",
    "\n",
    "    return ada, errors, stumps, alphas"
   ],
   "outputs": [],
   "metadata": {}
  },
  {
   "cell_type": "code",
   "execution_count": 49,
   "source": [
    "def predict(X, alphas, stumps):\n",
    "    return np.sign(np.dot(alphas, np.array([stump.predict(X) for stump in stumps])))"
   ],
   "outputs": [],
   "metadata": {}
  },
  {
   "cell_type": "markdown",
   "source": [
    "# Estimativa do erro esperado usando K-Fold Cross Validation \n",
    "\n",
    "Para obter uma estimativa do erro esperado, foi utilizada a técnica cross-validation, com k = 5. "
   ],
   "metadata": {}
  },
  {
   "cell_type": "code",
   "execution_count": 50,
   "source": [
    "def cross_validation (X, y, n_estimators=100, nfolds=5):\n",
    "\n",
    "    acc = []\n",
    "    roc = []\n",
    "    f1 = []\n",
    "\n",
    "    kfold = KFold(nfolds, True, 1)\n",
    "\n",
    "    X = np.asarray(X)\n",
    "\n",
    "    for train_index, test_index in kfold.split(X):\n",
    "\n",
    "        X_train, X_test = X[train_index], X[test_index]\n",
    "        y_train, y_test = y[train_index], y[test_index]\n",
    "\n",
    "        ada, errors, stumps, alphas = adaboost(X_train, y_train, n_estimators=n_estimators)\n",
    "\n",
    "        y_pred = predict(X_test, alphas, stumps)\n",
    "\n",
    "        acc.append(accuracy_score(y_test, y_pred))\n",
    "        roc.append(roc_auc_score(y_test, y_pred))\n",
    "        f1.append(f1_score(y_test, y_pred))\n",
    "\n",
    "\n",
    "    return acc, roc, f1"
   ],
   "outputs": [],
   "metadata": {}
  },
  {
   "cell_type": "markdown",
   "source": [
    "# Carregamento e preparação da base\n",
    "\n",
    "Foram necessárias realizar duas transformações no conjunto de dados, a primeira foi a de one-hot encoding para binarizar os atributos da base e a segunda foi uma transformação no atributo label. Nessa transformação, os rótulos \"positive\" receberam o valor \"1\" e os rótulos \"negative\" receberam o valor -1."
   ],
   "metadata": {}
  },
  {
   "cell_type": "code",
   "execution_count": 51,
   "source": [
    "df = pd.read_csv('tic-tac-toe-endgame.csv')\n",
    "dummies = pd.get_dummies(df[df.columns[:len(df.columns)-1]],drop_first = True)\n",
    "dummies"
   ],
   "outputs": [
    {
     "output_type": "execute_result",
     "data": {
      "text/plain": [
       "     V1_o  V1_x  V2_o  V2_x  V3_o  V3_x  V4_o  V4_x  V5_o  V5_x  V6_o  V6_x  \\\n",
       "0       0     1     0     1     0     1     0     1     1     0     1     0   \n",
       "1       0     1     0     1     0     1     0     1     1     0     1     0   \n",
       "2       0     1     0     1     0     1     0     1     1     0     1     0   \n",
       "3       0     1     0     1     0     1     0     1     1     0     1     0   \n",
       "4       0     1     0     1     0     1     0     1     1     0     1     0   \n",
       "..    ...   ...   ...   ...   ...   ...   ...   ...   ...   ...   ...   ...   \n",
       "953     1     0     0     1     0     1     0     1     1     0     1     0   \n",
       "954     1     0     0     1     1     0     0     1     0     1     1     0   \n",
       "955     1     0     0     1     1     0     0     1     1     0     0     1   \n",
       "956     1     0     0     1     1     0     1     0     0     1     0     1   \n",
       "957     1     0     1     0     0     1     0     1     0     1     1     0   \n",
       "\n",
       "     V7_o  V7_x  V8_o  V8_x  V9_o  V9_x  \n",
       "0       0     1     1     0     1     0  \n",
       "1       1     0     0     1     1     0  \n",
       "2       1     0     1     0     0     1  \n",
       "3       1     0     0     0     0     0  \n",
       "4       0     0     1     0     0     0  \n",
       "..    ...   ...   ...   ...   ...   ...  \n",
       "953     1     0     0     1     0     1  \n",
       "954     0     1     1     0     0     1  \n",
       "955     0     1     1     0     0     1  \n",
       "956     0     1     1     0     0     1  \n",
       "957     1     0     0     1     0     1  \n",
       "\n",
       "[958 rows x 18 columns]"
      ],
      "text/html": [
       "<div>\n",
       "<style scoped>\n",
       "    .dataframe tbody tr th:only-of-type {\n",
       "        vertical-align: middle;\n",
       "    }\n",
       "\n",
       "    .dataframe tbody tr th {\n",
       "        vertical-align: top;\n",
       "    }\n",
       "\n",
       "    .dataframe thead th {\n",
       "        text-align: right;\n",
       "    }\n",
       "</style>\n",
       "<table border=\"1\" class=\"dataframe\">\n",
       "  <thead>\n",
       "    <tr style=\"text-align: right;\">\n",
       "      <th></th>\n",
       "      <th>V1_o</th>\n",
       "      <th>V1_x</th>\n",
       "      <th>V2_o</th>\n",
       "      <th>V2_x</th>\n",
       "      <th>V3_o</th>\n",
       "      <th>V3_x</th>\n",
       "      <th>V4_o</th>\n",
       "      <th>V4_x</th>\n",
       "      <th>V5_o</th>\n",
       "      <th>V5_x</th>\n",
       "      <th>V6_o</th>\n",
       "      <th>V6_x</th>\n",
       "      <th>V7_o</th>\n",
       "      <th>V7_x</th>\n",
       "      <th>V8_o</th>\n",
       "      <th>V8_x</th>\n",
       "      <th>V9_o</th>\n",
       "      <th>V9_x</th>\n",
       "    </tr>\n",
       "  </thead>\n",
       "  <tbody>\n",
       "    <tr>\n",
       "      <th>0</th>\n",
       "      <td>0</td>\n",
       "      <td>1</td>\n",
       "      <td>0</td>\n",
       "      <td>1</td>\n",
       "      <td>0</td>\n",
       "      <td>1</td>\n",
       "      <td>0</td>\n",
       "      <td>1</td>\n",
       "      <td>1</td>\n",
       "      <td>0</td>\n",
       "      <td>1</td>\n",
       "      <td>0</td>\n",
       "      <td>0</td>\n",
       "      <td>1</td>\n",
       "      <td>1</td>\n",
       "      <td>0</td>\n",
       "      <td>1</td>\n",
       "      <td>0</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>1</th>\n",
       "      <td>0</td>\n",
       "      <td>1</td>\n",
       "      <td>0</td>\n",
       "      <td>1</td>\n",
       "      <td>0</td>\n",
       "      <td>1</td>\n",
       "      <td>0</td>\n",
       "      <td>1</td>\n",
       "      <td>1</td>\n",
       "      <td>0</td>\n",
       "      <td>1</td>\n",
       "      <td>0</td>\n",
       "      <td>1</td>\n",
       "      <td>0</td>\n",
       "      <td>0</td>\n",
       "      <td>1</td>\n",
       "      <td>1</td>\n",
       "      <td>0</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>2</th>\n",
       "      <td>0</td>\n",
       "      <td>1</td>\n",
       "      <td>0</td>\n",
       "      <td>1</td>\n",
       "      <td>0</td>\n",
       "      <td>1</td>\n",
       "      <td>0</td>\n",
       "      <td>1</td>\n",
       "      <td>1</td>\n",
       "      <td>0</td>\n",
       "      <td>1</td>\n",
       "      <td>0</td>\n",
       "      <td>1</td>\n",
       "      <td>0</td>\n",
       "      <td>1</td>\n",
       "      <td>0</td>\n",
       "      <td>0</td>\n",
       "      <td>1</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>3</th>\n",
       "      <td>0</td>\n",
       "      <td>1</td>\n",
       "      <td>0</td>\n",
       "      <td>1</td>\n",
       "      <td>0</td>\n",
       "      <td>1</td>\n",
       "      <td>0</td>\n",
       "      <td>1</td>\n",
       "      <td>1</td>\n",
       "      <td>0</td>\n",
       "      <td>1</td>\n",
       "      <td>0</td>\n",
       "      <td>1</td>\n",
       "      <td>0</td>\n",
       "      <td>0</td>\n",
       "      <td>0</td>\n",
       "      <td>0</td>\n",
       "      <td>0</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>4</th>\n",
       "      <td>0</td>\n",
       "      <td>1</td>\n",
       "      <td>0</td>\n",
       "      <td>1</td>\n",
       "      <td>0</td>\n",
       "      <td>1</td>\n",
       "      <td>0</td>\n",
       "      <td>1</td>\n",
       "      <td>1</td>\n",
       "      <td>0</td>\n",
       "      <td>1</td>\n",
       "      <td>0</td>\n",
       "      <td>0</td>\n",
       "      <td>0</td>\n",
       "      <td>1</td>\n",
       "      <td>0</td>\n",
       "      <td>0</td>\n",
       "      <td>0</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>...</th>\n",
       "      <td>...</td>\n",
       "      <td>...</td>\n",
       "      <td>...</td>\n",
       "      <td>...</td>\n",
       "      <td>...</td>\n",
       "      <td>...</td>\n",
       "      <td>...</td>\n",
       "      <td>...</td>\n",
       "      <td>...</td>\n",
       "      <td>...</td>\n",
       "      <td>...</td>\n",
       "      <td>...</td>\n",
       "      <td>...</td>\n",
       "      <td>...</td>\n",
       "      <td>...</td>\n",
       "      <td>...</td>\n",
       "      <td>...</td>\n",
       "      <td>...</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>953</th>\n",
       "      <td>1</td>\n",
       "      <td>0</td>\n",
       "      <td>0</td>\n",
       "      <td>1</td>\n",
       "      <td>0</td>\n",
       "      <td>1</td>\n",
       "      <td>0</td>\n",
       "      <td>1</td>\n",
       "      <td>1</td>\n",
       "      <td>0</td>\n",
       "      <td>1</td>\n",
       "      <td>0</td>\n",
       "      <td>1</td>\n",
       "      <td>0</td>\n",
       "      <td>0</td>\n",
       "      <td>1</td>\n",
       "      <td>0</td>\n",
       "      <td>1</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>954</th>\n",
       "      <td>1</td>\n",
       "      <td>0</td>\n",
       "      <td>0</td>\n",
       "      <td>1</td>\n",
       "      <td>1</td>\n",
       "      <td>0</td>\n",
       "      <td>0</td>\n",
       "      <td>1</td>\n",
       "      <td>0</td>\n",
       "      <td>1</td>\n",
       "      <td>1</td>\n",
       "      <td>0</td>\n",
       "      <td>0</td>\n",
       "      <td>1</td>\n",
       "      <td>1</td>\n",
       "      <td>0</td>\n",
       "      <td>0</td>\n",
       "      <td>1</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>955</th>\n",
       "      <td>1</td>\n",
       "      <td>0</td>\n",
       "      <td>0</td>\n",
       "      <td>1</td>\n",
       "      <td>1</td>\n",
       "      <td>0</td>\n",
       "      <td>0</td>\n",
       "      <td>1</td>\n",
       "      <td>1</td>\n",
       "      <td>0</td>\n",
       "      <td>0</td>\n",
       "      <td>1</td>\n",
       "      <td>0</td>\n",
       "      <td>1</td>\n",
       "      <td>1</td>\n",
       "      <td>0</td>\n",
       "      <td>0</td>\n",
       "      <td>1</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>956</th>\n",
       "      <td>1</td>\n",
       "      <td>0</td>\n",
       "      <td>0</td>\n",
       "      <td>1</td>\n",
       "      <td>1</td>\n",
       "      <td>0</td>\n",
       "      <td>1</td>\n",
       "      <td>0</td>\n",
       "      <td>0</td>\n",
       "      <td>1</td>\n",
       "      <td>0</td>\n",
       "      <td>1</td>\n",
       "      <td>0</td>\n",
       "      <td>1</td>\n",
       "      <td>1</td>\n",
       "      <td>0</td>\n",
       "      <td>0</td>\n",
       "      <td>1</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>957</th>\n",
       "      <td>1</td>\n",
       "      <td>0</td>\n",
       "      <td>1</td>\n",
       "      <td>0</td>\n",
       "      <td>0</td>\n",
       "      <td>1</td>\n",
       "      <td>0</td>\n",
       "      <td>1</td>\n",
       "      <td>0</td>\n",
       "      <td>1</td>\n",
       "      <td>1</td>\n",
       "      <td>0</td>\n",
       "      <td>1</td>\n",
       "      <td>0</td>\n",
       "      <td>0</td>\n",
       "      <td>1</td>\n",
       "      <td>0</td>\n",
       "      <td>1</td>\n",
       "    </tr>\n",
       "  </tbody>\n",
       "</table>\n",
       "<p>958 rows × 18 columns</p>\n",
       "</div>"
      ]
     },
     "metadata": {},
     "execution_count": 51
    }
   ],
   "metadata": {}
  },
  {
   "cell_type": "code",
   "execution_count": 52,
   "source": [
    "y = label_binarize(df['V10'], classes=['negative', 'positive'], neg_label=-1, pos_label=1)\n",
    "y = y.reshape(1,-1)[0]\n",
    "X = dummies"
   ],
   "outputs": [],
   "metadata": {}
  },
  {
   "cell_type": "markdown",
   "source": [
    "## Cross-validation\n",
    "\n",
    "Posteriormente, foi realizada a estimativa do erro esperado utilizando a técnica k-fold crossvalidation com k = 5. Além disso, foram avaliados 6 diferentes valores para o número de estimators. Sendo esses: 10, 20, 30, 100, 200 e 500.\n",
    "\n",
    "Para avaliar os resultados, foi utilizada as métricas:\n",
    "\n",
    "- Acurácia: Avalia a quantidade de acertos do algoritmo em relaçao ao total de registros da base;\n",
    "- F1-score: Média harmônica entre as métricas precision e recall;\n",
    "- AUC: Representa a área abaixo da curva ROC. O valore de AUC varia entre 0 e 1, onde 0 significa que 100% das predições estão corretas e 1 que 100% das predições estão corretas."
   ],
   "metadata": {}
  },
  {
   "cell_type": "code",
   "execution_count": 53,
   "source": [
    "metrics_mean ={'acc': [], 'roc': [], 'f1': []}\n",
    "metrics_std ={'acc': [], 'roc': [], 'f1': []}\n",
    "\n",
    "n_estimators = [10, 20, 50, 100, 200, 500]\n",
    "\n",
    "for i in n_estimators:\n",
    "\n",
    "    \n",
    "    acc, roc, f1 = cross_validation (X, y, n_estimators=i, nfolds=5)\n",
    "\n",
    "    metrics_mean['acc'].append(np.mean(acc))\n",
    "    metrics_mean['roc'].append(np.mean(roc))\n",
    "    metrics_mean['f1'].append(np.mean(f1))\n",
    "\n",
    "    metrics_std['acc'].append(np.std(acc))\n",
    "    metrics_std['roc'].append(np.std(roc))\n",
    "    metrics_std['f1'].append(np.std(f1))\n"
   ],
   "outputs": [],
   "metadata": {}
  },
  {
   "cell_type": "markdown",
   "source": [
    "A fim de analisar os resultados obtidos, foi representado gráficamente o valor da média e o desvio padrão das métricas nas 5 dobras para cada um dos valores de estimators. Os gráficos a seguir apresentam os resultados obtidos."
   ],
   "metadata": {}
  },
  {
   "cell_type": "code",
   "execution_count": 54,
   "source": [
    "fig = go.Figure()\n",
    "fig.add_trace(go.Scatter(x=n_estimators, y=metrics_mean['acc'],\n",
    "                    mode='lines+markers',\n",
    "                    name='accuracy'))\n",
    "fig.add_trace(go.Scatter(x=n_estimators, y=metrics_mean['roc'],\n",
    "                    mode='lines+markers',\n",
    "                    name='auc score'))\n",
    "fig.add_trace(go.Scatter(x=n_estimators, y=metrics_mean['f1'],\n",
    "                    mode='lines+markers', name='f1-scores'))\n",
    "fig.update_layout(title=\"Variação do valor médio das métricas nas 5 fold em relação ao número de iterações\")\n",
    "fig.show()"
   ],
   "outputs": [
    {
     "output_type": "display_data",
     "data": {
      "application/vnd.plotly.v1+json": {
       "config": {
        "plotlyServerURL": "https://plot.ly"
       },
       "data": [
        {
         "mode": "lines+markers",
         "name": "accuracy",
         "type": "scatter",
         "x": [
          10,
          20,
          50,
          100,
          200,
          500
         ],
         "y": [
          0.7222731239092496,
          0.7452334205933682,
          0.8130671902268762,
          0.8830333769633508,
          0.9717877399650959,
          0.9801537958115183
         ]
        },
        {
         "mode": "lines+markers",
         "name": "auc score",
         "type": "scatter",
         "x": [
          10,
          20,
          50,
          100,
          200,
          500
         ],
         "y": [
          0.6539840310488385,
          0.67905607827569,
          0.7501287564054221,
          0.8412601079299546,
          0.963381534012046,
          0.9738354367269781
         ]
        },
        {
         "mode": "lines+markers",
         "name": "f1-scores",
         "type": "scatter",
         "x": [
          10,
          20,
          50,
          100,
          200,
          500
         ],
         "y": [
          0.8044185293816948,
          0.821221787295133,
          0.8697435357653021,
          0.9163708608466786,
          0.9787198985301483,
          0.9849448019845684
         ]
        }
       ],
       "layout": {
        "template": {
         "data": {
          "bar": [
           {
            "error_x": {
             "color": "#2a3f5f"
            },
            "error_y": {
             "color": "#2a3f5f"
            },
            "marker": {
             "line": {
              "color": "#E5ECF6",
              "width": 0.5
             },
             "pattern": {
              "fillmode": "overlay",
              "size": 10,
              "solidity": 0.2
             }
            },
            "type": "bar"
           }
          ],
          "barpolar": [
           {
            "marker": {
             "line": {
              "color": "#E5ECF6",
              "width": 0.5
             },
             "pattern": {
              "fillmode": "overlay",
              "size": 10,
              "solidity": 0.2
             }
            },
            "type": "barpolar"
           }
          ],
          "carpet": [
           {
            "aaxis": {
             "endlinecolor": "#2a3f5f",
             "gridcolor": "white",
             "linecolor": "white",
             "minorgridcolor": "white",
             "startlinecolor": "#2a3f5f"
            },
            "baxis": {
             "endlinecolor": "#2a3f5f",
             "gridcolor": "white",
             "linecolor": "white",
             "minorgridcolor": "white",
             "startlinecolor": "#2a3f5f"
            },
            "type": "carpet"
           }
          ],
          "choropleth": [
           {
            "colorbar": {
             "outlinewidth": 0,
             "ticks": ""
            },
            "type": "choropleth"
           }
          ],
          "contour": [
           {
            "colorbar": {
             "outlinewidth": 0,
             "ticks": ""
            },
            "colorscale": [
             [
              0,
              "#0d0887"
             ],
             [
              0.1111111111111111,
              "#46039f"
             ],
             [
              0.2222222222222222,
              "#7201a8"
             ],
             [
              0.3333333333333333,
              "#9c179e"
             ],
             [
              0.4444444444444444,
              "#bd3786"
             ],
             [
              0.5555555555555556,
              "#d8576b"
             ],
             [
              0.6666666666666666,
              "#ed7953"
             ],
             [
              0.7777777777777778,
              "#fb9f3a"
             ],
             [
              0.8888888888888888,
              "#fdca26"
             ],
             [
              1,
              "#f0f921"
             ]
            ],
            "type": "contour"
           }
          ],
          "contourcarpet": [
           {
            "colorbar": {
             "outlinewidth": 0,
             "ticks": ""
            },
            "type": "contourcarpet"
           }
          ],
          "heatmap": [
           {
            "colorbar": {
             "outlinewidth": 0,
             "ticks": ""
            },
            "colorscale": [
             [
              0,
              "#0d0887"
             ],
             [
              0.1111111111111111,
              "#46039f"
             ],
             [
              0.2222222222222222,
              "#7201a8"
             ],
             [
              0.3333333333333333,
              "#9c179e"
             ],
             [
              0.4444444444444444,
              "#bd3786"
             ],
             [
              0.5555555555555556,
              "#d8576b"
             ],
             [
              0.6666666666666666,
              "#ed7953"
             ],
             [
              0.7777777777777778,
              "#fb9f3a"
             ],
             [
              0.8888888888888888,
              "#fdca26"
             ],
             [
              1,
              "#f0f921"
             ]
            ],
            "type": "heatmap"
           }
          ],
          "heatmapgl": [
           {
            "colorbar": {
             "outlinewidth": 0,
             "ticks": ""
            },
            "colorscale": [
             [
              0,
              "#0d0887"
             ],
             [
              0.1111111111111111,
              "#46039f"
             ],
             [
              0.2222222222222222,
              "#7201a8"
             ],
             [
              0.3333333333333333,
              "#9c179e"
             ],
             [
              0.4444444444444444,
              "#bd3786"
             ],
             [
              0.5555555555555556,
              "#d8576b"
             ],
             [
              0.6666666666666666,
              "#ed7953"
             ],
             [
              0.7777777777777778,
              "#fb9f3a"
             ],
             [
              0.8888888888888888,
              "#fdca26"
             ],
             [
              1,
              "#f0f921"
             ]
            ],
            "type": "heatmapgl"
           }
          ],
          "histogram": [
           {
            "marker": {
             "pattern": {
              "fillmode": "overlay",
              "size": 10,
              "solidity": 0.2
             }
            },
            "type": "histogram"
           }
          ],
          "histogram2d": [
           {
            "colorbar": {
             "outlinewidth": 0,
             "ticks": ""
            },
            "colorscale": [
             [
              0,
              "#0d0887"
             ],
             [
              0.1111111111111111,
              "#46039f"
             ],
             [
              0.2222222222222222,
              "#7201a8"
             ],
             [
              0.3333333333333333,
              "#9c179e"
             ],
             [
              0.4444444444444444,
              "#bd3786"
             ],
             [
              0.5555555555555556,
              "#d8576b"
             ],
             [
              0.6666666666666666,
              "#ed7953"
             ],
             [
              0.7777777777777778,
              "#fb9f3a"
             ],
             [
              0.8888888888888888,
              "#fdca26"
             ],
             [
              1,
              "#f0f921"
             ]
            ],
            "type": "histogram2d"
           }
          ],
          "histogram2dcontour": [
           {
            "colorbar": {
             "outlinewidth": 0,
             "ticks": ""
            },
            "colorscale": [
             [
              0,
              "#0d0887"
             ],
             [
              0.1111111111111111,
              "#46039f"
             ],
             [
              0.2222222222222222,
              "#7201a8"
             ],
             [
              0.3333333333333333,
              "#9c179e"
             ],
             [
              0.4444444444444444,
              "#bd3786"
             ],
             [
              0.5555555555555556,
              "#d8576b"
             ],
             [
              0.6666666666666666,
              "#ed7953"
             ],
             [
              0.7777777777777778,
              "#fb9f3a"
             ],
             [
              0.8888888888888888,
              "#fdca26"
             ],
             [
              1,
              "#f0f921"
             ]
            ],
            "type": "histogram2dcontour"
           }
          ],
          "mesh3d": [
           {
            "colorbar": {
             "outlinewidth": 0,
             "ticks": ""
            },
            "type": "mesh3d"
           }
          ],
          "parcoords": [
           {
            "line": {
             "colorbar": {
              "outlinewidth": 0,
              "ticks": ""
             }
            },
            "type": "parcoords"
           }
          ],
          "pie": [
           {
            "automargin": true,
            "type": "pie"
           }
          ],
          "scatter": [
           {
            "marker": {
             "colorbar": {
              "outlinewidth": 0,
              "ticks": ""
             }
            },
            "type": "scatter"
           }
          ],
          "scatter3d": [
           {
            "line": {
             "colorbar": {
              "outlinewidth": 0,
              "ticks": ""
             }
            },
            "marker": {
             "colorbar": {
              "outlinewidth": 0,
              "ticks": ""
             }
            },
            "type": "scatter3d"
           }
          ],
          "scattercarpet": [
           {
            "marker": {
             "colorbar": {
              "outlinewidth": 0,
              "ticks": ""
             }
            },
            "type": "scattercarpet"
           }
          ],
          "scattergeo": [
           {
            "marker": {
             "colorbar": {
              "outlinewidth": 0,
              "ticks": ""
             }
            },
            "type": "scattergeo"
           }
          ],
          "scattergl": [
           {
            "marker": {
             "colorbar": {
              "outlinewidth": 0,
              "ticks": ""
             }
            },
            "type": "scattergl"
           }
          ],
          "scattermapbox": [
           {
            "marker": {
             "colorbar": {
              "outlinewidth": 0,
              "ticks": ""
             }
            },
            "type": "scattermapbox"
           }
          ],
          "scatterpolar": [
           {
            "marker": {
             "colorbar": {
              "outlinewidth": 0,
              "ticks": ""
             }
            },
            "type": "scatterpolar"
           }
          ],
          "scatterpolargl": [
           {
            "marker": {
             "colorbar": {
              "outlinewidth": 0,
              "ticks": ""
             }
            },
            "type": "scatterpolargl"
           }
          ],
          "scatterternary": [
           {
            "marker": {
             "colorbar": {
              "outlinewidth": 0,
              "ticks": ""
             }
            },
            "type": "scatterternary"
           }
          ],
          "surface": [
           {
            "colorbar": {
             "outlinewidth": 0,
             "ticks": ""
            },
            "colorscale": [
             [
              0,
              "#0d0887"
             ],
             [
              0.1111111111111111,
              "#46039f"
             ],
             [
              0.2222222222222222,
              "#7201a8"
             ],
             [
              0.3333333333333333,
              "#9c179e"
             ],
             [
              0.4444444444444444,
              "#bd3786"
             ],
             [
              0.5555555555555556,
              "#d8576b"
             ],
             [
              0.6666666666666666,
              "#ed7953"
             ],
             [
              0.7777777777777778,
              "#fb9f3a"
             ],
             [
              0.8888888888888888,
              "#fdca26"
             ],
             [
              1,
              "#f0f921"
             ]
            ],
            "type": "surface"
           }
          ],
          "table": [
           {
            "cells": {
             "fill": {
              "color": "#EBF0F8"
             },
             "line": {
              "color": "white"
             }
            },
            "header": {
             "fill": {
              "color": "#C8D4E3"
             },
             "line": {
              "color": "white"
             }
            },
            "type": "table"
           }
          ]
         },
         "layout": {
          "annotationdefaults": {
           "arrowcolor": "#2a3f5f",
           "arrowhead": 0,
           "arrowwidth": 1
          },
          "autotypenumbers": "strict",
          "coloraxis": {
           "colorbar": {
            "outlinewidth": 0,
            "ticks": ""
           }
          },
          "colorscale": {
           "diverging": [
            [
             0,
             "#8e0152"
            ],
            [
             0.1,
             "#c51b7d"
            ],
            [
             0.2,
             "#de77ae"
            ],
            [
             0.3,
             "#f1b6da"
            ],
            [
             0.4,
             "#fde0ef"
            ],
            [
             0.5,
             "#f7f7f7"
            ],
            [
             0.6,
             "#e6f5d0"
            ],
            [
             0.7,
             "#b8e186"
            ],
            [
             0.8,
             "#7fbc41"
            ],
            [
             0.9,
             "#4d9221"
            ],
            [
             1,
             "#276419"
            ]
           ],
           "sequential": [
            [
             0,
             "#0d0887"
            ],
            [
             0.1111111111111111,
             "#46039f"
            ],
            [
             0.2222222222222222,
             "#7201a8"
            ],
            [
             0.3333333333333333,
             "#9c179e"
            ],
            [
             0.4444444444444444,
             "#bd3786"
            ],
            [
             0.5555555555555556,
             "#d8576b"
            ],
            [
             0.6666666666666666,
             "#ed7953"
            ],
            [
             0.7777777777777778,
             "#fb9f3a"
            ],
            [
             0.8888888888888888,
             "#fdca26"
            ],
            [
             1,
             "#f0f921"
            ]
           ],
           "sequentialminus": [
            [
             0,
             "#0d0887"
            ],
            [
             0.1111111111111111,
             "#46039f"
            ],
            [
             0.2222222222222222,
             "#7201a8"
            ],
            [
             0.3333333333333333,
             "#9c179e"
            ],
            [
             0.4444444444444444,
             "#bd3786"
            ],
            [
             0.5555555555555556,
             "#d8576b"
            ],
            [
             0.6666666666666666,
             "#ed7953"
            ],
            [
             0.7777777777777778,
             "#fb9f3a"
            ],
            [
             0.8888888888888888,
             "#fdca26"
            ],
            [
             1,
             "#f0f921"
            ]
           ]
          },
          "colorway": [
           "#636efa",
           "#EF553B",
           "#00cc96",
           "#ab63fa",
           "#FFA15A",
           "#19d3f3",
           "#FF6692",
           "#B6E880",
           "#FF97FF",
           "#FECB52"
          ],
          "font": {
           "color": "#2a3f5f"
          },
          "geo": {
           "bgcolor": "white",
           "lakecolor": "white",
           "landcolor": "#E5ECF6",
           "showlakes": true,
           "showland": true,
           "subunitcolor": "white"
          },
          "hoverlabel": {
           "align": "left"
          },
          "hovermode": "closest",
          "mapbox": {
           "style": "light"
          },
          "paper_bgcolor": "white",
          "plot_bgcolor": "#E5ECF6",
          "polar": {
           "angularaxis": {
            "gridcolor": "white",
            "linecolor": "white",
            "ticks": ""
           },
           "bgcolor": "#E5ECF6",
           "radialaxis": {
            "gridcolor": "white",
            "linecolor": "white",
            "ticks": ""
           }
          },
          "scene": {
           "xaxis": {
            "backgroundcolor": "#E5ECF6",
            "gridcolor": "white",
            "gridwidth": 2,
            "linecolor": "white",
            "showbackground": true,
            "ticks": "",
            "zerolinecolor": "white"
           },
           "yaxis": {
            "backgroundcolor": "#E5ECF6",
            "gridcolor": "white",
            "gridwidth": 2,
            "linecolor": "white",
            "showbackground": true,
            "ticks": "",
            "zerolinecolor": "white"
           },
           "zaxis": {
            "backgroundcolor": "#E5ECF6",
            "gridcolor": "white",
            "gridwidth": 2,
            "linecolor": "white",
            "showbackground": true,
            "ticks": "",
            "zerolinecolor": "white"
           }
          },
          "shapedefaults": {
           "line": {
            "color": "#2a3f5f"
           }
          },
          "ternary": {
           "aaxis": {
            "gridcolor": "white",
            "linecolor": "white",
            "ticks": ""
           },
           "baxis": {
            "gridcolor": "white",
            "linecolor": "white",
            "ticks": ""
           },
           "bgcolor": "#E5ECF6",
           "caxis": {
            "gridcolor": "white",
            "linecolor": "white",
            "ticks": ""
           }
          },
          "title": {
           "x": 0.05
          },
          "xaxis": {
           "automargin": true,
           "gridcolor": "white",
           "linecolor": "white",
           "ticks": "",
           "title": {
            "standoff": 15
           },
           "zerolinecolor": "white",
           "zerolinewidth": 2
          },
          "yaxis": {
           "automargin": true,
           "gridcolor": "white",
           "linecolor": "white",
           "ticks": "",
           "title": {
            "standoff": 15
           },
           "zerolinecolor": "white",
           "zerolinewidth": 2
          }
         }
        },
        "title": {
         "text": "Variação do valor médio das métricas nas 5 fold em relação ao número de iterações"
        }
       }
      }
     },
     "metadata": {}
    }
   ],
   "metadata": {}
  },
  {
   "cell_type": "code",
   "execution_count": 55,
   "source": [
    "fig = go.Figure()\n",
    "fig.add_trace(go.Scatter(x=n_estimators, y=metrics_std['acc'],\n",
    "                    mode='lines+markers',\n",
    "                    name='accuracy'))\n",
    "fig.add_trace(go.Scatter(x=n_estimators, y=metrics_std['roc'],\n",
    "                    mode='lines+markers',\n",
    "                    name='auc score'))\n",
    "fig.add_trace(go.Scatter(x=n_estimators, y=metrics_std['f1'],\n",
    "                    mode='lines+markers', name='f1-scores'))\n",
    "fig.update_layout(title=\"Variação do valor desvio padrão das métricas nas 5 fold em relação ao número de iterações\")\n",
    "fig.show()"
   ],
   "outputs": [
    {
     "output_type": "display_data",
     "data": {
      "application/vnd.plotly.v1+json": {
       "config": {
        "plotlyServerURL": "https://plot.ly"
       },
       "data": [
        {
         "mode": "lines+markers",
         "name": "accuracy",
         "type": "scatter",
         "x": [
          10,
          20,
          50,
          100,
          200,
          500
         ],
         "y": [
          0.040354668440382345,
          0.0317700533637067,
          0.039802581927728876,
          0.0287444591424502,
          0.013513332643413393,
          0.01419028589400449
         ]
        },
        {
         "mode": "lines+markers",
         "name": "auc score",
         "type": "scatter",
         "x": [
          10,
          20,
          50,
          100,
          200,
          500
         ],
         "y": [
          0.0372253966213144,
          0.03516220490090237,
          0.049694009312443346,
          0.03998686995961903,
          0.018138990451760133,
          0.017056492935921257
         ]
        },
        {
         "mode": "lines+markers",
         "name": "f1-scores",
         "type": "scatter",
         "x": [
          10,
          20,
          50,
          100,
          200,
          500
         ],
         "y": [
          0.031181296304494777,
          0.02273576983772843,
          0.027056330763413496,
          0.01905133155292103,
          0.010100971256421414,
          0.010689244002965571
         ]
        }
       ],
       "layout": {
        "template": {
         "data": {
          "bar": [
           {
            "error_x": {
             "color": "#2a3f5f"
            },
            "error_y": {
             "color": "#2a3f5f"
            },
            "marker": {
             "line": {
              "color": "#E5ECF6",
              "width": 0.5
             },
             "pattern": {
              "fillmode": "overlay",
              "size": 10,
              "solidity": 0.2
             }
            },
            "type": "bar"
           }
          ],
          "barpolar": [
           {
            "marker": {
             "line": {
              "color": "#E5ECF6",
              "width": 0.5
             },
             "pattern": {
              "fillmode": "overlay",
              "size": 10,
              "solidity": 0.2
             }
            },
            "type": "barpolar"
           }
          ],
          "carpet": [
           {
            "aaxis": {
             "endlinecolor": "#2a3f5f",
             "gridcolor": "white",
             "linecolor": "white",
             "minorgridcolor": "white",
             "startlinecolor": "#2a3f5f"
            },
            "baxis": {
             "endlinecolor": "#2a3f5f",
             "gridcolor": "white",
             "linecolor": "white",
             "minorgridcolor": "white",
             "startlinecolor": "#2a3f5f"
            },
            "type": "carpet"
           }
          ],
          "choropleth": [
           {
            "colorbar": {
             "outlinewidth": 0,
             "ticks": ""
            },
            "type": "choropleth"
           }
          ],
          "contour": [
           {
            "colorbar": {
             "outlinewidth": 0,
             "ticks": ""
            },
            "colorscale": [
             [
              0,
              "#0d0887"
             ],
             [
              0.1111111111111111,
              "#46039f"
             ],
             [
              0.2222222222222222,
              "#7201a8"
             ],
             [
              0.3333333333333333,
              "#9c179e"
             ],
             [
              0.4444444444444444,
              "#bd3786"
             ],
             [
              0.5555555555555556,
              "#d8576b"
             ],
             [
              0.6666666666666666,
              "#ed7953"
             ],
             [
              0.7777777777777778,
              "#fb9f3a"
             ],
             [
              0.8888888888888888,
              "#fdca26"
             ],
             [
              1,
              "#f0f921"
             ]
            ],
            "type": "contour"
           }
          ],
          "contourcarpet": [
           {
            "colorbar": {
             "outlinewidth": 0,
             "ticks": ""
            },
            "type": "contourcarpet"
           }
          ],
          "heatmap": [
           {
            "colorbar": {
             "outlinewidth": 0,
             "ticks": ""
            },
            "colorscale": [
             [
              0,
              "#0d0887"
             ],
             [
              0.1111111111111111,
              "#46039f"
             ],
             [
              0.2222222222222222,
              "#7201a8"
             ],
             [
              0.3333333333333333,
              "#9c179e"
             ],
             [
              0.4444444444444444,
              "#bd3786"
             ],
             [
              0.5555555555555556,
              "#d8576b"
             ],
             [
              0.6666666666666666,
              "#ed7953"
             ],
             [
              0.7777777777777778,
              "#fb9f3a"
             ],
             [
              0.8888888888888888,
              "#fdca26"
             ],
             [
              1,
              "#f0f921"
             ]
            ],
            "type": "heatmap"
           }
          ],
          "heatmapgl": [
           {
            "colorbar": {
             "outlinewidth": 0,
             "ticks": ""
            },
            "colorscale": [
             [
              0,
              "#0d0887"
             ],
             [
              0.1111111111111111,
              "#46039f"
             ],
             [
              0.2222222222222222,
              "#7201a8"
             ],
             [
              0.3333333333333333,
              "#9c179e"
             ],
             [
              0.4444444444444444,
              "#bd3786"
             ],
             [
              0.5555555555555556,
              "#d8576b"
             ],
             [
              0.6666666666666666,
              "#ed7953"
             ],
             [
              0.7777777777777778,
              "#fb9f3a"
             ],
             [
              0.8888888888888888,
              "#fdca26"
             ],
             [
              1,
              "#f0f921"
             ]
            ],
            "type": "heatmapgl"
           }
          ],
          "histogram": [
           {
            "marker": {
             "pattern": {
              "fillmode": "overlay",
              "size": 10,
              "solidity": 0.2
             }
            },
            "type": "histogram"
           }
          ],
          "histogram2d": [
           {
            "colorbar": {
             "outlinewidth": 0,
             "ticks": ""
            },
            "colorscale": [
             [
              0,
              "#0d0887"
             ],
             [
              0.1111111111111111,
              "#46039f"
             ],
             [
              0.2222222222222222,
              "#7201a8"
             ],
             [
              0.3333333333333333,
              "#9c179e"
             ],
             [
              0.4444444444444444,
              "#bd3786"
             ],
             [
              0.5555555555555556,
              "#d8576b"
             ],
             [
              0.6666666666666666,
              "#ed7953"
             ],
             [
              0.7777777777777778,
              "#fb9f3a"
             ],
             [
              0.8888888888888888,
              "#fdca26"
             ],
             [
              1,
              "#f0f921"
             ]
            ],
            "type": "histogram2d"
           }
          ],
          "histogram2dcontour": [
           {
            "colorbar": {
             "outlinewidth": 0,
             "ticks": ""
            },
            "colorscale": [
             [
              0,
              "#0d0887"
             ],
             [
              0.1111111111111111,
              "#46039f"
             ],
             [
              0.2222222222222222,
              "#7201a8"
             ],
             [
              0.3333333333333333,
              "#9c179e"
             ],
             [
              0.4444444444444444,
              "#bd3786"
             ],
             [
              0.5555555555555556,
              "#d8576b"
             ],
             [
              0.6666666666666666,
              "#ed7953"
             ],
             [
              0.7777777777777778,
              "#fb9f3a"
             ],
             [
              0.8888888888888888,
              "#fdca26"
             ],
             [
              1,
              "#f0f921"
             ]
            ],
            "type": "histogram2dcontour"
           }
          ],
          "mesh3d": [
           {
            "colorbar": {
             "outlinewidth": 0,
             "ticks": ""
            },
            "type": "mesh3d"
           }
          ],
          "parcoords": [
           {
            "line": {
             "colorbar": {
              "outlinewidth": 0,
              "ticks": ""
             }
            },
            "type": "parcoords"
           }
          ],
          "pie": [
           {
            "automargin": true,
            "type": "pie"
           }
          ],
          "scatter": [
           {
            "marker": {
             "colorbar": {
              "outlinewidth": 0,
              "ticks": ""
             }
            },
            "type": "scatter"
           }
          ],
          "scatter3d": [
           {
            "line": {
             "colorbar": {
              "outlinewidth": 0,
              "ticks": ""
             }
            },
            "marker": {
             "colorbar": {
              "outlinewidth": 0,
              "ticks": ""
             }
            },
            "type": "scatter3d"
           }
          ],
          "scattercarpet": [
           {
            "marker": {
             "colorbar": {
              "outlinewidth": 0,
              "ticks": ""
             }
            },
            "type": "scattercarpet"
           }
          ],
          "scattergeo": [
           {
            "marker": {
             "colorbar": {
              "outlinewidth": 0,
              "ticks": ""
             }
            },
            "type": "scattergeo"
           }
          ],
          "scattergl": [
           {
            "marker": {
             "colorbar": {
              "outlinewidth": 0,
              "ticks": ""
             }
            },
            "type": "scattergl"
           }
          ],
          "scattermapbox": [
           {
            "marker": {
             "colorbar": {
              "outlinewidth": 0,
              "ticks": ""
             }
            },
            "type": "scattermapbox"
           }
          ],
          "scatterpolar": [
           {
            "marker": {
             "colorbar": {
              "outlinewidth": 0,
              "ticks": ""
             }
            },
            "type": "scatterpolar"
           }
          ],
          "scatterpolargl": [
           {
            "marker": {
             "colorbar": {
              "outlinewidth": 0,
              "ticks": ""
             }
            },
            "type": "scatterpolargl"
           }
          ],
          "scatterternary": [
           {
            "marker": {
             "colorbar": {
              "outlinewidth": 0,
              "ticks": ""
             }
            },
            "type": "scatterternary"
           }
          ],
          "surface": [
           {
            "colorbar": {
             "outlinewidth": 0,
             "ticks": ""
            },
            "colorscale": [
             [
              0,
              "#0d0887"
             ],
             [
              0.1111111111111111,
              "#46039f"
             ],
             [
              0.2222222222222222,
              "#7201a8"
             ],
             [
              0.3333333333333333,
              "#9c179e"
             ],
             [
              0.4444444444444444,
              "#bd3786"
             ],
             [
              0.5555555555555556,
              "#d8576b"
             ],
             [
              0.6666666666666666,
              "#ed7953"
             ],
             [
              0.7777777777777778,
              "#fb9f3a"
             ],
             [
              0.8888888888888888,
              "#fdca26"
             ],
             [
              1,
              "#f0f921"
             ]
            ],
            "type": "surface"
           }
          ],
          "table": [
           {
            "cells": {
             "fill": {
              "color": "#EBF0F8"
             },
             "line": {
              "color": "white"
             }
            },
            "header": {
             "fill": {
              "color": "#C8D4E3"
             },
             "line": {
              "color": "white"
             }
            },
            "type": "table"
           }
          ]
         },
         "layout": {
          "annotationdefaults": {
           "arrowcolor": "#2a3f5f",
           "arrowhead": 0,
           "arrowwidth": 1
          },
          "autotypenumbers": "strict",
          "coloraxis": {
           "colorbar": {
            "outlinewidth": 0,
            "ticks": ""
           }
          },
          "colorscale": {
           "diverging": [
            [
             0,
             "#8e0152"
            ],
            [
             0.1,
             "#c51b7d"
            ],
            [
             0.2,
             "#de77ae"
            ],
            [
             0.3,
             "#f1b6da"
            ],
            [
             0.4,
             "#fde0ef"
            ],
            [
             0.5,
             "#f7f7f7"
            ],
            [
             0.6,
             "#e6f5d0"
            ],
            [
             0.7,
             "#b8e186"
            ],
            [
             0.8,
             "#7fbc41"
            ],
            [
             0.9,
             "#4d9221"
            ],
            [
             1,
             "#276419"
            ]
           ],
           "sequential": [
            [
             0,
             "#0d0887"
            ],
            [
             0.1111111111111111,
             "#46039f"
            ],
            [
             0.2222222222222222,
             "#7201a8"
            ],
            [
             0.3333333333333333,
             "#9c179e"
            ],
            [
             0.4444444444444444,
             "#bd3786"
            ],
            [
             0.5555555555555556,
             "#d8576b"
            ],
            [
             0.6666666666666666,
             "#ed7953"
            ],
            [
             0.7777777777777778,
             "#fb9f3a"
            ],
            [
             0.8888888888888888,
             "#fdca26"
            ],
            [
             1,
             "#f0f921"
            ]
           ],
           "sequentialminus": [
            [
             0,
             "#0d0887"
            ],
            [
             0.1111111111111111,
             "#46039f"
            ],
            [
             0.2222222222222222,
             "#7201a8"
            ],
            [
             0.3333333333333333,
             "#9c179e"
            ],
            [
             0.4444444444444444,
             "#bd3786"
            ],
            [
             0.5555555555555556,
             "#d8576b"
            ],
            [
             0.6666666666666666,
             "#ed7953"
            ],
            [
             0.7777777777777778,
             "#fb9f3a"
            ],
            [
             0.8888888888888888,
             "#fdca26"
            ],
            [
             1,
             "#f0f921"
            ]
           ]
          },
          "colorway": [
           "#636efa",
           "#EF553B",
           "#00cc96",
           "#ab63fa",
           "#FFA15A",
           "#19d3f3",
           "#FF6692",
           "#B6E880",
           "#FF97FF",
           "#FECB52"
          ],
          "font": {
           "color": "#2a3f5f"
          },
          "geo": {
           "bgcolor": "white",
           "lakecolor": "white",
           "landcolor": "#E5ECF6",
           "showlakes": true,
           "showland": true,
           "subunitcolor": "white"
          },
          "hoverlabel": {
           "align": "left"
          },
          "hovermode": "closest",
          "mapbox": {
           "style": "light"
          },
          "paper_bgcolor": "white",
          "plot_bgcolor": "#E5ECF6",
          "polar": {
           "angularaxis": {
            "gridcolor": "white",
            "linecolor": "white",
            "ticks": ""
           },
           "bgcolor": "#E5ECF6",
           "radialaxis": {
            "gridcolor": "white",
            "linecolor": "white",
            "ticks": ""
           }
          },
          "scene": {
           "xaxis": {
            "backgroundcolor": "#E5ECF6",
            "gridcolor": "white",
            "gridwidth": 2,
            "linecolor": "white",
            "showbackground": true,
            "ticks": "",
            "zerolinecolor": "white"
           },
           "yaxis": {
            "backgroundcolor": "#E5ECF6",
            "gridcolor": "white",
            "gridwidth": 2,
            "linecolor": "white",
            "showbackground": true,
            "ticks": "",
            "zerolinecolor": "white"
           },
           "zaxis": {
            "backgroundcolor": "#E5ECF6",
            "gridcolor": "white",
            "gridwidth": 2,
            "linecolor": "white",
            "showbackground": true,
            "ticks": "",
            "zerolinecolor": "white"
           }
          },
          "shapedefaults": {
           "line": {
            "color": "#2a3f5f"
           }
          },
          "ternary": {
           "aaxis": {
            "gridcolor": "white",
            "linecolor": "white",
            "ticks": ""
           },
           "baxis": {
            "gridcolor": "white",
            "linecolor": "white",
            "ticks": ""
           },
           "bgcolor": "#E5ECF6",
           "caxis": {
            "gridcolor": "white",
            "linecolor": "white",
            "ticks": ""
           }
          },
          "title": {
           "x": 0.05
          },
          "xaxis": {
           "automargin": true,
           "gridcolor": "white",
           "linecolor": "white",
           "ticks": "",
           "title": {
            "standoff": 15
           },
           "zerolinecolor": "white",
           "zerolinewidth": 2
          },
          "yaxis": {
           "automargin": true,
           "gridcolor": "white",
           "linecolor": "white",
           "ticks": "",
           "title": {
            "standoff": 15
           },
           "zerolinecolor": "white",
           "zerolinewidth": 2
          }
         }
        },
        "title": {
         "text": "Variação do valor desvio padrão das métricas nas 5 fold em relação ao número de iterações"
        }
       }
      }
     },
     "metadata": {}
    }
   ],
   "metadata": {}
  },
  {
   "cell_type": "markdown",
   "source": [
    "Com base nos dados apresentados nos gráficos acima, é possível concluir que um número bom de estimators é 200. Pois esse possui desempenho superior aos menores e similar aos com quantidade de estimator maior que 200. Sendo assim, nos próximos experimentos, será utilizado o número de estimators igual a 200.\n",
    "\n",
    "# Variações no desempenho de cada stump após o ajuste de pesos\n",
    "\n",
    "O objetivo do próximo experimento é avaliar o desempenho de cada um dos 200 stumps criados. Para isso, inicialmente foi realizada a divisão dos dados em treino e teste. Foram separados 20% da base para teste e 80% para treinamento. Posteriormente é realizado o treinamento do adaboost e os valores de alpha e o stump criados em cada iteração são recuperados. \n",
    "\n",
    "Após o treinamento é realizada a predição com cada um dos stumps e são calculadas as métricas accuracy, f1-score e AUC."
   ],
   "metadata": {}
  },
  {
   "cell_type": "code",
   "execution_count": 56,
   "source": [
    "X_train, X_test, y_train, y_test = train_test_split(X, y, test_size=0.2, random_state = 42)\n"
   ],
   "outputs": [],
   "metadata": {}
  },
  {
   "cell_type": "code",
   "execution_count": 57,
   "source": [
    "ada, errors, stumps, alphas = adaboost(X_train, y_train, n_estimators=200)"
   ],
   "outputs": [],
   "metadata": {}
  },
  {
   "cell_type": "code",
   "execution_count": 58,
   "source": [
    "def predict_2(X, alphas, stumps):\n",
    "    return  np.array([stump.predict(X) for stump in stumps])\n",
    "\n",
    "acc = []\n",
    "roc = []\n",
    "f1 = []\n",
    "\n",
    "y_preds = predict_2(X_test, alphas, stumps)\n",
    "\n",
    "for y_pred in y_preds:\n",
    "\n",
    "    acc.append(accuracy_score(y_test, y_pred))\n",
    "    roc.append(roc_auc_score(y_test, y_pred))\n",
    "    f1.append(f1_score(y_test, y_pred))"
   ],
   "outputs": [],
   "metadata": {}
  },
  {
   "cell_type": "code",
   "execution_count": 63,
   "source": [
    "fig = go.Figure()\n",
    "\n",
    "fig.add_trace(go.Violin(y=acc, name='accuracy'))\n",
    "fig.add_trace(go.Violin(y=f1, name='f1-score'))\n",
    "fig.add_trace(go.Violin(y=roc, name='auc'))\n",
    "\n",
    "fig.update_layout(title=\"Desempenho do classificador a cada ajuste de peso e treinamento\")\n",
    "\n",
    "fig.show()"
   ],
   "outputs": [
    {
     "output_type": "display_data",
     "data": {
      "application/vnd.plotly.v1+json": {
       "config": {
        "plotlyServerURL": "https://plot.ly"
       },
       "data": [
        {
         "name": "accuracy",
         "type": "violin",
         "y": [
          0.6875,
          0.625,
          0.6458333333333334,
          0.6197916666666666,
          0.59375,
          0.6302083333333334,
          0.4895833333333333,
          0.53125,
          0.5260416666666666,
          0.5052083333333334,
          0.5,
          0.546875,
          0.5520833333333334,
          0.515625,
          0.6302083333333334,
          0.4895833333333333,
          0.53125,
          0.3854166666666667,
          0.5260416666666666,
          0.5052083333333334,
          0.5,
          0.6458333333333334,
          0.3489583333333333,
          0.5520833333333334,
          0.6875,
          0.625,
          0.4114583333333333,
          0.6197916666666666,
          0.515625,
          0.3489583333333333,
          0.59375,
          0.4114583333333333,
          0.546875,
          0.4427083333333333,
          0.6302083333333334,
          0.4895833333333333,
          0.625,
          0.3854166666666667,
          0.5260416666666666,
          0.5,
          0.6458333333333334,
          0.59375,
          0.6875,
          0.53125,
          0.6197916666666666,
          0.3489583333333333,
          0.5520833333333334,
          0.515625,
          0.4114583333333333,
          0.5052083333333334,
          0.5,
          0.546875,
          0.6302083333333334,
          0.6197916666666666,
          0.4427083333333333,
          0.4114583333333333,
          0.6458333333333334,
          0.625,
          0.3854166666666667,
          0.5260416666666666,
          0.59375,
          0.4895833333333333,
          0.6875,
          0.53125,
          0.3489583333333333,
          0.515625,
          0.5520833333333334,
          0.4114583333333333,
          0.5052083333333334,
          0.5,
          0.6458333333333334,
          0.3489583333333333,
          0.6875,
          0.6197916666666666,
          0.5520833333333334,
          0.625,
          0.3489583333333333,
          0.515625,
          0.5260416666666666,
          0.4114583333333333,
          0.59375,
          0.546875,
          0.4427083333333333,
          0.6302083333333334,
          0.4895833333333333,
          0.625,
          0.3854166666666667,
          0.5260416666666666,
          0.5,
          0.5052083333333334,
          0.6458333333333334,
          0.6875,
          0.53125,
          0.4114583333333333,
          0.515625,
          0.5520833333333334,
          0.6197916666666666,
          0.4114583333333333,
          0.3489583333333333,
          0.59375,
          0.546875,
          0.6302083333333334,
          0.4427083333333333,
          0.6197916666666666,
          0.515625,
          0.625,
          0.5260416666666666,
          0.3489583333333333,
          0.5,
          0.6458333333333334,
          0.5520833333333334,
          0.3489583333333333,
          0.59375,
          0.6875,
          0.4895833333333333,
          0.3854166666666667,
          0.53125,
          0.4114583333333333,
          0.5052083333333334,
          0.5,
          0.6458333333333334,
          0.5520833333333334,
          0.6302083333333334,
          0.6197916666666666,
          0.515625,
          0.3489583333333333,
          0.625,
          0.5260416666666666,
          0.546875,
          0.4114583333333333,
          0.59375,
          0.4427083333333333,
          0.3854166666666667,
          0.4895833333333333,
          0.6875,
          0.53125,
          0.4114583333333333,
          0.5052083333333334,
          0.515625,
          0.5,
          0.6458333333333334,
          0.5520833333333334,
          0.3489583333333333,
          0.6197916666666666,
          0.5260416666666666,
          0.625,
          0.6302083333333334,
          0.546875,
          0.4114583333333333,
          0.59375,
          0.4427083333333333,
          0.515625,
          0.4895833333333333,
          0.5260416666666666,
          0.625,
          0.3489583333333333,
          0.6875,
          0.3854166666666667,
          0.5052083333333334,
          0.5,
          0.6458333333333334,
          0.5520833333333334,
          0.4114583333333333,
          0.53125,
          0.6197916666666666,
          0.515625,
          0.3489583333333333,
          0.6875,
          0.59375,
          0.546875,
          0.4114583333333333,
          0.5520833333333334,
          0.5260416666666666,
          0.625,
          0.3854166666666667,
          0.6197916666666666,
          0.6302083333333334,
          0.5052083333333334,
          0.5,
          0.3489583333333333,
          0.6458333333333334,
          0.4427083333333333,
          0.4895833333333333,
          0.5260416666666666,
          0.515625,
          0.625,
          0.3489583333333333,
          0.6875,
          0.5,
          0.59375,
          0.546875,
          0.4114583333333333,
          0.5520833333333334,
          0.53125,
          0.515625,
          0.6197916666666666,
          0.4114583333333333,
          0.6302083333333334,
          0.6458333333333334,
          0.3489583333333333
         ]
        },
        {
         "name": "f1-score",
         "type": "violin",
         "y": [
          0.7560975609756098,
          0.7096774193548386,
          0.732283464566929,
          0.7203065134099617,
          0.688,
          0.6728110599078342,
          0.5288461538461539,
          0.5754716981132074,
          0.6539923954372624,
          0.5497630331753555,
          0.6220472440944882,
          0.5837320574162679,
          0.653225806451613,
          0.6172839506172839,
          0.6728110599078342,
          0.5288461538461539,
          0.5754716981132074,
          0.39175257731958757,
          0.6539923954372624,
          0.5497630331753555,
          0.6220472440944882,
          0.732283464566929,
          0,
          0.653225806451613,
          0.7560975609756098,
          0.7096774193548386,
          0.44878048780487806,
          0.7203065134099617,
          0.6172839506172839,
          0,
          0.688,
          0.4145077720207254,
          0.5837320574162679,
          0.44559585492227977,
          0.6728110599078342,
          0.5288461538461539,
          0.7096774193548386,
          0.39175257731958757,
          0.6539923954372624,
          0.6220472440944882,
          0.732283464566929,
          0.688,
          0.7560975609756098,
          0.5754716981132074,
          0.7203065134099617,
          0,
          0.653225806451613,
          0.6172839506172839,
          0.44878048780487806,
          0.5497630331753555,
          0.6220472440944882,
          0.5837320574162679,
          0.6728110599078342,
          0.7203065134099617,
          0.44559585492227977,
          0.4145077720207254,
          0.732283464566929,
          0.7096774193548386,
          0.39175257731958757,
          0.6539923954372624,
          0.688,
          0.5288461538461539,
          0.7560975609756098,
          0.5754716981132074,
          0,
          0.6172839506172839,
          0.653225806451613,
          0.44878048780487806,
          0.5497630331753555,
          0.6220472440944882,
          0.732283464566929,
          0,
          0.7560975609756098,
          0.7203065134099617,
          0.653225806451613,
          0.7096774193548386,
          0,
          0.6172839506172839,
          0.6539923954372624,
          0.4145077720207254,
          0.688,
          0.5837320574162679,
          0.44559585492227977,
          0.6728110599078342,
          0.5288461538461539,
          0.7096774193548386,
          0.39175257731958757,
          0.6539923954372624,
          0.6220472440944882,
          0.5497630331753555,
          0.732283464566929,
          0.7560975609756098,
          0.5754716981132074,
          0.44878048780487806,
          0.6172839506172839,
          0.653225806451613,
          0.7203065134099617,
          0.4145077720207254,
          0,
          0.688,
          0.5837320574162679,
          0.6728110599078342,
          0.44559585492227977,
          0.7203065134099617,
          0.6172839506172839,
          0.7096774193548386,
          0.6539923954372624,
          0,
          0.6220472440944882,
          0.732283464566929,
          0.653225806451613,
          0,
          0.688,
          0.7560975609756098,
          0.5288461538461539,
          0.39175257731958757,
          0.5754716981132074,
          0.44878048780487806,
          0.5497630331753555,
          0.6220472440944882,
          0.732283464566929,
          0.653225806451613,
          0.6728110599078342,
          0.7203065134099617,
          0.6172839506172839,
          0,
          0.7096774193548386,
          0.6539923954372624,
          0.5837320574162679,
          0.4145077720207254,
          0.688,
          0.44559585492227977,
          0.39175257731958757,
          0.5288461538461539,
          0.7560975609756098,
          0.5754716981132074,
          0.44878048780487806,
          0.5497630331753555,
          0.6172839506172839,
          0.6220472440944882,
          0.732283464566929,
          0.653225806451613,
          0,
          0.7203065134099617,
          0.6539923954372624,
          0.7096774193548386,
          0.6728110599078342,
          0.5837320574162679,
          0.4145077720207254,
          0.688,
          0.44559585492227977,
          0.6172839506172839,
          0.5288461538461539,
          0.6539923954372624,
          0.7096774193548386,
          0,
          0.7560975609756098,
          0.39175257731958757,
          0.5497630331753555,
          0.6220472440944882,
          0.732283464566929,
          0.653225806451613,
          0.44878048780487806,
          0.5754716981132074,
          0.7203065134099617,
          0.6172839506172839,
          0,
          0.7560975609756098,
          0.688,
          0.5837320574162679,
          0.4145077720207254,
          0.653225806451613,
          0.6539923954372624,
          0.7096774193548386,
          0.39175257731958757,
          0.7203065134099617,
          0.6728110599078342,
          0.5497630331753555,
          0.6220472440944882,
          0,
          0.732283464566929,
          0.44559585492227977,
          0.5288461538461539,
          0.6539923954372624,
          0.6172839506172839,
          0.7096774193548386,
          0,
          0.7560975609756098,
          0.6220472440944882,
          0.688,
          0.5837320574162679,
          0.44878048780487806,
          0.653225806451613,
          0.5754716981132074,
          0.6172839506172839,
          0.7203065134099617,
          0.4145077720207254,
          0.6728110599078342,
          0.732283464566929,
          0
         ]
        },
        {
         "name": "auc",
         "type": "violin",
         "y": [
          0.6630447761194029,
          0.5908059701492537,
          0.6033432835820896,
          0.5625671641791045,
          0.5529552238805969,
          0.6502089552238807,
          0.511044776119403,
          0.5499701492537313,
          0.45594029850746265,
          0.523044776119403,
          0.442865671641791,
          0.5723582089552239,
          0.5105671641791045,
          0.4791044776119403,
          0.6502089552238807,
          0.511044776119403,
          0.5499701492537313,
          0.42065671641791047,
          0.45594029850746265,
          0.523044776119403,
          0.442865671641791,
          0.6033432835820896,
          0.5,
          0.5105671641791045,
          0.6630447761194029,
          0.5908059701492537,
          0.43026865671641784,
          0.5625671641791045,
          0.4791044776119403,
          0.5,
          0.5529552238805969,
          0.45104477611940297,
          0.5723582089552239,
          0.4854328358208955,
          0.6502089552238807,
          0.511044776119403,
          0.5908059701492537,
          0.42065671641791047,
          0.45594029850746265,
          0.442865671641791,
          0.6033432835820896,
          0.5529552238805969,
          0.6630447761194029,
          0.5499701492537313,
          0.5625671641791045,
          0.5,
          0.5105671641791045,
          0.4791044776119403,
          0.43026865671641784,
          0.523044776119403,
          0.442865671641791,
          0.5723582089552239,
          0.6502089552238807,
          0.5625671641791045,
          0.4854328358208955,
          0.45104477611940297,
          0.6033432835820896,
          0.5908059701492537,
          0.42065671641791047,
          0.45594029850746265,
          0.5529552238805969,
          0.511044776119403,
          0.6630447761194029,
          0.5499701492537313,
          0.5,
          0.4791044776119403,
          0.5105671641791045,
          0.43026865671641784,
          0.523044776119403,
          0.442865671641791,
          0.6033432835820896,
          0.5,
          0.6630447761194029,
          0.5625671641791045,
          0.5105671641791045,
          0.5908059701492537,
          0.5,
          0.4791044776119403,
          0.45594029850746265,
          0.45104477611940297,
          0.5529552238805969,
          0.5723582089552239,
          0.4854328358208955,
          0.6502089552238807,
          0.511044776119403,
          0.5908059701492537,
          0.42065671641791047,
          0.45594029850746265,
          0.442865671641791,
          0.523044776119403,
          0.6033432835820896,
          0.6630447761194029,
          0.5499701492537313,
          0.43026865671641784,
          0.4791044776119403,
          0.5105671641791045,
          0.5625671641791045,
          0.45104477611940297,
          0.5,
          0.5529552238805969,
          0.5723582089552239,
          0.6502089552238807,
          0.4854328358208955,
          0.5625671641791045,
          0.4791044776119403,
          0.5908059701492537,
          0.45594029850746265,
          0.5,
          0.442865671641791,
          0.6033432835820896,
          0.5105671641791045,
          0.5,
          0.5529552238805969,
          0.6630447761194029,
          0.511044776119403,
          0.42065671641791047,
          0.5499701492537313,
          0.43026865671641784,
          0.523044776119403,
          0.442865671641791,
          0.6033432835820896,
          0.5105671641791045,
          0.6502089552238807,
          0.5625671641791045,
          0.4791044776119403,
          0.5,
          0.5908059701492537,
          0.45594029850746265,
          0.5723582089552239,
          0.45104477611940297,
          0.5529552238805969,
          0.4854328358208955,
          0.42065671641791047,
          0.511044776119403,
          0.6630447761194029,
          0.5499701492537313,
          0.43026865671641784,
          0.523044776119403,
          0.4791044776119403,
          0.442865671641791,
          0.6033432835820896,
          0.5105671641791045,
          0.5,
          0.5625671641791045,
          0.45594029850746265,
          0.5908059701492537,
          0.6502089552238807,
          0.5723582089552239,
          0.45104477611940297,
          0.5529552238805969,
          0.4854328358208955,
          0.4791044776119403,
          0.511044776119403,
          0.45594029850746265,
          0.5908059701492537,
          0.5,
          0.6630447761194029,
          0.42065671641791047,
          0.523044776119403,
          0.442865671641791,
          0.6033432835820896,
          0.5105671641791045,
          0.43026865671641784,
          0.5499701492537313,
          0.5625671641791045,
          0.4791044776119403,
          0.5,
          0.6630447761194029,
          0.5529552238805969,
          0.5723582089552239,
          0.45104477611940297,
          0.5105671641791045,
          0.45594029850746265,
          0.5908059701492537,
          0.42065671641791047,
          0.5625671641791045,
          0.6502089552238807,
          0.523044776119403,
          0.442865671641791,
          0.5,
          0.6033432835820896,
          0.4854328358208955,
          0.511044776119403,
          0.45594029850746265,
          0.4791044776119403,
          0.5908059701492537,
          0.5,
          0.6630447761194029,
          0.442865671641791,
          0.5529552238805969,
          0.5723582089552239,
          0.43026865671641784,
          0.5105671641791045,
          0.5499701492537313,
          0.4791044776119403,
          0.5625671641791045,
          0.45104477611940297,
          0.6502089552238807,
          0.6033432835820896,
          0.5
         ]
        }
       ],
       "layout": {
        "template": {
         "data": {
          "bar": [
           {
            "error_x": {
             "color": "#2a3f5f"
            },
            "error_y": {
             "color": "#2a3f5f"
            },
            "marker": {
             "line": {
              "color": "#E5ECF6",
              "width": 0.5
             },
             "pattern": {
              "fillmode": "overlay",
              "size": 10,
              "solidity": 0.2
             }
            },
            "type": "bar"
           }
          ],
          "barpolar": [
           {
            "marker": {
             "line": {
              "color": "#E5ECF6",
              "width": 0.5
             },
             "pattern": {
              "fillmode": "overlay",
              "size": 10,
              "solidity": 0.2
             }
            },
            "type": "barpolar"
           }
          ],
          "carpet": [
           {
            "aaxis": {
             "endlinecolor": "#2a3f5f",
             "gridcolor": "white",
             "linecolor": "white",
             "minorgridcolor": "white",
             "startlinecolor": "#2a3f5f"
            },
            "baxis": {
             "endlinecolor": "#2a3f5f",
             "gridcolor": "white",
             "linecolor": "white",
             "minorgridcolor": "white",
             "startlinecolor": "#2a3f5f"
            },
            "type": "carpet"
           }
          ],
          "choropleth": [
           {
            "colorbar": {
             "outlinewidth": 0,
             "ticks": ""
            },
            "type": "choropleth"
           }
          ],
          "contour": [
           {
            "colorbar": {
             "outlinewidth": 0,
             "ticks": ""
            },
            "colorscale": [
             [
              0,
              "#0d0887"
             ],
             [
              0.1111111111111111,
              "#46039f"
             ],
             [
              0.2222222222222222,
              "#7201a8"
             ],
             [
              0.3333333333333333,
              "#9c179e"
             ],
             [
              0.4444444444444444,
              "#bd3786"
             ],
             [
              0.5555555555555556,
              "#d8576b"
             ],
             [
              0.6666666666666666,
              "#ed7953"
             ],
             [
              0.7777777777777778,
              "#fb9f3a"
             ],
             [
              0.8888888888888888,
              "#fdca26"
             ],
             [
              1,
              "#f0f921"
             ]
            ],
            "type": "contour"
           }
          ],
          "contourcarpet": [
           {
            "colorbar": {
             "outlinewidth": 0,
             "ticks": ""
            },
            "type": "contourcarpet"
           }
          ],
          "heatmap": [
           {
            "colorbar": {
             "outlinewidth": 0,
             "ticks": ""
            },
            "colorscale": [
             [
              0,
              "#0d0887"
             ],
             [
              0.1111111111111111,
              "#46039f"
             ],
             [
              0.2222222222222222,
              "#7201a8"
             ],
             [
              0.3333333333333333,
              "#9c179e"
             ],
             [
              0.4444444444444444,
              "#bd3786"
             ],
             [
              0.5555555555555556,
              "#d8576b"
             ],
             [
              0.6666666666666666,
              "#ed7953"
             ],
             [
              0.7777777777777778,
              "#fb9f3a"
             ],
             [
              0.8888888888888888,
              "#fdca26"
             ],
             [
              1,
              "#f0f921"
             ]
            ],
            "type": "heatmap"
           }
          ],
          "heatmapgl": [
           {
            "colorbar": {
             "outlinewidth": 0,
             "ticks": ""
            },
            "colorscale": [
             [
              0,
              "#0d0887"
             ],
             [
              0.1111111111111111,
              "#46039f"
             ],
             [
              0.2222222222222222,
              "#7201a8"
             ],
             [
              0.3333333333333333,
              "#9c179e"
             ],
             [
              0.4444444444444444,
              "#bd3786"
             ],
             [
              0.5555555555555556,
              "#d8576b"
             ],
             [
              0.6666666666666666,
              "#ed7953"
             ],
             [
              0.7777777777777778,
              "#fb9f3a"
             ],
             [
              0.8888888888888888,
              "#fdca26"
             ],
             [
              1,
              "#f0f921"
             ]
            ],
            "type": "heatmapgl"
           }
          ],
          "histogram": [
           {
            "marker": {
             "pattern": {
              "fillmode": "overlay",
              "size": 10,
              "solidity": 0.2
             }
            },
            "type": "histogram"
           }
          ],
          "histogram2d": [
           {
            "colorbar": {
             "outlinewidth": 0,
             "ticks": ""
            },
            "colorscale": [
             [
              0,
              "#0d0887"
             ],
             [
              0.1111111111111111,
              "#46039f"
             ],
             [
              0.2222222222222222,
              "#7201a8"
             ],
             [
              0.3333333333333333,
              "#9c179e"
             ],
             [
              0.4444444444444444,
              "#bd3786"
             ],
             [
              0.5555555555555556,
              "#d8576b"
             ],
             [
              0.6666666666666666,
              "#ed7953"
             ],
             [
              0.7777777777777778,
              "#fb9f3a"
             ],
             [
              0.8888888888888888,
              "#fdca26"
             ],
             [
              1,
              "#f0f921"
             ]
            ],
            "type": "histogram2d"
           }
          ],
          "histogram2dcontour": [
           {
            "colorbar": {
             "outlinewidth": 0,
             "ticks": ""
            },
            "colorscale": [
             [
              0,
              "#0d0887"
             ],
             [
              0.1111111111111111,
              "#46039f"
             ],
             [
              0.2222222222222222,
              "#7201a8"
             ],
             [
              0.3333333333333333,
              "#9c179e"
             ],
             [
              0.4444444444444444,
              "#bd3786"
             ],
             [
              0.5555555555555556,
              "#d8576b"
             ],
             [
              0.6666666666666666,
              "#ed7953"
             ],
             [
              0.7777777777777778,
              "#fb9f3a"
             ],
             [
              0.8888888888888888,
              "#fdca26"
             ],
             [
              1,
              "#f0f921"
             ]
            ],
            "type": "histogram2dcontour"
           }
          ],
          "mesh3d": [
           {
            "colorbar": {
             "outlinewidth": 0,
             "ticks": ""
            },
            "type": "mesh3d"
           }
          ],
          "parcoords": [
           {
            "line": {
             "colorbar": {
              "outlinewidth": 0,
              "ticks": ""
             }
            },
            "type": "parcoords"
           }
          ],
          "pie": [
           {
            "automargin": true,
            "type": "pie"
           }
          ],
          "scatter": [
           {
            "marker": {
             "colorbar": {
              "outlinewidth": 0,
              "ticks": ""
             }
            },
            "type": "scatter"
           }
          ],
          "scatter3d": [
           {
            "line": {
             "colorbar": {
              "outlinewidth": 0,
              "ticks": ""
             }
            },
            "marker": {
             "colorbar": {
              "outlinewidth": 0,
              "ticks": ""
             }
            },
            "type": "scatter3d"
           }
          ],
          "scattercarpet": [
           {
            "marker": {
             "colorbar": {
              "outlinewidth": 0,
              "ticks": ""
             }
            },
            "type": "scattercarpet"
           }
          ],
          "scattergeo": [
           {
            "marker": {
             "colorbar": {
              "outlinewidth": 0,
              "ticks": ""
             }
            },
            "type": "scattergeo"
           }
          ],
          "scattergl": [
           {
            "marker": {
             "colorbar": {
              "outlinewidth": 0,
              "ticks": ""
             }
            },
            "type": "scattergl"
           }
          ],
          "scattermapbox": [
           {
            "marker": {
             "colorbar": {
              "outlinewidth": 0,
              "ticks": ""
             }
            },
            "type": "scattermapbox"
           }
          ],
          "scatterpolar": [
           {
            "marker": {
             "colorbar": {
              "outlinewidth": 0,
              "ticks": ""
             }
            },
            "type": "scatterpolar"
           }
          ],
          "scatterpolargl": [
           {
            "marker": {
             "colorbar": {
              "outlinewidth": 0,
              "ticks": ""
             }
            },
            "type": "scatterpolargl"
           }
          ],
          "scatterternary": [
           {
            "marker": {
             "colorbar": {
              "outlinewidth": 0,
              "ticks": ""
             }
            },
            "type": "scatterternary"
           }
          ],
          "surface": [
           {
            "colorbar": {
             "outlinewidth": 0,
             "ticks": ""
            },
            "colorscale": [
             [
              0,
              "#0d0887"
             ],
             [
              0.1111111111111111,
              "#46039f"
             ],
             [
              0.2222222222222222,
              "#7201a8"
             ],
             [
              0.3333333333333333,
              "#9c179e"
             ],
             [
              0.4444444444444444,
              "#bd3786"
             ],
             [
              0.5555555555555556,
              "#d8576b"
             ],
             [
              0.6666666666666666,
              "#ed7953"
             ],
             [
              0.7777777777777778,
              "#fb9f3a"
             ],
             [
              0.8888888888888888,
              "#fdca26"
             ],
             [
              1,
              "#f0f921"
             ]
            ],
            "type": "surface"
           }
          ],
          "table": [
           {
            "cells": {
             "fill": {
              "color": "#EBF0F8"
             },
             "line": {
              "color": "white"
             }
            },
            "header": {
             "fill": {
              "color": "#C8D4E3"
             },
             "line": {
              "color": "white"
             }
            },
            "type": "table"
           }
          ]
         },
         "layout": {
          "annotationdefaults": {
           "arrowcolor": "#2a3f5f",
           "arrowhead": 0,
           "arrowwidth": 1
          },
          "autotypenumbers": "strict",
          "coloraxis": {
           "colorbar": {
            "outlinewidth": 0,
            "ticks": ""
           }
          },
          "colorscale": {
           "diverging": [
            [
             0,
             "#8e0152"
            ],
            [
             0.1,
             "#c51b7d"
            ],
            [
             0.2,
             "#de77ae"
            ],
            [
             0.3,
             "#f1b6da"
            ],
            [
             0.4,
             "#fde0ef"
            ],
            [
             0.5,
             "#f7f7f7"
            ],
            [
             0.6,
             "#e6f5d0"
            ],
            [
             0.7,
             "#b8e186"
            ],
            [
             0.8,
             "#7fbc41"
            ],
            [
             0.9,
             "#4d9221"
            ],
            [
             1,
             "#276419"
            ]
           ],
           "sequential": [
            [
             0,
             "#0d0887"
            ],
            [
             0.1111111111111111,
             "#46039f"
            ],
            [
             0.2222222222222222,
             "#7201a8"
            ],
            [
             0.3333333333333333,
             "#9c179e"
            ],
            [
             0.4444444444444444,
             "#bd3786"
            ],
            [
             0.5555555555555556,
             "#d8576b"
            ],
            [
             0.6666666666666666,
             "#ed7953"
            ],
            [
             0.7777777777777778,
             "#fb9f3a"
            ],
            [
             0.8888888888888888,
             "#fdca26"
            ],
            [
             1,
             "#f0f921"
            ]
           ],
           "sequentialminus": [
            [
             0,
             "#0d0887"
            ],
            [
             0.1111111111111111,
             "#46039f"
            ],
            [
             0.2222222222222222,
             "#7201a8"
            ],
            [
             0.3333333333333333,
             "#9c179e"
            ],
            [
             0.4444444444444444,
             "#bd3786"
            ],
            [
             0.5555555555555556,
             "#d8576b"
            ],
            [
             0.6666666666666666,
             "#ed7953"
            ],
            [
             0.7777777777777778,
             "#fb9f3a"
            ],
            [
             0.8888888888888888,
             "#fdca26"
            ],
            [
             1,
             "#f0f921"
            ]
           ]
          },
          "colorway": [
           "#636efa",
           "#EF553B",
           "#00cc96",
           "#ab63fa",
           "#FFA15A",
           "#19d3f3",
           "#FF6692",
           "#B6E880",
           "#FF97FF",
           "#FECB52"
          ],
          "font": {
           "color": "#2a3f5f"
          },
          "geo": {
           "bgcolor": "white",
           "lakecolor": "white",
           "landcolor": "#E5ECF6",
           "showlakes": true,
           "showland": true,
           "subunitcolor": "white"
          },
          "hoverlabel": {
           "align": "left"
          },
          "hovermode": "closest",
          "mapbox": {
           "style": "light"
          },
          "paper_bgcolor": "white",
          "plot_bgcolor": "#E5ECF6",
          "polar": {
           "angularaxis": {
            "gridcolor": "white",
            "linecolor": "white",
            "ticks": ""
           },
           "bgcolor": "#E5ECF6",
           "radialaxis": {
            "gridcolor": "white",
            "linecolor": "white",
            "ticks": ""
           }
          },
          "scene": {
           "xaxis": {
            "backgroundcolor": "#E5ECF6",
            "gridcolor": "white",
            "gridwidth": 2,
            "linecolor": "white",
            "showbackground": true,
            "ticks": "",
            "zerolinecolor": "white"
           },
           "yaxis": {
            "backgroundcolor": "#E5ECF6",
            "gridcolor": "white",
            "gridwidth": 2,
            "linecolor": "white",
            "showbackground": true,
            "ticks": "",
            "zerolinecolor": "white"
           },
           "zaxis": {
            "backgroundcolor": "#E5ECF6",
            "gridcolor": "white",
            "gridwidth": 2,
            "linecolor": "white",
            "showbackground": true,
            "ticks": "",
            "zerolinecolor": "white"
           }
          },
          "shapedefaults": {
           "line": {
            "color": "#2a3f5f"
           }
          },
          "ternary": {
           "aaxis": {
            "gridcolor": "white",
            "linecolor": "white",
            "ticks": ""
           },
           "baxis": {
            "gridcolor": "white",
            "linecolor": "white",
            "ticks": ""
           },
           "bgcolor": "#E5ECF6",
           "caxis": {
            "gridcolor": "white",
            "linecolor": "white",
            "ticks": ""
           }
          },
          "title": {
           "x": 0.05
          },
          "xaxis": {
           "automargin": true,
           "gridcolor": "white",
           "linecolor": "white",
           "ticks": "",
           "title": {
            "standoff": 15
           },
           "zerolinecolor": "white",
           "zerolinewidth": 2
          },
          "yaxis": {
           "automargin": true,
           "gridcolor": "white",
           "linecolor": "white",
           "ticks": "",
           "title": {
            "standoff": 15
           },
           "zerolinecolor": "white",
           "zerolinewidth": 2
          }
         }
        },
        "title": {
         "text": "Desempenho do classificador a cada ajuste de peso e treinamento"
        }
       }
      }
     },
     "metadata": {}
    }
   ],
   "metadata": {}
  },
  {
   "cell_type": "markdown",
   "source": [
    "A partir do gráfico acima é possível ver que a métrica accuracy e auc estão concentradas próximo ao valor de 0.5. Já a métrica f1 possui está concentrada em 0.68, porém, diferentemente da accuracy e auc ela possui módelos com métrica 0. O que chama bastante atenção. \n",
    "\n",
    "Vamos comparar agora o desempenho médio de cada modelo com o desempenho do modelo final, que utiliza a votação entre os stumps."
   ],
   "metadata": {}
  },
  {
   "cell_type": "markdown",
   "source": [
    "## Avaliação do desempenho na base de test"
   ],
   "metadata": {}
  },
  {
   "cell_type": "code",
   "execution_count": 60,
   "source": [
    "y_pred = predict(X_test, alphas, stumps)\n",
    "\n",
    "print(\"Accuracy: {} \".format(accuracy_score(y_test, y_pred)))\n",
    "print(\"AUC: {} \".format(roc_auc_score(y_test, y_pred)))\n",
    "print(\"F1: {} \".format(f1_score(y_test, y_pred)))"
   ],
   "outputs": [
    {
     "output_type": "stream",
     "name": "stdout",
     "text": [
      "Accuracy: 0.9635416666666666 \n",
      "AUC: 0.9512238805970149 \n",
      "F1: 0.9725490196078431 \n"
     ]
    }
   ],
   "metadata": {}
  },
  {
   "cell_type": "markdown",
   "source": [
    "No gráfico apresentada acima foi possível ver que, em geral, os stumps possuem um desempenho de 0.5 para Acurácia e AUC e de aproximadamente 0.68 para o F1-score. Oque é um desempenho muito ruim. Contudo, quando agregamos os stumps e realizamos a predição como sendo a votação o desempenho obtido é quase duas vezes melhor que o de 1 stump. O stump de melhor desempenho apresentou uma acurácia de 0.68, f1-score de 0.75 e AUC de 0.66. Quando combinamos os stumps constata-se que a combinação dos stumps gera um resultado superior que o melhor stump obtido.\n",
    "\n",
    "Além disso, com base nos resultados obtido na base de teste, verifica-se que a estimativa de erro obtida com cross-validation foi muito precisa, com diferença menor que 1%."
   ],
   "metadata": {}
  },
  {
   "cell_type": "markdown",
   "source": [
    "# Referências\n",
    "\n",
    "Freund, Y., Schapire, R., & Abe, N. (1999). A short introduction to boosting. Journal-Japanese Society For Artificial Intelligence, 14(771-780), 1612."
   ],
   "metadata": {}
  }
 ],
 "metadata": {
  "orig_nbformat": 4,
  "language_info": {
   "name": "python",
   "version": "3.8.10",
   "mimetype": "text/x-python",
   "codemirror_mode": {
    "name": "ipython",
    "version": 3
   },
   "pygments_lexer": "ipython3",
   "nbconvert_exporter": "python",
   "file_extension": ".py"
  },
  "kernelspec": {
   "name": "python3",
   "display_name": "Python 3.8.10 64-bit ('estudos': conda)"
  },
  "interpreter": {
   "hash": "8189520348208f2d7a55cd5e08e528a8ea0eb6334cdd1bc6eb49a516298cd84a"
  }
 },
 "nbformat": 4,
 "nbformat_minor": 2
}