SMS_Spam_Detection_JoaoLPacher.ipynb

{
 "cells": [
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "## <span style=\"color:blue\"> Business Problem: Classify SMS messages as “ok” or “blocked</span>\n",
    "**Author: João Luiz Pacher**<br>\n",
    "**2021, Feb 11st**"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 1,
   "metadata": {},
   "outputs": [],
   "source": [
    "# Importing the global required libraries\n",
    "import pandas as pd\n",
    "import numpy as np\n",
    "import seaborn as sns\n",
    "import matplotlib.pyplot as plt\n",
    "%matplotlib inline"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "## Reading and looking at the data \n",
    "First of all let´s look at the data and explore the available information.   \n",
    "\n",
    "We have 2 datasets:  \n",
    "    \"train_data\" to train the model   \n",
    "    \"validation_data\" to apply the model and get the classification results\n",
    "\n",
    "As we will confirm in the nexts cells:   \n",
    "    \"train_data\" has 1 attribute SMS and 1 target with 6000 instances.   \n",
    "    All data are non-nulls and dtype is object.  "
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "### ... about train_data"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 2,
   "metadata": {},
   "outputs": [],
   "source": [
    "# Reading the dataset and assigning it to an object\n",
    "train_data = pd.read_csv('train_data.csv')"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 3,
   "metadata": {},
   "outputs": [
    {
     "data": {
      "text/html": [
       "<div>\n",
       "<style scoped>\n",
       "    .dataframe tbody tr th:only-of-type {\n",
       "        vertical-align: middle;\n",
       "    }\n",
       "\n",
       "    .dataframe tbody tr th {\n",
       "        vertical-align: top;\n",
       "    }\n",
       "\n",
       "    .dataframe thead th {\n",
       "        text-align: right;\n",
       "    }\n",
       "</style>\n",
       "<table border=\"1\" class=\"dataframe\">\n",
       "  <thead>\n",
       "    <tr style=\"text-align: right;\">\n",
       "      <th></th>\n",
       "      <th>SMS</th>\n",
       "      <th>LABEL</th>\n",
       "    </tr>\n",
       "  </thead>\n",
       "  <tbody>\n",
       "    <tr>\n",
       "      <th>0</th>\n",
       "      <td>BOLETO REFERENTE AS PARCELAS EM ATRASO DO CONS...</td>\n",
       "      <td>blocked</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>1</th>\n",
       "      <td>050003DA0202|lcloud-apple-lnc.com/?iphone=VtBq...</td>\n",
       "      <td>blocked</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>2</th>\n",
       "      <td>060804CB5F0303|ps: //bancodobrasil.seguranca.gq</td>\n",
       "      <td>blocked</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>3</th>\n",
       "      <td>Atualizacao ainda nao foi executado em sua con...</td>\n",
       "      <td>blocked</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>4</th>\n",
       "      <td>0608042BE40303|ps://bancodobrasil.seguranca.cf/</td>\n",
       "      <td>blocked</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>5</th>\n",
       "      <td>Apple informa, seu iPhone foi localizado e ent...</td>\n",
       "      <td>blocked</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>6</th>\n",
       "      <td>Apple Inc. Mobile Apple foi localizado ?s 23:5...</td>\n",
       "      <td>blocked</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>7</th>\n",
       "      <td>06080446A50403|rápido e seguro.\\r\\r\\nAcesse: h...</td>\n",
       "      <td>blocked</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>8</th>\n",
       "      <td>Seu Dispositivo IMEI 3533****4383, Cor BLACK, ...</td>\n",
       "      <td>blocked</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>9</th>\n",
       "      <td>060804C5130303|bloqueio agora.\\r\\r\\nAcesse: ht...</td>\n",
       "      <td>blocked</td>\n",
       "    </tr>\n",
       "  </tbody>\n",
       "</table>\n",
       "</div>"
      ],
      "text/plain": [
       "                                                 SMS    LABEL\n",
       "0  BOLETO REFERENTE AS PARCELAS EM ATRASO DO CONS...  blocked\n",
       "1  050003DA0202|lcloud-apple-lnc.com/?iphone=VtBq...  blocked\n",
       "2    060804CB5F0303|ps: //bancodobrasil.seguranca.gq  blocked\n",
       "3  Atualizacao ainda nao foi executado em sua con...  blocked\n",
       "4    0608042BE40303|ps://bancodobrasil.seguranca.cf/  blocked\n",
       "5  Apple informa, seu iPhone foi localizado e ent...  blocked\n",
       "6  Apple Inc. Mobile Apple foi localizado ?s 23:5...  blocked\n",
       "7  06080446A50403|rápido e seguro.\\r\\r\\nAcesse: h...  blocked\n",
       "8  Seu Dispositivo IMEI 3533****4383, Cor BLACK, ...  blocked\n",
       "9  060804C5130303|bloqueio agora.\\r\\r\\nAcesse: ht...  blocked"
      ]
     },
     "execution_count": 3,
     "metadata": {},
     "output_type": "execute_result"
    }
   ],
   "source": [
    "# Looking at firsts rows from dataframe train_data\n",
    "train_data.head(10)"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 4,
   "metadata": {},
   "outputs": [
    {
     "data": {
      "text/plain": [
       "(6000, 2)"
      ]
     },
     "execution_count": 4,
     "metadata": {},
     "output_type": "execute_result"
    }
   ],
   "source": [
    "# Looking basic info from train dataframe\n",
    "train_data.shape"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "Train Dataframe has 6000 instances with information about SMS and LABEL.   "
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 5,
   "metadata": {},
   "outputs": [
    {
     "name": "stdout",
     "output_type": "stream",
     "text": [
      "<class 'pandas.core.frame.DataFrame'>\n",
      "RangeIndex: 6000 entries, 0 to 5999\n",
      "Data columns (total 2 columns):\n",
      " #   Column  Non-Null Count  Dtype \n",
      "---  ------  --------------  ----- \n",
      " 0   SMS     6000 non-null   object\n",
      " 1   LABEL   6000 non-null   object\n",
      "dtypes: object(2)\n",
      "memory usage: 93.9+ KB\n"
     ]
    }
   ],
   "source": [
    "# Looking for non-nulls and dtype in train_data\n",
    "train_data.info()"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "Train_data has 6000 non-null instances and dtype is object"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "## Exploring the target LABEL\n",
    "Let's explore if the target is balanced and some relations with the SMS content.  \n",
    "\n",
    "Addittional info: Maximum granted SMS lenght = 160.\n"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 6,
   "metadata": {},
   "outputs": [
    {
     "name": "stdout",
     "output_type": "stream",
     "text": [
      "LABEL: ['blocked' 'ok']\n"
     ]
    },
    {
     "data": {
      "text/plain": [
       "ok         4500\n",
       "blocked    1500\n",
       "Name: LABEL, dtype: int64"
      ]
     },
     "execution_count": 6,
     "metadata": {},
     "output_type": "execute_result"
    }
   ],
   "source": [
    "# Verifying if the target LABEL is balanced\n",
    "print('LABEL:', train_data['LABEL'].unique())\n",
    "train_data['LABEL'].value_counts()"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 7,
   "metadata": {},
   "outputs": [
    {
     "data": {
      "image/png": "iVBORw0KGgoAAAANSUhEUgAAAX0AAAElCAYAAAAIpDLLAAAABHNCSVQICAgIfAhkiAAAAAlwSFlzAAALEgAACxIB0t1+/AAAADh0RVh0U29mdHdhcmUAbWF0cGxvdGxpYiB2ZXJzaW9uMy4yLjIsIGh0dHA6Ly9tYXRwbG90bGliLm9yZy+WH4yJAAARQ0lEQVR4nO3df6zddX3H8efLgsAGCIQLlhYty7pNIIpSAadTJ4t0U9f+MWaZQmOIXZBFzZZs4JaJJp38ZRbMYMEfo0QnqZtCh2GOVXE6EXahaFdYQ6OTdkWogqEYZVLe++N86g7ltve29J4D9/N8JCfn+32f7/d73ofcvPj08/2e70lVIUnqwwvG3YAkaXQMfUnqiKEvSR0x9CWpI4a+JHXE0Jekjhj60ixKsihJJTlk3L1IYOirA0n+O8lPkjye5NEkX0xy8kE8/huTPNWOvzPJ5iTvOoDjXJHk0werL2kqhr568baqOhKYDzwEfOxADrKPEfv2dvyjgT8DPp7k1APqVJpFhr66UlU/Bf4B+HkgJ3lLkg1JHkuyNckVQ6/tnp65OMkDwJenOX5V1Y3Ao8PvMXS8k5KsS/JIki1J3t3qS4EPAG9v/2L41sH4vNKenGdUV5L8AvB24JtD5R8DFwGbgNOBW5Pc08J7tzcALwOemub4LwCWAccAG6fY5LPtfU4Cfq2913eq6p+T/BXwy1X1zgP6cNIMGPrqxY1JngSOBB4Gztv9QlXdNrTdt5N8lkHID4f+FVX1430c/6QkP2LwP4UHgAuranOSRbs3aOcRXge8tf2L454knwAuBNY/i88mzZihr14sr6p/TTKPwUj8q0lOrarvJzkbuJLBKP+FwGHA5/bYf+s0x99eVQun2eYk4JGq2jlU+x6wZMafQnqWnNNXV6pqV1V9HtjFYNQN8PfAOuDkqnoR8LdA9tz1ILz9duC4JEcN1V4C/M9BfA9pnwx9dSUDy4Bjgfta+SgGI/CfJjkL+IPZeO+q2gp8A/hIksOTvBy4GPhM2+QhYFE7LyDNCv+41It/SvI48BiwGlhZVZvaa+8BPpxkJ/CXwNpZ7OMCYBGDUf8XgA9W1a3ttd1TSj9Mcvcs9qCOxR9RkaR+ONKXpI4Y+pLUEUNfkjpi6EtSRwx9SerIc/4buccff3wtWrRo3G1I0vPKXXfd9YOqmtiz/pwP/UWLFjE5OTnuNiTpeSXJ96aqO70jSR0x9CWpI4a+JHXE0Jekjhj6ktQRQ1+SOmLoS1JHDH1J6shz/stZzxf50J6/rqcDVR/0Nx6k2eJIX5I6YuhLUkcMfUnqiKEvSR0x9CWpI4a+JHXE0Jekjhj6ktQRQ1+SOmLoS1JHDH1J6oihL0kdMfQlqSOGviR1xNCXpI4Y+pLUEUNfkjpi6EtSRwx9SerIjEM/ybwkG5Lc3NaPS3Jrkvvb87FD216eZEuSzUnOG6qfmWRje+2qJP6wrCSN0P6M9N8H3De0fhmwvqoWA+vbOklOBVYApwFLgauTzGv7XAOsAha3x9Jn1b0kab/MKPSTLATeAnxiqLwMWNOW1wDLh+o3VNUTVfVdYAtwVpL5wNFVdXtVFXD90D6SpBGY6Uj/r4E/BZ4aqp1YVQ8CtOcTWn0BsHVou22ttqAt71l/hiSrkkwmmdyxY8cMW5QkTWfa0E/yVuDhqrprhsecap6+9lF/ZrHq2qpaUlVLJiYmZvi2kqTpHDKDbV4L/G6S3wEOB45O8mngoSTzq+rBNnXzcNt+G3Dy0P4Lge2tvnCKuiRpRKYd6VfV5VW1sKoWMThB++WqeiewDljZNlsJ3NSW1wErkhyW5BQGJ2zvbFNAO5Oc067auWhoH0nSCMxkpL83VwJrk1wMPACcD1BVm5KsBe4FngQurapdbZ9LgOuAI4Bb2kOSNCL7FfpVdRtwW1v+IXDuXrZbDayeoj4JnL6/TUqSDg6/kStJHTH0Jakjhr4kdcTQl6SOGPqS1BFDX5I6YuhLUkcMfUnqiKEvSR0x9CWpI4a+JHXE0Jekjhj6ktQRQ1+SOmLoS1JHDH1J6oihL0kdMfQlqSOGviR1xNCXpI4Y+pLUEUNfkjpi6EtSRwx9SeqIoS9JHTH0Jakjhr4kdcTQl6SOGPqS1BFDX5I6YuhLUkcMfUnqiKEvSR0x9CWpI4a+JHXE0Jekjhj6ktSRaUM/yeFJ7kzyrSSbknyo1Y9LcmuS+9vzsUP7XJ5kS5LNSc4bqp+ZZGN77aokmZ2PJUmaykxG+k8Ab6qqVwBnAEuTnANcBqyvqsXA+rZOklOBFcBpwFLg6iTz2rGuAVYBi9tj6UH8LJKkaUwb+jXweFs9tD0KWAasafU1wPK2vAy4oaqeqKrvAluAs5LMB46uqturqoDrh/aRJI3AjOb0k8xLcg/wMHBrVd0BnFhVDwK05xPa5guArUO7b2u1BW15z/pU77cqyWSSyR07duzP55Ek7cOMQr+qdlXVGcBCBqP20/ex+VTz9LWP+lTvd21VLamqJRMTEzNpUZI0A/t19U5V/Qi4jcFc/ENtyob2/HDbbBtw8tBuC4Htrb5wirokaURmcvXORJJj2vIRwG8B/wWsA1a2zVYCN7XldcCKJIclOYXBCds72xTQziTntKt2LhraR5I0AofMYJv5wJp2Bc4LgLVVdXOS24G1SS4GHgDOB6iqTUnWAvcCTwKXVtWudqxLgOuAI4Bb2kOSNCLThn5VfRt45RT1HwLn7mWf1cDqKeqTwL7OB0iSZpHfyJWkjhj6ktQRQ1+SOmLoS1JHDH1J6oihL0kdMfQlqSOGviR1xNCXpI4Y+pLUEUNfkjpi6EtSRwx9SeqIoS9JHTH0Jakjhr4kdcTQl6SOGPqS1BFDX5I6YuhLUkcMfUnqiKEvSR0x9CWpI4a+JHXE0Jekjhj6ktQRQ1+SOmLoS1JHDH1J6oihL0kdMfQlqSOGviR1xNCXpI4Y+pLUEUNfkjpi6EtSRwx9SerItKGf5OQkX0lyX5JNSd7X6scluTXJ/e352KF9Lk+yJcnmJOcN1c9MsrG9dlWSzM7HkiRNZSYj/SeBP6mqlwHnAJcmORW4DFhfVYuB9W2d9toK4DRgKXB1knntWNcAq4DF7bH0IH4WSdI0pg39qnqwqu5uyzuB+4AFwDJgTdtsDbC8LS8DbqiqJ6rqu8AW4Kwk84Gjq+r2qirg+qF9JEkjsF9z+kkWAa8E7gBOrKoHYfA/BuCEttkCYOvQbttabUFb3rM+1fusSjKZZHLHjh3706IkaR9mHPpJjgT+EXh/VT22r02nqNU+6s8sVl1bVUuqasnExMRMW5QkTWNGoZ/kUAaB/5mq+nwrP9SmbGjPD7f6NuDkod0XAttbfeEUdUnSiMzk6p0AnwTuq6qPDr20DljZllcCNw3VVyQ5LMkpDE7Y3tmmgHYmOacd86KhfSRJI3DIDLZ5LXAhsDHJPa32AeBKYG2Si4EHgPMBqmpTkrXAvQyu/Lm0qna1/S4BrgOOAG5pD0nSiEwb+lX1daaejwc4dy/7rAZWT1GfBE7fnwYlSQeP38iVpI4Y+pLUEUNfkjpi6EtSRwx9SeqIoS9JHTH0Jakjhr4kdcTQl6SOGPqS1BFDX5I6YuhLUkcMfUnqiKEvSR0x9CWpI4a+JHXE0Jekjszk5xIlPZ9lbz98pwNSNe4OnhVH+pLUEUNfkjpi6EtSRwx9SeqIoS9JHTH0Jakjhr4kdcTQl6SOGPqS1BFDX5I6YuhLUkcMfUnqiKEvSR0x9CWpI4a+JHXE0Jekjhj6ktQRQ1+SOmLoS1JHpg39JJ9K8nCS/xyqHZfk1iT3t+djh167PMmWJJuTnDdUPzPJxvbaVYk/3ClJozaTkf51wNI9apcB66tqMbC+rZPkVGAFcFrb5+ok89o+1wCrgMXtsecxJUmzbNrQr6p/Ax7Zo7wMWNOW1wDLh+o3VNUTVfVdYAtwVpL5wNFVdXtVFXD90D6SpBE50Dn9E6vqQYD2fEKrLwC2Dm23rdUWtOU961NKsirJZJLJHTt2HGCLkqQ9HewTuVPN09c+6lOqqmuraklVLZmYmDhozUlS7w409B9qUza054dbfRtw8tB2C4Htrb5wirokaYQONPTXASvb8krgpqH6iiSHJTmFwQnbO9sU0M4k57Srdi4a2keSNCKHTLdBks8CbwSOT7IN+CBwJbA2ycXAA8D5AFW1Kcla4F7gSeDSqtrVDnUJgyuBjgBuaQ9J0ghNG/pVdcFeXjp3L9uvBlZPUZ8ETt+v7iRJB5XfyJWkjhj6ktQRQ1+SOmLoS1JHDH1J6oihL0kdMfQlqSOGviR1xNCXpI4Y+pLUEUNfkjpi6EtSRwx9SeqIoS9JHTH0Jakjhr4kdcTQl6SOGPqS1BFDX5I6YuhLUkcMfUnqiKEvSR0x9CWpI4a+JHXE0Jekjhj6ktQRQ1+SOmLoS1JHDH1J6oihL0kdMfQlqSOGviR1xNCXpI4Y+pLUEUNfkjpi6EtSRwx9SerIyEM/ydIkm5NsSXLZqN9fkno20tBPMg/4G+C3gVOBC5KcOsoeJKlnox7pnwVsqarvVNX/AjcAy0bcgyR165ARv98CYOvQ+jbg7D03SrIKWNVWH0+yeQS99eB44AfjbmI6uSLjbkHj8bz4+yTPm7/Pl05VHHXoT/Vfq55RqLoWuHb22+lLksmqWjLuPqSp+Pc5GqOe3tkGnDy0vhDYPuIeJKlbow79/wAWJzklyQuBFcC6EfcgSd0a6fROVT2Z5I+ALwHzgE9V1aZR9tA5p8z0XObf5wik6hlT6pKkOcpv5EpSRwx9SeqIoS9JHTH057gkZ05Re9s4epE0fp7IneOS3A2srKqNbf0C4P1V9YxvQkujkuRjTPHFzN2q6r0jbKcrjvTnvt8D1iR5WZJ3A+8B3jzmnqRJ4C7gcOBVwP3tcQawa4x9zXmO9DuQ5FeAGxnc92h5Vf1kzC1JACT5CvDmqvpZWz8U+Jeq+s3xdjZ3jfreOxqRJBt5+j+fj2Pwhbg7klBVLx9PZ9LTnAQcBTzS1o9sNc0SQ3/ueuvQ8rHAbzD4n8DXgB+NpSPpma4ENrQRP8AbgCvG187c5/TOHJfkvcC7gc8zuMvpcuDjVfWxsTYmNUlezP/fYv2Oqvr+OPuZ6wz9OS7Jt4HXVNWP2/ovArc7vaPngiQB3gH8UlV9OMlLgBdX1Z1jbm3O8uqduS88/WqIXUz9uwbSOFwNvAa4oK3vZPCTqpolzunPfX/H4OTtF9r6cuCTY+xHGnZ2Vb0qyQaAqnq03XZds8TQn+Oq6qNJbgNex2CE/66q2jDerqSf+1mSebQrzZJMAE+Nt6W5zTl9SWOT5B3A2xl8QWsNgy8T/kVVfW6sjc1hhr6ksUlyGHAKcC6Df4muBx6qqkf2uaMOmKEvaWySfJHBt8R3fyN3PnBzVT3jRoE6OLx6R9I43Qh8Lsm8JIsY/JTq5WPtaI5zpC9prJJcCiwFFgF/WFXfGG9Hc5uhL2nkkvzx8CpwIbAR2H3p5kfH0VcPvGRT0jgctcf6F/ZS10HmSF+SOuKJXEljk+TWJMcMrR+b5Evj7GmuM/QljdNEVf38Vt9V9Shwwhj7mfMMfUnjtKvdWROAJC9lH7+dq2fPE7mSxunPga8n+Wpbfz2waoz9zHmeyJU0VkmOB85pq9+sqh+Ms5+5zpG+pHH7dQYj/N1uHlcjPXCkL2lsklwJvBr4TCtdAExWlbdimCWGvqSxaT/neUZVPdXW5wEb/DnP2ePVO5LG7Zih5ReNrYtOOKcvaZw+AmxI8hUG9+B5Pd5lc1Y5vSNprNo99F/NIPTvqKrvj7mlOc3QlzRySV61r9er6u5R9dIbQ1/SyLXpnN2GQyhAVdWbRtxSNwx9SWOT5AjgPcDrGIT/14BrquqnY21sDjP0JY1NkrXAYzz9Ov1jqur3x9fV3GboSxqbJN+qqldMV9PB43X6ksZpQ5Ld990hydnAv4+xnznPkb6kkUuykcEc/qHArwIPtPWXAvdW1eljbG9OM/QljVy7b/5eVdX3RtVLbwx9SeqIc/qS1BFDX5I6YuhLUkcMfUnqiKEvSR35P6QgXQRqr3DzAAAAAElFTkSuQmCC\n",
      "text/plain": [
       "<Figure size 432x288 with 1 Axes>"
      ]
     },
     "metadata": {
      "needs_background": "light"
     },
     "output_type": "display_data"
    }
   ],
   "source": [
    "count_Class=pd.value_counts(train_data[\"LABEL\"], sort= True)\n",
    "count_Class.plot(kind = 'bar',color = [\"green\",\"red\"])\n",
    "plt.title('Bar Plot')\n",
    "plt.show();"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 8,
   "metadata": {},
   "outputs": [
    {
     "data": {
      "text/html": [
       "<div>\n",
       "<style scoped>\n",
       "    .dataframe tbody tr th:only-of-type {\n",
       "        vertical-align: middle;\n",
       "    }\n",
       "\n",
       "    .dataframe tbody tr th {\n",
       "        vertical-align: top;\n",
       "    }\n",
       "\n",
       "    .dataframe thead tr th {\n",
       "        text-align: left;\n",
       "    }\n",
       "\n",
       "    .dataframe thead tr:last-of-type th {\n",
       "        text-align: right;\n",
       "    }\n",
       "</style>\n",
       "<table border=\"1\" class=\"dataframe\">\n",
       "  <thead>\n",
       "    <tr>\n",
       "      <th></th>\n",
       "      <th colspan=\"4\" halign=\"left\">SMS</th>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th></th>\n",
       "      <th>count</th>\n",
       "      <th>unique</th>\n",
       "      <th>top</th>\n",
       "      <th>freq</th>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>LABEL</th>\n",
       "      <th></th>\n",
       "      <th></th>\n",
       "      <th></th>\n",
       "      <th></th>\n",
       "    </tr>\n",
       "  </thead>\n",
       "  <tbody>\n",
       "    <tr>\n",
       "      <th>blocked</th>\n",
       "      <td>1500</td>\n",
       "      <td>1495</td>\n",
       "      <td>Mercado BTC - Ha uma nova mensagem pendente. h...</td>\n",
       "      <td>3</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>ok</th>\n",
       "      <td>4500</td>\n",
       "      <td>4364</td>\n",
       "      <td>131.221.110.19 DNS-SECUNDARIO</td>\n",
       "      <td>13</td>\n",
       "    </tr>\n",
       "  </tbody>\n",
       "</table>\n",
       "</div>"
      ],
      "text/plain": [
       "          SMS                                                               \n",
       "        count unique                                                top freq\n",
       "LABEL                                                                       \n",
       "blocked  1500   1495  Mercado BTC - Ha uma nova mensagem pendente. h...    3\n",
       "ok       4500   4364                      131.221.110.19 DNS-SECUNDARIO   13"
      ]
     },
     "execution_count": 8,
     "metadata": {},
     "output_type": "execute_result"
    }
   ],
   "source": [
    "# Looking at the more frequent SMS for each kind of LABEL\n",
    "train_data.groupby('LABEL').describe()"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "There is not a big repetition of same SMS for each kind of LABEL."
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 9,
   "metadata": {},
   "outputs": [
    {
     "data": {
      "text/html": [
       "<div>\n",
       "<style scoped>\n",
       "    .dataframe tbody tr th:only-of-type {\n",
       "        vertical-align: middle;\n",
       "    }\n",
       "\n",
       "    .dataframe tbody tr th {\n",
       "        vertical-align: top;\n",
       "    }\n",
       "\n",
       "    .dataframe thead th {\n",
       "        text-align: right;\n",
       "    }\n",
       "</style>\n",
       "<table border=\"1\" class=\"dataframe\">\n",
       "  <thead>\n",
       "    <tr style=\"text-align: right;\">\n",
       "      <th></th>\n",
       "      <th>SMS</th>\n",
       "      <th>LABEL</th>\n",
       "      <th>LENGHT</th>\n",
       "    </tr>\n",
       "  </thead>\n",
       "  <tbody>\n",
       "    <tr>\n",
       "      <th>0</th>\n",
       "      <td>BOLETO REFERENTE AS PARCELAS EM ATRASO DO CONS...</td>\n",
       "      <td>blocked</td>\n",
       "      <td>142</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>1</th>\n",
       "      <td>050003DA0202|lcloud-apple-lnc.com/?iphone=VtBq...</td>\n",
       "      <td>blocked</td>\n",
       "      <td>52</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>2</th>\n",
       "      <td>060804CB5F0303|ps: //bancodobrasil.seguranca.gq</td>\n",
       "      <td>blocked</td>\n",
       "      <td>47</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>3</th>\n",
       "      <td>Atualizacao ainda nao foi executado em sua con...</td>\n",
       "      <td>blocked</td>\n",
       "      <td>109</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>4</th>\n",
       "      <td>0608042BE40303|ps://bancodobrasil.seguranca.cf/</td>\n",
       "      <td>blocked</td>\n",
       "      <td>47</td>\n",
       "    </tr>\n",
       "  </tbody>\n",
       "</table>\n",
       "</div>"
      ],
      "text/plain": [
       "                                                 SMS    LABEL  LENGHT\n",
       "0  BOLETO REFERENTE AS PARCELAS EM ATRASO DO CONS...  blocked     142\n",
       "1  050003DA0202|lcloud-apple-lnc.com/?iphone=VtBq...  blocked      52\n",
       "2    060804CB5F0303|ps: //bancodobrasil.seguranca.gq  blocked      47\n",
       "3  Atualizacao ainda nao foi executado em sua con...  blocked     109\n",
       "4    0608042BE40303|ps://bancodobrasil.seguranca.cf/  blocked      47"
      ]
     },
     "execution_count": 9,
     "metadata": {},
     "output_type": "execute_result"
    }
   ],
   "source": [
    "# Let's create a new feature LENGHT and check the number of characters in SMS\n",
    "# Knowing that the maximum number of characters in SMS = 160\n",
    "train_data['LENGHT'] = train_data['SMS'].apply(len)\n",
    "train_data.head()"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 10,
   "metadata": {},
   "outputs": [
    {
     "data": {
      "text/plain": [
       "Text(0, 0.5, 'Frequência')"
      ]
     },
     "execution_count": 10,
     "metadata": {},
     "output_type": "execute_result"
    },
    {
     "data": {
      "image/png": "iVBORw0KGgoAAAANSUhEUgAAAmgAAAEhCAYAAADLUFxwAAAABHNCSVQICAgIfAhkiAAAAAlwSFlzAAALEgAACxIB0t1+/AAAADh0RVh0U29mdHdhcmUAbWF0cGxvdGxpYiB2ZXJzaW9uMy4yLjIsIGh0dHA6Ly9tYXRwbG90bGliLm9yZy+WH4yJAAAgAElEQVR4nO3dfbgdZXnv8e+PgICiAhIwJIRgG19AL1FTtFUrFStvaqg9aGyraOlJzylW7LHVQD314GnaeHr00raiB19jfcGotaRiFRpF0SoYlIoQkAgIkQARQQEtlnCfP9ZEVjZ7Z6+d7LXXrL2+n+ta15p55pmZe83OenKvZ2aeSVUhSZKk9tht0AFIkiRpeyZokiRJLWOCJkmS1DImaJIkSS1jgiZJktQyJmiSJEktY4KmHUpyQ5LnjVN+dJJNu7jtRUkqye67sp1mW+PGKUnDJMkHk/zloOPQ4JmgSZIktYwJmiRJUsuYoKkXv5LkqiR3JPlAkr3GVkjyhCQXJbkzyZVJXtS1bO8kb03y/SQ/TvKVJHuPs43fbk5VPjHJbklWJPlektuTrEmyf1fdlzfbuz3Jn/fvo0vS9NtRm9lV5+FJvpjkb5NkEHFqcEzQ1IvfBY4Ffgl4LPDG7oVJ9gD+GbgAOBD4Y+AjSR7XVPm/wNOAXwP2B14P3D9mG68C3gI8r6q+A7wGOAl4DnAwcAfwzqbu4cC7gJc3yx4FLJjODyxJ/dJDm0mSRwHrgK9W1WvK5zKOnPg3144kuQFYVVXvbuZPAP4OOBX4cFUtSPJs4BPAwVV1f1PvY8A1wJuBe4BnVNW/j9n2IuB64M+A3weeX1WbmmUbgFdX1bpmfh5wI7A3cCZweFUta5Y9jE4Cd0JV/Wt/joQkTY9J2sxFdH7AHgWsrqq/GVScGqxdvntOI+Gmrunv0+m16nYwcNO2hqar3nzgAGAv4Hs72P6fAW/elpw1DgU+naR7m1uBg7btb1thVd2T5PYeP4skDdqO2kyAE4G7gXfPdGBqD09xqheHdE0vBG4es/xm4JAku42p9wPgh8B/0Dk9OpHnA29M8ttdZTcBx1fVvl2vvarqB8Dm7piSPJTOaU5JGgY7ajMB3gN8Dvhsc4ZAI8gETb04LcmC5iL9M4GPj1l+CZ3TmK9PskeSo4EXAuc2vxDfD7wtycFJ5iT51SR7dq1/JXAc8M6uC2XfDaxMcihAkrlJljbLPgm8IMmzkjyEzmlU/y1LGhYTtplddV5N55TnZ8a7qUqzn/+pqRcfpXMx63XNa7tBFKvq58CLgOPp9JidDbyiqq5uqvwpcAXwDeBHdG4G2G3MNv4deAHwniTHA+8A1gIXJLkL+Drw9KbulcBpTVyb6Vx/tkuD5krSTOmhzaS5KWA5nbMJ541397xmN28SkCRJahl70CRJklrGBE2SJKllTNAkSZJaxgRNkiSpZUzQJEmSWqYVTxI44IADatGiRYMOQ1KfXXbZZT+sqrmDjmMY2C5Ks9+O2sRWJGiLFi1i/fr1gw5DUp8l+f6gYxgWtovS7LejNtFTnJIkSS1jgiZJktQyJmiSJEktY4ImSZLUMiZokiRJLWOCJkmS1DImaJIkSS1jgiZJktQyrRioVqNp0Yrzt5u/YdWJA4pE0mxj+6JhZw+aJElSy5igSZIktYwJmiRJUsuYoEmSJLWMCZokSVLLmKBJkiS1jAmaJElSy5igSZIktYwJmiRJUsv0lKAl2TfJJ5NcnWRDkl9Nsn+SC5Nc27zv11X/jCQbk1yT5Nj+hS9JkjT79NqD9g7gc1X1eODJwAZgBbCuqhYD65p5khwOLAOOAI4Dzk4yZ7oDlyRJmq0mTdCSPAL4deB9AFX186q6E1gKrG6qrQZOaqaXAudW1b1VdT2wEThqugOXJEmarXrpQXsMsAX4QJJvJXlvkocBB1XVZoDm/cCm/nzgpq71NzVlkiRJ6kEvCdruwFOBd1XVU4B7aE5nTiDjlNWDKiXLk6xPsn7Lli09BStJkjQKeknQNgGbquqSZv6TdBK2W5PMA2jeb+uqf0jX+guAm8dutKrOqaolVbVk7ty5Oxu/JEnSrDNpglZVtwA3JXlcU3QMcBWwFjilKTsFOK+ZXgssS7JnksOAxcCl0xq1JEnSLLZ7j/X+GPhIkocA1wGvopPcrUlyKnAjcDJAVV2ZZA2dJO4+4LSq2jrtkUuSJM1SPSVoVXU5sGScRcdMUH8lsHIX4pIkSRpZPklAkiSpZXo9xSlJkjSUFq04f7v5G1adOKBIemcPmiRJUsuYoEmSJLWMCZokSVLLmKBJkiS1jAmaJElSy3gXpyRpqI29Q0+aDexBkyRJahkTNEnaCUnmJPlWks808/snuTDJtc37fl11z0iyMck1SY4dXNSShoUJmiTtnNOBDV3zK4B1VbUYWNfMk+RwYBlwBHAccHaSOTMcq6QhY4ImSVOUZAFwIvDeruKlwOpmejVwUlf5uVV1b1VdD2wEjpqpWCUNJxM0SZq6twOvB+7vKjuoqjYDNO8HNuXzgZu66m1qyh4kyfIk65Os37Jly/RHLWlomKBJ0hQkeQFwW1Vd1usq45TVeBWr6pyqWlJVS+bOnbvTMUoafg6zIUlT80zgRUlOAPYCHpHkw8CtSeZV1eYk84DbmvqbgEO61l8A3DyjEUsaOvagSdIUVNUZVbWgqhbRufj/C1X1e8Ba4JSm2inAec30WmBZkj2THAYsBi6d4bAlDRl70CRpeqwC1iQ5FbgROBmgqq5Msga4CrgPOK2qtg4uTEnDwARNknZSVV0EXNRM3w4cM0G9lcDKGQtM0tDzFKckSVLLmKBJkiS1jAmaJElSy5igSZIktYwJmiRJUsv0lKAluSHJFUkuT7K+Kds/yYVJrm3e9+uqf0aSjUmuSXJsv4KXJEmajabSg/YbVXVkVS1p5lcA66pqMbCumSfJ4XQGbzwCOA44O8mcaYxZkiRpVtuVcdCWAkc306vpjAX0hqb83Kq6F7g+yUbgKOBru7AvSZJm3KIV5283f8OqEwcUiUZNrz1oBVyQ5LIky5uyg6pqM0DzfmBTPh+4qWvdTU2ZJEmSetBrD9ozq+rmJAcCFya5egd1M05ZPahSJ9FbDrBw4cIew5AkSZr9eupBq6qbm/fbgE/TOWV5a5J5AM37bU31TcAhXasvAG4eZ5vnVNWSqloyd+7cnf8EkiRJs8ykCVqShyV5+LZp4PnAd4C1wClNtVOA85rptcCyJHsmOQxYDFw63YFLkiTNVr2c4jwI+HSSbfU/WlWfS/INYE2SU4EbgZMBqurKJGuAq4D7gNOqamtfopckSZqFJk3Qquo64MnjlN8OHDPBOiuBlbscnSRJ0gjySQKSJEktY4ImSZLUMrsyUK0kSdLQGTsAMbRvEGJ70CRJklrGBE2SJKllTNAkSZJaxgRNkiSpZUzQJEmSWsYETZIkqWVM0CRJklrGBE2SJKllTNAkSZJaxgRNkiSpZUzQJEmSWsYETZIkqWVM0CRJklrGBE2SJKllTNAkSZJaZvdBByBJg5LkSODZwAFAtpVX1V8MLChJwh40SSMqyXLgq8BzgTcATwJeB/zyIOOSJLAHTdLoej1wXFVdnOSOqvqtJMcDy3a0UpK9gC8De9JpQz9ZVW9Ksj/wcWARcAPwkqq6o1nnDOBUYCvwmqr6fJ8+kzSrLVpx/oPKblh14gAi6T970CSNqgOr6uJm+v4ku1XVvwAvnGS9e4HnVtWTgSOB45I8A1gBrKuqxcC6Zp4kh9NJ+o4AjgPOTjJn+j+OpNnEBE3SqNqUZFEz/V1gaZJnAz/f0UrVcXczu0fzKmApsLopXw2c1EwvBc6tqnur6npgI3DUdH0ISbNTzwlakjlJvpXkM838/kkuTHJt875fV90zkmxMck2SY/sRuCTtov8DPKGZfjPwYeALwFmTrdi0h5cDtwEXVtUlwEFVtRmgeT+wqT4fuKlr9U1NmSRNaCo9aKcDG7rm7c6XNLSq6oPNKU2a9/2A/arqXT2su7WqjgQWAEcleeIOqmecshq3YrI8yfok67ds2TL5h5A0a/WUoCVZAJwIvLer2O58SUMlSbqmd+t+AfcBP22me1JVdwIX0fkxemuSec2259HpXYNOj9khXastAG6eYHvnVNWSqloyd+7cKXwySbNNrw3R2+nc8XR/V5nd+ZKGzY+7pu8D/nPMa1vZhJLMTbJvM7038DzgamAtcEpT7RTgvGZ6LbAsyZ5JDgMWA5dOy6eRNGtNOsxGkhcAt1XVZUmO7mGbPXXnN2MQLQdYuHBhD5uVpF12RNf0YTu5jXnA6ubSjd2ANVX1mSRfA9YkORW4ETgZoKquTLIGuIpOAnhaVW3d6U8gaST0Mg7aM4EXJTkB2At4RJIP03TnV9XmnenOr6pzgHMAlixZMu71GJI0naqqu3f/FuD+qvpFj1mSPZjkzEJVfRt4yjjltwPHTLDOSmDlzsQsaTRNeoqzqs6oqgVVtYjOxf9fqKrfw+58ScPtQuBpY8qeBjiIrKSB25UnCazC7nxJw+tJwCVjyi4FnjyAWCRpO1NK0KrqIjp3LNmdL2nY/Rg4iM6pzm0OAu4ZTDiS9ACfJCBpVH0K+GiSJyZ5aJInAR8C1gw4LkkyQZM0sv6czuDblwJ3AV8HrgHOHGRQkgS7dg2aJA2tqvoP4LQkrwYOAH5YVd5RLqkVTNAkjawkjwQeB+zTzANQVV8YYFiSZIImaTQleSXwTuBu4Kddiwp4zCBikqRtTNAkjaqVwH/Z9sB0SWoTbxKQNKp2By4YdBCSNB4TNEmj6i3AG5PYDkpqHU9xShpVfwI8Gnh9ktu7F1TVwsGEJEkdJmiSRtXvDToASZqICZqkkVRVXxp0DJI0Ea+9kDSSkuyZZGWS65L8uCl7fjNwrSQNlAmapJGRZE6Sv2xm3w48EfhdOmOfAVwJ/PdBxCZJ3TzFKWmUXAC8q5k+Cfjlqronyf0AVfWDJPMHFp0kNexBkzRK7gf2bKZ/zpgfqUnmArePXUmSZpoJmqRRcizwy830J4DVSQ4DSDIP+Hvg3AHFJkm/YIImaWRU1f1VdVYzeyZwA3AFsC9wLXAzcNb4a0vSzPEaNEkjqap+DrwWeG1zavOHVVWTrCZJM8IETdJISvKYMUUPTwJAVV038xFJ0gNM0CSNqo10htdIV9m2HrQ5Mx+OJD3ABE3SSKqq7a7BTfJo4E3AxYOJSJIe4E0CkgRU1S10rkn760HHIkn2oEnSAx4HPHTQQWj6LVpx/oPKblh14gAikXozaYKWZC/gy3QGd9wd+GRVvSnJ/sDHgUV0blV/SVXd0axzBnAqsBV4TVV9vi/RS9JOSnIxD1xzBp3E7AjgzYOJSJIe0EsP2r3Ac6vq7iR7AF9J8i/Ai4F1VbUqyQpgBfCGJIcDy+g0dAcD/5rksVW1tU+fQZJ2xnvHzN8D/HtVXTuIYCSp26QJWjMu0N3N7B7Nq4ClwNFN+WrgIuANTfm5VXUvcH2SjcBRwNemM3BJ2hVVtXrQMUjSRHq6Bi3JHOAyOo9IeWdVXZLkoKraDFBVm5Mc2FSfD3y9a/VNTZkktUaSnk5lVtVf9DsWSRqrpwStOT15ZJJ9gU8neeIOqmecsgeNzp1kObAcYOHChb2EIUnTaTHw28A3gO8DC+n09n8K+I+mjk8WkDQQU7qLs6ruTHIRcBxwa5J5Te/ZPOC2ptom4JCu1RbQeb7d2G2dA5wDsGTJEhtBSTMtwMuq6lO/KEheDJxcVa8aXFiS1MM4aEnmNj1nJNkbeB5wNbAWOKWpdgpwXjO9FliWZM8kh9H5lXrpdAcuSbvoeOCfxpSdB5wwgFgkaTu99KDNA1Y316HtBqypqs8k+RqwJsmpwI3AyQBVdWWSNcBVwH3Aad7BKamFNgKnAX/bVfZHwPcGE44kPaCXuzi/DTxlnPLbgWMmWGclsHKXo5Ok/vkDOtfUvh74AZ2bme6jM4SQJA2Uj3qSNJKq6lt0LsF4GfA24HeAxVX1zcnWTXJIki8m2ZDkyiSnN+X7J7kwybXN+35d65yRZGOSa5Ic26ePJWmWMEGTJKCqvgw8JMnDeqh+H/C6qnoC8AzgtGaQ7hV0BvBeDKxr5hkzgPdxwNnNZSOSNC4TNEkjKcmTgO8C7wHe1xQ/B3j/ZOtW1eZtPW1VdRewgc4p0qV0Bu6meT+pmf7FAN5VdT2d69+OmqaPImkWMkGTNKreBfxFVT0e+M+m7EvAs6aykSSL6Fynewmw3QDeQPcA3jd1reYA3pJ2yARN0qg6AvhwM10AVXUPsHevG0iyD52BbV9bVT/ZUdVxysYdwDvJ+iTrt2zZ0msYkmYhEzRJo+oG4GndBUmOonP6cVJJ9qCTnH2kqv6xKb61GbibnR3Au6qWVNWSuXPnTuGjSJptTNAkjar/CZyf5Cw6NwecAXwCeONkKyYJnevWNlTV27oWOYC3pGkxpUc9SdJs0Qy4fTyd8dC+BBwKvLiqLuth9WcCLweuSHJ5U3YmsAoH8JY0DUzQJI2cZoiL7wKHV9UfTXX9qvoK419XBg7gLWkaeIpT0shpeq+2AnsNOhZJGo89aJJG1dvpnI78KzoX8f/irsqqum5gUUkSJmiSRkySR1fVLcDfN0XPY/vTlQU4yr+kgfIUp6RR812AqtqtqnYD1m6bbl4mZ5IGzgRN0qgZe3H/cwYShSTtgAmapFEzdgT/ie7GlKSB8Ro0SaNm9yS/wQOJ2Zwx81TVFwYSmSQ1TNAkjZrbgPd3zd8+Zr6Ax8xoRJI0hgmapJFSVYsGHYMkTcYETTNi0YrzBx2CJElDw5sEJEmSWsYETZIkqWVM0CRJklrGBE2SJKllJk3QkhyS5ItJNiS5MsnpTfn+SS5Mcm3zvl/XOmck2ZjkmiTH9vMDSJIkzTa99KDdB7yuqp4APAM4LcnhwApgXVUtBtY18zTLlgFHAMcBZyfx2XaSJEk9mjRBq6rNVfXNZvouYAMwH1gKrG6qrQZOaqaXAudW1b1VdT2wEThqugOXJEmaraZ0DVqSRcBTgEuAg6pqM3SSOODAptp84Kau1TY1ZZIkSepBzwlakn2ATwGvraqf7KjqOGVjH05MkuVJ1idZv2XLll7DkCRJmvV6StCS7EEnOftIVf1jU3xrknnN8nl0nm8HnR6zQ7pWXwDcPHabVXVOVS2pqiVz587d2fglSZJmnV7u4gzwPmBDVb2ta9Fa4JRm+hTgvK7yZUn2THIYsBi4dPpCliRJmt16eRbnM4GXA1ckubwpOxNYBaxJcipwI3AyQFVdmWQNcBWdO0BPq6qt0x65JEnSLDVpglZVX2H868oAjplgnZXAyl2IS5IkaWT5JAFJkqSWMUGTJElqmV6uQZMkSWqlRSvO327+hlUnDiiS6WUPmiRJUsuYoEmSJLWMCZokSVLLeA2aJEmaUWOvG4PZc+3YdDFBkyQNjfH+Y5dmI09xSpIktYwJmiRJUsuYoEmSJLWM16BJkqRZY7Zcp2iCJklTlOT9wAuA26rqiU3Z/sDHgUXADcBLquqOZtkZwKnAVuA1VfX5AYQ9o2br6O7STPEUpyRN3QeB48aUrQDWVdViYF0zT5LDgWXAEc06ZyeZM3OhShpGJmiSNEVV9WXgR2OKlwKrm+nVwEld5edW1b1VdT2wEThqRgKVNLRM0CRpehxUVZsBmvcDm/L5wE1d9TY1ZQ+SZHmS9UnWb9mypa/BSmo3r0GTpP7KOGU1XsWqOgc4B2DJkiXj1pGG0Wy5cH8m2YMmSdPj1iTzAJr325ryTcAhXfUWADfPcGyShow9aJI0PdYCpwCrmvfzuso/muRtwMHAYuDSgUQo7YDPx2wXEzRJmqIkHwOOBg5Isgl4E53EbE2SU4EbgZMBqurKJGuAq4D7gNOqautAApc0NEzQJGmKquplEyw6ZoL6K4GV/YtI0mzjNWiSJEktY4ImSZLUMpMmaEnen+S2JN/pKts/yYVJrm3e9+tadkaSjUmuSXJsvwKXJEmarXq5Bu2DwN8DH+oq2/ZIk1VJVjTzbxjzSJODgX9N8lgviJUkSVMx6mOnTdqD5iNNJEmSZtbO3sW53SNNknQ/0uTrXfUmfKSJJElqt7G9WI6LNnOme5iNnh9pkmQ5sBxg4cKF0xyGNLocbFKSht/OJmi3JpnX9J7t1CNNfOacJEkzw56w4bOzw2xse6QJPPiRJsuS7JnkMHykiSRJ0pRN2oPmI02Gk7+WJEkaXpMmaD7SRJIkaWb5LE5JklrKsyGjywRNkiSNvLYlwyZokqS+c/iX2cG/48zxYemSJEktY4ImSZLUMiZokiRJLWOCJkmS1DLeJCBJGkltu2tP6maCJklSj7yLUTPFBE2SNCX2PEn9Z4ImSWqt8XqspFFggiZJUp95anRyJuPbM0GTJInRShBG6bMOKxO0IeOvMEn9MtPti9eySRMzQRsRJnaS2m5Ye3WmK9Gcrs8/08dxWP9ubWeCJknSAJjYaEd8koAkSVLL2IMmSSPKHhxpYoO+NMgEbRawkZUkaXYxQZOGyKB/0UmanD+aNR1M0KQp6GeCtLONuv8ZaND8NzhYHv/ZyQRNkqQhYTI2OkzQpAGwkZUk7UjfErQkxwHvAOYA762qVf3a13gcobo/Ru0aqF4SqWFMtkbt79gGg24TJQ2XviRoSeYA7wR+E9gEfCPJ2qq6qh/7U/+MUvIxjJ+1V/0aodykrjfD3CbO5u+FNFUz2Qb2qwftKGBjVV0HkORcYCkwLY3Rzhyg6ewx6KXBGrvtNvZYDMN/2sP66JNhZPLVV31tE3vhd0AaLv1K0OYDN3XNbwKe3qd9tfLut9l6aqwXbfx7aOr8e0yrvraJ/q2kwehn50u/ErSMU1bbVUiWA8ub2buTXNOnWCZyAPDDGd7njrQtHmhfTMazYzMeT94yaZWxMR3at2DabdI2EWakXWzbv1loX0xtiwfaF1Pb4oEWxdS0i73GM2Gb2K8EbRNwSNf8AuDm7gpVdQ5wTp/2P6kk66tqyaD2P1bb4oH2xWQ8O9a2eKCdMQ3IpG0i9L9dbOPfo20xtS0eaF9MbYsH2hfTdMTTr4elfwNYnOSwJA8BlgFr+7QvSWo720RJU9KXHrSqui/Jq4HP07ml/P1VdWU/9iVJbWebKGmq+jYOWlV9Fvhsv7Y/DQZ2enUCbYsH2heT8exY2+KBdsY0EC1pE9v492hbTG2LB9oXU9vigfbFtMvxpOpB16lKkiRpgPp1DZokSZJ2kgmaJElSy5igSZIktYwJmiRJUsuMRIKW5Liu6UcmeV+Sbyf5aJKDBhDPI5OsSnJ1ktub14ambN8BxNOq49MVS5I8PcmLk/xWMz3eiOwzLsk+SZ46iL9XVwytPT7QjmM0ytr4vbbt6zkuv9uTxzDrj9FIJGjAX3VNvxXYDLyQzuCR/28A8awB7gCOrqpHVdWjgN9oyj4xgHjadnxI8nzgWuB/AScAJwJnAdc2y2Y6nrO7pp9F5yHXbwWuSHLCAOJp1fFpYmrVMVL7vtfY9k3K73ZP8YzGMaqqWf8Cvtk1ffmYZZfPZCzNPq/ZmWWjcnya/W4AFo1TfhiwYcDH6IvAU5vpxwDrR/34tPEYjfqrpd9r277JY/K77TGiqvo3UG3LHJjkf9B5YPEjkqSaI8dgehG/n+T1wOqquhWg6U5/JXDTAOJp2/GBziDKm8Yp/wGwxwzHMtYjquqbAFV1XZI5A4ihzccH2nGMRl0bv9e2fZPzuz25kThGo5KgvQd4eDO9ms5T5rckeTRw+QDieSmwAvhS0zgVcCudZ/O9ZADxtO34ALwf+EaSc3mg4T6EzjMM3zeAeB6f5Nt0GvJFSfarqjuS7MZgGoS2HR9o3zEadW38Xtv2Tc7v9uRG4hiNzJMEkjwemA9cUlV3d5UfV1WfG1xkkOTZwFHAFVV1wQD2/xrg01U1iF+wE0pyOPAiOn+30PnFtLaqrhpALIeOKbq5qv4zyQHAr1fVPw4gpicAS2nB8Wniad0xGnVtbveaOGz7xtGmtq+Jp3Xf7VFo/0YiQUvyx8Cr6Zy3PhI4varOa5Z9s6qeOsPxXFpVRzXTfwCcBvwT8Hzgn6tq1QzH82PgHuB7wMeAT1TVlpmMQdL0alu71+zXtk/q0ajcxbkceFpVnQQcDfzPJKc3ywZxW253d+cfAs+vqrPoNFK/O4B4rgMWAP8beBpwVZLPJTklycN3vGp/pH234z8iyV8n+YckvzNm2dkTrdfHeMYOD/DeDH4IhW8meWOSXxrE/vUgbWv3wLZvUm1r+5qYbP8mj2na279RSdDmbOver6ob6DRWxyd5G4NpqHZLsl+SR9HpxdzSxHYPcN8A4qmqur+qLqiqU4GDgbOB4+g0YIMw0e34dzKY2/E/QOffyqeAZUk+lWTPZtkzBhDP2OEBbmHwQyjsB+wLfDHJpUn+JMnBA4pF7Wv3wLavF21r+8D2rxfT3/4N4nbUmX4BXwCOHFO2O/AhYOsA4rmBzpf/+ub90U35Pgxm2I9v7WDZ3gP6m7Xtdvyxt+D/OfBV4FF03V49g/G0cXiA7pieTec/ulvo3HK+fBAxjfKrbe1es3/bvsljalXb1+zX9m9qMU1L+zcqd3G+gjG/zqrqPuAVSWY8266qRRMsuh/4rRkMZZuXTrSgqn42k4F0+X7adTv+nkl2q6r7AapqZZJNwJfp/Ocy09o4PMAvVNXFwMXNdVC/Seff2DmDjWrktKrda/a/aIJFtn0PaFvbB7Z/UzJd7d/AP8hMqKpNVXXLBMu+OtPxTKSqflpV1w9gv9+d6X324KV0fp19KcmPkvwIuAjYHzh5APH8M/Dc7oKqWg28Dvj5AOLZNjzAPjwwPAAZ7PAAD/p3VFVbq+pzVfWqQQQ0yoal3QPbvjHa1vaB7V8vpr39G4m7ODW7JHlVVX1g0HFsYzyTa2NM0rBp4/eobTG1LR7Y+ZhM0DR0ktxYVQsHHcc2xjO5NsYkDZs2fo/aFlPb4oGdj2lUrkHTkElnROZxFwEzfhu18UyujTFJw6aN36O2xdS2eKA/MZmgqa0OAo6lc7t5twD/NvPhGE8P2hiTNGza+D1qW0xtiwf6EJMJmtrqM8A+VfWgC9mLMZsAAAPWSURBVD6TXDTz4RhPD9oYkzRs2vg9altMbYsH+hCT16BJkiS1zEgMsyFJkjRMTNAkSZJaxgRNkqQRkOSiJH8w6DjUGxM0SdJIS/KsJP+W5MfN6P1fTfIrzbJXJqnmIfPd65zUlH+wq+zUJFcnuSvJrUnOT/LwCfbZqmQpyaLm83jzYEuYoEmSRlaSR9C5A+/v6DxOaT5wFnBvV7XvAS8dk7y8gq7H+yR5DvBXwMuq6uHAE4A1/Y1es5kJmiRplD0WoKo+1jw78WdVdUFVdQ88egtwBZ1xrkiyP/BrwNquOr8CfK2qvtVs70dVtbqq7ppqQEl+P8mGJHck+XySQ7uWVZL/luTaZvk7k6RZNifJW5P8MMn1SV49Tq/YoU0P4V1JLkhyQFP+5eb9ziR3J/nVqcat6WWCJkkaZd8FtiZZneT4JPtNUO9DdHrNAJYB57F9L9slwLFJzkryzCR77kwwSU4CzgReDMwFLgY+NqbaC+gkhE8GXkKTOAL/FTgeOBJ4KnDSOLv4HeBVwIHAQ4A/bcp/vXnft6r2qaqv7Uz8mj4maJKkkVVVPwGeBRTwHmBLkrVJxj6e59PA0UkeSSdR+9CY7VxMJ6l6KnA+cHuStyWZM8WQ/hD466raUFX30TltemR3LxqwqqrurKobgS/SScigk6y9o6o2VdUdwKpxtv+BqvpuVf2MzinYI8epoxYwQZMkjbQmGXplVS0AnggcDLx9TJ2f0Um83ggcUFVfHWc7/1JVL6RzLdtS4JXAVG8EOBR4R5I7k9wJ/IjO44Lmd9W5pWv6p8A+zfTBwE1dy7qnJ1tXLWOCJklSo6quBj5IJ1Eb60PA64B/mGQb91fVOuALE2xnR24C/rCq9u167V1VvTzPcTOwoGv+kCns18cKtYwJmiRpZCV5fJLXJVnQzB8CvAz4+jjVvwT8Jp07PsduZ2mSZUn2S8dRwHMm2M42uyfZq+u1B/Bu4IwkRzTbfWSSk3v8OGuA05PMT7Iv8IYe1wPYAtwPPGYK66iPTNAkSaPsLuDpwCVJ7qGTUH2HTk/ZdqpjXVX9aJzt3EHnIv1rgZ8AHwb+pqo+soN9vwv4WdfrA1X1aeAtwLlJftLEcnyPn+U9wAXAt4FvAZ8F7gO2TrZiVf0UWAl8tTm9+owe96k+8WHpkiTNQkmOB95dVYdOWlmtYw+aJEmzQJK9k5yQZPck84E30bn7VEPIHjRJkmaBJA+lc53c4+mcMj0fOL0ZSkRDxgRNkiSpZTzFKUmS1DImaJIkSS1jgiZJktQyJmiSJEktY4ImSZLUMiZokiRJLfP/Ad+RDbWQTQR9AAAAAElFTkSuQmCC\n",
      "text/plain": [
       "<Figure size 720x288 with 2 Axes>"
      ]
     },
     "metadata": {
      "needs_background": "light"
     },
     "output_type": "display_data"
    }
   ],
   "source": [
    "# Looking how LENGHT varies according to LABEL\n",
    "train_data.hist(column='LENGHT', by='LABEL', bins=50,figsize=(10,4))\n",
    "plt.xlabel(\"SMS Lenght\",fontsize = 12) \n",
    "plt.ylabel(\"Frequência\", fontsize = 12)"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "It's inconclusive. There is no trend of LABEL in terms of LENGHT."
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 11,
   "metadata": {},
   "outputs": [],
   "source": [
    "#Let's delete column LENGHT because it is unuseful\n",
    "del train_data['LENGHT']"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "## Implementation of Bag of Words Approach (BoW)\n"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "Bag of words approach is a way of extracting features from the text for use in machine learning algorithms.  \n",
    "\n",
    "In this approach, we use the tokenized words for each observation and find out the frequency of each token.   \n",
    "\n",
    "Using this process we will convert a collection of documents to a matrix, with each document being a row and each word(token) being the column, and the corresponding (row,column) values being the frequency of occurrence of each word or token in that document."
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "## Creating corpus"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "First step is to create a corpus, which is a collection of machine-readable texts, a large and structured set of texts.   \n",
    "\n",
    "We import ‘re’ package and remove punctuation, special characters and convert all characters to lower case.   \n",
    "\n",
    "After that, we need to split a message into words to remove stop-words and to perform stemming.   \n",
    "\n",
    "We import Portuguese stop-words from the NLTK package and removed them if found in the sentence.   \n",
    "\n",
    "While removing stop-words, we perform stemming that is if the word is not a stop-word, it will be converted to its root form. This is called stemming.   \n",
    "\n",
    "Then we are joining the words in the list again to form a message without any stop-words and all words will be present in its root form. Then we append messages to the corpus."
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 12,
   "metadata": {},
   "outputs": [],
   "source": [
    "import re\n",
    "import nltk\n",
    "from nltk.corpus import stopwords\n",
    "from nltk.stem.porter import PorterStemmer"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 13,
   "metadata": {},
   "outputs": [
    {
     "data": {
      "text/plain": [
       "'support appl Se requier verifiqu su ID appl visit el siguient enlac detal http icloudfind support hntw sG'"
      ]
     },
     "execution_count": 13,
     "metadata": {},
     "output_type": "execute_result"
    }
   ],
   "source": [
    "# Creating a corpus of refinned messages\n",
    "stemming = PorterStemmer()\n",
    "corpus = []\n",
    "\n",
    "for i in range (0,len(train_data)):\n",
    "    s1 = re.sub('[^a-zA-Z]',repl = ' ',string = train_data['SMS'][i])\n",
    "    s1.lower()\n",
    "    s1 = s1.split()\n",
    "    s1 = [stemming.stem(word) for word in s1 if word not in set(stopwords.words('portuguese'))]\n",
    "    s1 = ' '.join(s1)\n",
    "    corpus.append(s1)\n",
    "\n",
    "corpus[50]"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "## Tokenizing and vectorizing the text"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "We’ll import Count-Vectorizer from Scikit-learn library to tokenize and vectorize the text.   \n",
    "\n",
    "By using this Count-Vectorizer we’ll tokenize a collection of text documents and built a vocabulary, this vocabulary is also used to encode new documents.   \n",
    "\n",
    "To use this Count-Vectorizer, first, we’ll create an instance of Count-Vectorizer class.   \n",
    "\n",
    "Then fit () function is used to learn vocabulary from the document and the transform() function is used to encode as a vector.   \n",
    "\n",
    "This vector represents the length of the entire vocabulary and the count for the number of times each word appeared in the document.   \n",
    "\n",
    "Now we have a numeric vector that has been converted from a string of text."
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 14,
   "metadata": {},
   "outputs": [],
   "source": [
    "from sklearn.feature_extraction.text import CountVectorizer\n",
    "countvectorizer =CountVectorizer()\n"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 15,
   "metadata": {},
   "outputs": [],
   "source": [
    "X = countvectorizer.fit_transform(corpus).toarray()"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 16,
   "metadata": {},
   "outputs": [
    {
     "data": {
      "text/plain": [
       "(6000, 5221)"
      ]
     },
     "execution_count": 16,
     "metadata": {},
     "output_type": "execute_result"
    }
   ],
   "source": [
    "X.shape"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 17,
   "metadata": {},
   "outputs": [
    {
     "data": {
      "text/plain": [
       "['aa',\n",
       " 'aaa',\n",
       " 'aacd',\n",
       " 'aaef',\n",
       " 'aaj',\n",
       " 'aapp',\n",
       " 'aar',\n",
       " 'ab',\n",
       " 'abaixo',\n",
       " 'abatia',\n",
       " 'abb',\n",
       " 'abba',\n",
       " 'abc',\n",
       " 'abcd',\n",
       " 'abdia',\n",
       " 'abdovaldo',\n",
       " 'abdx',\n",
       " 'abel',\n",
       " 'abelardo',\n",
       " 'abelmar',\n",
       " 'aberto',\n",
       " 'abertura',\n",
       " 'abigaelli',\n",
       " 'abilio',\n",
       " 'abimael',\n",
       " 'abinadabia',\n",
       " 'abnb',\n",
       " 'abner',\n",
       " 'about',\n",
       " 'abraao',\n",
       " 'abreu',\n",
       " 'abrir',\n",
       " 'absalao',\n",
       " 'ac',\n",
       " 'academi',\n",
       " 'acassio',\n",
       " 'acc',\n",
       " 'accacio',\n",
       " 'accamargo',\n",
       " 'accentur',\n",
       " 'access',\n",
       " 'accesslcloud',\n",
       " 'account',\n",
       " 'acdigit',\n",
       " 'acdigita',\n",
       " 'ace',\n",
       " 'acelino',\n",
       " 'acesa',\n",
       " 'acesqjz',\n",
       " 'acess',\n",
       " 'acessando',\n",
       " 'acessar',\n",
       " 'acesso',\n",
       " 'acessoseguro',\n",
       " 'acion',\n",
       " 'ack',\n",
       " 'acknowledg',\n",
       " 'ackv',\n",
       " 'acl',\n",
       " 'acmadvocacia',\n",
       " 'acompanh',\n",
       " 'acompanhado',\n",
       " 'acompanhei',\n",
       " 'acordo',\n",
       " 'across',\n",
       " 'act',\n",
       " 'action',\n",
       " 'activ',\n",
       " 'actual',\n",
       " 'ad',\n",
       " 'ada',\n",
       " 'adaia',\n",
       " 'adailson',\n",
       " 'adailton',\n",
       " 'adalberon',\n",
       " 'adalberto',\n",
       " 'adalgisa',\n",
       " 'adalicio',\n",
       " 'adalton',\n",
       " 'adam',\n",
       " 'adani',\n",
       " 'adao',\n",
       " 'adaofaraco',\n",
       " 'adar',\n",
       " 'adason',\n",
       " 'adauto',\n",
       " 'adbor',\n",
       " 'addan',\n",
       " 'addit',\n",
       " 'addmark',\n",
       " 'address',\n",
       " 'adeild',\n",
       " 'adeildo',\n",
       " 'adelaid',\n",
       " 'adelia',\n",
       " 'adelino',\n",
       " 'adelma',\n",
       " 'adelmo',\n",
       " 'adelson',\n",
       " 'ademar',\n",
       " 'ademario',\n",
       " 'ademart',\n",
       " 'ademilson',\n",
       " 'ademir',\n",
       " 'ademyr',\n",
       " 'adi',\n",
       " 'adiana',\n",
       " 'adicionado',\n",
       " 'adicionai',\n",
       " 'adiel',\n",
       " 'adielson',\n",
       " 'adilio',\n",
       " 'adilson',\n",
       " 'adilur',\n",
       " 'adilvanir',\n",
       " 'adiv',\n",
       " 'adivania',\n",
       " 'adminip',\n",
       " 'adolfo',\n",
       " 'adolpho',\n",
       " 'adonia',\n",
       " 'adorani',\n",
       " 'adriana',\n",
       " 'adriann',\n",
       " 'adriano',\n",
       " 'adriel',\n",
       " 'adson',\n",
       " 'adv',\n",
       " 'advogado',\n",
       " 'ae',\n",
       " 'aeksqqvl',\n",
       " 'aeq',\n",
       " 'aeqteo',\n",
       " 'aeufr',\n",
       " 'af',\n",
       " 'afonso',\n",
       " 'afranio',\n",
       " 'aft',\n",
       " 'ag',\n",
       " 'again',\n",
       " 'agar',\n",
       " 'agda',\n",
       " 'age',\n",
       " 'agencia',\n",
       " 'agend',\n",
       " 'agendada',\n",
       " 'agendado',\n",
       " 'agendamento',\n",
       " 'agendar',\n",
       " 'agent',\n",
       " 'agentinfo',\n",
       " 'agnaldo',\n",
       " 'agnato',\n",
       " 'ago',\n",
       " 'agora',\n",
       " 'agostinho',\n",
       " 'agostinopasta',\n",
       " 'agradec',\n",
       " 'agrestina',\n",
       " 'agua',\n",
       " 'aguarda',\n",
       " 'aguardamo',\n",
       " 'aguida',\n",
       " 'aguinaldo',\n",
       " 'aguini',\n",
       " 'ah',\n",
       " 'ahava',\n",
       " 'ahdcfa',\n",
       " 'ahgfmw',\n",
       " 'ahx',\n",
       " 'ailon',\n",
       " 'ailton',\n",
       " 'ainda',\n",
       " 'air',\n",
       " 'aira',\n",
       " 'airbnb',\n",
       " 'aitaon',\n",
       " 'aj',\n",
       " 'ajd',\n",
       " 'ajxkeil',\n",
       " 'ajz',\n",
       " 'akadlrxb',\n",
       " 'al',\n",
       " 'alair',\n",
       " 'alan',\n",
       " 'alaor',\n",
       " 'alarm',\n",
       " 'alarmando',\n",
       " 'alauy',\n",
       " 'albertino',\n",
       " 'alberto',\n",
       " 'alcantara',\n",
       " 'alcebiad',\n",
       " 'alcid',\n",
       " 'alcimar',\n",
       " 'alcindo',\n",
       " 'alcion',\n",
       " 'aldenic',\n",
       " 'aldenira',\n",
       " 'aldlem',\n",
       " 'aleandro',\n",
       " 'alecsandra',\n",
       " 'alef',\n",
       " 'alegr',\n",
       " 'alem',\n",
       " 'alert',\n",
       " 'alerta',\n",
       " 'alessand',\n",
       " 'alessandra',\n",
       " 'alessandro',\n",
       " 'alex',\n",
       " 'alexandr',\n",
       " 'alexandro',\n",
       " 'alexia',\n",
       " 'alexsand',\n",
       " 'alexsandra',\n",
       " 'alexsandro',\n",
       " 'alfa',\n",
       " 'alfena',\n",
       " 'alfredo',\n",
       " 'algar',\n",
       " 'algo',\n",
       " 'algum',\n",
       " 'ali',\n",
       " 'alic',\n",
       " 'alin',\n",
       " 'alirio',\n",
       " 'alisson',\n",
       " 'aliv',\n",
       " 'aliza',\n",
       " 'all',\n",
       " 'allana',\n",
       " 'allcom',\n",
       " 'allef',\n",
       " 'allgvt',\n",
       " 'allison',\n",
       " 'almap',\n",
       " 'almeida',\n",
       " 'almirant',\n",
       " 'aloisio',\n",
       " 'alreadi',\n",
       " 'alricelia',\n",
       " 'alt',\n",
       " 'alta',\n",
       " 'altair',\n",
       " 'altamira',\n",
       " 'alteramo',\n",
       " 'alto',\n",
       " 'alu',\n",
       " 'aluizio',\n",
       " 'alv',\n",
       " 'alvaro',\n",
       " 'alvonir',\n",
       " 'alvorada',\n",
       " 'alwycb',\n",
       " 'am',\n",
       " 'amaia',\n",
       " 'amanda',\n",
       " 'amanh',\n",
       " 'amanha',\n",
       " 'amarelo',\n",
       " 'amarildo',\n",
       " 'amaro',\n",
       " 'amauri',\n",
       " 'amazon',\n",
       " 'amazonaw',\n",
       " 'amdoc',\n",
       " 'america',\n",
       " 'americana',\n",
       " 'americo',\n",
       " 'amilson',\n",
       " 'amilton',\n",
       " 'amparo',\n",
       " 'amper',\n",
       " 'amsterdam',\n",
       " 'amt',\n",
       " 'amwu',\n",
       " 'amz',\n",
       " 'an',\n",
       " 'ana',\n",
       " 'anadi',\n",
       " 'anajatuba',\n",
       " 'analis',\n",
       " 'ananindeua',\n",
       " 'anayara',\n",
       " 'anayra',\n",
       " 'and',\n",
       " 'andamento',\n",
       " 'anderson',\n",
       " 'andr',\n",
       " 'andrad',\n",
       " 'andrada',\n",
       " 'andrea',\n",
       " 'andreia',\n",
       " 'andrev',\n",
       " 'andrew',\n",
       " 'android',\n",
       " 'ane',\n",
       " 'anel',\n",
       " 'angel',\n",
       " 'angela',\n",
       " 'angelic',\n",
       " 'angelica',\n",
       " 'angelo',\n",
       " 'angra',\n",
       " 'ani',\n",
       " 'ann',\n",
       " 'anna',\n",
       " 'ano',\n",
       " 'anorm',\n",
       " 'anselmo',\n",
       " 'anterior',\n",
       " 'antero',\n",
       " 'anton',\n",
       " 'antonia',\n",
       " 'antonieta',\n",
       " 'antonio',\n",
       " 'anual',\n",
       " 'anuidad',\n",
       " 'anuidadefre',\n",
       " 'anuidadelivr',\n",
       " 'anytim',\n",
       " 'ao',\n",
       " 'ap',\n",
       " 'apacc',\n",
       " 'aparecida',\n",
       " 'aparecido',\n",
       " 'aparelho',\n",
       " 'apcwwyrf',\n",
       " 'apena',\n",
       " 'api',\n",
       " 'apjmjo',\n",
       " 'apl',\n",
       " 'aplicativo',\n",
       " 'apo',\n",
       " 'aposentado',\n",
       " 'app',\n",
       " 'appacesso',\n",
       " 'appbb',\n",
       " 'appbbdigit',\n",
       " 'appbrasil',\n",
       " 'appcc',\n",
       " 'appestilo',\n",
       " 'appi',\n",
       " 'appiecar',\n",
       " 'appieinc',\n",
       " 'appl',\n",
       " 'appleid',\n",
       " 'appleidrastreio',\n",
       " 'appleidsupport',\n",
       " 'applesuport',\n",
       " 'applesupport',\n",
       " 'applewhatch',\n",
       " 'appuni',\n",
       " 'apr',\n",
       " 'apresenta',\n",
       " 'aprovada',\n",
       " 'aprovado',\n",
       " 'aproveit',\n",
       " 'apztgp',\n",
       " 'aq',\n",
       " 'aqpax',\n",
       " 'aqrtdf',\n",
       " 'aquiraz',\n",
       " 'ar',\n",
       " 'aracaju',\n",
       " 'araguaina',\n",
       " 'araguari',\n",
       " 'arajyh',\n",
       " 'aranha',\n",
       " 'arapiraca',\n",
       " 'araponga',\n",
       " 'arara',\n",
       " 'ararangu',\n",
       " 'ararangua',\n",
       " 'araraquara',\n",
       " 'arax',\n",
       " 'araxa',\n",
       " 'arcoverd',\n",
       " 'are',\n",
       " 'areacaixa',\n",
       " 'areia',\n",
       " 'arez',\n",
       " 'arg',\n",
       " 'argemiro',\n",
       " 'ariagno',\n",
       " 'arino',\n",
       " 'arismar',\n",
       " 'aritusa',\n",
       " 'arleid',\n",
       " 'armacao',\n",
       " 'arnaldo',\n",
       " 'arno',\n",
       " 'aroga',\n",
       " 'arozii',\n",
       " 'arpf',\n",
       " 'arpfr',\n",
       " 'arqueiro',\n",
       " 'arroio',\n",
       " 'arssgpweb',\n",
       " 'art',\n",
       " 'arthur',\n",
       " 'artur',\n",
       " 'aryton',\n",
       " 'as',\n",
       " 'asahi',\n",
       " 'asap',\n",
       " 'asem',\n",
       " 'ashburn',\n",
       " 'asia',\n",
       " 'asiq',\n",
       " 'asr',\n",
       " 'assai',\n",
       " 'assi',\n",
       " 'assinada',\n",
       " 'assinatura',\n",
       " 'assist',\n",
       " 'associ',\n",
       " 'associado',\n",
       " 'assunto',\n",
       " 'asterisk',\n",
       " 'astpd',\n",
       " 'at',\n",
       " 'atalaia',\n",
       " 'atbr',\n",
       " 'ate',\n",
       " 'atella',\n",
       " 'atencao',\n",
       " 'atenciosament',\n",
       " 'atendimento',\n",
       " 'atendimentobr',\n",
       " 'athayd',\n",
       " 'atibaia',\n",
       " 'ativ',\n",
       " 'ativado',\n",
       " 'ativar',\n",
       " 'ativo',\n",
       " 'ato',\n",
       " 'atraso',\n",
       " 'atrav',\n",
       " 'atribuimo',\n",
       " 'atrium',\n",
       " 'att',\n",
       " 'attempt',\n",
       " 'attend',\n",
       " 'attent',\n",
       " 'atual',\n",
       " 'atualiza',\n",
       " 'atualizabb',\n",
       " 'atualizacao',\n",
       " 'atualizacaored',\n",
       " 'atualizaco',\n",
       " 'atualizar',\n",
       " 'atualizarmobil',\n",
       " 'atualizebbacesso',\n",
       " 'au',\n",
       " 'aug',\n",
       " 'augusta',\n",
       " 'aujpdwt',\n",
       " 'aurelian',\n",
       " 'ausent',\n",
       " 'aut',\n",
       " 'auth',\n",
       " 'authent',\n",
       " 'author',\n",
       " 'auto',\n",
       " 'automotiva',\n",
       " 'autoridad',\n",
       " 'autoriza',\n",
       " 'autorizado',\n",
       " 'autosi',\n",
       " 'auwypiz',\n",
       " 'av',\n",
       " 'avaliaco',\n",
       " 'avar',\n",
       " 'avaya',\n",
       " 'avec',\n",
       " 'aviso',\n",
       " 'avisobbsm',\n",
       " 'avon',\n",
       " 'away',\n",
       " 'ax',\n",
       " 'axa',\n",
       " 'axh',\n",
       " 'ay',\n",
       " 'azhxwr',\n",
       " 'azlfbkp',\n",
       " 'azul',\n",
       " 'ba',\n",
       " 'bacalhau',\n",
       " 'backbon',\n",
       " 'backoffic',\n",
       " 'backup',\n",
       " 'bad',\n",
       " 'badger',\n",
       " 'bae',\n",
       " 'baia',\n",
       " 'bairro',\n",
       " 'baix',\n",
       " 'balsa',\n",
       " 'bambui',\n",
       " 'banco',\n",
       " 'bancobrasil',\n",
       " 'bancodobrasil',\n",
       " 'bancodobrasilapp',\n",
       " 'bandeirant',\n",
       " 'bang',\n",
       " 'bangalor',\n",
       " 'bangkok',\n",
       " 'bangu',\n",
       " 'banknet',\n",
       " 'barbara',\n",
       " 'barcelona',\n",
       " 'barra',\n",
       " 'barreira',\n",
       " 'barreiro',\n",
       " 'barreto',\n",
       " 'barri',\n",
       " 'barril',\n",
       " 'barroso',\n",
       " 'barueri',\n",
       " 'base',\n",
       " 'basebox',\n",
       " 'basilia',\n",
       " 'bassem',\n",
       " 'basto',\n",
       " 'batista',\n",
       " 'baussmkcx',\n",
       " 'baw',\n",
       " 'bazar',\n",
       " 'bazelgeus',\n",
       " 'bb',\n",
       " 'bbapp',\n",
       " 'bbatualiazacao',\n",
       " 'bbatualiza',\n",
       " 'bbclient',\n",
       " 'bbdigit',\n",
       " 'bbestilo',\n",
       " 'bbinforma',\n",
       " 'bbrasildesbloqueio',\n",
       " 'bbresgat',\n",
       " 'bbsmile',\n",
       " 'bbsmilesgol',\n",
       " 'bc',\n",
       " 'bcae',\n",
       " 'bcgbm',\n",
       " 'bczziapi',\n",
       " 'bd',\n",
       " 'bde',\n",
       " 'bdkc',\n",
       " 'bdpx',\n",
       " 'be',\n",
       " 'bear',\n",
       " 'beatriz',\n",
       " 'beauharnoi',\n",
       " 'beberib',\n",
       " 'bebo',\n",
       " 'been',\n",
       " 'befor',\n",
       " 'behv',\n",
       " 'beij',\n",
       " 'bel',\n",
       " 'bela',\n",
       " 'belabiju',\n",
       " 'belford',\n",
       " 'beljevac',\n",
       " 'belo',\n",
       " 'below',\n",
       " 'bem',\n",
       " 'benedito',\n",
       " 'bennett',\n",
       " 'benoitindisponivel',\n",
       " 'benson',\n",
       " 'bento',\n",
       " 'bepay',\n",
       " 'ber',\n",
       " 'bergen',\n",
       " 'bernardi',\n",
       " 'bertioga',\n",
       " 'bet',\n",
       " 'betim',\n",
       " 'bexddjw',\n",
       " 'bf',\n",
       " 'bff',\n",
       " 'bflijwtv',\n",
       " 'bfmlq',\n",
       " 'bg',\n",
       " 'bgp',\n",
       " 'bgpsession',\n",
       " 'bh',\n",
       " 'bhft',\n",
       " 'bhm',\n",
       " 'bhopal',\n",
       " 'bi',\n",
       " 'bianca',\n",
       " 'bicanga',\n",
       " 'bid',\n",
       " 'big',\n",
       " 'bilac',\n",
       " 'bin',\n",
       " 'binatur',\n",
       " 'biosa',\n",
       " 'bird',\n",
       " 'birigui',\n",
       " 'biritiba',\n",
       " 'biscuit',\n",
       " 'bismarck',\n",
       " 'bit',\n",
       " 'bitcola',\n",
       " 'bitli',\n",
       " 'bitnuvem',\n",
       " 'bito',\n",
       " 'bituruna',\n",
       " 'bivhz',\n",
       " 'bix',\n",
       " 'biz',\n",
       " 'bizlr',\n",
       " 'bjvgpkkpu',\n",
       " 'bkhyjm',\n",
       " 'bkp',\n",
       " 'bksxu',\n",
       " 'bkwqxqvc',\n",
       " 'bkywpttf',\n",
       " 'bl',\n",
       " 'black',\n",
       " 'blk',\n",
       " 'blletrft',\n",
       " 'block',\n",
       " 'bloq',\n",
       " 'bloque',\n",
       " 'bloqueada',\n",
       " 'bloqueado',\n",
       " 'bloqueadoprocur',\n",
       " 'bloqueamento',\n",
       " 'bloquear',\n",
       " 'bloqueio',\n",
       " 'blue',\n",
       " 'blumenau',\n",
       " 'blustelecom',\n",
       " 'bmanagem',\n",
       " 'bme',\n",
       " 'bmi',\n",
       " 'bo',\n",
       " 'boa',\n",
       " 'boleto',\n",
       " 'bom',\n",
       " 'bommtempo',\n",
       " 'bonatto',\n",
       " 'bonfim',\n",
       " 'bonfinopoli',\n",
       " 'bonifacio',\n",
       " 'bonito',\n",
       " 'bonjour',\n",
       " 'book',\n",
       " 'boquim',\n",
       " 'boqwzgkp',\n",
       " 'borda',\n",
       " 'borg',\n",
       " 'bot',\n",
       " 'botafogo',\n",
       " 'botfpwuq',\n",
       " 'bottcher',\n",
       " 'bp',\n",
       " 'bpcpbhr',\n",
       " 'bpr',\n",
       " 'bq',\n",
       " 'bqapsogf',\n",
       " 'bqgyz',\n",
       " 'bqi',\n",
       " 'br',\n",
       " 'bra',\n",
       " 'braco',\n",
       " 'bradescard',\n",
       " 'bradesco',\n",
       " 'brallcom',\n",
       " 'branc',\n",
       " 'branca',\n",
       " 'branco',\n",
       " 'brasi',\n",
       " 'brasil',\n",
       " 'brasilia',\n",
       " 'brasilit',\n",
       " 'brasilprepago',\n",
       " 'brasilusa',\n",
       " 'bravo',\n",
       " 'braz',\n",
       " 'brazil',\n",
       " 'brazlandia',\n",
       " 'brb',\n",
       " 'brc',\n",
       " 'brcgi',\n",
       " 'brdol',\n",
       " 'brejo',\n",
       " 'brenda',\n",
       " 'breve',\n",
       " 'brf',\n",
       " 'brian',\n",
       " 'brivalda',\n",
       " 'brnetindisponivel',\n",
       " 'brodowski',\n",
       " 'brota',\n",
       " 'browser',\n",
       " 'brscltotvsdb',\n",
       " 'brsp',\n",
       " 'brspesocweb',\n",
       " 'brspesocwebdev',\n",
       " 'brspitop',\n",
       " 'brspladb',\n",
       " 'brspladev',\n",
       " 'brsplawa',\n",
       " 'brspnfeapp',\n",
       " 'brspnfedb',\n",
       " 'brspnfedev',\n",
       " 'brsprmccdb',\n",
       " 'brsprouter',\n",
       " 'brspsync',\n",
       " 'brsptt',\n",
       " 'brst',\n",
       " 'brt',\n",
       " 'brtssdb',\n",
       " 'brum',\n",
       " 'brumadinho',\n",
       " 'bruna',\n",
       " 'bruno',\n",
       " 'brunoalv',\n",
       " 'brusqu',\n",
       " 'brv',\n",
       " 'brwm',\n",
       " 'brwmsbld',\n",
       " 'brwmsnim',\n",
       " 'brwmswasdev',\n",
       " 'bs',\n",
       " 'bsp',\n",
       " 'bss',\n",
       " 'bt',\n",
       " 'btc',\n",
       " 'bu',\n",
       " 'bueno',\n",
       " 'bugr',\n",
       " 'bula',\n",
       " 'bullet',\n",
       " 'bumm',\n",
       " 'buq',\n",
       " 'buritama',\n",
       " 'buriti',\n",
       " 'buriticupu',\n",
       " 'busca',\n",
       " 'buscando',\n",
       " 'buscar',\n",
       " 'buscarmeudispositivo',\n",
       " 'butwxf',\n",
       " 'bv',\n",
       " 'bveb',\n",
       " 'bw',\n",
       " 'bwj',\n",
       " 'bwqt',\n",
       " 'bxco',\n",
       " 'bxjsk',\n",
       " 'by',\n",
       " 'byxjpbrxg',\n",
       " 'byymg',\n",
       " 'bz',\n",
       " 'bzmpla',\n",
       " 'bztpphj',\n",
       " 'ca',\n",
       " 'caba',\n",
       " 'cabaceira',\n",
       " 'cabo',\n",
       " 'cabonnet',\n",
       " 'cacaushow',\n",
       " 'cach',\n",
       " 'cachoeira',\n",
       " 'cacoal',\n",
       " 'cadastr',\n",
       " 'cadastrado',\n",
       " 'cadastrai',\n",
       " 'cadastramento',\n",
       " 'cadastramo',\n",
       " 'cadastro',\n",
       " 'caeihhq',\n",
       " 'caeta',\n",
       " 'caetan',\n",
       " 'caetano',\n",
       " 'cafelandia',\n",
       " 'caico',\n",
       " 'caieira',\n",
       " 'caina',\n",
       " 'caio',\n",
       " 'caixa',\n",
       " 'caixaeconomica',\n",
       " 'caixafinanceira',\n",
       " 'calda',\n",
       " 'call',\n",
       " 'callebaut',\n",
       " 'callin',\n",
       " 'calliop',\n",
       " 'camapua',\n",
       " 'cambara',\n",
       " 'camboriu',\n",
       " 'camburi',\n",
       " 'cameta',\n",
       " 'camila',\n",
       " 'camisa',\n",
       " 'campanha',\n",
       " 'campina',\n",
       " 'campo',\n",
       " 'can',\n",
       " 'cana',\n",
       " 'canaa',\n",
       " 'canal',\n",
       " 'cananeia',\n",
       " 'canavieira',\n",
       " 'cancelamento',\n",
       " 'cancelar',\n",
       " 'cancun',\n",
       " 'candidat',\n",
       " 'candido',\n",
       " 'canjea',\n",
       " 'cannot',\n",
       " 'canoa',\n",
       " 'canta',\n",
       " 'capao',\n",
       " 'capaz',\n",
       " 'capelinha',\n",
       " 'capim',\n",
       " 'capit',\n",
       " 'capitao',\n",
       " 'capivari',\n",
       " 'caq',\n",
       " 'car',\n",
       " 'cara',\n",
       " 'carapebu',\n",
       " 'carapicuiba',\n",
       " 'carat',\n",
       " 'card',\n",
       " 'cardoso',\n",
       " 'caresupport',\n",
       " 'cariacica',\n",
       " 'carin',\n",
       " 'carla',\n",
       " 'carlo',\n",
       " 'carlopoli',\n",
       " 'carmelo',\n",
       " 'carmem',\n",
       " 'carmen',\n",
       " 'carmo',\n",
       " 'carmopoli',\n",
       " 'carnav',\n",
       " 'caro',\n",
       " 'carol',\n",
       " 'carolalmeidagaldino',\n",
       " 'carolin',\n",
       " 'carolina',\n",
       " 'carpina',\n",
       " 'cart',\n",
       " 'cartao',\n",
       " 'carteira',\n",
       " 'caruaru',\n",
       " 'casa',\n",
       " 'casasbahia',\n",
       " 'casca',\n",
       " 'cascalheira',\n",
       " 'cascavel',\n",
       " 'case',\n",
       " 'caso',\n",
       " 'cassia',\n",
       " 'cassio',\n",
       " 'castanh',\n",
       " 'castro',\n",
       " 'catal',\n",
       " 'catalyst',\n",
       " 'catanduva',\n",
       " 'catch',\n",
       " 'cattan',\n",
       " 'catu',\n",
       " 'caucaia',\n",
       " 'cav',\n",
       " 'cavalcant',\n",
       " 'caxia',\n",
       " 'cb',\n",
       " 'cbun',\n",
       " 'cc',\n",
       " 'ccc',\n",
       " 'ccj',\n",
       " 'cclient',\n",
       " 'ccr',\n",
       " 'ccul',\n",
       " 'cd',\n",
       " 'cdc',\n",
       " 'cdevelay',\n",
       " 'cdn',\n",
       " 'cdo',\n",
       " 'cdyf',\n",
       " 'ce',\n",
       " 'ceap',\n",
       " 'cec',\n",
       " 'cedral',\n",
       " 'cedro',\n",
       " 'ceeduc',\n",
       " 'cef',\n",
       " 'cefmobisac',\n",
       " 'ceilandia',\n",
       " 'cek',\n",
       " 'cel',\n",
       " 'celi',\n",
       " 'celia',\n",
       " 'celina',\n",
       " 'celso',\n",
       " 'celular',\n",
       " 'cemaf',\n",
       " 'cemec',\n",
       " 'cemporcentosk',\n",
       " 'center',\n",
       " 'centerserv',\n",
       " 'central',\n",
       " 'centreon',\n",
       " 'centro',\n",
       " 'cerquilho',\n",
       " 'cerro',\n",
       " 'certif',\n",
       " 'certo',\n",
       " 'cesar',\n",
       " 'cess',\n",
       " 'cesta',\n",
       " 'ceu',\n",
       " 'cf',\n",
       " 'cfh',\n",
       " 'cfibbc',\n",
       " 'cfstzr',\n",
       " 'cg',\n",
       " 'cge',\n",
       " 'cgp',\n",
       " 'cgsantanna',\n",
       " 'ch',\n",
       " 'chama',\n",
       " 'chamada',\n",
       " 'chang',\n",
       " 'chapeco',\n",
       " 'charl',\n",
       " 'chave',\n",
       " 'che',\n",
       " 'check',\n",
       " 'chegada',\n",
       " 'chegar',\n",
       " 'chegou',\n",
       " 'chengdu',\n",
       " 'chennai',\n",
       " 'chequ',\n",
       " 'chiayi',\n",
       " 'chicago',\n",
       " 'chqbbmk',\n",
       " 'christiano',\n",
       " 'chrome',\n",
       " 'chsxz',\n",
       " 'chu',\n",
       " 'ci',\n",
       " 'cianort',\n",
       " 'cibel',\n",
       " 'cicera',\n",
       " 'cicero',\n",
       " 'cidad',\n",
       " 'cidreira',\n",
       " 'cient',\n",
       " 'cij',\n",
       " 'cincinnati',\n",
       " 'cinthia',\n",
       " 'cintia',\n",
       " 'cinza',\n",
       " 'cio',\n",
       " 'cipo',\n",
       " 'cirurgia',\n",
       " 'cisco',\n",
       " 'citi',\n",
       " 'citibank',\n",
       " 'citifon',\n",
       " 'cj',\n",
       " 'cjg',\n",
       " 'cjn',\n",
       " 'cjywqr',\n",
       " 'ck',\n",
       " 'cket',\n",
       " 'ckk',\n",
       " 'cl',\n",
       " 'claim',\n",
       " 'clara',\n",
       " 'claric',\n",
       " 'claro',\n",
       " 'claudeci',\n",
       " 'claudemir',\n",
       " 'claudenic',\n",
       " 'claudet',\n",
       " 'claudia',\n",
       " 'claudilen',\n",
       " 'claudio',\n",
       " 'claudiria',\n",
       " 'claudivania',\n",
       " 'clayton',\n",
       " 'cldl',\n",
       " 'cleartext',\n",
       " 'cleber',\n",
       " ...]"
      ]
     },
     "execution_count": 17,
     "metadata": {},
     "output_type": "execute_result"
    }
   ],
   "source": [
    "# Assigning all the words in an object\n",
    "# This feature_cols with be used further in the validation_data vectorizing process\n",
    "feature_cols = countvectorizer.get_feature_names()\n",
    "feature_cols"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 18,
   "metadata": {},
   "outputs": [
    {
     "name": "stdout",
     "output_type": "stream",
     "text": [
      "['blocked' 'blocked' 'blocked' ... 'ok' 'ok' 'ok']\n"
     ]
    }
   ],
   "source": [
    "y = train_data['LABEL'].values\n",
    "print(y)"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 19,
   "metadata": {},
   "outputs": [],
   "source": [
    "# Let's now balance the target with oversampling using the algorithm SMOTE (Synthetic Minority Oversample)\n",
    "from imblearn.over_sampling import SMOTE"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 20,
   "metadata": {},
   "outputs": [],
   "source": [
    "# Transforming the dataset\n",
    "oversample = SMOTE(random_state = 12)\n",
    "X, y = oversample.fit_resample(X, y)"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 21,
   "metadata": {},
   "outputs": [
    {
     "name": "stdout",
     "output_type": "stream",
     "text": [
      "Counter({'blocked': 4500, 'ok': 4500})\n"
     ]
    }
   ],
   "source": [
    "# Summarizing the new class distribution\n",
    "from collections import Counter\n",
    "counter = Counter(y)\n",
    "print(counter)"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "We confirm above that the target LABEL is now balanced."
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "## Train – Test Split"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "Let's split the train_data into Train and Test subsets with Scikit-learn."
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 22,
   "metadata": {},
   "outputs": [
    {
     "name": "stdout",
     "output_type": "stream",
     "text": [
      "(6300, 5221)\n",
      "(2700, 5221)\n"
     ]
    }
   ],
   "source": [
    "from sklearn.model_selection import train_test_split\n",
    "X_train,X_test,y_train,y_test = train_test_split(X,y,test_size = 0.3, stratify=y,random_state=100)\n",
    "print(X_train.shape)\n",
    "print(X_test.shape)"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "## Naïve Bayes Model"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "A good text classifier is a classifier that efficiently categorizes large sets of text documents in a reasonable time frame and with acceptable accuracy, and that provides classification rules that are humanly readable for possible fine-tuning.   \n",
    "\n",
    "Despite the limitation in assuming independence of the attributes Naive Bayes classifier yields good results.\n",
    "\n",
    "One of the major advantages that Naive Bayes has over other classification algorithms is its ability to handle an extremely large number of features. In our case, each word is treated as a feature and there are thousands of different words. Also, it performs well even with the presence of irrelevant features and is relatively unaffected by them.\n",
    "\n",
    "The other major advantage it has is its relative simplicity. Naive Bayes' works well right out of the box and tuning it's parameters is rarely ever necessary, except usually in cases where the distribution of the data is known.\n",
    "\n",
    "It rarely ever overfits the data.\n",
    "\n",
    "Another important advantage is that its model training and prediction times are very fast for the amount of data it can handle."
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 23,
   "metadata": {},
   "outputs": [
    {
     "data": {
      "text/plain": [
       "GaussianNB()"
      ]
     },
     "execution_count": 23,
     "metadata": {},
     "output_type": "execute_result"
    }
   ],
   "source": [
    "from sklearn.naive_bayes import GaussianNB\n",
    "model = GaussianNB()\n",
    "model.fit(X_train,y_train)"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 24,
   "metadata": {},
   "outputs": [],
   "source": [
    "# Predicting on test data\n",
    "y_pred = model.predict(X_test)"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "## Results"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "Let's evalute the quality of our model using the classification report grid and plotting it with a heatmap.  \n",
    "\n",
    "The more important metrics is the Recall for OK SMS, once we want to avoid spam messages to be delivered.  \n",
    "\n",
    "We can find below Recall (for OK) of our model is 98%. "
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 25,
   "metadata": {},
   "outputs": [
    {
     "name": "stdout",
     "output_type": "stream",
     "text": [
      "              precision    recall  f1-score   support\n",
      "\n",
      "     blocked       0.98      1.00      0.99      1350\n",
      "          ok       1.00      0.98      0.99      1350\n",
      "\n",
      "    accuracy                           0.99      2700\n",
      "   macro avg       0.99      0.99      0.99      2700\n",
      "weighted avg       0.99      0.99      0.99      2700\n",
      "\n"
     ]
    }
   ],
   "source": [
    "from sklearn.metrics import classification_report, confusion_matrix\n",
    "\n",
    "print(classification_report(y_test,y_pred))\n"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 26,
   "metadata": {},
   "outputs": [
    {
     "data": {
      "image/png": "iVBORw0KGgoAAAANSUhEUgAAAXwAAAEaCAYAAAASSuyNAAAABHNCSVQICAgIfAhkiAAAAAlwSFlzAAALEgAACxIB0t1+/AAAADh0RVh0U29mdHdhcmUAbWF0cGxvdGxpYiB2ZXJzaW9uMy4yLjIsIGh0dHA6Ly9tYXRwbG90bGliLm9yZy+WH4yJAAAgAElEQVR4nO3dd7wU5d3+8c8FiIKFoiIKKth7+9lijMEajT2PMYnlsYtRE00sMcZuosaYogYTiKgRC9Ekthh7sCEqPrFGMSiiYgEsSLEA8v39cc/Csuw5Zw9wmGXner9e57Vnp353dubae+6ZPUcRgZmZNb52eRdgZmaLhgPfzKwgHPhmZgXhwDczKwgHvplZQTjwzcwKYrELfEmbSXpI0seSQtJ5bbSew7Pl92uL5TeSbDtd1wbL3VHSk5KmZOs4fGGvwxauttoXFhZJ10lq83vRJT0saWxbrydbV81ZVXPgS+os6WRJj0n6SNIMSeMl/TNbYYcFqrq2GjoAfwPWBs4GDgX+3tbrzYukPtkbGZL+0cQ0S0iamE0zdgHWtV9bfXjOD0ndSO/t0sAppPf60UW4/m9KekLStGx/v1VS3/lcVl9JV0kaLekzSZMkjciOpyWrTF86gA+oMu5ASdMljZK06vzUUy8k/TTbrmPmd/+VNLbsGAlJsyS9L+kRSd9tg7IXazWFtKS1gLuBdYAHgYuBD4AewC7AtcAGwOltU+Zsa2Q/p0TE79t4XUOAocD0Nl5PLT4Hdpe0ckS8VzFuH2CFbJoFsR9wGHDefMzbCfhyAddfaSugK3BURCzSD3VJ3wL+CjwPnAZ0AU4GhkvaMiLebcWy9gVuJm2fP2fL7AzsAfwW+F9Je0TE+BqWdQzwR+A5YPeImNiqF1Z/LgI+Av5Neq/n1zjgp9nv7YFepH355uyY+e0CVdlAWgx8SZ2Af5CC9n+qHHy/lLQV6QBtaz2zx4/aekUR8SULP8Tm113A/qRW7qUV444EXiDt6MssqoKy/WJGRMyMiAX9sKmmTd5rSe2BJSPi0ybGLwFcCbwNfC0ipmbD7wH+j/SBeGyN69qI1GiYCOwYEa+Xjb48C/BBwC2S+kUzX3uXdDrwS9JZzt4RMbmWGurcmhExBkDSS8z//vtJRNxQPkDSQOA94HDSB6sBRESzP8APgAAuaWnaivn2A4YDU7Of4cC+VaYbCzwMrEc6i5gCfEJqYfUsm+7hrI7Knz6kNzWAflWW/zAwtmLYdsA9wPuklvE7wD+BbcumqbpMUmt6ACkQpmePA4DlK6Yrzb8TcCrwOvAF8F/gsBq3YZ9sGb8H7gBeqRi/MjCT1Pp8qcrr3Bq4Llvnp9m2HQ7sX2UbVdu2h2fjr8uerwhcA4wHZgF9svEBXFe2vBOyYWdXrGcVUvi9AnRu5nWPrVbPArwHu5C6AF8HZpReVxPr3qVa7dm4h0j75hI1vn9/y5a1RzPT3JJNs1eVug/Inl+UPb8L6NSKY/A7wJ3AW9m+9wFwO7DJ/B6HZdNvCNwLTCN9KN9AOuOfa19oRa3z7L81zjcWeKnK8HbAZGBkxfDryvelsuGbALcBH5Iy4WVSj0X7KtP2BK4AxmTbdQLwALBrxTFVeTwuD4zItuvOFfvc/cCkbN0vAMc18XqPBkZl630NOAk4gibyr/Knli6dUj/ioBqmBUDS8aQDcBTw86yYw4HbJfWPiMpl9co20G2kU+hNgf7AcsBu2TS/IIXVmVktj2XDW3VaK2ld0pvzPnA5Kbx6Al/N1vtkM/N2AZ4A1iIF37+BzYHvAztJ2joiplTMdhGpy2Mg6U36PnCdpNciYngrSr+GtP2+EhEjsmGHkc5CbiDtCJX2Jx3AtwBvkna4w4C/Szo4Im7KpvsF6QD5GuksouSJiuWVttuFpL71qdUKjYgBknYCzpU0LCIel9Quq3NZYJdoooWdOZnU5XEsafu9Uhoxn+/BZcASwJ9IIfBqM+sunamOqDLuSdIH+DrAf5pZBpKWAvYkdTfc28ykfwK+DfwP6Uy6XDtJfwCOA24iNRRmNrfeCieSwngQ6X1bk7RNh0vaIiJGV0xfy3FIdi3jMWBJUmPkbWDvFl5nW2ovaYXS76SG0EmkfW1gSzNL2hJ4hNQYGEDaVnuTzqg2BQ4um7YPKYdWAq4HniEdC9uSgvuBJtbRB7gvq+nrEfFcNvxYUjfdk6TjcBqwK/AHSWtGxGllyziZdLbyPCkHO5PepwktvcbZavgE/RCY3IpP3G6kIHgNWK5s+HKkFtYUoGvFJ3QAB1YsZ0A2fL2yYf0oa3lWaRHN8wlHxSct8MNs2q1beB3zLDN7QwI4vmLaUov2wirzPwt0LBveixT8N9ewLfswp4XfgbQjDiobPwr4a1MtJGDpKsvsTAq8l2tp+ZSPA25oYvw8rbpsPxhLal12I7WwAzixxv2o6ns6n+/BqzRzRlGxnCuzedavMu74bNxuNSxn42zaO1uYbvlsumeq1P169jgAUK3HYAvv//rZ/ndVxfCx1H4c3pQN27FsmEgfFHm08KudnX4GHNvUvlwxbDjpTHmTsmFiztlXeWv8n9mwb1RZdruy3x8uvR5gM1L30iiys+Js+MqkFv1NVZZ1Oakxt2b2vCvpw+Dl8n0Z6E3K25pa+LXcpbMcqVVUq11Jn3hXRFk/Y/b7laR+ul0q5nk3Im6pGPav7HGtVqy7Fp9kj/tmrbDW2J90RlF5hjKQdLq8f5V5roqI2Rd+I+IdUhfL2q1ZcaSW3RDgO9kdU18F1iW1cpuaZ1rp92ye5UmB/y9gfUnLtaYGUku51no/Bg4i7dT3AOeSwm9BL7bPz3vwh2j+jKJc5+zxiyrjPq+YpjmlbftJs1PNGd+lyriVs8dRkR3drVF6/5Usl7WCJ5I+ALepMkuLx2F2prY36QNqWNm6gnmvLy0qY0m5syvpTORw4ClSK/mI5maU1IPUxXtnRLxQGp69nouyp/tn03YHdgfujYj7KpcVEbOqLH8X0nWXscBXI2Js2egDSGdJgyWtUP5D6r5rB+ycTbsbab8bUL4vR8Q44MbmXmO5Wrp0JpNOQ2pVunWt2invS9njGhXDx1SZ9sPscflWrLsWQ4FDSKdEP5L0JOlUa2hEvNnCvH1JO/pcp9URMVPSq8AWVeZp6rWt3urKU7ifCnwL2BF4l1R7VdnO/HNgX1L/aqWutO7D/L+tmJaIeELSL4Gfkc5OjmzN/E2Yn/egNXWXDqZ5bpcEliqfJrtwXRnUn0TEZ8zZrtWCvFxzHwynkLqqrpCkiLiihWXNRdLmpO63fqRGWLk3qsxSy3HYg9RoG1Vl2pdbU1+tsm68ThWDJ0a6sQJgWkQ8WDHPjaSz6ysl3RkRH1Jdc3n1MulaVSmv1iK1/J+tsfSVSGcEL5POEiobHetnjw/StJWyx1INC7Tda2nhvwQsJ6kypJuiWldeprm7YWpZXnOtn7k+1CLii4jYldTCuThb9wXAKEnVWocLqqnX1urtFBGvkFouJwAHAteX7fRzL1wS6ULQYaS+xu+QWie7kk7JoZVfvGtFK7lUQ0fgG9nT7sBqrZl/IWpN3aVbLntVGVca9k72+B3SqXr5z3eycaNJZwmbt7C+0gfUi1XGTSRdM3iBdFfPyS0VXyJpNVLLshT6+5NaibuSwq3ae1/LcVh6bPUZxwK4nHm3c7PfQcgaBA+RPuiqnc2UtOY4bO1r/4jUINuEsusAVZb3v8w5Q6n8ubFi2mrrrvk11NLC/xuwA+mi4Jk1TF+69WxD0gYvt0H2WK0lsSBKt+51rzKuL+lizFwi4mngaYDsCyzPklrDtzWznjHAupI6lLcwlb4Qtg4L/3VVcw1zLkRd28x0m5AuOF0QEeeWj5BU7QJvWxzAFwNbku52OB0Yml0snNb8bM1q6/dgZPb4FeZteW1LarmXzhjuIx2U5f4DEBGfZ7dy7idp94ho6oJm6b2o+l2DiPhA0s5ZLb/NWvq13Ga4P6klvk951wtA1rVXrcuqFhNIfcbrVxm3QZVhC8OlpAv+5d6vYb4lssfmeihK+8uGVcatR/pgLE0zmnSctPQhXjKDdDb+F2CgpCUi4qqy8aWL5h9UnqFUUcrV9ZnTzUbZsJrU0sK7mtTnd2r2JZJ5SPp/2Z05kK5STwN+IGnZsmmWJd3iOZUmrmQvgNIBONe1AUnfI90KWD5sBeY1jtSaqvaBUe520q2JlYF5TDa8uQ+LhWUocD5wUkQ011VRaq3N9emf3Rte7UymdL95S9ugJpL2AH4E/DkifkXqV12HdAF6QbT1e/AIqQV5tKTZ94VL2pTUNXJrRMwAiIj3IuLBip/yL8adSwrWgdldGnORdCTpjOBR5r1DZ7aI+IDUl/s88BtJp9TwOpp6/49hznccWi07o/wHsKWkHcuWK9roi5cR8XKV7dzsdz+y63O7Z0//3cyyJ5Du+to7OzZK84s5X+a6LZv2I9L1qD2yvvnKdc7T0s72lQNJt7cOkHRS2ehbSPvH+Vn3YOXyumjON7EfIF2IPkFS57JpepOuldWkxRZ+RHwqaS/Svbm3S7o/W/mHpANsR9Jp+6XZ9JOyL4kMAJ7SnL+rcTipD6x/RLR0IatVIuJVSQ8C/bON/hzpyvj+pLuFliib/CxJu5F22jdIB8TepE/zli46XUq6hW6ApC1IZwWbA0eRPhTb/KJVdvH7vBomfYXU2jw920FeJQVuf1I3XWVf95Ok2/iuknQ3qXXyVERU6+ttlqSVSd8qHZ0tk4i4W9LlwEmS7ouIoa1dbqZN34OImJEdlH8BHpP0J1I/+49IjYJzm5u/YlkvSDqY1Dp9MTsWyr9puztpXz2wpYuyEfFhWUv/Mkntsg/SptxD6soaIun3wMekW4+/SWotLsifQjkrq/8fkq4kNZj2JuVBzSQdypxrWSsCHSWdlT1/MyKG1LioLpIOKS2W1Mg7hNTv/aeY9/bTSieRPugfk1S6LXMvUq7dFBHlPRUnkj4g7pH0Z9KX8TqRuo3GAj+pXHh2fel7pGPqd9nZ6a8jYpyk75Ma1a9IGkK6fXpF0l1e+5HOmsZGxMeSzibdOPGEpOtJ+9FxpOOstrOOlm7jKbv9pzNpp3+ctPPMIN3Dfjfp3u32FdPvn22YadnPE8B+VZY7Fni4yvB+VNyCWW1Y2biewK2kU+6ppB1+fea9LbMf6WAeS/rE/IjUL340Zbe+0fRtgSsCV5F28hnZ4wBghYrpqs4fFbdstbDN+2TL+H0N01a7LXP1bJtMJB38T2fvy3nZcstvEWtH2pnGkVqHs7czzdyymY2ffStetpwHSXe0bF4xXUdSa+sToG8Lr6e57bfA70EN23Mv0ofgp6T9/a9kt8jNx7LWIt1r/Xq2XSZny/4RsFQzr/2AKuO6Z9swgJ+0sN4dSMfrFNKXeu4GNqq2/9GK4zAbvjHpGlHpi1c30sovXtH0F/6iWi1NLGNslXmnkbrmvk/ZrZLN7cuk7s/bs9fyBanB1NQXr3pl7+dbpC/+jc+2xc4Vr61yG7djzi3OPy0b/lXSWcSEbHnvAsNIF+2XqlhGf1LDpvTFq5NpxRevlC3EzMwa3GL355HNzGz+OPDNzArCgW9mVhAOfDOzgmjz/1JVBDM+GOMr34uZZXv3y7sEa6XPP39rfr7Fb2XcwjczKwgHvplZQTjwzcwKwoFvZlYQDnwzs4Jw4JuZFYQD38ysIBz4ZmYF4cA3MysIB76ZWUE48M3MCsKBb2ZWEA58M7OCcOCbmRWEA9/MrCAc+GZmBeHANzMrCAe+mVlBOPDNzArCgW9mVhAOfDOzgnDgm5kVhAPfzKwgHPhmZgXhwDczKwgHvplZQTjwzcwKwoFvZlYQDnwzs4Jw4JuZFYQD38ysIBz4ZmYF4cA3MysIB76ZWUE48M3MCsKBb2ZWEA58M7OCcOCbmRWEA9/MrCAc+GZmBeHANzMrCAe+mVlBOPDNzArCgW9mVhAOfDOzgnDgm5kVhAPfzKwgHPhmZgXhwDczKwgHvplZQXTIuwBbtM666Dc8Ovxpunfryu03/BGAKwddz78eH0E7taN7ty784men0GPF5WfP8977E9jnkP4cf+TBHHHQAQDc8+AjDLp+KLO+nMUO223NKScclcvrKbLevVdm8ODfstJKKzJrVjB48E0MGHANQ4YMYJ111gCga9flmDRpMttss0fO1Vo9UETkXcNib8YHYxabjfjMcy/SuVMnzrzwstmBP3XaNJZZemkAbrj1Dl5/4y3OPf0Hs+c5+cyf066d2HiDdTnioAOY9MlkDjjiRG4ZfAXdu3XlzAsvY589dmbbLTfP5TXNj2V798u7hAXWs2cPevbswXPPvcQyyyzNiBF38+1vH8OoUaNnT3PJJWcxefIULrro8hwrXTg+//wt5V3D4s4tfEDSesC+QC8ggHeBOyPilVwLawNbbrYx77w3fq5hpbAH+Oyzz1HZYfXQo0/Qe5WedOq01Oxhb7/7Hn1W7UX3bl0B2HarzXng4eGLVeA3gvffn8D7708AYOrUaYwa9Rq9evWcK/APOGAvvvGN7+ZVotWZwvfhS/oJMBQQ8DQwMvv9Zkln5FnbonT5wOvYef9Dufv+YZx49KEAfPrZ51xzw60cf+TBc027Wq9VeOPNt3nnvfHMnPkl/3p0BO9PmJhH2ZZZffXebLbZhjz99LOzh22//daMH/8Br78+Nr/CrK4UPvCBo4CtIuKSiLgh+7kE2DobV5WkYyU9I+mZq6+/eZEV21ZO6n84D902hD1325Gb/nYXAAMGD+HQ7+xP586d5pq2y3LLcvapJ3LqORdz2PGn0mvllWjfvn0eZRuw9NKdufnmgZx66vlMmTJ19vADD9yXW265I8fKrN64SwdmAasAb1YMXzkbV1VEDAIGweLVh9+SPXfrx/GnnsuJRx/Ki/95lQeGPc5vrhrMlKnTkMSSHTty0AH70G/7bem3/bYA3HrHP2nXzm2HPHTo0IGhQwcydOht3HHHvbOHt2/fnn333Z3tttszx+qs3jjw4WTgIUmjgbezYasBawEn5lbVIvTm2++w+qq9ABj22JP0Xb03ANf/4bLZ0wwYfAOdOy3FQQfsA8CHH09i+W5d+WTyFIb+/W5+feFPF33hxsCBv2LUqNe44oqr5xq+007b89//vs4777yfU2VWjwof+BFxr6R1SF04vUj99+OAkRHxZa7FtYHTzr2Ekc++wKRJk9l5v0M4/qhDeWzESMa+NQ61E6v07ME5p/2gxeVc8rs/8uprYwA47oiD6LNa77Yu3Spst91WHHzw//Dii6/w1FP3AHDOOZdy333DOPDAffjLX+7MuUKrN74tcyFopC6domiE2zKLxrdlLjh3vJqZFYQD38ysIBz4ZmYF4cA3MysIB76ZWUE48M3MCsKBb2ZWEA58M7OCcOCbmRWEA9/MrCAc+GZmBeHANzMrCAe+mVlBOPDNzAqiIf4evqQXSf98fJ5RQETEJou4JDOzutMQgQ/slXcBZmb1riECPyJm/z9aSasDa0fEg5I60SCv0cxsQTVUH76kY4C/AgOzQb2B2/OryMysfjRU4AMnAF8FJgNExGigR64VmZnViUYL/C8iYnrpiaQOVL+Ya2ZWOI0W+I9IOhPoJGlX4FbgrpxrMjOrC40W+GcAE4EXgf7AP4Gzcq3IzKxONNQdLBExS9KfgadIXTmvRoS7dMzMaLDAl7Qn8EfgddKXrvpK6h8R9+RbmZlZ/hoq8IFfAztGxGsAktYE7gYc+GZWeI3Whz+hFPaZMcCEvIoxM6snDdHCl/St7Nf/SPoncAupD//bwMjcCjMzqyMNEfjA3mW/jwe+nv0+Eei26MsxM6s/DRH4EXFE3jWYmdW7hgj8EklLAUcBGwJLlYZHxJG5FWVmVica7aLtEKAn8A3gEdIfT5uSa0VmZnWi0QJ/rYg4G5gWEX8G9gQ2zrkmM7O60GiBPyN7nCRpI6AL0Ce/cszM6kdD9eEDgyR1A84G7gSWAc7JtyQzs/rQUIEfEVdnvz4CrJFnLWZm9aYhAl/Sj5sbHxG/WVS1mJnVq4YIfGDZvAswM6t3DRH4EXF+3jWYmdW7RrtLx8zMmuDANzMrCAe+mVlBNEQfvu/SMTNrWUMEPnPu0lkX2Ir0pStIfzb50VwqMjOrMw0R+KW7dCTdD2wREVOy5+cBt+ZYmplZ3Wi0PvzVgOllz6fjv6VjZgY0SAu/zBDgaUm3kf7F4f7A9fmWZGZWHxoq8CPiF5LuAb6WDToiIp7NsyYzs3rRaF06AJ2ByRFxOTBOUt+8CzIzqwcNFfiSzgV+Avw0G7QEcEN+FZmZ1Y+G6tIh9dlvDvwbICLeldTmf1it0ypfa3kiqyufjr0/7xLMFrmGauED0yMiSBdskbR0zvWYmdWNRgv8WyQNBLpKOgZ4ELi6hXnMzAqhobp0IuIySbsCk0nfuj0nIh7IuSwzs7rQUIEv6ZcR8RPggSrDzMwKrdG6dHatMmyPRV6FmVkdaogWvqTvA8cDa0p6oWzUssAT+VRlZlZfGiLwgZuAe4CLgTPKhk+JiI/yKcnMrL40RJdORHwSEWOBy4GPIuLNiHgTmCFpm3yrMzOrDw0R+GX+AEwtez4tG2ZmVniNFvjKvngFQETMonG6rczMFkijBf4YST+UtET2cxIwJu+izMzqQaMF/nHAdsA7wDhgG+DYXCsyM6sTDdXdERETgO/mXYeZWT1qiMCXdHpEXCrpSrI/nFYuIn6YQ1lmZnWlIQIfeCV7fCbXKszM6lhDBH5E3JU9/jnvWszM6lVDBL6ku6jSlVMSEfsswnLMzOpSQwQ+cFn2+C2gJ3P+reH3gLF5FGRmVm8aIvAj4hEASRdGxA5lo+6S9GhOZZmZ1ZVGuw9/RUlrlJ5I6gusmGM9ZmZ1oyFa+GV+BDwsqfTt2j5A//zKMTOrHw0V+BFxr6S1gfWyQaMi4os8azIzqxcN1aUjqTNwGnBiRDwPrCZpr5zLMjOrCw0V+MC1wHTgK9nzccDP8yvHzKx+NFrgrxkRlwIzACLiM0D5lmRmVh8aLfCnS+pE9iUsSWsC7sM3M6PBLtoC5wL3AqtKuhH4KnB4rhWZmdWJhgl8Se2AbqRv225L6so5KSI+yLUwM7M60TCBHxGzJJ0YEbcAd+ddj5lZvWm0PvwHJJ0qaVVJ3Us/eRdlZlYPGqaFnzkyezyhbFgAa1SZ1sysUBoq8COib941mJnVq4YKfElLAccD25Na9o8Bf4yIz3MtzMysDjRU4APXA1OAK7Pn3wOGAN/OrSIzszrRaIG/bkRsWvZ8mKTnc6vGzKyONNpdOs9K2rb0RNI2wPAc6zEzqxuN1sLfBvhfSW9lz1cDXpH0IhARsUl+pZmZ5avRAn/3vAswM6tXDRX4EfFm3jWYmdWrRuvDNzOzJjjwzcwKwoFvZlYQDnwzs4Jw4JuZFYQD38ysIBz4ZmYF4cA3MysIB76ZWUE48M3MCqKh/rSCLTx/GvRr9vzmLkyY+AGbbb5z3uUU2tm//D2PPvkM3bt24bZrLwfgymtuYtjwkbST6N6tCz//yQ/osUJ3nnjmOX436AZmzJzJEh06cMpxh7HNFhsDcNzpFzDxw4/58stZbLHJ+vzspGNo3759ni/NFjFFRN41LPY6dOzVcBvxa9tvw9Sp07j22ssbMvA/HXt/3iXU7Jnn/0PnTkvxs4uvmB34U6d9yjJLdwbgxr/dzetvvs05Pz6OV0aPYfluXemxQndGv/Emx51+IQ/devVc80QEPz73V+zWbzv22Gn73F5Xa3VcZUPlXcPizl06VtVjjz/FRx9PyrsMA7bcdEO6LLfsXMNKYQ/w2eefI6UsXH/tNeixQncA1uqzGl9Mn8706TPmmmfml18yY+ZMnJ7F4y4ds8XUFVffyJ33P8yyS3dm8G8vmGf8A4+OYL211qBjxyVmD+t/2gW8OGo022+9Bbt+/SuLslyrA27ht0DSEU0MP1bSM5KemTVr2qIuy4wfHn0wD97yJ/bcZQduvu2euca99sZb/HbQEM798XFzDR/4q3MY9rfBzJgxg6eefXFRlmt1wIHfsvOrDYyIQRGxZURs2a7d0ou6JrPZvrnz13jw0RGzn78/8QNOPueXXHTGD1m1V895pl+yY0f6bbcVw4aPXJRlWh1wlw4g6YWmRgErLcpazGrx5rh3Wb33KgAMe2IkfVfrBcDkqdM44YxfcNLRh7D5xuvPnv7Tzz5j2qefseLy3Zn55Zc89tT/scUmG+RSu+XHd+kAksYD3wA+rhwFPBERqzQ3fyPepXPDkAF8fYevsMIK3Rk//gPOv+Ayrr1uaN5lLTSL0106p1/4G0Y+9xKTPplC925dOOHw7/LYU/9m7NvvoHbtWGWlFTn7R/1ZacXlGTjkVgbf9HdW67Xy7PkH/uocIuDEM3/B9BkzmfXlLLbeYiNOP+FIOixGt2X6Lp0F58AHJA0Gro2Ix6uMuykiDmpu/kYM/Ea3OAW+JQ78BecuHSAijmpmXLNhb2a2uPBFWzOzgnDgm5kVhAPfzKwgHPhmZgXhwDczKwgHvplZQTjwzcwKwoFvZlYQDnwzs4Jw4JuZFYQD38ysIBz4ZmYF4cA3MysIB76ZWUE48M3MCsKBb2ZWEA58M7OCcOCbmRWEA9/MrCAc+GZmBeHANzMrCAe+mVlBOPDNzArCgW9mVhAOfDOzgnDgm5kVhAPfzKwgHPhmZgXhwDczKwgHvplZQTjwzcwKwoFvZlYQDnwzs4Jw4JuZFYQD38ysIBz4ZmYF4cA3MysIB76ZWUE48M3MCsKBb2ZWEA58M7OCcOCbmRWEA9/MrCAc+GZmBeHANzMrCAe+mVlBOPDNzArCgW9mVhAOfDOzglBE5F2D1TFJx0bEoLzrsNr4/bLmuIVvLTk27wKsVfx+WZMc+GZmBeHANzMrCAe+tcT9wYsXv1/WJF+0NTMrCLfwzcwKwoFvZlYQDnyrStLukl6V9JqkM/Kux5on6RpJEyS9lHctVr8c+DYPSe2BAcAewAbA9yRtkG9V1oLrgN3zLsLqmwPfqtkaeC0ixkTEdGAosG/ONVkzIuJR4KO867D65sC3anoBb5c9H5cNM7PFmAPfqlGVYb5/12wx5wSneX4AAAKBSURBVMC3asYBq5Y97w28m1MtZraQOPCtmpHA2pL6SuoIfBe4M+eazGwBOfBtHhExEzgRuA94BbglIv6Tb1XWHEk3AyOAdSWNk3RU3jVZ/fGfVjAzKwi38M3MCsKBb2ZWEA58M7OCcOCbmRWEA9/MrCAc+FaXJHWVdHwbLv9wSb9vYZrzJJ3ayuVOXbDKzNqOA9/qVVegauBnf83TzFrJgW/16hJgTUnPSfqVpH6Shkm6CXhRUp/yv/0u6VRJ52W/rynpXkn/J+kxSes1tyJJe0t6StKzkh6UtFLZ6E0l/UvSaEnHlM1zmqSRkl6QdP7CfelmbaND3gWYNeEMYKOI2AxAUj/Sn23eKCLekNSnmXkHAcdFxGhJ2wBXATs1M/3jwLYREZKOBk4HTsnGbQJsCywNPCvpbmAjYO2sHgF3Stoh+xPFZnXLgW+Lk6cj4o3mJpC0DLAdcKs0+49+LtnCcnsDf5G0MtARKF/HHRHxGfCZpGGkkN8e2A14NptmGdIHgAPf6poD3xYn08p+n8ncXZJLZY/tgEmlM4MaXQn8JiLuzM4kzisbV/m3R4LUqr84Iga2Yh1muXMfvtWrKcCyzYwfD/SQtLykJYG9ACJiMvCGpG8DKNm0hXV1Ad7Jfj+sYty+kpaStDzQj/SXRO8DjszOJpDUS1KP2l+aWT7cwre6FBEfShqeXZi9B7i7YvwMSRcAT5G6YEaVjT4Y+IOks4AlSP+i8flmVnceqQvoHeBJoG/ZuKezda8GXBgR7wLvSlofGJF1G00FDgEmzOfLNVsk/NcyzcwKwl06ZmYF4cA3MysIB76ZWUE48M3MCsKBb2ZWEA58M7OCcOCbmRXE/wc5tZCenC8WvwAAAABJRU5ErkJggg==\n",
      "text/plain": [
       "<Figure size 432x288 with 1 Axes>"
      ]
     },
     "metadata": {
      "needs_background": "light"
     },
     "output_type": "display_data"
    }
   ],
   "source": [
    "# Plotting the confusion matrix with heatmap\n",
    "mat = confusion_matrix(y_test, y_pred)\n",
    "sns.heatmap(mat.T, square = True, annot = True, fmt = 'd', cbar = False)\n",
    "\n",
    "plt.rc('font', size=28)\n",
    "plt.rc('axes', titlesize=18)\n",
    "plt.rc('axes', labelsize=28)\n",
    "plt.rc('xtick', labelsize=18)\n",
    "plt.rc('ytick', labelsize=18) \n",
    "plt.xlabel('true label')\n",
    "plt.ylabel('predicted label')\n",
    "plt.title('Confusion Matrix for 0-OK and 1-Blocked');"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "## Validation"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 27,
   "metadata": {},
   "outputs": [],
   "source": [
    "# Reading the dataset and assigning it to an object\n",
    "validation_data = pd.read_csv('validation_data_original.csv')"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 28,
   "metadata": {},
   "outputs": [
    {
     "data": {
      "text/html": [
       "<div>\n",
       "<style scoped>\n",
       "    .dataframe tbody tr th:only-of-type {\n",
       "        vertical-align: middle;\n",
       "    }\n",
       "\n",
       "    .dataframe tbody tr th {\n",
       "        vertical-align: top;\n",
       "    }\n",
       "\n",
       "    .dataframe thead th {\n",
       "        text-align: right;\n",
       "    }\n",
       "</style>\n",
       "<table border=\"1\" class=\"dataframe\">\n",
       "  <thead>\n",
       "    <tr style=\"text-align: right;\">\n",
       "      <th></th>\n",
       "      <th>SMS</th>\n",
       "    </tr>\n",
       "  </thead>\n",
       "  <tbody>\n",
       "    <tr>\n",
       "      <th>0</th>\n",
       "      <td>BB informa:Prezado cliente suas senhas de ACES...</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>1</th>\n",
       "      <td>BB informa:Prezado cliente suas senhas de ACES...</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>2</th>\n",
       "      <td>BB informa:Prezado cliente suas senhas de ACES...</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>3</th>\n",
       "      <td>Node Warning: SFRO01H000001.call.br Warning em...</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>4</th>\n",
       "      <td>BB informa:Prezado cliente suas senhas de ACES...</td>\n",
       "    </tr>\n",
       "  </tbody>\n",
       "</table>\n",
       "</div>"
      ],
      "text/plain": [
       "                                                 SMS\n",
       "0  BB informa:Prezado cliente suas senhas de ACES...\n",
       "1  BB informa:Prezado cliente suas senhas de ACES...\n",
       "2  BB informa:Prezado cliente suas senhas de ACES...\n",
       "3  Node Warning: SFRO01H000001.call.br Warning em...\n",
       "4  BB informa:Prezado cliente suas senhas de ACES..."
      ]
     },
     "execution_count": 28,
     "metadata": {},
     "output_type": "execute_result"
    }
   ],
   "source": [
    "# Looking at firsts rows from dataframe validation_data\n",
    "validation_data.head()"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 29,
   "metadata": {},
   "outputs": [],
   "source": [
    "# Keeping the original validation data in an object\n",
    "original_validation_data = validation_data"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 30,
   "metadata": {},
   "outputs": [
    {
     "data": {
      "text/plain": [
       "(2000, 1)"
      ]
     },
     "execution_count": 30,
     "metadata": {},
     "output_type": "execute_result"
    }
   ],
   "source": [
    "# Looking basic info from validation dataframe\n",
    "validation_data.shape"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "validation_data has 2000 instances and 1 attribute SMS"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 31,
   "metadata": {},
   "outputs": [
    {
     "name": "stdout",
     "output_type": "stream",
     "text": [
      "<class 'pandas.core.frame.DataFrame'>\n",
      "RangeIndex: 2000 entries, 0 to 1999\n",
      "Data columns (total 1 columns):\n",
      " #   Column  Non-Null Count  Dtype \n",
      "---  ------  --------------  ----- \n",
      " 0   SMS     2000 non-null   object\n",
      "dtypes: object(1)\n",
      "memory usage: 15.8+ KB\n"
     ]
    }
   ],
   "source": [
    "# Looking for non-nulls and dtype in validation_data\n",
    "validation_data.info()"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "All validation_data are non-null and dtype is object."
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 32,
   "metadata": {},
   "outputs": [
    {
     "data": {
      "text/plain": [
       "'BB informa prezado client senha acess serao bloqueada seguranca caso nao atual dado http bbseguro app celular mobi dkwz'"
      ]
     },
     "execution_count": 32,
     "metadata": {},
     "output_type": "execute_result"
    }
   ],
   "source": [
    "# Creating a corpus of refinned messages\n",
    "stemming = PorterStemmer()\n",
    "corpus = []\n",
    "\n",
    "for i in range (0,len(validation_data)):\n",
    "    s1 = re.sub('[^a-zA-Z]',repl = ' ',string = validation_data['SMS'][i])\n",
    "    s1.lower()\n",
    "    s1 = s1.split()\n",
    "    s1 = [stemming.stem(word) for word in s1 if word not in set(stopwords.words('portuguese'))]\n",
    "    s1 = ' '.join(s1)\n",
    "    corpus.append(s1)\n",
    "\n",
    "corpus[50]"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 33,
   "metadata": {},
   "outputs": [],
   "source": [
    "# Tokenizing and vectorizing\n",
    "countvectorizer = CountVectorizer(vocabulary = feature_cols)\n",
    "X_validation = countvectorizer.fit_transform(corpus).toarray()"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 34,
   "metadata": {},
   "outputs": [
    {
     "data": {
      "text/plain": [
       "(2000, 5221)"
      ]
     },
     "execution_count": 34,
     "metadata": {},
     "output_type": "execute_result"
    }
   ],
   "source": [
    "X_validation.shape"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 35,
   "metadata": {},
   "outputs": [
    {
     "data": {
      "text/plain": [
       "(2700, 5221)"
      ]
     },
     "execution_count": 35,
     "metadata": {},
     "output_type": "execute_result"
    }
   ],
   "source": [
    "X_test.shape"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 36,
   "metadata": {},
   "outputs": [
    {
     "data": {
      "text/plain": [
       "array(['blocked', 'blocked', 'blocked', ..., 'ok', 'ok', 'ok'],\n",
       "      dtype='<U7')"
      ]
     },
     "execution_count": 36,
     "metadata": {},
     "output_type": "execute_result"
    }
   ],
   "source": [
    "# Predicting on validation data\n",
    "y_pred = model.predict(X_validation)\n",
    "y_pred"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 37,
   "metadata": {},
   "outputs": [
    {
     "data": {
      "text/plain": [
       "Counter({'blocked': 218, 'ok': 1782})"
      ]
     },
     "execution_count": 37,
     "metadata": {},
     "output_type": "execute_result"
    }
   ],
   "source": [
    "# Counting the occurrences in the predicted LABEL\n",
    "import collections, numpy\n",
    "collections.Counter(y_pred)\n"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 38,
   "metadata": {},
   "outputs": [
    {
     "data": {
      "text/html": [
       "<div>\n",
       "<style scoped>\n",
       "    .dataframe tbody tr th:only-of-type {\n",
       "        vertical-align: middle;\n",
       "    }\n",
       "\n",
       "    .dataframe tbody tr th {\n",
       "        vertical-align: top;\n",
       "    }\n",
       "\n",
       "    .dataframe thead th {\n",
       "        text-align: right;\n",
       "    }\n",
       "</style>\n",
       "<table border=\"1\" class=\"dataframe\">\n",
       "  <thead>\n",
       "    <tr style=\"text-align: right;\">\n",
       "      <th></th>\n",
       "      <th>SMS</th>\n",
       "      <th>PREDICTED LABEL</th>\n",
       "    </tr>\n",
       "  </thead>\n",
       "  <tbody>\n",
       "    <tr>\n",
       "      <th>0</th>\n",
       "      <td>BB informa:Prezado cliente suas senhas de ACES...</td>\n",
       "      <td>blocked</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>1</th>\n",
       "      <td>BB informa:Prezado cliente suas senhas de ACES...</td>\n",
       "      <td>blocked</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>2</th>\n",
       "      <td>BB informa:Prezado cliente suas senhas de ACES...</td>\n",
       "      <td>blocked</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>3</th>\n",
       "      <td>Node Warning: SFRO01H000001.call.br Warning em...</td>\n",
       "      <td>ok</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>4</th>\n",
       "      <td>BB informa:Prezado cliente suas senhas de ACES...</td>\n",
       "      <td>blocked</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>...</th>\n",
       "      <td>...</td>\n",
       "      <td>...</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>1995</th>\n",
       "      <td>Faça login no aplicativo OlympTrad?: 179.240.1...</td>\n",
       "      <td>ok</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>1996</th>\n",
       "      <td>Faça login no aplicativo OlympTrad?: 179.240.1...</td>\n",
       "      <td>ok</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>1997</th>\n",
       "      <td>bitnuvem.com: Identificamos acesso do IP 179.2...</td>\n",
       "      <td>ok</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>1998</th>\n",
       "      <td>enter\u0011in\u0011application: 179.240.169.116, BR, For...</td>\n",
       "      <td>ok</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>1999</th>\n",
       "      <td>Faça login no aplicativo OlympTrad?: 179.240.1...</td>\n",
       "      <td>ok</td>\n",
       "    </tr>\n",
       "  </tbody>\n",
       "</table>\n",
       "<p>2000 rows × 2 columns</p>\n",
       "</div>"
      ],
      "text/plain": [
       "                                                    SMS PREDICTED LABEL\n",
       "0     BB informa:Prezado cliente suas senhas de ACES...         blocked\n",
       "1     BB informa:Prezado cliente suas senhas de ACES...         blocked\n",
       "2     BB informa:Prezado cliente suas senhas de ACES...         blocked\n",
       "3     Node Warning: SFRO01H000001.call.br Warning em...              ok\n",
       "4     BB informa:Prezado cliente suas senhas de ACES...         blocked\n",
       "...                                                 ...             ...\n",
       "1995  Faça login no aplicativo OlympTrad?: 179.240.1...              ok\n",
       "1996  Faça login no aplicativo OlympTrad?: 179.240.1...              ok\n",
       "1997  bitnuvem.com: Identificamos acesso do IP 179.2...              ok\n",
       "1998  enter\u0011in\u0011application: 179.240.169.116, BR, For...              ok\n",
       "1999  Faça login no aplicativo OlympTrad?: 179.240.1...              ok\n",
       "\n",
       "[2000 rows x 2 columns]"
      ]
     },
     "execution_count": 38,
     "metadata": {},
     "output_type": "execute_result"
    }
   ],
   "source": [
    "# Printing the SMS for validation and the Predicted Label side by side\n",
    "original_validation_data['PREDICTED LABEL'] = y_pred\n",
    "original_validation_data"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 39,
   "metadata": {},
   "outputs": [],
   "source": [
    "# Exporting Dataframe with results to excel csv file\n",
    "original_validation_data.to_csv('validation_data.csv', index=False)\n"
   ]
  }
 ],
 "metadata": {
  "kernelspec": {
   "display_name": "Python 3",
   "language": "python",
   "name": "python3"
  },
  "language_info": {
   "codemirror_mode": {
    "name": "ipython",
    "version": 3
   },
   "file_extension": ".py",
   "mimetype": "text/x-python",
   "name": "python",
   "nbconvert_exporter": "python",
   "pygments_lexer": "ipython3",
   "version": "3.8.3"
  }
 },
 "nbformat": 4,
 "nbformat_minor": 4
}