Spam Filter.ipynb

{
 "cells": [
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "# Building a Spam Filter with Naive Bayes\n",
    "In machine learning, Naive Bayes classifiers are a family of simple \"probabilistic classifiers\" based on applying Bayes theorem with strong (but naive) independence assumptions between the features. In probability theory and statistics, Bayes' theorem (alternatively Bayes law or Bayes rule) describes the probability of an event, based on prior knowledge of conditions that might be related to the event.\n",
    "\n",
    "In this notebook, I'll build a spam filter for SMS messages using the multinomial Naive Bayes algorithm. My goal is to write a program that classifies new messages with an accuracy greater than 95%.\n",
    "\n",
    "To train the algorithm, I'll use a dataset of 5,572 SMS messages that are already classified by humans. The dataset was put together by Tiago A. Almeida and José María Gómez Hidalgo, and it can be downloaded from the [The UCI Machine Learning Repository](https://archive.ics.uci.edu/ml/datasets/sms+spam+collection). The data collection process is described in more details on [this page](http://www.dt.fee.unicamp.br/~tiago/smsspamcollection/#composition), where you can also find some of the authors' papers."
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "## Data Exploration"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 93,
   "metadata": {},
   "outputs": [],
   "source": [
    "import pandas as pd\n",
    "import matplotlib.pyplot as plt\n",
    "%matplotlib inline"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 94,
   "metadata": {},
   "outputs": [],
   "source": [
    "data = pd.read_csv('SMSSpamCollection.txt', sep='\\t', header = None, names=['Label','SMS'], encoding = 'utf_8')"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 95,
   "metadata": {},
   "outputs": [
    {
     "name": "stdout",
     "output_type": "stream",
     "text": [
      "(5572, 2)\n"
     ]
    },
    {
     "data": {
      "text/html": [
       "<div>\n",
       "<style scoped>\n",
       "    .dataframe tbody tr th:only-of-type {\n",
       "        vertical-align: middle;\n",
       "    }\n",
       "\n",
       "    .dataframe tbody tr th {\n",
       "        vertical-align: top;\n",
       "    }\n",
       "\n",
       "    .dataframe thead th {\n",
       "        text-align: right;\n",
       "    }\n",
       "</style>\n",
       "<table border=\"1\" class=\"dataframe\">\n",
       "  <thead>\n",
       "    <tr style=\"text-align: right;\">\n",
       "      <th></th>\n",
       "      <th>Label</th>\n",
       "      <th>SMS</th>\n",
       "    </tr>\n",
       "  </thead>\n",
       "  <tbody>\n",
       "    <tr>\n",
       "      <th>0</th>\n",
       "      <td>ham</td>\n",
       "      <td>Go until jurong point, crazy.. Available only ...</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>1</th>\n",
       "      <td>ham</td>\n",
       "      <td>Ok lar... Joking wif u oni...</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>2</th>\n",
       "      <td>spam</td>\n",
       "      <td>Free entry in 2 a wkly comp to win FA Cup fina...</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>3</th>\n",
       "      <td>ham</td>\n",
       "      <td>U dun say so early hor... U c already then say...</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>4</th>\n",
       "      <td>ham</td>\n",
       "      <td>Nah I don't think he goes to usf, he lives aro...</td>\n",
       "    </tr>\n",
       "  </tbody>\n",
       "</table>\n",
       "</div>"
      ],
      "text/plain": [
       "  Label                                                SMS\n",
       "0   ham  Go until jurong point, crazy.. Available only ...\n",
       "1   ham                      Ok lar... Joking wif u oni...\n",
       "2  spam  Free entry in 2 a wkly comp to win FA Cup fina...\n",
       "3   ham  U dun say so early hor... U c already then say...\n",
       "4   ham  Nah I don't think he goes to usf, he lives aro..."
      ]
     },
     "execution_count": 95,
     "metadata": {},
     "output_type": "execute_result"
    }
   ],
   "source": [
    "print(data.shape)\n",
    "data.head()"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 96,
   "metadata": {},
   "outputs": [
    {
     "data": {
      "text/html": [
       "<div>\n",
       "<style scoped>\n",
       "    .dataframe tbody tr th:only-of-type {\n",
       "        vertical-align: middle;\n",
       "    }\n",
       "\n",
       "    .dataframe tbody tr th {\n",
       "        vertical-align: top;\n",
       "    }\n",
       "\n",
       "    .dataframe thead th {\n",
       "        text-align: right;\n",
       "    }\n",
       "</style>\n",
       "<table border=\"1\" class=\"dataframe\">\n",
       "  <thead>\n",
       "    <tr style=\"text-align: right;\">\n",
       "      <th></th>\n",
       "      <th>Label</th>\n",
       "      <th>SMS</th>\n",
       "    </tr>\n",
       "  </thead>\n",
       "  <tbody>\n",
       "    <tr>\n",
       "      <th>5567</th>\n",
       "      <td>spam</td>\n",
       "      <td>This is the 2nd time we have tried 2 contact u...</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>5568</th>\n",
       "      <td>ham</td>\n",
       "      <td>Will ü b going to esplanade fr home?</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>5569</th>\n",
       "      <td>ham</td>\n",
       "      <td>Pity, * was in mood for that. So...any other s...</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>5570</th>\n",
       "      <td>ham</td>\n",
       "      <td>The guy did some bitching but I acted like i'd...</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>5571</th>\n",
       "      <td>ham</td>\n",
       "      <td>Rofl. Its true to its name</td>\n",
       "    </tr>\n",
       "  </tbody>\n",
       "</table>\n",
       "</div>"
      ],
      "text/plain": [
       "     Label                                                SMS\n",
       "5567  spam  This is the 2nd time we have tried 2 contact u...\n",
       "5568   ham               Will ü b going to esplanade fr home?\n",
       "5569   ham  Pity, * was in mood for that. So...any other s...\n",
       "5570   ham  The guy did some bitching but I acted like i'd...\n",
       "5571   ham                         Rofl. Its true to its name"
      ]
     },
     "execution_count": 96,
     "metadata": {},
     "output_type": "execute_result"
    }
   ],
   "source": [
    "data.tail()"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 97,
   "metadata": {},
   "outputs": [
    {
     "data": {
      "text/plain": [
       "ham     0.865937\n",
       "spam    0.134063\n",
       "Name: Label, dtype: float64"
      ]
     },
     "execution_count": 97,
     "metadata": {},
     "output_type": "execute_result"
    }
   ],
   "source": [
    "data['Label'].value_counts(normalize=True)"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 98,
   "metadata": {},
   "outputs": [
    {
     "data": {
      "text/plain": [
       "<matplotlib.axes._subplots.AxesSubplot at 0x12e4b7190>"
      ]
     },
     "execution_count": 98,
     "metadata": {},
     "output_type": "execute_result"
    },
    {
     "data": {
      "image/png": "iVBORw0KGgoAAAANSUhEUgAAANQAAACxCAYAAAC855PxAAAABHNCSVQICAgIfAhkiAAAAAlwSFlzAAALEgAACxIB0t1+/AAAADh0RVh0U29mdHdhcmUAbWF0cGxvdGxpYiB2ZXJzaW9uMy4xLjMsIGh0dHA6Ly9tYXRwbG90bGliLm9yZy+AADFEAAAcG0lEQVR4nO2deXgUVdq37+rsJBAMhCVC2FPsi6yyC8rARBHFV0fadRwcZdpXBpEREDOoL6DjuLY4Op+MWzvigo4KKioCirKDAkLBsMpuSAghe9L1/XEqkI2kE6pTXd3nvq6+mq6uVP26yS9ne87zKLquI5FIzMFhtQCJJJiQhpJITEQaSiIxEWkoicREpKEkEhORhpJITEQaSiIxEWkoicREpKEkEhORhpJITEQaSiIxEWkoicREpKEkEhORhpJITEQaSiIxkXCrBUjqD8XlMXXzm+52KtW9r6rqHUBnTdMeMvO+gYxsoSQSE5EtlMTfDFJVdTmQCLwEZAB/AkpbtxuA7sBMoABoDfwDGAX0Ap7TNO2l+hZdV6ShJP6mCPgN0AZYBrwFpGqalquq6svGe0eAVkBvoC/wHtABuBT4EGFEWyC7fBJ/s1nTNB04DjQATgKvq6r6L6AnEGGct13TtCLgNLBX07RCIBOItkBznZEtlMTflJ0IiQfmAsnG6y853/ULimxB0lCS+uQMsA7YDOQgWqAkYL+VosxEkWnE6g/F5YkEOgKdEGOEZOORCMQZj1jj0QAoBHIRv3ylz6eAg8AB47Ef2Ke7nSfr75NILoQ0lB9RXJ5OwEBgkPEoO2YwmxPAJmAD8D2wVnc7z/jpXpILIA1lIorL0wgYB1wLXAU0tVCOF2GuT4CPdbdzm4VaQgZpqItEcXmaAxMRJhoJRFoq6MLsR5jrA93tXG21mGBFGqoOKC6Pglh4/CMwAf914/zFbuAV4HXd7Uy3WkwwIQ1VCxSXJwG4E7gbSLFYjhkUIBZOF+pu57dWiwkGpKF8QHF5mgHTgXsRM3HByCogTXc7V1ktxM5IQ1WD0SL9BXAhprFtjZ44xdwLpmVVG20eisiF3SpQXJ5wYCrwMGJ1XyLxCWmoCiguz+XAy0APq7XYHVVVU4DXEAGyxcAi4A7ElH4L4BVN015UVXUEkGb8WAPgNsSi9mLgF6At8A4iKr0PsFTTtFn19TlqgzSUgeLyXAI8AfyB8/FlkovjKsRi8zRgGNAVEUHeBxGYvU1V1feAbsAtmqYdVVV1FvA/gAdoD4wBYhDT/pciIkYOAgFpKBltDiguTyqwC5iMNJOZvAqkA58jxqHFwPeaphVompYHbEeEYB0BnldV9TXgCs4vQ+zTNC0LEYF+QtO0DE3T8gngQNqQbqEUlycCmI/4CyqNZD7XAt9qmjZXVdWbgXnAKVVVw4AoRMu0B/gYaK9pWraqqq9j4wj0kDWU4vK0RfTLB1osJZjZCLylqmoxYtz0AnA78BnQBHhc07R0VVXfBNapqpqJiElMskrwxRKS0+aKy3M18CbQ2GotNucD4Dbd7cz15WRVVUcC92ia9ju/qrKQkBtDKS7PFOA/SDOZwUTgG2PhW0IItVBG/N18xEKtxFz2AWN1t3OP1UKsJiQMZWzsWwQ4rdYSxJwCxutu5/dWC7GSoDeU4vJEI7p4Y6zWEgLkAam62/mN1UKsIqgNpbg8UcBHwFirtYQQZ4HfhGpLFbSTEkY83rtIM9U3ccAyxeXpa7UQKwhKQxkTEK8C463WEqLEA8sVlyfk4iGD0lCIFfnbrBYR4iQAXyguTyurhdQnQTeGUlyemxAREJLAYDMwzNfFX7sTVC2U4vL0REyPSwKHywih/5OgMZSxu/YjgmBnbRByk+LyTLNaRH0QFF0+YxLiC8T+G0lgUgyM1N3ONVYL8SfB0kL9L9JMgU448Lri8sRaLcSf2N5QRrrjeVbrkPhEB8Su6KDF1y6fjsjhFlC1ehSXxwGsBoZYrUXiMzpwpe52rrBaiD+wews1FWkmu6EAixSXp6HVQvyBbQ2luDzJwONW65DUiTbAX60W4Q9qa6gJiOwzZxGhPWVzes9GZKMpAI4Cfytz/ZWIpv63wDZE5pq3EFlsPkVEKW9FlHvxlXmIbDgSe+JSXJ4OVoswm9oYKhKRY+0wYsbm98DNxnuTEK1FDMI8sYjUxRMqXGMxkI0wlxPYgUgVdRhR8dun4sRG4OWkWmiXBB6RBOEERW0MpQDXI/KrlXa1ShOc7EHUTh2IqOo91zjeqcI1ngYGAwuM1ycRRrrCeN3dRy1PIbMUBQMTFZdnqNUizKQ2WY90REJ5OF8TtTQqYQOidboPGAqUhu5XnBUs3Xh2xHhei8gqetR4HVWTCMXlGY+owyQJDv5OEGWeqk0LVQiUGP8ufS5tJe5BmK0XsJDzaXUrtiL5xrPXeM6t8NoXHqvFuZLAZ4Di8gTNnjWzZvlcxvP9iHGWX/KqKS7POGo3cSGxB9OtFmAWZhnqoPG8FPgBUUcJzK+lNMPk60kCg9GKy9PbahFmYJahpgJrgETExrKZxvHLTbo+xhc+0qzrSQKOB6wWYAa2iTZXXJ5FiHKckuCkCGivu52HrRZyMdgiUsIIUwna9L0SQAQJ/N5qEReLLQwFXIeMiggFbJ8HxC7VN24u+6L1JQ1w39ifYR0SyS0sYfHmg8z8eCuFxV4cisLc1J7cdXkHGkSGsXTHUe59Zz1n8ouqvPAbtw3m1gHtyh1bsHwHMz/eSkSYg8eu7omzXzviosJ5e+MBpi3ZREGxmOV/YHQXZo7pRn5RCdOWbObdzWJuJjLcwf651/K3r3by7De7/PF9BCsdFJdniJ03IQa8oRSXJxG4suyx9+4axsC2TfnxcCYxkWFMG9WFnIJiHln6E3+//jKmXtGZPSfPkJFbwKR+bSkoLuH3b62t8vo9kuLJLSxm+c5j547tOJYFwP9d04sHr+zKzuNZ5BYVM2V4CpHhDia/vY4WjaJ54treTFuymbYJsfzz5oG8v+UQXl3nD4M7EuFw8MqakE/1XRduQkxw2ZKANxSiPOQ5nY1jIhnYtil7f82mzxPLiIkI4/i8iVzd/VKeW6nxp+Ep7DyeRZ8FywDQHhlPz6SqC22EORQ6N49n6+FMrvvn6krv3zusE8fP5NFr/jIUBXbMvpo7B7Vn1sdbaZMQS5jDwY5jWeQUFNMoJoKmcVFk5BQw48quPPPNLnILS6q4q6QGblBcnqm621mbxf6AwQ6GuqHsi7yiYnIKivHqOroOug6KAmfyixjesRkRYQ6W7zx2rlvW9pGPLnjhlGYNiY4IIyJM4VXnIApLvCxcvZttR0+TGBdFXFQEPx05TVGJuNa6A+l0TGzI5e0SWX8wnRKvl24t42mTEEtWXiHpZwu4Y1B74qMjeHH1bj9+JUFNS6A/sM5qIXUhoA2luDwxVNhAWFDsZeoHG3H/T382/2UcDSLFR0hb9hN9WycAcEmDSDb9ZRztm8Sx7Oej/GnxBk7nFVa6fo+kSwDom9yEvslNALi1fzv6/+0zdp04Q35RCSnNGhITEUZeUQmdmzcCxBju4235zPhoC3PGdqewxMsf31kPwENXdeWFVdoFx2wSnxiNTQ0V6LN8QxBh/uUIczjQgT6tE1CbN+JUTiGFxd5z5rptYHvCHQrHzuQxqV9b3rit6vXlnMJiVmjHuf/9jcRPf5cnvtxBbFQ4j6b2QtfhtXX7aBoXzU+zUtny0LhzpouOCAPg6RW7SHzoAy6d/SGLNx3kd33b0DI+hmdX7uKZiX35dcFEvv3zVbRqLDOb1ZLRVguoK4FuqFEVDyRfEsuLN/YjI7eA9mn/od+Tn9E0NoqP7h5x7pz1B9LpvWAZPeYtZdfxLK7p0YoWjSqnw1i6/QijX/ia51eKFuXJr34GYGBbYZwHlmxi0Q97aRIbSU5BCW+s2wdAbmFxlWJnjunGP77bQ4+kxky9ojPTP9xCy0YxzE2V4Ye1ZLBRhsh22M5Qg9o1Jczh4Mtdx9l/6iybDmWw8dApmjU8//1vO3oaXYcSr87WI5kAVbYSiXFR9EtOIC5KtGwFRWKsFO4QX0tuYQl3edaSMON9hj6znBzDSAdO5VS61vW9W9OhaRxPfbWTy4yu5zubDrDh0KlzXVGJz0Qj9s3ZjoA1lOLyxHF+X9U5SsdCPZMaoyhizadjosj38drafXi9OkM7NCMyXHy00nHPgYzKJpg3vjcbZozjzkFiJ/aYLi0B2HjoFAAfTh7OyfkTiY+JIDLcwaiUFhQWl/DD/vRK15o1phuv/rCXE9n5546VeHW89ojsCkRGWi2gLgTypERPqtC3cs8J/vtrNn1aJ/DjzFQiwxxc2rgBX+w8yuHTuby98QC3DGjH9lmp5BQW07tVAos3HST9bAHdWsbz+NW92PxLBo99vp1/fLeH2we25+nrL+O6Xq0Y0KYpxSVe5i/fAcDOE2eY0Ks16x8cS4lXR23eiKdX7Kw0wfHbbkn0SGp8bup962HRKvZr04QuzRux5XCGn7+qoKSX1QLqQrWGUlW12nzUmqY9ba6ccnSt6mBhsZdRz3/FU9ddxmi1BSVendfW7mXaks0ATP73OrILiri5b1saFpfw8nd7mP6heK9pXBQTerU+N6mw6VAGE15ZxeNX92JQ26bsP5XD7E+2nmuBHv1sGy0bRTOhZ2sKS7wsWL6Dhz/9sZKm2b/pzpvr9/NLptgv+c3uE7hXaXw+5Qq0E2dIW7rN/G8n+PE1HUJAUW20uaqq/6rmZ3VN0/wWzKi4PM8gtoVIQhMdiLNbGZxqWyhN08ptl1BVtbGmaaf9K+kc3erpPpLAREH8DmywWkht8GkMpapqCqJUTLyqqv2Br4HrNE3zZ+SnNJSkOzYzlK+zfG5EvoiTmqYdBV4AXvGXKMXlaYCf8lJIbEVbqwXUFl8N1UTTtC9LX2iathBo5B9JgIjnkkiaWS2gtvhqKF1V1WjEQBFVVVsAYX5TBS38eG2JfQhaQy1EVAhspqrqfESCyoV+UyWSvUgktjOUT5MSmqYtUlX1v0AqYu//5LJdQD8gY3UkEKyGMtiBGDcV4f+Zl6p3BEpCjSZWC6gtPnX5VFVNBTTgQWAOsFNV1eF+1BXIIVGS+qPS1p1Ax9df3MeAEZqm7QBQVfUyxLR5Pz/pkiGlErDhH1ZfBeulZgLQNG2zqqr+jFQPeUPFUJi36ZL5W5o4ckI2fZoXR6EoI2YfagqOLZ0c2KCq6nTgH4hKGXcA/iw6HPKGyiMy5p6zkxqvjH+mjaIQa7Uei8i2WkBtqamVSQd+Be4GngTOIMqBuoE/+1FXyBsKYHVRp663ZN+xU9epeotw8GO7z11TcKxVGxALLLpvwPF2wYB+HcLSv3s09tOgqvTnI5Uz6wQ4vgbHRiLWoOIQUcBhQEdN02b7SddJP13XljyW+9uhKWEnVt4SvWGk1VrqGdv9Hvg6KbEYUVy6JbAFUcJxpZ80gQ2/SH9za/adI9uHpa8eHLHfn8sVgcaRmk8JLHzt0vVG5Hf4D2LT3xD8G81wwo/Xti3DTj8wdH9Jgi3z1dWRoDXUMU3TioHdQHdjCj3ef7KkoarCi8PRLeORnpnemFDZUx+0hjqrquok4EfgRlVVe2B+uc9z6G5nJjYckNYHeUTGdMqY2ypfD99ntZZ6IGgN9SdEt+9LxDrUKuBZf4kykKUrLsApPe6SbhlzIkt0Jdhb8uA0lKZpezRNm6Fpmq5p2k2apiUA/+dnbaHSrakT+7yJrYacnn5a1zlT5QlthkBaFqQ+c/5Y8mCYvBJmHYN71kDKWN9u1rI3zDkFt39a+T0lDO7bArMreLv7DTBtFzy4DwZNKf/e5FUwdoEvdz7km8DA4WLWmRTTVFSNNFQNrCtup048c/deXa/QPY5vDRNeKn9yXHNwvgfNu8Ev6yGhA9z0NiRdVv1NFAdc/Rw4qpgQVhxwzXOQ0L6Kn3kWdiyB1U/CVY9BrLETo9MYaNYF1jxf08fLA3bWdFKgcTGG8nc0gzSUD3xY2LvPgznXb9R14/+jz21w92po3Kb8ie2GQ1gkfP0ovHktrFoAjjDock31Nxh4DyT1rny8WVe4Yxn0ubXyew2aQlRDOLlLPBzh0Li1eG/YdPjx35B9tKaPtom0LNtFSgRsKmakoXzm73lXDv5n/hCRtnb4dCgpgI2vlj9p23swLwnWG7l1Yo1N0XmZF75wwyQYOQuOV/Ff0WU8tB4IK+dXfi83HQqyIVGFZp3BWwynf4F2I+DSvrDGp+H3el9OCjRqCo7NpuqWSAH8XaPlIJCB3L3rE3886xzRMezXVaO+e2YE25dA51Tod1f5k7xFgAJ3fg7Jl8PRrbCxmlymv/2baMW+mFl5/PTLenhlBBz/CUbOLP+e7oVPp8KYxyEsCr6cAzknYdirsO19yNzvy0ey5XpbTZESlqXD1d1OXXF5VgMTrNJgN67Mun/4znVzv1fDsy5cuaJRkjATQF4GRMZCYRVB3WoqdL4aVjwGmQcrv7+vhs0G298Xj1JaDYC2Q2HhQBh8Pwy+D86egCV3w8kdVV0h+AylaVoV32S98g3SUD6joyi9Mmf3Pdxk1tamYpmjMmdPiK5fzxvFxMG1C8FzfflzImJh3JOQvhvWPAcNTcjqNnwG7PwEigthdJowaspYMdv3RqVx3AnSsqz+3asTgTyGApGhVlILCoiI6pgxt11heFzlNaq45uCIgKIc2Py66JolD6x8kaQ+EN8KmqbAnHSYaoyh2g4TU/G1pUUv6Dgavn0KWvQQ3chdn8KhH6BllUU2VtX+JoFBQBtKdzt3AL9YrcNuZOkN4h85Oy6q3MFRj8ADu6HvHeJ1i15iejvrcOUL5J4Sv/Clj70ryh+vLcMfhD3LxXirFL1EGLpqltT+JoGBHfbsf4bY4CipBceLYhoD6I7wQgUi+ekdGHSvWBNKGXe+Zfj+BfHcdjgM/CP892vYtAgWl9l6Hp8sWqmTP5c/7guJncUEyaIx4vWJHcJILXqJ9aiyJhPkA0tr/YEDhIBuoQwWWy3Aznyj98jWdfJJ3w2eG+DYVmjVV4ylPr4Ptr4lToxvJSYhWvQwV8Cw6bD/WzhsZJ7L3CfGT6lPiwXhz2dW/IkvSMs6a66I+qPa+lCBgOLyOBAhKJdarcWuTIletdYdt3iAotjiD+itpGW9ZbWIuhLwX7DudnqBf1utw84szB8x6Nm8Ud9arcMHCoFPrBZxMQS8oQxs+xcrUJiWc8OIpQXdAn327EvSsuowjRg42MJQutv5I7Ddah1255ozU4ZvK05aY7WOavBbzbH6whaGMnjRagF2R0dR+mbOHHDM22iT1VqqYD9Qhzn5wMJOhnoNmbzloikiLCIl46/qWT0y0LZGvEha1gUXpoDbgJ8R2zqOI1qzGGAkIt70Zc7njvwFuL3MzzYE3kDkmCxA5OkvGyavA1sRyzNHgUxgOiLV+I/GPT/BhyIWAT/LVxbF5ZkDPGq1jmCgpSPr14MJs/MjFG9rq7UgTJBczfipJ+IXPgv4AVARWbjcwAeIELVsxBrWTmAYIoC7P7ARYb7JiJwoR4ARCBO1RJhMN362CLHLYbBxLNt43QURpD0PqDZ1np1aKBBF3nKtFhEMHPPGJ/bJnFXs1ZVTVmsBXqphMqItwiD/ASYiMnBNB94tc04YcBnCLHOMY/cZz8sQf4i7AaMQC8dhCFOWEg1ci8jotcK434fAUER9afAhWNxWhtLdzlPAIqt1BAs7SpLaXZX1v8d13dI/UvnUnJ/kG8Sk1O2ILT0fINI0lx0LbgZK46hKp95TyrzeiWipdgOl0bjRZX5e53wMYWkui9XGc+luyPLhXFVgK0MZzANyrBYRLKwoUrvddfaW7bpOiUUS3KRlHa/hnGxgAGLc8yHQGWHCsmtrVdWSKh2TvYVYy/QCM4xrQPk0DoVlzi99zq3wukZsZyjd7TwGPGG1jmDiX/mDB8zLHfu9Bbc+gag9VhPjED0TLzAJaA0cQ3TxSqsc9gY6GP8uzT6jAU2B3yEmtO5GzCSWnmc6tjOUwVOcb94lJvBw7vhh7xX0WVnPt51JWlbVWZvKkwHcgJjp/QpRkrYlYtavdAwYhtg2vxKYj+jCPQ+cRkxmNENMUPyMmOQAP+SWtKWhdLczD6gUVSm5OG48M3nkxqLk+gpR2oAwiC+sQ2w03QoMApKBj4DxZc75GtGK9UWMgW41zi9GtGr/RcwOngJKc6tdfjEfoCpsNW1eFsXlUYC1iL61xCTC8JbsT3h4c+uw0/39eBsdGExa1loTrjUSMWnxBee7epZhyxYKRM4J4A+ItQOJSZTgCOuckdYtyxtdZaIHk/CYZKaAw7aGAtDdzm3A41brCDZyiWrQKWNuiwI97IAfLn8U/1a/tBRbG8pgHqI/LjGRX/WGTXpmPuwo0ZVfTbysF3CSlpVu4jVXIqa/Le/uQRAYSnc7i4FbkGtTprO7pHnyyNN/Ttd1zNpBO4+0rJUmXSsgsb2hAHS3czeiQojEZL4r7thlUvadu3T9oseq3wF/NUFSQBMUhgLQ3c7XEesOEpN5p6B/v4dzx19M4slMRFfPqmiMeiNoDGXwADKXn1+Ylzt26Ov5A+uy49cL3EFalu1K09QF265DXQjF5UlATFK0r+lcSe35tvHfVw+N2FubwtlTSMt6qebTgoNga6HQ3c4MRBi+rXMTBCojTv956L6SJr6uIT0aSmaCIGyhSlFcnkGIEqZ+qwUcqkRTlH+kyczdCY7cntWc9jJpWffUm6gAIehaqFJ0t3MtkIrckGg6+UREd8qYm5ynh++9wClLgCkXeC+oCVpDAehu52pEAGW+1VqCjQw9tnHXjLSYYl05VuGtr4FJNeSHCFqC2lAAutv5NXA9ItGGxEQOeJskXX56xlldPzde/QhIJS2rwEpdVhK0Y6iKKC7PQMRW6ESrtQQbEyK3bnmv0T83hiv6vaGw1lQdIWMoAMXl6YBI2JFS07mSWvGY7nY+YrWIQCCkDAWguDxNENlzhlitJQgoBO7R3c5qCvWGFkE/hqqIkTnpSuD/Wa3F5uwHhkgzlSfkWqiyKC7PrcBLQKzVWmzGEuD3utspF88rENKGAlBcnhTAg0i7K6meQuBB3e2UQcgXIOS6fBUxtn4MRqSzKrRYTiCzFugvzVQ9Id9ClUVxeVRElY/RVmsJILIQGaZeNorfSapBGqoKFJfnd8DTiNxvocy7wP2621lTZleJgTTUBVBcnkbAQ4iE86EWYLsSmKO7nd9ZLcRuSEPVgLFu9QDgQtQZCma+RxhphdVC7Io0lI8YxpqGMFYji+WYzWpgge52fma1ELsjDVVLFJcnFrgRkXh+kMVyLoaziIoUbt3t/MlqMcGCNNRFoLg83RGV8W5BVLgLdLzAGuBN4N+622lWejCJgTSUCSguTzgix/YERDGvZEsFlScfsXP5I+AT3e00M3GlpALSUH5AcXm6IjKZDkYUM6jPOrZ5iLItaxGt0Ve62ymTgNYT0lD1gOLyNEcYqz/QA2iDqBt7yUVctgg4BOwF9iHqHv0A/Ki7nbKAgkVIQ1mI4vI0RJgrGTElHwM0MJ5jEEXE8oxHNqJ4WCbCSId0tzOkN/MFItJQEomJhHxwrERiJtJQEomJSENJJCYiDSWRmIg0lERiItJQEomJSENJJCYiDSWRmIg0lERiItJQEomJSENJJCYiDSWRmIg0lERiItJQEomJSENJJCYiDSWRmIg0lERiItJQEomJSENJJCYiDSWRmIg0lERiItJQEomJ/H99o4Oh4ckHhAAAAABJRU5ErkJggg==\n",
      "text/plain": [
       "<Figure size 216x216 with 1 Axes>"
      ]
     },
     "metadata": {},
     "output_type": "display_data"
    }
   ],
   "source": [
    "plt.style.use('tableau-colorblind10')\n",
    "autopct = '%.2f%%'\n",
    "textprops = {'color':\"w\",'fontsize': 14,'weight':'bold'}\n",
    "data['Label'].value_counts(normalize=True).plot.pie(figsize=(3,3),autopct=autopct,textprops=textprops,legend=True)"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "## Training and Test Set\n",
    "I'll need to split the data in to a training set and a test set. The training set will account for 80% of the data, and the test set for the remaining 20%."
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 99,
   "metadata": {},
   "outputs": [],
   "source": [
    "# Randomize the dataset\n",
    "data_randomized = data.sample(frac=1, random_state=1)\n",
    "\n",
    "# Calculate index for split\n",
    "training_test_index = round(len(data_randomized) * 0.8)"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 100,
   "metadata": {},
   "outputs": [
    {
     "name": "stdout",
     "output_type": "stream",
     "text": [
      "Training Set\n",
      "(4458, 2)\n",
      "Test Set\n",
      "(1114, 2)\n"
     ]
    }
   ],
   "source": [
    "# Training/Test split\n",
    "training_set = data_randomized[:training_test_index].reset_index(drop=True)\n",
    "test_set = data_randomized[training_test_index:].reset_index(drop=True)\n",
    "\n",
    "print('Training Set')\n",
    "print(training_set.shape)\n",
    "#print('\\n')\n",
    "print('Test Set')\n",
    "print(test_set.shape)"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "A sample of a population has to be a representative of the population, otherwise the results obtained can be faulty or skewed. Thus it becomes very important to check for this criterion before moving forward with the project."
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 101,
   "metadata": {},
   "outputs": [
    {
     "data": {
      "text/plain": [
       "ham     0.86541\n",
       "spam    0.13459\n",
       "Name: Label, dtype: float64"
      ]
     },
     "execution_count": 101,
     "metadata": {},
     "output_type": "execute_result"
    }
   ],
   "source": [
    "training_set['Label'].value_counts(normalize=True)"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 102,
   "metadata": {},
   "outputs": [
    {
     "name": "stdout",
     "output_type": "stream",
     "text": [
      "Training Set\n"
     ]
    },
    {
     "data": {
      "text/plain": [
       "<matplotlib.axes._subplots.AxesSubplot at 0x12da66710>"
      ]
     },
     "execution_count": 102,
     "metadata": {},
     "output_type": "execute_result"
    },
    {
     "data": {
      "image/png": "iVBORw0KGgoAAAANSUhEUgAAANQAAACxCAYAAAC855PxAAAABHNCSVQICAgIfAhkiAAAAAlwSFlzAAALEgAACxIB0t1+/AAAADh0RVh0U29mdHdhcmUAbWF0cGxvdGxpYiB2ZXJzaW9uMy4xLjMsIGh0dHA6Ly9tYXRwbG90bGliLm9yZy+AADFEAAAcOklEQVR4nO2deXgUVdaH30pIAmFJQlgCBggiFMgSNllElpFRQRBkGRdaEZ0RdSxHR1E/YSDjBuq42+rozDAMY7uM67gwAi4gqCCLICAUyCKyhSULIWTrdH1/3ArZkw6pTnV13/d5+im6+nbV6aZ/ufeee+45imEYSCQSa4iw2wCJJJSQgpJILEQKSiKxECkoicRCpKAkEguRgpJILEQKSiKxECkoicRCpKAkEguRgpJILEQKSiKxECkoicRCpKAkEguRgpJILEQKSiKxkEZ2GyBpOBTNY+nmN8PtUmp6XVXVGUB3Xdf/z8r7BjOyh5JILET2UJJAM0RV1WVAa+BlIAO4HSjp3aYCvYAHgAKgA/BX4GIgFXhO1/WXG9ros0UKShJoioDLgE7AEuA1YJyu66dVVX3FfO0gkAz0BQYAbwNdgHOA9xFCdARyyCcJNBt1XTeAI0AscBT4l6qq/wT6AFFmu626rhcBWcBuXdcLgUygsQ02nzWyh5IEmrKOkDjgQaCj+Xw5pUO/kMgWJAUlaUhOAmuBjUAuogdqD+y10ygrUWQasYZD0TwxiLlBV/PYETEJbwM0A5qWOcYChcBpxI8v1/x3BrDPfOw1j3sMt+tog30QSbVIQQUQRfN0BQYDQ8xjKqVzBqs5AmwA1gPfAGsMt+tkgO4lqQYpKAtRNE8LYCwwAeG9SrTRHB9CXB8CHxpu1xYbbQkbpKDqiaJ52iDWUiYCo4BoWw2qnr3AR8C7htv1ld3GhCpSUGeBonkUxMLjLcCVBG4YFyh2Aq8Ciwy364TdxoQSUlB1QNE8LYGbgJkIx4LTKQDeA1423K5VdhsTCkhB+YE5rJsF3IbwwoUiK4E0w+1aabchTkYKqgbMHul+QEO4sR2N0fr31l4wLbvGaPNwRC7sVoGieRoBdwJzEav7EolfSEFVQNE8QxHRzn3stsXpqKraDViECJD1AguBGQiXfhLwqq7rL6qqOhJIM98WC0xHLGq/BfwCpABvIqLS+wGf6Lo+u6E+R12QgjJRNE8C8DjwO0rjyyT14xLEYvPdwHDgfEQEeT9EYPYWVVXfBnoC1+m6fkhV1dnAbwAPcC5wKdAE4fY/BxEt8jMQlIKS0eaAonkuB3YANyPFZCX/AI4DnyLmoV7gG13XC3RdzwO2IkKwDgLPq6q6CPgVpcsQe3Rdz0ZEoKfrup6h63o+QRxIG9Y9lKJ5ooAFiL+gUkjWMxFYpev6g6qqXgvMB06oqhoJxCB6pl2IaI5zdV3PUVX1Xzg4Aj1sBaVonhTEGH2QzaaEMuuB11RV9SLmTS8ANwD/Q4RlPaLr+nFVVf8NrFVVNRNIR0SgO5KwdJsrmmc88G8g3m5bHM67wHTD7TrtT2NVVUcBt+q6fk1ArbKRsJtDKZrn98AHSDFZwRTgS3PhW0IY9VBm/N0CxEKtxFr2AGMNt2un3YbYTVgIStE80Yg1EJfdtoQwJ4AJhtv1jd2G2EnIC0rRPI0RQ7zL7LYlDMgDxhlu15d2G2IXIS0oc8v5+4hNf5KG4RRwWbj2VCHrlDDj8d5CiqmhaQYsUTTPALsNsYOQFJTpgFiIWFiUNDxxwFJF8/Sy25CGJiQFhViRv95uI8KcRGCZonmS7TakIQm5OZSiea5GRCZLgoONwHB/F3+dTkj1UIrmSUUM9STBQ3/gn3Yb0VCEjKAUzZOIcI87fmdtCHKVonnusduIhiAkhnymE2IZ8Gu7bZFUSzEwynC7VtttSCAJlR7qDqSYgp1IYJGieZrabUggcbygzHTHC+y2Q+IXXRC7okMWf4d8BiKHW1DV6lE0TwSwCrjQblskfmMAvzbcri/sNiQQOL2HuhspJqehAAsVzdPcbkMCgWMFpWieDsBDdtshOSs6AX+224hAUFdBXYnIPnMKkYCjbE7vOYhsNAXAIeAvZa6/AtHVXw5sQWSueQ2RxeZjRJTyJuqWums+IhuOxJloiubpYrcRVlMXQUUjcqwdQOSiuAm41nxtGvAI4ge+AlEwbBZCgGV5C8hBiMsFbEOkijqAqJ3kV3FiM/BS7m1yNtGEoIOiLoJSgMmI/GqPmOcGm8ddiNqpgxH7jh40z1dMqP80Ys7zmPn8KEJIvzKf+xtM+SQyS1EoMEXRPBfZbYSV1CXrkYFIKA+lNVFLohLWIXqnO4CLgJLQ/YpewZKNZwfN4xpEVtFD5vOY2oxQNM8ERB0mSWjwFKV/mB1PXXqoQsRqN2WOJb3ErQixpQIvUZpWt2Ivkm8efebxdIXn/iAdEaHFIEXzjLHbCKuwysunmcc7EfOsgORVUzTPWIRoJaHFLLsNsAqrBPWzefwE+BZRRwmsr6V0n8XXkwQHo82dAo7HKkHdBXwNtAZaAg+Y54dadP2SrRmjrLqeJOgIiV7KMdHmiuZZCNxotx2SgFEEnGu4XQfsNqQ+OCJSwgxTCdn0vRJABAncZLcR9cURggImIaMiwoHpdhtQX5wiqGlln3RIiOW/t4wk44mpHHhkEk9N7k90I/FRIhSFh8encujRyWT95Td4ZgyjReOoKi8KsHj6hRhuV7nHggl9K7Wb2CcZw+3in9cNOXPu2oEpHHx0Escem8Jdv+perv36+8bw7JSwzKRVH7oommeY3UbUh6AvZ6NontbA6LLn3v7tcAantGLzgUyaREdy98U9yC3wMu+TH3hqcn/u+lV3dh09ScbpAqYNTKHAW8xNr62p8vq928dxutDLsu2Hz5zbdji7XJum0Y144TcDy52LUBReuWYQf//mJ/Zl5PLUpP68sX4f6Tn5XN6zPb3axTPxVVlQ/Sy4GuHgciRBLyjgKsrYGd8kmsEprdh9LId+jy+hSVQkR+ZPYXyvc3huhc7tI7qx/Ug2/R5bAoA+bwJ92lddaCMyQqF72zg2Hchk0t++qtaAR69IpUNC+Y2mrZvF0LxxFNsOZ7MvI5dGkRF0atmU9Jx85lzWi399t4eDWXkWfPywY6qiee4y3K66LPYHDU4Q1JSyT/KKvOQWePEZBoYBhgGKAifzixhxXhuiIiNYtv0wBV7x/5Ey74NqL9ytTXMaR0USFanwD9cQCot9vPTVTrYcyjrTpn+Hltw+ohubD2SSmpxw5vyxUwXk5Bdxfrs4msY0wlvs4+eMXC7u1pZBnRK5fnFYZiK2gnaIInhVDymCnKCeQymapwkVNhAWeH3c9e56OiY0ZeP9Y/n+/y4HIG3JD3ROFL1IQmw0G+4fS+YTYg4V3yS6yuv3bi8EMqBjIjcN7cKtF3Xl23suo0dSC0AM6169dhDHcwtIW/JDuff6DINb3vyOawZ0Yt7Y3tz7wfek5+TzpzG9eGPDz+w5fsrS7yLMGF17k+AkqAUFDKOKgNnIiAgMoF+HlqhtW3Ait5BCr4/YaNHhTh98Lo0iFA6fzGPawBQWT696fTm30MsX+hHufGc9cbP+w+PLt9E0phEPjROL9n8YpTKgYyL3vLeRrLzCSu9/Y/0+zpnzPq3uf4dnv9zB0M6tGHleW+Yv3cq9vz6f9AVT2PzA5fSuZsgpqZaL7TbgbAl2QVX6YjsmNOXFqwaScbqAc9P+y8An/kerpjF8MHPkmTbf7TtO38eW0Hv+J+w4ks0VvZNJalE5HcYnWw8y+oXPeX6Fzsn8Ip747EcABqckck58Ex4a14cv9CO8vn6fX8bOHdOb9zb/QoHXx4IJqTz9xXZy8r3S21d3LjTLEDkOxwlqSOdWREZEsHzHEfaeOMWG/Rms33+CNs1Lv/8th7IwDCj2GWw6mAlAcnzl/Jetm8UwsGNLmsWInq2gSMy7GkVE8Gu1Hc0bR3GxmoThdrHizksAmDGkC3sfrFyDoF9yApf1aMejS7fSNzmByIgIPvjhAKt2H2VAx5YWfBVhRWPE6MRxBK2gFM3TjNJ9VWcoGXr1aR+PokB0owjOay3yfSxaswefz+CiLm3OrEt1byvmQ/sycivdY/6Evqy7byw3DhE7sS/t0Q6A9ftPsD8zlw82/3LmsXr3UQD2Z+SybMfhSteaO7Y3S348xKYDmWfOFfsMfA4J7QpCRtltwNkQzF6+XlRh34pd6fx0LId+HVqy+YFxREdGcE58LEu3H+JA1mleX7+P6wZ1ZuvsceQWeumb3JK3NvzM8VMF9GwXxyPjU9n4SwYPf7qVv67exQ2Dz+Xpyf2ZlJrMoE6t8Bb7WLBsG9/uPc6XO9PP3Hdk1zasuPMSvth5hFve+K6cTecnxTGxdzLDnlkGwA8HM/H5DPolJ9CrfTzflxGZxG8cGX1eo6BUVb27ptd1XX/aWnPKUeV2+EKvj4uf/4wnJ/VntJpEsc9g0Zrd3P3eRgBufmMtOQVFXDsghebeYl5ZvYtZ74vXWjWL4crUDjSOigRgw/4Mrnx1JY+MT2VISiv2nshlzkeb+Hbv8ToZOmdML77clc4a8327j59izsebefmaQaSfzGfaIseuU9qJI2tL1RhtrqpqTVUTDF3XAxbMqGieZxDbQiThiQE0N9yuymP1IKbGHkrX9XLbJVRVjdd1Pau69hbjyL9QEstQgPMR+Uocg19zKFVVuyFKxcSpqnoB8DkwSdf1HQG0TQpK0guHCcpfL58bkS/iqK7rh4AXgFcDZZSieWKBpEBdX+IYUuw2oK74K6hEXdeXlzzRdf0loEVgTAJEPJdE0sZuA+qKv4IyVFVtjJgooqpqEqLeT6CQvZMEQlhQLwFLgTaqqi5ARAK/FDCrRLIXicRxgvLLKaHr+kJVVX8CxiH2/t9cdggYABJqbyIJA0JTUCbbEPOmIgLveZHh2RKARLsNqCt+DflUVR0H6MC9wFxgu6qqIwJoVzCHREkajqo3sgUx/v5wHwZG6rq+DUBV1f4It/nAGt919siIUgk48A+rvwYbJWIC0HV9o6qqQRupHgo0oTBvQ8KC7xMjcsM2fZqPiEKnlQGrLTi2ZCPPOlVVZwF/RVTKmAEEsuhw2PdQeUQ3ue3UtfFfxj3bSVFoWvs7QpIcuw2oK7X1MseBY8BM4AngJKIcqBv4Y2BNk6ws6nb+jJzpPxrGmfJB4YbXbgPqSm3BsXYN62T+LZPFBUMu6Bp5dNWfmn463G5bbKByIo8gx9/g2GjEGlQzRBRwJHCerutzAmTXsQBd15HMPT1hePdGR1ZMjdk0ym5bGhjH/Q78dUq8hSgu3Q74HlHCcUWAbAJIr71JePGbkzNHbUyYv7pfowMhVZO2Fg7W3iS48HdI1xeR3+G/iE1/wxB1oALF0QBe27EMyrx/yKHiuPV229GAhKygDuu67gV2Ar1MF3pc4MySgqoKL5GN1My07jm+mO1229JAOK5WlL+COqWq6jRgM3CVqqq9sb7cZ1kyceCEtCE4ZTRupmb+uVWREfGL3bY0ACHbQ92OGPYtR6xDrQSeDZRRhttlIHpDSRUc9sW17pc52+szyLDblgATmoLSdX2Xruv36bpu6Lp+ta7rLYFHA2zb1gBf39FsK27f+bLsOw4aRjVLDJ2GQVo2jHum9FzHC+HmFTD7MNz6NXQb49/N2vWFuSfgho/Ln09KhRn/g9lH4Pb1oI4rfa3XVLh7B9y7B4b8vvz7bl4JYx7z586O64Xrs86kWGZF1WwJ8PUdz2dFPXrfcmraD4ZB+dIvcR3gypfLN27WFlxvQ9ue8Mt30LILXP06tO9f802UCBj/HERUcAi3SIYbPoJzBsCBdRDfAab+U1xXiYDxz8K29+CrJ+CSh6GpuROj66XQpgd8/XxtHy8P+LG2RsFGfQQV6PAg2UP5wd/yLxr8VN7o1WdO9JsOM7+C+E7lG3YeAZHR8PlD8O+JsPIxiIiEHlfUfIPBt0L7yhUdGXo7NI6Dj++CxVfA5w9CfhZ0GASxrSCmORzdIR4RjYTgAIbPgs1vQM6h2j7aRtKyQytSwmakoPzk3twpI7pHpq8YH7N1FCNmQXEBrP8HDPxtaaMtb8O2D0TvAdDU3BSdV0NW2+btYdRsOLIFknqXf62zuXtHF4XtWPOSeIC4R0EOtFYhOhZ8Xsj6BTqPFD3a+7f487HW+tMo2KgtODaHqnsiBaicfd9a9gIncOAmMzuYcPK2kT8kPPJ1r9XPDGPre9B9XHlBAfiKAAVu/BQ6DoVDm2B9DblML/+L6MWWPlB5/hTfCYqLYPAtMFSDnCPwxcOw/UMwfKLnuvQRiIyB5XMh9ygM/wdseQcy9/rzkRwpqNqGfL2A3lU8eiEiJwKG6elbFch7hBIGitI/c/ag9HVvb6Qgu/qGLdoLMQHkZUB0NYHs6jjoPh5WPQWZP1d+PSoWIqPgwjvh4PcQ3xGmLiqdk219B57uDn/pLHqu5EGQchGsfkq8Z9ZPwjHSpmd1ljpSULUFx1bxTTYoK4ArbbbBMRQRGdUt48HzDiU+oDcFtcpGp9Jhfnvoc5VwHEx8CTyTy7eJagpjn4DjO+Hr56B5FVndvPlCjO/cCLuWQp+rYdKr0H86HNpYuf2I+2D7R+AthNFpojfrNkZ4+xZXmselk5Zt92/vrAj2TYKf2W2A0zhpNGnRPSMtrjiyceXJUbO2EBEFRbmw8V9iaNZxcOWLtO8HccnQqhvMPQ53mQ7XlOHCFQ9w0nQqpJtT3YOmiJq3r3y9pFQ4bzSselLMxSIiYcfHsP9baFdlkY3qK4gHOUEtKMPt2oYDw0/s5oAvIenx/DFF5U5ePA/u2QkDZojnSanCeZBdxdd7+oT4wZc8dn9R/jwIMYBwNAC07i6OWVV0LCPuhV3L4EiZOsVGsRB01bxX22cMVoLZy1fC/4Cb7TbCaezMb9EGwEApViCSH96EIbeJNaFuY0t7hm9eEMeUEcLB8NPnsGEhvFVm63lcR9FLHf2x9PyaF8Ww8YrnIPVa0asVF8GGCk6O1t2Fg2ThpeJ5+jYhpKRUsR51pHwxcCAf+LjiSacQ1D2UyZt2G+BkNkb2PG4YGBzfCZ6pcHgTJA8Qc6kP74BNr4mGccnCCVHRPV4dx3bA61fBMR2SL4CMPeL50QprscNnwd5VYvEXIHOPmD+NexpangufPlDxyktJyz5Vn89sJzXWhwoGFM0TAewHzrHbFqfibvbmytubfDWy9pZBwfWkZb9mtxFnS9D3UIbb5QPesNsOJ6OdumbkZ4XqSrvt8INC4EO7jagPQS8oE4/dBjidS7P/MEL3tvnGbjtqYRlp2SftNqI+OEJQhtu1CZEKWnKWGChKauacAcd9TTfZbUsN/N1uA+qLIwRlUmt4sqRmCoiK6ZrxYOc8I+onu22pgr3AR3YbUV+cJKjFyK3x9SbLiI07P2NerNdQDtttSwVeJC272oUpYDpiO0cecASRCrwJMAoRb/oKpbkjfwFuKPPe5ojfzzGgAJGn//oyrxvAJkT+yUOIHeOzEKnGN5v3/Ag/ilgEvZevLIrmmQc8aLcdocAFjfbtXBv/RJKiBLQSpb+cBDqSll1dEGIfxA8+G/gWEVZ1LiLh6rvAl4gss/nAdmA4IoD7AmA9Qnw3I3aBHwRGIkTUDiEyw3xvEWIf3oXmuRzzeQ9EUqL5QI2p85zUQwG8iEyCaQnrvCndppycudswgiJ3x8s1iAlErV0FkXVrCiID1yzgP2XaRAL9EWKZa567wzwuAR4CegIXA5+Y7csGeDcGJiIyen1h3u994CJEfWnwo5C6owRluF0ngIV22xEqvF/Yt999uZPWGYatueTzgWdqafMlYn/cDUAGolfyAhvKtNlIaZhayVysW5nn2xE91U6gJBq3cZn3G4hcKVCay6IkprBkN2RMLXY6S1AmjyLyq0ss4Mm8S4YtzB9q5xrVi6Rl15bYNAcYhJj3vA90RyQJKru9p6paUiVzstcQa5k+4D7zGlA+jUNhmfYlx9MVnteK4wRluF2HEZNPiUX87tT1o1YVdrFDVOmIoVhtjEWMTHzANKADcBgxxCvZgNoX6GL+uyT7jA60Aq5BOLRmIuIES9pZjuMEZfIkMgrdUkZl/3H4nuLENQ1829l+LuRmAFOBRYgtPesQDoUfEbu6QcyJvkPsoVuAGMI9D2QhnBltEA6KHxFODghAbklHCspwu/KA2XbbEUr4iIjomTGvb6avSUNlm1oH1LD/vhxrERtNNwFDgI7AB8CEMm0+R/RiAxBzoOvN9l5Er/YTwjt4gtI529B6fYIqcJTbvCyK5lGANYixtcQiEpVTmQcTZ2fFKN7OAbyNAQwlLduKbe6jEE6LpZQO9WzDkT0UnMk58TtkymZLOWE0S+iZ8aeoYkMJ5CL6vy0SU9DhWEEBGG7XFgKfwTbs2O1rkzwy6+4ThhEQb+phxBpSSOJoQZnMR4zHJRbytbdLj2k5N+4wDEvLcvqA60jLtrKQ2gqE+9v24R6EgKAMt8sLXEfpmoHEIt4suGDg3NNXWOn5m09adiCLnduO4wUFYLhdO4Hf19pQUmcePT32otfzB66w4FKrgT9bcJ2gxrFevqpQNM9zwB/stiMUWRv/+KpBUT+fbeHsDKAvadmOq6ZRV0KihyrDPYj1CInFXJh174X7ixO+O4u3+oAbw0FMEGI9FICieRIRTopArqOEJbEUnD6c+MDeFhH51eZPrgKNtOwXA2ZUkBFqPVRJRPpERMiJxEJOExPbNePPbQuNSH/TJD8cTmKCEBQUnFmfGoOIUpZYyFGjRavUzDmKz1CO19L0FdKy5zWIUUFESAoKwHC71gLjke50y9lRnNRxdPYf0g2j2u/2XcLU6xqyggIw3K6vEEGVBXbbEmqsKFJ7zsiZvs0wKK7w0ueAq5b8ECFLSAsKwHC7lgOTkVvnLWdxwZAL5p8eUzbX33+B8aRlh+0fsJDz8lWHonkGI7ZCt7bbllDj3Ravrpgcs2kPMJO07Io9VlgRNoICUDRPF0Q1j6522xJiPGS4XWl2GxEMhJWg4Mw61YeIVFGS+lEEzDTcrkV2GxIshPwcqiLmOtVoQiDtr83sBYZJMZUn7Hqosiia5zrgr0A1lZsl1fAu8FvD7aopl15YEtaCAlA0T1fgdUTaXUnNFAD3GG5XWEU/1IWwG/JVxHC7diHmUw8ht9PXxFpgkBRTzYR9D1UWRfN0A15CzLEkgixEhqlXzOJ3khqQgqoCRfNcCzwNJNlti828AfzRcLtqy+wqMZGCqgZF88QB9yMSzlueEDHI+QqYa4ZuSeqAFFQtmOtWswCN0BfW10Ca4XbJTZpniRSUn5QR1m1AnM3mWM0q4FHD7VpqtyFORwqqjiiaJxa4GpF4fojN5tSHXMQcyW24XZvtNiZUkIKqB4rm6Y0QlgtIsNkcf/AhhnWLgTcNt0uWBbIYKSgLUDRPI0SO7cnAOEQy+2AhD1iO2FrxseF2yTrFAUQKKgAomqcncBmiusMgGlZgeYhqfmsQc6Plhtsldy03EFJQDYCieZIQwhqMKICcYj7qM0wsAvYDexCBqlsQItpsuF1F9biupB5IQdmIonlaIISVjHDJx1Z4gKhBm4+olJ6FSBr5M3DAcLvCejNfMCIFJZFYSNgHx0okViIFJZFYiBSURGIhUlASiYVIQUkkFiIFJZFYiBSURGIhUlASiYVIQUkkFiIFJZFYiBSURGIhUlASiYVIQUkkFiIFJZFYiBSURGIhUlASiYVIQUkkFiIFJZFYiBSURGIhUlASiYVIQUkkFiIFJZFYyP8D5nh748W7xjkAAAAASUVORK5CYII=\n",
      "text/plain": [
       "<Figure size 216x216 with 1 Axes>"
      ]
     },
     "metadata": {},
     "output_type": "display_data"
    }
   ],
   "source": [
    "print('Training Set')\n",
    "training_set['Label'].value_counts(normalize=True).plot.pie(figsize=(3,3),autopct=autopct,textprops=textprops,legend=True)"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 103,
   "metadata": {},
   "outputs": [
    {
     "data": {
      "text/plain": [
       "ham     0.868043\n",
       "spam    0.131957\n",
       "Name: Label, dtype: float64"
      ]
     },
     "execution_count": 103,
     "metadata": {},
     "output_type": "execute_result"
    }
   ],
   "source": [
    "test_set['Label'].value_counts(normalize=True)"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 104,
   "metadata": {},
   "outputs": [
    {
     "name": "stdout",
     "output_type": "stream",
     "text": [
      "Test Set\n"
     ]
    },
    {
     "data": {
      "text/plain": [
       "<matplotlib.axes._subplots.AxesSubplot at 0x12d9bf110>"
      ]
     },
     "execution_count": 104,
     "metadata": {},
     "output_type": "execute_result"
    },
    {
     "data": {
      "image/png": "iVBORw0KGgoAAAANSUhEUgAAANQAAACxCAYAAAC855PxAAAABHNCSVQICAgIfAhkiAAAAAlwSFlzAAALEgAACxIB0t1+/AAAADh0RVh0U29mdHdhcmUAbWF0cGxvdGxpYiB2ZXJzaW9uMy4xLjMsIGh0dHA6Ly9tYXRwbG90bGliLm9yZy+AADFEAAAcdElEQVR4nO2deXgUVdaH30pIIAuEsIYl7FIg+yaoLO7ioCxuKC3uMjq2Mwo6CjpEHPdPncUeHNx1aBUVcQNFRdlkJ4IiWAKyBpAlSScha6fr++NWSIeEpEOqU13d932ePJWuul19utO/3HvPPfccRdd1JBKJOURZbYBEEk5IQUkkJiIFJZGYiBSURGIiUlASiYlIQUkkJiIFJZGYiBSURGIiUlASiYlIQUkkJiIFJZGYiBSURGIiUlASiYlIQUkkJiIFJZGYSAOrDZDUH4rTbermN93lUKq7rqrqzUAPTdMeMvN1QxnZQ0kkJiJ7KEmwGaaq6ldAS+AlIBO4Gyjr3a4GegPTgSIgFfgvcAHQD/iXpmkv1bfRp4sUlCTYlACXAh2BRcBcYIymafmqqs4xrmUA7YH+wCDgA6Ar0A5YgBCiLZBDPkmwSdc0TQcOAfHAYeAtVVXfAPoCMUa7LZqmlQDZwE5N04qBLKCRBTafNrKHkgQbf0dIEjAL6GA8/pryoV9YZAuSgpLUJznAWiAdOI7ogdoCu6w0ykwUmUas/lCc7ljE3KC7ceyI+G/dHEgEGvsdGwKFQAGQ73c8DOwG9hg/u4GdustxuP7eieRUSEEFEcXp7gqc7ffTh+CNCg4h/vOvA1YDa3WXwxOk15KcAikoE1Gc7ibAH4DxwPlAKwvN8SHE9Rnwme5y/GShLRGDFFQdUZzu1sAEykUUa61Fp2Q3Qlwf6C7HCottCVukoE4DxelWEAuPdwHjsJ9z5xfgVeAt3eU4arUx4YQUVC1QnO5k4GbgToRjwe4UIxZOXbrLsdJqY8IBKagAUJzuVsADiB4pwWJzgsVS4FHd5VhmtSF2RgqqGowe6UHgHsQqv63RW/7J3BumeaqNNo9E7Db2rxcUpzsGuBeYATS12ByJjZCCOgnF6T4HmIOIgJbUAVVVuwNvIgJkvcDriDmoD0gBXtY07T+qqo4C0oynxQM3IuZ384B9QCfgPcTfZACwUNO0GfX1PmqDFJSB4nQ3BZ4GplAeXyapGxcDG4GpwAjgTEQE+QBEYPZPqqp+APQCbtA07YCqqjOAawA30AW4BIhDhCe1Q0SL7EGMHkIOGW0OKE73WGAb8EekmMzkNeAo8CXgRPRSqzRNK9I0rQDYggjBygD+rarqm4i1vLII9N80TfMgItB/1zQtU9O0QkI4kDaieyhjrvQMcJ/VtoQp44AVmqbNUlX1euBJ4JiqqtGIWMVewHbgU6CLpmm5qqq+hY0j0CNWUIrT3QExRh9mtS1hzAZgrqqqXsS86UXgJuALREDw45qmHVVV9X/AWlVVs4DfERHotiQi3eaK0z0GeBtoZrUtNucjYLLucuQH0lhV1fOAOzVNuy6oVllIxM2hFKf7HsQQQ4qp7lwJfGcsfEuIoB7KiL97CrFQKzGX34DLdJfjV6sNsZqIEJThfHgNmGy1LWHMMWCc7nJ8b7UhVhL2glKc7jhEAOilVtsSARQAf9BdjqVWG2IVYS0oxeluCHyCFFN9kgdcqrscq6w2xArC1imhON0NgPeRYqpvEoFFitM9yGpDrCAsBWU4IF4DxlptS4SSBHylON19rTakvglLQSFi8m602ogIpxnwpeJ0t7PakPok7OZQitM9CRFYKQkN0oERgS7+2p2w6qEUp7sf8IrVdkgqMBB4w2oj6ouwEZSxu/YjwmBnbRhyreJ0T7PaiPogLIZ8itMdBSwERltti+SUlAIjw92dHi491L1IMYU60cBbitMd1iMI2wtKcbrPAB632g5JQHQDnrXaiGBi6yGfMdRbBgy32hZJwOjAxbrLscRqQ4JBoD2UjqgEEWrcgxST3VCA14088GGHbYd8itPdCbGlWmI/OgAzrTYiGNhWUIi9TWE9wQ1z7lGc7i5WG2E2tRXUeEQ6pzxErFyM37WHEemdioADwP/53X8pYtj4B+AnRCqouYi0UJ8jwv43IWqu1ojidA8GJtbSdkloEUsYOihqI6hYRNLC/YjkLrcC1xvXJiE8bXEI8SQA9yME6M88IBchLgfwMyL32n6gH4FX+34Wme4rHLhKcbrDag5cG0EpiBwCIyh3Uw81jtsRxYiHIrZLzDLOn3HSPV4AzkEEr4Iob9kPkYsNAsjWqjjdf/BrL7E/z1ttgJnURlA6wkUN5UWGy+Yw64FvEV63dYjhHkCjk+7xnXHMMI5rEGl6DxiPGwZgx1OBmyyxAWcpTnfY7FmrjaCKEeEj+B3Lhl13IsTWD5hNeZ7qk4dlZa53n3HMP+lxtShO92UEOM+S2Ir7rTbALMzy8jmN418Q86xgJSp8IEj3lVjLRYrT3d9qI8zALEHtMY4LERXI7zIeJ5p0/7KtGXLuFL6ERS9llqDuBb4HWiJ2ak43zp9t0v1BzM8k4ctExelOtdqIumKLWD7F6U4CDiLc8pLwJU13OR6z2oi6YJdIiQlIMUUCts8DYpfqGxWSy6cmx+O6dggjurYkv7iUeel7mP7pJoq9PqIUhVlj+nLb2V2Jj41m4c8HuOu9deQUllR54wn9UnlsTF86N0/kt2N5zPh0E59vyThxfUzvdrxw5UA6JCewbs8xbn9nDdsP5wJwQffWvHz9UJonNOTV1Tt4YMEPJ5730R0jiVJg/MvLg/F5hCtdFaf7HDtvQgz5HkpxulsAF/qf++C2EYzt0569mfkcL/Yy9YKePHKpWBN+/sqBPDK6N3lFJezPzmfS4E788+qqU8T1btuUebcOR23dhNW7jtC5eQLzbx9Bj9YiELpz80Q+vG0E7ZvGszkji5HdWvHJlFFER4nVgBevGcKmjCxumbua+y88k8EdRP2BM1OSGNenPU8s/jlYH0s4Y+vKHCEvKOAq/HrSpnGxDO3Ugp1HchnwzCIGPL2I3MISLu/djuYJDbl7ZHe2HfLQ58mFDHrmC/ZkHqdv26rrTl+kphATHcXjX27hYte33P3+emIbRHNJzzYA3DKsC41iopn2UTrDnlvMhz/spWdKEqO6iWITXVoksu1QDj8dyDYeNwbgkdG9+UY7xPo9x4L5uYQrVxv73GyJHQy/1v9BQYmX40VefLqOroOug6JATmEJI7u1IiY6iq+2HaTI66PI66PTzI8Z/OyXVd742PEiAEp9FR0zZcPDYZ1bALBq1xEAVuw8DMBZncT5347m0aN1E/oYgt15NJduLRtzzYAOPLF4iylvPgJpAwyx2ojTJaTnUIrTnYCIHTxBkdfHvfM34LpmCOkPXkZ8rHgLaYt+ZFCqGHIlx8ey8cHL6NI8kUVbD3D3vPVkFxRXuv+89D1cN6gjfxvdm1FntGJYpxas232UeRvFslrbJOEHyTxeXOHYzjj/5w83MOe6s7i4RwrPLdnKxr2ZvH7DMFbvOsryHYeD8IlEDBcCa6024nQIaUEhAmljTj4ZHRWFDgwwBLQn8zjFXt8Jcd04tAs/ZmRxMKeASYM70bhhA8bOWXbybYhSFKIUhYYx0VzcQwzzMjwFKEbAVKMG0QCUlIrIKK9PHONixPkl2iG6zfr0xP06JCfgGNyJK+YswzGkE8+MG0BBSSlT3l3Ld7/+bsLHETFciE03j4b6kO+8k090SE7gP9cOJjO/iC5pnzD42S9okdCQj6eMOtFm3e6j9H96EX2eXMgvhzxc0ac9KU1OjtOFaRf0ZPSZbXljzU4aT5vH419uYUK/VJ4dPxCAQq8IWSxzQsREi4+roKS00r0AHrrkTDZnZLPqtyO8NPEsPvxhL5v2Z/HqpKFVtpecknMUp7vyH8wG2E5Qwzq3IDoqiq9/OcSuY3ls3JvJhr3HaNW4/PP/6UA2ui7mRpsysgBo37Ty5t5zu7QE4O21u8gr8vLa6h3iRc8QTodDOSKWNzk+tsJxf3blrMJtkuK4ZVhXnli8hTNaNqZxoxi+2HqAb7RDdGnRmKZxsaf9IUQgjYBzrTbidAhZQRn52ypNTsvmQn3bNkVRILZBFN1aCu/am2t+w+fTGd61FbENxFsrc4Hvzjxe6TXK7tW/fTIAvdsI50KZkDbuzQRgRFchsDIBrqvCe/fAhT3ZfiSXT37cf+Jcqa7js0EkSogyquYmoUcoz6EGUcX8aen239lxJJcBqc3YPH0MsdFRtGsaz+JtB9ifnc87G3Zzw1md2TJjDMeLvfRv34x5G/dwNK+IXm2SePzyfqTvy+TvX27htdU7mTiwI8+OH8AVvdsxpGNzAF5dJXqq19fs5N7zVZ6/ciA3De3CsM4t2HrQw7LtFR0OLRIbMuXcM7j9nTUAbD+SS15RCf3bJdOhWQK7j+VV6RSRVEs/qw04HaoVlKqqU6u7rmnaC+aaU4GeVZ0s9vq44N/f8NyEgVyoplDq03lzzU6mfpQOwB3vriW3qITrB3WisbeUOSu3c/8Cca1FYkPG90ulkZ9TYcIry5l5WW+GdmrB/ux8XliQznuGl0/7PYcJryzn+QkDGdA+mRU7DnPbO2sq9Tr3nd+DDE8+76fvBSCvyMvd76/nmXEDKPL6TghNUitq3L0dilQbHKuqanVVE3RN02413ySB4nT/AxHFLolMdCDRbmVwqu2hNE27xf+xqqpNNU3LDq5JJ6iyh5JEDArQC5FewTYENIdSVbU78DGQpKrqEGAJMEHTtF+CaJsUlKQ3NhNUoF4+F2J7+2FN0w4ALwIvB8sow8Nn+81mkjrTyWoDakuggmquadrXZQ80TZsNBDM3dTtk3j0JtLLagNoScLEAVVUbISaKqKqagqj3Eyxs90FKgoLtvgeBCmo2sBhoparqU4h8erODZpUNP0hJULDd9yAgp4Smaa+rqroDGINYbL3DfwgYBKrewCSJNMJTUAY/I+ZNJQTf85IU5PtL7EFzqw2oLQEN+VRVHQNoiESTfwO2qao6Moh2yYQsEhAFKmxFoD3U34FRmqb9DKCq6kCE23xwkOwKKDWzJOwJ5VjTKgnUYL1MTACapqWrqhrMSPWqNxxFEHEUF6xPfjq9ZVRexBaV8xFVLKoe2YeagmObGb+uV1X1fuC/iN7jZkS1jWAR8YIqIDbu9twbWqxq+lyKokTsnDLPagNqS0091FHE2lPZIqt/xTmd4OWjjnhBAazxdlGvyLlr82dNXlIVpVJpoEig6mSKIUxNwbFWbUD0WvS6IcfC4j79puRNWvty4juDFSWoi+mhiO02kQUaHBuLWINKRPRW0UA3TdMeDpJdWUG6ry15tXD40PZR2SvSEhaNqLl1WHHEagNqS6BOiXmIWrhtgB8QpT+XBskmEIUBJH48mn/5iNSorGW3xq225dbw0ySj5iahRaBDuv6ILemfIDb9nYsoWxMspKCq4La8yaO+Ku5ROR9a+BK2gjqoaZoX+BXobbjQg+l5koI6BaM994zc7G230mo76okDNTcJLQIVVJ6qqpOAzcC1qqr2wcTqhCejuxw5lNfflfihoyiDsx4atq+06TqrbakHwraHuhsx7PsasQ61DPhnsIwy2Bvk+9sWL9EN1MxH+xzzxW+22pYgE56C0jRtu6Zpf9U0Tdc0baKmac2AJ4Js209Bvr+tKSA2rlvmY53y9Zhfq2zQ8VxI88CYf5Sf63AO3LEUZhyEO7+H7qOrf5Ghd4EzHaZnwO3fQsfhFa8PnwbTtovr4+dAjF9Qx9n3wAO74L5t0OvK8vPRsTD1Fxj2p0De5u5AGoUSdVlnCvaO2nD/71tnsvX4pO6ZjyaV6FH7KlxISoXxL1VsnNgaHB9A616wbx006woT34G2A6u++YDJMPppaJQE+zdCm37g+BCadRHX+1wDF84EdPDsh37XifZlr3XRLFj+DGxdAFf8CxTjqzbwRoiKgY1v1vT28oFtgX8aoUFdBBXslKhSUAGQ4Utu3TfrYV+prog1mwE3wpTl0LRjxYadR4reYclj8L9xsOxpiIqGnldUfeN+k8Rx7gT431j4/l8QE1fe2wwyMsi9dTn891zIPQh9rhW9VFKquPfhX8RPwyYQ3xyiGsA5f4E1s6GkxinyD6R5bLfAH7KpmJGCCphfStt0HJ49LVPXyWHk/VBaBBteq9jopw/gybawzsitkyDSSlNwijX0Fc/Bwmnwu1Hn6riRLTcmHlCg7QAoyIajv4LPC/s3CMG17gWefeArhVY9oKUKhR7IPwb9rodGTWD9K4G8LVs6XWoKjs2l6p5IAYIaBa27HPsUpzuT4K53hQ1rvF3UcTl3bvpk5T97K1vmN6DHGBh8W8VGvhJAgVu+hA5nw4FNsOEUuUx3Lin/PSoG+htR34d+hPhmQjy5h8rbFBrpGhu3hf3r4ZuZMPKvUFoMn98nrp17nxB0UU4gbyn8BIX16XBXAmMttsE2fFbct/8fl65cOyfRM1g5VRKdJm2FmAAKMiE2AYpzq7mrIuZjKX0hcxdoi8p7N59f7KrPGJ3FGDG8q13ip4w+10Dj1mK4d+lT0Hei6N3m3wY5VTrzwk9QmqbtqS9DTsG3SEHVilcKhw9tH5W1YuZJlR9PkPe7GPr1vRYu/yeMmw3uK6tsCsCY54UYSvLho9tFj+MV1UlQ/DQbZXyVSgqrvs/wqaI3bNVLePg+vkv0YOdNh0+dJ7c+Sprnt4DecIgRynMogG+sNsCOpOVfMWJdTP/KWX0TW4vhW8lxSH8LdB90qKYY3KiHxLCxtBjevxEyNojzBVngLYI4v1w6jYzfq+pteo6F5M6w6t/CWwiwZT4cSIc2/at6Zdv+3UNaULrL8TNygfe0mJ3Zt0eFExfMhGm/wqCbxeOUfsKV7dlf+ckAnUfBqAfF7wvuhB1+Sa50n3BWxDeHFt1F79RuIJQUwOGtle81fBr88L9yxwaAXiruUzUfBfIeQ5GQFpTBF1YbYGeyY1OE5+DH98Sw7eK/w+RPYfLHosGqF8Wx00iY6C53h5//sBBccR70vlJc87++4XVxvOlzsUjcpB38OK+yO/yMS6D1mcLtDnDIWK9vOwBaqMLJUZFCYJFZ77++sYOgbPvfKhT4IKdbq32lTddx9FdwXw0HN0H7QWIu9ek9sGmuaJjUHnpcDil9oFEypBpDwdhEcb7sJ6WPOL9pLiyZZTw3VYhp8fTKBoy4X1zLMXrC3cuFp88xXzg1lj518jO+Is1TudykTai2PlQooDjd0YhhX1urbbErcRQXZDSfvj05qqCv1bYEwE2ked622ojTJeR7KN3lKAXesdoOO1NAbFyXzL93zNdjtlttSw2UAJ9ZbURdCHlBGbxltQF2x4j7a1KiR53CCxESfE2ax9bpD2whKN3l2AJsstoOu2PE/ZWeiPsLPV6quUloYwtBGbxqtQHhwC+lbTqOyJ56TNcJKP6nHtmFjb17ZdhJUG8g8gRK6shqb9ce43P++JuuU2S1LX7MJs1TXQruG4GtQAFwCJEKPA44DxFvOgeRNzIH2Afc5PfcxsDbiCxKRYg8/ZP9ruuIEdAUxLb7LETOycGIIO0CxNyuxqowIe/l80dxumcCs6y2I1z4Y6MVa15KfHdICOT7ywFSSfOcqtfsi/jCe4DVgIrIwuUC5gPfAbmINaxtiLArBRgCbECI7w5ETpQMYBRCRG0QItON55YgNraeY5zLNR73RARpPwlUmzrPTj0UiA/Qdul5Q5U5hSOGPZE/epXVdgBzqhETiFq7CiLr1lWIDFz3A+/7tYkGBiLE8jfj3D3GcRHwGKKq/AXAQqN9F7/nNwLGITJ6fWu83gJgOKK+NAQQLG4rQekuRyYQ0GYaSWD8LX/siLcLz7IyNVkhNecn+Q7YghjGZSJ6JS+w0a9NOlDmwSxzvXf3e7wN0VP9CpTtqvRPb60jcqVAeS6L5caxLPtSwxrstJegDJ5HjGklJnFT7s2jvi3ubpWoXiDNU1O6sFzgLMS8ZwHQAyHCFX5tqqolVTYnmwu8azz+q3EPqJjGodivfdkx/6THNWI7QekuRwYVixZITOAiz19G/uRt+309v+xBoFLsURVcBryO+GJPAlKN5w6kvMphf6Cr8XtZ9hkNaAFcBxxGOB0+92tnOrYTlMGzCE+OxCR0FGVg1vSh+0ubBrvcqz8Pk+YJZE6cCVwNvInY2rEe4VDYChwz2kQjNiUuRYhUB/4NZCOcGa0QDoqtCCcHBCG3pC0Fpbsc+cCDVtsRboh8f2m9snxx9ZHCLR0hkEBYC4xHePqGAR2Aj6m4+XQJohcbhJgDTTbaexG92g6Ed/AYUJZb7ey6vIGqsJXb/GQUp3slwisjMZFmyvHsfc1nHIlXSs4I4suMJM2zouZmNXIewmmxmPKhnmXYsofy425sWJQr1MnUE5qqmY82DmLc37smiSnksLWgdJdjM/C41XaEI/t9ySn9sx72+nTF7OiUvUBAaWPtiK0FZfAkYrIpMZmtpW06jcyeekTXqS4tUm3wAZNJ82SbdD8QTgiFEBjuQRgISnc5yiadtt3lGcp87+3a86qcKTtNivt7ijTP8pqb2RfbCwpAdzm2Ux5mIjGZBcX9+zvzJv6g64EvcFbBWuBRk0wKWcJCUAC6y/EGYbCfJlSZXThq2NMFl57uwm8uMMmOucprS9gIyuDPiPUISRCYcXzciLmFQ5bW8mle4Dq7Jq6sLbZeh6oKxelORqyYd7PalnDl26R/LDs/dnugxbNvJ83zWs3NwoNw66HQXY4sRDSxx2pbwpULPfeO/NmbEsjwb1YkiQnCsIcqQ3G6RyCSZCZYbUs4EkNpya5mj2xuF+0ZfIomr5Hmub1ejQoBwq6HKkN3OVYAY5DFr4NCCdEx3TMfPTO76ri/L4A769umUCBsBQWguxzLEMM/uX8qCOTTML5r5mOpBXrMDr/Ti4GrI8GjVxVhO+TzR3G6L0Ls2mxUU1tJ7UmNyjy4s9nM0hjFtw64njRPsdU2WUVY91Bl6C7HN8DFyKxJQWGfr1mbodl/nQ9cG8liggjpocpQnO6uiIQd3WtqK6kVT+guxyNWGxEKRJSgABSnuxmiokeg6yiSU1MA3G1EqUiIkCGfP0bmpEsQuzslp89O4BwppopEXA/lj+J03wj8hyDkFghzPgFu0l0OuXh+EhHXQ/mjuxxvI3IQ1GdiEjtThEjDNUGKqWoiuocqQ3G6GyCyjc4AGlhsTqiyErhDdzkqF8OWnEAKyg/F6e4FvAicb7UtIUQO8BDwX93lkF+WGpCCqgLF6Z4IPAe0t9oWi5kP3Ku7HKFcpC2kkII6BYrTnYCotHAvomxKJLEEmKG7HOusNsRuSEHVgOJ0twKmIjL1NLbYnGCzHpiuuxxyk+ZpIgUVIMbGxb8gdgUnW2yO2XwHvKC7HJ9bbYjdkYKqJYrT3Ri4HrgVGGqxOXXhOOAGZhv5DSUmIAVVBwyv4K2IPNotLTYnELyIPHYfAO/pLkeo1dm1PVJQJqA43TGIynhjEfuvUq21qAIliIoV84GPdZfjWA3tJXVACioIKE73mYh4wfMRhY/b1uPLFyKcCyuNn+9lVEP9IQVVDyhOdwoixGkQokhYV0QvllSH25YCu4HtiDKX2xElYjboLkdE70myEikoC1Gc7kTE4nEqoiBYI0Qd11jj2BAxZMszfjIR9Y0OA7t0l0NWHgkxpKAkEhOJ6GhzicRspKAkEhORgpJITEQKSiIxESkoicREpKAkEhORgpJITEQKSiIxESkoicREpKAkEhORgpJITEQKSiIxESkoicREpKAkEhORgpJITEQKSiIxESkoicREpKAkEhORgpJITEQKSiIxESkoicREpKAkEhP5fxA0jfLmUbURAAAAAElFTkSuQmCC\n",
      "text/plain": [
       "<Figure size 216x216 with 1 Axes>"
      ]
     },
     "metadata": {},
     "output_type": "display_data"
    }
   ],
   "source": [
    "print('Test Set')\n",
    "test_set['Label'].value_counts(normalize=True).plot.pie(figsize=(3,3),autopct=autopct,textprops=textprops,legend=True)"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "The `train_set` and `test_set` both have similar proportions of ham and spam messages as the original data set."
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "## Data Cleaning\n",
    "To calculate all the probabilities required by the algorithm, I'll first need to perform a bit of data cleaning to bring the data in a format that will allow me to easily extract all the information I'll need.\n",
    "\n",
    "### Letter Case and Punctuation\n",
    "I'll begin with removing all the punctuation and standardizing text to lower case."
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 105,
   "metadata": {},
   "outputs": [
    {
     "data": {
      "text/html": [
       "<div>\n",
       "<style scoped>\n",
       "    .dataframe tbody tr th:only-of-type {\n",
       "        vertical-align: middle;\n",
       "    }\n",
       "\n",
       "    .dataframe tbody tr th {\n",
       "        vertical-align: top;\n",
       "    }\n",
       "\n",
       "    .dataframe thead th {\n",
       "        text-align: right;\n",
       "    }\n",
       "</style>\n",
       "<table border=\"1\" class=\"dataframe\">\n",
       "  <thead>\n",
       "    <tr style=\"text-align: right;\">\n",
       "      <th></th>\n",
       "      <th>Label</th>\n",
       "      <th>SMS</th>\n",
       "    </tr>\n",
       "  </thead>\n",
       "  <tbody>\n",
       "    <tr>\n",
       "      <th>0</th>\n",
       "      <td>ham</td>\n",
       "      <td>Yep, by the pretty sculpture</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>1</th>\n",
       "      <td>ham</td>\n",
       "      <td>Yes, princess. Are you going to make me moan?</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>2</th>\n",
       "      <td>ham</td>\n",
       "      <td>Welp apparently he retired</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>3</th>\n",
       "      <td>ham</td>\n",
       "      <td>Havent.</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>4</th>\n",
       "      <td>ham</td>\n",
       "      <td>I forgot 2 ask ü all smth.. There's a card on ...</td>\n",
       "    </tr>\n",
       "  </tbody>\n",
       "</table>\n",
       "</div>"
      ],
      "text/plain": [
       "  Label                                                SMS\n",
       "0   ham                       Yep, by the pretty sculpture\n",
       "1   ham      Yes, princess. Are you going to make me moan?\n",
       "2   ham                         Welp apparently he retired\n",
       "3   ham                                            Havent.\n",
       "4   ham  I forgot 2 ask ü all smth.. There's a card on ..."
      ]
     },
     "execution_count": 105,
     "metadata": {},
     "output_type": "execute_result"
    }
   ],
   "source": [
    "# Before cleaning\n",
    "training_set.head()"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 106,
   "metadata": {},
   "outputs": [
    {
     "data": {
      "text/html": [
       "<div>\n",
       "<style scoped>\n",
       "    .dataframe tbody tr th:only-of-type {\n",
       "        vertical-align: middle;\n",
       "    }\n",
       "\n",
       "    .dataframe tbody tr th {\n",
       "        vertical-align: top;\n",
       "    }\n",
       "\n",
       "    .dataframe thead th {\n",
       "        text-align: right;\n",
       "    }\n",
       "</style>\n",
       "<table border=\"1\" class=\"dataframe\">\n",
       "  <thead>\n",
       "    <tr style=\"text-align: right;\">\n",
       "      <th></th>\n",
       "      <th>Label</th>\n",
       "      <th>SMS</th>\n",
       "    </tr>\n",
       "  </thead>\n",
       "  <tbody>\n",
       "    <tr>\n",
       "      <th>0</th>\n",
       "      <td>ham</td>\n",
       "      <td>yep  by the pretty sculpture</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>1</th>\n",
       "      <td>ham</td>\n",
       "      <td>yes  princess  are you going to make me moan</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>2</th>\n",
       "      <td>ham</td>\n",
       "      <td>welp apparently he retired</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>3</th>\n",
       "      <td>ham</td>\n",
       "      <td>havent</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>4</th>\n",
       "      <td>ham</td>\n",
       "      <td>i forgot 2 ask ü all smth   there s a card on ...</td>\n",
       "    </tr>\n",
       "  </tbody>\n",
       "</table>\n",
       "</div>"
      ],
      "text/plain": [
       "  Label                                                SMS\n",
       "0   ham                       yep  by the pretty sculpture\n",
       "1   ham      yes  princess  are you going to make me moan \n",
       "2   ham                         welp apparently he retired\n",
       "3   ham                                            havent \n",
       "4   ham  i forgot 2 ask ü all smth   there s a card on ..."
      ]
     },
     "execution_count": 106,
     "metadata": {},
     "output_type": "execute_result"
    }
   ],
   "source": [
    "# After cleaning\n",
    "training_set['SMS'] = training_set['SMS'].str.replace('\\W', ' ')\n",
    "training_set['SMS'] = training_set['SMS'].str.lower()\n",
    "training_set.head()"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "### Creating the Vocabulary\n",
    "Now I'll create the vocabulary, which in this context means a list with all the unique words in the training set."
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 107,
   "metadata": {},
   "outputs": [],
   "source": [
    "training_set['SMS'] = training_set['SMS'].str.split()\n",
    "\n",
    "vocabulary = []\n",
    "for sms in training_set['SMS']:\n",
    "    for word in sms:\n",
    "        vocabulary.append(word)\n",
    "        \n",
    "vocabulary = list(set(vocabulary)) #set ingnores duplicates"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "### The Final Training Set\n",
    "Now I'll transform the vocabulary in to a usable data format"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 108,
   "metadata": {},
   "outputs": [],
   "source": [
    "word_counts_per_sms = {unique_word: [0] * len(training_set['SMS']) for unique_word in vocabulary}\n",
    "\n",
    "for index, sms in enumerate(training_set['SMS']): #enumerate function to get both the index and SMS message\n",
    "    for word in sms:\n",
    "        word_counts_per_sms[word][index] += 1"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 109,
   "metadata": {},
   "outputs": [
    {
     "data": {
      "text/html": [
       "<div>\n",
       "<style scoped>\n",
       "    .dataframe tbody tr th:only-of-type {\n",
       "        vertical-align: middle;\n",
       "    }\n",
       "\n",
       "    .dataframe tbody tr th {\n",
       "        vertical-align: top;\n",
       "    }\n",
       "\n",
       "    .dataframe thead th {\n",
       "        text-align: right;\n",
       "    }\n",
       "</style>\n",
       "<table border=\"1\" class=\"dataframe\">\n",
       "  <thead>\n",
       "    <tr style=\"text-align: right;\">\n",
       "      <th></th>\n",
       "      <th>sold</th>\n",
       "      <th>gotta</th>\n",
       "      <th>reacting</th>\n",
       "      <th>suffers</th>\n",
       "      <th>espe</th>\n",
       "      <th>watches</th>\n",
       "      <th>freaky</th>\n",
       "      <th>rewarding</th>\n",
       "      <th>colourful</th>\n",
       "      <th>choice</th>\n",
       "      <th>...</th>\n",
       "      <th>box61</th>\n",
       "      <th>transcribing</th>\n",
       "      <th>sipix</th>\n",
       "      <th>breathe</th>\n",
       "      <th>valuable</th>\n",
       "      <th>requires</th>\n",
       "      <th>finding</th>\n",
       "      <th>teachers</th>\n",
       "      <th>working</th>\n",
       "      <th>crazy</th>\n",
       "    </tr>\n",
       "  </thead>\n",
       "  <tbody>\n",
       "    <tr>\n",
       "      <th>0</th>\n",
       "      <td>0</td>\n",
       "      <td>0</td>\n",
       "      <td>0</td>\n",
       "      <td>0</td>\n",
       "      <td>0</td>\n",
       "      <td>0</td>\n",
       "      <td>0</td>\n",
       "      <td>0</td>\n",
       "      <td>0</td>\n",
       "      <td>0</td>\n",
       "      <td>...</td>\n",
       "      <td>0</td>\n",
       "      <td>0</td>\n",
       "      <td>0</td>\n",
       "      <td>0</td>\n",
       "      <td>0</td>\n",
       "      <td>0</td>\n",
       "      <td>0</td>\n",
       "      <td>0</td>\n",
       "      <td>0</td>\n",
       "      <td>0</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>1</th>\n",
       "      <td>0</td>\n",
       "      <td>0</td>\n",
       "      <td>0</td>\n",
       "      <td>0</td>\n",
       "      <td>0</td>\n",
       "      <td>0</td>\n",
       "      <td>0</td>\n",
       "      <td>0</td>\n",
       "      <td>0</td>\n",
       "      <td>0</td>\n",
       "      <td>...</td>\n",
       "      <td>0</td>\n",
       "      <td>0</td>\n",
       "      <td>0</td>\n",
       "      <td>0</td>\n",
       "      <td>0</td>\n",
       "      <td>0</td>\n",
       "      <td>0</td>\n",
       "      <td>0</td>\n",
       "      <td>0</td>\n",
       "      <td>0</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>2</th>\n",
       "      <td>0</td>\n",
       "      <td>0</td>\n",
       "      <td>0</td>\n",
       "      <td>0</td>\n",
       "      <td>0</td>\n",
       "      <td>0</td>\n",
       "      <td>0</td>\n",
       "      <td>0</td>\n",
       "      <td>0</td>\n",
       "      <td>0</td>\n",
       "      <td>...</td>\n",
       "      <td>0</td>\n",
       "      <td>0</td>\n",
       "      <td>0</td>\n",
       "      <td>0</td>\n",
       "      <td>0</td>\n",
       "      <td>0</td>\n",
       "      <td>0</td>\n",
       "      <td>0</td>\n",
       "      <td>0</td>\n",
       "      <td>0</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>3</th>\n",
       "      <td>0</td>\n",
       "      <td>0</td>\n",
       "      <td>0</td>\n",
       "      <td>0</td>\n",
       "      <td>0</td>\n",
       "      <td>0</td>\n",
       "      <td>0</td>\n",
       "      <td>0</td>\n",
       "      <td>0</td>\n",
       "      <td>0</td>\n",
       "      <td>...</td>\n",
       "      <td>0</td>\n",
       "      <td>0</td>\n",
       "      <td>0</td>\n",
       "      <td>0</td>\n",
       "      <td>0</td>\n",
       "      <td>0</td>\n",
       "      <td>0</td>\n",
       "      <td>0</td>\n",
       "      <td>0</td>\n",
       "      <td>0</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>4</th>\n",
       "      <td>0</td>\n",
       "      <td>0</td>\n",
       "      <td>0</td>\n",
       "      <td>0</td>\n",
       "      <td>0</td>\n",
       "      <td>0</td>\n",
       "      <td>0</td>\n",
       "      <td>0</td>\n",
       "      <td>0</td>\n",
       "      <td>0</td>\n",
       "      <td>...</td>\n",
       "      <td>0</td>\n",
       "      <td>0</td>\n",
       "      <td>0</td>\n",
       "      <td>0</td>\n",
       "      <td>0</td>\n",
       "      <td>0</td>\n",
       "      <td>0</td>\n",
       "      <td>0</td>\n",
       "      <td>0</td>\n",
       "      <td>0</td>\n",
       "    </tr>\n",
       "  </tbody>\n",
       "</table>\n",
       "<p>5 rows × 7783 columns</p>\n",
       "</div>"
      ],
      "text/plain": [
       "   sold  gotta  reacting  suffers  espe  watches  freaky  rewarding  \\\n",
       "0     0      0         0        0     0        0       0          0   \n",
       "1     0      0         0        0     0        0       0          0   \n",
       "2     0      0         0        0     0        0       0          0   \n",
       "3     0      0         0        0     0        0       0          0   \n",
       "4     0      0         0        0     0        0       0          0   \n",
       "\n",
       "   colourful  choice  ...  box61  transcribing  sipix  breathe  valuable  \\\n",
       "0          0       0  ...      0             0      0        0         0   \n",
       "1          0       0  ...      0             0      0        0         0   \n",
       "2          0       0  ...      0             0      0        0         0   \n",
       "3          0       0  ...      0             0      0        0         0   \n",
       "4          0       0  ...      0             0      0        0         0   \n",
       "\n",
       "   requires  finding  teachers  working  crazy  \n",
       "0         0        0         0        0      0  \n",
       "1         0        0         0        0      0  \n",
       "2         0        0         0        0      0  \n",
       "3         0        0         0        0      0  \n",
       "4         0        0         0        0      0  \n",
       "\n",
       "[5 rows x 7783 columns]"
      ]
     },
     "execution_count": 109,
     "metadata": {},
     "output_type": "execute_result"
    }
   ],
   "source": [
    "word_counts = pd.DataFrame(word_counts_per_sms)\n",
    "word_counts.head()"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 110,
   "metadata": {},
   "outputs": [
    {
     "data": {
      "text/html": [
       "<div>\n",
       "<style scoped>\n",
       "    .dataframe tbody tr th:only-of-type {\n",
       "        vertical-align: middle;\n",
       "    }\n",
       "\n",
       "    .dataframe tbody tr th {\n",
       "        vertical-align: top;\n",
       "    }\n",
       "\n",
       "    .dataframe thead th {\n",
       "        text-align: right;\n",
       "    }\n",
       "</style>\n",
       "<table border=\"1\" class=\"dataframe\">\n",
       "  <thead>\n",
       "    <tr style=\"text-align: right;\">\n",
       "      <th></th>\n",
       "      <th>Label</th>\n",
       "      <th>SMS</th>\n",
       "      <th>sold</th>\n",
       "      <th>gotta</th>\n",
       "      <th>reacting</th>\n",
       "      <th>suffers</th>\n",
       "      <th>espe</th>\n",
       "      <th>watches</th>\n",
       "      <th>freaky</th>\n",
       "      <th>rewarding</th>\n",
       "      <th>...</th>\n",
       "      <th>box61</th>\n",
       "      <th>transcribing</th>\n",
       "      <th>sipix</th>\n",
       "      <th>breathe</th>\n",
       "      <th>valuable</th>\n",
       "      <th>requires</th>\n",
       "      <th>finding</th>\n",
       "      <th>teachers</th>\n",
       "      <th>working</th>\n",
       "      <th>crazy</th>\n",
       "    </tr>\n",
       "  </thead>\n",
       "  <tbody>\n",
       "    <tr>\n",
       "      <th>0</th>\n",
       "      <td>ham</td>\n",
       "      <td>[yep, by, the, pretty, sculpture]</td>\n",
       "      <td>0</td>\n",
       "      <td>0</td>\n",
       "      <td>0</td>\n",
       "      <td>0</td>\n",
       "      <td>0</td>\n",
       "      <td>0</td>\n",
       "      <td>0</td>\n",
       "      <td>0</td>\n",
       "      <td>...</td>\n",
       "      <td>0</td>\n",
       "      <td>0</td>\n",
       "      <td>0</td>\n",
       "      <td>0</td>\n",
       "      <td>0</td>\n",
       "      <td>0</td>\n",
       "      <td>0</td>\n",
       "      <td>0</td>\n",
       "      <td>0</td>\n",
       "      <td>0</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>1</th>\n",
       "      <td>ham</td>\n",
       "      <td>[yes, princess, are, you, going, to, make, me,...</td>\n",
       "      <td>0</td>\n",
       "      <td>0</td>\n",
       "      <td>0</td>\n",
       "      <td>0</td>\n",
       "      <td>0</td>\n",
       "      <td>0</td>\n",
       "      <td>0</td>\n",
       "      <td>0</td>\n",
       "      <td>...</td>\n",
       "      <td>0</td>\n",
       "      <td>0</td>\n",
       "      <td>0</td>\n",
       "      <td>0</td>\n",
       "      <td>0</td>\n",
       "      <td>0</td>\n",
       "      <td>0</td>\n",
       "      <td>0</td>\n",
       "      <td>0</td>\n",
       "      <td>0</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>2</th>\n",
       "      <td>ham</td>\n",
       "      <td>[welp, apparently, he, retired]</td>\n",
       "      <td>0</td>\n",
       "      <td>0</td>\n",
       "      <td>0</td>\n",
       "      <td>0</td>\n",
       "      <td>0</td>\n",
       "      <td>0</td>\n",
       "      <td>0</td>\n",
       "      <td>0</td>\n",
       "      <td>...</td>\n",
       "      <td>0</td>\n",
       "      <td>0</td>\n",
       "      <td>0</td>\n",
       "      <td>0</td>\n",
       "      <td>0</td>\n",
       "      <td>0</td>\n",
       "      <td>0</td>\n",
       "      <td>0</td>\n",
       "      <td>0</td>\n",
       "      <td>0</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>3</th>\n",
       "      <td>ham</td>\n",
       "      <td>[havent]</td>\n",
       "      <td>0</td>\n",
       "      <td>0</td>\n",
       "      <td>0</td>\n",
       "      <td>0</td>\n",
       "      <td>0</td>\n",
       "      <td>0</td>\n",
       "      <td>0</td>\n",
       "      <td>0</td>\n",
       "      <td>...</td>\n",
       "      <td>0</td>\n",
       "      <td>0</td>\n",
       "      <td>0</td>\n",
       "      <td>0</td>\n",
       "      <td>0</td>\n",
       "      <td>0</td>\n",
       "      <td>0</td>\n",
       "      <td>0</td>\n",
       "      <td>0</td>\n",
       "      <td>0</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>4</th>\n",
       "      <td>ham</td>\n",
       "      <td>[i, forgot, 2, ask, ü, all, smth, there, s, a,...</td>\n",
       "      <td>0</td>\n",
       "      <td>0</td>\n",
       "      <td>0</td>\n",
       "      <td>0</td>\n",
       "      <td>0</td>\n",
       "      <td>0</td>\n",
       "      <td>0</td>\n",
       "      <td>0</td>\n",
       "      <td>...</td>\n",
       "      <td>0</td>\n",
       "      <td>0</td>\n",
       "      <td>0</td>\n",
       "      <td>0</td>\n",
       "      <td>0</td>\n",
       "      <td>0</td>\n",
       "      <td>0</td>\n",
       "      <td>0</td>\n",
       "      <td>0</td>\n",
       "      <td>0</td>\n",
       "    </tr>\n",
       "  </tbody>\n",
       "</table>\n",
       "<p>5 rows × 7785 columns</p>\n",
       "</div>"
      ],
      "text/plain": [
       "  Label                                                SMS  sold  gotta  \\\n",
       "0   ham                  [yep, by, the, pretty, sculpture]     0      0   \n",
       "1   ham  [yes, princess, are, you, going, to, make, me,...     0      0   \n",
       "2   ham                    [welp, apparently, he, retired]     0      0   \n",
       "3   ham                                           [havent]     0      0   \n",
       "4   ham  [i, forgot, 2, ask, ü, all, smth, there, s, a,...     0      0   \n",
       "\n",
       "   reacting  suffers  espe  watches  freaky  rewarding  ...  box61  \\\n",
       "0         0        0     0        0       0          0  ...      0   \n",
       "1         0        0     0        0       0          0  ...      0   \n",
       "2         0        0     0        0       0          0  ...      0   \n",
       "3         0        0     0        0       0          0  ...      0   \n",
       "4         0        0     0        0       0          0  ...      0   \n",
       "\n",
       "   transcribing  sipix  breathe  valuable  requires  finding  teachers  \\\n",
       "0             0      0        0         0         0        0         0   \n",
       "1             0      0        0         0         0        0         0   \n",
       "2             0      0        0         0         0        0         0   \n",
       "3             0      0        0         0         0        0         0   \n",
       "4             0      0        0         0         0        0         0   \n",
       "\n",
       "   working  crazy  \n",
       "0        0      0  \n",
       "1        0      0  \n",
       "2        0      0  \n",
       "3        0      0  \n",
       "4        0      0  \n",
       "\n",
       "[5 rows x 7785 columns]"
      ]
     },
     "execution_count": 110,
     "metadata": {},
     "output_type": "execute_result"
    }
   ],
   "source": [
    "training_set_usable = pd.concat([training_set, word_counts], axis=1)\n",
    "training_set_usable.head()"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "## Model Building \n",
    "### Calculate Constants \n",
    "\n",
    "Now that I've cleaned the data and have a training set to work with I can begin building the model.\n",
    "\n",
    "The Naive Bayes algorithm's formula requires the following terms. Precalculating them once saves will save time during model training.\n",
    "- `P(spam)` - probability of spam\n",
    "- `P(ham)` - probability of non-spam\n",
    "- `Nspam` - number of words in spam messages\n",
    "- `Nham` - number of words in non-spam messages\n",
    "- `Nvocab` - number of words in vocabulary\n",
    "- `Alpha` - Laplace smoothing constant\n",
    "\n",
    "Alpha, the Laplace smoothing constant, is required because there can be certain words of the input text that only appear in spam messages and not in non-spam messages. If excluded, calculating `P(word|non-spam)` would yield 0.\n"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 111,
   "metadata": {},
   "outputs": [],
   "source": [
    "# Isolate Spam and Ham messages\n",
    "spam = training_set_usable[training_set_usable['Label']=='spam']\n",
    "ham = training_set_usable[training_set_usable['Label']=='ham']\n",
    "\n",
    "# P(Spam) and P(Ham)\n",
    "p_spam = len(spam) / len(training_set_usable)\n",
    "p_ham = len(ham) / len(training_set_usable)\n",
    "\n",
    "#N_Spam\n",
    "n_words_per_spam_message = spam['SMS'].apply(len)\n",
    "n_spam = n_words_per_spam_message.sum()\n",
    "\n",
    "# N_Ham\n",
    "n_words_per_ham_message = ham['SMS'].apply(len)\n",
    "n_ham = n_words_per_ham_message.sum()\n",
    "\n",
    "# N_Vocabulary\n",
    "n_vocabulary = len(vocabulary)\n",
    "\n",
    "# Laplace smoothing\n",
    "alpha = 1"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "### Calculate Parameters\n",
    "\n",
    "Now that I have the constant terms calculated, it is time to calculate the parameters \n",
    "- `P(w_i|Spam)` \n",
    "- `P(w_i|Ham)`\n",
    "\n",
    "wi stands for each word in the vocabulary. Each parameter will thus be a conditional probability value associated with each word in the vocabulary.These calculations are static like the ones done previously. Calculating them once will avoid meaningless calculations during training phase. \n",
    "\n",
    "Two separate dictionaries, one for spam and one for ham are created, mapping each word to its probability of appearing in the respective label."
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 112,
   "metadata": {},
   "outputs": [],
   "source": [
    "# Initiate parameters\n",
    "parameters_spam = {unique_word:0 for unique_word in vocabulary}\n",
    "parameters_ham = {unique_word:0 for unique_word in vocabulary}\n",
    "\n",
    "for word in vocabulary:\n",
    "    n_word_given_spam = spam[word].sum()   # spam already defined in a cell above\n",
    "    p_word_given_spam = (n_word_given_spam + alpha) / (n_spam + alpha*n_vocabulary)\n",
    "    parameters_spam[word] = p_word_given_spam\n",
    "    \n",
    "    n_word_given_ham = ham[word].sum()   # ham_messages already defined in a cell above\n",
    "    p_word_given_ham = (n_word_given_ham + alpha) / (n_ham + alpha*n_vocabulary)\n",
    "    parameters_ham[word] = p_word_given_ham"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "### Build Model\n",
    "\n",
    "Now that the static components required for the model are calculated, I can build the model. The model will:\n",
    "\n",
    "- Takes an input message (w1, w2, ..., wn).\n",
    "- Calculates P(Spam|w1, w2, ..., wn) and P(Ham|w1, w2, ..., wn).\n",
    "- Compares the values of P(Spam|w1, w2, ..., wn) and P(Ham|w1, w2, ..., wn), and:\n",
    "    - If P(Ham|w1, w2, ..., wn) > P(Spam|w1, w2, ..., wn), then the message is classified as ham.\n",
    "    - If P(Ham|w1, w2, ..., wn) < P(Spam|w1, w2, ..., wn), then the message is classified as spam.\n",
    "    - If P(Ham|w1, w2, ..., wn) = P(Spam|w1, w2, ..., wn), then the algorithm may request human help."
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 113,
   "metadata": {},
   "outputs": [],
   "source": [
    "import re\n",
    "\n",
    "def classify(message):\n",
    "\n",
    "    message = re.sub('\\W', ' ', message)\n",
    "    message = message.lower()\n",
    "    message = message.split()\n",
    "\n",
    "    p_spam_given_message = p_spam\n",
    "    p_ham_given_message = p_ham\n",
    "\n",
    "    for word in message:\n",
    "        if word in parameters_spam:\n",
    "            p_spam_given_message *= parameters_spam[word]\n",
    "            \n",
    "        if word in parameters_ham:\n",
    "            p_ham_given_message *= parameters_ham[word]   \n",
    "\n",
    "    print('P(Spam|message):', p_spam_given_message)\n",
    "    print('P(Ham|message):', p_ham_given_message)\n",
    "\n",
    "    if p_ham_given_message > p_spam_given_message:\n",
    "        print('Label: Ham')\n",
    "    elif p_ham_given_message < p_spam_given_message:\n",
    "        print('Label: Spam')\n",
    "    else:\n",
    "        print('Needs Human Classification!')"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "Test the function with: \n",
    "- WINNER!! This is the secret code to unlock the money: C3421.\n",
    "- Sounds good, Tom, then see u there"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 114,
   "metadata": {
    "scrolled": true
   },
   "outputs": [
    {
     "name": "stdout",
     "output_type": "stream",
     "text": [
      "P(Spam|message): 1.3481290211300841e-25\n",
      "P(Ham|message): 1.9368049028589875e-27\n",
      "Label: Spam\n"
     ]
    }
   ],
   "source": [
    "classify('WINNER!! This is the secret code to unlock the money: C3421.')"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 115,
   "metadata": {},
   "outputs": [
    {
     "name": "stdout",
     "output_type": "stream",
     "text": [
      "P(Spam|message): 2.4372375665888117e-25\n",
      "P(Ham|message): 3.687530435009238e-21\n",
      "Label: Ham\n"
     ]
    }
   ],
   "source": [
    "classify('Sounds good, Tom, then see u there')"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "## Model Testing\n",
    "The preliminary test above looks good. The next step is to test the model on the test data set. Below I'll change the function to return instead of print"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 116,
   "metadata": {},
   "outputs": [],
   "source": [
    "def classify_test_set(message):\n",
    "\n",
    "    message = re.sub('\\W', ' ', message)\n",
    "    message = message.lower()\n",
    "    message = message.split()\n",
    "\n",
    "    p_spam_given_message = p_spam\n",
    "    p_ham_given_message = p_ham\n",
    "\n",
    "    for word in message:\n",
    "        if word in parameters_spam:\n",
    "            p_spam_given_message *= parameters_spam[word]\n",
    "\n",
    "        if word in parameters_ham:\n",
    "            p_ham_given_message *= parameters_ham[word]\n",
    "\n",
    "    if p_ham_given_message > p_spam_given_message:\n",
    "        return 'ham'\n",
    "    elif p_spam_given_message > p_ham_given_message:\n",
    "        return 'spam'\n",
    "    else:\n",
    "        return 'Needs Human Classification!'"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "With the function set to  return labels instead of printing them, I can use it to create a new column in the test set."
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 117,
   "metadata": {},
   "outputs": [
    {
     "data": {
      "text/html": [
       "<div>\n",
       "<style scoped>\n",
       "    .dataframe tbody tr th:only-of-type {\n",
       "        vertical-align: middle;\n",
       "    }\n",
       "\n",
       "    .dataframe tbody tr th {\n",
       "        vertical-align: top;\n",
       "    }\n",
       "\n",
       "    .dataframe thead th {\n",
       "        text-align: right;\n",
       "    }\n",
       "</style>\n",
       "<table border=\"1\" class=\"dataframe\">\n",
       "  <thead>\n",
       "    <tr style=\"text-align: right;\">\n",
       "      <th></th>\n",
       "      <th>Label</th>\n",
       "      <th>SMS</th>\n",
       "      <th>predicted</th>\n",
       "    </tr>\n",
       "  </thead>\n",
       "  <tbody>\n",
       "    <tr>\n",
       "      <th>0</th>\n",
       "      <td>ham</td>\n",
       "      <td>Later i guess. I needa do mcat study too.</td>\n",
       "      <td>ham</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>1</th>\n",
       "      <td>ham</td>\n",
       "      <td>But i haf enuff space got like 4 mb...</td>\n",
       "      <td>ham</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>2</th>\n",
       "      <td>spam</td>\n",
       "      <td>Had your mobile 10 mths? Update to latest Oran...</td>\n",
       "      <td>spam</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>3</th>\n",
       "      <td>ham</td>\n",
       "      <td>All sounds good. Fingers . Makes it difficult ...</td>\n",
       "      <td>ham</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>4</th>\n",
       "      <td>ham</td>\n",
       "      <td>All done, all handed in. Don't know if mega sh...</td>\n",
       "      <td>ham</td>\n",
       "    </tr>\n",
       "  </tbody>\n",
       "</table>\n",
       "</div>"
      ],
      "text/plain": [
       "  Label                                                SMS predicted\n",
       "0   ham          Later i guess. I needa do mcat study too.       ham\n",
       "1   ham             But i haf enuff space got like 4 mb...       ham\n",
       "2  spam  Had your mobile 10 mths? Update to latest Oran...      spam\n",
       "3   ham  All sounds good. Fingers . Makes it difficult ...       ham\n",
       "4   ham  All done, all handed in. Don't know if mega sh...       ham"
      ]
     },
     "execution_count": 117,
     "metadata": {},
     "output_type": "execute_result"
    }
   ],
   "source": [
    "test_set['predicted'] = test_set['SMS'].apply(classify_test_set)\n",
    "test_set.head()"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 118,
   "metadata": {},
   "outputs": [
    {
     "name": "stdout",
     "output_type": "stream",
     "text": [
      "Correct: 1100\n",
      "Incorrect: 14\n",
      "Accuracy: 0.9874\n"
     ]
    }
   ],
   "source": [
    "correct = 0\n",
    "total = len(test_set)\n",
    "\n",
    "for row in test_set.iterrows():\n",
    "    row = row[1]\n",
    "    if row['Label'] == row['predicted']:\n",
    "        correct += 1\n",
    "        \n",
    "\n",
    "print('Correct:', correct)\n",
    "print('Incorrect:', total - correct)\n",
    "print('Accuracy:', round(correct/total,4))  "
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "## Conclusions\n",
    "\n",
    "In this notebook I successfully built a spam filter for SMS messages using the multinomial Naive Bayes algorithm. The filter had an accuracy of 98.74% on the test set, which is an excellent result. There are 14 messages that were misclassified and require further attention, but overall the model learned the training data well and preformed well on the test set. "
   ]
  }
 ],
 "metadata": {
  "kernelspec": {
   "display_name": "Python 3",
   "language": "python",
   "name": "python3"
  },
  "language_info": {
   "codemirror_mode": {
    "name": "ipython",
    "version": 3
   },
   "file_extension": ".py",
   "mimetype": "text/x-python",
   "name": "python",
   "nbconvert_exporter": "python",
   "pygments_lexer": "ipython3",
   "version": "3.7.6"
  }
 },
 "nbformat": 4,
 "nbformat_minor": 4
}