arrhythmia_detector.ipynb

{
 "cells": [
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "# Detecting Cardiac Arrhythmias\n",
    "\n",
    "### Training a deep neural network on ECG signals to detect cardiac arrhythmias.\n",
    "\n",
    "##### Written by Chance Alvarado: [LinkedIn](https://www.linkedin.com/in/chance-alvarado/), [GitHub](https://github.com/chance-alvarado/)\n",
    "\n",
    "##### Explore the full repository [here](https://github.com/chance-alvarado/arrhythmia-detector)."
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "<img src='resources/images/heart_graphic.jpg' />\n",
    "\n",
    "Image courtesy of [Pixabay](https://pixabay.com/)."
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "\n",
    "---\n",
    "\n",
    "## Table of Contents\n",
    "\n",
    "- [Introduction](#intro)\n",
    "    - [What is a Cardiac Arrhythmia?](#i1)\n",
    "    - [What is an ECG?](#i2)\n",
    "    - [Analysis Goals](#i3)\n",
    "- [Setup and Data Overview](#s1)\n",
    "    - [Imports](#s11)\n",
    "    - [Overview of Training and Testing Data](#s12)\n",
    "    - [Target Notation](#s13)\n",
    "    - [Previewing the Data](#s14)\n",
    "    - [Breakdown of Classes for Training Data](#s15)\n",
    "- [Signal Visualization](#s2)\n",
    "    - [Visualizing ECG Signal of a Normal Heartbeat ](#s21)\n",
    "    - [Visualizing ECG Signal of a Supraventricular Premature Beat ](#s22)\n",
    "    - [Visualizing ECG Signal of a Premature Ventricular Contraction ](#s23)\n",
    "    - [Visualizing ECG Signal of the Fusion of a Ventricular and Normal Beat](#s24)\n",
    "    - [Visualizing ECG Signals of an Unclassifiable Beat](#s25)\n",
    "    - [Trends in ECG Signals for Varying Types of Arrhythmias](#s26)\n",
    "- [Model Construction](#s3)\n",
    "    - [Data Preparation](#s31)\n",
    "    - [Training the Neural Network and Tuning Hyperparameters](#s32)\n",
    "    - [Preparing the Testing Data](#s33)\n",
    "    - [Making Predictions](#s34)\n",
    "- [Conclusions](#conclusions)\n",
    "    - [Findings](#findings)\n",
    "    - [Limitations](#limitations)\n",
    "    - [Further Research](#further)"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "<a id=\"intro\"></a>\n",
    "\n",
    "## Introduction"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "<a id=\"i1\"></a>\n",
    "\n",
    "### What is a Cardiac Arrhythmia?\n",
    "\n",
    "Cardiac arrhythmias occur when the electrical impulses of the heart don't function properly. This can cause your heart to beat too quick, too slow, or beat irregularly. For some this may feel like a fluttering or pain in the chest. Arrhythmias may also manifest as anxiety, fatigue, sweating, dizziness, or fainting.\n",
    "<br/><br/>\n",
    "The causes of arrhythmias varies greatly. These can range from smoking, drinking, and genetics to heat attacks, high blood pressure, and diabetes. Certain types of arrhythmias may put one at higher risk of serious medical conditions such as stroke and heart failure.\n",
    "<br/><br/>\n",
    "Preventing cardiac arrhythmias often involves adopting a heart-healthy lifestyle. This includes:\n",
    "\n",
    "- Eating a heart-healthy diet\n",
    "- Staying active and maintaining a healthy weight\n",
    "- Limiting caffeine and alcohol\n",
    "- Reducing stress\n",
    "- Carefully using certain over-the-counter medicines that can trigger arrhythmias\n",
    "<br/><br/>\n",
    "\n",
    "Information above retrieved from [Mayo Clinic](https://www.mayoclinic.org/diseases-conditions/heart-arrhythmia/symptoms-causes/syc-20350668)."
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "<a id=\"i2\"></a>\n",
    "\n",
    "### What is an ECG?\n",
    "\n",
    "An electrocardiogram (ECG or EKG) measures the the electrical activity of the heart. A normal heartbeat presents certain notable features in the ECG signal:\n",
    "\n",
    "- The left and right atria make the first wave, the P-wave.\n",
    "- The left and right ventricles make the following wave, the QRS complex.\n",
    "- Finally, the electrical recovery back to a resting state creates the final wave, the T-wave. \n",
    "<br/><br/>\n",
    "\n",
    "ECGs give medical professionals indespinsable knowledge about the function of the heart. They show the speed and regularity of the electrical activity of the heart. They may also indicate abmormalities in the features of the heart.\n",
    "\n",
    "Information above retrieved from [American Heart Association](https://www.heart.org/en/health-topics/heart-attack/diagnosing-a-heart-attack/electrocardiogram-ecg-or-ekg).\n"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "##### Typical Sinus Rhythm Produced from an ECG"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "<img src='resources/images/ecg.jpg' width=\"400\" height=\"20\"/>\n",
    "\n",
    "Image courtesy of [Wikipedia](https://commons.wikimedia.org/w/index.php?curid=1560893)."
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "<a id=\"i3\"></a>\n",
    "\n",
    "### Analysis Goals\n",
    "\n",
    "The goal of this analysis is to train a deep neural network to accurately detect cardiac arrhythmias from an ECG signal representing a single heartbeat. This model should be able to properly differentiate between a normal heartbeat and four common types of arrhythmias."
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "\n",
    "---\n",
    "\n",
    "<a id=\"s1\"></a>\n",
    "\n",
    "## Setup and Data Overview\n"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "<a id=\"s11\"></a>\n",
    "\n",
    "### Imports\n",
    "\n",
    "The code responsible for this analysis has been refactored and split among multiple scripts found in the `resources/scripts` directory. We will primarily be using functions and classes from `arrhythmia_analysis.py`.\n",
    "\n",
    "- Further explanation on the methods used in this analysis can be found in these scripts.\n",
    "- Feel free to modify and adapt any part of these modules for your projects!"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 1,
   "metadata": {},
   "outputs": [],
   "source": [
    "from resources.scripts import arrhythmia_analysis\n",
    "\n",
    "DataProcessing = arrhythmia_analysis.DataProcessing()\n",
    "ModelEvaluation = arrhythmia_analysis.ModelEvaluation()"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "<a id=\"s12\"></a>\n",
    "\n",
    "### Overview of Training and Testing Data\n",
    "\n",
    "Data necessary for this analysis is divided among two csv files and can be found in the `resources/data` directory:\n",
    "\n",
    "- `mitbih_train`\n",
    "    - This dataset contains 87553 ECG signals of a single heartbeat measured at 187 instances. Each instance notes the signal's normalized amplitude ranging between 0 and 1. The signal's respective arrhythmia type is also noted.\n",
    "    - Modified data from this dataset will be used to train and validate our neural network.\n",
    "- `mitbih_test`\n",
    "    - This dataset contains 21891 instances of ECG signals in the same fashion as `mitbih_train`.\n",
    "    - Data from this dataset will be used to validate the accuracy of our neural network.\n",
    "    \n",
    "Acknowledgements to the rights holders can be found in this project's [README](https://github.com/chance-alvarado/arrhythmia-detector/blob/master/README.md)."
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "<a id=\"s13\"></a>\n",
    "\n",
    "### Target Notation\n",
    "\n",
    "Our data represents varying arrhythmia types as integers ranging from 0 to 4. These values correspond to the American Heart Association's standard heartbeat notation. The standard notation and full name are as follows:\n",
    "\n",
    "- 0: **N**\n",
    "    - Normal heartbeat\n",
    "- 1: **S**\n",
    "    - Supraventricular premature beat\n",
    "- 2: **V**\n",
    "    - Premature ventricular contraction\n",
    "- 3: **F**\n",
    "    - Fusion of ventricular and normal beat\n",
    "- 4: **Q**\n",
    "    - Unclassifiable beat\n",
    "\n",
    "For consistency, we will mostly refer to the arrhythmia types by their integer value in plots and graphics."
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "<a id=\"s14\"></a>\n",
    "\n",
    "### Previewing the Data\n",
    "\n",
    "To get a better feel of the raw data we will be working with, we will first observe a sample DataFrames built on `mitbih_test`."
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "##### Relevant paths to data"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 2,
   "metadata": {},
   "outputs": [],
   "source": [
    "test_dir = 'resources/data/mitbih_test.csv'"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "##### Import a sample of the testing dataset and preview"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 4,
   "metadata": {},
   "outputs": [
    {
     "data": {
      "text/html": [
       "<div>\n",
       "<style scoped>\n",
       "    .dataframe tbody tr th:only-of-type {\n",
       "        vertical-align: middle;\n",
       "    }\n",
       "\n",
       "    .dataframe tbody tr th {\n",
       "        vertical-align: top;\n",
       "    }\n",
       "\n",
       "    .dataframe thead th {\n",
       "        text-align: right;\n",
       "    }\n",
       "</style>\n",
       "<table border=\"1\" class=\"dataframe\">\n",
       "  <thead>\n",
       "    <tr style=\"text-align: right;\">\n",
       "      <th></th>\n",
       "      <th>0</th>\n",
       "      <th>1</th>\n",
       "      <th>2</th>\n",
       "      <th>3</th>\n",
       "      <th>4</th>\n",
       "      <th>5</th>\n",
       "      <th>6</th>\n",
       "      <th>7</th>\n",
       "      <th>8</th>\n",
       "      <th>9</th>\n",
       "      <th>...</th>\n",
       "      <th>178</th>\n",
       "      <th>179</th>\n",
       "      <th>180</th>\n",
       "      <th>181</th>\n",
       "      <th>182</th>\n",
       "      <th>183</th>\n",
       "      <th>184</th>\n",
       "      <th>185</th>\n",
       "      <th>186</th>\n",
       "      <th>187</th>\n",
       "    </tr>\n",
       "  </thead>\n",
       "  <tbody>\n",
       "    <tr>\n",
       "      <th>0</th>\n",
       "      <td>1.000000</td>\n",
       "      <td>0.758264</td>\n",
       "      <td>0.111570</td>\n",
       "      <td>0.000000</td>\n",
       "      <td>0.080579</td>\n",
       "      <td>0.078512</td>\n",
       "      <td>0.066116</td>\n",
       "      <td>0.049587</td>\n",
       "      <td>0.047521</td>\n",
       "      <td>0.035124</td>\n",
       "      <td>...</td>\n",
       "      <td>0</td>\n",
       "      <td>0</td>\n",
       "      <td>0</td>\n",
       "      <td>0</td>\n",
       "      <td>0</td>\n",
       "      <td>0</td>\n",
       "      <td>0</td>\n",
       "      <td>0</td>\n",
       "      <td>0</td>\n",
       "      <td>0</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>1</th>\n",
       "      <td>0.908425</td>\n",
       "      <td>0.783883</td>\n",
       "      <td>0.531136</td>\n",
       "      <td>0.362637</td>\n",
       "      <td>0.366300</td>\n",
       "      <td>0.344322</td>\n",
       "      <td>0.333333</td>\n",
       "      <td>0.307692</td>\n",
       "      <td>0.296703</td>\n",
       "      <td>0.300366</td>\n",
       "      <td>...</td>\n",
       "      <td>0</td>\n",
       "      <td>0</td>\n",
       "      <td>0</td>\n",
       "      <td>0</td>\n",
       "      <td>0</td>\n",
       "      <td>0</td>\n",
       "      <td>0</td>\n",
       "      <td>0</td>\n",
       "      <td>0</td>\n",
       "      <td>0</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>2</th>\n",
       "      <td>0.730088</td>\n",
       "      <td>0.212389</td>\n",
       "      <td>0.000000</td>\n",
       "      <td>0.119469</td>\n",
       "      <td>0.101770</td>\n",
       "      <td>0.101770</td>\n",
       "      <td>0.110619</td>\n",
       "      <td>0.123894</td>\n",
       "      <td>0.115044</td>\n",
       "      <td>0.132743</td>\n",
       "      <td>...</td>\n",
       "      <td>0</td>\n",
       "      <td>0</td>\n",
       "      <td>0</td>\n",
       "      <td>0</td>\n",
       "      <td>0</td>\n",
       "      <td>0</td>\n",
       "      <td>0</td>\n",
       "      <td>0</td>\n",
       "      <td>0</td>\n",
       "      <td>0</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>3</th>\n",
       "      <td>1.000000</td>\n",
       "      <td>0.910417</td>\n",
       "      <td>0.681250</td>\n",
       "      <td>0.472917</td>\n",
       "      <td>0.229167</td>\n",
       "      <td>0.068750</td>\n",
       "      <td>0.000000</td>\n",
       "      <td>0.004167</td>\n",
       "      <td>0.014583</td>\n",
       "      <td>0.054167</td>\n",
       "      <td>...</td>\n",
       "      <td>0</td>\n",
       "      <td>0</td>\n",
       "      <td>0</td>\n",
       "      <td>0</td>\n",
       "      <td>0</td>\n",
       "      <td>0</td>\n",
       "      <td>0</td>\n",
       "      <td>0</td>\n",
       "      <td>0</td>\n",
       "      <td>0</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>4</th>\n",
       "      <td>0.570470</td>\n",
       "      <td>0.399329</td>\n",
       "      <td>0.238255</td>\n",
       "      <td>0.147651</td>\n",
       "      <td>0.000000</td>\n",
       "      <td>0.003356</td>\n",
       "      <td>0.040268</td>\n",
       "      <td>0.080537</td>\n",
       "      <td>0.070470</td>\n",
       "      <td>0.090604</td>\n",
       "      <td>...</td>\n",
       "      <td>0</td>\n",
       "      <td>0</td>\n",
       "      <td>0</td>\n",
       "      <td>0</td>\n",
       "      <td>0</td>\n",
       "      <td>0</td>\n",
       "      <td>0</td>\n",
       "      <td>0</td>\n",
       "      <td>0</td>\n",
       "      <td>0</td>\n",
       "    </tr>\n",
       "  </tbody>\n",
       "</table>\n",
       "<p>5 rows × 188 columns</p>\n",
       "</div>"
      ],
      "text/plain": [
       "        0         1         2         3         4         5         6    \\\n",
       "0  1.000000  0.758264  0.111570  0.000000  0.080579  0.078512  0.066116   \n",
       "1  0.908425  0.783883  0.531136  0.362637  0.366300  0.344322  0.333333   \n",
       "2  0.730088  0.212389  0.000000  0.119469  0.101770  0.101770  0.110619   \n",
       "3  1.000000  0.910417  0.681250  0.472917  0.229167  0.068750  0.000000   \n",
       "4  0.570470  0.399329  0.238255  0.147651  0.000000  0.003356  0.040268   \n",
       "\n",
       "        7         8         9    ...  178  179  180  181  182  183  184  185  \\\n",
       "0  0.049587  0.047521  0.035124  ...    0    0    0    0    0    0    0    0   \n",
       "1  0.307692  0.296703  0.300366  ...    0    0    0    0    0    0    0    0   \n",
       "2  0.123894  0.115044  0.132743  ...    0    0    0    0    0    0    0    0   \n",
       "3  0.004167  0.014583  0.054167  ...    0    0    0    0    0    0    0    0   \n",
       "4  0.080537  0.070470  0.090604  ...    0    0    0    0    0    0    0    0   \n",
       "\n",
       "   186  187  \n",
       "0    0    0  \n",
       "1    0    0  \n",
       "2    0    0  \n",
       "3    0    0  \n",
       "4    0    0  \n",
       "\n",
       "[5 rows x 188 columns]"
      ]
     },
     "execution_count": 4,
     "metadata": {},
     "output_type": "execute_result"
    }
   ],
   "source": [
    "arrhythmia_analysis.sample_dataframe(test_dir).head()"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "##### Discussion\n",
    "\n",
    "Our raw data's first 187 columns represent the ECG signal in question. The last column specifies the arrhythmia type in question. In this instance, our last columns are all 0s, representing normal heartbeats. We can also note many trailing zeros; this likely represents a heartbeat that concluded before the end of the standardized time duration.  "
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "<a id=\"s15\"></a>\n",
    "\n",
    "### Breakdown of Classes for our Training Data\n",
    "\n",
    "Let us further analyze the density of each arrhythmia type in our eventual training data. To do so, we will plot the count of each arrhythmia as a bar graph."
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "<img src='resources/plots/class_bar.jpg' />"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "##### Discussion\n",
    "\n",
    "From this visualization we note a *severe* class imbalance. The vast majority of our data represents normal heartbeats with vary types of arrhythmias paling in comparison. This will need to be addressed later before we begin to train our model - possibly through over or under-sampling."
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "\n",
    "---\n",
    "\n",
    "<a id=\"s2\"></a>\n",
    "\n",
    "## Signal Visualization\n",
    "\n",
    "For details on how these and other visualizations were created please see `resources/scripts/visualization_creation.py`."
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "<a id=\"s21\"></a>\n",
    "\n",
    "### Visualizing ECG Signal of a Normal Heartbeat "
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "<img src='resources/plots/ecg_signal_0.gif' />"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "##### Discussion\n",
    "\n",
    "The ECG signal for our normal heartbeat is relatively unremarkable. We are able to pick out some of the basic structures of the signal (notably the QRS complex). We also note the beat concludes about halfway through the time interval."
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "<a id=\"s22\"></a>\n",
    "\n",
    "### Visualizing ECG Signal of a Supraventricular Premature Beat "
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "<img src='resources/plots/ecg_signal_1.gif' />"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "##### Discussion\n",
    "\n",
    "In this signal of a superventricular prematures beat we notes an abnormal P-wave. This is common this this type of arrhythmia. This type of arrhythmia also usually shows as a wider QRS complex but that is not imeediately obvious in this signal."
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "<a id=\"s23\"></a>\n",
    "\n",
    "### Visualizing ECG Signal of a Premature Ventricular Contraction "
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "<img src='resources/plots/ecg_signal_2.gif' />"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "##### Discussion\n",
    "\n",
    "For a premature ventricular contraction there is often a large 'dip' in the signal. We can assume that this correlates to the low signal start and higher than average active signal before quickly dropping back to 0."
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "<a id=\"s24\"></a>\n",
    "\n",
    "### Visualizing ECG Signal of the Fusion of a Ventricular and Normal Beat "
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "<img src='resources/plots/ecg_signal_3.gif' />"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "##### Discussion\n",
    "\n",
    "To an untrained eye there is not much strikingly different from our example of a normal heartbeat. This makes sense based on the 'fusion' aspect of the arrhythmia. We will have to wait and see if our model is able to pick up on the subtle differences in the signals."
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "<a id=\"s25\"></a>\n",
    "\n",
    "### Visualizing ECG Signal of an Unclassifiable Beat "
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "<img src='resources/plots/ecg_signal_4.gif' />"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "##### Discussion\n",
    "\n",
    "This signal looks much different than our previous signals. Finding a trend in irregular and unclassifiable beats may be dificult as this label acts as sort of a catch-all for irregular heart beats."
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "<a id=\"s26\"></a>\n",
    "\n",
    "### Trends in ECG Signals for Varying Types of Arrhythmias\n",
    "\n",
    "To conclude our initial data exploration we will attempt to note any general trends in the signals of each arrhythmia type. We will plot 100 of each type of arrhythmia as a scatter plot of rounded amplitude versus time and overlap them.\n"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "<img src='resources/plots/ecg_scatter.jpg' />"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "##### Discussion\n",
    "\n",
    "- Arrhythmia Type 0:\n",
    "    - The signals of our normal heartbeats are suprisignly irregular. We do note that the bulk of our data is concentrated in the lower amplitude range.\n",
    "- Arrhythmia Type 1:\n",
    "    - Our visualization clearly highlight the elevated P-wave present in this type of arrhythmia. We also note signals not falling to zero as readily as in other arrhythmia types.\n",
    "- Arrhythmia Type 2:\n",
    "    - We note the telltale 'dip' for this type of arrhythmia clearly. However we can note that many signals in this form do not have this feature as evident. This may be difficult for our model to generalize.\n",
    "- Arrhythmia Type 3: \n",
    "    - The scatter plot of the fusion of a ventricular and normal beat perhaps has the most notable consistent form of all arrhythmias analyzed. These heartbeats seems quick with notable peaks.\n",
    "\n",
    "- Arrhythmia Type 4: \n",
    "    - We note that most of the visualized signals in this plots fall to 0 at around the same time. We are able to note a relatively distinct average upper bound on our signals as well. The pattern for 'unclassifiable' heartbeats is suprisignly consistent."
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "\n",
    "---\n",
    "\n",
    "<a id=\"s3\"></a>\n",
    "\n",
    "## Model Construction"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "<a id=\"s31\"></a>\n",
    "\n",
    "### Data Preperation"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "Before we begin training our model we will need to process our data to enhance the training process. The following processing techniques will be applied to our test data:\n",
    "\n",
    "- Under-sample our overrepresented class, the normal heartbeat. We will randomly select 20000 signals from this class. \n",
    "<br/><br/>\n",
    "- Over-sample our underrepresnted clases, all forms of arrhythmias. We will randomly duplicate signals so we have 20000 signals from each class. This leads to a total training dataset of 100000 signals and their targets.\n",
    "<br/><br/>\n",
    "- Randomly shuffle the rows of the DataFrame holding our training data so our model cannot learn from the order of the signals. \n",
    "<br/><br/>\n",
    "- Add noise to the signals. We will add random uniform noise to every signal. This will ensure that each signal our model trains on is unique in hopes of preventing overfitting.\n",
    "<br/><br/>\n",
    "- Split our DataFrame into a feature matrix of the ECG signals and a target vector of their respective arrhythmia types.\n",
    "<br/><br/>\n",
    "- One-hot encode our target vector. This changes our one-column vector to a binary matrix of five columns.\n",
    "\n",
    "To find more about the specifics of preparing our test data please see `model_training.py` and `arrhythmia_analysis.py` found in `resources/scripts`."
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "##### Raw Signal"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "<img src='resources/plots/ecg_no_noise.jpg' />"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "##### Same Signal with Random Noise Added"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "<img src='resources/plots/ecg_noise.jpg' />"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "##### One-hot Encoding our target vector"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "$$\\begin{bmatrix} 0 \\\\ 1 \\\\ 2 \\\\ 3 \\\\ 4 \\\\ \\vdots \\end{bmatrix} \\rightarrow \\begin{bmatrix} 1&0&0&0&0 \\\\ 0&1&0&0&0 \\\\ 0&0&1&0&0 \\\\ 0&0&0&1&0 \\\\ 0&0&0&0&1 \\\\ \\vdots&\\vdots&\\vdots&\\vdots&\\vdots\\end{bmatrix}$$"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "<a id=\"s32\"></a>\n",
    "\n",
    "### Training our Neural Network and Tuning Hyperparameters\n",
    "\n",
    "Now that we have a robust amount of processed data we can begin to build and train our deep neural network. We will do this through TensorFlow's Keras library.\n",
    "\n",
    "##### Structure of the Network\n",
    "\n",
    "Our deep neural network will follow the following general architecture\n",
    "\n",
    "- A dense layer using the rectified linear unit (ReLU) activation function\n",
    "- A dropout layer \n",
    "- Another dense layer using the ReLu activation function\n",
    "- A final dropout layer\n",
    "- An output layer of five nodes using the softmax activation function.\n",
    "\n",
    "##### Parameter Selection\n",
    "\n",
    "To identify the optimal architecture of our model we will test the following hyperparamters:\n",
    "\n",
    "- Weight sequence of our hidden layers:\n",
    "    - 128, 96\n",
    "    - 128, 128\n",
    "    - 216, 128\n",
    "- First dropout layer rate:\n",
    "    - 0\n",
    "    - 0.25\n",
    "    - 0.5\n",
    "- Second dropout layer rate:\n",
    "    - 0\n",
    "    - 0.25\n",
    "    - 0.5\n",
    "\n",
    "Since we are exhaustively testing these hyperparameters, we will be building and evaluating a total of 27 models and selecting the one that performs best on our validation data.\n",
    "\n",
    "##### Training Process\n",
    "\n",
    "The training of our model will follow the following process:\n",
    "\n",
    "- Select a combination of testing hyperparameters\n",
    "- Build a model with the specified layer units and dropout rates\n",
    "- Train our model until accuracy on our validation set no longer increases\n",
    "- Store the model metrics and repeat testing with the next set of hyperparameters\n",
    "- Rebuild the model with the optimized hyperparameters (based on validation accuracy) and train for 100 epochs."
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "##### Import the best model found from `resources/scripts/model_training.py`"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 5,
   "metadata": {},
   "outputs": [],
   "source": [
    "model = ModelEvaluation.import_best_model()"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "##### Inspect the optimized hyperparameters for our model"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 6,
   "metadata": {
    "scrolled": false
   },
   "outputs": [
    {
     "name": "stdout",
     "output_type": "stream",
     "text": [
      "Input Layer Nodes: 187\n",
      "Dense Layer Nodes: 216\n",
      "Dropout Rate: 0\n",
      "Dense Layer Nodes: 128\n",
      "Dropout Rate: 0\n",
      "Dense Layer Nodes: 5\n"
     ]
    }
   ],
   "source": [
    "ModelEvaluation.best_parameters(model)"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "##### Visualize our Model's Architecture"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "<img src='resources/plots/model_architecture.png' />"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "#### Change in Validation Accuracy and Loss during Training with Optimized Hyperparamters"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "<img src='resources/plots/model_training.jpg' />"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "##### Discussion\n",
    "\n",
    "We note that with our optimized hyperparameters our deep neural network quickly approaches an accuracy in the upper 90s on the validation set. After a quick drop-off the loss on our validation set steadily begins to rise."
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "<a id=\"s33\"></a>\n",
    "\n",
    "### Preparing our Testing Data\n",
    "\n",
    "Now that we have a deep neural network with optimized hyperparamters we can test its accuracy on our testing data. Before this, though, we will need to process our testing data into a form consistent with our training data."
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "##### Construct DataFrame of test data"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 7,
   "metadata": {},
   "outputs": [],
   "source": [
    "df_test = arrhythmia_analysis.create_dataframe(test_dir)"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "##### Split our data into a feature matrix and target vector"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 8,
   "metadata": {},
   "outputs": [],
   "source": [
    "feature_mat_test, target_vect_test = DataProcessing.feature_target_split(df_test)"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "##### One hot encode our target vector to match training samples"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 9,
   "metadata": {},
   "outputs": [],
   "source": [
    "target_vect_test_enc = DataProcessing.one_hot_encoder(target_vect_test)"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "<a id=\"s34\"></a>\n",
    "\n",
    "### Making Predictions\n",
    "\n",
    "Next, we will allow our deep neural network to make predictions on the 21891 signals present in our testing data.\n",
    "\n",
    "**Note:** Due to the ineherently stochastic process of machine learning results may vary slightly from run to run."
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "##### Make predictions on our test data"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 10,
   "metadata": {},
   "outputs": [
    {
     "name": "stdout",
     "output_type": "stream",
     "text": [
      "Accuracy on testing data:  0.9544582366943359\n"
     ]
    }
   ],
   "source": [
    "ModelEvaluation.evaluate_model(model, feature_mat_test, target_vect_test_enc)"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "##### Create and display the confusion matrix for our optimized model"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 11,
   "metadata": {},
   "outputs": [
    {
     "data": {
      "image/png": "iVBORw0KGgoAAAANSUhEUgAAAbUAAAGYCAYAAADFmHw1AAAABHNCSVQICAgIfAhkiAAAAAlwSFlzAAALEgAACxIB0t1+/AAAADh0RVh0U29mdHdhcmUAbWF0cGxvdGxpYiB2ZXJzaW9uMy4xLjMsIGh0dHA6Ly9tYXRwbG90bGliLm9yZy+AADFEAAAgAElEQVR4nOzdeVxU9f4/8NfAjCabIoiACpiKpNcFwW6au2Jd6ybSjVwTwzRy6/66t9tX6YuauaSp5JrmlqWGpJnbV7xlWSAaaCIuoAIuKSiorCObn98fysgwLDNHZuH0ej4e51F85nPmvM/bw7z5nPM5ZxQABIiIiGTAytwBEBER1RcWNSIikg0WNSIikg0WNSIikg0WNSIikg0WNSIikg0WNfpT6tatG/773//izp07EEIgIiLCKNsZP348hBDo37+/Ud5fToQQ2LRpk7nDoAaORY1MqkmTJpgxYwaOHj2KnJwclJSUIDMzE/v378f48eNhbW1t9Bisra3x7bffokOHDvjwww8xduxY7Nq1y+jbNRdPT08IISCEwN69e6vto1QqcevWLQghkJ6eLnlbw4cPN9ofCET6Ely4mGJp166duHDhghBCiJiYGPGvf/1LhISEiPfff1/ExMQIIYRYtGiR0ePo0KGDEEKIf/7zn0bflpWVlWjcuLFQKBRmy7unp6cQQoiioiJRWloqXF1ddfoEBQVp+qSnp0ve1qZNm4QQQtK6jRs3Fkql0mx54iKPRQkiE3jqqaewb98+PP300wgKCsLu3bu1Xv/kk0/g7++Pnj17Gj0WV1dXAMCdO3eMvq0HDx6guLjY6NvRx969ezFixAiMGzcOixcv1nrtzTffxOnTp2FtbQ07OzuTxfTUU0+htLQU5eXlFpMnavjMXlm5yH+ZOnWqEEKIBQsWGLTe8OHDxa+//iry8/NFfn6++PXXX8Urr7yi0y89PV0cOXJEdOzYUezbt0/k5eWJe/fuiZ07d4qWLVtq+h05ckRUx9PTU4wfP14IIUT//v113v/IkSM6I5hevXqJAwcOiJs3bwq1Wi2uX78u9u/fL/76179q+tT0nk5OTmLlypXi6tWrori4WFy9elWsXLlSNG/eXKtfxfoDBw4U7733nrh06ZK4f/++SElJEW+88YZeOawYqa1YsUJ899134ty5c1qvu7q6itLSUjFjxgxx5swZnf3s2bOn2LRpk0hJSRGFhYUiLy9P/PrrryIwMFAnR9UZP368AB6P4pydncWGDRtEZmamKC8vF56engKAEEKITZs2ad4vLCxMCCFEeHi41nbc3NzErVu3xLlz50STJk3MfmxzsayFIzUyiX/84x8AgHXr1um9TlhYGFavXo3z589j3rx5EEIgJCQEe/bswaRJk7B+/Xqt/q1atcJPP/2E3bt349///je6deuGyZMnw8HBAS+88AIA4OOPP0ZsbCxmzZqFzz//HL/88gsA4Pbt2wbtj7e3Nw4fPozMzExERkYiKysLrq6ueP7559GtWzccP368xnUdHBwQFxeH9u3bY+PGjTh58iR8fX0RFhaGQYMG4dlnn0VBQYHWOvPnz0eTJk3w+eefo7i4GGFhYdiyZQsuXbqEuLg4vePeuHEj9uzZg+eeew7x8fEAHk5mKS8vx1dffYWJEyfqrDNixAj4+PggKioKV65cgZOTE8aPH4/du3dj9OjR2L59O4CHubWyskK/fv0wduxYzfpV46vI20cffQRbW1udfa2wZs0aDB48GBEREThy5AhiY2OhUCjw1Vdfwd7eHkOGDIFardZ73+nPw+yVlYv8l+zsbJGbm6t3/2bNmon8/Hxx8eJFYW9vr2m3t7cXly5dEnl5eaJp06aa9vT0dCGEEK+99prW+6xcuVIIIUTHjh01bf3799caQVQshozUpk2bJoQQomfPnrXuR3XvOW/ePCGEEGFhYVp933nnHSGEEHPnztVZ/+TJk0KlUmna3d3dxf3798W2bdvqzGXlkZq1tbW4efOm+PzzzzWvnz9/XuzcuVMAqHakZmNjo/OeTZo0ERcuXBBnz57Vaq/tmlrFa1u3bq329aojtYrjID09XVy5ckU0a9ZMhIeHCyGEmDJlitmPaS6WuXD2I5mEg4MD8vLy9O4fEBAAOzs7fPbZZ8jPz9e05+fnY8WKFZq/1Cv7448/sHPnTq22H3/8EQDQvn37J4heV25uLoCHs/0aN25s0LojRozArVu3dEatn3/+OW7fvo0RI0borLN69WqUlpZqfr5x4wZSU1PRoUMHg7ZdXl6OrVu34vXXX0eTJk3Qu3dv+Pj4YOPGjTWuU1RUpPn/Jk2aoHnz5rCxscGPP/6ITp06wd7e3qAYlixZonffe/fuYfTo0XBzc8PBgwcRERGBPXv2YNWqVQZtk/48WNTIJPLy8gz68Gvbti0A4OzZszqvJScnAwCefvpprfa0tDSdvjk5OQAAJycnvbetjx07duDw4cOYNWsW7ty5gx9++AHvv/8+PDw86ly3bdu2SElJQXl5uVZ7eXk5UlJSdPYLqHnfpOzXxo0b0bRpUwQFBeHNN9/EH3/8gUOHDtXYv0WLFvj888+RmZmJoqIi5OTkIDs7G2FhYQCAZs2aGbT91NRUg/ofO3YMixYtwnPPPYfbt2/jzTffNGh9+nNhUSOTSE5ORtOmTTXFqi4KhcLgbVQtEoa+38OzZtVTKrUvP5eUlGDo0KF49tlnsWDBApSXl2Pu3Lm4cOECAgMD9Q9aTzXtm5Q8XbhwAfHx8ZgyZQqCg4Px5Zdf4sGDBzX2j4mJwfjx4/Hll18iODgYL7zwAoYMGYKvv/4aAGBlZdjHiKHXwVQqleaaaPPmzfX6w4H+vFjUyCS+/fZbAKh2IkJ1Ll++DADo3LmzzmudOnUCUP3o5UlUTPFv3ry5zms1FePffvsN8+bNw9ChQ9G+fXsUFhZi3rx5tW4nLS0NHTt21LnR3NraGt7e3vW+X9XZuHEjevXqBXt7+1qf4tG1a1d0794dCxcuxPvvv4+dO3ciJiYGP/zwQ7U3ytf2h4FUCxYsQM+ePfHvf/8beXl52LFjB2xsbOp9OyQPLGpkEl988QUuXLiAf/3rX3jllVeq7dOjRw/NKa3Dhw+joKAA06ZN07pvys7ODtOmTUN+fj4OHz5crzFWnBareq1u5MiRaNWqlVZbdaf9rl+/jtu3b1dbFCv77rvv4OLiolPg33rrLbi4uOjcw2cMO3bswOzZszF9+nRcvHixxn4VI8SqI8LOnTtXe+2vYiajo6NjvcT54osv4p///Cc2b96MJUuWICQkBB07dsTKlSvr5f1Jfjiln0xCrVbj5Zdfxv79+7Fnzx4cOnQIhw8fRk5ODlq0aIGBAwfihRdewCeffALg4USM999/H6tXr8bx48exefNmAEBISAg6dOiASZMmGTTxRB+pqak4fPgwJk+eDIVCgd9//x3du3fHiBEjcPHiRahUKk3f8PBwDB06FPv27UN6ejoUCgX+/ve/45lnnsGiRYtq3c4nn3yC1157DatWrUKPHj1w6tQp+Pr6IjQ0FBcuXNDkwJjy8/MxZ86cOvudP38eycnJeP/992FjY4OUlBR4e3tj8uTJSE5Ohp+fn1b/+Ph4TJs2DatXr8b+/ftRWlqK48ePIyMjw+AYXV1dsWXLFly8eBFTp04FABw4cADLly/Hu+++i0OHDuGbb74x+H1J/sw+BZPLn2dp0qSJePfdd8Uvv/wi7ty5I0pKSkRmZqbYt2+fGDt2rLCystLqHxgYKGJjY0VBQYEoKCgQsbGxYvjw4TrvW3HzddX26qbv1zSlH4Bo2bKliIqKErm5uSI/P18cOHBA+Pj46Ezp79+/v9ixY4dIT08XRUVFIicnR8THx4vQ0FCt96vpNgFnZ2exatUqce3aNVFSUiKuXbsmVq5cKZycnPRaH6j+hvDqlspT+uvqW92Ufg8PDxEVFSVu3bolCgsLxfHjx0VgYKCIiIjQ3Lhe0VehUIjFixeLa9euibKysmpvvq5p25Wn9CsUCnH48GGhVqtF9+7dtfqpVCqRmJgo7t27J7y8vMx+THOxrEXx6H+IiIgaPF5TIyIi2WBRIyIi2WBRIyIi2WBRIyIi2WBRIyIi2WBRIyIi2WBRIyIi2WBRIyIi2WBRIyIi2WBRIyIi2WBRIyIi2WBRIyIi2WBRIyIi2WBRIyIi2WBRIyIi2bD4b75++J2CREREDykUihpfs/iiBgCoZQeoGo/+EFjBvBlk2qO8hTJvBtnAvElSkbfaPqBJV10DHZ5+JCIi2WBRIyIi2WBRIyIi2WBRIyIi2WBRIyIi2WBRIyIi2WBRIyIi2WBRIyIi2WBRIyIi2WBRIyIi2WBRIyIi2WBRIyIi2WBRIyIi2WBRIyIi2WBRIyIi2WBRIyIi2WBRIyIi2WBRIyIi2WBRIyIi2WBRIyIi2WBRIyIi2WBRIyIi2WBRIyIi2WBRIyIi2WBRIyIi2WBRqxAWBqSlAWo1kJAA9OlTc99NmwAhdJeCAu1+77wDnDsHFBUBFy4A48YZdx/MoEtYGN5IS0OYWo3XExLgXlveALj364fXExIQplbjjcuX8ZfJk7Ve9/vgAwSfOIHJubmYeOsWXv7+ezTv3NmYu2A2A8PCsDAtDWvVanyYkIAOdeTOu18/fJiQgLVqNRZevoz+VXLn3bcvpu3ZgyXXr2ODEHh+/Hhjhm82zJvxhYWFIS0tDWq1GgkJCehTR44tCYsaAAQHA5GRwPz5gK8vEBcHHDwItGlTff8ZMwBXV+3l8mUgKupxn7ffBhYtAubOBTp3BiIigFWrgJdfNs0+mUCH4GD0jYxE4vz52OHri5txcfj7wYOwqyFvDl5eeOXAAdyMi8MOX18kLliAfitWoF1QkKZPqwEDcGb1auzs3Ru7Bw3Cg7IyBP73v2js6Giq3TKJnsHBGBkZiQPz52OOry8ux8Xh3YMH0byG3Dl7eeHdAwdwOS4Oc3x9cWDBAoxesQJ+lXLX2M4OfyQnY/uMGSguKjLVrpgU82Z8wcHBiIyMxPz58+Hr64u4uDgcPHgQbWr6PLQwCgDCVBtzdHTEhg0bMHToUGRnZ+N//ud/sH379lrXEUIACoVxA4uPB5KSgEmTHrelpgLR0cDMmXWv37s3EBv78L/Hjj1si40Fjh8H/t//e9xvyRLgr38F+vat3/irEg//SVcYOW+vxccjJykJP1bK27jUVFyKjsaxavLWe+FCtAsKwlZvb03boPXr0bxzZ0T37l3tNlS2tpiUm4v9gYHI2Lev/neikmmP8hZq7OMNwKz4eFxPSsKWSrmbn5qKhOho7Komd/9YuBA9goIws1Luxq9fj1adO2N+NblblZ+PbVOnInbLFuPsQCUbmDdJKvKmMEHeDBEfH4+kpCRMqpTj1NRUREdHY6Y+n4dGJoSoNWcmHamtWrUKJSUlaNmyJcaMGYM1a9agU6dOpgxBl0oF+PkBMTHa7TExD4uUPt56C0hOflzQAKBxY+D+fe1+ajXw7LOAUvlkMVsAK5UKLn5+uFolb1djYuBWQ95ce/XS7X/oEFz8/WFVQ05U9vawsrZG8d279RO4BbBWqeDp54ezVXJxNiYG7WvIXbtevXT7HzoET39/WMvgeNIH82Z8KpUKfn5+iKmSs5iYGPTW9/PQzExW1GxsbPDqq6/iww8/RGFhIWJjY/H9999jnLmvMzk7PywyWVna7VlZD08r1sXBAXjtNWD9eu32Q4eAN98E/P0f/uznB0ycCDRq9HCbDVwTZ2dYKZUoqpK3oqws2NSQNxtX12r7W6tUeKqGnPSLjMTtU6eQWfkPhgbO3tkZ1kol8qrkIi8rC01ryJ2Dq2u1/ZUqFexkcDzpg3kzPmdnZyiVSmRVyVlWVhZc9fk8tAAmK2re3t4oLy/HxYsXNW2nT59G5zomAaxbt87YoT0kqpyFVSh026ozdixgbQ1s3ard/tFHwP79D6/PlZYCe/YAFac0ysvrJ2ZLUCVHirryVl2eq2sH0OfTT+HWpw8OvPoqxIMHTxqpxRHV5K5qW239K3JX2zpyxLwZn6E5tiQmK2p2dnbIzc3VasvNzYW9vX2t61U+r2sU2dlAWZnuqMzFRXf0Vp233gK+/Raoenrs/n0gNBSwsQG8vAAPDyAjA8jLe7jNBk6dnY0HZWU6o7ImLi46o7EKRZmZOv1tXFxQXlqK+zk5Wu19li6F96hR+G7QIOSlp9dv8GaWn52N8rIyndGFvYuLzqiiQl5mpk5/BxcXlJWWorBK7uSKeTO+7OxslJWV6YzKXFxcdEZvlspkRa2goAAODg5abQ4ODsjPzzdVCNUrLQUSE4GAAO32gICHo6za9OwJdO+ue+qxsrIy4I8/gAcPgJEjgX379BsBWrgHpaW4lZiINlXy1iYgADdryFvmsWNoM2SITv9bCQl4UFamaeu7fDk6jh6N3YMG4W5KSv0Hb2blpaW4kpiITlVy1ykgAJdqyN3lY8fQqUruOgUE4EpCAsor5U7OmDfjKy0tRWJiIgKq5DggIABxdX0eWgiTFbXU1FQolUq0b99e09atWzecPXvWVCHUbOlSICTk4cjKxwdYvhxwdwfWrn34+pYtj08dVjZp0sNZkj//rPtahw4PT022b/+w+G3fDvzlL/rNpmwgfl+6FM+EhKBTaCgcfXzQd/ly2Lq7I/lR3gK2bEFApbwlr10Lu9at0XfZMjj6+KBTaCieCQnBqSVLNH36r1yJThMm4NCoUSi+exc2LVvCpmVLqGxtTb5/xhSzdCmeDwlB39BQuPn4YNTy5Wjm7o6fH+UudMsWhFbK3U9r18KxdWuMXLYMbj4+6BsaiudDQnCoUu4a29qiTbduaNOtGxRWVmju4YE23brVON29IWLejG/p0qUICQlBaGgofHx8sHz5cri7u2NtxeehhTPZ9J+ioiLs2rULc+fOxcSJE9G9e3cMHz7cMmbUREUBTk5AeDjg5vZwJuOwYcDVqw9f9/DQXcfO7uHIa+7c6t/T2vrhdP6OHR+OBo8ceTib8soV4+2HiV2MisJTTk7oGR4OWzc35CQnY++wYch/lDe7KnnLy8jA98OGoe+yZegSFoaCGzdwdPp0XN61S9On65QpAIARP/6ote7x2bNxYs4cI++R6fwWFQU7Jye8HB6Opm5u+CM5GZHDhiHnUe6aV8lddkYGlg8bhpHLlmFAWBju3biBbdOnI7FS7rz8/fH+Tz9pfg6cOxeBc+cidvNmbJwwwST7ZWzMm/FFRUXByckJ4eHhcHNzQ3JyMoYNG4arFZ+HFs7k96lt3LgRAQEByMnJwQcffGAZ96nJjYnuU5MbU96nJiemvE9NTiz1PjVLV9d9aia9UePu3bsYMWKEKTdJRER/InxMFhERyQaLGhERyQaLGhERyQaLGhERyQaLGhERyQaLGhERyQaLGhERyQaLGhERyQaLGhERyQaLGhERyQaLGhERyQaLGhERyQaLGhERyQaLGhERyQaLGhERyQaLGhERyQaLGhERyQaLGhERyQaLGhERyQaLGhERyQaLGhERyQaLGhERyQaLGhERyQaLGhERyQaLGhERyYYCgDB3ELURwqLDIyIiE1MoFDW+xpEaERHJhtLcAeijtqpMuipGtyuYN4NMe5Q3Hm+GEcybJMybNHWdveNIjYiIZINFjYiIZINFjYiIZINFjYiIZINFjYiIZINFjYiIZINFjYiIZINFjYiIZINFjYiIZINFjYiIZINFjYiIZINFjYiIZINFjYiIZINFjYiIZINFjYiIZEPvojZw4EDs3LkTiYmJaNWqFQBg/Pjx6Nevn9GCIyIiMoReRe0f//gH9u7di9u3b+OZZ55Bo0aNAAA2Njb44IMPjBogERGRvvQqarNmzcLbb7+Nd955B2VlZZr2uLg4+Pr6Gi04IiIiQ+hV1Ly9vXH06FGd9ry8PDRr1qzegyIiIpJCr6KWmZmJ9u3b67Q///zzSEtLq/egiIiIpNCrqG3YsAHLly9Hjx49IIRAy5YtERwcjMWLF2PdunXGjpGIiEgvSn06zZ8/H82bN0dcXBxUKhViY2NRVlaGyMhIREZGGjtGIiIivSgACH07Ozg4oEuXLrCyssKZM2dw7949I4b2kBACCoXC6NuREyEe/pOuYN4MMu1R3ni8GUYwb5Iwb9LUVRP0GqlVKCwsRGZmJgAgPz//ySIjIiKqZ3pdU1MqlViwYAHu3buHlJQUpKSk4N69e1i4cCFUKpWxYyQiItKLXkVt5cqVGD9+PGbMmIEuXbqgS5cumDFjBsaNG4cVK1YYO0aLExYWhrS0NKjVaiQkJKBPnz7mDslsuoSF4Y20NISp1Xg9IQHudeTCvV8/vJ6QgDC1Gm9cvoy/TJ6s9brfBx8g+MQJTM7NxcRbt/Dy99+jeefOxtyFBoHHnDTMmzQNOW96FbWRI0diwoQJ2LhxI86fP4/z589j48aNCA0NxciRI40do0UJDg5GZGQk5s+fD19fX8TFxeHgwYNo06aNuUMzuQ7BwegbGYnE+fOxw9cXN+Pi8PeDB2FXQy4cvLzwyoEDuBkXhx2+vkhcsAD9VqxAu6AgTZ9WAwbgzOrV2Nm7N3YPGoQHZWUI/O9/0djR0VS7ZXF4zEnDvEnT0POm10SRzMxMDBgwABcuXNBq9/HxwdGjR+Hi4qLXxqZMmYKQkBB06dIF27dvx4QJE+pcx9ImisTHxyMpKQmTJk3StKWmpiI6OhozZ840Y2SPmWqiyGvx8chJSsKPlXIxLjUVl6KjcayaXPReuBDtgoKw1dtb0zZo/Xo079wZ0b17V7sNla0tJuXmYn9gIDL27av/najEUieKWPoxZ6kTHpg3aRpC3mrLmV4jtTVr1mDmzJla18+USiU++OADrFmzRu9gbty4gXnz5mHjxo16r2NJVCoV/Pz8EBMTo9UeExOD3jV8KMuVlUoFFz8/XK2Si6sxMXCrIReuvXrp9j90CC7+/rBSVj9nSWVvDytraxTfvVs/gTcwPOakYd6kkUPe9CpqnTt3xvDhw3Ht2jUcPHgQBw8exLVr1xAYGAgfHx988803mqU2u3fvxp49e5CTk1MvwZuas7MzlEolsrKytNqzsrLg6upqpqjMo4mzM6yUShRVyUVRVhZsasiFjatrtf2tVSo85exc7Tr9IiNx+9QpZB47Vj+BNzA85qRh3qSRQ970mtJfVlaG/fv3a7UdOXLEKAFVZYlPLKk4bVBBoVDotP1pVJOLqm219UfFaYRq1unz6adw69MH3/bpA/HgwZNG2qDxmJOGeZOmIedNr6I2evRoY8dRo0mTJmFylRly5pKdnY2ysjKdv1hcXFx0/rKRO3V2Nh6UlemMypq4uOiMxioUZWbq9LdxcUF5aSnuVxm991m6FN4jR2L3wIHIS0+v3+AbEB5z0jBv0sghb3qdflywYEG1DzT+syktLUViYiICAgK02gMCAhAXF2emqMzjQWkpbiUmok2VXLQJCMDNGnKReewY2gwZotP/VkICHlT6SqO+y5ej4+jR2D1oEO6mpNR/8A0IjzlpmDdp5JA3vYpaQEAALly4gJ9//hnjxo3DU089Zey4LNbSpUsREhKC0NBQ+Pj4YPny5XB3d8fatWvNHZrJ/b50KZ4JCUGn0FA4+vig7/LlsHV3R/KjXARs2YKALVs0/ZPXroVd69bou2wZHH180Ck0FM+EhODUkiWaPv1XrkSnCRNwaNQoFN+9C5uWLWHTsiVUtrYm3z9LwWNOGuZNmoaeN71OP/r7+6NLly4IDQ3Fp59+ihUrVuCbb77Bhg0bcOLECb03Zm1tDaVSCWtra1hbW6Nx48YoKytDeXm55B0wtaioKDg5OSE8PBxubm5ITk7GsGHDcPXqVXOHZnIXo6LwlJMTeoaHw9bNDTnJydg7bBjyH+XCzsNDq39eRga+HzYMfZctQ5ewMBTcuIGj06fj8q5dmj5dp0wBAIz48UetdY/Pno0Tc+YYeY8sE485aZg3aRp63gx6oDHwcCp/YGAgJkyYgICAAKSkpOCLL77Axo0b63weZEREBGbPnq3VNnv2bMyp5cPK0u5Tawj4QGNpLPU+NUtnqfdbWTrmTZp6uU+t6huWl5dr/kFyc3MxdepUXLt2Da+++mqt686ZMwcKhUJrqa2gERERGULvota5c2d8+umn+OOPP7B27VqcP38eXbp0QZ8+fdChQwd8/PHHWLlypTFjJSIiqpVepx9PnDgBX19fHDlyBOvXr8fu3btRVmm2GgC0aNECmZmZsLa2rtcAefrRcDz9KA1PP0rD02jSMG/S1Mv3qcXExCA4OBgZGRk19rl9+zaaNGlicIBERET1pdaR2uXLl9GzZ0/cuXPHhCFp40jNcBypScORmjQccUjDvEnzRBNFvLy86v10IhERkbEYPPuRiIjIUtV5Ta1Xr164W8fXfvzyyy/1FhAREZFUdRa1Xbt21Xr+UggBZQ3fhUVERGRKdVajzp074/bt26aIhYiI6InUWdTu3Llj1tmPRERE+uJEESIiko1ai9rPP/+MkpISU8VCRET0RGo9/Tho0CBTxUFERPTEePqRiIhkg0WNiIhkg0WNiIhkg0WNiIhkQ+9HgTg6OuLFF1+Eh4cHGjVqpPXaRx99VO+BERERGUqvovbXv/4V+/fvR3FxMVq0aIE//vgDbm5uKC4uRkZGBosaERFZBL1OPy5evBhff/01WrVqhfv372PQoEHw8PBAQkICFi1aZOwYiYiI9KJXUevatStWrlwJACgvL0fjxo1x69Yt/Oc//8Hs2bONGR8REZHe9CpqlZ8qkpWVBU9PTwBAQUEB3N3djRMZERGRgfS6pnby5En07NkTFy9exE8//YR58+ahZcuWGDt2LJKSkowdIxERkV70GqnNmjULN27cAACEh4fj9u3bWLFiBRwdHTFp0iSjBkhERKQvBQBh7iBqI4So9UtKSZcQD/9JVzBvBpn2KG883gwjmDdJmDdp6qoJDaKoERERVaitqNV4Te306dPo378/7t27h6SkpFqLS7du3Z4sQiIionpQY1H79ttvUVxcDACIjo42WUDV4fDcMDytIU1F3k4wbwZ5lsebJPw9laaus3cN4vQj/9ENw3yRNUoAACAASURBVF8WaVjUpGFRk4a/p9LUVRP4QGMiIpINve5Ta9asGWbPno2BAwfCxcUFVlbatbBly5ZGCY6IiMgQehW1L7/8Ep07d8aWLVuQlZXFGYlERGSR9LqmlpeXh/79++PUqVMmCEkbr6kZjufqpeE1NWl4TU0a/p5KUy/X1C5fvqxzypGIiMjS6FWpZsyYgQULFqBr164sbkREZLH0uqZ26dIlNGnSBCdPnqz+TZR6f4E2ERGR0ehVjbZv346mTZti+vTpnChCREQWS6+i5u/vj2effRZnz541djxERESS6XWB7Ny5c3BwcDB2LERERE9Er6IWHh6OpUuXYvDgwXBxcYGjo6PWQkREZAn0uk+tvLxc8/+Vr6cpFAoIIYw6UYT3qRmO979Iw/vUpOF9atLw91SaumqCXtVo4MCB9RYQERGRsfAp/TLEvwCl4UhNGo7UpOHvqTT1MlIDgCZNmqB79+7VPtB49+7d0iMkIiKqJ3oVtcGDB2P79u1wcnLSec3Y19SIiIj0pdfsx8jISOzfvx+tW7eGtbW11sKCRkRElkKviuTl5YVXXnkFN2/eNHY8REREkuk1UouNjUXHjh2NHQsREdETqXGk5uvrq/n/tWvXYsmSJXB3d8eZM2dQWlqq1dcc37NGRERUVY1T+svLy/WaTs+bry0PpwpLwyn90nBKvzT8PZVG8pT+tm3bGiUgIiIiY6mxqF29elXz/3379kVcXJzW47IAwNraGr1799bqS0REZC56PVGkrKwMbm5uuH37tlZ78+bNcevWLZ5+tDA8rSENTz9Kw9OP0vD3VJq6aoJesx8rHlxclZOTEwoLC6VHR0REVI9qLWp79uzBnj17IITAV199pfl5z5492LdvHw4fPoy4uDhTxWoxwsLCkJaWBrVajYSEBPTp08fcITUYzN1jLmFh6JaWBn+1Gp0TEmBXRy7s+/VD54QE+KvV6Hr5MlpMnqzTR+Xqiqc3b4bvrVvwV6vR5exZ2PfrZ6xdsHg83qRpyHmrtajl5OQgJycHCoUCd+/e1fyck5OD69evY+3atRg7dqypYrUIwcHBiIyMxPz58+Hr64u4uDgcPHgQbdq0MXdoFo+5e6x5cDA8IiNxY/58JPv6oiAuDh0PHkSjGnLRyMsL3gcOoCAuDsm+vri5YAE8V6yAY1CQpo9106Z4JjYWUCiQ+tJLSHrmGVyZNg2lt26ZarcsCo83aRp63vS6pva///u/WLJkCYqKiiRvqFGjRli9ejWGDBmC5s2b49KlS5g5cyb+7//+r9b1LO2aWnx8PJKSkjBp0iRNW2pqKqKjozFz5kwzRvaYpZ6rt/TcmfKaWqf4eBQlJSGjUi66pqbiTnQ0rleTi9YLF6J5UBCSvL01bV7r18Omc2ec6937YZ+PP4Z9//44b+K/qi31mlpDOd6YN8PUyzU1R0fHJ57ir1Qqce3aNfTv3x9NmzbFhx9+iKioKHh6ej7R+5qSSqWCn58fYmJitNpjYmLQ+9EHC1WPuXtMoVLB1s8PuVVykRsTA7sacmHXq5du/0OHYOPvD8WjiVqOgYEoPH4c7XbsgG9WFjqfOgWXKVOMsxMWjsebNHLIm15FrWfPnjh9+jSOHz+Ot956C/b29gZvqKioCHPmzMGVK1cghMD+/fuRnp4OPz8/g9/LXJydnaFUKpGVlaXVnpWVBVdXVzNF1TAwd48pnZ2hUCpRViUXpVlZUNWQi0auriit0r8sKwtWKhWUzs4AgMZPPw2Xd95BcVoaUl54AVmRkWizcOGfsrDxeJNGDnnTq6j16dMHnTp1wpEjRxAREYEbN25gy5Yt6PcEF6BdXFzg7e2Ns2fP1tpv3bp1krdhLFVngtY0O5R0MXeP6ey3QgHUlovq+ldut7JC4cmTuD5zJop+/x3Zmzcj67PP0PJPWNQq8HiTpiHnTa+iBjw8p/rBBx+gTZs2GDlyJOzs7BATE4PU1FT85z//gaOjo94bVSqV+Prrr7FlyxakpKTU2rfyeV1zy87ORllZmc5fLC4uLjp/2ZA25u6xsuxsiLIynVGZysVFZzRWoSQzU6e/0sUFD0pLUZaTAwAovXkT6nPntPqoz59HIw+Peoy+YeDxJo0c8qZ3UaugUqng4OCApk2bwtraGlevXsW4ceNw9epVjBo1qs71FQoFtm7dipKSEkydOlVS0OZSWlqKxMREBAQEaLUHBAT8KW9tMARz95goLUVhYiKaVslF04AAFNSQi4Jjx+AwZIhO/6KEBIiysod9YmPRpMq3aTzl7Y2SK1fqMfqGgcebNHLJm9Bn8fPzE6tWrRI5OTni2rVrYt68ecLLy0vz+vTp00VmZmad77Nx40bx448/iqeeekqv7YqHY16LWYKDg0VxcbEIDQ0VPj4+Yvny5SI/P194eHiYPbbKObO0vDWE3FU4Dhh9uRgcLMqLi0VaaKg47eMjbi5fLsry88UpDw9xHBC3t2wRt7ds0fQ/5eUlygoKxM1ly8RpHx+RFhoqyouLRWpQkKZPsr+/KC8pEVdnzhS/t2snUv/xD1F6755If+cdo+4Lj7cnO97MHUdDzFsdfep+k6SkJFFcXCy+//578fe//11YWVnp9HF2dhbl5eW1vs+aNWvEsWPHhK2tbX3ugMmXsLAwkZ6eLu7fvy8SEhJE3759zR5T1ZxZYt4sPXemLGrHAZEeFibup6eL8vv3RUFCgjjXt6/mtdwjR0TukSNa/c/16ycKEhNF+f374n5amkibPFnnPS8MGyYKf/9dlKvVQp2SIjKmTTP6fvB4e7LjzdxxNMS81fa6XvephYeHY+PGjbhx40ZdXWvk4eGBK1eu4P79+yh7dLoEACZPnoxt27bVuJ6l3afWEFjq/S+Wjs9+lMZS71OzdPw9laaumqBXUTMnFjXD8ZdFGhY1aVjUpOHvqTSSv0+tquDgYAwePBguLi6wstKeXzJ8+HDpERIREdUTvYraJ598gnfffRdHjhzBjRs3Gsz9CkRE9Oei1+nHzMxMTJkyBd9++60JQtLG04+G42kNaXj6URqefpSGv6fS1MuzH62srPD777/XW1BERETGoFdRW7du3Z/uK2aIiKjhqfGaWmRkpOb/raysMGbMGAQEBCApKQmlpaVafWfMmGG8CImIiPRUY1Hr0qWL1s8Vpx99fHy02jlphIiILIVeE0WaNGmC4uJiPHjwwAQhaeNEEcPxArQ0nCgiDSeKSMPfU2meeKKIlZUVcnNz0bHKg1KJiIgsTZ1F7cGDB7hy5QoaNWpkiniIiIgk02v240cffYSFCxfCycnJ2PEQERFJptc1taSkJLRt2xYqlQrXr19HYWGh1uvdunUzVny8piYBz9VLw2tq0vCamjT8PZWmXp79GB0dXW8BERERGUudRc3a2hq//fYbjh8/jjt37pgiJiIiIknqvKZWXl6OXbt2wd7e3hTxEBERSabXRJHTp0+jffv2xo6FiIjoiehV1GbPno1PP/0Uw4cPR+vWreHo6Ki1EBERWQK9Zj+Wl5dr/r/yY7EUCgWEEFAq9f6uUYNx9qPhOKtKGs5+lIazH6Xh76k09TL7ceDAgfUWEBERkbHoVdSOHj1a42vt2rWrt2CIiIiehF7X1Kpq3LgxxowZg59++gkpKSn1HRMREZEkBhW17t27Y+XKlbhx4wYWL16Ms2fPYsiQIcaKjYiIyCB1nn50cHDAmDFj8NZbb6F9+/bYs2cP7O3t0adPH5w/f94UMRIREeml1pHal19+ievXryMoKAjLly+Hq6srxo0bZ6rYiIiIDFLrlP7S0lIsWrQIS5Yswb179zTtJSUl6Natm0lGavxmbSIiqkzyl4QGBweje/fuuH79Onbt2oXAwECj3pNGRET0JPS6+bpVq1aYMGECQkJC0KxZMzRr1gxvvPEGtm3bZvQAefO14XhTpzTMmzQVeQtl3gyygcebJHXVBL2KWmWDBw/GxIkTMXz4cOTl5WHXrl145513njTOGrGoGY4fztIwb9KwqEnDoiZNvRe1ChWjtTfffBPdu3eXGl+dWNQMxw9naZg3aVjUpGFRk8ZoRc1UWNQMxw9naZg3aVjUpGFRk6aumiDpiSJERESWiEWNiIhkg0WNiIhkg0WNiIhko8Y7qQ35Ruu7d+/WSzBERERPosailp2dXecjqkzxzddERET6qrEa8duuiYiooamxqNX2bddERESWyODzhi1btkSjRo202q5du1ZvAREREUmlV1FzcHDAZ599huDgYJ2CBoDX1IiIyCLoNaV/yZIl6NatGwIDA3H//n2MHj0a//73v3H9+nW8/vrrxo6RiIhIL3oNsf72t79h1KhR+PXXX1FeXo7ExERERUXh5s2bmDx5Mr799ltjx0lERFQnvUZqzZo1w5UrVwAAubm5cHJyAgAcO3YMvXv3Nl50REREBtCrqF2+fBlPP/00AOD8+fMYOXIkACAoKAh37twxXnREREQG0Kuobd68GV27dgUALFy4EJMnT0ZJSQkWL16MRYsWGTVAIiIifUn6PrU2bdrA398fFy9eRHJyshHCeozfp2Y4fi+YNMybNPw+NWn4fWrS1FUTJM3Fv3btGu9NIyIii6NXUfvnP/9Z6+vLli2rl2CIiIiehF6nH9PS0rR+VqlUcHNzg1qtxq1bt9CuXTtjxcfTjxLwNJo0zJs0PP0oDU8/SlMvpx8rZj5W5uLigk2bNmH9+vXSoyMiIqpHkiaKVOjevTuioqLg7e1djyFp40jNcBxxSMO8ScORmjQcqUlTV014om++trKyQsuWLZ/kLYiIiOqNXqcfR4wYofWzQqGAm5sbpkyZgl9++cUogRERERlKr5FadHS01hIVFYXw8HCcOnUKEydONHaMFicsLAxpaWlQq9VISEhAnz59zB1Sg8HcScO8PTYwLAwL09KwVq3GhwkJ6FBHLrz79cOHCQlYq1Zj4eXL6D95svbrffti2p49WHL9OjYIgefHjzdm+A1CQz7e9Cpq1tbWWotSqYSbmxvGjBmDzMxMY8doUYKDgxEZGYn58+fD19cXcXFxOHjwINq0aWPu0CwecycN8/ZYz+BgjIyMxIH58zHH1xeX4+Lw7sGDaF5DLpy9vPDugQO4HBeHOb6+OLBgAUavWAG/oCBNn8Z2dvgjORnbZ8xAcVGRqXbFYjX0402viSLjxo3DN998g5KSEq12lUqFkSNHYuvWrXptbOvWrRg8eDBsbW2RmZmJTz75BBs2bKh1HUubKBIfH4+kpCRMmjRJ05aamoro6GjMnDnTjJE9ZqkTHiw9d8ybNKacKDIrPh7Xk5KwpVIu5qemIiE6GruqycU/Fi5Ej6AgzKw0mW38+vVo1bkz5lfzMPZV+fnYNnUqYrdsMc4OVGKpE0UawvH2xBNFNm3ahKZNm+q029vbY9OmTXoHs2DBAnh5eaFp06Z45ZVXMG/ePPTo0UPv9c1NpVLBz88PMTExWu0xMTH8toI6MHfSMG+PWatU8PTzw9kquTgbE4P2NeSiXa9euv0PHYKnvz+s+eXGOuRwvOlV1BQKheavsco8PDyQm5ur98bOnTunGe0JISCEMOqN2/XN2dkZSqUSWVlZWu1ZWVlwdXU1U1QNA3MnDfP2mL2zM6yVSuRVyUVeVhaa1pALB1fXavsrVSrYOTsbLdaGSg7HW61/qiQlJWmKz88//4yysjLNa9bW1vD09MSBAwcM2uCqVasQEhICGxsbnDx5ss71161bZ9D7m0LVAl9T0SddzJ00zNtjhuZC57VHp67+rPnTR0M+3motatHR0QCAv/zlL9i/fz8KCgo0r5WUlCAjI8Pgb72eMmUKpk2bhl69emHAgAEoLi6utf+kSZMwucpsJXPJzs5GWVmZzl8sLi4uOn/ZkDbmThrm7bH87GyUl5XpjMrsXVx0RmMV8jIzdfo7uLigrLQUhTk5Rou1oZLD8VZrUZs7dy4AICMjAzt27NCZKCLVgwcPEBsbi7FjxyIsLAwrVqyol/c1ttLSUiQmJiIgIEBT8AEgICDA4OL+Z8PcScO8PVZeWooriYnoFBCAhEq56BQQgMQacnH52DH4BgZqtXUKCMCVhASUVzrzRA/J4XjT60ppRkYGnnvuORw9elSrvV+/fhBCSL4BW6lUNqhragCwdOlSbN26FSdOnEBsbCzefvttuLu7Y+3ateYOzeIxd9Iwb4/FLF2KiVu3Iv3ECVyKjcWAt99GM3d3/PwoF6GPZi1ueHSv2U9r12LQ1KkYuWwZfv78c7R//nk8HxKCdaNGad6zsa0tXNq3BwAorKzQ3MMDbbp1Q+GdO7jzJ/yKrYZ+vOlV1JYtW6YZtVXm4OCA2bNnw9/fv873aNGiBQYNGoR9+/ZBrVZjyJAhGDVqFEaPHm141GYUFRUFJycnhIeHw83NDcnJyRg2bBiuXr1q7tAsHnMnDfP22G9RUbBzcsLL4eFo6uaGP5KTETlsGHIe5aK5h4dW/+yMDCwfNgwjly3DgLAw3LtxA9umT0firl2aPl7+/nj/p580PwfOnYvAuXMRu3kzNk6YYJL9siQN/XjT6z61goIC/OUvf0FGRoZWu6enJ5KTk2Fvb1/nhpydnREdHY1u3brBysoKV65cwWeffYYvvvii1vUs7T61hsBS77eydMybNHygsTSWep+apauXr55Rq9Vwd3fXKWqtW7fW+zpbdnY2BgwYoFdfIiIiKfS6T+3QoUNYuHAhmjVrpmlzdHTE/PnzcejQIaMFR0REZAi9Tj+6urri6NGjcHFxQVJSEgCga9euuHXrFvr374+bN28aLUCefjQcT6NJw7xJw9OP0vD0ozR11QS9vyS0SZMmGDNmDLp37w6FQoGTJ09i27ZtUKvV9RVrtVjUDMcPZ2mYN2lY1KRhUZOm3opaTQYPHowffvjhSd6iVixqhuOHszTMmzQsatKwqEljlG++dnd3x6xZs5CWlsZrakREZDH0LmpWVlYIDAzE/v37kZGRgREjRmDNmjVo/+imRSIiInOrc0q/t7c3Jk6ciDfeeAOFhYXYtm0bAgICMG7cOJw/f94UMRIREeml1pHa0aNHER8fj2bNmiE4OBjt2rXDhx9+aKrYiIiIDFLrSK1Xr15YtWoV1q9fj7Nnz5oqJiIiIklqHan5+/tDqVTil19+wcmTJ/Huu++iZcuWpoqNiIjIILUWtdOnT2Pq1Klwc3PD0qVLMXz4cFy7dg1WVlZ46aWXtJ4wQkREZG4G36fWrl07zcQRJycn/Pjjjxg2bJiRwuN9alLwfitpmDdpeJ+aNLxPTZp6v0/t8uXL+J//+R+0adMGwcHB9fbFoURERE/qiZ8oYmwcqRmOIw5pmDdpOFKThiM1aYzyRBEiIiJLxKJGRESywaJGRESywaJGRESywaJGRESywaJGRESywaJGRESywaJGRESywaJGRESywaJGRESywaJGRESy0SCe/UhERFSBz34kIqI/BaW5A9AHn2JtGD5tXhrmTRrmTZqKvP3GvBmkZx1n7zhSIyIi2WBRIyIi2WBRIyIi2WBRIyIi2WBRIyIi2WBRIyIi2WBRIyIi2WBRIyIi2WBRIyIi2WBRIyIi2WBRIyIi2WBRIyIi2WBRIyIi2WBRIyIi2WBRIyIi2WBRIyIi2WBRIyIi2WBRIyIi2WBRIyIi2WBRIyIi2WBRIyIi2WBRIyIi2WBRIyIi2WBRIyIi2WBRIyIi2WBRkyAsLAxpaWlQq9VISEhAnz59zB1Sg8HcScO8ScO8PdYiLAxd09Lgp1ajU0IC7OrIhX2/fuiUkAA/tRpdLl9Gi8mTtV7vmp6OnkLoLB327TPmbtSJRc1AwcHBiIyMxPz58+Hr64u4uDgcPHgQbdq0MXdoFo+5k4Z5k4Z5e6x5cDA8IiNxY/58nPX1RUFcHLwPHkSjGnLRyMsLHQ4cQEFcHM76+uLmggXwWLECjkFBmj7nevbEKVdXzXLW1xfiwQPciYoy1W5VSwFAmHqj7du3x5kzZxAdHY1x48bV2lcIAYVCYaLI6hYfH4+kpCRMmjRJ05aamoro6GjMnDnTjJE9JsTDf1JLyhtg+blj3qRh3qSpyNtvJsjbM/HxUCclIaNSLrqkpuJudDSuV5OL1gsXwjEoCGe8vTVtXuvXo0nnzjjfu3e123CbOROu//43Tru744FaXf878UjPOmqCWUZqq1atwm+//WaOTT8RlUoFPz8/xMTEaLXHxMSgdw3/0PQQcycN8yYN8/aYQqWCrZ8fcqvkIjcmBrY15MKuVy/d/ocOwcbfHwqlstp1nENDkfPVV0YtaPoweVF7/fXXce/ePfzwww+m3vQTc3Z2hlKpRFZWllZ7VlYWXF1dzRRVw8DcScO8ScO8PaZ0doZCqURplVyUZWVBVUMuVK6uKKvSvzQrC1YqFZTOzjr9HQIC8NTTT+P2F1/UX+ASmbSo2dvbY+7cuXjvvff0XmfdunVGjEiaitMGFRQKhU4bVY+5k4Z5k4Z5q6TqfisUum1a3XVzV107ALR46y0UnDgB9enTTx7nEzJpUfvoo4+wYcMGXL9+Xe91Kp8PN7fs7GyUlZXp/KXn4uKi8xchaWPupGHepGHeHivLzoYoK9MZlSldXHRGbxVKMzOr7f+gtBTlOTna7S1aoNnw4bi9fn39Bi6RyYpat27dMGTIECxbtsxUm6x3paWlSExMREBAgFZ7QEAA4uLizBRVw8DcScO8ScO8PSZKS1GYmAiHKrloGhCAwhpyUXDsGByGDNFqcwgIQFFCAkRZmVa7c0gIRHEx7uzYUb+BS1T9FT8jGDBgALy8vHD16lUAgJ2dHaytrdGpUyf4+fmZKowntnTpUmzduhUnTpxAbGws3n77bbi7u2Pt2rXmDs3iMXfSMG/SMG+PZS1dirZbt6LwxAkUxMbC5e23oXJ3x61HuWi7ZQsAIH38eADArbVr4TJ1KtosW4bbn38Ou+efh3NICNJGjdJ57xYTJ+LOjh14UFBguh2qhcmK2rp167CjUiX/17/+BS8vL4SFhZkqhHoRFRUFJycnhIeHw83NDcnJyRg2bJimWFPNmDtpmDdpmLfH7kRFwdrJCe7h4VC5uUGdnIzUYcNQ8igXjTw8tPqXZGTg4rBhaLNsGVzCwlB64wauTp+Ou7t2afWzHzAAT3l7I23sWJPtS13Mcp8aAERERKB9+/YN7j61hsBS7xuydMybNMybNKa8T01O6rpPzWxFTV8saobjh4w0zJs0zJs0LGrSWOTN10RERMbAokZERLLBokZERLLBokZERLLBokZERLLBokZERLLBokZERLLBokZERLLBokZERLLBokZERLLBokZERLLBokZERLLBokZERLLBokZERLLBokZERLLBokZERLLBokZERLLBokZERLLBokZERLLBokZERLLBokZERLLBokZERLLBokZERLLBokZERLLBokZERLKhACDMHURthLDo8IiIyMQUCkXNr8HCixoREZG+ePqRiIhkg0WNiIhkg0WNiIhkg0WNiIhkg0WNiIhkg0WNiIhkg0WNiIhkg0WNiIhkg0VNAkdHR+zatQsFBQXIyMjAqFGjzB1SgzBlyhT89ttvuH//PjZt2mTucBqERo0a4YsvvkBGRgby8vJw8uRJvPjii+YOq0HYunUrbty4gdzcXKSkpCA0NNTcITUo7du3h1qtxtatW80disEEF8OWbdu2iR07dghbW1vx/PPPi3v37olOnTqZPS5LX0aMGCGGDx8uVq9eLTZt2mT2eBrCYmNjIyIiIoSnp6dQKBTipZdeEnl5ecLT09PssVn60qlTJ9GoUSMBQHTs2FHcvHlT9OjRw+xxNZTl0KFD4ujRo2Lr1q1mj8WQhSM1A9nY2ODVV1/Fhx9+iMLCQsTGxuL777/HuHHjzB2axdu9ezf27NmDnJwcc4fSYBQVFWHOnDm4cuUKhBDYv38/0tPT4efnZ+7QLN65c+dQUlICABBCQAiBdu3amTmqhuH111/HvXv38MMPP5g7FIOxqBnI29sb5eXluHjxoqbt9OnT6Ny5sxmjoj8LFxcXeHt74+zZs+YOpUFYtWoVCgsLkZKSgps3b+LAgQPmDsni2dvbY+7cuXjvvffMHYokLGoGsrOzQ25urlZbbm4u7O3tzRQR/VkolUp8/fXX2LJlC1JSUswdToMwZcoU2Nvbo0+fPti1axeKi4vNHZLF++ijj7BhwwZcv37d3KFIwqJmoIKCAjg4OGi1OTg4ID8/30wR0Z+BQqHA1q1bUVJSgqlTp5o7nAblwYMHiI2NRevWrREWFmbucCxat27dMGTIECxbtszcoUimNHcADU1qaiqUSiXat2+PS5cuAXh4IPB0EBnThg0b0LJlSwwbNgxlZWXmDqdBUiqVvKZWhwEDBsDLywtXr14F8PDMlLW1NTp16tSgruOafbZKQ1u2b98utm3bJmxsbETv3r05+1HPxdraWjRu3FjMnz9ffPnll6Jx48bC2tra7HFZ+rJmzRpx7NgxYWtra/ZYGsrSokUL8frrrwtbW1thZWUlhg4dKgoKCsQrr7xi9tgseWnSpIlo2bKlZlm8eLHYuXOncHZ2NntsBixmD6DBLY6OjmL37t2ioKBAXLlyRYwaNcrsMTWEJSIiQlQVERFh9rgsefHw8BBCCKFWq0V+fr5mGT16tNljs+TF2dlZ/PTTT+Lu3bsiNzdXJCUliYkTJ5o9roa2RERENLgp/fzmayIikg1OFCEiItlgUSMiItlgUSMiItlgUSMiItlgUSMiItlgUSMiItlgUaMG4cyZM4iIiND8nJ6ebpYHrvr5+UEIAU9PT5Nu98iRI1ixYkW9v6+npyeEEEZ5WkRERATOnDlT7+9LVBsWNZJk06ZNmq/zKCkpweXLl7F48WLY2NiYZPs9e/bE6tWr9eo7fvx4szybc8SIESgrK8NXX31l8m1XZ9OmTdi7d6/JtrdkyRL0799f8vpHIRHTDAAAC81JREFUjhzRHGPVLenp6fUYLckFn/1Ikh0+fBjjxo2DSqVC37598cUXX8DW1hbvvPNOtf2VSmW9PbcwOzu7Xt7HmCZOnIhPPvkE06dPR7NmzXDv3r1a+1eXH2tra5SXlxszTKMpLCxEYWGh5PWDgoLQqFEjAEDz5s1x7tw5BAUFIS4uDgAabF7IuDhSI8mKi4uRlZWF69evY/v27fj6668RGBgIAOjfvz+EEPjb3/6G48ePo7i4GC+88AIA4OWXX0ZCQgLUajXS0tIwb948qFQqzfu2aNEC3333HYqKipCRkYEJEybobLvq6Ud7e3usXr0aN27cgFqtxrlz5xAcHIz+/ftj8+bNsLOz0/yFX3EaU6VSYeHChbh27RoKCgpw4sQJDB06VGs7L7zwAs6fPw+1Wo2jR4/C29tbr9y0atUKAwcOxJIlSxAfH48xY8ZovV5TfipO2Y0fPx6XLl1CcXExbG1tAQBWVlb4+OOPcfv2bWRlZWHx4sVQKBQAgA8//LDaU32//vorIiMjERERgZCQELz88suaPFQeRXl6eiImJgaFhYU4e/YshgwZohPriy++iISEBBQVFeHo0aNo1aoV+vXrh99//x35+fnYu3cvmjdvrlmv6ulHf39/HDp0CLdv30Zubi5++eUXPPfcczXm8O7du8jKykJWVhZu3boFALhz546mLTs7Gx9//DF+++03nXUTEhKwaNEiAMD27duxc+dOzJ49G1lZWcjNzcXnn3+uKZjAw29BmDlzJtLS0lBUVITTp0/jtddeqzE2smxmf1YXl4a3bNq0Sezdu1erLTIyUty+fVsAEP379xdCCJGUlCQCAgJE27ZthbOzsxg6dKjIzc0VISEh4umnnxYDBgwQFy5cEIsXL9a8z/79+0VycrLo3bu36N69uzhy5IjIz8/Xek5kenq6eO+99zQ///rrr+Ls2bPihRdeEG3bthUvvviiCAwMFCqVSkyfPl0UFBRoHtJa8WDgr776Shw7dkz07dtXtG3bVkyZMkUUFxeLrl27CgCidevWQq1Wi88++0x07NhRvPbaa+LatWtCCCE8PT1rzU94eLj47rvvBAAxfvx4cerUKa3Xa8pPRESEKCgoEIcOHRK+vr6ic+fOwtraWhw5ckTcu3dPzJkzR3To0EG89tprorS0VIwcOVIAEK1atRKlpaWiZ8+emm14e3sLIYTo2rWrsLW1FTt27BAxMTGaPKhUKuHp6SmEEOL8+fPi5ZdfFu3btxebN28W2dnZmjxVxHr8+HHRp08f0aVLF3HmzBnx66+/iv/+97/i2WefFX5+fiItLU189tlnmu1HRESIM2fOaH4eOHCgGDt2rPDx8REdO3YUK1asEHfu3BFOTk51Hm9OTk5CCCH69++v1d62bVtRXl4uunTpomnr0qWLEEKIjh07CuDhA8jz8vLEV199JTp16iT+9re/iZs3b4pFixZp1lmyZIlITk4WAQEBwsvLS7zxxhuiqKhIDB482Oy/a1wMXsweAJcGuFQtaj179hS3b98WO3bsEMDjD8KgoCCt9X7++WcRHh6u1TZ8+HCRn58vAIgOHToIIYTo3bu35nUPDw9RVlZWY1EbMmSIKC8vFz4+PtXGOn78eM37VyxPP/20KC8vF23atNFq3717t1i1apUAID7++GORkpKi9fqsWbP0KmqXL18Wr776qgAgbG1tRUFBgejRo4fm9ZryExERIUpKSoSLi4tW+5EjR0RcXJxWW0xMjFi/fr3m571794o1a9Zofl64cKH47bffavw3A6ApapMmTdK0ubu7CyGEeP7557ViHTp0qKbPlClThBBC+Pr6asVeuYhV/bm65caNG2LMmDF1Hm81FTUA4vDhw2LZsmWan5cvXy5++eUXzc/bt28Xt27dEo0bN9a0hYaGisLCQqFSqUTTpk1FcXGx8Pf313rfNWvWiG+//dbsv2tcDFt4+pEke/HFF5Gfnw+1Wo1jx47h6NGjmDZtmlafhIQErZ/9/Pwwa9Ys5Ofna5Zt27bBzs4Orq6ueOaZZ1BeXo4TJ05o1rl69Spu3LhRYxy+vr64efMmLly4oHfsPXr0gJWVFc6dO6cVy0svvaT5zq1nnnkG8fHxWusdO3aszvcePHgwHB0dNZMyCgsL8d1332HixIk6favmB/j/7d1fSFN9GMDx745MMyMycGVgeVHhRUI3RpRIQRdGIJFdFGblEusqWAMV6aYICi+iXWTKLK2s1UaOSAuD9UcY4qiNipquP24WkjmpMdjIU9t7Ee3tbOprVvNt/D7wu9j5nfPwnLPBs985jw7evXsXu932o6dPnypej4yMoNFoYq+NRiO7du1i3rx5SJJEZWUl58+f/89842N/v9Y/xo7fZ3R0FEBxe3F0dDThmB/l5OTQ3NzM4OAgnz59IhgMotFoWL58+YxynIrRaKSiogK1Wo1araaioiLhvF0ul+JXr/v6+pg/fz75+fkUFhaSnp7O/fv3FZ+Fqqoq8ftrfyHRKCLMWm9vLzU1NciyzMjIyKRNIPGNApIkcezYMSwWS8K+Y2NjsWdEP2M2x0iSRCQSoaioCFmWFXPhcHjWceFbg0h2drbi3FUqFcFgEL1eH4sPiddnqm1AQp7RaBRJ+vd7aXd3N6FQiPLycgKBAIsWLcJkMs0o5/jYgCJ2/D7RaBRA8Z7H5xPv4sWLLFmyBJ1Oh9fr5fPnz9hsNsWzrdmwWq2cPXuWsrIyVCoVGRkZmM3mGR//PefS0lLev3+vmJuYmPil3ITkE0VNmLVQKMTr169/6hin00lBQcGUx7ndbtLS0igqKoqtivLy8li2bNm0MXNzcykoKJh0tTYxMUFaWppim8vlQpIkli5dyoMHDyaN++LFC8rLyxXbpmtsAMjOzmb79u3s3bsXp9OpmLPZbOzcuZPLly9PG2O2vn79Snt7O1qtlkAgQGdnJ4FAIDY/2XVIpuLiYg4fPszt27eBbyvB3NzcX44ryzKXLl1Cq9WiUqm4fv06oVBIsc/atWtJT0+PFan169fHGpH8fj+yLJOXl4fdbv/lfIS5JYqakFTHjx+nq6sLn8+H2Wzmy5cvrFmzhnXr1lFXV4fH4+HOnTu0tLRQU1NDOBzm9OnTitVNPJvNRn9/Pzdu3ECn0+HxeFi5ciVZWVncvHkTr9dLZmYmW7ZsweVyEQqFePnyJR0dHbS3t6PX63E6nSxevJhNmzbx5s0brFYrzc3N6PV6zpw5Q1NTE4WFhRw6dGja86usrCQYDHLlyhUikYhirrOzk+rq6j9W1ABaW1upq6sjEokkdHJ6vV62bt3K6tWrGR8fVxS8ZPB4POzZs4f+/n6ysrJobGz8bSsho9HIs2fPUKlUFBcXJ8xnZmbS2trKyZMnWbFiBSdOnKCpqQlZlvn48SMGgwGDwYBarcZut7Nw4UI2bNhAOBymra3tt+QoJId4piYk1d27d9m2bRubN2/G4XDgcDior69neHg4ts/+/fsZGhri3r173Lp1i6tXr+L1eqeM+b013m6309HRgdvtxmAwxG5r9fX1ce7cOUwmE36/n9raWgCqqqpoa2ujsbGRgYEBurq6KCkpwefzAfD27Vt27NhBaWkpT548QafTUV9fP+35HThwAKvVmlDQACwWCyUlJaxatepnL9uMDQ0N8fDhQ4aHhxNWoEajEbfbzaNHj/D7/WzcuPGP5TEZrVbLggULePz4MdeuXePChQvTvq8/Y2BgAIfDweDgYMJzUICenh58Ph+9vb2YzWa6u7s5evRobL62tpZTp07R0NCA2+2mp6eHsrIy8Qfef6k571YRQwwxft94/vx5tKGhYc7zSPZ49epV9MiRIwnbTSZT1GKxzHl+YiRniNuPgpAicnJy2L17N/n5+bS0tMx1Okmj0WjYt28fGo1mxt2eQuoSRU0QUsSHDx8YGxvj4MGDjI+Pz3U6SZGRkRH7jyPV1dVJf04o/P+o+LZkEwRBEIS/nmgUEQRBEFKGKGqCIAhCyhBFTRAEQUgZoqgJgiAIKUMUNUEQBCFl/ANqrb77GLWDDAAAAABJRU5ErkJggg==\n",
      "text/plain": [
       "<Figure size 504x432 with 1 Axes>"
      ]
     },
     "metadata": {},
     "output_type": "display_data"
    }
   ],
   "source": [
    "ModelEvaluation.plot_confusion_matrix(model, feature_mat_test, target_vect_test)"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "##### Discussion\n",
    "\n",
    "Our optimized deep neural network returned an accuracy on our test data of ~95%. From our confusion matrix we note that the statistically significant majority our false predictions involve classifying the signal of a normal heartbeat as an arrhythmia. In terms of medical diagnostics, this is certainly better than letting a serious arrhythmia go undetected. We also note that our model failed to learn any notable patterns on type 1 arrhythmias - supraventricular premature beats."
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "\n",
    "---\n",
    "\n",
    "<a id=\"conclusions\"></a>\n",
    "\n",
    "## Conclusions\n",
    "\n",
    "\n",
    "<a id=\"findings\"></a>\n",
    "\n",
    "### Findings\n",
    "\n",
    "After training our optimized deep neural network on 100000 ECG signals we were able to make accurate predictions on ~95% of our testing data. Of the incorrect prediction, the statistically significant ones were where our model classified a regular signal as that of one signifying an arrhythmia. As mentioned prior, this is certainly better than letting a serious arrhythmia go undetected. A more balanced testing dataset could yield different results, though.\n",
    "\n",
    "<a id=\"limitations\"></a>\n",
    "\n",
    "### Limitations\n",
    "\n",
    "Perhaps the most evident limitation to our data is the severe class imbalance. A higher amount of authentic training data for the underrepresented arrhythmia types could have helped our model accurately predict these on our testing data. Though this was attempted to be combated by resampling and adding noise to our signals, this does not make up to high-quality authentic data.  Modifying the number of epochs our final model trained for could possible increase performance as well. Minimal computing power prevents a a more exhaustive hyperparamter search from being completed; it is likely that a different combination of layers, nodes, and evaluation techniques would yield better performance.\n",
    "\n",
    "<a id=\"further\"></a>\n",
    "\n",
    "### Further Research\n",
    "\n",
    "Different types of neural networks could yield higher accuracies. For example, [Gregoire DC](https://www.kaggle.com/gregoiredc)'s model hosted on [Kaggle](https://www.kaggle.com/gregoiredc/arrhythmia-on-ecg-classification-using-cnn) achieved a high level of accuracy using a convolutional neural network (CNN). A long short term memory network (LSTM) have been shown to provide high accuracy on temporal and signal data such as our ECGs.\n",
    "\n",
    "---\n"
   ]
  }
 ],
 "metadata": {
  "kernelspec": {
   "display_name": "Python 3",
   "language": "python",
   "name": "python3"
  },
  "language_info": {
   "codemirror_mode": {
    "name": "ipython",
    "version": 3
   },
   "file_extension": ".py",
   "mimetype": "text/x-python",
   "name": "python",
   "nbconvert_exporter": "python",
   "pygments_lexer": "ipython3",
   "version": "3.7.4"
  },
  "varInspector": {
   "cols": {
    "lenName": 16,
    "lenType": 16,
    "lenVar": 40
   },
   "kernels_config": {
    "python": {
     "delete_cmd_postfix": "",
     "delete_cmd_prefix": "del ",
     "library": "var_list.py",
     "varRefreshCmd": "print(var_dic_list())"
    },
    "r": {
     "delete_cmd_postfix": ") ",
     "delete_cmd_prefix": "rm(",
     "library": "var_list.r",
     "varRefreshCmd": "cat(var_dic_list()) "
    }
   },
   "types_to_exclude": [
    "module",
    "function",
    "builtin_function_or_method",
    "instance",
    "_Feature"
   ],
   "window_display": false
  }
 },
 "nbformat": 4,
 "nbformat_minor": 4
}