logistic_regression_notebook.ipynb

{
 "cells": [
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "<center> \n",
    "\n",
    "# BIL 570 - MACHINE LEARNING\n",
    "\n",
    "## HOMEWORK - 1\n",
    "\n",
    "### Furkan KÜÇÜK - 181211050 </center>"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "Firstly, one should import the libraries will be used in the assignment. "
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 1,
   "metadata": {},
   "outputs": [],
   "source": [
    "import numpy as np\n",
    "import matplotlib.pyplot as plt"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "The parameters for the assignment is below. These parameters can be adjusted as desired.\n",
    "\n",
    "#### Denotations\n",
    "- dataset_name: File name of the dataset.\n",
    "- lr: Desired learning rate. Please note that adjusting learning rate is crucial. If the learning rate is too low, the model can stuck in a local optima and/or takes long to converge. Too high learning rate obstruct the convergence.\n",
    "- batch_size: Desired batch size for mini batches.\n",
    "- n_iterations: Maximum number of iterations if no convergence is detected.\n",
    "- patience: Desired patience level for convergence detection. The usage of this parameter is explained at the corresponding markdown section.\n",
    "- min_delta: Desired threshold level for convergence detection. The usage of this parameter is explained at the corresponding markdown section.\n",
    "\n",
    "P.S: Please note that dataset file should be located at the root directory of this notebook file."
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 2,
   "metadata": {},
   "outputs": [],
   "source": [
    "dataset_name = \"ionosphere.data\"\n",
    "\n",
    "lr = .1\n",
    "batch_size = 16\n",
    "n_iterations = 10000\n",
    "\n",
    "patience = 2\n",
    "min_delta = 1e-2"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "Development of a logistic regression model was assigned in this assignment. The general formulation of logistic regression is like the equation below.\n",
    "\n",
    "<h3><center> $ f(x) = \\sigma(\\beta_0 + \\beta_1 * x_1 + \\beta_2 * x_2 + ... + \\beta_n * x_n) $ </center></h3>\n",
    "\n",
    "\n",
    "The equation above refers to a set of parameters($\\beta_i$) being multiplied by predictors (also known as features) from each example and going through from an activation function which is called \"sigmoid\" function.\n",
    "\n",
    "Logistic regression is a method that is being used for classification. It uses a similar (but not the same) mathematical approach as linear regression method. Each example in dataset is expected to have some number of predictors and a class value. Predictors are the inputs of constructed function that will be used for make predictions. They are being expected to contain valuable information about class predictions each example belongs to. In this assignment this class values are binary as datasets contain only 2 different classes. \n",
    "\n",
    "As mentioned before, logistic regression is a method for classification. Hence, it is expected to output a value which is a probability. Because of that reason, unlike linear regression, an activation function called \"sigmoid function\" is employed for the output. Mathematical expression of the sigmoid function is shown below.\n",
    "\n",
    "<h3><center> $ h(z) =  \\frac{\\mathrm{1} }{\\mathrm{1} + e^{-z} }  $ </center></h3>\n",
    "\n",
    "The sigmoid function is implemented as a lambda function which is located at below."
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 3,
   "metadata": {},
   "outputs": [],
   "source": [
    "sigmoid = lambda z : 1 / (1 + np.exp(-z))"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "A loss function should be implemented for the training process. A loss function is used for measuring how good a trained model performing. Logarithmic loss (log-likelihood) function is implemented for this assignment. Logarithmic loss function is widely employed for classification tasks because of ease of computation and being a good measure for how confident the model of its predictions. Logarithmic loss function is implemented as a lambda function which can be seen below."
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 4,
   "metadata": {},
   "outputs": [],
   "source": [
    "logloss = lambda y_hat, y : np.sum(-y * np.log(y_hat) - (1 - y) * np.log(1 - y_hat)) / len(y_hat)"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "Gradient descent is widely used for any kind of model training task. Gradient descent method simply computes the gradients of the predictors, and that computed gradients is used for moving towards an optimal solution. The gradient descent function can be constructed from the derivative of the model function for each predictor. The simplifed version of computed gradients is shown in the equation below.\n",
    "\n",
    "<h3><center> $ \\frac{\\delta J(\\beta)}{\\delta\\beta_p} = \\frac{1}{m} * X^T(\\sigma(X * \\beta) - y)$ </center></h3>\n",
    "\n",
    "The equation above is a vectorized implementation of partial derivations (gradients) of each predictors which is meant to make the computation process faster. For the sake of computational speed enhancement, all gradients are calculated in a single series of matrix operations. \"y\" denotes the ground truth obtained from dataset and \"m\" denotes the number of samples.\n",
    "\n",
    "Calculated gradients are multiplied with a factor which is called \"learning rate\" in order to make sure the model is learning at a desired speed and summed or substracted with the current predictors(depends on if it's a maximization or minimization task). Increasing learning rate too much may lead the model not to learn. Decreasing it too much may cause a slow learning rate and in some conditions (non-convex loss for the given dataset) may lead to converge to a local optima.\n",
    "\n",
    "The implemented gradient descent method is located at below."
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 5,
   "metadata": {},
   "outputs": [],
   "source": [
    "def gradient_descent(X, y, beta, lr):\n",
    "    y = y.reshape(-1, 1)\n",
    "    gradients = np.dot(X.T, sigmoid(np.dot(X, beta.T)) - y) / len(y)\n",
    "    new_betas = beta - lr * gradients.T\n",
    "\n",
    "    return new_betas"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "In the assignment it's been told that a mini-batch gradient descent should be employed. With this method, instead of calculating gradients for the full dataset, the given dataset is divided into batches. Each batch contains the same number of samples. Thus, one should split the dataset into mini-batches for the learning process. The function implemented for preparing batches task is located at below."
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 6,
   "metadata": {},
   "outputs": [],
   "source": [
    "def prepare_batches(X, y, batch_size):\n",
    "    X_batch_list = list()\n",
    "    y_batch_list = list()\n",
    "    \n",
    "    for i in range(len(y) // batch_size):\n",
    "        X_batch_list.append(X[i * batch_size : i * batch_size + batch_size, :])\n",
    "        y_batch_list.append(y[i * batch_size : i * batch_size + batch_size])\n",
    "    \n",
    "    if len(y) % batch_size > 0:\n",
    "        X_batch_list.append(X[len(y) // batch_size * batch_size:, :])\n",
    "        y_batch_list.append(y[len(y) // batch_size * batch_size:])\n",
    "\n",
    "    return X_batch_list, y_batch_list"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "In order to make predictions after the training process, a prediction function should be implemented. Since the model outputs a probability, one can determine a threshold at 0.5 and predict 1 if the predicted probability greater than that threshold and predict 0 if it's the otherwise. The implementation of prediction function is located at below. "
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 7,
   "metadata": {},
   "outputs": [],
   "source": [
    "predict = lambda X: sigmoid(np.dot(X, betas.T)) > .5\n",
    "predict_with_output = lambda X: (X > .5) * 1"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "Before the training process, the dataset should be imported. The code for importing dataset is located below. The code below can successfully import any dataset that has the same structure with the datasets given. \n",
    "\n",
    "P.S: Please note that the ground truth of each dataset can vary. Since it's known that both of the datasets consist binary classes, a function called \"encoder\" is employed to cast the dataset char into 0 and 1 values."
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 8,
   "metadata": {},
   "outputs": [],
   "source": [
    "f = open(dataset_name, \"r\")\n",
    "\n",
    "X = list()\n",
    "y = list()\n",
    "\n",
    "encode_char = \"\"\n",
    "encoder = lambda y : 0 if y is encode_char else 1\n",
    "\n",
    "for row in f:\n",
    "    split_row = [x.strip() for x in row.split(',')]\n",
    "    if encode_char is \"\":\n",
    "        encode_char = split_row[-1]\n",
    "    y.append(encoder(split_row[-1]))\n",
    "    X.append([np.array(split_row[:-1]).astype(np.float)])"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "If it is desired, the dataset's feature matrix and ground truth vector can be seen with the code below. For simplicity, the whole dataset is not used for this purpose. One can change the corresponding index and see other samples' feature matrices and ground truths too."
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 9,
   "metadata": {},
   "outputs": [
    {
     "name": "stdout",
     "output_type": "stream",
     "text": [
      "[array([ 1.     ,  0.     ,  0.99539, -0.05889,  0.85243,  0.02306,\n",
      "        0.83398, -0.37708,  1.     ,  0.0376 ,  0.85243, -0.17755,\n",
      "        0.59755, -0.44945,  0.60536, -0.38223,  0.84356, -0.38542,\n",
      "        0.58212, -0.32192,  0.56971, -0.29674,  0.36946, -0.47357,\n",
      "        0.56811, -0.51171,  0.41078, -0.46168,  0.21266, -0.3409 ,\n",
      "        0.42267, -0.54487,  0.18641, -0.453  ])]\n",
      "0\n"
     ]
    }
   ],
   "source": [
    "print(X[0])\n",
    "print(y[0])"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "The dataset should be divided into 2 sub-datasets, training and test datasets. Before the division process, the dataset is shuffled in order to ensure both datasets contain samples from both of the classes.\n",
    "\n",
    "2 sub-datasets are also divided into 2 new sets, feature matrix and ground truth vector."
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 10,
   "metadata": {},
   "outputs": [],
   "source": [
    "permutations = np.random.permutation(len(X))\n",
    "\n",
    "X, y = np.asarray(X).squeeze(), np.asarray(y)\n",
    "\n",
    "X = X[permutations, :]\n",
    "y = y[permutations]\n",
    "\n",
    "#To add beta 0\n",
    "temp = np.ones((X.shape[0], X.shape[1] + 1))\n",
    "temp[:, 1:] = X\n",
    "X = temp\n",
    "\n",
    "len_test = len(X) // 5 \n",
    "len_train = len(X) - len_test\n",
    "X_test, y_test, X_train, y_train = X[:len_test, :], y[:len_test], X[len_test:, :], y[len_test:]"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "Please note that the integer of \"1\" is padded at the beggining of the dataset. This can be an efficient implementation for calculating the $\\beta_0$ predictor since that implementation requires one less sum. (For example, without the padding process, prediction function should be like \"predict = lambda X: sigmoid(np.dot(X, betas.T) + beta_zero) > .5\") "
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "The shapes of the constructed predictor matrices and ground truth vectors can be seen with the code below. It is a good practice to check the outputs and matrix shapes before moving on. Desired samples' predictor vector and ground truth can also be seen with the code below. "
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 11,
   "metadata": {},
   "outputs": [
    {
     "name": "stdout",
     "output_type": "stream",
     "text": [
      "Shape of X matrix is: (351, 35)\n",
      "Shape of y matrix is: (351,)\n",
      "Shape of X_test matrix is: (351, 35)\n",
      "Shape of y_test matrix is: (351, 35)\n",
      "Shape of X_train matrix is: (351, 35)\n",
      "Shape of y_train matrix is: (351, 35)\n",
      "Desired samples feature vector: [ 1.       1.       0.       0.90116  0.16607  0.79299  0.37379  0.7299\n",
      "  0.50515  0.59784  0.72997  0.44303  0.81152  0.24412  0.87493  0.06438\n",
      "  0.85038 -0.12611  0.87396 -0.28739  0.79617 -0.46635  0.65924 -0.57135\n",
      "  0.53805 -0.68159  0.39951 -0.71844  0.25835 -0.72369  0.11218 -0.71475\n",
      " -0.05525 -0.67699 -0.19904]\n",
      "Desired samples ground truth: 0\n"
     ]
    }
   ],
   "source": [
    "print(\"Shape of X matrix is: \" + str(X.shape))\n",
    "print(\"Shape of y matrix is: \" + str(y.shape))\n",
    "print(\"Shape of X_test matrix is: \" + str(X.shape))\n",
    "print(\"Shape of y_test matrix is: \" + str(X.shape))\n",
    "print(\"Shape of X_train matrix is: \" + str(X.shape))\n",
    "print(\"Shape of y_train matrix is: \" + str(X.shape))\n",
    "\n",
    "print(\"Desired samples feature vector: \" + str(X[2]))\n",
    "print(\"Desired samples ground truth: \" + str(y[2]))"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "The initialization of predictors should be before the training process. There are various types of initialization for predictors in the literature. One can simply initialize the predictors with uniform random decimals ranging from 0 to 1. In this implementation, a uniform random initialization is employed as below."
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 12,
   "metadata": {},
   "outputs": [],
   "source": [
    "betas = np.random.random(X.shape[1]).reshape(1, -1)"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "Employed strategy to stop-training is as follows:\n",
    "- Collect test error values for each iteration.\n",
    "- Average test error values collected from a full pass of the whole train set (simply for all mini batches). This process might be necessary since mini-batch gradient descent can cause small oscilations.\n",
    "- Observe that average for each whole train pass if it's changed for a desired minimum delta over iterations or not. \n",
    "- If it's not, wait for a patience number to see if it's decreased further.\n",
    "- If the patience number is exceeded, assume the model as it's converged to an optimum point.\n",
    "\n",
    "The trained model is expected to generalize the data well and will be tested on the test dataset. Obviously, training procedure for model is being conducted on train dataset. However, the model can overfit on the training dataset and this condition should be checked since it prevents the model to generalize the data. Hence, in training stage, the model should have no relation for training since the model will be tested on the test set if it generalized the data well or not.\n",
    "\n",
    "The training process is as follows:\n",
    "- Mini batches preperation\n",
    "- Iteration phase:\n",
    "    - Calculation of new predictors with gradient descent for each mini batch.\n",
    "    - Calculation of models current error with logloss function with train set\n",
    "    - Calculation of models current error with logloss function with test set\n",
    "    - Check if the model is converged with the employed stop-training strategy.\n",
    "    - If the model convergance takes a large number of iteration, stop it at a desired iteration number.\n",
    "    \n",
    "The code is located below."
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 13,
   "metadata": {},
   "outputs": [],
   "source": [
    "train_error_hist = list()\n",
    "test_error_hist = list()\n",
    "test_acc_hist = list()\n",
    "\n",
    "X_batch_list, y_batch_list = prepare_batches(X_train, y_train, batch_size)\n",
    "\n",
    "n_batches = len(y_batch_list)\n",
    "\n",
    "prev_average = 10000\n",
    "\n",
    "patience_counter = 0\n",
    "iteration_counter = 0 \n",
    "while iteration_counter < n_iterations:\n",
    "    for i in range(n_batches):\n",
    "        X_batch = X_batch_list[i]\n",
    "        y_batch = y_batch_list[i]\n",
    "\n",
    "        betas = gradient_descent(X_batch, y_batch, betas, lr)\n",
    "        \n",
    "        y_hat = sigmoid(np.dot(X_batch, betas.T))\n",
    "        train_error_hist.append(logloss(y_hat, y_batch) / len(y_batch))\n",
    "              \n",
    "        y_hat = sigmoid(np.dot(X_test, betas.T))\n",
    "        test_error_hist.append(logloss(y_hat, y_test) / len(y_test))\n",
    "        test_acc_hist.append(np.mean((predict_with_output(y_hat) == y_test.reshape(-1, 1)) * 1))\n",
    "          \n",
    "        iteration_counter += 1\n",
    "        \n",
    "    current_average = np.mean(train_error_hist[-n_batches:])\n",
    "        \n",
    "    if np.abs(prev_average - current_average) < min_delta:\n",
    "        patience_counter += 1\n",
    "    else:\n",
    "        patience_counter = 0\n",
    "        \n",
    "    prev_average = current_average\n",
    "    \n",
    "    if patience_counter == patience:\n",
    "        break"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "The model's performance is being tracked in terms of the determined loss function during training. (\"train_error_hist\" and \"test_error_hist\") The performance plot on both sets is located below."
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 14,
   "metadata": {},
   "outputs": [
    {
     "data": {
      "text/plain": [
       "<matplotlib.legend.Legend at 0x7fd0444ca908>"
      ]
     },
     "execution_count": 14,
     "metadata": {},
     "output_type": "execute_result"
    },
    {
     "data": {
      "image/png": "iVBORw0KGgoAAAANSUhEUgAAAYUAAAEWCAYAAACJ0YulAAAABHNCSVQICAgIfAhkiAAAAAlwSFlzAAALEgAACxIB0t1+/AAAADl0RVh0U29mdHdhcmUAbWF0cGxvdGxpYiB2ZXJzaW9uIDMuMC4yLCBodHRwOi8vbWF0cGxvdGxpYi5vcmcvOIA7rQAAIABJREFUeJzsnXecXGX1/99nd2d7LymbTe+VEBJIgBCaBBAFpSv6lWIAUfEniqgoTRBFEQEVMXQQROmdBAhJCElIJ4WQtkk2bXvvO8/vj+fendk+SXZ2spnzfr3mNTO3zZl2P88pz7lijEFRFEVRACJCbYCiKIpy5KCioCiKojSjoqAoiqI0o6KgKIqiNKOioCiKojSjoqAoiqI0o6KgKAEgIgtE5JoQvXaciLwhImUi8t921v9KROaGwjY/G94Rkf8LpQ1K9xAVagOU0CAiucA1xpj5obblUBARA6wHjjHGeJ1lvwNyjDHfC6VtQeAioC+QYYxpbL3SGHOP+1hEhgA7AE9723YHInI7MMIYc4WfDecE47WUnkc9BaU3kw1cFmojDgaxHOz/bjDwZbBO8v6IiA4UwxwVBaUNIvJ9EdkqIsUi8rqIZDvLRUT+IiL5TihjnYhMcNadKyIbRaRCRPaIyM/aOW6MiJS6+zjLskSkRkT6iEimiLzpbFMsIou6OIH+EbijvROZiJwqInmtluWKyJnO49tF5L8i8qxj8+ciMkpEfum8v90iclarww4XkeXOe39NRNL9jj1dRJY4tq8VkVP91i0QkbtF5BOgGhjWjr1jne1KRWSDiHzdWX4H8FvgUhGpFJGr29n3dhF51nm60Lkvdbaf4WxzlYhsEpESEXlPRAb77W9E5AYR2QJscZb91fkMykVkpYjMdJafDfzKz561fu/xGudxhIjcKiI7nc/yaRFJcdYNcV7v/0Rkl4gUisiv/Ww5XkRWOK97QETub/1+lSBjjNFbGN6AXODMdpafDhQCU4AY4CFgobNuNrASSAUEGAv0d9btA2Y6j9OAKR287uPA3X7PbwDedR7/HngE8Di3mYB0cBwDjHTsucZZ9jvgSefxqUBeR+8ZuB2odd5TFPA0Nuzya+e1vw/s8Nt3AbAHmAAkAC8BzzrrBgBFwLnYgdZXnOdZfvvuAsY7r+VpZZcH2Io92UY730EFMNrP1mc7+S5v97NliPPZRPmtv8A5/ljn9W8FlrT6LOcB6UCcs+wKIMPZ/iZgPxDbkT3Oe3S/h6uc1xsGJAIvA8+0su9fQBxwDFAHjHXWfwp8x3mcCEwP9X8l3G7qKSit+TbwuDFmlTGmDvglMMOJVTcAScAY7Ml6kzFmn7NfAzBORJKNMSXGmFUdHP/fwOV+z7/lLHOP0R8YbIxpMMYsMs7ZoQMM8BvgtyISc9DvFBYZY94zNizzXyALuNcY0wC8AAwRkVS/7Z8xxqw3xlQ5r3uJiERiT6BvG2PeNsZ4jTHzgBVYkXB50hizwRjT6Bzfn+nYE+C9xph6Y8yHwJu0/JwOh2uB3zvfVyNwDzDZ31tw1hcbY2oAjDHPGmOKHHv/jB0gjA7w9b4N3G+M2W6MqcT+hi5r5dHdYYypMcasBdZixQHsb2CEiGQaYyqNMUsP+V0rh4SKgtKabGCn+8T5UxcBA5yT1cPA34ADIvKoiCQ7m16IPQnuFJGP3bBFO3wIxInICc5JaTLwirPuPuwI830R2S4it3RlrDHmbewofM7BvlHggN/jGqDQGNPk9xzsydplt9/jndgRfiY25n+xE/opFZFS4GSswLW3b2uygd3GSZj7HX9AwO+kcwYDf/WzrRjr6fkfv4V9InKTE24qc/ZJwb7XQGjxG3IeR2GT5S77/R5X4/ucrwZGAV+IyGcicl6Ar6l0EyoKSmv2Yk8iAIhIAjaMsAfAGPOgMeY4bChkFPBzZ/lnxpjzgT7Aq8CL7R3cOfG9iB0Ffwt40xhT4ayrMMbcZIwZBnwN+KmInBGAzbdiwz7xfsuq/J87I/qsAI7VGQP9Hg/CjmoLsSfUZ4wxqX63BGPMvX7bd+bx7AUGtsqfDML5zA+S9l5nN3BtK/vijDFL2tvPyR/8ArgESDPGpAJlWCHp6r1Aq98Q9r000lKE2zfemC3GmMuxv6M/AP9zfoNKD6GiEN54RCTW7xaFDeVcKSKTnZDMPcAyY0yuiExzRvge7Em3FmgSkWgR+baIpDihkXKgqcNXta9xKTbM4IaOEJHzRGSEiIjfMTo7DgDGmAXA54B/nfyXQKyIfNWx91ZsCORwuEJExolIPHAn8D/Hs3gW+JqIzBaRSOezPFVEcgI87jLs53mziHicJPXXsCGsg6UA8NIymf0I8EsRGQ8gIikicnEnx0jCnsQLgCgR+S2Q7Lf+ADa01tH543ng/4nIUBFJxP6G/mMCqJ4SkStEJMsZPJQ6i7v8DSjdh4pCePM2Nkzi3m43xnyAjZe/hE0eD8dX9pmMTRCWYEMCRcCfnHXfAXJFpBy4DhtnbxdjjHsSzAbe8Vs1EpgPVGITjn93TviBcCs2Ueq+RhnwA2AudsRdBeS1v2vAPAM8iZN0BX7svNZu4HxsorgAOzL/OQH+v4wx9cDXgXOwnsffge8aY744WAONMdXA3cAnTrhoujHmFeyo+wXn+1nvvFZHvIf9Xr7Efs+1tAwvuRPoikSkvdzR49jPaiE2eV8L/CjAt3A2sEFEKoG/ApcZY2oD3FfpBqTzPJ6iKIoSTqinoCiKojSjoqAoiqI0EzRRcJJty8XO7tzgzMxsvU2MiPxH7OzZZU4tvKIoihIigukp1AGnG2OOwdainy0i01ttczVQYowZAfwFmwxTFEVRQkTQml85M1Ernadu24LWWe3zsVPmAf4HPCwi0tks1szMTDNkyJDuNVZRFOUoZ+XKlYXGmC7n6gS1I6IzYWglMAL4m1OK6M8AnFI3Y0yjiJRhJ0oVtjrOHJwZq4MGDWLFihXBNFtRFOWoQ0R2dr1VkBPNxpgmY8xkIAc4Xvy6YzpIe7u1c5xHjTFTjTFTs7IOd1KqoiiK0hE9Un1kjCnFdlE8u9WqPJzWAc5s2hRsXxZFURQlBASz+ijL7TApInHAmUDrGZqv42tNcBHwYRddMRVFUZQgEsycQn/gKSevEAG8aIx5U0TuBFYYY14HHgOeEZGtWA+hV11FS1GUg6ehoYG8vDxqa7V7RTCIjY0lJycHj8dzSPsHs/poHXBsO8t/6/e4FuisMZeiKEcZeXl5JCUlMWTIEGzvQ6W7MMZQVFREXl4eQ4cOPaRj6IxmRVF6lNraWjIyMlQQgoCIkJGRcVhemIqCoig9jgpC8DjczzY8RaFiP2x6M9RWKIqiHHGEpyiseAJe/A40tb5UrqIoRztFRUVMnjyZyZMn069fPwYMGND8vL6+PuDjPP744+zfv7/rDXsZQZ3RfMRSXQTGC421EHloGXpFUXonGRkZrFmzBoDbb7+dxMREfvaznx30cR5//HGmTJlCv379DsmOxsZGoqKiOnwe6H7dTXiKQq1zlb/GOohJCq0tiqIcMTz11FP87W9/o76+nhNPPJGHH34Yr9fLlVdeyZo1azDGMGfOHPr27cuaNWu49NJLiYuLY/ny5URHRzcfZ8uWLfzwhz+ksLCQhIQE5s6dy6hRo7jiiivo27cvq1atYtq0aURHR1NQUMD27dvp168fjz76KNdddx2rVq3C4/HwwAMPcMoppzB37lzmz59PZWUldXV1zJs3L2ifQXiKQk2JvW/UOmlFCSV3vLGBjXvLu/WY47KTue1r4w96v/Xr1/PKK6+wZMkSoqKimDNnDi+88ALDhw+nsLCQzz//HIDS0lJSU1N56KGHePjhh5k8eXKbY82ZM4e5c+cyfPhwPvnkE374wx/y/vvvA7Bt2zY++OADIiIiuPXWW1m9ejULFy4kNjaWP/zhD0RHR/P555+zYcMGzj33XLZs2QLAp59+ypo1a0hLSzuMT6drwlQU/DwFRVEUYP78+Xz22WdMnToVgJqaGgYOHMjs2bPZvHkzN954I+eeey5nnXVWp8cpLS1l6dKlXHjhhc3LGhsbmx9ffPHFRET40rnnn38+sbGxACxevJif//znAIwfP57s7Gy2bt0KwFlnnRV0QYCwFQXXU1BRUJRQcigj+mBhjOGqq67irrvuarNu3bp1vPPOOzz44IO89NJLPProo50eJzMzszlv0ZqEhIQOn3fW5af1fsEiPKuPmnMKGj5SFMVy5pln8uKLL1JYaDv3FxUVsWvXLgoKCjDGcPHFF3PHHXewatUqAJKSkqioqGhznLS0NPr3788rr7wCgNfrZe3atQHZcMopp/Dcc88BsGnTJvbt28eIESO64+0FTPh5Cl6vegqKorRh4sSJ3HbbbZx55pl4vV48Hg+PPPIIkZGRXH311RhjEBH+8Ad7gcgrr7ySa665pt1E8wsvvMD111/P7bffTn19PVdccQXHHHNMlzb86Ec/4tprr2XixIl4PB6efvrpFsftCaS3NSWdOnWqOayL7NSWwb2D7OPvvArDT+sewxRFCYhNmzYxduzYUJtxVNPeZywiK40xU7vaN/zCR26SGdRTUBRFaUUYikKJ77HmFBRFUVoQfqJQq56CoihKR4SfKKinoCiK0iFhKArqKSiKonREGIqCegqKoigdEX6iUFsKEc70DPUUFCXs6I7W2VdeeSWbN28OsqWhIfwmr9WUQHwGVBVCk4qCooQbgbTONsZgjGnRo8ifJ554otvtOtRW2l3ZerCEn6dQUwJxaRAVo+EjRVGa2bp1KxMmTOC6665jypQp7Nu3jzlz5jB16lTGjx/PnXfe2bztySefzJo1a2hsbCQ1NZVbbrmFY445hhkzZpCfn9/m2JWVlXzve9/j+OOP59hjj+WNN94AYO7cuVx22WWcd955nHPOOcyfP58zzzyTyy67jGOPPRaAP/7xj0yYMIEJEybw0EMPdWhrdxGGnkKpFYXKAxo+UpRQ884tsP/z7j1mv4lwzr2HtOvGjRt54okneOSRRwC49957SU9Pp7GxkdNOO42LLrqIcePGtdinrKyMWbNmce+99/LTn/6Uxx9/nFtuuaXFNnfeeSdnn302Tz75JCUlJZxwwgl85StfAVq2xJ4/fz5Lly5l48aNDBo0iOXLl/Pcc8+xfPlympqaOP7445k1axbx8fFtbO0uwtBTKIXYVIiKVU9BUZQWDB8+nGnTpjU/f/7555kyZQpTpkxh06ZNbNy4sc0+cXFxnHPOOQAcd9xx5Obmttnm/fff5+6772by5Mmcdtpp1NbWsmvXLqBtS+wZM2YwaJBtxbNo0SIuvPBC4uPjSUpK4oILLmDx4sXt2tpdhJ+nUFtqRxJRMeopKEqoOcQRfbDwb0+9ZcsW/vrXv7J8+XJSU1O54oorqK1tO5D0b1gXGRnZ4toJLsYYXn31VYYPH95i+cKFC4+4Vtph6CmUQJx6CoqidE55eTlJSUkkJyezb98+3nvvvUM+1uzZs3nwwQebn69evTqg/U455RReeeUVampqqKys5LXXXmPmzJmHbEcghJen0NQA9ZV+iWb1FBRFaZ8pU6Ywbtw4JkyYwLBhwzjppJMO+Vi33XYbP/nJT5g4cSJer5cRI0bw2muvdbnf8ccfz+WXX94cJrr++uuZOHFi89XYgkF4tc6uLIA/jYBz7oP1/wNPHHy36y9GUZTuQ1tnBx9tnR0objM89RQURVHaJbxEwW1xEZemOQVFUZR2CFNRSFVPQVFCSG8LW/cmDvezDTNR8A8fqaegKKEgNjaWoqIiFYYgYIyhqKiI2NjYQz5GeFUfuZ5CrOspBNb8SlGU7iMnJ4e8vDwKCgpCbcpRSWxsLDk5OYe8f9BEQUQGAk8D/QAv8Kgx5q+ttjkVeA3Y4Sx62RhzJ8HCTTTHpkCk9j5SlFDg8XgYOnRoqM1QOiCYnkIjcJMxZpWIJAErRWSeMab1PPFFxpjzgmiHj5oSiEmGyCgnfKQ5BUVRFH+CllMwxuwzxqxyHlcAm4ABwXq9gKgptUlm0C6piqIo7dAjiWYRGQIcCyxrZ/UMEVkrIu+IyPgO9p8jIitEZMVhxSFrSmw+Aayn4G0Ab9OhH09RFOUoI+iiICKJwEvAT4wx5a1WrwIGG2OOAR4CXm3vGMaYR40xU40xU7Oysg7JjgWb89mwfRd1nhS7ICrG3msISVEUpZmgioKIeLCC8Jwx5uXW640x5caYSufx24BHRDKDYUtkhBDTUEZlRJJdEOWUbGkISVEUpZmgiYKICPAYsMkYc38H2/RztkNEjnfsKQqGPYPS40mRKkqN025WPQVFUZQ2BLP66CTgO8DnIrLGWfYrYBCAMeYR4CLgehFpBGqAy0yQZrRkp8RiqGR7UzzDQT0FRVGUdgiaKBhjFgPSxTYPAw8HywZ/PE01IE3sr3fEQD0FRVGUNoRPmwtn4lperSMG6ikoiqK0IXxEwWlxsbPKuXSeegqKoihtCCNRsJ7C7toYquoa1VNQFEVphzASBesplJpEdpdU+zyFJm2KpyiK4hI+opAxnPxjbmCfSWdXkZ8oqKegKIrSTPiIQt/xRM++nVKS2FVc7Rc+0pyCoiiKS/iIApAS5yEpNordxeopKIqitEdYiYKIMCg9vpWnoKKgKIriElaiAPiJgpakKoqitCYsRWF3SQ3eCA0fKYqitCbsRGFgejz1jV7yq50WS+opKIqiNBN2ojAoPR6AXSU1ziU51VNQFEVxCV9RcPMK6ikoiqI0E3aikJ0aR4Tgq0BST0FRFKWZsBOF6KgI+qfE+eYqqKegKIrSTNiJAviVpUaqKCiKovgT3qIQFauioCiK4kd4ikJGPAUVdTRFRrfMKSy6H5Y9GjrDFEVRQkwwr9F8xDKmXxIA1U1RJPl7Cmueg7I9MOFCSMgIkXWKoiihIyw9hbH9kwEob4ps6SlUF0FjDSz/Z4gsUxRFCS1hKQr9U2JJjo2irD7Sl1Noamy+OhvL/gl1FaEzUFEUJUSEpSiICGP7J1NUJz5PobYUMDDxEvt45VMhtVFRFCUUhKUogA0hFdYKxhWF6iJ7P2o2DJkJnz6slUmKooQdYSwKSVQ3ReFtaCUK8Rlw8k+gYh98/r/QGagoihICwlgUkqnD074oDD8DopNg/7rQGagoihICwlYURvVNoh4P4oaI/EVBBKLjoaEmdAYqiqKEgLAVhVhPJPHx8USZejDGTxTS7X1UrIqCoihhR9iKAkBKsp3ERmMdVBeDJwE8cXaZJ97OWVAURQkjwloUMlLsJLaKqirrKcT7zWL2qKegKEr4EdaikJVqRWHbvkJHFNJ9Kz3x0KDXWlAUJbwIa1Hok54CwPZ9RW09hahYaKgOkWWKoiihIaxFITXJ5hRyD5S0Ez6K06uyKYoSdgRNFERkoIh8JCKbRGSDiNzYzjYiIg+KyFYRWSciU4JlT7s2emIB2J1fbBPNrUVBPQVFUcKMYLbObgRuMsasEpEkYKWIzDPGbPTb5hxgpHM7AfiHc98zRFlRKC0tBlPejiiop6AoSngRNE/BGLPPGLPKeVwBbAIGtNrsfOBpY1kKpIpI/2DZ1IaoGAAS6/Ltc/9Ec1ScVh8pihJ29EhOQUSGAMcCy1qtGgDs9nueR1vhCB6Op9BPiu3zNjkFFQVFUcKLoIuCiCQCLwE/McaUt17dzi6mnWPMEZEVIrKioKCg+4xzPIX+7YpCPDTVg7ep+15PURTlCCeooiAiHqwgPGeMebmdTfKAgX7Pc4C9rTcyxjxqjJlqjJmalZXVfQZ26inYdRpCUhQlnOhSFERkiIhEO49PFpEfiEhyAPsJ8BiwyRhzfwebvQ5816lCmg6UGWP2HYT9h0dkNAA5kSX2eWtPAVQUFEUJKwLxFF4FjIgMB54GxgL/DmC/k4DvAKeLyBrndq6IXCci1znbvA1sB7YC/wJ+cNDv4HBwPIXsCNdTSG+zTvMKiqKEE4GUpHqNMQ0i8k3gAWPMgyKyuqudjDGLaT9n4L+NAW4IzNQg4OQU0r0lVJJAYqTHt85tjKeegqIoYUQgnkKjiFyMHfW/6SzzdLJ978HxBiLwUmiSaPL65bhVFBRFCUMCEYWrgNOAPxpjtovIUOD54JrVQzieAkCxSeRAud9kNRUFRVHCkC7DR8aY9TixfhFJAeKMMXcH27AeISISIjzgbaDYJNFQXE12qiMGUc695hQURQkjAqk++kBEkkUkDfgc+LeI3Bd803oIJ4RUYpLYVezX60g9BUVRwpBAwkfpzqSzbwJPGWMmA7ODa1YP4oSQSkhit4qCoihhTiCiECUiWcDFwBtBtqfncTyFxth0dpf4CYCKgqIoYUggonA38DGw2xizXESGATuCa1YP4ngKkQkZLcNHzTkF7ZSqKEr4EEii+QXgBb/n27HdTY8OHE8hJiWLXfvaCx/pNRUURQkfAkk0Z4vIiyKyz7n9R0Sye8K4HsFtn53al4KKOmrqnQZ4Gj5SFCUMCSR89AQwDxji3OY5y44OHFFIy7KXccgrcTyDiEjbG0lFQVGUMCIQUehrjPmXMabOuc0F+gbbsB7DEYXMvtb5aZNXUFFQFCWMCEQUikXkMvFxKVAcbMN6jKhYkAiy+/YDaFuWqpPXFEUJIwJtc/FdoBAowPZAuiqYRvUoUTEQl0Zmchxxnkh2FfuXpcaqp6AoSlgRSPVRLnCu/zIR+SHwcJBs6ln6HwPeJkSEQenx7Cqu8q3zxKsoKIoSVgTSOrs9buZoEYWZNzU/HJQRT26hnyhEqaegKEp4caiX4+z0Ogm9laGZCewsrsbrttD2xOvkNUVRwopDFQXT9Sa9j8EZ8dQ3etnvttD2xOnkNUVRwooOw0ciUkL7J38BkoJmUQgZkpEAQG5hlW2h7YmFBvUUFEUJHzrLKWT2mBVHCEMyHVEoqubEETiJZvUUFEUJHzoUBWNMU08aciTQPzmW6KgIdhY5yeaoWM0pKIoSVhxqTuGoJCLClqXmuqKgJamKooQZKgqtGJIRz84iJ2Skk9cURQkzVBRaMSQjgdyiKluW6okHbwM0NYbaLEVRlB7hUKuPjDEmPWhWhZDBmQnUNnjJr6ijn3OtBRprIPKoLLhSFEVpgVYftWJIRjwAuUVV9PO/pkKMioKiKEc/HYaPjDFN/jcgBdsy270dlbhzFXYWVemFdhRFCTsCufLaV0XkSyAPWObcfxhsw0JFdmocnkght6haRUFRlLAjkETz3cBJwGZjzEBgNrAgmEaFksgIYWC60xgvyhEFvaaCoihhQiCi0GiMKQAiRESMMfOAKUG2K6TYCiT1FBRFCT8CaZ1dJiIJwGLgaRHJB7zBNSu0DMlIYOn2IkxUim0Hq6KgKEqYEIincAFQC/wEGzbaA5wXRJtCzpDMeKrrmyhpcDRTRUFRlDAhEFH4pVOB1GCMecwYcz/w02AbFkoGOxVIeZXOAu1/pChKmBCIKJzdzrKvdrchRxLuXIVdFc7cPe2UqihKmNChKIjItSKyGhgtIqv8bluAjV0dWEQeF5F8EVnfwfpTRaRMRNY4t98e+tvoXgakxhEVIWwrcRrF6jUVFEUJEzpLNL8IfAD8HrjFb3mFMSY/gGM/ib2O89OdbLPIGHPE5SeiIiOYmJPCsjwnl6CegqIoYUJnM5pLjDFbjTEXA3HAV5xbViAHNsYsBIq7xcoQcOLwDD7Lc8RAcwqKooQJgcxovgHrNQxybi+KyA+66fVniMhaEXlHRMZ3YsMcEVkhIisKCgq66aU758ThmTR4haaIaPUUFEUJGwKZp3AtcLwxphJARO4BlgB/P8zXXgUMNsZUisi5wKvAyPY2NMY8CjwKMHXq1PY6t3Y7xw1OIzoygnqJIU5zCoqihAmBVB8J0OD3vMFZdlgYY8pdoTHGvA14ROSI6cwa64lkyuBUqrwe9RQURQkbOrueQpQxphF4BlgqIi85q74BPHW4Lywi/YADxhgjIsdjBarocI/bnZw4PJOKPA9JtdXEhNoYRVGUHqCz8NFyYIox5o8i8hEwE+shXGeM+ayrA4vI88CpQKaI5AG3AR4AY8wjwEXA9SLSCNQAlxljeiQ0FCgnjcig9uNoSsvKjt5e4YqiKH50JgrNISJHBLoUAn+MMZd3sf5hbMnqEcuknFQ2SgzlFRUqCoqihAWdiUKWiHTYzsJpd3FU44mMIDo2gZrqilCboiiK0iN0lmiOBBKBpA5uYUFCYhI01HCgXCuQFEU5+unMU9hnjLmzxyw5QklJTqK2oJ5FWwq56LicUJujKIoSVDrzFA677PRoIDkpmcTIBt5YuzfUpiiKogSdzkThjB6z4ghGPHEkRzWyaEsBBRV1oTZHURQlqHTW+6jX9i3qVjzxxFGP16DegqIoRz2BzGgOb6JiiWyqZcKAZF5dsyfU1iiKogQVFYWu8MSDt5FvTOrDurwytuZXdr2PoihKL0VFoSs8sQB8fVw6EQKvqbegKMpRjIpCV3jiAMiK9XLSiExeWb2HI6wbh6IoSrehotAVUVYUaKzhG8cOIK+khg82BXLhOUVRlN6HikJXOJ4CDTWcNymbEX0Sue31DVTXN4bWLkVRlCCgotAVnnh731BDNI38Y8pe9pZW8cD8LaG1S1EUJQioKHSFk2imoQY+vJORC67j12MLeGzxDjbsLQutbYqiKN2MikJXuJ7Ctg9hie30/a2hVaTFe/jVy5/T5NWks6IoRw8qCl0R5XgKi++HtMEQm0J82TZu/eo41uaV8d8Vu0Nrn6IoSjeiotAVrqdgDFzwCGSNgcIvOX9yNtOGpHHfe5spq2no/BiKoii9BBWFrohPB4mEk34Mg2dA5igo2IyIcNvXxlNcXc+DH3SQdG6ogSatUlIUpfegotAV8enw49Vw5h32edYYqC6EqiImDEjhsmkDeWpJLlvz27k622NfgQ/v6ll7FUVRDgMVhUBIGwziXF4ia7S9L9wMwM/OGk1cdCS3vrqe2oYm3z6NdbB/PeRv6mFjFUVRDh0VhYMlc5S9L7CikJEYw2/OG8eyHcVc8s9P2V/mXLazZCdgoHJ/aOxUFEU5BFQUDpaUgTb5XPhl86JLpg7kn1ccx7b8Sr7+8GLW7i6Fkh12ZcWBEBlugthZAAAgAElEQVSqKIpy8KgoHCwREZAxotlTcDlrfD9e+sGJeCIj+NHzq/EWbbcrqvLB29TOgRRFUY48VBQOBacstTVj+iXzi3PGsKu4mr07NtqFxgtVhT1s4CFQvCPUFnQPxsAbP4Hdn4XaEkXplagoHApZo6BsN9S1veDO7PF9yUiIpjjPz5M40vMK+9fDg5Nh55JQW3L41JTAyidg46uhtkRReiUqCodCplOBVNR2fkJMVCSXTBtIfOVumhL62oVHel6haKu9z98YWju6g+oie1+WF1o7FKWXoqJwKLhlqa3yCi6XT81hoBSwNXaiXXCkewoV++x96a7Q2tEduKE6FQVFOSRUFA6F9GEQEeUTBWOgsb559SBPKTHSwBslg+yCI91TKN9r70t2htaO7qD6KBOF126Aj+4JtRVKGKGicChEeqwwFH4JVUXw5FfhHydacYDmpO3q2r7Ue1J6gafg2Hc0eQqVB1oIda9l2wLYMi/UVihhhIrCoZI1GvJWwL9Og52f2PyCG5N35ig0JA8irzGZmuK9ITQ0AI7G8BEGKo7wz70rjIGqAt+cF0XpAVQUDpXM0dYDaKiBS562y7YvsPfFOyAiituumE2+N5XtudsprT6IUeu6F2HR/d1ucoe44aPqwnYrqnoV1X7lv709hFRXAU11tqKqpjTU1hw+6/4L79wSaiuULlBROFQmXgyTLoXvfwjjzrcT2rZ/bNeV7ICUgYzPyWD4sOEkNxZxzVMrqKkPcBLbqqdhxRPBs90fY2z4KNGplCrr5deHqCq0+R7o/aJQVeB7fDR4C5teg8/+dXSE9Y5igiYKIvK4iOSLyPoO1ouIPCgiW0VknYhMCZYtQaHPGPjmo5A60D4fOsuGkZoarKeQPhSArP6DyI4sY+WuYm58YXVgV2or3gE1xUE03o+6cmiogoEn2Oe9PYRUXQh9xtrHR4PAuRRvD50d3UVlPngboXhbqC05fPJWwDPfhIbaUFvS7QTTU3gSOLuT9ecAI53bHOAfQbQl+Aw7FeorYc9KO6pLs6JAYl8ivfXcPXsg7288wF1vbsSYToShsQ7K99hjNdYF3243yTxohr3v7RVIVYWQnAPxmb3fU/APhR0NM84r8+390TAfZvtHsO0D2Lsq1JZ0O0ETBWPMQqCz4e75wNPGshRIFZH+wbIn6Aw5GRDY8CrUljV7CiT1A+Bb42K45uShPLkkl8cWd/IHL90FOKJR3QPegptP6D/JXnq09CgQhYQMSMnp/aLgho8ioo6O8FGzKBwF7eTdwdTu5aG1IwiEMqcwAPD37/OcZW0QkTkiskJEVhQUFLS3SeiJT4fsybD2efvcz1MAoHI/vzp3LOdM6Mfdb2/i9+9sorq+nauyleT6HvdECMmtPErqD6mDenf4yBg7ozk+0xGFPaG26PBwRaHvBCjODakph01dpQ1TwtElCnlHX4+tUIqCtLOs3biKMeZRY8xUY8zUrKysIJt1GAydBbVOlUgrT4GKA0RECH+5dDIXH5fDPz/ezlfuX8i8jQfw+ucZ/MMEPeEptBGFXuwp1JaBtwESsmyL87LdvrkjvZGqQohJsQ0Ye7unUOV4CRJ5dISP/EWhN//G2iGUopAHDPR7ngP07sLyYaf6HqcNsfd+ngJArCeSP150DP+ZM5346Ei+//QKTvrDh9z15kY27i3veU+hfB/EpkB0PKQO7t2egtv3KCETUgbYvExtWWhtOhyqCux7SR9qw3y9Oanpho4GTLEDn/rq0NpzuFQesAJXeaD3hylbEUpReB34rlOFNB0oM8bsC6E9h8+g6RAZY4UgOsEui0myF+VxW12U5MKfx3KCbOStH8/kgUsnMz47mac/zeVrDy9mz46NEJtqt+0pTyEp2z5OHWRr4mvLg/+6wcANt7jhI+jdf9iqAuv1pA8DTO/24lxRGHYqYNptPd9rcMu4h860z4+yEFIwS1KfBz4FRotInohcLSLXich1ziZvA9uBrcC/gB8Ey5YewxMHw0+H/pN9y0SsSLitLta/bGfavnMz0eLlgmMHMPf/pvHZr8/kxOEZVO7bwu64MXbbnsopJDv5/VSnV5PrLZTlweZ3g29Dd+GWcCZk2PAR9HJRKLSegpuf6s0VSJXOoGjYqfa+N+cVqottmHL4GbY4I29FqC3qVqKCdWBjzOVdrDfADcF6/ZBxcTuTzpL6+TyFL96CmGQbV131FEy7GoDU+Gge++5UuKeApwsm8l1PLPUl+SQG297yfTZmDZA22N6X7oJ+E+Ctn8GW9+GXeTa8dKTjlnDGZ9r+VNC75ypUFdj5I25+qjfnFaoKAIGcaRAZ3bvzCu4ALyUHso+FvKOrAklnNHc3njh788f1FMr3wp4VcNKPYfDJ8NHdLWLe0bUFRJs6coaNo9CbwLufbeL//WcNXx6oCI6t3iY7gktyPQU/USjZCV++C6YJDmwIzut3N82eQiYk9IEIT+/1FLxemyNJyIT4DIhO6v2eQkImRMXYFjG92VNwk8xJ/SFnKuxb2zNzinoIFYWewPUUNr9tn4/5Gpx9j3VDF97n285JMp89cwaZWf2ZmN7Eexv2c85fF/Hb19ZTUtXN7QGqCuxJ362Qis8AT4KNXa94jOZisP1ru/d1g0VVIUQnWlGOiLDJ5vJeWpZaU2Iv5ZqQZUOQ6UN696zmynwr1GBnnB8VotDXej5N9bD/89Da1I2oKPQEiX2hvsI2BMsYYTus9j8Gjv02LH3EN4HMHQmmDSU6KYPRyQ188ovT+fYJg3h26U5O/dMC7nvvi+7zHNzXTXYSzSI2r1CwGVY9A2O/ZpPe+wIUhZ2f2jYf3UFtub3W8sE0gqsutMLmktyLJ7C5SfOETHufNrR3h48q8yHRTxTK846MyrCNr8HH93W9nT9u+CixH+Qcbx8fRclmFYWewC1L3b0UxnzVnnwBTv6pTVitf9k+L8kFxPZTik+H6iLSEqK58/wJvH3jTI4bnMY/Fmzj2gf+w+rfzWTe/HdpaPIeul3+brBL6iA7fb+mGI6fY2c671vX9bG2L4AnzrYdXruDnZ/Yay3v+DjwfdzErIv/rOZNb8IDk0JzwSNjYMEfoOAgKm6aRcGZl5M+1Ib0vAE2VQwmn82Fze8c3D4tRGGcvc//onvtOhTW/gc+fejg9qnYb+ePRMfbIo3kHBUF5SBJ6ut7POY83+OM4bZSaf1L9nnJDnsii4qBuPQWJalj+iXz+Pem8dkPx/BGyn0c27gO78f3cfqfF/DvZbvIr6i1M3gPpruqe70Bf1Fwk81ZY2DITOg3ySYFu/IAljqtq3YvbbncGGhqZ+Z2V7hhH/95G11RXWiTzC4pOdYbOrABXrnWhsVa29cTVBXCgntg3QsHsU8rUUgbagcQR0I4bNH9sOyfgW9vjJ285u8pABQcASGkqgLrsRyM11Kxv+V/Omcq5H7S++deOASt+kjxI9GJ2Sf2hQFTW66bcCHM+42NFxfv8E16i0+3s6O9XhsfB6jMJ+Oli8HUYMZ+jbO+eIv/xJTxq1c+51evwDMJDzCzaTl3bR5AZUw/Gr2GitoGoqoOUNvopTgiHYDjBqdxzcyh9K/YDxLh+7OCryz1+O9bj6b/ZBszLdhsK5Lao2gbfPkeIG3L81Y/A/PvgJ9utGIXKGWHIApVRdDvGN/zlBybM3n2IptnaKixsd9x5wd+zO6g0m+OSqA0J839PAWwvxH3OwoF7oV/Dua7rCuHxlqfx5wy0OZ+joS8giu+pbuhX0pg+1Qe8L0XgKlX2jDU2z+DC/7e/Ta6rH/ZDtb6jgvea6CeQs/gjsRHn+s7wbuM/4a9X/+yPWm4ohCXbhONtX4x9ecvs/MKvv0ictbdiDE8NnEDr91wEn852cvMJlsaV5a7lo+/LODTbYXsKq7mhuLf86uK35EUG0WcJ5Inl+Ryyh8/4rN1G6iNyWRXSZ2v1cbI2VaoJl1mn/efZO/3dxJCWv4viIi0f478TS0nv218zY7gKw5yXuLBegruySrBL6fgzlWoPAAXPwmZowILhXmb4OU5tny4O2gWhYOYfFZVYAU7Ls0+Tx9m7ze+Zr2yD3/XsrV2T1FbagcJpbvtgCUQKl2vxxl8RETYnNqa52HDK8GxM1Dcz/BgSpcr9rX0roedCrNuhjXPwepnu9M6H16v/U1+/t/gHN8P9RR6goQM+Or9MPKstutSB8LA6fYHVZXvGxHG21E9NSX2cU2Jbct9+q125jTAyLOQVU9zzKxfcMzCp22cs66MP50aDSedYbfxeuH326G+imcuHgTJ/dldXM3cRdupXZnHFyRywX0fkRAdyZDMBIZkJjCyzy2cVwYj+mAT4554ezKd/K229tdV2D/C+G/YfMmKx2Hvahg2y15MZecSu135Xp/gBcLBegp15Ta84h8+yhptO4yedbftYtt/EuxY1PWxVjwO6/5jP/MxXw3c5o44JE+hwCbNIyLt8+QBtix1xWO+bWJT4MQfHb59B4N7gvc22JNjSrs9LFvt47x/f4/0/L/BS9fAf78HW+fD2X+AmG6eldNYDx/9Dmb8CBLb6ZnWUGMLQMCKXCAYY/NS/uEjgFm/gF1L4a2brHfdkVd9qFQV2M/cLQoJIuop9BTTrvZdkKc1Ey70lRv6ewrgyyu4J5TM0S2PWXkA5t9uJ5md/BM7gvF3y0t2+LpTbnkPgIHp8dxx/gRO6tvAkCEj+P03J3Lx1IFkJcWwYU8ZD36whTPv/5hv/P0THl2cS3HiSEp3rGD9njIKKupaNvBb82/7xzrhehhwnF3mJt3yPoMGJ85afpBtrVxPoXRXYMlV/zkKLqkD4ZbdMN2ZRN9vos2jdDbCriyAD+6yj3ct657ErntSPJjLnbotLlwiImHOApjzMfx8O6QMgj0h6OXvNraDwNtutCcK6UPhqndh5s9g9XOw4PfdZ6PLvjXwyV9tCLM9/H8HZQH2/KottZdITWrV5T8iEi6ca6v13vlFx/vXVdpB1MFeXrXcKZhw27cEEfUUjgTGnQ/v/sKGi9JaewqtRMH1JABGnGnjy58+bEfIx8+B3EUtRcGtn46MtnH/477XvCqiYh+pQ07i8uNbxqjzK2p5dfUe/rsij3ve/oK4qCzOj/yErz20EEMEkRFCnCeSRKr5r/yZiojR3DeviYHpe7gxdgilqxcw33yD2flvMRhBMAcnCsbZPi7NjtbL9zTH0Y0xVNQ1Ul7TQFlNA+kJ0fRPifNrhtdqROg/E7ufXyhs+Ontv/b8262QzbwJFv3ZJtn7TQzc9vaobHUi7Tu+632qWpXXAmSO8D0ecKz1HLuiLA9e/YH1ms4NoPRy1zJ47Qa48p32R9ct3ssuGHxi18d04/aJrUbXkR444zd2RnDu4q6Pc7C4IaGtH8DMn3ZsFwTeCNKt2Gv9XsCK3oQLrTfX1OCbVe+y/WN4/Uf2N/DJX+HyF2yxSSC4nnMPeAoqCkcCSX1teGPHQj9PwYklt/YU3FnHYEcnx10JH9xhvYSYRMgaa8MfboL6wHrbzXHSpfD5/6zL7IlzGt+Vth3xAH2SYplzynC+P3MYpdUNNHyWR/KC+Tx5fha5ph/5FbU01VTw7S13kl1VyMN9b+JAeR0rd5ZwjHcws2pW8/t3NnFc9DxKGM7IiD288cFSHlo4nlhPJCnxHk6QDfSr28HChvF8VpVJXYPB67QgPqFvE8811VGQfjJZez5g9bo1bEuIYOn2IpZsLWRvWctuoeOzk7m27xd8HTDxGe32ZAd8J/d9HYjC7uWw5lk46UaY8n9WFHZ+2rUo5K2A3ctgRgddW9wTCdi8QkCiUGDj7h2RPcXmF6qKWuZR/PnyfXhlji/0OPuetieq1uxaAkVbbEWc62G1tsv/vQSC21HU9X5bM+A4WPKQ7QLriW25rrbMVtRtnW+T0/Hpttpn6lVdv65bjrx7qc1zxSa3ei+OpxCTEnj4qLmMu18H72UKLP2bHUy4358x8N6v7fL04fDVP8OHd8PcM+CSp2HoKV2/bvOcIvUUwodTfg4ZI30eQnueQnxG2x/28XNszH/qlfZ5n7HQWAOluTY5uX89ZI6E8RdYN3rHIhh1Fiz+CyDt5zkcRIS0hGgYOQ0WwKzkfcwaP93+wZ67CKo3wSVPcOO487nR3WnFHnjzYz6/LpvEp7axdvD3qNk3nwmxlZyYk0lNQxPlNQ1cuvdvDG3K5XtAuSeT+aNuZlvGLBqbDKXbbfjpkdz+/MYDz7+3kBebIkmL9zBjeAb/l5NKWkI0ybFR7CyqZt7GAyxZt5mve+CyZ7dw7KR4jh2UypCMBAZnxBPrifR9pikD284+rSqEzx6D5f+0HWNPudl2uU0eYE+SJ8xxvosSWPyATSq6XXDBJtrXvWBDeyPPbPtBVubbqpGCLwLPK1QXtvV6/HFDdXtXt3xNb5OdM7L2eZuU7DsBTvyxHTjsWQWDTuj8dd0R8+cvti8Klfk2AZ6QdRDho3y7fesii+b3MtVeu3n/OhjoTAZraoQP77LfS32F9fJqSq34rvk3jD2/YzF0cUXB22gHXGPPa7neFbjsyYH3Ympvbo8/2cfa+z2rfKJQvN0KwjHfsoIQHW+b6T1/mb3O849W+krBO6I8zzbfi+9AWLsRFYUjhaGntBwxxKTYP5/rKRT7XffZn5jEln9etwY8f5MVhQPrbVO1wSfbFhZfvmtDCUsfgWMu81UXdUafcXakl7fC9nhZ/Bco3GKb/7Uu78yZBkDSir+BaWLyrAtg0Tay6qv408V+I6d78mHcJTDkZJI/vItvRi6G2dfa9V9shxfgogu+iXn7eX56XDRzTj6FYZmJRES09QOunTWcyvmLYTEkpvfjX4u20+TkPSLEluDOHt+PWaOyGJgxjqi9a6mpbSAxJgpZ9TS8c7MtmRx5Fpz+G1/Cc9AMG9YwxpbnLnsUPnnAJvpHn+MzwD3Rv3sLDF1CVVMEOwqr2FZQSVJsFKeU7yey/wSkbA+U5OL1Gj7fU8aHX+SzYmcxRZX1lFY34DWG0f2SmNgvnptry2iIzaDDcX32ZECsB+CKQt5K+M+3bQI4NgVm/NAWJjTUwAd32omAgYrCnpW21Lh1eKMq34Yq04a2CLk0eQ2b91dwoLyW4qp6quob6ZMUS05aHKPL9+Pxzye0pjkXtcInCptes5/1uPPtJM/syb5t5p5hr5E88aLO30tZnq04K98HW+c1i0J1fSP7y2oZULafGLCj+x0fQ0MNxfWRLNlWyKfbisgrqaG4qp7SmnoSoqPITIzhsvrVnAfk1iUx2BhEWv0e04fZvMLeVYAzUHMnYM68yRfOTB8Kl/0bHppiw7ruwAN8F+3xP3b5Xhs6av16QUBF4UglIsK62/6egnPC7ZQsJxGdv8me1Mp224S0JxaGn2Z/gLWl9sd1+m8Cs8UTa8Xm04ft88zRcPnzMGp2O68/1nouG162I5uc4+2Ie/sC3zaVB2zcPmcaHPd/9g/rP1Jz4qdjx06CJQPp590PfZI6NTGxsRQ88Tz2/VlU1Dawo7CKHYVVfHmggg825fO7tzbxu7c28ZOoBH4cuZXjb3+duCj4OOoWciOH8cfY69mycwBxzxXTP2Up/VPiOKNyKOdW/o/n3v0YUnK4cOm/iAU2bVzLlzWTqG1ooqquiUsPbKU2JoeMoi089Pub+HPV2S1sWxezh7dKRzGNDPavWMkPlr9PeW0jIjAhO4WB6fFMyvHQ5IVN+8p5fckWbvbAXQsKyN+zkuOHpjM4I57BGQkMyYgnKjLCXqcjc1TLC8cvvt+Wi17ytC0tdkMxnjgbAtv+sfVyXIq2Wc8pKtq3rHSX/V7yVtCw5j+sG3E9a3aXsnZ3KVV1jdxcsp0sSaXCm0Fm4Wpe/jSXlTtLWLilkOIOenO9Hr2Nxtg0Fs3fwvRh6QzJTCArMcYn8Mn97W/EP0fy5Xs2hHrRE74KLLAj8dhU2NZKFOqrbd4s0u+UVrabprShVCUOJWrje/yZDazYVcqGPWU0eg2/jlrBtyNj+OOnXm4HLvrDi6yotIUKiTFRDM1MICMxmuFZCVTWNVFUVUdFYR6VJpZTH/yMrKQYhmYmMCg9noFp8eSkxZGTFse4jElE5n7G6q02PDVm/QckxfdlVVkqVQUHqKpvwhMhxHqSmJ40mJq1b7Ip45s0eQ21DU2MXHkHsWU7+N+4hyisrCO/oo4f79xETVM8Sz7cwg9PH9nxH6EbUFE4knFaXdDUYEc9Ey/uep+YJFuZkr/J1920rxMTH3U2fPEmrM+zVR+BlBO6TPk/2PYhTLvGxuM7CgVERtl4987FVpQ8sdbVrthvQwKRUb4eT27SvM84OyfAzXeU59k/eHyGzbH4h1zqKqxo9BnT8nWrfS0ukmI9TMpJZVKOvVjRz2ePYVdRNctzi+mzJ5+IVS/z+5MiSNu/hKQ91byRfSOZiWPIjoygsr6RfaU1LNlWyNaaPpwbAWsWv0W1ieXb0TbcsGzlSm5fZj/TWOq4KraQx7yXMD0ml+83/Zf4Wd8iO2cIw/skUlFRTvKzNaT2yaG6qpwRDXmcNyabaUPSmDWqD+kJ0bSmIS8J5sKYYcN4b3cJ727w5STiPJFMzEnhmJwULpDhDN6xlIfe3khGRDVXb36XrUMu59OSSVQttqGTlDgPqfEeJqVOI+fLZ1izdQ/GE09M6VbGvzqbrVN+zReDLqespoGiijp+UJTLh8nnkxVRS9rHT3HhvEmA0D8llpQ4D3Wl+1nvjWNNqYcbIvdxx2trSUmI59RRWZwyKotBGfGkx0cTHxNJfnkdeSXVDH29kuUyjAc++BIz376PmKgIBmfEMzwrkeFZiXwjejQZWz7lH+9sIibCcP3Gd8nLOJl3PtpOZV0jdQ1NxEVHkRAdyVeTp9F303v8J2M7RoTGxgYuXnYJ25KO4/nMH1FcVU9BRR3PFe/gjT392OQdxD2e91iy7FOSBo7n2lnDGJaZyKTlz1JXkk6f7BGQC+cOrOfUgaM4cUQmkwakWPH1NsHHf7Tl2GmDMS8+SsPebO6aPoHVu0rYXVzNoi0FHCj3dUm9KSqd6yMXc/XchdQSzYqYhbzpncRP/7WszXd9W9QYLi//kKvmLqKOaOKpZUXMS0Tg5S/zviAhNpqspBgyvUXkJh3DiD5Bb6avonBE47a6KMuzM3MDrfPvM9bGrw+st8/dmml3ZJ+QZRPTB8MJc1q6uJ2RM9WKwrBZ9nlytrW/Kt8+dhu7pfmJgvHaWdPZk+1JPznbCk/aENj0hu/Y835rE+kjZ8MZv/W9t6qClnMUWjEoI55BGU4sdxVckL4LNv4XRp3Nr751Wfs7eU/D3HcPv59UibdoHQ0Vg2mMSuCihHpO+fosYjyRJJZthSfgRxd9BRlwHPx9Olc3/BsmOv10om1V1DnTJ8MBD6xYyT0XTOg0DOCptSPMb51+HJcPPJ7iqnpyi6rZWVTF53vKWL2rlKeW7KSWPtzlKeadJSs52awm0tPIT78Yw4ZNbePjsyIyeSq6nvsff4bF3oncG/UoE6KaWLFsEb/8xApsJmXcGFvPl3VpFGTO5or8P/H8V6MZdswp9E22Xod5oJ66/pOZkH0ikR+8yqc3jCFjwKiWYT1jIH8TfbLHMiE7GV4u4YwZE1gx40w+31PG7pIadhVVsaOwms37K3h/4wEapC+/9HzIq5+sY6B3Lz+NLuOvu4fx5s4vifVEEBMVSU1DE/WNXnZHDuGPnnm88Pb7fGkGcmrEaq6N3sGeamFZeTHpCdEMSjSkFlcyZNho+o++AN5/jDfPqSXypBk+OzfWQlQ2P7jgNHgArhofBVNbjcK3L4CP77V5qMv/jVQeIDolm+9MH8x3pvvyALUNTewrqyWvpJqEHSVELXmVly5IpMmTSOYb5YydcS7/HnUCCTFRxEdH0tBkqGtsIm5nHbEfvMfr50FFzgz65r5G/AIrMJtuGk9M1jArTL8rImviBJjQQS6jG1FROJKJT3eubdBqZN0VfcbYmOveNfZE6ZbPJfaB035tE2AxnYdjDoths2DJg74kdrLjkZTvsyf74h02X+K2a2hukLbJikL5Hl+VRdoQ6y3Vltvk7sbXbfhq91J45GQYfJI9zoENnVfruKQMtOGHj++zCcxTbu5424gIZNAMoja/BXVlcNbv8OxZCfvWkpjljNgO2FCXpA218fcxX7VhGhe3AV9iX6ivskUAlfltJz/tW2eTodOvbzHnQkTISIwhIzGG4wan8c0p9nMxxsCeLJj7JAu/lYz30w00Vo3i0e9cRVy0PfEAlNc0UFLdQE3VZLzP3c89k4vYM2owJ7z+CXjhvAEVTP3mKSTHeUgvWWcF7htn2LzJnx5kRuWHkOx8j8YglQXEpvYjdoAtj81q2A8Ro1u+lxWP2UlcX3/I9vryNkBiXzISYzh1dNvcQkOTF5ObCM88z7LvpmF2fYH5JJJ7bv4Jf0lKxxMZ0WLbuqLx8Pd/8drZddRPO4vEl5+ErTAxJp9PfnGaFdyCzfA3mDn1WJg4DVaNInL7B3DSD30vXFVgf5tJ/W3OrL1ZzRucZpWb37I5m4r9NgfRilhPJEMzExiamQB9z4AlMJ5t0GhtHzvjPEhrZ9DS/xz4OJbRFUthyDdg0Rt2wqW3kZjSbZA1zP5evI0H59kfBjp57UjG9RTc8EnAnsI4G1v+8h07kvYflc66uf1cQHcy/HS46Utf6aVbW93cusJt/OeETtKH2Wtb52/wbef+Adz3XLrTVp5UF8Kpv4Ab19ra86Z6m8irKmh5GdSOELHx9foKa2fOcZ1vP2iGFYSoOJj8bV+C1W3y1/q7yRpr1zfU2Of+E7f834s/+9fDU+fB+7+2cwTcCpdOqo9EBOk30V5IaMOrROxeStSxlzEgLZ70hGhiPZHEeiLpkxzL6H5JTB4+gIicaQwq/YwZBf8lwjTB0FkklW9nZN8k+ibH4qlwToqpAyEu1apFOOQAABL6SURBVIr6hpd9ic/6SitqiX38rtLX6r00NdoafID3bvW1Xe8k0eyJjCA6Z4odKOxZgXz5HjL4RJJTM1sIgrttYp8hkDmauF0LSKnbQ+TWeVbs6yt9vzH3BO9O9hpxZtumdVVOhVdklBWH1mWpjfXWSx1zng1lfniX0wyvi9F6cn+7zZ5VVuhTB3VcXeSJswUmW963A4jtH8Gx37Hr3OtYN5ejqigo8Wk20VySa2PsXf0YXdzLa9aW2ZLEUOA/8alZFPyuG+FfSRUZBVmjrKfg9fo8CvCdSEtybd4hMhpGfMUmIc/4LVwzzzbb+00RnP7rwGxzPYrOvAQXd3LWpIut55Y+zI7a3JNOSa6t6nJnUmeOBAwUbbXPXVFI6uc7MfjnSIq2wTPfsDX402+wpaSL77fvsytvLirGCq87mu0q5zRslvUeVzwOY79uBwc1xXauA/iqidyeUUNPsfa778GduJbQx3pyEtl20teGV+yys35nBfs1Z2TeWfUR+ObYbHzdDg66GriMOMO2UFn6DysmZ/zWLi/YbO/dclRXFIaeYmciuz28mntlOb/T1EFtPYXtH9n/0JTv2gqo7R85otjOxLXWDDjOXmUxd3HX8xBGfMWWrS68z4ZRp19vRahZFJz3oqKgEJduSyXzN9kfrX8VRmdkjQZ3CtfhzsbtDuIz7EnObdVdvL1tKKzPODiw0eYdvA2+P4C/KGx6A4bOajtXAzpOfLfH9OvhG4/C4Bldb5s9xVZpnfpL+9xtTOe2JSndaW10vbHMUfbe/UNXHrAnrfgMX7jMnfRVsR+ePt+eCL7zqr0a36xb7InIveJaV7jlnG4YrTOGzgKM7RN10o/b2lq624qt+/lmjmy5vnlmst/o2n8CmzG2jDRztBW402/1ndACOpFO8bXTHnV259sOP8P+N5b9E8aca5vS+dtalmdFy+1Q7A6UXNGoLbXi3iwKA9sK3PqXbahx2Gm2gs8dlAUyOMs+1v5GakthSBei4JYUfzbXertZo+13U7jFLldPQWnGnaiyZ1X7cxQ6whPnO+mGylPwR8SO/Mv32hNeTXHb99NnnBUNt2LKHeHFpdma+y/esifg1hOQDoWUHDjm0sC2jYiAU37m81zcz9XN85TktgwNZIwAxPeHrjzgTNyKtN9LYj+fp/DR3Xb9d162nhLAqbfAV+7yhRC6wo1vT7qk621zptqmekNmWjFpfdIv3dVSWNw+W+6J1N9TAPu+/cNHW+fb4oaTbrSf2/TrfWXUXXkKrn1ghTdjROfbDj7RhhwxtiIuIcuewP09heRsX4lq6iBbIt0scK1ak6cMtPM73OuGNNTa39zY82yY0xPnK+ftqIeZP/55h6EzO9+2+f0a23kA7HfjL3A9NHENNNF8ZNPcFK/w4DqMgj3Jlu72jQZDTZIjCq3LUV3cZPNWp2bRf1SUNgR2fQqIbT8eShL72fxC8Q47Mi7J9Y1SwZbgpg1ueSL1HyW7JbbFO+zM3KlXt0yQi9hRfKCM/Zo9ViDlypEe+O6rvpFuysCWJ8rSXT5xAntSjU70CZzbDM89wacOsmXKLov/Yr8315aISNuyfMdCX9uWznC9nlFnd+0lRcfbz710p/WAROwI2/9E6t88LiLSnmjd76X15U5TB1qPrXyP/Y62zrd5J7e1PdiWMllj7WTQrnBnNmeMDKxf0aizYdkjtncS2P/tqqdtTrEHJ66BisKRjf/I4GBF4YTrbD8l/4lJoSQ5205Oal2O6uJeOGTLPGf7VqKwb62tiAlkxBlM3DLZ4u32xNJQ3fa78Xf9K/a3EoXBNha+8E+2yuTk/3d49sSm2KZygeKOxsGeKDNGWFuNsaIw8iu+9SLOiNUVOOdE6pb+pg62o+uGWtuBd+cntr+S/28uJaf9luvt0We8DdUF4vUAfPNReyL3D925lwkt2912smfmaNt8D9pe2c7No5Tutt/nhpftoGzoLN/+IoGFHMGK4JCZgTUMBNt6e9Ilvqo0dzBXtNWpxuuZ0BFo+OjIJu4wRGHoTOu+Hym44SM3Ft/aU0geYFt7FG1p6yq77707rm3QHaQPs6PzjqrCMkfZ9+Ftat9TKMuzCeWpV9lKlVDihimqCm0StXVeInN0S08hLt0XknHDZrmLbEJ5wHEw7fuHbosbqgv0ynJxqS1/J1mjrVddVWjnuvz/9u49Rq6yjOP497e7bGmBUtctBEqhpXZboCjFBcpFuSoXsW0CCFiEIkhUCBQFgkQxkCASDCqRYBBqqyEUgUYqBglCo9zadKGFgkAp0EKh0C3Xyh15/ON9z+zZ6czOtrszc2bm+SSb3Zk5O/uefXfPc97b8+anmR45IQS+j99LdYWlBpohBJMV94QB7z2ml04g2JeZd8Ghl/Tv2C2H924xJt1n61fEdTseFBz0/oPv7xqFrBq+Y5j98cpj4U4zf2aN1JO3afio3k3l7SeFqZe7fbNy5e1L29jQ4snfAyPR3hEGQd9eHS6k6TUJI3YBLAy8H7iJCwjLob0jlDPpdtkoKIwPd6ofbYgBrsDWrXecAQiOn13dlmkymLzqgTBZIT8o5AbWn4tjCupJT54cu/RmmDcjzOrqbxqYchixS/gbWfd0/zczGiQeFLIs3VJIp8yuRUm/6uqHe2bw5EuCQv4/wKTjwrqETW0tlUvb2HDRf2lReLzRhTRefF5aHGa45LcUIM5m6ceMnHJr7whdMEluqvxzSXJprV+x8cY/yd/kh+/A9OuqXz/J7z0Zl9o2b0A4/1yGpVo9LUPCeNHqB0OSyFPvrNjAbkHNLSHN9qoHQzaACuyjkPAxhSxraQ0DfVsMHfytCistaf5+8GbxVk8y2JyfM76puaJ3SiUlQe35+8Og7RZDe7+eXHxWxa0/03fXo/eDI6+EyTPKX87+KHUhTd9d/3dd71k12+wQJhDseVw2WnHbjg7JGFfGwe/8lkLbuDBNtfvZjQMchPGW99+Ab98axmqqrX18T4qXCuyjkPCgkHVD24pv6FFL0nc6xabXJoPNFbwr2ixJ+d9eHVY85xvWFrolckEhVX/NLbD/D8tfxv5K+q5fXRqmdOavAWnbNQyI5y6kqQDX1ASzlvfOTFpNTU3hfJIFavlBoaU13JCsfzZuTpQXFL7159BtWaFZPiW1dwBxNXkF/ye8+yjrdp+68Z4FtWir7cIiLijeUth+UrhDS6bzZdW2o8OFEop3mbR39CyGqvaMqb60DgtZdbHCA7zNW4TAsPbxkEYif4vOrASERNJKGzK88N1++4RUgMvLRdTUlJ2AAL2nk1dgb+aEB4WsO/IKOOCc0sdlXXNLzx1zsZbC0BFw0YvZmWVUTHNLzwW0r6CQ6M9q3mpKFrEVm/XT3hHyTkHvlkIWJQvuil1ER3aECQIbXut7Z7ssSOqlZWj/1nkMEg8KrnLyVwUX0tScrbu1YpJxhVJBoXXr7I8HJWUtNpmhvSO0EiDbrR7oWXxXNChMDIP/H2+onaBQwYVr4EHBVdLwHWLyuIz/M/ZH0topFRSyfhGF/rUUElmvu1IthV7nUnz/jUwYsk0YyK/wJIuyBgVJR0l6VtJKSRcXeH2mpG5Jy+LHmeUsj6uyzu+GJGm10BIoZbuJYSZL27jCryd3rFnvOoKeWV/FpgqnU19kPci17Rp+58XSqNdSgIOwWn3/ynYfl22USFIzcB3wNWANsETSAjPL3xbqVjOrg05zV9K4w8JHPZh8KoyesvHAayLJK1QLQWHnKXDK/JANtJBaupC2tML5T/VMBMg3ZOswvfPdNdkfH4H+pwgZROWcOrAvsNLMXgCQNA+YBmy8V6BztaaltWcr0EKamkPG0yxkqS1FCvsTFJN0Y3zyXljklXWlUlOM7IhBIePdR1VSzu6jUUB614o18bl8x0l6QtLtkgrmpJV0lqQuSV3d3d3lKKtzg++g83snmKtlIyf0Xm9Ry5JxBw8KBZWzpVCo49jyHv8NuMXMPpL0fWAusFH/gpndANwA0NnZmf8ezrlyO/IXPTOQat2XZ4ZuvyysWs6gcgaFNUD6zn8n4NX0AWb2RurhH4Crylge59zmSlab14PtJoYPV1A5u4+WAOMljZXUCpwELEgfICmdN3gq8HQZy+Occ66EsrUUzOxTSecA9wDNwGwze0rS5UCXmS0AzpU0FfgUeBOYWa7yOOecK01mtdVF39nZaV1dXdUuhnPO1RRJj5pZZ6njfEWzc865HA8KzjnncjwoOOecy/Gg4JxzLseDgnPOuZyam30kqRtYvZnf3g6sH8TiZJGfY33wc6wPWTrHXcysZEbDmgsKAyGpqz9TsmqZn2N98HOsD7V4jt595JxzLseDgnPOuZxGCwo3VLsAFeDnWB/8HOtDzZ1jQ40pOOec61ujtRScc871wYOCc865nIYJCpKOkvSspJWSLq52eQaDpNGSFkp6WtJTks6Lz7dJulfSc/Hz56pd1oGQ1CxpqaS74uOxkhbH87s17tdR0ySNiFvSPhPrc/96qkdJ58e/0Scl3SJpy3qoR0mzJa2T9GTquYL1puDaeA16QtLe1St5cQ0RFCQ1A9cBRwO7AydLqoetpD4FfmxmuwFTgLPjeV0M3Gdm44H74uNadh69N2C6Cvh1PL+3gDOqUqrB9VvgH2Y2EfgS4Xzroh4ljQLOBTrNbBJhf5WTqI96nAMclfdcsXo7GhgfP84Crq9QGTdJQwQFYF9gpZm9YGYfA/OAaVUu04CZ2Vozeyx+vYFwIRlFOLe58bC5wPTqlHDgJO0EfAO4MT4WYR/v2+MhNX1+AJKGA18FbgIws4/N7G3qqB4JG3oNldQCDAPWUgf1aGb/JmwQllas3qYBf7JgETAib/fJTGiUoDAKeDn1eE18rm5IGgNMBhYD25vZWgiBA9iueiUbsN8AFwGfxcefB942s0/j43qoy12BbuCPsZvsRklbUSf1aGavAL8CXiIEg3eAR6m/ekwUq7eauA41SlBQgefqZi6upK2BO4BZZvZutcszWCQdC6wzs0fTTxc4tNbrsgXYG7jezCYD71GjXUWFxD71acBYYEdgK0JXSr5ar8dSauJvt1GCwhpgdOrxTsCrVSrLoJK0BSEg3Gxm8+PTryfN0vh5XbXKN0AHAlMlrSJ0+R1GaDmMiN0QUB91uQZYY2aL4+PbCUGiXurxCOBFM+s2s0+A+cAB1F89JorVW01chxolKCwBxsfZDq2EQa4FVS7TgMX+9ZuAp83smtRLC4DT4tenAXdWumyDwcx+YmY7mdkYQp3db2YzgIXA8fGwmj2/hJm9BrwsaUJ86nDgP9RJPRK6jaZIGhb/ZpPzq6t6TClWbwuAU+MspCnAO0k3U5Y0zIpmSccQ7jKbgdlmdkWVizRgkg4CHgCW09PnfglhXOEvwM6Ef8gTzCx/MKymSDoEuMDMjpW0K6Hl0AYsBU4xs4+qWb6BkrQXYTC9FXgBOJ1w01YX9SjpMuBEwoy5pcCZhP70mq5HSbcAhxBSZL8O/Bz4KwXqLQbE3xFmK70PnG5mXdUod18aJig455wrrVG6j5xzzvWDBwXnnHM5HhScc87leFBwzjmX40HBOedcjgcF1zAkXSnpEEnTk0y5kuZIOj5+PUvSsEH8edPTiRclXS7piMF6f+fKwYOCayT7EdZwHExY35FvFiFZW7/FDLzFTCdk5QXAzC41s39uyvs7V2keFFzdk3S1pCeAfYBHCAunrpd0aeqYcwl5eRZKWhif+7qkRyQ9Jum2mGMKSaskXSrpQeAESd+TtETS45LuiCt3DwCmAldLWiZpXF6r5PCY/G55zMk/JPXel8WfuVzSxPj8wfF9lsXv26Ziv0DXUDwouLpnZhcSAsEcQmB4wsy+aGaXp465lpCH5lAzO1RSO/BT4Agz2xvoAn6UetsPzewgM5sHzDezfcws2QfhDDN7mJDW4EIz28vMnk++UdKWsSwnmtmehIR4P0i99/r4M68HLojPXQCcbWZ7AV8BPhiUX45zeTwouEYxGVgGTCTk3SllCqHr5yFJywg5bHZJvX5r6utJkh6QtByYAexR4r0nEBLErYiP5xL2U0gkiQ0fBcbErx8CroktmhGplNPODaqW0oc4V7tiTqE5hIyU6wljBooX+v37+lbgXjM7ucjr76W+ngNMN7PHJc0k5MLps1glXk/y//yP+D9qZr+U9HfgGGCRpCPM7JkS7+PcJvOWgqtrZrYsdrmsINz53w8cGbt08rtgNgBJX/0i4EBJXwCI4wQdRX7MNsDamMZ8RpH3S3sGGJO8N/Ad4F99nYekcWa23MyuInRlTezreOc2lwcFV/ckjQTeMrPPgIlmVqz76AbgbkkLzawbmAncEgepF1H8QvwzwqymewkX/MQ84MI4MDwuedLMPiRkQb0tdjl9Bvy+xGnMUtj0/nHCeMLdJY53brN4llTnnHM53lJwzjmX40HBOedcjgcF55xzOR4UnHPO5XhQcM45l+NBwTnnXI4HBeecczn/ByflLrS5uNarAAAAAElFTkSuQmCC\n",
      "text/plain": [
       "<Figure size 432x288 with 1 Axes>"
      ]
     },
     "metadata": {
      "needs_background": "light"
     },
     "output_type": "display_data"
    }
   ],
   "source": [
    "plt.plot(test_error_hist)\n",
    "plt.plot(train_error_hist)\n",
    "plt.xlabel(\"#Iterations\")\n",
    "plt.ylabel(\"Total Loss\")\n",
    "plt.title(\"Loss vs Number of iterations\")\n",
    "plt.legend((\"Test error\", \"Train error\"))"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "Finally, the accuracy of the model on the test dataset is located below."
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 15,
   "metadata": {},
   "outputs": [
    {
     "data": {
      "text/plain": [
       "Text(0.5, 1.0, 'Accuracy vs Number of iterations')"
      ]
     },
     "execution_count": 15,
     "metadata": {},
     "output_type": "execute_result"
    },
    {
     "data": {
      "image/png": "iVBORw0KGgoAAAANSUhEUgAAAYUAAAEWCAYAAACJ0YulAAAABHNCSVQICAgIfAhkiAAAAAlwSFlzAAALEgAACxIB0t1+/AAAADl0RVh0U29mdHdhcmUAbWF0cGxvdGxpYiB2ZXJzaW9uIDMuMC4yLCBodHRwOi8vbWF0cGxvdGxpYi5vcmcvOIA7rQAAIABJREFUeJzt3Xl8XHW9//HXJ0mTdF/TfaVL2hQoS2VVaKEiULZ7AaXiVdwqCrK5IQpXQL3qFQEvKCIoqCwi+tOClU1aNgFboFDSNm0pXdK0TbpkaZu0WT6/P85JmKZJM2nnZDKT9/PxyKNzzvnOOZ8zM53PnO92zN0REREByEh2ACIi0nkoKYiISBMlBRERaaKkICIiTZQURESkiZKCiIg0UVIQiZCZXWZmLyfx+F82sy1mttPMBjbbNjpcn5nE+C41s2eSdXzZn5JCmjCzhWa2w8xykh1LZ2ZmD5iZm9lxMesmmFnaDdgxs27Az4Az3L2Xu2+L3e7u68P19WH5hWb2hQjjGRu+9lkxMTzk7mdEdUxpPyWFNGBmY4GPAA6c18HHzmq7VKezHfh+soNor4N4rYcAuUBhBOHsJ5lXHJI4Sgrp4dPAa8ADwGdiN5hZdzO7zczWmVmFmb1sZt3DbR82s3+ZWbmZbTCzy8L1+/xibF4FEv7au8LMVgGrwnV3hvuoNLM3zOwjMeUzzewGM3vPzKrC7aPM7G4zu61ZvE+Y2TXNT9DM7jGznzZb9zczuy58/C0z2xjuv8jMTj/A6/UgcKSZndrSRjNba2azYpa/Z2Z/CB83/tr9bHi+O8zscjP7kJm9E76Wd+2/S/u/8PVfERubmfU1s/vNbFMY//cbv1zD1/0VM7vdzLYD32sh1hwzu8PMSsK/O8J1k4CisFi5mT3fwnObfrmb2Q8IfljcFVYp3RWWmWxmz5rZ9vB1/XjM8x8ws1+a2Xwz2wXMNLPZZvZW+DnYYGaxMb8YE89OMzuxhc/WSWa2KHytFpnZSTHbFprZreFrUmVmz5jZoHBbrpn9wcy2he/BIjMb0tL7K21wd/2l+B+wGvgKcCxQCwyJ2XY3sBAYAWQCJwE5wGigCpgDdAMGAkeFz1kIfCFmH5cBL8csO/AsMADoHq77VLiPLOBrwGYgN9z2DWApkA8YMC0sexxQAmSE5QYBu2PjjznmKcAGwMLl/kA1MDzc7wZgeLhtLDC+ldfqAYKrhKsazwmYEPxXaCqzFpgVs/w94A8x+3bgHoJf4WcANcBfgcHh61wKnBrz2tUB14av8yeACmBAuP2vwK+AnuHz/w18qdlzvxq+rt1bOJ9bCH4QDAbygH8BtzaLNauV12Kf7S287z3D1/Wz4fGPAbYCU2NeywrgZIIfmLnADOCIcPlIYAtwQWvxEPPZIvg87QD+KzzenHB5YEx87wGTgO7h8o/CbV8CngB6EHzOjwX6JPv/Zir+6UohxZnZh4ExwGPu/gbBf5pPhtsygM8BV7v7Rnevd/d/ufse4FLgOXd/xN1r3X2buy9px6H/x923u3s1gLv/IdxHnbvfRpB48sOyXwC+6+5FHng7LPtvgi+Vxl/OlwAL3X1LC8d7ieALpfEK5CLgVXcvAerD4xWYWTd3X+vu77UR/6+A0WZ2VjvOOdat7l7j7s8Au4BH3L3U3TeGsR4dU7YUuCN8nf9I8At+dvhL9izgGnff5e6lwO0Er0OjEnf/v/B1rW4hjkuBW8JjlwE3E3ypJsI5wFp3/214/DeBPxO89o3+5u6vuHtD+HosdPel4fI7wCNAi1dkLZgNrHL334fHewRYAZwbU+a37r4yfC0eA44K19cS/NCYEH7O33D3yoM/9a5LSSH1fQZ4xt23hssP80EV0iCCX28tfUGOamV9vDbELpjZ18xseXjZXw70DY/f1rEeJLjKIPz39y0VcncHHiX49QhB4nso3LYauIbgF32pmT1qZsMPFHyYGG8N/+xAZVsRm7iqW1juFbO8MYy/0TqCK5wxBFcPm8Iqj3KCZDU4puw+r3MLhof7a77vRBgDHN8YWxjfpcDQ1uIzs+PNbIGZlZlZBXA5H3wO2tL8XAiXR8Qsb455vJsPXuffA08Dj4bVaD+xoKFd2klJIYVZ0DbwceBUM9tsZpsJqimmmdk0gkv9GmB8C0/f0Mp6CH759ohZHtpCmaYvubD94FthLP3dvR/BFUDjl+2BjvUH4Pww3ikE1SmteQS4yMzGAMcT/GoNgnF/2N0br5oc+PEB9tPotwTJ6z+arY/n/NtjhJnFJp7RBNVmG4A9wCB37xf+9XH3qTFl2+oVVUJwzs33fTCaH2sD8EJMbP086K305QM852FgHjDK3fsSVLNZK2Wba34uEJzPxjYDD67Cbnb3AoIq0nMI2tqknZQUUtsFBFUnBQSX0UcRfLG+BHza3RuA3wA/M7PhFjT4nmhBt9WHgFlm9vGwoXGgmTVeii8B/tPMepjZBODzbcTRm6DuuwzIMrObgD4x2+8DbjWziRY40sI+8+5eDCwi+KX351aqSAjLvhUe4z7gaXcvBzCzfDM7LTyvGoJf6vVtvXjuXkdwdfGtZpuWAJeYWTczm86+1SUHYzBwVbi/iwneo/nuvgl4BrjNzPqYWYaZjbdWGsBb8QjwXTPLCxtdbyJItAdjC3BYzPKTwCQz+68w9m4WNKhPOcA+egPb3b3Ggm6/n4zZVgY0NDtGrPnh8T4ZfiY/QfDZfrKtwM1sppkdYUEjfSVBdVKbnwHZn5JCavsMQR3renff3PgH3AVcakEXxq8TNPIuIuiK+WOCht31wNkEjcLbCb4Ip4X7vR3YS/Al8SBhNc0BPA38A1hJcLlfw77VCj8jqP99huA/7P0EDYWNHiRonGyx6qiZR4BZBL9IG+UAPyK4MtpM8CV8Qxz7atzfpmbrbiS4stlBUEf/cPMntdPrwMQwvh8AF/kHYwY+DWQDy8LjPQ4Ma8e+vw8sBt4heJ/f5OC7295JcCW2w8x+7u5VBA3plxD8it9M8Pk50FiYrwC3mFkVQYJ6rHGDu+8mOP9XwuqoE2KfGL4m5xB8JrcB3wTOiakaPZChBK9dJbAceIGDT45dWmNPDpGkMbNTCP4Djw2vbkQkSXSlIEkVNgZeDdynhCCSfJEmBTM7MxzwstrMrm9h+xgz+6cFg34WmtnIKOORziWsmy4nqC65I8nhiAgRVh+FDT4rgY8CjY2Jc9x9WUyZPwFPuvuDZnYa8Fl3T1QfaxERaacorxSOA1a7+xp330vQx/z8ZmUKgH+Gjxe0sF1ERDpQlJOZjWDfHijFBH3LY70NXEjQ6+E/gN5mNtCbzeZoZnOBuQA9e/Y8dvLkyZEFLSKSjt54442t7p7XVrkok0JLo0Sb11V9nWACrssIJsvaSNDffd8nud8L3Aswffp0X7x4cWIjFRFJc2bWfLR4i6JMCsUE0xs0GkmzkZbhvDX/CWBmvYAL3b0iwphEROQAomxTWARMNLNxZpZNMABmXmwBMxsUTtoG8G2C0bciIpIkkSWFcAqBKwlGuy4nmMWz0MxuMbPGG8HMAIrMbCXBDUF+EFU8IiLStpQb0aw2BRGR9jOzN9x9elvlNKJZRESaKCmIiEgTJQUREWkSZZdUEZHIvPreNvr37MbkoX3aLtxBFqwo5a31OyLb/+lThjBtVL/I9g9KCiKSgjZs381nfvtvBvTI5vmvn0qP7OR/ldXWN3DVI29RtacOO5gbvMZhcJ9cJQURkeb+5x/LAdhcWcM9C9/jujPykxwRvLFuB1V76rjnU8dy5uGHegfX5FGbgoiklNfWbGP+0s1cOXMC500bzq9eXEPxjt3JDouFRWVkZRgnTxiY7FAOiZKCiKSM+gbn5ieWMaJfd+aechjXnzUZM/iff6xIdmgsLCrlQ2MH0Du3W7JDOSSqPhKRpHF37nvpfTaWV8dVvrSqhuWbKrnrk0eT2y2T4f26c/mp47njuVX0yX2HnKxMsrMymHvKYQzqdaBbSSdWSXk1KzZXccPZqT+Ds5KCiCRN8Y5qfjB/Od27ZdItM77W2fOPGs7sI4Y1LX/plPH8a/U2/v7OJgAqa+rok5vFladNjCTmlrywsgyAGfmDO+yYUVFSEJGkKSwJJkV+ZO4JHHWQvWq6Z2fy2OUnNi2fd9fLLCgq69CksGBFKSP6dWfi4F4ddsyoqE1BRJLm3Y2VZGYYk4f2Ttg+Z+QP5q31OyjfvTdh+zyQvXUNvLJ6K6fm52FR9UXtQEoKIpI0hSUVjM/rSW63zITtc0Z+Hg0OL67amrB9HsjitdvZtbeemWlQdQRKCiKSRIUllUwd3jeh+5w2sh/9e3Rj4YrShO63Nc+vKCU7M4OTxqd2V9RGSgoikhRlVXsordrD1OGJnaYiM8M4dVIeL6wso6Eh2lsDbN25hz8u2sCM/Dx65qRHE62SgogkRWMjc6KvFCBoV9i2ay9LN0Z7d9/bnimiuraeb52V+l1RGykpiEhSFJZUAlCQ4CsFgFMm5WEGC4qiq0J6d2MFjy7awGdOGsv4vNTvddRISUFEkmJZSSWjBnSnb/fEjwAe0DObaSP7saCoLOH7hmDQ3S1PLqN/j2yuOr3jur52BCUFEUmKwpIKpg5LfNVRo5n5g3mnuJyyqj0J3/c/3t3Mv9/fznUfnRRJUksmJQUR6XBVNbWs3bY74Y3MsWYfOQwD7l6wOqH7ramt54fzlzN5aG8u+dCohO67M1BSEJEOtyxsTzh8RHRXChMG92LOcaP5/WvrWLWlKmH7ve+lNRTvqOamcwvIyky/r9D0OyMR6fQaG5mjvFIAuO6jk+iRncktTy7D/dC7p26uqOEXC9/jzKlDOWn8oARE2PkoKYhIhyssqWRQrxwG98mN9DgDe+Vw9ekTeWnVVp5PwGC2nzy1grp654azpyQgus4pPUZbiEjKcHeWbiyP/Cqh0adPHMvD/17PNx9/h/wDzLGUYcbnPzJun+kqSqtquPmJZezYtRd3eHXNNr48YzyjB/boiNCTQlcKItKhnl9RysotOzltcsfMFZSdlcFtF08jf2hvausbWv1bXbqTrz32NhXVtU3P/dE/VvBM4WZq6xuoa2jgvGnDuWLmhA6JO1l0pSAiHWZvXQPf//tyDsvrySePH91hxz16dH8e/uIJByzz7sYKzr3rZX7+z1XceE4BSzaU85c3N3L5qeO5Po1GLLcl0isFMzvTzIrMbLWZXd/C9tFmtsDM3jKzd8zs7CjjEZHk+t2ra3l/6y5uPKeAbp2s587hI/ryiemjePBfa1ldupNbnihkUK8crjwtva8MmovsXTGzTOBu4CygAJhjZgXNin0XeMzdjwYuAX4RVTwiklxbd+7hzn+uYkZ+XqedZvprZ+TTvVsm/3X/67y5vpxvnplPrzSZ6C5eUZ7tccBqd18DYGaPAucDy2LKONDY2tQXKIkwHpEux9354fzlvLZm+37beuZk8v0LjmBCzN3CXl61lZ8+U0R9BLOL7ti9l+q99Xx3dvPfhp1HXu8cvnr6BH44fwVHjOjLRceMTHZIHS7KpDAC2BCzXAwc36zM94BnzOyrQE9gVks7MrO5wFyA0aM7rh5SJNU9u2wLv37pfY4d03+/6RgWrd3Od/+6lEe+eAJmxu69dXz9T2+TYTB5WOJ7BuX1DrqHTujkt6y87KRxbN25lwuPGUlGRurfSa29okwKLb2azX9+zAEecPfbzOxE4Pdmdri7N+zzJPd7gXsBpk+fHu0E6SJpYk9dPT+Yv5yJg3vx6NwT9qvD//2ra7nxb4U8XbiZMw8fxj0vrGFzZQ2PX34i08cOSE7QnUB2VkZaj0NoS5QtPcVA7MQgI9m/eujzwGMA7v4qkAuk5zBBkQ72wCtrWbdtd6uNunOOG03+kN58/+/LWVO2k1+98B7nThvepROCRJsUFgETzWycmWUTNCTPa1ZmPXA6gJlNIUgK0cx1K9KFlFXt4f+eX83pkwdzyqS8FstkZWZw07kFFO+o5uJ7XsWMLtX1UloWWfWRu9eZ2ZXA00Am8Bt3LzSzW4DF7j4P+BrwazO7lqBq6TJPxAQlImlsTdlOrn3sbSp27221zM49deypq+c7sw9cDXLyhEF8tGAIzy7bwtWnT2REv+6JDldSTKR9rdx9PjC/2bqbYh4vA06OMgaRdOLu/Pe8QtaU7uS0KQfu1jlryhAOi+OOYLeefzhTh/fhS6eMT1SYksK6VgdckRT3/IpSXlq1lRvPKeDzHx6XkH0O7ZvLNbMmJWRfkvo615BCEWlV7BQRnz5xTLLDkTSlpCCSIpqmiJjd+aaIkPSh6iORBLrusSWMGdCTq2cd/M3cb/h/S3n63c37ra+oruXUSXnM7KDZRaVrUlIQSZCtO/fwlzc3kplhnH3EUCYOaX3u/tYsWFHKw6+v57TJgxneb98b0ORmZfKFjxyWqHBFWqSkIJIgL64MhthkmnHLk8v43eeOwyz+aRJq6xu49e/LOGxQT+751LFkZ6mKSDqePnUiCbKgqIxBvXL45pn5B3X7x9+9uo41Zbv4zuwpSgiSNPrkiSRAXX0DL64sY0Z+Hp85aSyH5fXk+39fzt66hrafDGzftZc7n1vJKZPyOuyOZCItUfWRSAK8XVxORXUtM/MH0y0zgxtnF/DZBxZRcNNTZMRRhVQfDuS/cfaUdlU5iSSakoJIAixYUUZmhvHhicF8jjMnD+a2i6exumxn3Pv40Nj+B9U4LZJISgoiCbCgqJRjR+97z4ILj+16N2iR1Kc2BZFDVFpZQ2FJJTMmtzwbqUgq0ZWCyEHaU1ePO/wz7GU0Y5IaiCX1KSmIHIS7F6zmf58ualoe0ieHKcPUHiCpT0lBpJ3WbdvFnc+t4sMTBnHShIEATB8zQL2GJC0oKYi00w/+vpxumcbPPj6NwX1y236CSApRQ7NIO7yyeivPLNvCV2ZOUEKQtKSkIBKnuvoGbnliGaMGdE/YDW5EOhslBZE4PVW4maItVdxw1hRyu2UmOxyRSCgpiMTpjXU76N4tkzOmDk12KCKRUVIQiVNhSSWTh/UmM0O9jCR9KSmIxKGhwVleUsnhw/smOxSRSCkpiMRhw47dVO2pY+rwPskORSRSSgoicXh3YyUAU3WlIGlOSUEkDoUlFWRlGJOG9kp2KCKRijQpmNmZZlZkZqvN7PoWtt9uZkvCv5VmVh5lPCIHq7CkkgmDe5GTpa6okt4im+bCzDKBu4GPAsXAIjOb5+7LGsu4+7Ux5b8KHB1VPCKHorCkkhn5mhpb0l+Ucx8dB6x29zUAZvYocD6wrJXyc4D/jjAeESAYmbx8UxWOt1omOyuD/CG9MTNKK2vYunOPGpmlS4gyKYwANsQsFwPHt1TQzMYA44DnI4xHBICvPPQmzyzb0ma5a2ZN5JpZkygsUSOzdB1RJoWWRvi09tPsEuBxd69vcUdmc4G5AKNHj05MdNIlvbCyjGeWbeFzJ4/j5HDa65Y88u8N/GLhe1x4zEgKSyoAdL8E6RKiTArFwKiY5ZFASStlLwGuaG1H7n4vcC/A9OnTW7/mFzmA2voGbn1yGWMH9uBbZ+UfsNG4YHgfTvvpC/zoHyuob3DGDuxB79xurZYXSRdR9j5aBEw0s3Fmlk3wxT+veSEzywf6A69GGIsID722jtWlO/nO7II2exEN69udy08dz9+XbuLFVWWqOpIuI7IrBXevM7MrgaeBTOA37l5oZrcAi929MUHMAR51d10BSEKs2lJFadWefdbV1jdwe3i3tFlT4ruX8txTDuOPi9ZTUlHD1BFqZJauIdI7r7n7fGB+s3U3NVv+XpQxSNfy8qqtfOr+11vc1i3TuPGcgrhvm9k9O5MbZk/hyoff4pjR/RMZpkinpdtxStqoq2/g5icKGT2gB/970ZH7ffkP75fLyP492rXPc44czpEj+jFqQPdEhirSaSkpSNp46PX1rCrdyT2fOpbjD2u9Z1F7jR7YvkQikso095GkhfLde7n9uZWcNH4gH5s6JNnhiKQsXSlIhymr2sMb67aHS8aHJw6iV07LH8H6BuelVWXU1LY4dGU//3h3M5XVtdx0bvxtBiKyPyUF6RA1tfX8xy9eoXhHddO648YN4I9zT2jxS/y2Z4r4xcL32nWMy04ay+Sh6iUkciiUFKRD3PfSGop3VHPnJUcxaUhvFhSV8pOnipi/dDOzjxy2T9n123Zz30vvM/vIYVw5c0Jc+8/KMCYM1rTWIodKSUEit6Wyhl8sfI+PTR3C+UeNAGDSkN7MW1LCD+cv5/Qpg8nt9sFgsh/OX05mhnHj7AKG9s1NVtgiXZIamiVyP35qBXX1znfOLmhal5lh/Pe5U9lYXs2vX1zTtP5f723lqcLNfGXGeCUEkSTQlUIntHJLFUs2BPcbMmBG/mDyeucc9P5WbK7kneKKBEXXPpXVtfzlzY18ecb4/bp2njh+IGcdPpRfLHyPQb1zyMww7n/pfUb0684XTzksKfGKdHVKCp3Q1x57m6UbP/gSv+jYkfz04mkHta+S8mouuPsVamobEhVeu40a0J0rWmkbuOHsKbyyeivf/stSILiC+MWlx+xTnSQiHUdJoRNav303/3n0CK47YxK3PrmMhUVlNDQ4GRnt72r546dW4A7zrjyZAT2zI4i2bYN65bT6JT9qQA/+9e3TKd+9F4Ae2VlJi1NElBQ6nd1766iormXCkF6M7N+Dj00dytOFWygsqeSIke2bqfONddv525ISvnraBI4c2S+iiA9dr5ysVscriEjHUkNzJ1NSXgPA8L7BXDunTMrDDBYWlbZrPw0Nzs1PLGNInxwuP3V8wuMUkfSkn2edTEl5MLhreL8gKQzqlcORI/qyoKiUr54+EQimgZ63pITy6tpW97Nu2y7eKa7gZx+fRk/9CheROLX5bRHeE+Ehd9/RAfF0eZsqgqQwLKY75oz8wfz8+VVs37WXAT2zuWfhe9z27Mo29/WRiYO4IBwXICISj3h+Qg4FFpnZm8BvgKd1Q5zobCyvwYx9+ujPnDyYO/+5ipdWlXH8uIFNA8F+ctGBeyT1zsk6qMZpEem62kwK7v5dM7sROAP4LHCXmT0G3O/u7ZucRtq0qbyawb1z6Jb5QXPPkSP6MqBnNguLynihqIz6hmAgWN/uumewiCRWXJXN7u5mthnYDNQR3FP5cTN71t2/GWWAXU1JRXVTe0KjjAzj1El5PPXuZqpr61scCCYikght9j4ys6vM7A3gJ8ArwBHu/mXgWODCiOPrcjaV1zT1PIo1Iz+P6tp68nrntDoQTETkUMVzpTAI+E93Xxe70t0bzOycaMLqmtydjeXVnDZ5/xvLz5g0mBH9unP9WZPVp19EIhPPt8t8oPHOKJhZb6DA3V939+WRRdYF7dhdy566hv2qjwD69ujGK9efloSoRKQriWfw2i+BnTHLu8J1kmAfjFHQ7KAikhzxJAWL7YLq7g1o0Fskmg9cExHpaPEkhTVhY3O38O9qYE2bz5J221QRTHExrIWGZhGRjhBPUrgcOAnYCBQDxwNzowyqqyopryY7K4OBmiVURJIknsFrpcAlHRBLl1dSUcOwvrkahSwiSRPP3Ee5wOeBqUBTC6i7fy6O554J3AlkAve5+49aKPNx4HuAA2+7+yfjDT7dlJRXtzhGQUSko8RTffR7gvmPPga8AIwEqtp6kpllAncDZwEFwBwzK2hWZiLwbeBkd58KXNOu6NPMpvJqhqnnkYgkUTxJYYK73wjscvcHgdnAEXE87zhgtbuvcfe9wKPA+c3KfBG4u3EG1rCqqkuqq29gc2XLo5lFRDpKPEmhcdL+cjM7HOgLjI3jeSOADTHLxeG6WJOASWb2ipm9FlY37cfM5prZYjNbXFZWFsehU09p1R4aXN1RRSS54kkK95pZf+C7wDxgGfDjOJ7XUmtp8ym3s4CJwAxgDnCfme1330h3v9fdp7v79Ly8vDgOnXoaxyio+khEkumADc1mlgFUhtU7LwKHtWPfxcComOWRQEkLZV5z91rgfTMrIkgSi9pxnLRQEo5RGKErBRFJogNeKYSjl688yH0vAiaa2Tgzyybo1jqvWZm/AjMBzGwQQXVSlxwY13Sl0FdXCiKSPPFUHz1rZl83s1FmNqDxr60nuXsdQUJ5GlgOPObuhWZ2i5mdFxZ7GthmZsuABcA33H3bQZ5LSttUXk3v3Cx65+rGOSKSPPHMYdQ4HuGKmHVOHFVJ7j6fYJbV2HU3xTx24Lrwr0tbUlzBxMG9kh2GiHRx8YxoHtcRgXRlW3fu4Z3icq6dNSnZoYhIFxfPiOZPt7Te3X+X+HC6phdXluEOM/P3v7mOiEhHiqf66EMxj3OB04E3ASWFBFlYVMagXjlMHd4n2aGISBcXT/XRV2OXzawvwdQXkgD1Dc4LK8uYNWWIJsITkaSLp/dRc7sJxhJIAizZsIOK6lpmTk7PQXkiklriaVN4gg9GImcQTG73WJRBdSULVpSRmWF8ZIKSgogkXzxtCj+NeVwHrHP34oji6XIWrizlmNH96NtD4xNEJPniSQrrgU3uXgNgZt3NbKy7r400si6gtLKGdzdW8o2P5Sc7FBERIL42hT8BDTHL9eE6OUQvrAxmfFVXVBHpLOJJClnh/RAACB/rJsIJUFhSSY/sTKYM653sUEREgPiSQlnMXEWY2fnA1uhC6jrWbtvF2IE9MVNXVBHpHOJpU7gceMjM7gqXi4EWRzlL+6zbtpuCYRqwJiKdRzyD194DTjCzXoC5e5v3Z5a21dU3sGH7bs4+YmiyQxERadJm9ZGZ/dDM+rn7TnevMrP+Zvb9jggunRXvqKauwRkzsGeyQxERaRJPm8JZ7l7euBDehe3s6ELqGt7ftguAcYOUFESk84gnKWSaWU7jgpl1B3IOUF7isHZrkBTG6kpBRDqReBqa/wD808x+Gy5/FngwupC6hrVbd9ErJ4tBvdS7V0Q6j3gamn9iZu8AswADngLGRB1Yunt/227GDuqh7qgi0qnEO0vqZoJRzRcS3E9heWQRdRHrwjEKIiKdSatXCmY2CbgEmANsA/5I0CV1ZgfFlrZq6xso3lHNuUcOT3YoIiL7OFD10QrgJeBcd18NYGbXdkhUaW7D9t3UNzhj1fOFVPTmAAAQW0lEQVRIRDqZA1UfXUhQbbTAzH5tZqcTtCnIIVrb1B21R5IjERHZV6tJwd3/n7t/ApgMLASuBYaY2S/N7IwOii8tvb91N6DuqCLS+bTZ0Ozuu9z9IXc/BxgJLAGujzyyNLZ26y5652YxoKe6o4pI59KuezS7+3Z3/5W7nxZVQF3B2m27GDdIs6OKSOfTrqQgibF22y7NeSQinVKkScHMzjSzIjNbbWb7VTmZ2WVmVmZmS8K/L0QZT2ewt66BjTuqGTdQjcwi0vnEM83FQTGzTOBu4KME92BYZGbz3H1Zs6J/dPcro4qjs1m/fTcNjrqjikinFOWVwnHAandfE97C81Hg/AiPlxLeb5wIT0lBRDqhKJPCCGBDzHJxuK65C83sHTN73MxGtbQjM5trZovNbHFZWVkUsXaY5ZsqMYNJQ3RfZhHpfKJMCi11rfFmy08AY939SOA5Wpl91d3vdffp7j49Ly8vwWF2rMKSCsYO7EmvnMhq7kREDlqUSaEYiP3lPxIoiS3g7tvcfU+4+Gvg2Ajj6RQKSyopGK77MotI5xRlUlgETDSzcWaWTTC53rzYAmY2LGbxPNJ89tWK3bUU76hmqpKCiHRSkdVhuHudmV0JPA1kAr9x90IzuwVY7O7zgKvM7DygDtgOXBZVPJ1B4aYKAKYO75vkSEREWhZpxba7zwfmN1t3U8zjbwPfjjKGzmRZSSWArhREpNPSiOYOVFhSyZA+OQzqpVtci0jnpKTQgd7dWKGqIxHp1JQUOkj13nreK9vJ4ao6EpFOTEmhg6zYXEmDQ4GuFESkE1NS6CCFamQWkRSgpNBBCksq6du9GyP7d092KCIirVJS6CDLSiooGNZHN9YRkU5NSaED1NU3sGJzlaqORKTTU1LoAO+V7WJPXQNTRygpiEjnpqTQAQpLNL2FiKQGJYUOUFhSSU5WBofpxjoi0skpKXSAdzdWMGVYH7Iy9XKLSOemb6mIuTvLNlWqkVlEUoKSQsQ2bK+mqqZO7QkikhKUFCL2QSOzrhREpPNTUohYYUklmRlG/tDeyQ5FRKRNSgoRKyypYEJeL3K7ZSY7FBGRNikpROzdEjUyi0jqUFKIUGlVDWVVeyhQUhCRFKGkEKEPpstWzyMRSQ1KChFaFiYFXSmISKpQUohQYUkFowf0oG/3bskORUQkLkoKESpUI7OIpBglhYhU1tSybttuJQURSSlKChFZpkZmEUlBkSYFMzvTzIrMbLWZXX+AcheZmZvZ9Cjj6Ugf9DzSlYKIpI7IkoKZZQJ3A2cBBcAcMytooVxv4Crg9ahiSYbCkgoG9cphcJ/cZIciIhK3KK8UjgNWu/sad98LPAqc30K5W4GfADURxtLhlpVUcrhuvykiKSbKpDAC2BCzXByua2JmRwOj3P3JA+3IzOaa2WIzW1xWVpb4SBOspraeVaU7VXUkIiknyqRgLazzpo1mGcDtwNfa2pG73+vu0919el5eXgJDjMbKLVXUN7gamUUk5USZFIqBUTHLI4GSmOXewOHAQjNbC5wAzEuHxmY1MotIqooyKSwCJprZODPLBi4B5jVudPcKdx/k7mPdfSzwGnCeuy+OMKYOUVhSQe+cLEb175HsUERE2iWypODudcCVwNPAcuAxdy80s1vM7LyojtsZvLuxkoLhfcjIaKkGTUSk88qKcufuPh+Y32zdTa2UnRFlLB2lvsFZsbmSTx43JtmhiIi0m0Y0J9iasp3U1DaoPUFEUpKSQoI1NTJrjIKIpCAlhQQrLKkgOyuD8Xm9kh2KiEi7RdqmkM5Wl1bxp8XFHwy8CD27bAuTh/amW6byrYikHiWFg3T7c6uYv3QTuVmZ+227ePqoFp4hItL5KSkchLr6Bl5cWcbFx47kJxdNS3Y4IiIJozqOg/Dm+nKqauqYmT842aGIiCSUksJBWFhUSlaGcfLEQckORUQkoZQUDsKCojKOHdOfPrndkh2KiEhCKSm00+aKGpZvqmTmZFUdiUj6UVJopxdWlgKoPUFE0pKSQjstWFHGsL65TBqiwWkikn6UFNphb10DL6/eyoz8wZhpBlQRST9KCu3wxrod7NxTx8z8zn/3NxGRg6Gk0A4Li0rplmmcNEFdUUUkPSkptMOColKOGzeAXjkaCC4i6UlJIU4by6tZuWWneh2JSFpTUojTwqKgK+oMtSeISBrrMknhqXc3M+fe16itb9hn/SP/Xs+f3yhu8/kLVpQxsn933SdBRNJal0kKmRnGq2u28YfX1jWtK9pcxXf+31Ku/8s7vL91V6vP3VNXz7/e28qM/Dx1RRWRtNZlksKsKYP58IRB3PHcKnbs2ou7c+uTy+id243szAx+8PflrT530fs72L23Xu0JIpL2ukxSMDNuPKeAqppabn9uJc8tL+Xl1Vu5dtZErjhtAs8t38JLq8pafO6ColKyszI4cfzADo5aRKRjdZmkAJA/tDefOmEMD72+nhv/+i4TBvfi0hPG8LmTxzF6QA9ufXIZdc3aHCBICsePG0CPbHVFFZH01qWSAsC1sybRKyeLzZU13HhOAd0yM8jtlskNZ09h5ZadPPzv9fuUf3/rLtaU7VLVkYh0CV0uKfTvmc0dnziKr58xiVMnfdC99GNTh3DS+IH87NmVlO/e27T+tmeKyO2WwdlHDEtGuCIiHarLJQWAmZMHc+VpE/dZZ2bcdG4BldW13PHcKgAWrd3Ok+9s4kunjGdo39xkhCoi0qEiTQpmdqaZFZnZajO7voXtl5vZUjNbYmYvm1lBlPG0ZfLQPsw5bjS/f20dRZuruPmJQob1zeXyU8cnMywRkQ4TWVIws0zgbuAsoACY08KX/sPufoS7HwX8BPhZVPHE67qPTqJndiafuv913t1YyfVnTaZ7dmaywxIR6RBRXikcB6x29zXuvhd4FDg/toC7V8Ys9gQ8wnjiMrBXDlfPmkRZ1R6OHdOf86YNT3ZIIiIdJso+liOADTHLxcDxzQuZ2RXAdUA2cFpLOzKzucBcgNGjRyc80OY+feIYKnbv5YKjR2gEs4h0KVFeKbT0bbrflYC73+3u44FvAd9taUfufq+7T3f36Xl50U9I1y0zg+vOyOcwzXMkIl1MlEmhGBgVszwSKDlA+UeBCyKMR0RE2hBlUlgETDSzcWaWDVwCzIstYGax/UJnA6sijEdERNoQWZuCu9eZ2ZXA00Am8Bt3LzSzW4DF7j4PuNLMZgG1wA7gM1HFIyIibYt0Mh93nw/Mb7buppjHV0d5fBERaZ8uOaJZRERapqQgIiJNlBRERKSJkoKIiDQx96TPLNEuZlYGrGuzYMsGAVsTGE5npHNMDzrH9NCZznGMu7c5+jflksKhMLPF7j492XFESeeYHnSO6SEVz1HVRyIi0kRJQUREmnS1pHBvsgPoADrH9KBzTA8pd45dqk1BREQOrKtdKYiIyAEoKYiISJMukxTM7EwzKzKz1WZ2fbLjSQQzG2VmC8xsuZkVmtnV4foBZvasma0K/+2f7FgPhZllmtlbZvZkuDzOzF4Pz++P4dTsKcvM+pnZ42a2InwvT0zD9/Da8DP6rpk9Yma5qf4+mtlvzKzUzN6NWdfi+2aBn4ffP++Y2THJi/zAukRSMLNM4G7gLKAAmGNmBcmNKiHqgK+5+xTgBOCK8LyuB/7p7hOBf4bLqexqYHnM8o+B28Pz2wF8PilRJc6dwFPuPhmYRnCuafMemtkI4CpgursfTjCV/iWk/vv4AHBms3WtvW9nARPDv7nALzsoxnbrEkkBOA5Y7e5r3H0vwV3ezk9yTIfM3Te5+5vh4yqCL5MRBOf2YFjsQVL4jnZmNpLgBkz3hctGcC/vx8MiqX5+fYBTgPsB3H2vu5eTRu9hKAvobmZZQA9gEyn+Prr7i8D2Zqtbe9/OB37ngdeAfmY2rGMibZ+ukhRGABtilovDdWnDzMYCRwOvA0PcfRMEiQMYnLzIDtkdwDeBhnB5IFDu7nXhcqq/l4cBZcBvwyqy+8ysJ2n0Hrr7RuCnwHqCZFABvEF6vY+NWnvfUuY7qKskBWthXdr0xTWzXsCfgWvcvTLZ8SSKmZ0DlLr7G7GrWyiayu9lFnAM8Et3PxrYRQpXFbUkrFc/HxgHDAd6ElSnNJfK72NbUuZz21WSQjEwKmZ5JFCSpFgSysy6ESSEh9z9L+HqLY2XpuG/pcmK7xCdDJxnZmsJqvxOI7hy6BdWQ0Dqv5fFQLG7vx4uP06QJNLlPQSYBbzv7mXuXgv8BTiJ9HofG7X2vqXMd1BXSQqLgIlhb4dsgkaueUmO6ZCF9ev3A8vd/Wcxm+bxwf2uPwP8raNjSwR3/7a7j3T3sQTv2fPufimwALgoLJay5wfg7puBDWaWH646HVhGmryHofXACWbWI/zMNp5j2ryPMVp73+YBnw57IZ0AVDRWM3U2XWZEs5mdTfArMxP4jbv/IMkhHTIz+zDwErCUD+rcbyBoV3gMGE3wH/Jid2/eIJZSzGwG8HV3P8fMDiO4chgAvAV8yt33JDO+Q2FmRxE0pGcDa4DPEvxgS5v30MxuBj5B0GPuLeALBHXqKfs+mtkjwAyC6bG3AP8N/JUW3rcwGd5F0FtpN/BZd1+cjLjb0mWSgoiItK2rVB+JiEgclBRERKSJkoKIiDRRUhARkSZKCiIi0kRJQboMM/sfM5thZhc0zpRrZg+Y2UXh42vMrEcCj3dB7MSLZnaLmc1K1P5FoqCkIF3J8QRjOE4lGN/R3DUEk7XFLZyBtzUXEMzKC4C73+Tuz7Vn/yIdTUlB0p6Z/a+ZvQN8CHiVYODUL83sppgyVxHMy7PAzBaE684ws1fN7E0z+1M4xxRmttbMbjKzl4GLzeyLZrbIzN42sz+HI3dPAs4D/tfMlpjZ+GZXJaeHE+AtDeflz4nZ983hMZea2eRw/anhfpaEz+vdYS+gdClKCpL23P0bBIngAYLE8I67H+nut8SU+TnBXDQz3X2mmQ0CvgvMcvdjgMXAdTG7rXH3D7v7o8Bf3P1D7t54L4TPu/u/CKY2+Ia7H+Xu7zU+0cxyw1g+4e5HEEyK9+WYfW8Nj/lL4Ovhuq8DV7j7UcBHgOqEvDgizSgpSFdxNLAEmEww705bTiCo+nnFzJYQzGMzJmb7H2MeH25mL5nZUuBSYGob+84nmCBuZbj8IME9FRo1Tmz4BjA2fPwK8LPwiqZfzJTTIgmV1XYRkdQVziv0AMGslFsJ2gws/KI/8UBPBZ519zmtbN8V8/gB4AJ3f9vMLiOYD+eAYbWxvXH+n3rC/6Pu/iMz+ztwNvCamc1y9xVt7Eek3XSlIGnN3ZeEVS4rCX75Pw98LKzSaV4FUwU01tW/BpxsZhMAwnaCSa0cpjewKZzG/NJW9hdrBTC2cd/AfwEvHOg8zGy8uy919x8TVGVNPlB5kYOlpCBpz8zygB3u3gBMdvfWqo/uBf5hZgvcvQy4DHgkbKR+jda/iG8k6NX0LMEXfqNHgW+EDcPjG1e6ew3BTKh/CqucGoB72jiNayy46f3bBO0J/2ijvMhB0SypIiLSRFcKIiLSRElBRESaKCmIiEgTJQUREWmipCAiIk2UFEREpImSgoiINPn/4ipYfX1PIJsAAAAASUVORK5CYII=\n",
      "text/plain": [
       "<Figure size 432x288 with 1 Axes>"
      ]
     },
     "metadata": {
      "needs_background": "light"
     },
     "output_type": "display_data"
    }
   ],
   "source": [
    "plt.plot(test_acc_hist)\n",
    "plt.xlabel(\"#Iterations\")\n",
    "plt.ylabel(\"Accuracy\")\n",
    "plt.title(\"Accuracy vs Number of iterations\")"
   ]
  }
 ],
 "metadata": {
  "kernelspec": {
   "display_name": "Python 3",
   "language": "python",
   "name": "python3"
  },
  "language_info": {
   "codemirror_mode": {
    "name": "ipython",
    "version": 3
   },
   "file_extension": ".py",
   "mimetype": "text/x-python",
   "name": "python",
   "nbconvert_exporter": "python",
   "pygments_lexer": "ipython3",
   "version": "3.6.8"
  }
 },
 "nbformat": 4,
 "nbformat_minor": 2
}