linear_regression.ipynb

{
 "cells": [
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "## Linear Regression in plain Python\n",
    "\n",
    "In linear regression we want to model the relationship between a **scalar dependent variable** $y$ and one or more **independent (predictor) variables** $\\boldsymbol{x}$.\n",
    "\n",
    "**Given:** \n",
    "- dataset $\\{(\\boldsymbol{x}^{(1)}, y^{(1)}), ..., (\\boldsymbol{x}^{(m)}, y^{(m)})\\}$\n",
    "- with $\\boldsymbol{x}^{(i)}$ being a $d-$dimensional vector $\\boldsymbol{x}^i = (x^{(i)}_1, ..., x^{(i)}_d)$\n",
    "- $y^{(i)}$ being a scalar target variable\n",
    "\n",
    "The linear regression model can be interpreted as a very **simple neural network:**\n",
    "- it has a real-valued weight vector $\\boldsymbol{w}= (w^{(1)}, ..., w^{(d)})$\n",
    "- it has a real-valued bias $b$\n",
    "- it uses the identity function as its activation function\n",
    "\n",
    "A linear regression model can be trained using either  \n",
    "a) gradient descent or  \n",
    "b) the normal equation (closed-form solution): $\\boldsymbol{w} = (\\boldsymbol{X}^T \\boldsymbol{X})^{-1} \\boldsymbol{X}^T \\boldsymbol{y}$ \n",
    "\n",
    "where $\\boldsymbol{X}$ is a matrix of shape $(m, n_{features})$ that holds all training examples.  \n",
    "The normal equation requires computing the inverse of $\\boldsymbol{X}^T \\boldsymbol{X}$. The computational complexity of this operation lies between $O(n_{features}^{2.4}$) and $O(n_{features}^3$) (depending on the implementation).\n",
    "Therefore, if the number of features in the training set is large, the normal equation will get very slow. \n",
    "\n",
    "* * *\n",
    "The training procedure of a linear regression model has different steps. In the beginning (step 0) the model parameters are initialized. The other steps (see below) are repeated for a specified number of training iterations or until the parameters have converged.\n",
    "\n",
    "**Step 0: ** \n",
    "\n",
    "Initialize the weight vector and bias with zeros (or small random values)\n",
    "\n",
    "**OR**\n",
    "\n",
    "Compute the parameters directly using the normal equation\n",
    "* * *\n",
    "\n",
    "**Step 1: ** (Only needed when training with gradient descent)\n",
    "\n",
    "Compute a linear combination of the input features and weights. This can be done in one step for all training examples, using vectorization and broadcasting:\n",
    "$\\boldsymbol{\\hat{y}} = \\boldsymbol{X} \\cdot \\boldsymbol{w} + b $\n",
    "\n",
    "where $\\boldsymbol{X}$ is a matrix of shape $(m, n_{features})$ that holds all training examples, and $\\cdot$ denotes the dot product.\n",
    "* * *\n",
    "\n",
    "**Step 2: ** (Only needed when training with gradient descent)\n",
    "\n",
    "Compute the cost (mean squared error) over the training set:\n",
    "\n",
    "$J(\\boldsymbol{w},b) = \\frac{1}{m} \\sum_{i=1}^m \\Big(\\hat{y}^{(i)} - y^{(i)} \\Big)^2$\n",
    "* * *\n",
    "\n",
    "**Step 3: **  (Only needed when training with gradient descent)\n",
    "\n",
    "Compute the partial derivatives of the cost function with respect to each parameter:\n",
    "\n",
    "$ \\frac{\\partial J}{\\partial w_j} = \\frac{2}{m}\\sum_{i=1}^m \\Big( \\hat{y}^{(i)} - y^{(i)} \\Big) x^{(i)}_j$\n",
    "\n",
    "$ \\frac{\\partial J}{\\partial b} = \\frac{2}{m}\\sum_{i=1}^m \\Big( \\hat{y}^{(i)} - y^{(i)} \\Big)$\n",
    "\n",
    "\n",
    "The gradient containing all partial derivatives can then be computed as follows: \n",
    "\n",
    "$\\nabla_{\\boldsymbol{w}} J = \\frac{2}{m} \\boldsymbol{X}^T \\cdot \\big(\\boldsymbol{\\hat{y}} - \\boldsymbol{y} \\big)$\n",
    "\n",
    "$\\nabla_{\\boldsymbol{b}} J = \\frac{2}{m} \\big(\\boldsymbol{\\hat{y}} - \\boldsymbol{y} \\big)$\n",
    "* * *\n",
    "\n",
    "**Step 4: ** (Only needed when training with gradient descent)\n",
    "\n",
    "Update the weight vector and bias:\n",
    "\n",
    "$\\boldsymbol{w} = \\boldsymbol{w} - \\eta \\, \\nabla_w J$  \n",
    "\n",
    "$b = b - \\eta \\, \\nabla_b J$  \n",
    "\n",
    "\n",
    "where $\\eta$ is the learning rate.\n"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 2,
   "metadata": {
    "ExecuteTime": {
     "end_time": "2018-03-26T14:37:04.956748Z",
     "start_time": "2018-03-26T14:37:04.391421Z"
    }
   },
   "outputs": [],
   "source": [
    "import numpy as np\n",
    "import matplotlib.pyplot as plt\n",
    "from sklearn.model_selection import train_test_split\n",
    "\n",
    "np.random.seed(123)"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "## Dataset"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 3,
   "metadata": {
    "ExecuteTime": {
     "end_time": "2018-03-26T14:37:06.048159Z",
     "start_time": "2018-03-26T14:37:05.749381Z"
    }
   },
   "outputs": [
    {
     "data": {
      "image/png": "iVBORw0KGgoAAAANSUhEUgAAAfIAAAGDCAYAAADQ75K0AAAABHNCSVQICAgIfAhkiAAAAAlwSFlzAAALEgAACxIB0t1+/AAAADl0RVh0U29mdHdhcmUAbWF0cGxvdGxpYiB2ZXJzaW9uIDIuMi4wLCBodHRwOi8vbWF0cGxvdGxpYi5vcmcvFvnyVgAAIABJREFUeJztvX2cHFd55/t7ZtQWMwY8ciyIPdhIcFkpGGMJD2AQIZFJEImxmbVJBAsbAsn1kmRJLJxJxgvBMoH17EfJNSRsLvEmLGFtjPzGrBwnEWxkYK+CTEbMyEIg8eY32rAIpDHYM7JGM8/9o7tG1dXnVJ2qrtfu3/fz0UdSd72cU1Vdz3neRVVBCCGEkGrSV/QACCGEEJIcCnJCCCGkwlCQE0IIIRWGgpwQQgipMBTkhBBCSIWhICeEEEIqDAU5IYQQUmEoyAnpAkTkYRGZF5GfisisiPyLiLxbRCJ/4yKyRkRURFZkPMZczkNIr0FBTkj3cIWqPgvA8wFMAPhjAH9b7JAIIVlDQU5Il6GqT6jqLgBbAbxDRF4iIpeLyLSI/EREHhOR7b5dvtT8e1ZEnhSRV4nIC0Vkj4j8WER+JCK3iciQt4OI/LGI1JsWgCMi8rrm530iMi4i32nue4eInG07T8aXgpCegIKckC5FVb8C4HsAfh7AUwB+A8AQgMsB/I6IjDY3fW3z7yFVfaaqfhmAALgJwHkAfg7A+QC2A4CIrAPwHwG8vGkB2ALg4eYxfh/AKIBfaO57HMB/DTkPIaRDKMgJ6W4eB3C2qn5BVQ+q6pKqPgjgdjSErRFV/baqfl5Vn1bVowD+H9/2iwBWAnixiNRU9WFV/U7zu/8A4H2q+j1VfRoN4f9m+sUJyQ4KckK6m2EAx0TklSJyv4gcFZEnALwbwDm2nUTkOSLymab5/CcAbvW2V9VvA7gWDSH9w+Z25zV3fT6AzzYD7mYBfAMNwf/crCZISK9DQU5IlyIiL0dDkP9/AD4NYBeA81X1LAAfR8N8DgCmFog3NT9/qao+G8DbfdtDVT+tqq9BQ3ArgP/S/OoxAL+iqkO+P89Q1brlPISQDqEgJ6TLEJFni8gbAXwGwK2qehDAswAcU9UTIvIKAP/Ot8tRAEsAXuD77FkAnkQjMG0YwJjv+OtE5DIRWQngBIB5NLRuoLFA+LCIPL+57WoReVPIeQghHUJBTkj3cK+I/BQNrfh9aPi139n87ncBfLD5/QcA3OHtpKpzAD4MYG/TJH4pgBsBvAzAEwDuA3CP7zwr0Uhv+xGAHwB4DoD/1Pzuo2ho/p9rnmsfgFeGnIcQ0iGiSmsXIYQQUlWokRNCCCEVhoKcEEIIqTAU5IQQQkiFoSAnhBBCKgwFOSGEEFJhKlE28ZxzztE1a9YUPQxCCCEkF/bv3/8jVV3tsm0lBPmaNWswNTVV9DAIIYSQXBCRR1y3pWmdEEIIqTAU5IQQQkiFoSAnhBBCKgwFOSGEEFJhKMgJIYSQCkNBTgghhFQYCnJCCCGkwlCQE0IIIRWGgpwQQgipMBTkhBBCSIWpRIlWQgghJMjkdB07dh/B47PzOG9oAGNb1mF043DRw8odCnJCCCGVY3K6juvvOYj5hUUAQH12HtffcxAAek6YZ2ZaF5FPiMgPReRrhu/+UERURM7J6vyEEEK6lx27jywLcY/5hUXs2H2koBEVR5Y+8k8CeEPwQxE5H8AvA3g0w3MTQgjpYh6fnY/1eTeTmSBX1S8BOGb46mYAfwRAszo3IYSQ7ua8oYFYn3czuUati8iVAOqqesBh22tEZEpEpo4ePZrD6AghhFSFsS3rMFDrb/lsoNaPsS3rChpRceQW7CYigwDeB+D1Ltur6i0AbgGAkZERau+EEEKW8QLaGLWeb9T6CwGsBXBARADgeQC+KiKvUNUf5DgOQgghXcDoxuGeFNxBchPkqnoQwHO8/4vIwwBGVPVHeY2BEEII6TayTD+7HcCXAawTke+JyG9ldS5CCCGkV8lMI1fVt0Z8vyarcxNCCCG9AmutE0IIIRWGgpwQQgipMBTkhBBCSIWhICeEEEIqDAU5IYQQUmEoyAkhhJAKQ0FOCCGEVBgKckIIIaTCUJATQgghFYaCnBBCCKkwFOSEEEJIhaEgJ4QQQioMBTkhhBBSYSjICSGEkAqTWRtTQgghJG8mp+vYsfsIHp+dx3lDAxjbsg6jG4eLHlamUJATQgjpCian67j+noOYX1gEANRn53H9PQcBoKuFOU3rhBBCuoIdu48sC3GP+YVF7Nh9pKAR5QMFOSGEkK7g8dn5WJ93CxTkhBBCuoLzhgZifd4tUJATQgjpCsa2rMNArb/ls4FaP8a2rCtoRPnAYDdCCCFdgRfQxqh1QgghpKKMbhzuesEdhKZ1QgghpMJQkBNCCCEVhoKcEEIIqTAU5IQQQkiFoSAnhBBCKgwFOSGEEFJhKMgJIYSQCkNBTgghhFQYCnJCCCGkwrCyGyGEEGJhcrpe+pKvFOSEEEKIgcnpOq6/5+Byj/P67Dyuv+cgAJRKmNO0TgghhBjYsfvIshD3mF9YxI7dRwoakRkKckIIIcTA47PzsT4vCgpyQgghxMB5QwOxPi8KCnJCCCHEwNiWdRio9bd8NlDrx9iWdQWNyAyD3QghhBADXkAbo9YJIYSQijK6cbh0gjsITeuEEEJIhaEgJ4QQQioMBTkhhBBSYSjICSGEkApDQU4IIYRUmMwEuYh8QkR+KCJf8322Q0QOi8iDIvJZERnK6vyEEEJIL5ClRv5JAG8IfPZ5AC9R1ZcC+CaA6zM8PyGEkBIwOV3Hpok9WDt+HzZN7MHkdL3oIXUVmeWRq+qXRGRN4LPP+f67D8Cbszo/IYSQ4qlKBzGPKrQtDVKkj/xdAP7R9qWIXCMiUyIydfTo0RyHRQghJC2q0kEMOL3oqM/OQ3F60VF2C0IhglxE3gfgFIDbbNuo6i2qOqKqI6tXr85vcIQQQlKjDB3EXE37VVp0+Mm9RKuIvAPAGwG8TlU17/MTQgjJj/OGBlA3CO28OojFMe2XYdGRhFw1chF5A4A/BnClqs7leW5CCCH5U3QHMZuWfe3OmTbtvCptS4NkmX52O4AvA1gnIt8Tkd8C8DEAzwLweRGZEZGPZ3V+QgghxTO6cRg3XXURhocGIACGhwZw01UX5RZAFqZNB33gRS86kiJVsG6PjIzo1NRU0cMghBBSMTZN7DGa9v0MDw1g7/hlAFqj1ocGa1AFnphfyD2CXUT2q+qIy7ZsY0oIIaQrMKWOjW1Zh207ZxCmsvq1dq9taZXS5liilRBCSKWZnK5jw42fw7U7Z9pSxwCECnHA7AOvUgQ7NXJCCOkCqljIJA2CmrMfT/AOWyLnAbsPvEoR7NTICSGk4lS1kEkamDRnP4/PzhuD2ABgaKBmDbyrUgQ7BTkhhFScKpmB0yZKQz5vaMAYOf+RrRswc8PrrVaLKkWw07ROCCEVp0pm4LSxFZwBWgWvF8TmirdtFdwVFOSEEFJxiq6e1gmd+vbHtqwz+shXDdZwwxUXdiR44wr/oqAgJ4SQimMSZkWbgV0EtC3Fa+qRY7j/8FEn4e59vn3XIczOLwBIR4hXCQpyQgipOGUzA7vmYNt8+7fte3Q5Zcw1f/vpU0vL/z4+t1DanO8soCAnhJAuoExm4LDgO/8YbT78YN63ad8k5+tWKMgJIYSkimvw3VkDtWVzeNJjxjlfmpQpb5+CnBBCSKq4Bt+JxDtmp+eLi63u+tBgDU+eOIWFpYbtoOjyrcwjJ4QQkiquOdizc27aeFTgXhY538EiO8fnFjA7v7D8b0+IexSZt0+NnBBCSKq4Bt+F5YD3i2BJ1clsnUWwX1TFOBNF5e1TkBNCCDHSiR/YJfhubMs6jN11AAuLrdptrU+w49cujl3AJU2zdhKhXFTePgU5IYSQNlxSyDoN+PK2vfHeQzjeNLMPDdSw/cric8DDrAUmBMDm9auzG1DYuVWjGrwVz8jIiE5NTRU9DEII6Rk2TewxCrLhoQHsHb/M2HVsoNZvbUJSNcK6qgFAf59gMeAnT3P+IrJfVUdctqVGTgghpI2olC5b7vZ1dxwAkF70dp5pXsFzXX3J8HKFOX/U+nlDA3jq6VNtqXNF5a5TkBNCCAHQKsj6RLBosNh6fmCboF9UTS0Vy7VCXBqYznX3/jpuuuoiAO2BdNt2zhiPU0TAG9PPCCGEtKVbmYS4P6UrLLArrVQsm9Z/7c4ZbJrYk2q/ddu5brz3UFuv9207Z9qqz3kUEfBGjZwQkpgyVbeqEmW8blHpVv0iuPqS05HhnlZqE2hpaKZhwWZpa+e28R435Lrb5lxUoxpq5ISQRAQ1OO/FmqaW1I2U9bpFCd5FVdy9v748ztGNw1aBBqSjmfZHlH5LswhLp+MdHhooLNCPgpwQkoiwRhXETifXbXK6jk0Te7B2/L7UTcsugiw4zmHLPgKkopmazPtBXDX/qGtnqg7nigDYO34Za60TQqpFEY0qqoxnTreZi6Oum2vgV1KzvamnuYn67Dw2TexZjuSu9UlLuVIB8LZLL0hFqA075HK7LEBM127bzhlMPXIMHxptBLMFq8PFScwuqhCMBzVyQkgibC+vol9qZcRvTrcRdd1cNPlOzPajG4dx01UXYXhoAAK7WVuax/VqjkMaRVwEDcF789YNy8KxU6K0ZG8sUdYJ07VTALfte7Rlv9GNw9g7fhkemrjcam0IUpRf3A8FOSEkEVk0quhWogLJXK6biwWkU3eHX5D9+a9f3HZ/Be2BXguLijNXrsBDE5enbl4OLi5WDdYwNFBrG0vUgiWs77nt2ria2leuKF6M0rROCElEFo0qupUws/mw43VzadUZx90RZYI33d+kbgEX/K6H/mYOu3dt9o5f1rKtqepcWDGWJGMPzv8ZtT7MLyy1bTc7v1BoC1OAgpwQ0gFpN6roVmyCxCt36oLJhx3U5F37crv624P311a2NY2+3/7xeEFutnHFjc/YvH41bt33qPE7RWNem9evXq7i5l/YeOe1zR0orqKbR/E2AUJI15JllHWVSMMNETQzm9KdXM+T1ASflTslzPVgGlec+IzJ6Tru3h/+3NVn53HrvkdDYwuirA5FBnlSIyeEZEKe5TXLTlpuiCgLiOt5bELHH5Huam43HT9u5HxcIelinYjKEogiqGVHdUMrMsiTgpwQkglhWl+vCXIgOzeESWhGmettQsmLAgfaF17B89y8dYNxPu+fPIjb9j3aFojmHSfOePzfB+c7v7DY5kv3t1d1SaWLwr+ACEvPKzrIk6Z1QkgmMM88e5Kmm5lM5KaIdG/h5Xqeyel6ixD3H2f7rkOxxuPhF5LBNL5F1eXv/YuEqCwBV/wLCL9rAzidnldkRTcPauSEkExwDbwiyUlq9Ygbke56nh27j1gLqczOL2Byum4c1+jGYUw9cgy3P/BYSzW3oKbtOo40FosmLbuswZ0U5ISQTHDxY3YbeTdD6cTq4RqRPjRYcz5P1HltCwwvIM0vxE2atus4whYm/Zb2rGee0Y+hwTMqmUpJQU4IyYReyzMvIrgvTavH2JZ1GLvrABYWW4XckydO4ayBGmbn27uABc8T5eu2CeKoKHrXHun+uZgWkTdddZG1j/jcyUUc+qBbKmDZoI+cEJIZ/kphRTaVyAPXEqpppuPFTQcLO//oxmGceUa7brewpBCB03miqqHZFhhhUfRxeqT752JL1bONwcsnr2KKpKhDd5miGRkZ0ampqaKHQQipMFmbvdeO32f0DwuAhyYuN0ZSe1piJ+NwnVfY+QGEpmoJgJu3bnA+z/Zdh9o0+LC52sz6NjO4x9BADSLA7NyCcyrc5vWrcff+ujUYLo17kgYisl9VR5y2pSAnhHQ7WQlRPzZh5FVvi/o+a2znXzVYw4mFpdAo7yRjjLNwst2fsDHV+gQQtLgCgvfUdtyrLxnG/YePWhcued2TMOIIcvrICSFdTx457VHBfUWn49nOc3yu3fftJ2mAoj+YzhPq23bOLGvFwXKoN111UZvgD7MS+FuneswvLOLanTPYsfvI8v6m+37/4aPYO36Z1YqSpDZ9kVCQE0JSp6iXnu28eQjRqOC+otPxogLRTLg2dAnDFATor3vu+cFvuuoioxacpLBLfXYeY3ceMAp74PR9T7s2fVFQkBNCUqWol17YebMSonGqqmWRjhdnwWQ7/8oVfcaI9LTMyy7FWWzWEe//191xINRXbsIL0jPt5t1313tS9iqFFOSEkFSJ+9JLS3sPO6+tvObcyVMtRUo68etGLVjSTsdL6/xAu9Ybd4ERdt1crR5R7URNqXFRqLb72v1z67Q2fVmqFFKQE0JSJW5P7LS097DzescKRlMfnzvdSxqAcSx3Tj2Kfd89jkVV9Ivgra88Hx8avSiRlhYUHF5qWtoLl7Dz275LusCIuoeuJv0w68joxmFjJDwQHdlu8r3HXbgV7RaJIrM8chH5hIj8UES+5vvsbBH5vIh8q/n3qqzOTwgphjgtJpO200xy3tGNwzhzZbvu4p3PNpa93zm2LCgWVXHrvkfx/smDibS0pLXR45wniZbo5fvfvHUDAGDbzhnnnOqoexiVW+4RZQF4wiDEAWBJFasGa8bvVg3WrLUM4tyLrNq3pkWWBWE+CeANgc/GAfyzqr4IwD83/08IKSFJi5fEeemlKYxczht2vjjnvP2Bx2ItWDzyXLjE5f2TB7Ft50zsRUbUPQw2GzHhCdwwwuZ7wxUXotYvLZ/X+gU3XHGh9Xhx7oVLL/giyUyQq+qXABwLfPwmAH/X/PffARjN6vyEkOR0ojnGeemlKYxczht2vjjnXFS1dhDbvH61db+8Fi5xF2FhXctsiwzvHDajdrBz2N7xy/CRrRuMYw4TuB5h8x3dOIwdb764pTPZwqIud24zEfdelLlKYd4+8ueq6vcBQFW/LyLPyfn8hBAHOo3Sde0SZQtCq8/OY9PEntjBYFHnjYpSdk116pPTHbv8AlAB3L2/jpHnn525r9U1eM0l7iCsa5lLbEMQmwWmk4C/qH29v13nXna/dxxKG+wmItcAuAYALrjggoJHQ0hvkdT/G/cFbRKGHlmkrbkIEv93a35mAHu/EzQsAtDGfO8/fNSqxZrGnCQFLey6mhYumyb2xF6Ehd1X19gGj6jc805agUbtG2cB2k3d+fIW5P9HRM5tauPnAvihbUNVvQXALUCjRGteAySExNdWbJHLU48ca6vgFXyhmoShR5QASrp48G/jmYhtx/i5P/lHzC8stRxjCacFvomoVCrTmIM1ylcN1nD5S89tqQvusrhJsgiz3W+BOQjNdiwBMilt6nqf48y9m7rz5S3IdwF4B4CJ5t//M+fzE0IciKut2DQhv6ZtE0JR/mHb92mkrrkc40RAiPvHlcQ8a9IqJ6frbZXIjs8ttFRA84ha3CQZk+l+C4C3XXpBai6CpPUC4txn27i8zmbBc3ZiHSgTkcFu0uDtIvKB5v8vEJFXOOx3O4AvA1gnIt8Tkd9CQ4D/soh8C8AvN/9PCCkIW1BU3Chdm7B1CZ6K8knavk8jAtzlGGEBcmmlJe3YfcRaTtRE2OInyZhM9/vmrRvwodGLUjmHKXhy284ZrHEIxrPdo+27DrU9u2Gpbp2k+qXdfjZtXDTyv0LDknQZgA8C+CmAuwG8PGwnVX2r5avXxRkgISQbJqfrLdWy6rPzGLvrAIDTmoqrthKnjndQCNkC3oBw4eBqRk1Sdcz/eZh1Ii3zbNyo9SiNP8mY4tzvuOcwCeMoS42H7drMzi8suyH89dq9AjCm5zFJWdWy11kH3AT5K1X1ZSIyDQCqelxEzsh4XISQjLnx3kNtJS8XFhU33nso9gsqTBgHCQohv1Coz84vV+qKCppyMe+aXsLbds7g2p0zGB4awNBgzdj9K5g65Y3PNeDMFW+REaaLC1otGy4afx4m4zjniFqohAlY10WidwwvNSxOZ7MwwiwCVRLkCyLSj+azJCKr0dDQCSEVxta+MqqtpYmgMLYhMKeWJRE8Ln78KE2w1ieo9UtbT+ugoMxCMEalcAGNoiZbX35+ZMBg2XERxjYBG2eR6D9GWullYRYBf53+InER5H8B4LMAniMiHwbwZgDvz3RUhJDK4Qm7TRN7rC9tV3Oq6/mAcPNulPa1sKQYGqjhzJUrcheUUV3BVg3WcMMVF5ZCUHSKizC2CVjTfZ47eSrSkpJWelnYIqQy3c9U9TYR2Y+Gb1sAjKrqNzIfGSEkU4YGasYmFEMD5rrVrtgioOPkW7vgEgXtogk+Mb+AmRten2gMcQiO1zYuAfDQxOVOx/CEUtlTqIIWm7juAlPKYJSQTit+YWzLOly7c8b4XSW6n4lIH4AHVfUlAA7nMyRCSB5sv/JCvPeOGfiDpfuk8XknmF6gNqGV9EXoGoDUiSaYJqbxmhY3tvFMTtdx472HWrTQ5eBExXLEexkDsTz8wrjT1rWuQjoNl8joxuG2a+9RlipwoYJcVZdE5ICIXKCq7QmNhJBK0y+CJV8LyH4R67ZxXr7BF6jN3J70RehawatTTTAJputk89W7jCfMl27qz92ppSNtbM9NGgI2rznecMWFpa4C5+IjPxfAIRH5CoCnvA9V9crMRkUIyRxT7vLCkhqFQKcpOGmXw4yj4aepCUZhu042i4DidD/tfhFcfUm7cIrypZt4fHY+87m6UIXULRfKXgXORZDfmPkoCCG5E6ecZRpNVLzjdPoinJyuxzJLB8eR5cvXdp08YR1EgJZe58GGK5PTdef8fD9nDdRKIUA7fW7KRJmrwLkEu30xj4EQUhXKoOmkQZz0nDTab6b1IrTlXdvqgueJ7XosqmKg1h8rANDTZqPoQ2s+8ECtHyJIRYB2+qzbrkfS7nbEjEuJ1p+KyE+af06IyKKI/CSPwRFSNjrp093JObMoDxmnzGaafcM9ks4rqhxskaU0bdfDK3HrL4Ea1TbU1aTe3y8YGqi1lNKdtdQCiLPwSuNZD3s+Ov3tlL1sap5ECnJVfZaqPrv55xkArgbwseyHRkj5SFrfO+lLJ8uFQ5x66mnVFPdIOq/J6Tr6LAF5Q01zcp6LrCBh12l04zD2jl+GhyYux97xyzAcsjiKY1JfWFScuXLF8nFHNw5bBWifiPMzaHvWr7vjgPM1Dat97h0vTm18jyIW1GUmdvczVZ0UkfEsBkNI2UnapzupvzJrH6OruTsNH7ffTNtn8Bm7tCy9/p6DRl9zmubkqLGHzT3OdbIFAG5ev9rJpO7HtX69d+06aYe6qOr8/LpU/EuSghi1oO4G11ccIgW5iFzl+28fgBGY40wI6XqSlH3sRBin4ZtOi05rivsFi0kYA+Hzspma+0Vw01UXYVvKRTs84R1MW4sSgt518vbftnMGO3YfMbbQ9OYVlaoWRVj9+qQLp7D8/7iBjmEV/5K4Z8J879t2zqRaQbAKRJrWAVzh+7MFje5nb8pyUISUlSQm5k6EcRa+6SJwFU5h87JdryXVUHNykmvlN90Cbu1Ybft7pt+xuw5gw42fazFtB83toxuHYy88bM+f/9hLCRZOUWbxuONM0z0Tdk/j3qtuwEWQ/42qvrP55/9W1Q8DeFHWAyOkjMTxK3t0ImDS9k1HkVUAkctLP2peUdcxzWvlsvCIaz1YWFTMzi9E+nSjnouhgVqs5y/smFHtUG+66iJrkaC4C6Qkvx0bUYuMIGUppZoVLj7yvwTwMofPCOkJ4pqYOymGkmchiiyLd0TVPPfM42HnibqOaV6rpNYSvzk+CpN5enK6jrmTp6z7DNT6sf3K+I1Ukj6D3nnSKuaTVgqii+/dT9UsWHGxCnIReRWAVwNYLSLv9X31bADuSyFCepxOBUxehSiyDKyLqnnumcfDGN04jKlHjuH2Bx6zVkJL61pFLTzillK14V8w2Pb3/PPB/uxxS+YCyZ7BslY1i/K9e5SplGpWhGnkZwB4ZnObZ/k+/wkarUwJIY6UuSqUR5aBdd7cr7vjgDHQzUVjmpyu4+799dBKaGkR1sEtKFA9Og1Ss+1/3tAA9o5f1vJZEutJJ89gmZ/fJPeq27AK8mZFty+KyCdV9ZEcx0QIKYAkEflx6NRMm2e5zyRaaKdBanmWzM2CoioeltVikCcuPvI5EdkB4EIAz/A+VNXL7LsQQspG1Is27cYmJoIv3aHBGlRhTdHyk3cqXlwt1KX3uYcAbW6BvEvmpknRzVHKbDHIA5eo9dvQ6EW+Fo0GKg8D+NcMx0QISRljOtSdB7Dxg6fToQCkFlUcNg5PiJ81UMOTT59yiuQGyp+KFyeSWgHcf/ho5P5VKJkLRBdoYTnVbHER5D+jqn8LYEFVv6iq7wJwacbjIoSkiDEdaklxfK5ViAJoy2tOi+BiYnZ+oa2fdljOb96peHEZ3TiMqy+J31DEE3Lbds5g5Yo+rBqsVaZkbnAups9ZTjV7XEzrXvX974vI5QAeB/C87IZECEkbF5Nr1j7W7bsOOQWD2cZaFl9omIsiqGWH4dVU95ukZ+cXMFDrx81bN4TOK+1r0anPPcwtUEZ/frfhIsg/JCJnAbgOjfzxZwPYlumoCMmIMrQgLWIMrv7brHysk9N1zM6bO3IFCcvP9q5ZlKBLA9N9AhDqC3a9fp723ImQS9Mv3KnPPSy+Iu3SuaQdl37kf9/85xMANmc7HEKyo+iAnCLHEJXH7ZGVv9m1RKb38g/60p86eWrZDB/3mk1O13HjvYdwvNnac2igFllUxXafVq7oCxW8tgXT0EANZ65c0bZ4K4uQ6zRjIcxCYCvaUpbYhm7ApWnKvwHw/wJ4rqq+REReCuBKVf1Q5qMjJEXKYOIragzBF21QOALZ+pvDBNOqwRpm5xasWq9Jk3e9ZpPTdYzddaBlnrPzCxi78wCA9oVAWGW2+YVF60LIm59NM7UtHLJO+XMljYwFm4Ugj2yIXsfFtP7fAIwB+GsAUNUHReTTACjISaUoQ8pOkWMIvmjzNPHbBNaqwRqmP/D6ls82TezpyJfuZ8fuI20BdUAj0M9UHjVuZTYPT/DG9V1vXr8at+571Ph5EpLeU9NCT8QtLTDusXsxzztrXAT5oKp+RVoL59uLARNSUsqg/ZRhDB70yOJJAAAgAElEQVR55t7atLIbrriwbVvXRY3LNQs7VvA7l8psqwZrOLGw1Lbd3MlTy93MgoLLcyuYrrUtOC5O0JxHp24bb+xZuH96Pc87a1zSz34kIi9EszuciLwZwPczHRUhGRCVspNHrmtaaUNVy8uN0/nKRUC7XrOwYwW/i1pAeAuPm666CEMDtZbvjs8tLKdUxUm3StNCE5XLHcT2DMU9DikeF4389wDcAmC9iNQBPATgbZmOipAMCDPx5RWEloaZ0TTWa3fO4MZ7D+GGK+J3xsoLV63MpL3X+gTPfMaKFl+667GCPnLveMGFQFhkf7Bm947dR9p8935h5xoHkaaFJs6iIOx5L4MLisQjrPvZH6jqRwGcq6q/JCJnAuhT1Z/mNzxC0sUmTPKu493JMW0mYE8r9M5RVdJa7Pj9vQuLS3jqZOOa2aLWbeZ/k+UgibAzfZdmIFicRUHY814m9w9xI8y0/s7m338JAKr6FIU46VaK0kKSmMjDxpSXCdR13EldAKMbh7F3/DLcvHUDgEbQlev+pgpySwp8ZOsGPDxxOWZueL1xUZCG+f+8oYFY5VPjnDOKOG6bsOe97BX0SDthpvVviMjDaPQjf9D3uQBQVX1ppiMjJEeK0ELimPP9GmafiLEVqEceiw+XcXfqrkiy/+R03dgqNe0iK1GadBwtO61AsDiWjLDnnVHm1UM05IUgIj8LYDeAK4Pf5dnadGRkRKempvI6HekSolJxwoqOAHazalpsmtjjlJIVNy1q2NC/Ok1s4w6e13W7Ts/jEXWdBMBDE5dHntf/XHjd2Z6Yb/fNB58fESz78DevX437Dx8trSA0Xausn3cSDxHZr6ojLtuGBrup6g8AXJzKqAjJkShtzlTjutYnbcVJvG2z0E5smvPxuYXlVCbA7hMXNFNJfKRpArXN29UN0am7Iu7+UeljLtaV4HPhVYMD2p+hsHStu/fXl4Widx237ZwpjVCn1t1duEStE1I5ooLXbN3ABs9YEaoNpxnNHhYl7TcDhwm+j2zdkMnLOGze1jKkg60pWa7b2Yjr7gi7Tq4LnKjFgMlEH5WuVXRZYBudmvSTWC64YMgGlzxyQipHlDbnqu1lmVMbJlj84wgLnvKCwtJuOxo277Et61Drl7Z9njxxqiUYzXU7G3GDrsI0bm/sUed1sRbEsTx0a052MKDw+NyCsa88W5jmAwU56UqiIoddI4uzjGYf3TjcVljENI4ioojD5j26cRhnntFuzPPKnnq4bmfDJaLbHxX/1NOnjAsHDxch4mJ+D24T9ix1a062q+WiWxcyZcMqyEXkXhHZZfuT5yAJiUuU8HMVjnFSiZKw/coLI8eRZoqSn7DUsKh521qSBgXUE47b2QizOJjSzKCNYEEB0C/tQj1KiNisCB6mZyTsWcr6+SkKV8tFty5kykaYj/zPmn9fBeBnAdza/P9bATyc4ZgI6ZioYB7/9/XZefSLtLzkve+z7tzkGnSUdq3qKN9/2Lwnp+vGQDvArK3G8XPH8adGxTmsHb/PuF+YEBndOIztuw4ZFyr9IsYFVNQ97MbOXy797b17zOIy2WMV5Kr6RQAQkT9V1df6vrpXRL6U+cgI6ZAo4Wd60Zoik4Hui+6NCgYMm/emiT1GIS5o9/vHWQjFDSyM0vaS1gawWRGWVEMDuEwpcWlXqcvy+Ytznqj+9klz6kkyXKLWV4vIC1T1uwAgImsBJOuxR0jKxHn5mLaN8uH5t79564bUX6B51XgP4mLytC2EbPsq2sccR5DFLZMbJaiTWlOijpvnPcvrXHHPE7yvYVHr/u26ZSFcNlwE+TYAXxCR7zb/vwbAf8hsRKQrSVOr8I5Vn51vMfFGVUYzvahsGkXw+zRfoFFV2vzCKyttrJNKdrZ9hy37uroF4vpTowS1bREBNIrN2K5p1HHjLDg6FcR59QBIch7X+5q2W4i0EynIVfWfRORFANY3Pzqsqk9nOyzSTaSpVQSPFTTx2l4+thdVv6Xcqeczdzm2f2xRQjc4flup1frsPNaM3+e8UIlLJ77/TvYNu0ZxFxcu2n5QiLg8i1HHjbPg6FQQJw0Wi7sA7DQojbnixeJaEOYSNDTxFQAuFhGo6qeSnlREtgH4bTTeUQcBvFNVTyQ9HsmXuD/aNLWKqLQXwPzysb2QFlVR65e20qy2c9iO47pYcRm/H9eFSlw68d0m3beTALuwscS5Fq7PYthx4yw4OhWQSSwnSRbOnVhoinIPkdNE5pGLyP9AI4L9NQBe3vzjVP/VcrxhAL8PYERVXwKgH8Bbkh6P5EvcAg+T03VrdGuSFBSXffpE2lKqQl9IvpQlL7XLZia2Hcc1XzaNtJu0Unc6KSaTZN+oa5RVmp2fNNKh4uT1d5p+lqSGQJLc7U5qFTBXvHhcNPIRAC/WsO4qyc47ICILAAYBPJ7isUmGJPEP2kiSguKS9uKZq/2aQViUrak0KxAdbeu3TNh+HEEBYRu/zcRvIu51K4vZs5MAu7RIo8tdHItEp+mLSawfSRYrnVhomCtePC6C/Gto5JF/P40TqmpdRP4MwKMA5gF8TlU/l8axSfaE/WiDAmPu5Cmn9JQ4hAlkkzD0FhleatC1O2es4/cT9WJz7UgWFBC2F/tNV12EbTtnrAsC/7ZxrluZzJ5JhajrQsRlu7TqAsQJ9AI6i9o2+fnDgvVs17lPpKUZT9I5BSmiBTBpxUWQnwPg6yLyFQDLQW6q2tba1AURWQXgTQDWApgFcKeIvF1Vbw1sdw2AawDgggsuSHIqkgFhjTCCAiOMpCbT0Y3DmHrkGG7b92ib0LNptJ6QHt04vBztHsT00gl7sbn4uk0CIuzFbhubx3CCjmx5RT27kESIpt37PK5gTcOakaaVwWWetsXuomomi7isiyaRaFwE+faUz/lLAB5S1aMAICL3AHg1TleOAwCo6i0AbgEa/chTHgNJiO1Hq4pY/bI7eZHcf/hopObqJ1i3PI2XTpjAleY5bS9924vdNDYB8LZLL8CHRi9a/sz0Mt+2cwZTjxxr2Q4ol9nTxcoRJ88/KojQVg/AVrwlSJmsGR4u18P7+7o7DoSmNqZFGlYH0hku6WdfFJHnohHkBgBfUdUfdnDORwFcKiKDaJjWXwdgqoPjkRyx/Wi3WUzWQdJYqccVQnMnTy2bFNN66YT5tJMWjvGPzSsbu6iK+w8fjexPrgBu2/coRp5/tpOZNanZs1MN1baIiZvn79qBrJN6AGWyZni4LsxGNw5bf5NZLOKYK14skYJcRH4dwA4AX0BDQfhLERlT1buSnFBVHxCRuwB8FcApANNoat6kGph+tDaz8NBADWeuXJHqSt0l4M3P8bmFtrKrnY4hLDCtkxe9t1+Y8AmrrBY8d5pmzyw11Lh5/q413ZPUA/AokzXDI87CjL7r3sGljen7ALxcVd+hqr8B4BUA/qSTk6rqDaq6XlVfoqr/ngVmqo8tfWX7lRem3i/bdC4/SbpexcWWngZ0/qK3CbXr7jiAyel66IvYpJmlkdI1OV3HdXccyCzNKCzP3yUtyvb8RcVNhFHGzmVx0sSKaH9LisHFR94XMKX/GOxjTgLk6SfzjnnjvYdwfG6h5bskxVyS4LkTXDqAebiapcOE2vX3HMTVlwwbg/1s5+7UAuFp4p0IxSjCyr56vvKo7nBA+/MXJ7gxiM2asXn96tCocRtpBc4Bbr8z+q57B4lKDxeRHQBeCuD25kdbARxU1T/KeGzLjIyM6NQU3eikHVuAlE0ouAQ5ufL+yYNtAtVLJbPl1JvSzoLR16YgpeA8Nq9f7XTuNITHpok9kdH0nV5X1+uT93GD12/z+tW4e3899vGymh/pXkRkv6o6FV+LFOTNA16FRmU3AfAlVf1sZ0OMBwV5+pSlSEgW5PnSdL2ONmHoF4KuuekC4KGJyyPPndZ1WDt+nzVLIM3rmma+eCfbh+FyH9Pcj/QucQS5S7DbWgD/oKr3NP8/ICJrVPXhzoZJiqIsaTVZLSbyNvOnVf3KtQ77WQM1p3OnFXUdFkiW5uLI5VomeXbTjKhOGgAXFlW/dvy+rltMk3xx8ZHfiUaet8di87OXmzcnZacMaTVxX8iT0/UWn/jQQA3br7wwl5d3GrhEELv6mmfnF/D+yYNtOeNB0oq6DqtGl/c1LvrZTRoJHpZp4e9ZALDRCImPS9DaClU96f2n+e8zshsSyZoypNXEabQwOV3H2F0HWgLbZucXMHbnAWuzliLwSmcGG7YA0RHEk9N19Bmi7W3ctu/RyLmnFXWdVuR7EoLXNM0GPElIGgkelWkBsNEISY6LRn5URK5U1V0AICJvAvCjbIdFsqQM+aVxezr724x6LCxpocU5/NiqrV27c2Y58vqmqy4ymvujosJNmHLGg3SaQ150HIXpmvr7s/vJ69lN6rYJ7ufaZIcQF1wE+bsB3CYi/xWN39D3APxGpqPqAYp8SZahNrLLYsK7RmER02V58dmqrQGnzaY3XXWRMbApbo9yj6i5m+rS9zkq/WWIo7Bd06AwD1o2on5XWVWni7OfzbrAYi0kCZGmdVX9jqpeCuDnAFyoqq9W1W9nP7TuJW5P77Qp0lTq4WJq9q5RGGV58UUJ1TCzadLFiEvXsJ1feaxF6D11chFjd0W7JMrQYzqsgp3p2XX5XRX92/NgsRaSJi5R688F8J8BnKeqvyIiLwbwKlX928xH16UUHbADFB8MFmWidNFSa31SmhefS9lYm2CKW3IWcHvp79h9BAtLBpfEYrRLoqg4Cr+23Gcpz2pL2XL5XZXhtwewWAtJFxfT+icB/Hc0SrUCwDcB7ARAQZ6QMgSblYGwxUTUtYiKWs+bsD7pHjYNemzLOozddaAlDqDWL9j68vNx/+GjeHx2HkODNagCT8wvOL/0w65h1PU9a6CG2fkF4+dZEMxKAMz17MMWMC6/qzL99opeTJPuwakfuareISLXA4CqnhKR+A49soxrsFnRwUZFEqalZl1EI8l1D3YuC/PjGgnKLAVGnn92aIpZ1DjDrmGUWd4WQB8jsN6ZqEI4/SJYUo28Fy6/q7wXKITkgYsgf0pEfgbNV42IXArgiUxH1eW4BJuVIdioKCan63jq6VPW77PUnkzX/dqdM9i+61CkBcCvYblUXQszIUdF5Ls8H2Nb1mHszgNt5vVaf7RLYnauXdiFfd4JUW6UJVU8NHF55HFcfld5LlAIyQsXQf5eALsAvFBE9gJYDeDNmY6qy3Hxj5XFl5cVNkHnUqY0ywA3m1CZnV+ItZAKM5sG55ikGYnL8+H9vX3XoWUtdNVgDTdcEe2SyDNFMWph5npOl99VngsUQvIiUpCr6ldF5BcArEMj8+OIqvKpDxDXHBvlHyuTLy9twrTJKO0s68jesKCzJAsp03Nx472HnNLN4rQrtX2e1A+bZ4pimAsg7jmj5tvJAqWXXV2k3FgFuYi8HMBjqvqDpl/8EgBXA3hERLar6rHcRllysjCDx3nhFPmCSXLuMG0ybKEyHHH8NK5DvyVS2iPOQsr0XASD2mxECbCsNeY8o6ptgYJZBDQmXaD0squLlJ8wjfyvAfwSAIjIawFMAHgPgA0AbgHN68tkYQZ3feFk/YIJE45Jzx3WQGI4pC91WIBbWtchqrpaVFBUpO87RIi7BnUB2WjMpnudR2euvJvcmM4FILTHeLe7uki1CRPk/T6teyuAW1T1bgB3i8hM9kOrDlmYwV1fblm+YKKEY9Jz27RJAaz9nl1yptO4DraFhMfs/AI2TeyxVgxz8X3b+PNfv9h5rGkLv6I1zjxTsYLncpl7N7u6SPUJFeQiskJVTwF4HYBrHPfrObIyc7q83LJ8wWzf1e7L9QvHJOeenK5j7qQ5Il0B3H/4qLUmeRhZdvoKYhNySUutAg0zMhCuFQZJU/j1ssbpMvcy9CcgxEaYQL4dwBdF5EcA5gH8bwAQkf8LTD9rocja5XFqlscRjJPTdWO+LXBaOMZ9ublEpD8+O59IQKX1ovXOe90dB0I1apOQc1k01PoFULSkhA3U+vHGi88tVCOOuxDqpsAvl7mXoT8BITastdZV9cMArkOjsttrVJffan1o+MpJkyJrl8epWR6ntnRYTW1POMatF+2isSbVcNKsXT26cRh//usXR7ad9L/ow9qQ9ossPxc73nwxdvzaxW3Pyv2HjxZa2zxOy9Oy1CtPC5e5l6E/ASE2Qk3kqrrP8Nk3sxtO9QhqJjdv3ZDaj9tF60lSs9zFZBqmXXrCMa6fNkpj7UTDMeVMnzi1iGt3zmDH7iOJOlwFjxfEe9GHtSEdqPUbX/jB/2/baQ47iRsln1RLjqNxdpsZ3nXuLKlKygp93R2QZYBQnGMnqVnuUoTDGJAmDaGzY/cRbF6/erkWuIvgiCq76ip4wgTW06eWlrfz5GrS++IF9JkEueD0gsZmaegXcdbabNemTwRrx++LvL6dPotxFmV5BX7lZb7PM2qekCwQjRlZWwQjIyM6NTVV9DDasPUUdq0FHvai6vTYnY7RxZ8dxKZ9hh0zap84xwjTnoH4125yuo5rLZoyADzcLBu6dvy+tlLpQEPYe6VFXUq2Rl3vsGtlu8/9IrGi4V2wnWvVYA2DZ6ywpiraKvm5VviL+6wQUmVEZL+qjrhsG9mPnNjpRDOJ8jOmGYWdxHcc9An2OxSjjvLphvkZJ6fr2DSxB2vH78OmiT1Wf6stkj5KiAPJirnYGPb5T6N8rC4+ZZfrnaSn+aIqrt05gw03fi41H7bpmar1C548cco4R9v83z950HpdytAPnZCqQNN6B3QSKR3lZ0w7CjuJ2dBvsl87fp/T+aKEpckN4GoWDoukjxLiQLxrFxaY5y2EPIET1e3M1afscr2T9jQ31YpParo2PVNPPX2q7R74Ba9p/rc/8FhbXEFUhT/mbRPSDgW5j7gvtk5SUqJeVGmmu6QRpBMlKPzbxcVV0HWijcW9dmECY+WKPly7c6ZFeCuw/P+gvz+JUIq7kHPJf/df0zR86v7t4i48gPBmMUXkbXdTSh3pLWhab5IkpaaTlJQoc2zZ0l1M5tQgSRcaroIuTCisGrSXTk1y7Wz3R3Ba+ze0EEe/iLEveJxzAI3rXetrNa/X+uztR73nJcoFUp+dx9rx+3DdHQdSNV2HzdH2nW2snhBNK53QhW5LqSO9BQV5k6Q+udGNw9g7fhkemrgce8cvi5XuE/WiSnrsLDAtLN5+6QUdLTQ8v7gt3DJY1MaWp+215jRdz49s3ZDo2pnuT9B8bmJRtU0A2HzKTz19KjwmIDjdiDAF1/x3RbLWqWGEPc+27976yvOt++S9kKVPnlQZmtab5O2Tq2LKS5p5tFFR2qaiNrY8bX9/bc9n3S/S8iKOO27T/XFxLQD2vuDesYYGa3jyxGmfssmsvWP3kbYGKwuLas3V9puFzxqooU+Ap07GLxeb1HRtyrt/Rq2v5TvTsz7y/LOtv4E887bpkydVhoK8SRE+ueCLytNQqyLYw4jyN4YFkwV9zK552t7fQd/vtp0zmHrkGD40epHz+Lzj+T+zpV2ZCOsLvmliD47PmQPDkvjVg4ui2fkFDNT68fZLL1jO83dJMk3DdO3P4z8+1xpgZ3qWy1JkhbXUSZWhab1J3j65IGX30bmmh3nbBueybecM1vj2tQkqAdpM4bZtl1TbhIBJ6CuA2/Y9ujzmpNfaJU7Af07bdXIR0jYBMjRYa7sPNrPw/YePLrtmhkP81GmZrqtsni76909IJ1Ajb1K0qbuTspdZR9uGRTgDreZUL+jMJEz9+w4N1tq0UsAswOJoSzYhqcDytYwSOGGmXu97F83cFgnuMh9TFLqXq+1dN+/4NsuGS9OPNP3OVTZPF/37J6QTKMh9FGnmS/oSzKOPtE3w3XjvITx54lRLJy+TcA4yv7CIlSv6MFDrd0qvi5OKF+bL9q6l7ZoGBaPpWnrPiKuZ3bQYc5lPnFztfhFj/ECw6UfweGkLqqqbp8ti5ickLjStl4QkKUpAPuZMm+A7PrfQIsTjMDu/gJUrTj9+qwZrVu0wTgTz2JZ11uBu71qGpUO5Xss4ZnaTv9xlPsGshScsRW8WVZ3MwllnQdA8TUgxUCMvCUkLwORhzowTse2KPx8bAE4sLNk3hru2NLpxGFOPHMNt+x61VlqzXWsXE7X/PECrhjt38pSzu8B1Pn63SZ9F8/aCA4s2C9M8TUgxUJCXhKQvwU7Mma6+dZPgi8qpHhqoQcRuag/um2YbzA+NXhSZ1gS0X2ub79t2LU1ZB2lV4zMdz5Z+582tDAKzLOMgpJegIC8QkyCN050LSK7Jx22TCpwO8ooS4rV+wfYrL8SO3UecfOYenVoR4gT92QTO2J0HWtwFXjW1NHrDxyUs7W5JlRovIQQABXlhpBWkllR4xI2Sdw3y8qqsjW4cxraQFqAm4gRFBQXr5vWrcff+eudBf4ZqalOPHHM+dpoaaVjandcelRQD67KTMkFBHpO0fsCdpJsFcREewXFHRXbbCMv/DgqXOL71OCZo0yIo6BMH2q+nS5EaUzW1sC5dWb68qx4Fnid5CtY8MkUIiQMFeQxMP+Cxuw5g+65DeGJ+IdYLJCwFau34fam+jEzjtpnHo4SETbj0iWByuo7RjcOh7T1NBCu5RWEr+mKiPju/bEXwj8X08g3r6W0i6/xom9tk8/rVXVMBMA3yFqxpLsIJSQMK8hiYfsALixpaM9tGmLbqrzbmcqwk4/a33fRw0Ypt7TK9ZiFBM7TpPEH8cQEumlUcASrA8nWO0tht98QlTzsLTG6T1FwIBZOmBp23YK1y4RvSnTCP3JHJ6XqsAiBRuOQhp5UPHlbtLG53KS8H2tSCcn5hEbc/8Jiztuzx/smGIHItnRrWYjT4/6hzB6ufuXbpAoCnnj6VeQndYO73/YePVrYMqkfa5YjzFqxJaz4QkhUU5A54Lx5XXF4gwaIgnRwrirACKEk0otGNw1iymJttZugwvDrorsVtbAL3bYG2qi4jCVY/MxVq+dDoRbjpqovaep7Pzi/kXg+/G7TBtIsY5S1YWfiGlI1CTOsiMgTgbwC8BA2l6V2q+uUixuJCWKcuE64vkGBHLJPG7zXfcBW0JpNlmDkcSGaejWuGDsOrg+4qpFwj9aMi7G3Vz2xR+6Z0urx9o90QAJf2YiRpCmZSWPiGlI2iNPKPAvgnVV0P4GIA3yhoHE6EvWBqfa36dNIXSJip3dX0aDNZAmjRNG1m8TgaURwzdJjFwcN7IZqwVUaLKjdqGqM3liTdvsqgDXeDNpi2Bh2nhG9aZF3ulpA45K6Ri8izAbwWwG8CgKqeBHAy73HEwaYFpVkaM6qzlovmF2ay9L9s1o7fZ9w/jkAK00qCVdWCAVomvP3T1KyiNKe4/d/T0IY7DfLqBm0wCw2aFeVIL1OEaf0FAI4C+O8icjGA/QD+QFWfKmAsToS9eNJ8gXjHWjt+n9G/6xe0JoHgqjGmZZ4NChVPozddE0+4m87rv5b+46UhpGz3x5ayNPXIMdx/+Kjx/J0KoDSLAFVdaK1c0bd8HfxFhAgh8SlCkK8A8DIA71HVB0TkowDGAfyJfyMRuQbANQBwwQUX5D5IP1kImDDNLErQ2gTCWQO1tjaX/v08bD7zuZOnlnPBXecQp8yrS2GWvISUzXrhLywTnE+nzwHzj8316KMa5hBCwhFNEGXc0QlFfhbAPlVd0/z/zwMYV1VrzcmRkRGdmprKaYTZY2uu4fn1or63BXGtGqzhxMKSdb/gGLbvOtQm+G3bm7CNY3hoIHbN+LyxWT1MpDUf2zlNVfG6lSo/M4TkiYjsV9URl21z18hV9Qci8piIrFPVIwBeB+DreY8jKMjyNO9FaWY2zQ8Ij8Q+PreAj2zd4KQxelHYQUEeR0N0MeWXtSZ1nPKxpnn6q9d5kfpRFeq6IeK8U8oQMEhIt1FUZbf3ALhNRM4A8F0A78zz5JPT9bYuV8fnFjB21wEA2VfIci3PGqx4ZjKH+/Eisl01m05fqkldAEDxVcjGtqxrewZsBAWtrb1o1PyyTpMq66LJDxczhKRPIelnqjqjqiOq+lJVHVXV43mef8fuI8YX+MKitqRgeVHNa8fvw6aJPaHpX3G2DXtp2SpdueSyK4Dr7jjgNIawcbi+VG0pc56vPe3CH6kTyIvr7xOndMKwexE2vyzTpNKulpYV3ZA+R0jZ6Mla62Eap/ddHG0yrua5ef1qY7cuP0ETt6uWHKfIS6caonfcoK/9+NxCqPWgDGZUU6ezxSXFswdrGDxjRUe13sO+jxPMF0fDrkogXTekzxFSNnpSkIf5R4eaZThdXox+P2kQ20t0crqOu/fXnQKt/AIhjk83agweabxUw3ztRTUbccEmbGfnFjD9gdeH7ht1L9KYX9zFYZV8z92QPkdImehJQR7mH33yRMMsHOXHHhqs4ckTp0J9rKZjxCn36vc1z84lq5kT9SJP46Ua1v5zoNafW+nMOBpsWDvWsDayk9N1PPX0KesY0ppfXA2bvmdCepeebJoyunEYO37tYmPp0IWlhp88yo99fG4hMlDKdAxXrdoTCJPTdYzddQBPnWwX/iv6ooufJn2Rp+Hz93zAeZTOjOsjtvn3F1Wt+3vnCFofvNuQ5vziatj0PRPSu/SkRg40hPm2nTPG7x6fncfNWzdERomHYXuJ2szNfQKce9ZAmza5aWJPmy/X41TEQiLqRW7TYOOadfOqfBdGXA026FboM9yX4P42a8q5Z6WfAx1Xw6bvmZDepWcFORD+sgy+GOOUzQnLJ7Z1BltSc9pYUh9nVE5zmLDuVCgWIUSS+Ij9iwyX+vN5+qGTBCLS90xIb9LTgjzqZel/MUa1xPT2jTKtDoc0YDGRJMjNpUpWmLC2nS9sHEULkU59xC775+mHLsPiiBBSDXpakMd5WZqEfq1fcOYZK/DE/ILzizaupjW2ZR3G7jpgNa8HcQEQwg4AABK6SURBVPWLhmmXNvO/v/1pnMCyLAuV+DMHBGixnMTxEbvcF5dt0pxr0YsjQkg16GlBDri/LNPSkOIex/v8xnsP4fhce0OUJIsJIFy7tGnennDPMsc+DsFjK7AszKNcC0Fc7kvUNmWuZEcI6V5yb5qShG5rmtIJaWl8YY1ZbOZ1z2Qfp/FFlk0yXI6dZ9lSNgQhhKRFqZumVIWy1q1Oy9wapV2GmZDjBH1lGSAWdey8NeQqFWUhhHQPFOQGesVEalsURAn5MLN8cAE0NFgzugTSCBCLCj7Lu2wpi7IQQoqgJwvCRFH6Zh85MLpxGHvHL8NDE5dj7/hlLYLPVnxk8/rVbUVZnjxxCrX+6EYkSYgqghJWnc+lqUza4yGEkCygIDdAE2k4ti5e9x8+2rYAWlhSnHnGikyqu0V1EwvThLPoDpZldzNCCLHBYDcDZQhaKquPPoy14/cZC+cIgIcmLs97OE493BmIRggpIwx265BO23smwS+4gw1Zkvro814MlM1H7Pf121LqaGUhhFSdnjat2xqD5G0i9RqjeL5lU0OWuD76uE1E0qCMPmLP1x9WOY8QQqpMz2rkUZHpaaV5uWjFN957yKlym0l7tB0/r4jt4PmvvmQY9x8+WjqXQBFWFkIIyYOeFeRpCbowQe2axmZKzzIR1B7Djp9HwJ7p/Hfvr5cywIu1ywkh3UrPCvI0BF2UoE5TKzZpj2HHt/mrFY1gvjSEWN552p3SK7XLqxgoSQhJTs/6yG2+Uf/nNh+6R1S+uetiYWigZtxOmn9MPvrJ6XpoAJfJX+1h8pdHzdVlHlGfk+wpIjaCEFIsPSvIowKzXF6IUYLMZbEAAG+8+Ny2bWp9gpu3bjAWZPHGZsPrp+4F7JnwLziSvvxd50fyg8WMCOk9elaQR0Wmu7wQowSZSxT35HQdd+9vF5gLS4odu48YhalpbKbjexHbYtzy9IIj6cs/aZR6Eu2fuEErCSG9R88KcqAh6Ma2rMN5QwN4fHa+RXC6vBCjBNnoxmFcfcnwch/vfhFcfUmrnzZMKNs047CXsinQLGrBkfTlnyRNj6bfbKGVhJDeo6cFuUmobNs5gzXj96FPzHqs/4UYJcg8bdvr472oirv3153M8x4mzdj2Uh5umtSDRC04Onn5h9VkN0HTb7aUMZefEJItPRu1DpiFipfNvWgoXeu9ECen67jx3kPLaWNDAzXcvHVDmxBzieq2RZf7CQr7uDnRUalXeeZY0/SbLUyzI6T36GlB7iI8+kWwpLr8QgSAsbsOtBRwmZ1fwHvvmAHQmh/uap6Pqgce1IyTvKzDUq/yfPmXrYxrN9IraXaEkAY9LchdtOEl1ZaGH5sm9hirsC1po0Kb/wXqIrSC9cAFaGk8YtOMO3lZ2/KM83j5s8IaIYSkS0/7yMNyrT2CmmKYFh+s0Obqr/T8zA9PXI6bt27ItMZ70cFmbPVJCCHp0tMaeZQ2DABzJ09hcroey6dtOn4aJvA0KEM1Npp+CSEkPdiP3MfkdB3bdx3C7HyrZu0J+OGhAWxevxq37nvUuP/QQA0zN7w+83F2Qtl6hhNCCGknTj/ynjatBxndOIwzV7YbKTzB5zUF2fTCs9u2qfUJtl95YcYj7JxuyDNmQRlCCDkNBXkAl7zuh388j48EfNk7fu3iRObivIVS1fOMi/bxE0JI2ehpH3mQyek6+kSMOeR+Hp+dT8XP69rmNE2qnmdcBh8/IYSUiZ4W5P40rKHBGp48cSpSiAPpmaGLEkpFBpt517w+O4/+5qJpOMZiggVlCCGklZ4V5EFtOJg6ZiNNM3SvCaXgNfcWTXEsESwoQwghrfSsjzysWUmQLHKePTO+iSihVNVgr7Br7lpvveo+fkIISZue1chdtd7hoQHsHb8s1XN7mqnNjP/U062566Z98/Srp0XUNXe5J2n4+G2V7QghpIr0rCB3KeySlaYXZQ2YnV+wCucqB3tFXXNX83in5WmruhAihBATPWtaN5loa/2CoYFa5qVDXTRPm6m5KL96Gub8sJK4eZnH2UaVENJt9KxGXmQalmuZV5NwLiLYKy0tNlgSN0nUeqf0WoAhIaT76VlBDhSXhuXSuhQwC+ciuoelac4vus46o94JId1Gz5rWiyTYAWzVYA21vtYI9rD2pXl3D+smLZZR74SQbqOnNfIiCWqmcSKp89Zqu0mLrXplO0IICVJY9zMR6QcwBaCuqm8M2zav7mfETNBHDjS0WPYRJ4SQbIjT/axIjfwPAHwDwLMLHEMpKHteM7VYQggpL4UIchF5HoDLAXwYwHuLGENZqEpec9FBaoQQQswUFez2EQB/BGDJtoGIXCMiUyIydfTo0VRPXqYSp8xrJoQQ0gm5a+Qi8kYAP1TV/SLyi7btVPUWALcADR95WuePowHnYfLupohwQggh+VOEaX0TgCtF5FcBPAPAs0XkVlV9ex4nd82JDhP43nHSEPDdFBFOCCEkf3I3ravq9ar6PFVdA+AtAPbkJcQBdw3YJvC37zqE6+85iPrsPBSnBXxS8zzzmgkhhHRCzxWEsWm6wc9tAn92fiFVn3YRBV4IIYR0D4UWhFHVLwD4Qp7ndC1x6loP3aMTnzYjwgkhhCSl5zRyVw3YZvJeNVgzHpc+bUIIIUXQkyVaXTRgWxEUAJEafdkLvBBCCOkeelKQuxIm8G2CuioFXkxwAUIIIdWDgjwBUQI+rZafeVLlBQghhPQyPecjz5qqFnhhhTlCCKkmFOQp45reVjaqugAhhJBeh4I8Zapa4KWqCxBCCOl1KMhTpqoFXqq6ACGEkF6HwW4ZUMUCL+w5Tggh1YSCvAlTr6q5ACGEkF6HghxMvSKEEFJd6CMHU68IIYRUFwpyMPWKEEJIdaEgB1OvCCGEVBcKciRLvZqcrmPTxB6sHb8Pmyb2YHK6nvUwCSGEkDYY7Ib4qVcMjiOEEFIWKMibxEm9qmpjFEIIId0HTesJYHAcIYSQskBBngAGxxFCCCkLFOQJYF1yQgghZYE+8gSwLjkhhJCy0LOCvNPa6qxLTgghpAz0pCBn+hghhJBuoSd95KytTgghpFvoSUHO9DFCCCHdQk8KcqaPEUII6RZ6UpAzfYwQQki30JPBbkwfI4QQ0i30pCAHmD5GCCGkO+hJ0zohhBDSLVCQE0IIIRWGgpwQQgipMBTkhBBCSIWhICeEEEIqDAU5IYQQUmEoyAkhhJAKQ0FOCCGEVBgKckIIIaTCUJATQgghFUZUtegxRCIiRwE8ksGhzwHwowyOmzecR/nolrl0yzyA7plLt8wD6J65ZDGP56vqapcNKyHIs0JEplR1pOhxdArnUT66ZS7dMg+ge+bSLfMAumcuRc+DpnVCCCGkwlCQE0IIIRWm1wX5LUUPICU4j/LRLXPplnkA3TOXbpkH0D1zKXQePe0jJ4QQQqpOr2vkhBBCSKXpSkEuIm8QkSMi8m0RGTd8v1JEdja/f0BE1vi+u775+RER2ZLnuE04zOW9IvJ1EXlQRP5ZRJ7v+25RRGaaf3blO/K2cUbN4zdF5KhvvL/t++4dIvKt5p935DvytnFGzeNm3xy+KSKzvu/KdD8+ISI/FJGvWb4XEfmL5jwfFJGX+b4rzf1ojidqLm9rzuFBEfkXEbnY993DInKweU+m8hu1cZxR8/hFEXnC9wx9wPdd6HOZNw5zGfPN42vN38bZze/KdE/OF5H7ReQbInJIRP7AsE3xvxVV7ao/APoBfAfACwCcAeAAgBcHtvldAB9v/vstAHY2//3i5vYrAaxtHqe/5HPZDGCw+e/f8ebS/P+TRd+PGPP4TQAfM+x7NoDvNv9e1fz3qrLOI7D9ewB8omz3ozmW1wJ4GYCvWb7/VQD/CEAAXArggbLdjxhzebU3RgC/4s2l+f+HAZxT9P1wnMcvAvh7w+exnssyzCWw7RUA9pT0npwL4GXNfz8LwDcN767CfyvdqJG/AsC3VfW7qnoSwGcAvCmwzZsA/F3z33cBeJ2ISPPzz6jq06r6EIBvN49XFJFzUdX7VXWu+d99AJ6X8xhdcLknNrYA+LyqHlPV4wA+D+ANGY0zirjzeCuA23MZWUxU9UsAjoVs8iYAn9IG+wAMici5KNf9ABA9F1X9l+ZYgfL+RlzuiY1Ofl+ZEHMuZf6dfF9Vv9r8908BfAPAcGCzwn8r3SjIhwE85vv/99B+4Ze3UdVTAJ4A8DOO++ZJ3PH8FhorQ49niMiUiOwTkdEsBuiI6zyubpqm7hKR82PumwfOY2m6ONYC2OP7uCz3wwXbXMt0P5IQ/I0ogM+JyH4RuaagMcXhVSJyQET+UUQubH5W2XsiIoNoCLe7fR+X8p5IwwW7EcADga8K/62syOKgBSOGz4Kh+bZtXPbNE+fxiMjbAYwA+AXfxxeo6uMi8gIAe0TkoKp+J4NxRuEyj3sB3K6qT4vIu9GwmFzmuG9exBnLWwDcpaqLvs/Kcj9cqMpvxBkR2YyGIH+N7+NNzXvyHACfF5HDTW2yjHwVjbKdT4rIrwKYBPAiVPieoGFW36uqfu29dPdERJ6JxmLjWlX9SfBrwy65/la6USP/HoDzff9/HoDHbduIyAoAZ6FhBnLZN0+cxiMivwTgfQCuVNWnvc9V9fHm398F8AU0VpNFEDkPVf2xb+z/DcAlrvvmSJyxvAUBc2GJ7ocLtrmW6X44IyIvBfA3AN6kqj/2Pvfdkx8C+CyKdaWFoqo/UdUnm//+BwA1ETkHFb0nTcJ+J6W4JyJSQ0OI36aq9xg2Kf63UmQgQRZ/0LAyfBcNs6YX+HFhYJvfQ2uw2x3Nf1+I1mC376LYYDeXuWxEI9DlRYHPVwFY2fz3OQC+hYICYBznca7v3/8WwL7mv88G8FBzPqua/z67rPNobrcOjYAdKeP98I1pDeyBVZejNYDnK2W7HzHmcgEa8S6vDnx+JoBn+f79LwDeUOJ5/Kz3TKEh3B5t3h+n57JMc2l+7ylQZ5b1njSv76cAfCRkm8J/K11nWlfVUyLyHwHsRiOa8xOqekhEPghgSlV3AfhbAP9DRL6NxoP0lua+h0TkDgBfB3AKwO9pq2k0VxznsgPAMwHc2YjXw6OqeiWAnwPw1yKyhIblZUJVv17iefy+iFyJxnU/hkYUO1T1mIj8KYB/bR7ug9pqhssNx3kAjeCdz2jz19ykNPcDAETkdjSioM8Rke8BuAFADQBU9eMA/gGNaNxvA5gD8M7md6W5Hx4Oc/kAGjEwf9X8jZzSRoOL5wL4bPOzFQA+rar/lPsEmjjM480AfkdETgGYB/CW5jNmfC4LmMIyDnMBGgv2z6nqU75dS3VPAGwC8O8BHBSRmeZn/wmNxWFpfius7EYIIYRUmG70kRNCCCE9AwU5IYQQUmEoyAkhhJAKQ0FOCCGEVBgKckIIIaTCUJATUgGktXPajIisEZEREfmLGMcYEpHfDfn+95tdnm5LML41IvLv4u5HCOkcpp8RUgFE5ElVfabjtiu00UMg+PkaNLpnvcSy32EAv6KNhkFxx/eLAP5QVd8Yc7/+Ims1ENINUCMnpKJIoz/13zf/vV1EbhGRzwH4lIhcKCJfaWrvD4rIiwBMAHhh87MdgWN9HI02mLtEZJuInCmNntL/KiLTIvKm5nZrROR/i8hXm39e3TzEBICfbx57mzT6y3/Md/y/bwp7iMiTIvJBEXkAjSYgl4jIF5tNMnY3O0cRQhzpuspuhHQpA77KUg+p6r81bHMJgNeo6ryI/CWAj6rqbSJyBhoVv8YBvERVNwR3VNV3i8gbAGxW1R+JyH9Go0f0u0RkCMBXROR/AfghgF9W1RPNxcHtaDTrGYdPIxeR3wyZy5lolO78QLOO9RfRqIF+VES2AvgwgHfFvD6E9CwU5IRUg3mTAA6wS1Xnm//+MoD3icjzANyjqt9qlr105fUArhSRP2z+/xlolKV8HMDHRGQDgEUA/ybOQZss4nTbynUAXoJGlyugseD4foJjEtKzUJAT0j0s16xW1U83TdeXA9gtIr+NRmMNVwTA1ap6pOVDke0A/g+Ai9FwzZ2w7H8Kra67Z/j+fcLnFxcAh1T1VTHGRgjxQR85IV1Is+f5d1X1LwDsAvBSAD8F8CzHQ+wG8B5pqski4rVcPQvA91V1CY1mEv3Nz4PHfhjABhHpE5HzYW9FeQTAahF5VfM8NRG50HGMhBBQkBPSrWwF8LWmX309gE9pow/3XhH5WjDYzcCfotGt6kER+Vrz/wDwVwDeISL70DCre1aAB9HownVARLYB2ItG28aDAP4MwFdNJ1HVk2h09fovInIAwAyAV5u2JYSYYfoZIYQQUmGokRNCCCEVhoKcEEIIqTAU5IQQQkiFoSAnhBBCKgwFOSGEEFJhKMgJIYSQCkNBTgghhFQYCnJCCCGkwvz/DA+i0rzbq4kAAAAASUVORK5CYII=\n",
      "text/plain": [
       "<Figure size 576x432 with 1 Axes>"
      ]
     },
     "metadata": {},
     "output_type": "display_data"
    }
   ],
   "source": [
    "# We will use a simple training set\n",
    "X = 2 * np.random.rand(500, 1)\n",
    "y = 5 + 3 * X + np.random.randn(500, 1)\n",
    "\n",
    "fig = plt.figure(figsize=(8,6))\n",
    "plt.scatter(X, y)\n",
    "plt.title(\"Dataset\")\n",
    "plt.xlabel(\"First feature\")\n",
    "plt.ylabel(\"Second feature\")\n",
    "plt.show()"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 4,
   "metadata": {
    "ExecuteTime": {
     "end_time": "2018-03-26T14:37:06.920607Z",
     "start_time": "2018-03-26T14:37:06.906905Z"
    }
   },
   "outputs": [
    {
     "name": "stdout",
     "output_type": "stream",
     "text": [
      "Shape X_train: (375, 1)\n",
      "Shape y_train: (375, 1)\n",
      "Shape X_test: (125, 1)\n",
      "Shape y_test: (125, 1)\n"
     ]
    }
   ],
   "source": [
    "# Split the data into a training and test set\n",
    "X_train, X_test, y_train, y_test = train_test_split(X, y)\n",
    "\n",
    "print(f'Shape X_train: {X_train.shape}')\n",
    "print(f'Shape y_train: {y_train.shape}')\n",
    "print(f'Shape X_test: {X_test.shape}')\n",
    "print(f'Shape y_test: {y_test.shape}')"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "## Linear regression class"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 5,
   "metadata": {
    "ExecuteTime": {
     "end_time": "2018-03-26T14:37:08.615318Z",
     "start_time": "2018-03-26T14:37:08.519693Z"
    }
   },
   "outputs": [],
   "source": [
    "class LinearRegression:\n",
    "    \n",
    "    def __init__(self):\n",
    "        pass\n",
    "\n",
    "    def train_gradient_descent(self, X, y, learning_rate=0.01, n_iters=100):\n",
    "        \"\"\"\n",
    "        Trains a linear regression model using gradient descent\n",
    "        \"\"\"\n",
    "        # Step 0: Initialize the parameters\n",
    "        n_samples, n_features = X.shape\n",
    "        self.weights = np.zeros(shape=(n_features,1))\n",
    "        self.bias = 0\n",
    "        costs = []\n",
    "\n",
    "        for i in range(n_iters):\n",
    "            # Step 1: Compute a linear combination of the input features and weights\n",
    "            y_predict = np.dot(X, self.weights) + self.bias\n",
    "\n",
    "            # Step 2: Compute cost over training set\n",
    "            cost = (1 / n_samples) * np.sum((y_predict - y)**2)\n",
    "            costs.append(cost)\n",
    "\n",
    "            if i % 100 == 0:\n",
    "                print(f\"Cost at iteration {i}: {cost}\")\n",
    "\n",
    "            # Step 3: Compute the gradients\n",
    "            dJ_dw = (2 / n_samples) * np.dot(X.T, (y_predict - y))\n",
    "            dJ_db = (2 / n_samples) * np.sum((y_predict - y)) \n",
    "            \n",
    "            # Step 4: Update the parameters\n",
    "            self.weights = self.weights - learning_rate * dJ_dw\n",
    "            self.bias = self.bias - learning_rate * dJ_db\n",
    "\n",
    "        return self.weights, self.bias, costs\n",
    "\n",
    "    def train_normal_equation(self, X, y):\n",
    "        \"\"\"\n",
    "        Trains a linear regression model using the normal equation\n",
    "        \"\"\"\n",
    "        self.weights = np.dot(np.dot(np.linalg.inv(np.dot(X.T, X)), X.T), y)\n",
    "        self.bias = 0\n",
    "        \n",
    "        return self.weights, self.bias\n",
    "\n",
    "    def predict(self, X):\n",
    "        return np.dot(X, self.weights) + self.bias"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "## Training with gradient descent"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 7,
   "metadata": {
    "ExecuteTime": {
     "end_time": "2018-03-26T14:37:25.473465Z",
     "start_time": "2018-03-26T14:37:25.201638Z"
    }
   },
   "outputs": [
    {
     "name": "stdout",
     "output_type": "stream",
     "text": [
      "Cost at iteration 0: 66.45256981003433\n",
      "Cost at iteration 100: 2.2084346146095934\n",
      "Cost at iteration 200: 1.2797812854182806\n",
      "Cost at iteration 300: 1.2042189195356685\n",
      "Cost at iteration 400: 1.1564867816573\n",
      "Cost at iteration 500: 1.121391041394467\n"
     ]
    },
    {
     "data": {
      "image/png": "iVBORw0KGgoAAAANSUhEUgAAAfEAAAGDCAYAAAA72Cm3AAAABHNCSVQICAgIfAhkiAAAAAlwSFlzAAALEgAACxIB0t1+/AAAADl0RVh0U29mdHdhcmUAbWF0cGxvdGxpYiB2ZXJzaW9uIDIuMi4wLCBodHRwOi8vbWF0cGxvdGxpYi5vcmcvFvnyVgAAIABJREFUeJzt3XmcXXV9//HXe7bMlpnJMgkhCQlLWFUQAwVRKyC4Cz/rRrGipfKzdV+qUn/1V/uzVWvrVq0tikoVFUUtuAJGcBcIKJshEJZITMi+TbbJzHx+f5zvJHeGm2QmmTvnnrnv5+Mxj3vPej/3m8y87/d7zj1HEYGZmZkVT13eBZiZmdnBcYibmZkVlEPczMysoBziZmZmBeUQNzMzKyiHuJmZWUE5xK1mSLpF0l/lXcd4knScpN9K2irpLXnXU86h/rtI+qGkS8ayprEkqUfSUWO9rhk4xG2cSXpU0o4UKpsk/UrSGyT5/2IZY/DB493ALRExOSI+NVZ17YukL0n6YKVfp1REPD8irhrr/Up6tqQVh7qfiGiPiIfHel0zcIhbPl4cEZOBecCHgfcAV+Zb0oQ1D7gv7yIqQZlc/4ZJasjz9c0c4pabiNgcEdcDrwQukfQkAEmTJP2rpD9IWi3pPyW1pGVLJL1ocB+SGiStk3Rqmj4j9e43SbpL0rPLvbakOkn/R9JySWsk/bekzrRsvqSQdJmklZJWSXpnybb/IOmbkr6SRhTukXSspMvTvh6TdH7J+p2Srkz7+aOkD0qqT8teK+kX6f1ulPSIpOenZf8EPBP4dBpm/fQ+3stLJN2X3vMtkk5I838CnF2y/bFltp0q6YvpfW6U9D8ly14vaZmkDZKul3R4mi9JH0/vdbOkuyU9SdJlwMXAu9PrfXcf9Z4n6f607acBDWvbr5RMD/5bNKTpWyT9k6RfAtuBo0pHK/bXnmn5kZJ+lv7dfizpM6WvV7JeG/BD4PD0XnokHZ7quzb9228BXivpdEm/Tu2/StKnJTWV7CskHZOefym95vdTDbdKOvog1z1f0tLUjv8h6aeqscNF5hC3KhARtwEryAIL4CPAscApwDHAbOD9adnXgItKNn8usC4i7pQ0G/g+8EFgKvAu4FuSusu87GvTz9nAUUA7MDwkzwYWAOcD75X0nJJlLwa+DEwBfgvcQPb7NBv4R+C/Sta9CuhL7+WpaX+lf2z/BFgKTAf+BbhSkiLifcDPgTelYdY3DX8TKZi/BrwN6AZ+AHxXUlNEnDNs+wfKtMOXgVbgJGAG8PG033OADwGvAGYBy4Gvp23OB55F9m/URfYhbH1EXAFcDfxLer0Xl6l3OvAt4P+k9/sQcFaZuvbnL4DLgMmpruHKtmda9lXgNmAa8A9pX08QEduA5wMr03tpj4iVafEFwLVk7/1qoB94e3q9M4Fzgb/ZT/0XAR8g+7+zDPin0a6b2vFa4PL0XpYCT9/PfmyCcohbtVgJTE1/bF8PvD0iNkTEVuCfgVel9b4KvERSa5r+8zQP4NXADyLiBxExEBE3AYuBF5R5vYuBj0XEwxHRQ/bH8FUaOjz6gYjYFhH3AF9k6IeHn0fEDRHRB3yTLEA/HBG7ycJuvqQuSTPJwuBtaV9ryILyVSX7Wh4Rn4uIfrLAnwXMHGG7vRL4fkTclF77X4EWRvAHXdKsVNsbImJjROyOiJ+WtM8XIuLOiNiV2udMSfOB3WQBejygiFgSEatGWO8LgN9HxLWp3k8Aj49w20Ffioj7IqIv7WO4su0p6QjgNOD9EdEbEb8Arh/lawP8OiL+J/0f2xERd0TEb1I9j5J9gPvT/Wz/7Yi4Lf3fuZrsw+po130BcF9EfDst+xSjb0ebAHw8x6rFbGADWRi2Anfs7TwhoB4gIpZJWgK8OA3XvoSsdwvZ8d+XSyrtATYCN5d5vcMZ2otbTvb7UBqejw1b/uSS6dUlz3eQjQb0l0xD1rs/PNWwquT91A3b954/vhGxPa3XXqbmcoa8j4gYkPQYWXseyFxgQ0Rs3Md+7yzZb4+k9cDsiPhJGgb/DHCEpO8A74qILSOsd897j4hI9Y7GgdbfV3tOJ3u/24fta+6hvH4aDfkYsJDs/24DcMdI6iM7JLC/f+t9rVuuHQ/5JDwrHvfELXeSTiMLnV8A68hC8KSI6Eo/nRFR+oducEj9ArJe3bI0/zHgyyXbdUVEW0R8uMzLriQL/UFHkA15l4bz3GHLVzJ6jwG7gOklNXVExEkj3P5Atxkc8j7SSMZc4I8jrG2qpK4R7LeNbNj2jwAR8amIeBrZMPyxwN+OsN5VlLRrSb2DtpEF4aDDyuzjYG+9uIrs/Zbuf38Bvq/XGT7/s8D9wIKI6AD+jpLj/BWyCpgzOJHacc6+V7eJyiFuuZHUoewkta8DX4mIeyJiAPgc8HFJM9J6syU9t2TTr5Mdl/1r9g6lA3yFrIf+XEn1kpqVfU2o3B+3rwFvTyc6tZMN2V+ThiYH/b2kVkknAa8Drhnte0zDzDcC/5beb52koyXtb7i11GqyY/b78g3ghZLOldQIvJPsQ8OvRljbD4H/kDRFUqOkZ6XFXwVeJ+kUSZPI2ufWiHhU0mmS/iS93jZgJ9lx4ZHU+33gJEkvTYcu3sLQoP4d8CxJRyg70fDyA72PkYqI5WSHV/5BUpOkM8nObdiX1cC0VMf+TAa2AD2Sjif7f1lp3weeLOnC1I5vpPwHHpvgHOKWh+9K2krWE3wf2VDk60qWv4fsJJ7fpDOAfwwcN7gwhc+vyY77XlMy/zGy3vnfAWvT/v+W8v/Pv0B2UtfPgEfIgujNw9b5aapjEfCvEXHjwb1dXgM0Ab8HNpKdkDRrhNt+EniZsjOtn/A974hYSnYuwL+TjWK8mOwrfL0j3P9fkB3jvh9YQ3aCHBGxCPh7spPQVgFHs/c4fgfZB62NZEP568mOxUP2VcET05nae850L6l3HfBysq8Wric7cfCXJctvIvs3vZtsSPp7I3wfI3Ux2cln68lOgLyG7EPPE0TE/WQf9h5O7+fwfezzXWTnZmwla5dRf9gbrZJ2/Bey93Ii2QeUsu/FJi5FHOzIlNnElE7eegRoHNYztwlG0jXA/RHxf/Ou5VAo+778CuDiiCh3DohNUO6Jm1nNSIcCjk6HNZ5HNnLzhBGDIkiHjbrS4Y7B4/C/ybksG2c+O93MaslhwLfJTtJbAfx1RPw235IO2plk5y4MHqq5MCJ27H8Tm2g8nG5mZlZQHk43MzMrKIe4mZlZQRXimPj06dNj/vz5eZdhZmY2Lu644451EVHuvg9DFCLE58+fz+LFi/Muw8zMbFxIKndznyfwcLqZmVlBOcTNzMwKyiFuZmZWUA5xMzOzgnKIm5mZFZRD3MzMrKAc4mZmZgXlEDczMysoh7iZmVlBOcTNzMwKyiFuZmZWUDUX4vc/voVfPbQu7zLMzMwOWc2F+H/99GHefe3deZdhZmZ2yGouxNsnNdCzqy/vMszMzA5ZzYX45OYGtu7sIyLyLsXMzOyQ1FyItzc30D8Q7Nw9kHcpZmZmh6TmQnxycyMAW3fuzrkSMzOzQ1N7IT6pAYCtPi5uZmYFV3sh3pyFeM9Oh7iZmRVbzYV4+2BP3CFuZmYFV3MhPnhMvGeXj4mbmVmx1WCIZz3xLe6Jm5lZwdVsiPuYuJmZFV3NhXibj4mbmdkEUXMh3lhfR0tjvY+Jm5lZ4dVciEN21Tb3xM3MrOhqMsQnNzf4Yi9mZlZ4tRnik9wTNzOz4qvJEG9vbqDH1043M7OCq8kQnzyp0T1xMzMrvJoM8fbmBnp8TNzMzAquJkN8ss9ONzOzCaA2Q3xS1hMfGIi8SzEzMztotRni6SYo23rdGzczs+KqyRBvb/alV83MrPhqMsT33ATFJ7eZmVmB1WSIt++5CYq/K25mZsVVkyE+eEzcw+lmZlZkFQ1xSV2SrpV0v6Qlks6UNFXSTZIeTI9TKllDOZN9TNzMzCaASvfEPwn8KCKOB04GlgDvBRZFxAJgUZoeVz4mbmZmE0HFQlxSB/As4EqAiOiNiE3ABcBVabWrgAsrVcO++Ji4mZlNBJXsiR8FrAW+KOm3kj4vqQ2YGRGrANLjjArWUFZbUwMS9Hg43czMCqySId4AnAp8NiKeCmxjFEPnki6TtFjS4rVr145pYXV1or2pgS0OcTMzK7BKhvgKYEVE3JqmryUL9dWSZgGkxzXlNo6IKyJiYUQs7O7uHvPifBMUMzMruoqFeEQ8Djwm6bg061zg98D1wCVp3iXAdZWqYX+ym6D4mLiZmRVXQ4X3/2bgaklNwMPA68g+OHxD0qXAH4CXV7iGstonuSduZmbFVtEQj4jfAQvLLDq3kq87EpObG9m4vTfvMszMzA5aTV6xDdIxcZ/YZmZmBVazId7R7LPTzcys2Go4xBvZ4hPbzMyswGo3xFsa6e0bYOfu/rxLMTMzOyg1HeIAW3a4N25mZsVUsyHemUJ8s0PczMwKqmZDvCPdyczHxc3MrKhqNsTdEzczs6Kr2RDfe0zcXzMzM7NiqtkQH+yJezjdzMyKqmZDvKM5Dadvd4ibmVkx1WyINzXU0dJY7564mZkVVs2GOEBHS4NPbDMzs8Kq6RDvbGn0iW1mZlZYNR3iHc2N7ombmVlh1XSId7b4JihmZlZcNR3iHS3uiZuZWXHVdIhnx8Qd4mZmVkw1HeIdzQ1s3dXHwEDkXYqZmdmo1XaItzQSAVt3+Qx1MzMrnpoPcfA9xc3MrJhqOsR9JzMzMyuymg7xweunuyduZmZFVNMh7juZmZlZkdV0iHe0NAAeTjczs2Kq6RDf0xP39dPNzKyAajrE25oaqJN74mZmVkw1HeJ1daLD1083M7OCqukQh+wMdZ+dbmZmRVTzId7pm6CYmVlB1XyId7Q0sGWnT2wzM7PiqfkQd0/czMyKquZDvKPZIW5mZsVU8yHe2ZqFeIRvR2pmZsVS8yHe1dJEb98AO3b3512KmZnZqNR8iE9pza7atnG7h9TNzKxYaj7Eu1qbANi0vTfnSszMzEanoZI7l/QosBXoB/oiYqGkqcA1wHzgUeAVEbGxknXsz2BPfJN74mZmVjDj0RM/OyJOiYiFafq9wKKIWAAsStO5GeyJb3RP3MzMCiaP4fQLgKvS86uAC3OoYQ8fEzczs6KqdIgHcKOkOyRdlubNjIhVAOlxRrkNJV0mabGkxWvXrq1YgXuOiW9zT9zMzIqlosfEgbMiYqWkGcBNku4f6YYRcQVwBcDChQsr9iXupoY62prq2eQLvpiZWcFUtCceESvT4xrgO8DpwGpJswDS45pK1jASXa1NPiZuZmaFU7EQl9QmafLgc+B84F7geuCStNolwHWVqmGkulobfXa6mZkVTiWH02cC35E0+DpfjYgfSbod+IakS4E/AC+vYA0jMsU9cTMzK6CKhXhEPAycXGb+euDcSr3uwehqbWTlph15l2FmZjYqNX/FNshC3D1xMzMrGoc42XD65h27GRjwnczMzKw4HOJkZ6cPBGzZ6ZPbzMysOBziQFeLr59uZmbF4xAHprQNXnrVx8XNzKw4HOKU3o7UPXEzMysOhzjZiW3gnriZmRWLQxwfEzczs2JyiAMdLY1IsMk9cTMzKxCHOFBfJzpbGn1PcTMzKxSHeOLrp5uZWdE4xJPOlkY2+57iZmZWIA7xZIqvn25mZgXjEE+mtDaxcZt74mZmVhwO8aSrtclnp5uZWaE4xJOpbY1s6+1n5+7+vEsxMzMbEYd4Mq19EgAbtrk3bmZmxeAQT6a2ZZdedYibmVlROMST6e1ZiK/r2ZVzJWZmZiPjEE+mtnk43czMisUhnng43czMisYhnnQ0N9BYL9b1OMTNzKwYHOKJJKa2NbFhm4+Jm5lZMTjES0xrm+ThdDMzKwyHeIlp7U0eTjczs8JwiJfIhtMd4mZmVgwO8RIeTjczsyJxiJeY1t5Ez64+Xz/dzMwKwSFewt8VNzOzInGIl5iWQny9T24zM7MCcIiXmJaun77e3xU3M7MCcIiXmObrp5uZWYE4xEtMbfdwupmZFYdDvMTkSdn109e7J25mZgXgEC8hKX1X3MfEzcys+jnEh5na1uThdDMzK4SKh7ikekm/lfS9NH2kpFslPSjpGklNla5hNKa1N3k43czMCmE8euJvBZaUTH8E+HhELAA2ApeOQw0jNs3XTzczs4KoaIhLmgO8EPh8mhZwDnBtWuUq4MJK1jBaU9smsb7Hx8TNzKz6Vbon/gng3cBAmp4GbIqIvjS9AphdbkNJl0laLGnx2rVrK1zmXtMnN7Gtt58dvb5+upmZVbeKhbikFwFrIuKO0tllVo1y20fEFRGxMCIWdnd3V6TGcrrbswu+rN3q3riZmVW3hgru+yzgJZJeADQDHWQ98y5JDak3PgdYWcEaRm1GRzMAa3t2csS01pyrMTMz27eK9cQj4vKImBMR84FXAT+JiIuBm4GXpdUuAa6rVA0Hwz1xMzMrijy+J/4e4B2SlpEdI78yhxr2qXtyFuJrHOJmZlblKjmcvkdE3ALckp4/DJw+Hq97MKa2NVEn98TNzKz6+Yptw9TXientkxziZmZW9RziZXRPnuThdDMzq3oO8TK6J7snbmZm1c8hXsYMh7iZmRWAQ7yM7smTWNezi4GBstehMTMzqwoO8TK62yfRNxBs3O4boZiZWfVyiJex96ptHlI3M7Pq5RAvY88FX7Y4xM3MrHo5xMvwpVfNzKwIHOJlDPbEPZxuZmbVzCFeRtukBtqa6j2cbmZmVc0hvg/dkye5J25mZlXNIb4P2VXbduZdhpmZ2T45xPdhxuRmXz/dzMyq2ohCXNKXRzJvIvH1083MrNqNtCd+UumEpHrgaWNfTvXonjyJrTv72NHbn3cpZmZmZe03xCVdLmkr8BRJW9LPVmANcN24VJiTw9JV21Zv8XFxMzOrTvsN8Yj4UERMBj4aER3pZ3JETIuIy8epxlzM6sxCfNVmh7iZmVWnkQ6nf09SG4CkV0v6mKR5FawrdzNTiD++ZUfOlZiZmZU30hD/LLBd0snAu4HlwH9XrKoqMDic7p64mZlVq5GGeF9EBHAB8MmI+CQwuXJl5a9tUgMdzQ2sdoibmVmVahjhelslXQ78BfDMdHZ6Y+XKqg6zOlvcEzczs6o10p74K4FdwF9GxOPAbOCjFauqSszsbOZxn51uZmZVakQhnoL7aqBT0ouAnRExoY+JA8zqaOZx98TNzKxKjfSKba8AbgNeDrwCuFXSyypZWDU4rLOZtT272N0/kHcpZmZmTzDSY+LvA06LiDUAkrqBHwPXVqqwajCrs5kIWLN1F7O7WvIux8zMbIiRHhOvGwzwZP0oti2sPd8V3+zvipuZWfUZaU/8R5JuAL6Wpl8J/KAyJVWPWXtC3DdCMTOz6rPfEJd0DDAzIv5W0kuBZwACfk12otuENqsjG0Jf5Z64mZlVoQMNiX8C2AoQEd+OiHdExNvJeuGfqHRxeetoaaC5sc5nqJuZWVU6UIjPj4i7h8+MiMXA/IpUVEUkMauzxd8VNzOzqnSgEG/ez7KaOF37MH9X3MzMqtSBQvx2Sa8fPlPSpcAdlSmpuszqbPalV83MrCod6Oz0twHfkXQxe0N7IdAE/K9KFlYtZnY2s2brTgYGgro65V2OmZnZHvsN8YhYDTxd0tnAk9Ls70fETypeWZU4vLOZ3f3Bup5dzOjY39EFMzOz8TWi74lHxM3AzRWupSrNnpId+l+xaYdD3MzMqkrFrromqVnSbZLuknSfpA+k+UdKulXSg5KukdRUqRrGwpwprQCs2OjvipuZWXWp5KVTdwHnRMTJwCnA8ySdAXwE+HhELAA2ApdWsIZDNnjN9D86xM3MrMpULMQj05MmG9NPAOew98YpVwEXVqqGsdA2qYEprY2s2Lg971LMzMyGqOhNTCTVS/odsAa4CXgI2BQRfWmVFcDsStYwFuZMaeWPm9wTNzOz6lLREI+I/og4BZgDnA6cUG61cttKukzSYkmL165dW8kyD2h2V4uPiZuZWdUZl9uJRsQm4BbgDKBL0uBZ8XOAlfvY5oqIWBgRC7u7u8ejzH2aM6WFP27cQUTZzxtmZma5qOTZ6d2SutLzFuA5wBKyr6q9LK12CXBdpWoYK7OntLBjdz8btvXmXYqZmdkeleyJzwJulnQ3cDtwU0R8D3gP8A5Jy4BpwJUVrGFM+GtmZmZWjUZ0sZeDke5+9tQy8x8mOz5eGHu+ZrZpByfP7cq5GjMzs8y4HBMvuj1XbfPXzMzMrIo4xEegs6WRjuYGX/DFzMyqikN8hGZPafUxcTMzqyoO8RGaM6XFF3wxM7Oq4hAfocELvvi74mZmVi0c4iM0Z0oLPbv62Lxjd96lmJmZAQ7xETtiavZd8eXrfYa6mZlVB4f4CM2f3gbA8g0OcTMzqw4O8RHa0xNfty3nSszMzDIO8RFqbqxnVmczj3o43czMqoRDfBTmTWtl+Xr3xM3MrDo4xEdh/rQ298TNzKxqOMRHYd60Ntb17KJnV1/epZiZmTnER2PetMGvmXlI3czM8ucQH4W9Ie4hdTMzy59DfBTmTcu+K/6oe+JmZlYFHOKj0D6pgentk1i+zj1xMzPLn0N8lOZPa3VP3MzMqoJDfJTmTWvzMXEzM6sKDvFRmj+tlce37GRHb3/epZiZWY1ziI/S4I1QPKRuZmZ5c4iP0tHd7QAsW9OTcyVmZlbrHOKjdFR3G5JD3MzM8ucQH6XmxnrmTmll2VqHuJmZ5cshfhCOmdHOQ+6Jm5lZzhziB+GYGe08vG4b/QORdylmZlbDHOIH4Zjudnr7Bnhsg78vbmZm+XGIH4SjZ2RfM/PJbWZmlieH+EE4pnsygE9uMzOzXDnED0JnayPT2ye5J25mZrlyiB+kY2a08ZB74mZmliOH+EE6ZkY7y9b0EOEz1M3MLB8O8YN0THc7W3f2sWbrrrxLMTOzGuUQP0jHHpad3Lb08a05V2JmZrXKIX6QTjisA4D7H9+ScyVmZlarHOIHaUpbE4d1NLNklXviZmaWD4f4ITh+1mSWrHJP3MzM8lGxEJc0V9LNkpZIuk/SW9P8qZJukvRgepxSqRoq7YRZHTy0tofevoG8SzEzsxpUyZ54H/DOiDgBOAN4o6QTgfcCiyJiAbAoTRfS8YdNZnd/+KIvZmaWi4qFeESsiog70/OtwBJgNnABcFVa7SrgwkrVUGknzvLJbWZmlp9xOSYuaT7wVOBWYGZErIIs6IEZ+9jmMkmLJS1eu3bteJQ5akdOb6Opoc7Hxc3MLBcVD3FJ7cC3gLdFxIjTLiKuiIiFEbGwu7u7cgUegob6Oo6d2c79/q64mZnloKIhLqmRLMCvjohvp9mrJc1Ky2cBaypZQ6WdcFiHe+JmZpaLSp6dLuBKYElEfKxk0fXAJen5JcB1laphPBw/q4N1Pb2s2boz71LMzKzGVLInfhbwF8A5kn6Xfl4AfBg4T9KDwHlpurBOOjw7ue2+le6Nm5nZ+Gqo1I4j4heA9rH43Eq97nh70uxOJLj7sc2cfVzZc/TMzMwqwldsO0Ttkxo4prudu1ZsyrsUMzOrMQ7xMfCUOV3cvWKT7y1uZmbjyiE+Bk6e28m6nl5WbvbJbWZmNn4c4mPgKXO6ALj7MQ+pm5nZ+HGIj4ETZk2msV7ctWJz3qWYmVkNcYiPgUkN9Rx/WAd3++Q2MzMbRw7xMfKUOZ3cs2IzAwM+uc3MzMaHQ3yMnDyni627+nh43ba8SzEzsxrhEB8jp87LTm67c/nGnCsxM7Na4RAfI0d3tzOltZHbH92QdylmZlYjHOJjRBJPmzfVIW5mZuPGIT6GTj9yCo+u3+47mpmZ2bhwiI+hhfOnAnDHoz4ubmZmlecQH0NPOryT5sY6bvOQupmZjQOH+BhqaqjjlLldLHZP3MzMxoFDfIydNn8q963cTM+uvrxLMTOzCc4hPsYWzp/KQMAd/r64mZlVmEN8jJ02fwpN9XX8ctm6vEsxM7MJziE+xlqbGjh1Xhc/f9AhbmZmleUQr4BnLuhmyaotrOvZlXcpZmY2gTnEK+CsY6YDeEjdzMwqyiFeAU+e3UlnS6ND3MzMKsohXgH1deLpR0/jFw+uI8L3Fzczs8pwiFfIMxZMZ+Xmnb6/uJmZVYxDvEKetaAbgJvvX5NzJWZmNlE5xCtk7tRWjj9sMjf9fnXepZiZ2QTlEK+g55wwk8XLN7JxW2/epZiZ2QTkEK+g55w4k/6B4JYHPKRuZmZjzyFeQU+Z3cmMyZM8pG5mZhXhEK+gujpx7gkz+enStezq68+7HDMzm2Ac4hV2/okz2dbbz68eWp93KWZmNsE4xCvs6cdMY3JzA9+9a2XepZiZ2QTjEK+wSQ31PO+kw7jxvtXs3O0hdTMzGzsO8XHwklMOp2dXny/8YmZmY8ohPg7OPGoa09ubuN5D6mZmNoYc4uOgob6OFz55FovuX8PWnbvzLsfMzCaIioW4pC9IWiPp3pJ5UyXdJOnB9DilUq9fbV5yyuH09g3ww3sfz7sUMzObICrZE/8S8Lxh894LLIqIBcCiNF0TTj1iCkd1t3HN7Y/lXYqZmU0QFQvxiPgZsGHY7AuAq9Lzq4ALK/X61UYSF512BHcs38gDq7fmXY6ZmU0A431MfGZErAJIjzPG+fVz9WdPm0NTfR1fu+0PeZdiZmYTQNWe2CbpMkmLJS1eu3Zt3uWMialtTZx/0ky+fecf/Z1xMzM7ZOMd4qslzQJIj/v84nREXBERCyNiYXd397gVWGl/fvoRbN6xm+/dvSrvUszMrODGO8SvBy5Jzy8Brhvn18/dmUdPY8GMdq78xSNERN7lmJlZgVXyK2ZfA34NHCdphaRLgQ8D50l6EDgvTdcUSfzVM49kyaot/HKZb4piZmYHr6FSO46Ii/ax6NxKvWZRXHDKbD56wwN87ucP84wF0/Mux8zMCqpqT2ybyJob67nkzHn89IG1LH3cXzczM7OD4xDPyavPmEdrUz2fvnlZ3qWYmVlBOcRzMqWtidedNZ/v3rWSJau25F2OmZkVkEM8R69/5lFMntTAx296IO9SzMysgBziOepqbeKvnnkUN/5+NXev2JRQiQshAAAP10lEQVR3OWZmVjAO8Zz95TPmM7WtiQ9+b4m/N25mZqPiEM/Z5OZG3nX+cdz26Aa+66u4mZnZKDjEq8ArT5vLSYd38KEfLGF7b1/e5ZiZWUE4xKtAfZ34wEtOYtXmnXxy0YN5l2NmZgXhEK8SC+dP5VWnzeVzP3uY3z3mk9zMzOzAHOJV5O9eeAIzO5p51zfv8q1KzczsgBziVaSjuZEPvfTJLFvTw0dvWJp3OWZmVuUc4lXm2cfN4DVnzuPKXzzCjfc9nnc5ZmZWxRziVeh9LzyBJ8/u5J3fvIs/rN+edzlmZlalHOJVaFJDPf9x8akIeP1/L2bLzt15l2RmZlXIIV6l5k5t5bOvfhoPre3hjVffye7+gbxLMjOzKuMQr2JnHTOdf37pk/n5g+t4z7V30z/gy7KamdleDXkXYPv3ioVzWbNlJ/964wM01IsPv/Qp1NUp77LMzKwKOMQL4E3nLKC3b4BP/WQZEfChlz6ZhnoPopiZ1TqHeEG8/bxjQeJTix5kw7ZePv3np9LSVJ93WWZmliN35wpCEu8471g+eOGTuHnpGl72n7/y18/MzGqcQ7xgXn3GPD5/yUIe27CdF/37z1m0ZHXeJZmZWU4c4gV0zvEz+f5bnskR01q59KrFvP+6e+nZ5VuYmpnVGod4Qc2d2sq1b3g6rztrPl/+zXKe+/GfccvSNXmXZWZm48ghXmDNjfX83xefxLVvOJPmxjpe+8Xbed0Xb2Pp41vzLs3MzMaBQ3wCeNq8qXz/Lc/k8ucfz+LlG3n+J3/GO675ncPczGyCU0T1XwVs4cKFsXjx4rzLKIRN23v5zM3L+Mpv/sCO3f2cfVw3lzx9Ps9c0E29LxJjZlYIku6IiIUHXM8hPjFt3NbLl3+znKt+9Sjrt/VyWEczf/a02bz01Dkc3d2ed3lmZrYfDnEDYFdfP4uWrOGbix/jpw+sZSDg6O42zjvxMM47cQYnz+ny1d/MzKqMQ9ye4PHNO/nRvav48ZI1/Obh9fQNBG1N9Zw6bwpnHDWN04+cykmHd9Da5Av5mZnlySFu+7V5x25+8eA6bn1kPbc+vIGlq7OT4OoER05v46TDOznx8A6O7m7nyOmtzJ3ayqQGX+bVzGw8OMRtVDZu62Xx8o3ct3Iz963cwu9XbuGPm3bsWV4nOLyrhfnT2pjZ0cxhnZM4rKOZmSU/Xa2NNDc66M3MDtVIQ9zjpgbAlLYmzjtxJuedOHPPvM3bd/PI+m08sq6HR9Zt59F121i+YTsPPbSONVt3lb2/eXNjHV0tTXS1NtLZ0khXayNdLU20NzfQ2lRPS1M9rY31tDY1ZM+bsueDy5rq62hsqKOxXkyqr6exQTTW19FQJySfXW9mVsohbvvU2drIKa1dnDK36wnL+geC9T27eHzLTlZv2cXqLTvZvGM3m7b3smn7bjbt2J19CFi3jU3bN9Gzq48du/s52IEfCRrr67KQr8+CvbG+jqaGOuoE9XWiTqK+TkOfS9TVUWZe9ji4/t7lILJtQNQpe+06CZHdiEZK65Qso8y80vX3Tj9xXl2apuT54KNKtitdn5Jp9uw3q6H0s07p9oPLSrehZN7ghyQN2x9Dttn3/hg2b8/+tHefPKGGJ+6v9N98z/4GaznU/Q1bXrq/4fWX29/ggtL3WG5/w+vf265D/80GX8PsYDnE7aDU14kZHc3M6Gge8TYRwc7dA2zv7WN7bz87dvezvbef7b197EjTu/sH6O0boLc/2N03sGd6d3+a1z9Qss4Au/uDgYGgfyDoj/Q8sumBwccB2N0/MGTekOUBfQMDDAxkH06CIAKygYZsecTexwgISualdQgYiBiyzGy0SkM/m9aw6cHlQ1fc1/ID7e+J249sOw3bwRPX33cdwz+4lH7wGZPaD1ADB1r/AHVQZrvTj5zK5c8/gfHmELdxI4mWNGw+Le9ixklp6A/seZ4eS0J/cBll5sWeDwYlywZiz6jGnv2xd13S+gxZlq07uKy0lr3rj3J/aaMh+0/LYehrDN8fw9picHnp/kpfb1/7o9yyIduk1xy2bbn9Mez9lrZ96f7Kbjts3hPXHdrGQ9t8SCMPa/N9rz98OU9YPrLtho+QxUHWwfDlZbYbde1PWL/8cvZV48HWPso6WnI6H8ghblZBg0PnAPVo/yubmY1SLlf5kPQ8SUslLZP03jxqMDMzK7pxD3FJ9cBngOcDJwIXSTpxvOswMzMrujx64qcDyyLi4YjoBb4OXJBDHWZmZoWWR4jPBh4rmV6R5g0h6TJJiyUtXrt27bgVZ2ZmVhR5hHi5s3viCTMiroiIhRGxsLu7exzKMjMzK5Y8QnwFMLdkeg6wMoc6zMzMCi2PEL8dWCDpSElNwKuA63Oow8zMrNDG/XviEdEn6U3ADUA98IWIuG+86zAzMyu6XC72EhE/AH6Qx2ubmZlNFLlc7MXMzMwOnUPczMysoBziZmZmBeUQNzMzKyjF8HvQVSFJa4HlY7jL6cC6Mdxf0bk9hnJ77OW2GMrtMZTbY6ixbI95EXHAK50VIsTHmqTFEbEw7zqqhdtjKLfHXm6LodweQ7k9hsqjPTycbmZmVlAOcTMzs4Kq1RC/Iu8CqozbYyi3x15ui6HcHkO5PYYa9/aoyWPiZmZmE0Gt9sTNzMwKr+ZCXNLzJC2VtEzSe/OuZzxI+oKkNZLuLZk3VdJNkh5Mj1PSfEn6VGqfuyWdml/lY0/SXEk3S1oi6T5Jb03za7U9miXdJumu1B4fSPOPlHRrao9r0h0HkTQpTS9Ly+fnWX8lSKqX9FtJ30vTtdwWj0q6R9LvJC1O82rydwVAUpekayXdn/6GnJl3e9RUiEuqBz4DPB84EbhI0on5VjUuvgQ8b9i89wKLImIBsChNQ9Y2C9LPZcBnx6nG8dIHvDMiTgDOAN6Y/g/UanvsAs6JiJOBU4DnSToD+Ajw8dQeG4FL0/qXAhsj4hjg42m9ieatwJKS6VpuC4CzI+KUkq9O1ervCsAngR9FxPHAyWT/T/Jtj4iomR/gTOCGkunLgcvzrmuc3vt84N6S6aXArPR8FrA0Pf8v4KJy603EH+A64Dy3RwC0AncCf0J2wYqGNH/P7w3ZLYTPTM8b0nrKu/YxbIM5ZH+IzwG+B6hW2yK9r0eB6cPm1eTvCtABPDL83zjv9qipnjgwG3isZHpFmleLZkbEKoD0OCPNr5k2SsOfTwVupYbbIw0f/w5YA9wEPARsioi+tErpe97THmn5ZmDa+FZcUZ8A3g0MpOlp1G5bAARwo6Q7JF2W5tXq78pRwFrgi+lwy+cltZFze9RaiKvMPJ+eP1RNtJGkduBbwNsiYsv+Vi0zb0K1R0T0R8QpZL3Q04ETyq2WHidse0h6EbAmIu4onV1m1QnfFiXOiohTyYaG3yjpWftZd6K3RwNwKvDZiHgqsI29Q+fljEt71FqIrwDmlkzPAVbmVEveVkuaBZAe16T5E76NJDWSBfjVEfHtNLtm22NQRGwCbiE7V6BLUkNaVPqe97RHWt4JbBjfSivmLOAlkh4Fvk42pP4JarMtAIiIlelxDfAdsg95tfq7sgJYERG3pulryUI91/aotRC/HViQzjZtAl4FXJ9zTXm5HrgkPb+E7Njw4PzXpDMrzwA2Dw4VTQSSBFwJLImIj5UsqtX26JbUlZ63AM8hO1nnZuBlabXh7THYTi8DfhLpgF/RRcTlETEnIuaT/W34SURcTA22BYCkNkmTB58D5wP3UqO/KxHxOPCYpOPSrHOB35N3e+R9skAOJye8AHiA7Ljf+/KuZ5ze89eAVcBusk+Hl5Idu1sEPJgep6Z1RXYG/0PAPcDCvOsf47Z4BtmQ1t3A79LPC2q4PZ4C/Da1x73A+9P8o4DbgGXAN4FJaX5zml6Wlh+V93uoULs8G/heLbdFet93pZ/7Bv9e1urvSnqPpwCL0+/L/wBT8m4PX7HNzMysoGptON3MzGzCcIibmZkVlEPczMysoBziZmZmBeUQNzMzKyiHuNkYkxSS/q1k+l2S/mGM9v0lSS878JqH/DovT3dpunnY/MMlXZuenyLpBWP4ml2S/qbca5lZeQ5xs7G3C3ippOl5F1Iq3cVvpC4F/iYizi6dGRErI2LwQ8QpZN+xH00NDftZ3AXsCfFhr2VmZTjEzcZeH3AF8PbhC4b3pCX1pMdnS/qppG9IekDShyVdrOxe3/dIOrpkN8+R9PO03ovS9vWSPirp9nTv4v9dst+bJX2V7IITw+u5KO3/XkkfSfPeT3ZRnP+U9NFh689P6zYB/wi8Utm9pl+ZrvD1hVTDbyVdkLZ5raRvSvou2c002iUtknRneu0L0u4/DByd9vfRwddK+2iW9MW0/m8lnV2y729L+pGy+zn/S0l7fCnVeo+kJ/xbmE0E+/tUbGYH7zPA3YOhMkInk918ZAPwMPD5iDhd0luBNwNvS+vNB/4UOBq4WdIxwGvILut4mqRJwC8l3ZjWPx14UkQ8Uvpikg4nuwf208juk32jpAsj4h8lnQO8KyIWlys0InpT2C+MiDel/f0z2aVH/zJdyvU2ST9Om5wJPCUiNqTe+P+KiC1ptOI3kq4nu5nEkyK7GcvgXeYGvTG97pMlHZ9qPTYtO4XsbnS7gKWS/p3sTlKzI+JJaV9d+296s2JyT9ysAiK7M9p/A28ZxWa3R8SqiNhFdqnGwRC+hyy4B30jIgYi4kGysD+e7LrWr1F2S9FbyS4FuSCtf9vwAE9OA26JiLWR3UrzamB/d6k6kPOB96YabiG7LOkRadlNETF4cxAB/yzpbuDHZLdnnHmAfT8D+DJARNwPLAcGQ3xRRGyOiJ1k17KeR9YuR0n6d0nPA/Z3pzqzwnJP3KxyPgHcCXyxZF4f6cOzJAFNJct2lTwfKJkeYOjv6vBrJQdZML45Im4oXSDp2WS3TCyn3K0SD4WAP4uIpcNq+JNhNVwMdANPi4jdyu4a1jyCfe9Labv1Aw0RsVHSycBzyXrxrwD+ckTvwqxA3BM3q5DU8/wG2Uligx4lG74GuABoPIhdv1xSXTpOfhSwFLgB+Gtlt1lF0rHK7jy1P7cCfyppejrp7SLgp6OoYyswuWT6BuDN6cMJkp66j+06ye7bvTsd2563j/2V+hlZ+JOG0Y8ge99lpWH6uoj4FvD3ZLeMNJtwHOJmlfVvQOlZ6p8jC87bgOE91JFaSha2PwTekIaRP082lHxnOhnsvzjASFtkt0W8nOxWm3cBd0bEdfvbZpibgRMHT2wD/h/Zh5K7Uw3/bx/bXQ0slLSYLJjvT/WsJzuWf+/wE+qA/wDqJd0DXAO8Nh122JfZwC1paP9L6X2aTTi+i5mZmVlBuSduZmZWUA5xMzOzgnKIm5mZFZRD3MzMrKAc4mZmZgXlEDczMysoh7iZmVlBOcTNzMwK6v8Dj3L+kKLPpGUAAAAASUVORK5CYII=\n",
      "text/plain": [
       "<Figure size 576x432 with 1 Axes>"
      ]
     },
     "metadata": {},
     "output_type": "display_data"
    }
   ],
   "source": [
    "regressor = LinearRegression()\n",
    "w_trained, b_trained, costs = regressor.train_gradient_descent(X_train, y_train, learning_rate=0.005, n_iters=600)\n",
    "\n",
    "fig = plt.figure(figsize=(8,6))\n",
    "plt.plot(np.arange(600), costs)\n",
    "plt.title(\"Development of cost during training\")\n",
    "plt.xlabel(\"Number of iterations\")\n",
    "plt.ylabel(\"Cost\")\n",
    "plt.show()"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "## Testing (gradient descent model)"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 8,
   "metadata": {
    "ExecuteTime": {
     "end_time": "2018-03-26T14:37:27.326617Z",
     "start_time": "2018-03-26T14:37:27.312972Z"
    }
   },
   "outputs": [
    {
     "name": "stdout",
     "output_type": "stream",
     "text": [
      "Error on training set: 1.0955\n",
      "Error on test set: 1.0\n"
     ]
    }
   ],
   "source": [
    "n_samples, _ = X_train.shape\n",
    "n_samples_test, _ = X_test.shape\n",
    "\n",
    "y_p_train = regressor.predict(X_train)\n",
    "y_p_test = regressor.predict(X_test)\n",
    "\n",
    "error_train =  (1 / n_samples) * np.sum((y_p_train - y_train) ** 2)\n",
    "error_test =  (1 / n_samples_test) * np.sum((y_p_test - y_test) ** 2)\n",
    "\n",
    "print(f\"Error on training set: {np.round(error_train, 4)}\")\n",
    "print(f\"Error on test set: {np.round(error_test)}\")"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {
    "ExecuteTime": {
     "end_time": "2018-03-10T12:46:26.518968Z",
     "start_time": "2018-03-10T12:46:26.516036Z"
    }
   },
   "source": [
    "## Training with normal equation"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 9,
   "metadata": {
    "ExecuteTime": {
     "end_time": "2018-03-26T14:37:30.245175Z",
     "start_time": "2018-03-26T14:37:30.236084Z"
    }
   },
   "outputs": [],
   "source": [
    "# To compute the parameters using the normal equation, we add a bias value of 1 to each input example\n",
    "X_b_train = np.c_[np.ones((n_samples)), X_train]\n",
    "X_b_test = np.c_[np.ones((n_samples_test)), X_test]\n",
    "\n",
    "reg_normal = LinearRegression()\n",
    "w_trained = reg_normal.train_normal_equation(X_b_train, y_train)"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "## Testing (normal equation model)"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 10,
   "metadata": {
    "ExecuteTime": {
     "end_time": "2018-03-26T14:37:32.088447Z",
     "start_time": "2018-03-26T14:37:32.077406Z"
    }
   },
   "outputs": [
    {
     "name": "stdout",
     "output_type": "stream",
     "text": [
      "Error on training set: 1.0228\n",
      "Error on test set: 1.0432\n"
     ]
    }
   ],
   "source": [
    "y_p_train = reg_normal.predict(X_b_train)\n",
    "y_p_test = reg_normal.predict(X_b_test)\n",
    "\n",
    "error_train =  (1 / n_samples) * np.sum((y_p_train - y_train) ** 2)\n",
    "error_test =  (1 / n_samples_test) * np.sum((y_p_test - y_test) ** 2)\n",
    "\n",
    "print(f\"Error on training set: {np.round(error_train, 4)}\")\n",
    "print(f\"Error on test set: {np.round(error_test, 4)}\")"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "## Visualize test predictions"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 13,
   "metadata": {
    "ExecuteTime": {
     "end_time": "2018-03-26T14:38:03.737624Z",
     "start_time": "2018-03-26T14:38:03.447771Z"
    }
   },
   "outputs": [
    {
     "data": {
      "image/png": "iVBORw0KGgoAAAANSUhEUgAAAfIAAAGDCAYAAADQ75K0AAAABHNCSVQICAgIfAhkiAAAAAlwSFlzAAALEgAACxIB0t1+/AAAADl0RVh0U29mdHdhcmUAbWF0cGxvdGxpYiB2ZXJzaW9uIDIuMi4wLCBodHRwOi8vbWF0cGxvdGxpYi5vcmcvFvnyVgAAIABJREFUeJzsvXucHGWZsH3d3dOT9BDMEBLFTMJhdTesHCNBXRNdATG6QMiiBvF8YNF1PxF0A0ERhqwuI/EVZVffBVFRQUxEHAPoRpeAbrKiJk4IZoX1XUGSCUoCmUAyk0zPzPP9Ud2d6u46dldXV03f1+8XmO6uquep6uq6n/ssxhgURVEURUknmVZPQFEURVGU+lFBriiKoigpRgW5oiiKoqQYFeSKoiiKkmJUkCuKoihKilFBriiKoigpRgW5klhE5BMicmud+94mIp/2+NyIyEvrn10yEJEHReTi4t/vEJEf13mcH4nIe6Kdne+Yfy8ifxKRfSJyZJxjJ5VGvkOlfVFBPskQkSdEZEREnheRIRH5LxH5kIgE+q5F5NiikOto8jx9xzHG/LMx5uJmzmMyYYy5wxjzBr/tRKRXRG6v2vdNxphvNG92NXPIAZ8H3mCMmWaMeSaCYz4hIq+P4DjvFZENjR7H5dieC8ig36Gi2FFBPjk5zxhzOHAM0AdcCXy1tVNS/Gj24ilhvAiYCmwLu6NY6LOrAdrsXpv06I9hEmOM2WuMWQtcCLxHRE4EEJFzRGRARJ4Tke0i0mvb7WfF/w8VTZ5/JSIvEZH1IvKMiOwWkTtEpLu0g4hcKSKDRSvAYyJyVvH9jIisEJH/Le67RkRmuI1TPX+75mjT4N8jIk8W5/FJn0swU0R+UpzXT0XkGKeN7Obp4usKjUxEji8e59ni+S3zGdd+bCMil4rI74tzXlUSQsVxNorIjSLyLNBbfP/9IvJbEdkjIuvs8xaRs0XkURHZKyL/CojHvE+wzftPRVfFG4FPABcWr/vD1deg+L1dLSJ/EJGnReSbIjK9+Jnn9yAirxCRTcV7608i8nmHa/IXwGPFl0Misr74/qtF5FfFc/uViLy66jv6jIhsBIaBP6s65reAo4F7iud1RfH9V4lllRoSkYdF5HVV1+v3xfvjcbHM2n8J/BvwV8XjDLl8rzX72j5z/P5EpHTPP1w89oUux7V/h0Ysi9rvisf7kohI9X7FbaeIyBdEZGfx3xdEZErxs9eJyA6xfqt/BL4uIkeIyL0isqt47HtFZE7VNf+n4j36vIj8WERm2j5/d/EeeUZEPiU2i4h4//aVqDHG6L9J9A94Ani9w/tPAn9f/Pt1wElYC7mTgT8BS4ufHQsYoMO270uBs4EpwCwsIfyF4mfzgO3AbNv+Lyn+fRnwEDCnuO/NwJ1u4zjMuRe4vWr7rwB54BTgIPCXLvveBjwPvLY49heBDbbPDfDS4t8PAhfbPntvaVvgsOL5vQ/oAF4O7AZOCPh9GOABYAaWoPmf0ljFccaAjxSPnQeWAv8P+Mvie1cD/1XcfibwHPAWIAdcXtz/Yod5Hw48BXwcS/M9HHhl9XW1zfNB23HeX5zDnwHTgLuBbwX5HoCfA+8q/j0NeJXLdan4/ovXZw/wruJ5X1R8faRtfk8CJxQ/z/nd+0AP8AzwN1j3+tnF17OK3+tzwLziti8ufaf26+gyd699Xb+/6vvO5dgVYxe3vxfoxrp/dgFvdNl3Jdbv7YXFc/wv4J9sv/kx4LNYv4c8cCTwZqCreH98F+ivuif+F/iL4vYPAn3Fz14G7AMWAZ3A54BC6frj8dvXf9H/a/kE9F/EX6i7IH8I+KTLPl8Abiz+XfGAddl+KTBQ/PulwNPA66sfrsBvgbNsr19c/LF3BBynl1pBPsf2+S+Bt7nsexvwHdvracA4MLf4OqggvxD4z6pj3wxcG/D7MPYHL/Bh4H7bOE9Wbf8j4AO21xksDfQY4N3AQ7bPBNiBsyC/qPQdeV1X23sP2o5zP/Bh22fzHL43x+8Ba5F3HTDT57pUfP9YAvyXVdv8HHivbX4rw9z7WC6lb1Vtsw54D5YwHsISZPmqbcrX0WUcr31dv7/q+87l2BVjF7dfZHu9Bljhsu//An9je70YeKL49+uAUWCqx9inAnuq7omrq+7dfy/+fQ02wYy1GBjlkCB3/e0H+d3ov3D/1LTePvQAzwKIyCtF5IGiSW0v8CEsbc8REXmhiHxHLPP5c8Dtpe2NMf8Pa/XdCzxd3G52cddjgO8XzZpDWD/ucSz/aL380fb3MJaAdmN76Q9jzD6s85/tvrkjxwCvLJ1D8TzeARwV4hjbbX//oWoO26u2PQb4om2sZ7EEdk9xP/s5GYf9S8zFerDXw+ziPO1z7qDye3P7Hj6ApcE9WjSPn1vnmKVxe2yv3c7VjWOAt1Z9d4uAFxtj9mMt0j4EPCUi94nI8UEO6rOv1/dXL0HveafvzX6v7TLGHCi9EJEuEbm5aB5/DmsR1i0i2QBjV9+Lw1jWjhLN+O0rLqggbwNE5HSsB0nJ9/ZtYC2WdjodyydY8rsZh0NcX3z/ZGPMC4B32rbHGPNtY8wirB+vwTLfgfVDf5Mxptv2b6oxZtBlnKiZW/pDRKZhmW93Omy3H0ujKGEX0tuBn1adwzRjzN/XMw8s86h9DtXXYTvwwarx8saY/8IyldvPSaqOXX2cl7h85nftd2J9l/Y5j2G5YDwxxvzOGHMRlnn3s8BdInKY334OY5bGHbQf3m/4qtfbsTRy+7U8zBjTV5zrOmPM2Vja4qNY7oIg43jt6/X9NRun783rXvs4lrXllcXf9WuL7zv64Kt4Cstsbu0gUjLVl/D67SsRo4J8EiMiLyhqRN/BMqU+UvzocOBZY8wBEXkF8HbbbruACSqDiQ7H8ocNiUgPsNw2xjwRObMYVHMAGMFaeYO1QPiMLdhnloic7zFO1PyNiCwSkU7gn4BfGGOctLotwAVFDeWlWFpliXuBvxCRd4lIrvjv9GJQVCk46QmfeSwvBhbNBT4KrPbY9t+Aq0TkhOLxp4vIW4uf3QecICIXiBV1fCnuloF7gaNE5LJiENThIvLK4md/Ao4V98jvO4HLReS44gLon4HVxpgxn/NERN4pIrOMMRNY5mc4dD948UOs6/x2EekoBoK9rHgeQfkTlffT7cB5IrJYRLIiMrUY9DVHRF4kIkuKi4yDWPf3uO04c4r3jdM5eu3r9f05zTFK7gSuLv7OZmKZv2/32P5wrN/rUDEQ7doQY92FdW1fXbxO11G5APD67SsRo4J8cnKPiDyPtSr+JFa+7vtsn38YWFnc5hosvxtQNpF9BthYNIu9CutH+nJgL5Ywudt2rClYKW67scxwL8SKigYrwGwt8OPiWA8Br/QYJ2q+jfVwehY4Dcsk7sSNWP69PwHfAO4ofWCMeR54A/A2LO3mjxwKGAJLI97oM48fAJuxFgz34ZEKaIz5fvH43ymaO38DvKn42W7grVjX+xngz93GLs77bOC84px/B5xR/Pi7xf8/IyK/dtj9a8C3sEytj2Mt0D7ic44l3ghsE5F9WN//2+zmXDeMlUd+LpaW+AxwBXBu8ZyDcj2WIBsSkX8sLtrOx7ofd2H9HpZjPfcyxbF2Yt0ff431uwBYj5UW90cRcRrfdV+v769IL/CN4hwDZz8E5NPAJmAr8Ajw6+J7bnwBK4htN9Zv89+DDmSM2YZ1T3wHSzt/HitW5mBxE9ffvhI9YrnZFEWpB7GqcH3UGPNbl88N8OfFWAJFmZQULTdDWPf6462eT7uhRQEUpQGMVuFS2hQROQ8rw0Gw0s8ewcocUGJGTeuKoihKPZyP5V7YieXmeZtRE29LUNO6oiiKoqQY1cgVRVEUJcWoIFcURVGUFJOKYLeZM2eaY489ttXTUBRFUZRY2Lx5825jzKwg26ZCkB977LFs2rSp1dNQFEVRlFgQkeqSxa6oaV1RFEVRUowKckVRFEVJMSrIFUVRFCXFqCBXFEVRlBSjglxRFEVRUowKckVRFEVJMSrIFUVRFCXFqCBXFEVRlBSjglxRFEVRUowKckVRFEVJMako0aooiqIoJfoHBlm17jF2Do0wuzvP8sXzWDq/p9XTahkqyBVFUZTU0D8wyFV3P8JIYRyAwaERrrr7EYC2FeZNM62LyNdE5GkR+Y3DZ/8oIkZEZjZrfEVRFGXysWrdY2UhXmKkMM6qdY+1aEatp5k+8tuAN1a/KSJzgbOBJ5s4tqIoijIJ2Tk0Eur9dqBpgtwY8zPgWYePbgSuAEyzxlYURVEmJ7O786HebwdijVoXkSXAoDHm4QDbXiIim0Rk065du2KYnaIoipJ0li+eRz6XrXgvn8uyfPG8Fs2o9cQW7CYiXcAngTcE2d4YcwtwC8CCBQtUe1cURVHKAW0atX6IOKPWXwIcBzwsIgBzgF+LyCuMMX+McR6KoihKilk6v6etBXc1sQlyY8wjwAtLr0XkCWCBMWZ3XHNQFEVRlMlGM9PP7gR+DswTkR0i8oFmjaUoiqIo7UrTNHJjzEU+nx/brLEVRVEUpV3QWuuKoiiKkmJUkCuKoihKilFBriiKoigpRgW5oiiKoqQYFeSKoiiKkmJUkCuKoihKilFBriiKoigpRgW5oiiKoqQYFeSKoiiKkmJUkCuKoihKilFBriiKoigpRgW5oiiKoqQYFeSKoiiKkmJUkCuKoihKimlaG1NFURRFiYv+gUFWrXuMnUMjzO7Os3zxPJbO72n1tGJBBbmiKIqSavoHBrnq7kcYKYwDMDg0wlV3PwLQFsJcTeuKoihKqlm17rGyEC8xUhhn1brHWjSjeFFBriiKoqSanUMjod6fbKggVxRFUVLN7O58qPcnGyrIFUVRlFSzfPE88rlsxXv5XJbli+e1aEbxosFuiqIoSqopBbRp1LqiKIqipJSl83vaRnBXo6Z1RVEURUkxKsgVRVEUJcWoIFcURVGUFKOCXFEURVFSjApyRVEURUkxKsgVRVEUJcWoIFcURVGUFKOCXFEURVFSjApyRVEURUkxWtlNURRFURzoHxhMRdlXFeSKoiiKUkX/wCBX3f1Iuc/54NAIV939CEDihLma1hVFURSlilXrHisL8RIjhXFWrXusRTNyRwW5oiiKolSxc2gk1PutRAW5oiiKolQxuzsf6v1WooJcURRFUapYvnge+Vy24r18LsvyxfNaNCN3NNhNURRFUaooBbRp1LqiKIqipJSl83sSKbirUdO6oiiKoqQYFeSKoiiKkmJUkCuKoihKilFBriiKoigpRgW5oiiKoqSYpglyEfmaiDwtIr+xvbdKRB4Vka0i8n0R6W7W+IqiKIrSDjRTI78NeGPVez8BTjTGnAz8D3BVE8dXFEVRWkz/wCAL+9Zz3Ir7WNi3nv6BwVZPadLRtDxyY8zPROTYqvd+bHv5EPCWZo2vKIqitJY0dRBLS8tSJ1rpI38/8CO3D0XkEhHZJCKbdu3aFeO0FEVRlChISwex0oJjcGgEw6EFR1qsBy0R5CLySWAMuMNtG2PMLcaYBcaYBbNmzYpvcoqiKEoktLqDWFCzfloWHG7EXqJVRN4DnAucZYwxcY+vKIqixMPs7jyDDkI7jg5iYcz6rV5wNEqsGrmIvBG4ElhijBmOc2xFURQlXlrZQSyMlp2mlqVONDP97E7g58A8EdkhIh8A/hU4HPiJiGwRkX9r1viKoihKa1k6v4frLziJnu48AvR057n+gpNiCSILo2WnqWWpE82MWr/I4e2vNms8RVEUJXm0qoNYGLN+mlqWOqFtTBVFUZRU45Q6tnzxvAofOXhr2WlpWeqElmhVFEVRUkn/wCDzV/6Yy1ZvqUkdA1pm1o8b1cgVRVFSTpqLmdRLdVS6nVJQ28YVZ0766wAqyBVFUVJNmqqnRYlTVLqdtKSORYGa1hVFUVJM2ouZ1IufoE5L6lgUqCBXFEVJMWkvZlIvXoI6TaljUaCCXFEUJcWktZhJo13RnHK/AbrzuUkb1OaGCnJFUZQUk8ZiJlE0KSkVm+nO58rvHdGVo3fJCW0lxEEFuaIoSqppZfW0eonSr39wbKL8957hQqq6lkWFRq0riqKknLQVM3Hz3ztVYvPCa0GQpuvRKKqRK4qiKLEy3WYOryaMNt2KQL9GffvNQDVyRVEUJVZE3D8Lo003s02qU5EdIJE5+6qRK4qiKLEyNFxw/SyMNt2sQD+3YLzr7tmWyJx91cgVRVGUWHHTpEufBaVZXcvcfO9uleRanbOvglxRFEWpoNm125cvnsfyux6mMG4q3s9lJLQ23YxAv7CCudU5+2paVxRFUcpEkePtx9L5Pax6yykc0XUo6K07n2PVW09JRLS5m2DuzucSmbMvxhj/rVrMggULzKZNm1o9DUVRlEnPwr71jmbvnu48G1ec2YIZxY9TZ7V8Lsv1F5wERG/Kd0JENhtjFgTZVk3riqIoSpmwKV3NMsPH3Zq1erw3n9bDA4/uchw/CVYDOyrIFUVRlDJBU7r6Bwa57p5t7LFFoEeVjhV3a1an8b63ebBCA7989RZWrXvMEujZjXD/Sti7A6bPgbOugZOXRT6voKhpXVEURSnjZVYuCVGnbew0aoZ3M++Xjh21du423hFdOQ4UJjh7/Kdc0bGG2bKbPWYah8sBOmXs0Ia5PJx3U6TCXE3riqIoCSVuk3FYgqR0OaVn2Wk0Hctr/2Zo527j7RkusCSzgc/lbikL7iNlX+2GhRFLQ2+RVq6CXFEUJSbiNhnXi19Kl5+gbjQdyyvPHKKvp+403pLMBq7oWEOP7PasRFdm745I5lIPmn6mKIoSE410/UpSjW8vQR1FOpZbr3E7URZhqR5vSWYDfblbmZMJKMTB8pW3CNXIFUVRmkzJnO6mZfoJpaCafDPN9vZjd3flyGWEwkRljFV3Ppp+4HbzfhQV4MDj2mxdw9IHV3J+dgd/ys7k+tG3srxjDV0yGvzgubwV8NYiVJAriqI0Eb/AMPAXSkHadTbTbF997D3DBXJZoTufY+9IoSm+/pJ53y34LozWX32M0577Caf3/x3mB7sRBDAIcBS7+OJhX8cU/LX9cWPteaDrKLre1Dr/OKggVxRFaSp+gWFBhFKQ3O5m9uZ2OnZh3HDYlA62XPuGho7tRxT11Evzv67ja7wzez8ZjM1kXpW5VRjBSAYxE67HO2iyLC98kJ9NOYPeN5zA0pNbG9+gglxRFKWJeJnNg6ZSBcntDlPIJawJPq6+327zaqie+tY1rB2+nBlTrGjzID7vjJlgLDuVjvED5fcmAAzsNDO5YWwZaycWwUghEcGKGuymKIrSRNzM5qVc6yACIEi7TrdxnAq5hK2lHvTYjdCMGu+/Wnszo3d/iCMz+xAJJsQBmD6XjvP/BabPBQSmzyVzwVd4Tf77LBq9yRLiRZLQxlQFuaIoShOJomf20vk9XH/BSfR05xGsRYC9QEuYceqJnG9W3+9G5+VF/8AgszffQCfubg1HSoFrJy+Dy38DvUPW/09eFptlIixqWlcURWkiUfXM9jMvBx2nHmEU5hzqjZyPSkjaMwR+P2V3oH0mjKWty/S5nuVWg5avjRsV5IqiKE2mGT2zwVlo+pVGrUcYBRXOjUTO+80ryByqx99pZjJH3IW5MbCfKXyi8AHumVjE473neM5x+eJ5DUfQNwM1rSuKoqSQen3KYc3kYcZxM49fd8823/Pxmlf1HC7Z9yXO7T8R0zsdrpsB937McfwbxpZx0NQWljEGxg18c/z1nHjw66ydWBRIqw7i4mgFqpEriqKkkHrTzcKa+sOM41WzvH9gsO55LexbX57DdR1f493Z/zgUuGbGYdNXi+OfVXHMtROLoADXdnyTGcUa6cMd07m28C7uGn11ebswWnWzrCuNoIJcURQlAuJuhtKITzmMMAozjleN9CD57BXz2roG7r8UfrCD1RNHckPGSvl6R3a9c/T55tuY3X1uzfhrJxaxdnRRRarfooFBfp7gxjVhUUGuKIrSIK1ohhJX4FWYcZYvnsdlq7c4HidU0NrWNXDPpVZXMWBOZjd9uVuhAFlcCrWYcVcfdrX5O4ladSOoj1xRFKVBok6dCkI9vu56mq6EGWfp/B668znH4wRaYGxdAzeeCHf/XVmIl+iSUa7oWMO4m9iSbGgfdpIa0TSCauSKokx6mm32bkV+cdiUMDeLgd8xwvrUe5ecEC6y+96PwebbLF+3D7PlGb41flaljxyryKqc9t7yfO3151ete4zLV2+pmXdaWsoGQQW5oiiTmjge2K3KLw5qInazGPSu3cbBsQnfaxPGFB1K8N/7sXKgWhB2miO5duz9ALwju54sE4yTYbU5i8N6Ps5S27Z+33sza9PHjRhj/LdqMQsWLDCbNm1q9TQURUkhC/vWOwrZUonUKHDr0JWE1CSA41bcV90axJMor42dX629mbm/XsULzS6ellm8kF2B/bvDppMVhYsryqNW01MV6e71vbtdEwEe76vMJ487kBFARDYbYxYE2VY1ckVRIqcVDz63MeMwe0dVva1ZeEWTOxHZtdm6Bu5fCXt3cDA3nVNGn6dTxkGslqHGYElOD4yBQXujEg8Gh0ZY/t2HPc+h9H5QK0oaTPAqyBVFiZRWPPi8xozL7B13JHSYxZJbNPfUXIY9w4Wa7Ru+NlvXwI+uhJFny29NKQzVCG2/JiZj2an844EP0D++MPDQhQlD79ptvt970CptaTDBa9S6oiiREiaCO6qoYa8xnaKuAfYfHEttlHLYqm5u0dzXnndCQ81Qqr+/X629GT57nBV1bhPiXrh6d4sdyKadflGg49gZGin4RtsHjXBPaqMUO6qRK4oSKUEffFFq7l5jlo513T3bKrTPoQC9pK/uf4Q7f7GdcWPIinDRK+fy6aUnhZqbnahcDvVoiV4Wg3rmVP39nfbcTzhx860go6HOZVimcpgUrKh1ycJp74VzP1/+/IEfrnfcLyvCuEeMVxB3RxArSlIbpdhpmiAXka8B5wJPG2NOLL43A1gNHAs8ASwzxuxp1hwURYmfoA++KE2WfmOWopSrzche413d/wi3P/Rk+fW4MeXX9QjzuBYuYakWeCXLid+cttx3Cz+R25k9ZTc7zUy65AD5sELcdPLfp63k9CUfdN3G7ZwmjOGIrpyja+CILiuXPQp3R1Ibpdhppmn9NuCNVe+tAO43xvw5cH/xtaIoCaMRk3fQAiJRCqMgY4Yd785fbA/1vh9RFo1x0wbr0RL7BwZZftfDFWb65Xc97P2d3/sxril8gTmZ3WTEqrx2BPsCjTduYMIIOyZm8k/yIU8hDt7neu15J5DLVjrac1nh2vNOCDSXICS1UYqdpmnkxpificixVW+fD7yu+Pc3gAeBK5s1B0VRwtOo5hg0gjtKk2WQMcOO52a29TLnehH1wsVNSwxrvr/unm0UxivPqTBuuO6ebc77bV0Dm75GJmTgGsBBk2V54YOsnVhkpeed72/Z8DpX+/c+ODRCVoTCuAlsVQhK0ku6xu0jf5Ex5ikAY8xTIvLCmMdXFMWHKEzeQR58Tg9osBYOC/vWh/Yf+40Z1kTq5oMNIrCciGPhAoRehDmZpgFec+ABuPFjsHcHTJ8DZ10DJy+z0slcstKNcbo+AhiG8y/mhsKF3HPwFRX53vWea+n90v+TniLWTBIb7CYilwCXABx99NEtno2itA9xRekund/Dpj88yx0PPVkjFprxIA6b633RK+dW+MhLCPi25HSiHl+rl3bttHCxt/ssEWYRtiSzgSs61tAju63vZG/xg73brSYmYAl2F0Y7pzOl6wW1wh/oAnqL/8Lit0hLQ4pYM4lbkP9JRF5c1MZfDDzttqEx5hbgFrAqu8U1QUVpd+KM0n3g0V2uFcf8HsT1RIBXC4RSLIDTMT699CTu3ryD4UJlt60JE6wlp9PY4LyQcIqOX3DMjNBaZj2LsO58jqGRAt/MfYbXZLaVNeoaxbowYmnj0+dYgr0GYcp5nysL7igI+h2nIUWsmcQtyNcC7wH6iv//QczjK4riQ5xRun4PWrfPo4gAD3KMkYJzy8x6BYSTZukWHX/35h014/stbupZhH1l/uP85eZPcRgH/d0Ge3fABbdUtBi1EFjwflchXs+iK8x37HbeBupy06QN36h1sXiniFxTfH20iLwiwH53Aj8H5onIDhH5AJYAP1tEfgecXXytKEqCiDNK10/Ld/s8igjwIMeIMjrcDbco+GpLQAmvRUTglqOldqG90zn911cyTQIIcbC08ZOXwXk3wfS5gFj/v+CWitxvO2GL15QI8x27Ff0hxHhuc09Dm9MgGvmXgQngTGAl8DzwPeB0r52MMW7leM4KM0FFUZpDWP9rM3ALeANvK0CYojNu5xjkGHFYJ8JGwXstIgLFAWxdU6VRBxw/l7d83mAJ84Am9Hr912HM5dXR69XU4y9PQ431EkEE+SuNMS8XkQEAY8weEels8rwURWkiSXlIOaUPjRvjG9UcxITsdI6Xr97CZau30NOdZ3rRN+x1jGY1Q7EvMNwQgakd2dCLCN9F2P0rq8ziAZg+tyJwLQz1+q/DuglK5+3W1SysO8RtAeKaltdCggjygohkKS7bRGQWloauKEpKSVKUbz3afxBN2ekcSw/4waERclkhlxEKE8b1GPXOzwunlqdOvOOVR7PgmBnRd1TziDqv4bi/hvesbWi4eoMn67WGRBWs6Sb49wwX6spaaCZBBPlNwPeBF4rIZ4C3AFc3dVaKojSVtEf5BtGU/c6lMG6V+Ozq7Ii19ajTAsNOdU33yOfjGnVuQzJw2vtc/d5hqFcg12sNicod4tX6NWlpbb6C3Bhzh4hsxvJtC7DUGPPbps9MUZSmkYZGEH74acpBenAPDRcYuOYNUU+tBrsp3c0jLcDjfef47l8h0Gz9vqvztl056xrnqHNMQyZ0NxpxT9RjDYnKHbJ88TwuW73F8bOkLXg9BbmIZICtxaYnj8YzJUVRms3yxfNY/t2HK8zKuYwkqhFEo3gF0pWIY+ES1JTuNper+x+pKJpTimfo2X4vpz9y7SGBbC/a4iWIS5+FXQA0QNwlTqMYb+n8HnrXbvONo0gCnoLcGDMhIg+LyNHGmNoSR4qzcc9lAAAgAElEQVSipJfqdCOP9KOo2m/GSXUgXVHnLNOM3Hin6+RnSveaS//AoGPlu5HCOHN/vQqo0gxLRVv8hHKIqPNGSON9Y6d3yQmJ73wGIMYn9UFE1mOlmv0S2F963xizpLlTO8SCBQvMpk2b4hpOUSY9C/vWO5qde7rzbFxxZsV7ThplPpdtWQeoeoVDs4WK23XyEuICnnNx+54Afj/l7TWNS0pH7T9/W8sFaNLum3pp1WJERDYbYxYE2TZIsNt1Dc5HUZSEESbYLUkR7o2kzTXbvOt2ndyarzgtmirYuobVw1eV+33fMLaMtROLyh8/LbM4il01uw3nj0pEamGS7ptGSHrnMwhQ2c0Y81Onf3FMTlGSRloqPfkRpmJZkiLco+zpHTVu12PcmGDV1uwUi7bY+3335W5lSWZDeZMvchFj2amV++Xy3FC4MJJr1Oi97nY9St3t0vrbSSJBSrQ+LyLPFf8dEJFxEXkujskpSpKot9RkEglcypPmlSmtR1AkaVFRjdv1KJW4DVXy1qFoS5eMckXHmvLrOw+8ihWFixnOv5hyqdTzbuIb+5wraIe5RlHc6173RzuUTY2TIBr54caYFxT/TQXeDPxr86emKMkiydpgWMLUUw8j9INSj6DoHxgk41IQPAlRxF7Xaen8HjauOJPH+85h44ozD6WO3Xgi9HZb/99qCen+gUEmXIq2zJZnKl7fNfpqzjZfht4huPw3cPIy12tRaiASRPC53esfX/NwYMHpVf+8dLx6rASTZTEdJaG7nxlj+kVkRTMmoyhJJsnaYD0E9f1FkZdbHTA0PDoWyn9aeoA7+ZqTEkUc6jpV1zsvpo796ok9XPWrY1ggRzIns7tmt53myNr3qu4/r7S7oP5yLzdBmJgEcK9/7jWOG35+97RHydeLryAXkQtsLzPAAgJX2VeUycNkKKJSL40E/DgFqLnh9mB3S+HKijQlCrpegeB6naoLt4zur613Xhhh7q9XMVL4IjdkltGXu5UuGS1/PGw6uWGsNmWs+v6LooGIVzGdMAFrpevhFn0fVdnUwaER5q/8MfsOjJVrIyS5yUnU+JrWgfNs/xZjdT87v5mTUpQk0gwTczsQJI+6hNuD3e0BPmFMU4R4lObbX629mZG7/79iWVRj/X/kWcdtX2gsLXztxCJWFC5mx8RMJoywY2ImKwoXV0Stg/v9VzLlu5UG8NOE/cziYTXpqH47XoJ/z3ChosARpNf1FZYgpvVbjTEb7W+IyELg6eZMSVGSSbM6YU12gj70vR7scVpDIkmbKmrgZu8O5huhQ4L1mXpaZpb/XjuxiLWjlYK7O5/jsCnBa8PXe91Kx/z4mocd3Rlhr3uUZVODVMmzk1bXVxiCCPJ/AV4e4D1FmfSkIac0aQSpee7XtjSOvuAl6o2FKJnjFzz3E/o6v0qegwjQIcE8kWPZqXxx/CLXz/O5LL1LTgh1/zVy3UrjRHXdoyqbCt5+92rawfXlKshF5K+AVwOzRORjto9eALjbXBRFUWz4aVEC3oVRsB7gm/7wLHf+YjvjxpAV4c2nNWdRFUqL3boGfnQlZuRZzjfwWqZhOiDPQf+B8jOg8zDYu4Ph/FFcs//N3DX6qopNMgITpnahE9SH36gmnEQrlJ/f3U67uL68NPJOYFpxm8Nt7z+H1cpUURTFlyjMtP0Dg3xv82B5/3Fj+N7mQRYcMyNyoeKrxRaFt93PLcX/zGBfoEjgEaaQf9Nny/XOz+5bz+BorVB68XT/krl+QV2NasJJtUI5fU+5rHBYZwd7RwqJWHTEhasgL1Zv+6mI3GaM+UOMc1IUZZLRqJk2znKfnlroN5bA496FLV1S3RkzGTIYnuJIdp52BafbmpaktWRuiVakfSXRWtAqgvjIh0VkFXACUK4HaIzxtoUpipIoWp1j28iDN+4c/gotdOsauP9S+MH2wPsbUynQh01nOeo8lxVWzT2F023bhzHnJ62eQSP17xslqdaCuAkiyO8AVgPnAh8C3gMOlfoVRUksrXzY2qn3wduyHP7qwi0B2cM0hiemMlueYac5sqLhSWHc1GjPYYLSor4WjS7wtEhL6wkiyI80xnxVRD5qM7dr0xRFSRFJNMeGIZao9eqiLWdd41jz3I9R00Fv4d01Od92StqzXch1d+WY0pHx9e9GeS2iWOB5WQiSsoCc7AQR5IXi/58SkXOAncCc5k1JUZSoSYI5thHNrKn+UIfgNfZuh/4Pw0TBfb8qDLDHTPMV4mBpz9VCbs9wgXwuy40Xnup5XlFeiygWeF4WgrQvINNCEEH+aRGZDnwcK3/8BcDlTZ2VokxiWmFqbHV52UY0s+rr5SfoQuFlOp8oMG4g61YerUS2E87/En/27cMCRayXtOdGhFxUvuEoFnheFoLLV29p+PiKP76C3Bhzb/HPvcAZzZ2OokxuWmVqjLOgihNhhJZdcE/P59g/OkZhvL762f0Dg1x3zzb2DFua9UVTH+JT+e/SNfJH95rnNjJYgWr2mudlJAunvRfO/TwAs3/onNfsVo0tCUIuigWel4XArXBLOxRpiZMgTVP+Avi/wIuMMSeKyMnAEmPMp5s+O0WZZLTK1NjqVJ2gml/1QmdopNa0HfR69Q8Msvyuh8uLgCWZDXzK3ErXSFEo7w0Whb6icDFXdKxhduYZMiXf+cm1zUvcFktu1dim53OO5xenkItqgedmIWj1ArJdCGJa/wqwHLgZwBizVUS+DaggV5SQtNJX3cpUnaCaX9AGK0Gu16p1j/Em859c0bmG2bKbCTKBa56X2MO0cs1zAR7vPcd12zCLpf6BQfaPjtW8n8tI3UKuHpeN05zPOH4Wq9Y9xuWrtzS84Gv1ArJdCCLIu4wxv5TKKge1d6CiKL602lfdKoJqZkEXNEGu14LnfsL1tlagGcIJ8VL0udOYbkKzWnCVOm9VC65V6x4rWwrsTJvaUZeQa8RlY593M1w/muvdfIII8t0i8hKKPchF5C3AU02dlaLETFwBaFGYGtOYlxtUMwvSYMX1elWlj13XOUQXDr7tmgOWap5vB8lizDg7zUw+WziU+20f00vYAYEEoduCZWg4eJS8nahcNhplnk6CCPJ/AG4BjheRQeBx4B1NnZWixEicAWiNmhrTnJcbRDNzrJ+dEaZN7WBo2CO/ujr6fO92pgeZVC4PtprnYNVN/9XAIJvXPYY4fEdewq70t9Nn9jlHbZmJymWThDRFJTxe3c8+aoz5IvBiY8zrReQwIGOMeT6+6SlK84lbC2nE1DjZNabAC53q4i0O0eduWWMTkiFjzKGiLw6Ba17fUT3CrvqzqIPAoloYtKvrJ+1kPD57X/H//wJgjNmvQlyZjKRJC0nKXPsHBlnYt57jVtzHwr719A8M1rWNE0vn97BxxZnceOGpAFy+ekvl/iXte+92wFj/txdzsVHTbC2XJ/O3N0PvEFz+G0ch7oebUJvdnff8zM7S+T1cf8FJ9HTnEaw2pddfcFLdi7Hli+eRz1V2l65nYRDVcZR48TKt/1ZEnsDqR77V9r4AxhhzclNnpigxkQYtpOQXdys4Eudcg5j3G3UBeO7/YPCyqeWa5z6pY2Hw06aDatpRBoFFFR2uUebpxKuN6UUichSwDlgS35QUpflUFx3JZaUiijhJWki1UKsm7rkGMe836gJYte4xzh7/aTl1bKeZyQ1jy1i1rpOlB3Y47uPUcaxULtUvdaxE6b4YHBohK8K4MfRUCbMgwq4VgjCqhYFGmacPz2A3Y8wfgVNimouixIJT0ZFcRjiiK+cdUNUivHKrq4VMVHhFxgcx7zfqAqhOHZsju+nL3cpVzwGz5jgWc/HqOBbEYlF9X4wb92pyXsKuOp0rqpxsRXEjSNS6okwqnARjYcLQ1dnBwDVvaNGs3HETfgJsXHFm5OP5mcWDuCLctpmez7kPbAtg+z+dQkdV3neXjHJV53fhrH+uqY9u176rCWqx8Fow1RNQmOYMAy+qF3lnHD+LBx7dpab4FuIV7KYok5KkBIwFJWgAVVT4pVedcfwsx/3s7y9fPI9cpjZufP/oWGXQ2o0nQm83fPY4q9tYMYCtWoiXeBG7LR/3eTfB9LmAwPS53JD7sGvHsdLc/YLt/L7/sPeH33VMI6XFyeDQCAZrcXL7Q09WvL7q7kcCBzYq0aCCXGk74haMjRJ3JLHfQueBR3c5fm5/f+n8HqZNrTT4Lcls4IHsR1jygxNqBDcjzwZqGSrTix2UT15G/+vWsXDq3Rz3p8/SP76QnEebsiACxu/7D3t/pG3BGIQgJXTTvlhJI1555PeAe1c+Y4wGwCmpJG2NHJoRSezlA/cznbtVXqsWUPYqZdd1fI13Zf+DspLuki7mSS5vRZ3jH+eQKQaq2fEzjy9fPI/l332YwkTtY6+eGuhpyIYIS9BFSJoXK2nEy0f+ueL/LwCOAm4vvr4IeKKJc1KUpmIXjKXoZLsWkUT/XpSRxH6+W6+FTv/AoJV/6nDcagH1ufw3OX/ix2SLZnLx6+vthGTBTNQUb/GLczhuxX2Oh/MSMEvn91S0PLVTTw30tC0YgxCkhG5pOyU+vNLPfgogIv9kjHmt7aN7RORnTZ+ZojSR0kO51cFIraib7pca5mUBWNi33lGICxwSUFvXwD2XcYHZX5/wLs3JdPKbl3+a05d8sOYzP7N1vdqwW63zemqgR2FJSVpdfafFSTVpX6ykkSBR67NE5M+MMb8HEJHjAOdoF0VJEX7BSM1+gLYqqjmI79bNAuC2b2/H11i6dj384FCQWlgZPmo6eN5M5QjZX04f2/zff85GByeen6CuVxtOkjk8iVHvbm1PNWq9tQQR5JcDD4rI74uvjwVql8iK4kFUmkWUGoqbUCo9MJv1ALUXHamm2o/bDI2sEWFl33dJZgNXdKyhR3bjam/3Iz8DRvawY6Iy77uEuHxHfoLaTRsGWNi33vV6RmkOb1QQJ7WuvhaMSR6+gtwY8+8i8ufA8cW3HjXGHGzutJTJRFSaRdQaiptAK/nM7fg9QIMKXL8qbXBogdEsjawRYbV88Tw2fP/LfEJu4wj21W06nwAeP+ZtvOR9NwNwYd/6UIuLIGbraoET5HpGGVjYqCCuN+o9bnN80sz/7UjQgjCnYWniHcApIoIx5pv1DioilwMXY63hHwHeZ4w5UO/xlGSTpF7J9odOd1eOXEYqopTzuayrkHV7gIYRuEHSdzIiHLfivroir4PQiLBamt3Iublb6RgP/3M1xvrBl8qt/uT3f831A4O+AXZe59GM+ycqjbPR9LN6LCdxm+OTaP5vR3wFuYh8C3gJsAUo/QoMUJcgF5Ee4FLgZcaYERFZA7wNuK2e4ynJxcuEDM3pleylHVQ/dPYMF8hlhe58jr0jh0qzus3Z7QEaZoER5JxLwrtaiIc5hh91C6v7V9YlxMcRbh8/i2vH3n/ozYlgAXZREXded6P+9noWN3Gb45Nq/m83gmjkC7CEbj0eMK9x8yJSALqAnREeW0kAQUzIUfdK9tMOHFOWxg2HTelgy7WVpVn9HqD2BYPbD8NJQHil72QdNHAnWpras9e5YYkr+Rnwps/y0m8f5nidggTYRUXcgWyN+tvrWdzEvViZjEVv0kiQym6/wcojjwRjzCBWjvqTwFPAXmPMj6M6vtI67P2nP77m4chTVJwqnAEMF8t++kWhB33o+PWKri5T6YaTgHCr0vaFC09lIoAQb2pqj71k6o0nWq+rKVVW8yN3GFzwFbjycTh5Wd3V9IL2NA+yXdwV8qLoOV7qzf543zlla5HXObpdz+4ujxr3DZC2KomTlSAa+Uzgv0Xkl0A5yK3eym4icgRwPnAcMAR8V0TeaYy5vWq7S4BLAI4++uh6hlJixK1zlBP1duwqbd+7dhtDI4fyevcMF7hs9RbX/erJLfbSDoP4ub16UJeOUa1luZn0syJMGFOxbeQBRvd+DDZ9jXLo+d7tVmMSqOzffdY1NQ1LrBO2NG+3Xt/1aKdB/a9Btwur4UZxjaOyMgQ9x+WL57H8rocrWvIC7DtgLXajtnhMxqI3aUT8LOYi8tdO75cKxoQeUOStwBuNMR8ovn438CpjzIfd9lmwYIHZtGlTPcMpMbHQJeq4mp7ufMMdu4KOVT2mk7k/n8uG1pKOW3GfpyZe70Il6Pz6BwZrSonmMsKqt54SfMyta+BHV/qXSp0+Fy7/Te2+xS5l1RXX/M4vjGB0+56r76Gg24UhqnslKsKc46nX/bhioeu1bRRo1HpzEJHNxpgFQbYNkn72UxF5EXB68a1fGmOebmB+TwKvEpEuYAQ4C1ApnXKC+MSiWqmH9b/tP1ipjTT60PHyczfysHSyOEzN1Xq/etduq6kHXpgw9K7dFuxctq6BH/wDjI/6b2vziR96YB/G7O6bWH5+uGsXVjsN6gpphp82aUFcYc5xr4MQ9zpGo2heeesJErW+DFgFPIhV9uFfRGS5MeauegY0xvxCRO4Cfg2MAQPALfUcS0kOXjnZ1WbhZo3lxtBIocIM2egcli+e52rKj+JheXDsUHW0PcOFGhOqk7bl9X6t9h2iekvRJ96KNKOgrpBmBLElLYgrzDkmqTqdEg9Bgt0+CZxujHmPMebdwCuATzUyqDHmWmPM8caYE40x79ICM+nHLZDo/yw7hcf7zmHjijMje+C7Bb2VyDpUKYmyteLS+T0c4RI81OjD0k0TvGz1Fs9gL0e2rrHahd79d1Um9KAJKAJnXUP/wKBj8GKz21UGDU5rRhBb0oK4wpxj3EF9SusJEuyWqTKlP4P2MVeqiCMPuHosp05V9RR0qYdrzzshVJBPUD+i1xxLWnBXLsNwYaLm84rFxdY1zkFpgRFY8H76xxdy1d2PNDWn3Y2g91Qz7r2og7ga9SOHOcc4f4tKMggS7LYKOBm4s/jWhcAjxpgrmjy3Mhrsprjh9IB0i/6OOtinkbKsboFTQQL5jujKse/gWEVkci4rrHqLLdjtxhOtyPN6mD63HMDmN59mBVAlgSj7AyQpcE5JB1EHuy0XkQuARVjOtVuMMd9vcI5Ki5kskaZuPu84UmKC+tvDBE4FaRM5NFzgxgtPZct9t3Dx6O3MzjzDgfxRdGVXAlb0uNm7w7/7mGSsXt/gmj7mpXG3wlwb530bVRBX0gLnlMlHkGC344AfGmPuLr7Oi8ixxpgnmj05pTlM9vrISTMthg2cmprLeAry7q4cS7MbWSo3Q8Y6RtfIUxV5339iJkexy31SmRws/bJv2phXEGPcGmVa71uvLnvHrbiv5fenkn6C+Lq/i9WsqMR48T0lpfhVQEsqQat8QWVFrCgD7eohaOBUSVBV+/2r2TtSYPhH19T6vwsjVn43cP3oWxk2nRUfG2P9Iz8jkBAH7yDGuK9pWu9brwA5w6EFSahARkWxEUSQdxhjygmnxb87PbZXEk7SUmuCUF0WNWkPP69FRtAoYidBtSSzgQ2dl/L7KW9nQ+elLMlsYMLA1JE/Ok+kmPe96QVns6JwMTsmZjJhhB0TM/lo4cMsyn+/XDY1CFGUGY2KNN634J9lAelYkCjJJUjU+i4RWWKMWQsgIucDu5s7LaWZpDHPNMl+RieT7+Wrt3DZ6i3lKm/XX3CSr6m/WiAtyWygL3crXWKto+fIbvpyt0IBdk4cyZyMw8+wmPdt+dpHWTu6qPxRPpfl+oA+7STGUKTxvoVaV0+YJjuKEoQggvxDwB0i8iUsS9AO4N1NndUkp9UPybTUR663w1jcOC0ySvMtWQ6uv+Ak3+ju90z7JR8ZvZUZsg+ACYSsVJ55l4xyRccabhhbxk2Hfb3SvJ7LW9HmOKfoTekIljWaVF+0333b6t+VF/bAObdMgKQvSJTk4vvLNsb8rzHmVcBfAicYY15tjPl/zZ/a5CQJJuIkmUvdaKTDWNz4LSY8zaZb18BnZkPvdK4d+wJHZvYhAiLUCPESs+UZfjblDDjvJitVDLH+f95NNSbzA7Z881KFO797Lam+aK/7Ngm/q6BowRYlaoJErb8I+GdgtjHmTSLyMuCvjDFfbfrsJiFJMREnvT5yIx3G4iZIyVhHYb91Ddz9QUqxpL7pYkWe4kh6l5wAJ/d4+rrrvdda5YsOolG73bdJ+V0FIWlZFUr6CWJavw34OlapVoD/AVYDKsjrIK0BO3HjdT0EEvXwC5L7Pbs7X9s1bHQ/lQkh/owwhZ2nXRHovOu916bnc45126fnm9PTun9gsKZKX1hzftBzTYr5PekLaSVdBOpHboxZIyJXARhjxkTEW1VSXAkTsJOUh04rcLtOSawkZtewBodGalqSvKXzv/jM+O1w99ChN8NUXZOsVbhl+hzyZ13D6UUt3O/+qDc4zKFUvef7jeBU9axEGI06yLkm1fevKI0SJPplv4gcSfHZJCKvAvY2dVaTmKD+sTT5/KKklMZVEoh24jSlh8lZh0N560/0ncONF55a9uO+d9ov6cvdypTCkOf+rmQ74W//DXqHrL7gNiHud3/U64sdcsljd3u/EfxcKEEtVUHONam+f0VplCAa+ceAtcBLRGQjMAt4S1NnNYkJ6h9Lk88vLG6aZLXGZDjUcLMnRovE1f2PcMdDT9ZEnkMwza3CbHrjlbD3gOu2xtRquqX2B3uYxozzb3T0gwe5P+r1xcaZ5uUnqIOOGeRc1a2lTFaC1Fr/tYj8NTAP67n6mDEm+qV5iglrAg/iH5usDx0v86ZbGlec5vT+gcEKIV7CcxFV7fsuNhzpHxhkyd4dnmavPUwjN1FgmlidfCcQbh8/i2vH3m+d98nO5x30/qjHFxtneqJXoGDYMf3OtZEFSju7uZTk4yrIReR0YLsx5o9Fv/hpwJuBP4hIrzHmWbd924lm+d3S4EuvZ1wvTbKRxUtU12DVusdc090qvo+y8K7yde/dDvdcyq+e2MNVvzqGBeJSuAUYNp30Ft7N2olFNZ/5CbFmas1xRlW7BQp253P0Ljkh0jHrXaCob11JOl4a+c3A6wFE5LVAH/AR4FTgFtS8DjTPBB70odPsh0xQM3jQcb0aSHS7REv7Cacor4FftPyv1t7M6b/tgxGPdWxhhNmbb2CkcBM3ZJZVVGcDy3T+rJnGdWOHhLgIzJ6eDyw4m601xxVV3Yo+9lEuPlWQK0nAS5BnbVr3hVjtS78HfE9EtjR/aumgWSbwJPjSw5rBg4zrZUrdPzpGLiMUJg7pxEGEU5TXwGl+SzIbuKJjDbNlN/LrYMd5Mc8AWIK6QHH/Z9hpjuSGsWU1WrgxhHIfTKZc5DhTseoZa7K6uZTJg6cgF5EOY8wYcBZwScD92opmmzhb6UvvXbstcjP4GcfPcvRBAxTGDUd05ejq7AglnKK8BtWa7nUdX+Nd2f8gEzL1aqc5svz32olF5ZrnWRHGTe3Zd+dzLOxbH+q8NRc5HtJa411pH7wE8p3AT0VkNzAC/CeAiLwUTT8r0+q65X4PmXp9x/0Dg45mbqB8rLAPt/6BQb63edCz5OrQcIGBa97gO7/qMaN60C7NbuQN065h6vBTDDGNbvaFFuITBm4Yq400z+eyvPm0Hr63ebDifsllhP2jY+XrrT7YZNHq37ii+OEaUGuM+QzwcazKbouMKasRGSxfuULr65Z75c82kovulVtbWhCEzVEOUna1HuEbWe3qrWvgnkvpGnmKjMAMqU+If2v89WXTeVak4r749NKTau6XaVM7KIxXLm80vzk5tPo3rih+eJrIjTEPObz3P82bTvpoZsR40NrT4OwrXdi3vm7fsZdZ2j6PMOfuZ+quV8txmssZx89i1brHuHz1luDfy/0rK7uJBcBgCW8BdpqZFf7vfC7r+MCvNokft+I+x2OHdQ1oilTzUDeGkmTU190AzYwYD3Nst4dMI75jN3O1CFy+egur1j3G8sXzQgVoeQW6hSn44iawSvsGunZOud97dwQ+FwvhblnMxw/UdvXNigTW2tyuS3dXcL/5ZEuR0kWJogRHjEPgTdJYsGCB2bRpU6unUYNbX+GgBUy8HlaNHrvRY3jVwC7hpnGGOWaUx4BD9c6deO+0X9Kb+6Zz6lguDx1577QyDlVdGzQzmfOW6znu24c5+vwFeLzvnPKcvYSS0znlsgKGmgh+t2vl9l1353NsuTZczEHUeKUwBklthPD3iaKkHRHZbIxZEGRb1cgboNECJl4aVBSR2I0E6VSbqzMO0dZhU7yiSJlyi6TvXbuNg2MTNZ9VpI4VgDGXAxdGLEGey1eY10dNB8+bqRwh+ytSx0pV12b/0FmA2oMN/TRlp+uy/+BYTbCh1/V2uy+GRgqcet2PIy+uEhS389/0h2crgv6iSG1UlHZFBXkDNBIt7fewiiISu1HBaTdXR+XHbcTX6BVJb3+/JLx7xKqoFrhr18geuOAWuH8lE3t3sHPCOefbvhjyWywFFUqN+s293BZDI4WaxUNcpmu387/zF9tdF4aat60o4VBB3gCNaLx+D6uoUl6iCtJJQi5tkCjuevO+ActXfvIyOHkZL1lxn2uanN3E67dYqlcohb3eyxfP47LV7nWa7IuHOP3pbufplEtf2r4V95r65JU0E6SNqeJCI2kpbg+l0vtJS3mJLMWrAZyEwpLMBjZ0Xsrvp76DX0+5pH4hnstbAW9F3L6fnu68Y7DhxhVn8njfOWxccWbF537fsxvLF88jV3UiuYy4Xu+l83s4oivneczS9YuznafbeWZdzCT1pjY2Qru2DFYmD6qRN0i9Gm8QjTtJKS/NKAkaRgvqHxis8dMvyWzgc7lb6BTL8T1D9tU3kfwMeNNnK9qFRmURcTpOLivsPzjGcSvu8z7valnns0C59rwTPAMUS0I1TtO123V0KoxTur5xl59Vn7ySdlSQt4g01sqOcmERxrxb2rbaHNub+2ZZiNeFgwAv4ff9BF2EVB+nuyvHvgP+VdxWrXuspkhMYdy4CpfSfEYK44gciq4vn6ptEfpgqksAAB4jSURBVBKn6drrOi44ZobrNYxzEas+eSXtaPpZgphMfjq/cwmTGtf76Wu5ePR2ZsvuiqIrj095e/BANgAEFrwfzv18Q9e6kfSooOd9nIuP3p7W5jWfXEaYNrWDoeGC4yJE07sOEUWqp6JEjaafpZAkF/QIK/SczuWy1VvoXbutnAYVWAu692NcU/gqmWI0xxzZTV/uVnAOXq/BGEBAps+1fOAnL2v4WruZYnvXbvO9TkHP201rzojUmOSd5lOYMHR1djjWrU+jNaiZaC11Je2oIE8ISfXTeQk9sPK6S2biI7pyXHveCa411e1pUG6C6j3Tfgk3XmlVWcsfASPP1gSvdckoV3SsYa8cTjfP1xxj3IAg5bzveyYW8XjvIS22EUEM3jnbfibzoGZtJ+FinZupOX49puEkxV+0Gl3YKGlHBXlCSKqfzkvo7T84VlF5bM9wgeV3PVzj263et1TetSSoKvK+xzjUW8+jytpseYbNL/8sp2/5JEwcUs8PmizLCx+syP3uqRKSjQhi8M7ZdjpX+/5Btb8wBXmSkBqYdnRho6QZTT9LCPWmKTUbL6FnF+IlCuPGNbWo+phLsxv59ZRL+GLuy8zJ7EbENzC7zIGuozh9yQdh6Zdh+lxAGM6/mE+av68Q4k5CMug1dUvJckqPcqP6+oVJK7SntU145F0nITVQUZTWoRp5Qkiqny6o9mln3BjyuaxrGtT0fI4N3/8yK+UWumS0jlkJXW9aaf1ZLOAC0AUsGhjk5z4mUjeztRNOCxknU+zw6Bh7hmsd906LhqDanz02wUkjLx1fTcOK0t6oIE8IrXgYBwlic8yDzlhCxUEhBw51MvvE3VsZLkzUfD46Ns5l8p26hTgL3u+YMgbBhGSjgthpHLdI8HoXYtXHcxLi9uOraVhR2hcV5C3ESZDGle4SNHK7WuhNz+fYPzrGRK18BqxiJ/Zo6mEHjXa4MMHsKbuDTTTbCZ3TrDropXajDkI8bGR9EEEswBnHz2q4L3w9uAUMZkWYMEa1bkVRymgeeYtodS5vvbmzbvvBoah1e6MVt7trQ+elzMm4CXMBjOX7dhDc1YL1jONnOVYJC3str+5/hDseerJizrmM5bi3B/DF8T2FySNX4mUy1XtQkovmkaeAVqeb1Rsl7/a5QE3Ospd//YaxZfTlbq0wrxtAPKqtgbMloVr4gvO19HsAP/DorprjOAX0xfE9aSR6cOIUrEmu96C0LyrIG6Teh4ibQBwcGvGvwR0B9QqK6v2WZDZwbcc3mZHZB71wMNfN9ea9fGPfK5iez5HLimM62tqJRWTGhH/MrmZ25hkO5I+yAthcBHgJpwWQm9Zvv8ZBHsBhUv2anRboFfyoGuEh4hasrV6AK4oTmn7WAI10TfISmHF0YKo3Zcm+3zdzn+GLuS9zZGZfOW1sSmGITxT+lfMyG6ycbA/PTf/4Qjb97c/I9A7RdeWjvkIcwglQ+zUO0vErjLbbbM3YLU0NmBSduvoHBlnYt57jVtzHwr71dc8/zk5ukNx6D0p7o4I8IE4PnkYeIkFykZv5QKq3TerS7EY2T7uM3099O6/JbHOsdd4pY1zRsQawTNNeueGXr97C1f2PeGxRiZsArR6jelES5AHs9J3kMkIuW3n0uNICndqjxi24mkGUbUPjFqxJrfegtDcqyAPg9uBx8/8GeYhUC1I3mvFAKi1KLl+9BYAbLzy1po92BVvXwI0nQu90uPsSukaeIgOeDUtmyzPlv73CKQ1wx0NPBn6Iu1kS3vGqoz0XJUEewE6Lm1VvPYVVbzklMX3hJ4NGGOViJG7BqsV3lCTSEh+5iHQDtwInYj3L32+M+Xkr5hIEtwdP1qNIRxDsKVBu0eCm+FlQP6if/zSwT3HrGrh/JezdTjmKvDwjf3aaIwNtVzpiUB9jvWleYUqjerUjbTWTIQguysVI3IWUtPiOkkRaFez2ReDfjTFvEZFOrKJcicXtAeNUwazeh4hXtbGgATxBhHSgYJ2ta+CeS6FQOu9wKYqjpoMbxix/dz6XZWou41hsxU6Yh3g9xU8mywM4qRUAwxDlYqQV36sW31GSRuyCXEReALwWeC+AMWYUqKfEV2y4PXhKFcyieIjYH0hOYwWJjA0ipANpQ/evtAnxcIxJjn/u+AfuOfiK8vUBfEuixqFR2h/AJcvF5au3pEqoT4YFSdSLERWsSrvTCo38z4BdwNdF5BRgM/BRY8z+FswlEF4PnigfIqVjuRUDqU6nqn6YBxHS9kVJqevYbNnN0zILtu63Isf37gg/ecnCae+l49zP0wv0OmzitkiJW6N0s1xs+sOzPPDorsQLyDQLLnuQaMk11ZPga60oaaAVgrwDeDnwEWPML0Tki8AK4FP2jUTkEuASgKOPPjr2Sdpphhbk5cv2Mz26CaLp+Vy5DafTfnBoUXL2+E8rCrIcxS7LnA5WKdS92z1m7115zQknbbhVAtPNcmEvLKOFPqLHqX68fUGsKEp9xF6iVUSOAh4yxhxbfP0aYIUxxrXu5GQr0epXntXvc7fAuCO6chwoTNTs983T/8Dpv+0r9/c+mOtmpDBON8/XTq4knCt85FCP8E4qXqVjq/ErWasEp96ywIrSjiS6RKsx5o8isl1E5hljHgPOAv477nm0Uiv082W7WQDAu9b50HCBGy88tWK/L7zsd5y+5ZMwcUhTn1IYYorb5PbuOCSk719pvfZoVuJFqzVvN8K0ZnVyV5TOa3BoRM3DIZgMqXOKkkRaFbX+EeCOYsT674H3xTl4q+slBy3PatdSnLT0aqbnc7X+0xsvrRDivkyfY/3f1ue7Hlp9jb1winmwJ9jZqQ7Cc2sv2urzS+qiyc5kSJ1TlCTSkoIwxpgtxpgFxpiTjTFLjTF74hy/1dWx6inP6tbW0k5hfKK27GWYwLVc3tK8I6DV19gLp8Iv73jV0YEKfXh9D606vygrpTUTLaaiKM2hLZumtNLE1z8wyP6DY77bBU0bKzctkX0A7BmZRm/m3awdWsRVdz/CG6YdRdfIU86D5GdA52ENmc/dSLoZ1Snye8ExM3y12nq7w4UljIadlkYekyF1TlGSSFsKcjcTX0aE/oHBwA+WsObMIOZxO35pYz2yGwNkbKVSZ7CPz+VugQKsLSzihsKF9Ga+VGtez3Z6tgttlDSaUYOkdfn516M4v7BuiaQvmuykOXVOUZJKW9Zad2tYMm6Mp0nS3jhl/sofs/y7D4cyZ7qZZbMuRctnd+fpHxhk/sofVwjxvtytzMnsRqRSiJewNy35xr5XwNIvW9p3ifwMOP9LTY08b4UZtZGOWkH3PeP4Wa618aM6v7BuCW3koSjtTVtq5CWN4ONrHq6ple5mkqzWkpxKjvqZM8OWej3j+Fksv+vhin7eV3SsKed+e1FqWjK7Ow8nnxN7uljcZtRGguuC7ts/MMj3Ng86BsVFGbUeVsOeDGVbFUWpn7YU5GA9oEvdv6pxemAGCTZz27dE2FKvq9Y9ViHEAWbLbt85gNW0xO9h3uxI5zjNqI34iYPu63YPRJ0HHdYtob5nRWlv2laQQ7gHZlB/o5c5M1Cp161r4P4r4Qc7WDBxJDdklrF2YtGheZiZzPER5qOmg1s738n157i32/TTQtOQzmSnET9x0H3j8kXXo2Gr71lR2pe29JGXCOPHDeJvDPKwrU57quhtXeo6tnc7YJiT2U1f7laWZDaUj3HD2DKGTWfFcSfsSnt+Bp1v/r/0Xn1d3Q1W0pLOZKcRP3HQfePyRfveJ4qiKDZiL9FaD80s0RpU83SKOM9lhcM6O9g7UgivtZb7fdtSv8r9vyvZMTGTRaM3lV8fanbyDDvNkdza+U56r74u1Hm7lSkVvF0AJRNyGI09Du3er6xtFPsGKa2bJiuGoijJJdElWpNGUJNkZH7IrWvgR1eW654DlvCuqW1+iNmZZyper51YxNpRy9yez2W5/pyTws0Bb7eCnwk5TGBZXBXeGvl+gu7rtV2SK9kpijK5aXuNPFZKpnO3Xt+SBeMQUDd9Llz+GyA67dZLu3RrN1rSyMM0v2hlo4w4NWRtCKIoSpSoRh4BTREC9690F+JgCfFcvnKbqrKpUQU1+WmhXsFWYYK+WlWsJG4NOU1FWRRFmVyoIHegYSHg5P8+eZl/3fNSi9AGu44FxW1R4Cfk3czy3V05Fvatr9inVRXe4i5bmsZKdoqiTA7UtO5AaDOpXXDnj4CDz1eWRM3l4bybXIPZKrZJQZ9vt8A/DBRsIfT5XJY3n9bD9zYP1hWE1ghePceb0XK0kWA7RVGUasKY1ts6/cyNUGbSqpQxRp6trWteGLGE+FnXWAK7mvyM1AhxcE6POqyzo0KIg6UBP/DorpakUnlpws1Ip9OUMUVRWoWa1h3wNZPaNXDJOAeoVbN3xyFBHcB0nvRUpmqz/HEr7nPcbufQSEuKlTgVVbHTDDO7FmVRFKUVqCB3wLOy1r0fg01fg5LhNogQB0togyW0fQR3d1eOfQfGyhpuvYFacS4GkuYjtvv53bqVaSCaoiiTgbY2rbt1vHI1k2Y3VgrxoFRFnjvNY/ldhzqp7RkuOJqp3bpfuR0zzupsreh25sfS+T1sXHEmPdodTFGUSUzbauR+kelL5/dYgrtkBn9wDozuJ5AQz3ZC5zTMyB7+xEyu3/9WNv1wJsvHnXudX3fPtprmKE44aZBuWnccUdvVY7/5tB4eeHRX4twB2h1MUZTJTNsKcidBdzMrec0PtsEPSu8IZcHtFm1eZMxkyIjhQP4out60kv7xhZXCw8M87tQS1YlqDdJrMdLsvGansb+3eTCRAV7aHUxRlMlM2wrykkAr1S3vKXYUk4qtgpnQJwx8rPAh1k4sIj+e5frxkyLXiJ00SK8xmu2zjjtPu1HaJRAt6UGSiqJET9sK8tndeS7Z9yXelf0PMuK/vRsTBr41/vpyq9GSMAujEXfncwyN1GrlpWm5PZC9xrjxwlMDm5PrefhrJbPkofXeFaU9aVtB/oWX/Y7TNocX4gdz3UzpOhz27mDHxJHcMFbZLxwoC8SgGvG5p7yY2x96suK9XEZY9dZTPB/AXmMENSfX+/BPWpS6kj4riaIo0dC2Ueun/++/hBbiw6aT6817rQYmvUNc2PWVGiEOhzToIFHc/QODfG9zZSS5ABe+Yq7vw9dvjFLU9uN957BxxZmOx/N6+DcythtumQJK46iVRFHak7YV5L51zwFj4IDJMmGEHRMzWVG4mG/se0X5cy9hVkph687nyp9NzdVebidBaoAHHt3lO78oqonV+/CvZ+y4U+LaDTdriFpJFGVy07amdabP8YxEN8B/TpzAuwufrHjfnpMcxHx9cGyi/Pee4UKN2bpRLarRIK5GTORhx1bTb3PRNDtFaU/aV5CfdY1Db3CBBe+Hcz/PD4raI9Q+FPsHBrnunm3ltLHufI4bLzy1RhgFEVyt9jXH+fBX029z0TQ7RWlP2leQ+9Q9d3soAiy/6+GKAi5DIwU+tnpLxX4QTHC1WouK8+Hf6kVLO9AuaXaKohyifQU5uNY9L+H0UFzYt96xCtsE0Lt2W8X2QQRX3FqUW6pZHA//Vi9aFEVRJiPtLcjrwMsMXJ0LHlRwxSVIW51nrKZfRVGU6FFB7oBXgRQ3LduJpAmuJASbqelXURQlWlSQV+GktV6+eguXrd5CT3eeM46fVVO8pcQRXbma95IkuDTYTFEUZfLRvnnkLrjldcOhxiALXzKjZr9cVrj2vBNimGH9TIY8Yy0ooyiKUokK8ir8tNORwjhPPDPCFy48taIYyqq3eJdTdSNOwZTEnuFh0IIyiqIotahpvYogPvCdQyORmMzjDj5Lms8+LEnw8SuKoiSNthbkTkFtTpHm1aS5FWirffalaz44NEJWhHFj6NGOa4qiKHXTtqZ1NzMtUK4hDtX9yaM1RbebYLJfc4BxY0UfBDWRTwYfv6IoStS0rSD304Y3rjiTJ/rO4cYqX3jYpiRe1CuY0hrw5XTNSzSz45qiKMpkpm1N60G14Waaot3M+MOjY/QPDDqO2+qiLo3gZ2kI0nENGvfxe9UJUBRFSRttK8iTUPe7JDx6126rqArn1CWtRJoDvvwCCZvRca2aNC+EFEVRnGhb03pSzLRL5/dw2JTa9ZSbqTnNfnWna14irmvvtRBSFEVJI22rkScpFSuMcE6CJaFe7Ne8nqj1KEjzQkhRFMWJthXk0PpUrBJhhHPaO4i1+pqneSGkKIriRNua1pNEGDP/0vk95fS4ZkTST3aS4lJRFEWJirbWyJNCWDN/q7XaNJMkl4qiKEoUiDHGf6tmDCySBTYBg8aYc722XbBggdm0aVM8E1MURVGUFiMim40xC4Js20qN/KPAb4EXtHAOiUDzmhVFUZR6aYmPXETmAOcAt7Zi/CShHb0URVGURmhVsNsXgCuACbcNROQSEdkkIpt27doV6eBJKnGqec2KoihKI8RuWheRc4GnjTGbReR1btsZY24BbgHLRx7V+GEqe8Vh8ta8ZkVRFKURWqGRLwSWiMgTwHeAM0Xk9rgGD6oBx2Xy1o5eiqIoSiPELsiNMVcZY+YYY44F3gasN8a8M67xg2rAcZm8Na9ZURRFaYS2KwgTVAOOy+StBV4URVGURmhpQRhjzIPAg3GOGbTEaZylPLXAi6IoilIvbaeRB9WA1eStKIqipIG2LNEaRAPWUp6KoihKGmhLQR6UdjN5a4U5RVGU9KGCXAHC5dcriqIoyaHtfOSKM1phTlEUJZ2oIFcArTCnKIqSVlSQK4BWmFMURUkrKsgVQNPtFEVR0ooGuymAptspiqKkFRXkRTT1qv3S7RRFUSYDKsjR1CtFURQlvaiPHE29UhRFUdKLCnI09UpRFEVJLyrI0dQrRVEUJb2oIKe+1Kv+gUEW9q3nuBX3sbBvPf0Dg82epqIoiqLUoMFuhE+90uA4RVEUJSmoIC8SJvXKKzhOBbmiKIoSJ2parwMNjlMURVGSggryOtDgOEVRFCUpqCCvA61LriiKoiQF9ZHXgdYlVxRFUZJC2wryRmura11yRVEUJQm0pSDX9DFFURRlstCWPnKtra4oyv/f3t3HylGVcRz//mwLSCHQWlRCCwWDKCW8tMRIQQXfKKCtBBPqW0AkhhdBMZigJEggGgwmKhICqCTWQAERTG3AgoJowLbU2ncoLS0iaSOFKlApkJbHP865OHfZ2ztLubszc3+f5KazZ87MPc89O312Zs7OMWuKYZnI/fUxMzNrimGZyP31MTMza4phmcj99TEzM2uKYTnYzV8fMzOzphiWiRz89TEzM2uGYXlp3czMrCmcyM3MzGrMidzMzKzGnMjNzMxqzInczMysxpzIzczMasyJ3MzMrMacyM3MzGrMidzMzKzGnMjNzMxqTBHR6zYMStIm4B9DsOtxwLNDsN9ucxzV0pQ4oDmxOI5qaUocMHSxHBAR+5SpWItEPlQkLYqIo3vdjp3lOKqlKXFAc2JxHNXSlDigGrH40rqZmVmNOZGbmZnV2HBP5Df2ugFvEcdRLU2JA5oTi+OolqbEARWIZVjfIzczM6u74X5GbmZmVmuNTOSSpklaLWmtpEvarN9V0m15/QJJEwvrvp3LV0s6sZvtblUijm9KWiVpmaQ/SjqgsG67pCX5Z053W/5GJWI5U9KmQpvPLqw7Q9Ka/HNGd1v+hnYOFsePCjE8Luk/hXWV6RNJN0l6RtKKAdZL0jU5zmWSJhfWVak/BovjC7n9yyQ9LOmIwronJS3P/bGoe61u287B4jhe0vOF989lhXU7fE92U4k4vlWIYUU+JsbmdVXqjwmSHpD0qKSVkr7epk51jpGIaNQPMAJ4AjgI2AVYChzaUuc84Pq8PBO4LS8fmuvvChyY9zOiwnGcAOyel8/tiyO/3tLrvugwljOBa9tsOxZYl/8dk5fHVDWOlvoXADdVtE8+DEwGVgyw/mTgHkDAB4EFVeuPknFM7WsfcFJfHPn1k8C4XvdFyTiOB+bu7Huy13G01P00cH9F+2NfYHJe3hN4vM3/WZU5Rpp4Rv4BYG1ErIuIV4FbgRktdWYAv8zLdwAfk6RcfmtEvBIR64G1eX+9MGgcEfFARLyUX84Hxne5jWWV6ZOBnAjcFxGbI+LfwH3AtCFq52A6jeNzwOyutKxDEfFnYPMOqswAZkUyH9hb0r5Uqz8GjSMiHs7thAofIyX6YyA7c2y95TqMo8rHx8aIWJyXXwQeBfZrqVaZY6SJiXw/4J+F10/zxg54vU5EbAOeB95Rcttu6bQtXyF9Ouyzm6RFkuZL+sxQNLADZWM5LV+iukPShA637YbSbcm3OQ4E7i8UV6lPBjNQrFXqj061HiMB3Cvpb5K+2qM2deIYSUsl3SNpUi6rZX9I2p2U3H5TKK5kfyjdej0KWNCyqjLHyMih3HmPqE1Z69D8geqU2bZbSrdF0heBo4GPFIr3j4gNkg4C7pe0PCKeGIJ2llEmlt8BsyPiFUnnkK6YfLTktt3SSVtmAndExPZCWZX6ZDB1OEZKk3QCKZEfVyg+NvfHO4H7JD2WzyiraDHpkZ1bJJ0M/BY4mJr2B+my+kMRUTx7r1x/SNqD9GHjGxHxQuvqNpv05Bhp4hn508CEwuvxwIaB6kgaCexFuhxUZttuKdUWSR8HLgWmR8QrfeURsSH/uw74E+kTZa8MGktEPFdo/8+AKWW37aJO2jKTlsuGFeuTwQwUa5X6oxRJhwM/B2ZExHN95YX+eAa4i97dRhtURLwQEVvy8t3AKEnjqGF/ZDs6PirRH5JGkZL4zRFxZ5sq1TlGejGQYCh/SFcZ1pEua/YN/pjUUud8+g92uz0vT6L/YLd19G6wW5k4jiINdDm4pXwMsGteHgesobcDYMrEsm9h+VRgfl4eC6zPMY3Jy2OrGkeudwhp4I6q2ie5HRMZeHDVKfQfyLOwav1RMo79SWNdpraUjwb2LCw/DEyrcBzv7ns/kRLcU7lvSr0nqxJHXt934jS6qv2R/7azgB/voE5ljpHGXVqPiG2SvgbMI43ovCkiVkq6AlgUEXOAXwC/krSW9IaambddKel2YBWwDTg/+l8arVocVwN7AL9OY/V4KiKmA+8HbpD0Gumqy1URsaoXcUDpWC6UNJ30d99MGsVORGyWdCXwSN7dFdH/clzXlIwD0iCeWyMf1Vml+kTSbNJI6HGSnga+C4wCiIjrgbtJo3LXAi8BX87rKtMfUCqOy0jjX67Lx8i2SBNcvAu4K5eNBG6JiN93PYCsRByfBc6VtA3YCszM76+278kehACUigPSB/V7I+K/hU0r1R/AscCXgOWSluSy75A+GFbuGPGT3czMzGqsiffIzczMhg0ncjMzsxpzIjczM6sxJ3IzM7MacyI3MzOrMSdysxpQ/5nTlkiaKOloSdd0sI+9JZ23g/UX5tmebn4T7Zso6fOdbmdmO89fPzOrAUlbImKPknVHRppDoLV8ImkGrcMG2O4x4KRIEwZ12r7jgYsj4lMdbjeiV89qMGsKn5Gb1ZTSHNVz8/Llkm6UdC8wS9IkSQvz2fsySQcDVwHvyWVXt+zretJUmHMkXSRptNLc0o9I+rukGbneREl/kbQ4/0zNu7gK+FDe90VK88tfW9j/3JzskbRF0hWSFpAmApki6cE8Wca8PIOUmZXUuCe7mTXU2wtPmFofEae2qTMFOC4itkr6KfCTiLhZ0i6kp35dAhwWEUe2bhgR50iaBpwQEc9K+j5pruizJO0NLJT0B+AZ4BMR8XL+cDCbNGHPJRTOyCWduYNYRpMe4XlZfp71g6TnoG+SdDrwPeCsDv8+ZsOWE7lZPWxtl4BbzImIrXn5r8ClksYDd0bEmvz4y7I+CUyXdHF+vRvp8ZQbgGslHQlsB97byU6z7fx/+spDgMNIs11B+sCx8U3s02zYciI3a47Xn10dEbfkS9enAPMknU2aXKMsAadFxOp+hdLlwL+AI0i35l4eYPtt9L91t1th+eXCfXEBKyPimA7aZmYFvkdu1kB5zvN1EXENMAc4HHgR2LPkLuYBFyifJkvqm3J1L2BjRLxGmlRiRC5v3feTwJGS3iZpAgNPSbka2EfSMfn3jJI0qWQbzQwncrOmOh1Yke+rvw+YFWku7ockrWgd7NbGlaRZq5ZJWpFfA1wHnCFpPumyet9VgGWkmbiWSroIeIg0feNy4IfA4na/JCJeJc3s9QNJS4ElwNR2dc2sPX/9zMzMrMZ8Rm5mZlZjTuRmZmY15kRuZmZWY07kZmZmNeZEbmZmVmNO5GZmZjXmRG5mZlZjTuRmZmY19j+UGTep1A4gQQAAAABJRU5ErkJggg==\n",
      "text/plain": [
       "<Figure size 576x432 with 1 Axes>"
      ]
     },
     "metadata": {},
     "output_type": "display_data"
    }
   ],
   "source": [
    "# Plot the test predictions\n",
    "\n",
    "fig = plt.figure(figsize=(8,6))\n",
    "plt.title(\"Dataset in blue, predictions for test set in orange\")\n",
    "plt.scatter(X_train, y_train)\n",
    "plt.scatter(X_test, y_p_test)\n",
    "plt.xlabel(\"First feature\")\n",
    "plt.ylabel(\"Second feature\")\n",
    "plt.show()"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": null,
   "metadata": {},
   "outputs": [],
   "source": []
  }
 ],
 "metadata": {
  "anaconda-cloud": {},
  "kernelspec": {
   "display_name": "Python [conda root]",
   "language": "python",
   "name": "conda-root-py"
  },
  "language_info": {
   "codemirror_mode": {
    "name": "ipython",
    "version": 3
   },
   "file_extension": ".py",
   "mimetype": "text/x-python",
   "name": "python",
   "nbconvert_exporter": "python",
   "pygments_lexer": "ipython3",
   "version": "3.6.4"
  },
  "toc": {
   "nav_menu": {},
   "number_sections": true,
   "sideBar": true,
   "skip_h1_title": false,
   "title_cell": "Table of Contents",
   "title_sidebar": "Contents",
   "toc_cell": false,
   "toc_position": {},
   "toc_section_display": true,
   "toc_window_display": false
  }
 },
 "nbformat": 4,
 "nbformat_minor": 2
}