FMML_Module4_Lab3_2024.ipynb

{
  "nbformat": 4,
  "nbformat_minor": 0,
  "metadata": {
    "colab": {
      "provenance": [],
      "include_colab_link": true
    },
    "kernelspec": {
      "name": "python3",
      "display_name": "Python 3"
    },
    "language_info": {
      "name": "python"
    }
  },
  "cells": [
    {
      "cell_type": "markdown",
      "metadata": {
        "id": "view-in-github",
        "colab_type": "text"
      },
      "source": [
        "<a href=\"https://colab.research.google.com/github/Harsha-070/FMML_IIITH/blob/main/FMML_Module4_Lab3_2024.ipynb\" target=\"_parent\"><img src=\"https://colab.research.google.com/assets/colab-badge.svg\" alt=\"Open In Colab\"/></a>"
      ]
    },
    {
      "cell_type": "markdown",
      "source": [
        "# FOUNDATIONS OF MODERN MACHINE LEARNING, IIIT Hyderabad\n",
        "# Module 4: Perceptron and Gradient Descent\n",
        "## Lab 3: Gradient Descent\n",
        "\n",
        "Gradient descent is a very important algorithm to understand, as it underpins many of the more advanced algorithms used in Machine Learning and Deep Learning.\n",
        "\n",
        "A brief overview of the algorithm is\n",
        "\n",
        "\n",
        "*   start with a random initialization of the solution.\n",
        "*   incrementally change the solution by moving in the direction of negative gradient of the objective function.\n",
        "*   repeat the previous step until some convergence criteria is met.\n",
        "\n",
        "The key equation for change in weight is:\n",
        "$$w^{k+1} \\leftarrow w^k - \\eta \\Delta J$$\n",
        "\n",
        "In this lab, we will discuss stochastic gradient descent, mini-batch gradient descent and batch gradient descent.\n"
      ],
      "metadata": {
        "id": "XYxxkQg6xCjD"
      }
    },
    {
      "cell_type": "code",
      "metadata": {
        "id": "Fr-MnaGs7JmZ"
      },
      "source": [
        "import numpy as np\n",
        "import matplotlib.pyplot as plt"
      ],
      "execution_count": 1,
      "outputs": []
    },
    {
      "cell_type": "code",
      "metadata": {
        "id": "7ob_zZms7VOu"
      },
      "source": [
        "np.random.seed(42)"
      ],
      "execution_count": 2,
      "outputs": []
    },
    {
      "cell_type": "markdown",
      "metadata": {
        "id": "v4Kix4bcChiy"
      },
      "source": [
        "# Creating the Data\n",
        "\n",
        "Let's generate some data with:\n",
        "\\begin{equation} y_0= 4 \\end{equation}\n",
        "\\begin{equation} y_1= 3 \\end{equation}\n",
        "\n",
        "and also add some noise to the data."
      ]
    },
    {
      "cell_type": "code",
      "metadata": {
        "id": "MtAS7eFZ9hX6"
      },
      "source": [
        "X = 2 * np.random.rand(100, 1)\n",
        "y = 4 + 3 * X + np.random.randn(100, 1)"
      ],
      "execution_count": 3,
      "outputs": []
    },
    {
      "cell_type": "markdown",
      "metadata": {
        "id": "zD95NaF-CxM-"
      },
      "source": [
        "Let's also plot the data we just created"
      ]
    },
    {
      "cell_type": "code",
      "metadata": {
        "colab": {
          "base_uri": "https://localhost:8080/",
          "height": 466
        },
        "id": "3IiEP4BQ7Wja",
        "outputId": "0154f5ed-9da1-4f59-9f16-8260833ca500"
      },
      "source": [
        "plt.plot(X, y, 'b.')\n",
        "plt.xlabel('x')\n",
        "plt.ylabel('y', rotation=0)"
      ],
      "execution_count": 4,
      "outputs": [
        {
          "output_type": "execute_result",
          "data": {
            "text/plain": [
              "Text(0, 0.5, 'y')"
            ]
          },
          "metadata": {},
          "execution_count": 4
        },
        {
          "output_type": "display_data",
          "data": {
            "text/plain": [
              "<Figure size 640x480 with 1 Axes>"
            ],
            "image/png": "iVBORw0KGgoAAAANSUhEUgAAAikAAAGwCAYAAABsEvUIAAAAOXRFWHRTb2Z0d2FyZQBNYXRwbG90bGliIHZlcnNpb24zLjcuMSwgaHR0cHM6Ly9tYXRwbG90bGliLm9yZy/bCgiHAAAACXBIWXMAAA9hAAAPYQGoP6dpAAAxyElEQVR4nO3df3BVZX7H8U8SzIUiuYo/CNmEiCTCioBYhYk/KrKRyDIO7LQqVF3UuDCWHaXuuiszVXRsN+A6SO1QBCcrqIvUqqD1F/7YBKugIsoW7dYFl2iugLRbyQ2o0U1O/7jNJTe5+XGSe+7znHPer5k7MTcn8XnuCTmf+zzf5zk5juM4AgAAsEyu6QYAAACkQ0gBAABWIqQAAAArEVIAAICVCCkAAMBKhBQAAGAlQgoAALDSINMN6K+2tjbt379fw4YNU05OjunmAACAPnAcR83NzSoqKlJubs9jJb4NKfv371dJSYnpZgAAgH5obGxUcXFxj8f4NqQMGzZMUqKTBQUFhlsDAAD6Ih6Pq6SkJHkd74lvQ0r7FE9BQQEhBQAAn+lLqQaFswAAwEqEFAAAYCVCCgAAsBIhBQAAWImQAgAArERIAQAAViKkAAAAKxFSAACAlQgpAADASoQUAABgJUIKAACwEiEFAACXYjGpri7xEd7xJKS8/vrruvzyy1VUVKScnBxt3rw55etPP/20ZsyYoZNOOkk5OTnatWuXF80AACDjamul0lJp+vTEx9pa0y0KLk9CytGjRzVp0iStWrWq269feOGFWr58uRf/ewAAPBGLSQsWSG1tic/b2qSFCxlR8cogL37ozJkzNXPmzG6/fu2110qSGhoavPjfAwDgiT17jgWUdq2t0t69UnGxmTYFmSchxQstLS1qaWlJfh6Pxw22BgAQRuXlUm5ualDJy5PKysy1Kch8UzhbU1OjaDSafJSUlJhuEgAgZIqLpbVrE8FESnxcs4ZRFK/4JqQsWbJETU1NyUdjY6PpJgEAQqi6WmpoSKzuaWhIfA5v+Ga6JxKJKBKJmG4GAAAqLmb0JBt8M5ICAADCxZORlCNHjmjv3r3Jz/ft26ddu3Zp+PDhGjVqlP73f/9Xn376qfbv3y9J+uijjyRJhYWFKiws9KJJAADAZzwZSXn33Xc1efJkTZ48WZJ06623avLkybrzzjslSc8++6wmT56sWbNmSZLmzp2ryZMn68EHH/SiOQAAwIdyHMdxTDeiP+LxuKLRqJqamlRQUGC6OQAAoA/cXL+pSQEAAFYipAAAACsRUgAAgJUIKQAAoItYLLFhncmbJxJSAABAitpaqbRUmj498bG21kw7CCkAACApFpMWLDh2E8W2NmnhQjMjKoQUAACQtGdP6l2eJam1VeqwR2vWEFIAAEBSebmU2ykd5OVJZWXZbwshBQAAJBUXS2vXJoKJlPi4Zo2ZGyr65i7IAAAgO6qrpaqqxBRPWZm5Oz4TUgAAQBfFxebCSTumewAAgJUIKQAAwEqEFAAAYCVCCgAAsBIhBQAAWImQAgAArERIAQAAViKkAAAAKxFSAACAlQgpAADASoQUAAA8FItJdXWJj3CHkAIAgEdqa6XSUmn69MTH2lrTLfIXQgoAAB6IxaQFC6S2tsTnbW3SwoWMqLhBSAEAwAN79hwLKO1aW6W9e9Mfz7RQV4QUAECPuHj2T3m5lNvpKpuXJ5WVdT2WaaH0CCkAgG5x8ey/4mJp7dpEMJESH9esSTzfkalpIT+ET0IKACAtaioGrrpaamhIhIGGhsTnnbmdFsoEv4RPQgoAIC0TF88gKi6Wpk3rOoLSzs20UCb4KXwSUgAAaWX74hlWfZ0WyhQ/hU9CCgAgrWxfPMOsL9NCmeKn8DnIdAMAAPaqrpaqqhLvssvKCCheKi7OzuvbHj4XLkyMoNgcPj0ZSXn99dd1+eWXq6ioSDk5Odq8eXPK1x3H0Z133qmRI0dqyJAhqqys1J49e7xoCgBggHqrqYD/ZHPkZiA8CSlHjx7VpEmTtGrVqrRfv/fee/XAAw/owQcf1Ntvv62hQ4eqqqpKX3/9tRfNAQAAnfghfHoy3TNz5kzNnDkz7dccx9HKlSv1d3/3d5o9e7Yk6ZFHHtGIESO0efNmzZ0714smAQAAn8l64ey+fft08OBBVVZWJp+LRqOaOnWqtm/f3u33tbS0KB6PpzwAAEBwZT2kHDx4UJI0YsSIlOdHjBiR/Fo6NTU1ikajyUdJSYmn7QQAAGb5ZgnykiVL1NTUlHw0NjaabhIAAPBQ1kNKYWGhJOnzzz9Pef7zzz9Pfi2dSCSigoKClAcAAAiurIeU0aNHq7CwUK+99lryuXg8rrffflsVFRXZbg4AALCUJ6t7jhw5or0d9tfdt2+fdu3apeHDh2vUqFFavHix/v7v/17l5eUaPXq07rjjDhUVFWnOnDleNAcAAPiQJyHl3Xff1SWXXJL8/NZbb5UkzZ8/X+vWrdPPfvYzHT16VAsWLNDhw4d14YUX6qWXXtLgwYO9aA4AAPChHMdxHNON6I94PK5oNKqmpibqUwAArsRiiRvtlZfbvZlZELm5fvtmdQ8AAJlQWyuVlkrTpyc+1taabhG6Q0gBAIRGLCYtWCC1tSU+b2tL3GgvFjPbroGIxRL34PFzH7pDSAEAhMaePccCSrvW1sRdnv0o6KNChBQAQGiUl0u5na58eXlSWZmZ9gxEEEeFOiOkAABCo7hYWrs2EUykxMc1a/xZPBu0UaF0PFmCDACAraqrpaqqxMW8rMyfAUU6NirUMaj4dVSoO4ykAABCp7hYmjbNvwFF6n5USApOIS0hBQAAi/W0eqe6WmpoSHy9oSHxXJAKaQkpAABYqi+rd9pHhaTgFdISUgAAsJDb1TtBLKQlpACAh4K80ZYU/P6Z5DZ0BGl5dTtCCgB4JOgbbQW9f6a5DR1BWl7djhsMAoAHYrHEhbvz8tCGBn9fNNoFvX+2qK1NTPG0th4LHdXVPX9PLGb38mo312/2SQEAD/Q0VG/jhcOtoPfPFv3Z06W4ODjngJACAB4I+kZbQe+fTYIUOtyiJgUAPBDE+oCOgt4/vwtKQTM1KQDgIdvrAwYq6P3zo9raY0uXc3MTYbK3OpZscnP9JqQAABAQfihodnP9ZroHAICACNqGboQUAAACImgbuhFSAAAIiKAVNLMEGQCAAOnP3iq2IqQAABAwQdlbhekeAABgJUIKAACwEiEFABBYQdl5NawIKQCAQKqtTWxsNn164mNtrekWwS1CCgAgcGKxY1vDS4mPCxcyouI3hBQAQOAEbefVsCKkAAACJ2g7r4YVIQUAEDhB23k1rNjMDQAQSEHaeTWsjI2kNDc3a/HixSotLdWQIUN0/vnna8eOHaaaAwAIoOJiado0AopfGQspN954o1555RU9+uij2r17t2bMmKHKykp99tlnppoEAPAp9kMJJiMh5auvvtJTTz2le++9V3/xF3+hsrIy3XXXXSorK9Pq1atNNAkA4FPshxJcRkLKn/70J7W2tmrw4MEpzw8ZMkRvvPFG2u9paWlRPB5PeQAAwo39UILNSEgZNmyYKioqdM8992j//v1qbW3VY489pu3bt+vAgQNpv6empkbRaDT5KCkpyXKrAQC2YT+UYDNWk/Loo4/KcRx95zvfUSQS0QMPPKB58+Ypt/PC9v+3ZMkSNTU1JR+NjY1ZbjEAwDbshxJsxkLKmDFjtHXrVh05ckSNjY1655139O233+r0009Pe3wkElFBQUHKAwAQbuyHEmzG90kZOnSohg4dqi+++EJbtmzRvffea7pJAAAf8dN+KLFYYoqqvNzudtrCWEjZsmWLHMfR2LFjtXfvXt12220aN26crr/+elNNAgD4VHHxsYu+rUGgtvZYkW9ubmIEqLraXHtsfZ06Mjbd09TUpEWLFmncuHH64Q9/qAsvvFBbtmzRcccdZ6pJAACfs3U5sm2rkGx9nTrLcRzHMd2I/ojH44pGo2pqaqI+BQCgWCxxwe242icvT2poMD9SUFeXCATpnp82LbttMf06ubl+c4NBAEAg2Lwc2aZVSDa/Tp0RUgAAgWBTEOjMplVINr9OnRFSAAD9ZtM9c2wKAulUVyemVOrqEh9NFc3a/jp1RE0KAKBfbFut0i4W88dyZNNMvU5urt+EFACAa6aLL23ghyW8NqJwFgDgKT8VX3rBL0t4/Y6QAgBwzU/Fl5lm254nQUZIAQC45qfiy0wL+yhSNhm/dw8AwJ/8dM+cTGofRepcjxOGUaRsYyQFQGjYtFw2KIqLEzumhiWgSOEeRco2QgqAUKDQEZlky54nQccSZACBx3JZwB4sQQaADih0dM/N1BjTaPAKIQVA4IV5uWx/uJka8/s0GgHLboQUAIFHoWPfudkDxO/7hfg9YIUBIQVAKFDo2Ddupsb8PI3m94AVFuyTAiA0iosZPemNmz1A/LxfSE8Bi98RezCSAgBIcjM15udpNOqU/IElyACALmKxvu8k6+ZYm9TWJqZ4WluPBSymAb3n5vpNSAEAGBGLJaZdysvNhRu/Biw/Y58UAIDV3K6s8WqpcBi39fcTQgoAIKvcrqxhqXB4EVIAAFnlZukyS4XDjZACAMgqNytr/LwXCwaOkAIABoVxW3Y3S5dZKhxuhBQAMCTMtRZ93QHYz3uxYOBYggwABsRiiWDSebfWhgYuwOmwVDg43Fy/2RYfAAxgW3Z3uKVBODHdAwAGUGsB9I6QAgAGUGvRvTAWEyM9QgoAGNLX4tEwCXMxMbqicBYAYAWKicPBF/fuaW1t1R133KHRo0dryJAhGjNmjO655x75NDMBAAaIjdvQmbHVPcuXL9fq1au1fv16jR8/Xu+++66uv/56RaNR3XzzzaaaBQAwpL2YuPNICsXE4WVsJGXbtm2aPXu2Zs2apdNOO01/9Vd/pRkzZuidd94x1SQA8AzFoL2jmBidGQsp559/vl577TX9/ve/lyT99re/1RtvvKGZM2emPb6lpUXxeDzlAQB+QDFo31VVSRs2SE88QTExDIaU22+/XXPnztW4ceN03HHHafLkyVq8eLGuvvrqtMfX1NQoGo0mHyUlJVluMQC4x118+649zF11lTR3rrRli+kWwTRjIeWJJ57Qr3/9a23YsEHvvfee1q9fr/vuu0/r169Pe/ySJUvU1NSUfDQ2Nma5xQDgnm3FoLZOOxHmkI6xwtnbbrstOZoiSRMmTNAnn3yimpoazZ8/v8vxkUhEkUgk280EgAGxqRi0tvZYEMjNTdR/2DKdwm0CsicWS7ze5eX2v7bGRlK+/PJL5XbaEzovL09tnX9LASCDsj2SYEsxqO0jFdwmIDv8Vh9lLKRcfvnl+od/+Ac9//zzamho0KZNm7RixQr94Ac/MNUkAAFn6g+0DTvL2jbt1JktYS7IbA+q6Rjbcba5uVl33HGHNm3apEOHDqmoqEjz5s3TnXfeqfz8/F6/nx1nAbgR9t1M/dL/WCwRnMrK7GpXENTVJQJ6uuenTcteO9xcv43VpAwbNkwrV67UypUrTTUBQIiEveahfaRi4cJEv20dqSgutq9NQWFTfVRfcYNBAKFAzYMd004wx49TaoQUAKHgxz/QXiguTgzt96Xf2S4ytnV5dJD4LagSUgCEht/+QJuU7SJjv6068TM3QdU0Y4WzA0XhLAB4I9tFtn4p6kVmuLl+M5ICAEiR7eXKti+PhjmEFABAimwXGVPUjO4QUgAAKbJdZExRM7pDTQoAIK1sb6zGRm7h4IvN3AAAdsv2xmps5IbOmO4BAABWIqQAAAArEVIAD7GDJgD0HyEF8Ag7aALAwBBSAA/EYtKCBcc2qGprS9x9lhEVAOg7QgrgAXbQBICBI6QAHmAHTQAYOEIK4AF20AwXCqQzh9cSHRFSAI9UVyfu4lpXl/hYXW26RfACBdKZw2uJztgWHwD6KRZLXEw71h/l5SVCKaNm7vBahoeb6zcjKQDQTxRIZw6vJdLpc0h55JFHdNJJJ6mlpSXl+Tlz5ujaa6/NeMPgf8wtI+gokM4cXkuk0+eQcsUVV6i1tVXPPvts8rlDhw7p+eef1w033OBJ4+BfYZpbJoyFFwXSmcNriXRc1aT8zd/8jRoaGvTCCy9IklasWKFVq1Zp7969ysnJ8ayR6VCTYq8wzS3X1h7btC03N/FHlgLZ9GKxxJB+eXnwfg9iscS0RFlZ8PqWbbyWwedZTcqPfvQjvfzyy/rss88kSevWrdN1112X9YACu4VlbpldZfsu6CNrxcXStGlcVDOB1xIduQopkydP1qRJk/TII49o586d+vDDD3Xdddd51DT4VVjmlsMSxgaKMAegv1yv7rnxxhu1bt06Pfzww6qsrFRJSYkX7YKPhWVuOSxhbKAIcwD6y3VI+eu//mvFYjE99NBDFMyiW2HYyCwsYWygCHMA+st1SIlGo/rLv/xLHX/88ZozZ44HTUJQhGFuOQxhbKAIcwD6a1B/vumzzz7T1VdfrUgkkun2AL5TXMwFtzfV1VJVFas2ALjjKqR88cUXqq+vV319vf75n//ZqzYBCCDCHAC3XIWUyZMn64svvtDy5cs1duxYr9oEAK4EeQ+WvuI1QBC5CikNDQ0eNQP9wR8lgA31JF4DBJexGwyedtppysnJ6fJYtGiRqSb5StA3xwL6gj1YeA0QbMZCyo4dO3TgwIHk45VXXpGUuEcQesYfJSAhyHuw9PWeUEF+DQBjIeWUU05RYWFh8vHcc89pzJgxuvjii001yTf4owQkBHUPFjcjpUF9DQDJYEjp6JtvvtFjjz2mG264odv7ALW0tCgej6c8woo/SjDFtjs+B3EPFrcjpUF8DYB2VoSUzZs36/Dhwz3eB6impkbRaDT5CPN2/PxRggm21kEFbUO9/oyUBu01ANrlOI7jmG5EVVWV8vPz9W//9m/dHtPS0qKWlpbk5/F4XCUlJX261XNQcUtzeKnj6jEpEUw6Xjzz8hIXRH73MisW47VGsMXjcUWj0T5dv/u142wmffLJJ3r11Vf19NNP93hcJBJhh9tO2BwLXum8pPXWW7t/d8/vYGa1j5QuXJh4jRkpRZgZH0m56667tGbNGjU2NmrQoL5nJjdJDEDfdfdOvq1N6vjXgnf33mKkFEHl5vpttCalra1NDz/8sObPn+8qoADwTnc1ET/5CXVQ2RSGG3QCvTGaDF599VV9+umnuuGGG0w2A0AH7avHOo+k3HJL4mHru3t2YAaCx+hIyowZM+Q4js444wyTzQDQQU+rx2x9d2/byiPblmoDfmW8JqW/qEkBvOWXmgjbVsNwHx2gZ76pSQGCzs/vqG0dNenMph2YuWUFkFmEFMAjtk1BBJVNOzDbFJiAICCkAB7gHXX22LQDs02BCQgCQgrgAd5RZ5ct28LbFJiAIGBzEsAD3S3j5R21d2zZgbm6Wqqq8kfRMWA7RlIAD/COOtz8UnQM2I6RFMAjvKP2HzaEA+zCSArgId5R+wersQD7EFIAhB6rsQA7EVIAhB6rsQA7EVIAhB77mwB2IqQACD1WYwF2YnUPAIjVWICNCCmApVgOm322bAgHIIHpHsBCLIcFAEIKYB2WwwJAAiEFsAzLYQEggZCCwIvFEnfH9ctIBMthASCBkIJA82NtR1+Ww/oteAFAf+Q4juOYbkR/xONxRaNRNTU1qaCgwHRzYKFYLBFMOk6d5OVJDQ3+WMERi6VfDltbe6xmJTc3EWiqq821EwDccHP9ZiQFgeX32o50NyekqDaBkSQgHAgpCKwg1nb4PXhlgh+n8AD0DyEFgRXErc6DGLzcYCQJCBdCCgKtujpRg1JXl/jo99oNE8HLpqkVRpKAcGFbfARe0LY6z+Y9Zmwr0m0fSepcDB2WkSQgbBhJAXwoXVFtptk4tRLEKTwA3WMkBUBaPU2teB2OerqxIncrBsKDkRQAaZko0u3ryp1sjCQBMI+QAiCtbE+t2Di9BMAspnvgid6G7OEP2ZxaMTW9BMBejKQg49hsK1iyNbUSlD1gbFqyDfid0ZDy2Wef6ZprrtFJJ52kIUOGaMKECXr33XdNNgkDxJA9+isIK3cI6EBmGZvu+eKLL3TBBRfokksu0YsvvqhTTjlFe/bs0YknnmiqScgAhuwxEH5eudNdQK+q8lc/AJsYCynLly9XSUmJHn744eRzo0ePNtUcZAibbWGg/Lr5HgEdyDxj0z3PPvuszj33XF1xxRU69dRTNXnyZD300EPdHt/S0qJ4PJ7ygH2CMGRvI+oc7BeUmhrAJsZCyh/+8AetXr1a5eXl2rJli2666SbdfPPNWr9+fdrja2pqFI1Gk4+SkpIstxh9FbT75ZhGnYM/ENCBzMtxHMcx8T/Oz8/Xueeeq23btiWfu/nmm7Vjxw5t3769y/EtLS1qaWlJfh6Px1VSUqKmpiYVFBRkpc0INhuXTcdiiWDSefqsocGeNiJVLObPmhogW+LxuKLRaJ+u38ZGUkaOHKkzzzwz5bnvfve7+vTTT9MeH4lEVFBQkPIAMsXtaEW2pl+466//sBsukDnGQsoFF1ygjz76KOW53//+9yotLTXUIoSV22XT2Zx+oc4BQJgZCyl/+7d/q7feeku/+MUvtHfvXm3YsEFr167VokWLTDUJIeVmtCLb+8BQ5wAgzIyFlPPOO0+bNm3S448/rrPOOkv33HOPVq5cqauvvtpUk7KGlRp2cTNaYWL6hUJkAGFlrHB2oNwU3tiktvbYO/Hc3MS7ZC46x5gqXq2tTYyItLYeG61Id14oZM0OG4uYAWSGLwpnw4gt43tmcqltd6MVnUe9mH7xHkuuAbRjJCWL6uoSf3jTPT9tWtabYxUbRyh6GvVimak3bPw9AJBZjKRYipUa3bNtqW1vo14sM/VGf34PqPECgouQkkVMFXTPtgBnW2gKC7e/B0wNAcFGSMkyVmqkZ1uAsy00hYWb3wNqvIDgM3YX5DDz611eM6GnVRvV1Ynb2ttQ69F+sey84ies5y2b+vp7wF2HgeCjcBZZ48fl1xTI2osiW8CfKJyFdfw6NE+BrL1smyJsRyEvkDmEFGQFhajwgm01XhTyApnFdA+ygqH57GG3VjP4HQf6hukepLBh+NnWofmg4Z28OYwWApnHSErA2VasSiGqd3gnbxavP9A3jKRAkp3FqhSieod38mYxWghkHvukBEzHegT2kQiX9g3oOr+TZwO67LFprx8gCBhJCZDO9Qg7d7JrapjwTt4OjBYCmUNNSkB0Nx9eUyMtWZK6a6rpZZrwFnU/AGzm5vrNdE9AdDe1c955icI9LlrhEebbLgAIFkJKQPRUj8BFCwDgR9SkZIjpvUioRwAABA0hJQNs2UDLti3CET6mwzqAYCGkDJBte5F4tbIgiBefIPbJJFvCOoDgIKQMUBg20ArixSeIfTLJtrAOIBgIKQPUXrDaUZD2IgnixSeIfTItDGEdQPYRUgYo6AWrQbz4BLFPpgU9rAMwg5CSAUEuWA3ixSeIfTIt6GEdgBmElAwJ6lbYfbn4pCtAtbkolQuqN4Ic1gGYwbb46JPutlqvrT1W35Gbm7j4S12fs/GCxfbxAJB9bq7fhBT0W3f3C2prkzr+VuXlJd5ZEwQAAG6u30z3oN+6K0DtHHuDWJRq83QWAAQFIQWS+nfR7a4ANSen63NBKkpljxUAyA5CCvp90e2uAPWhh4JblMoeKwCQPdSkuBSLJaY5yssHfuHN5M8aSBvS1ZW4qSFJV4Aa1KLUurpEmEv3/LRpWW8OAPiOL2pS7rrrLuXk5KQ8xo0bZ6o5fZLJYX5bpgwysbFZuuXXQV2SzR4rAJA9Rqd7xo8frwMHDiQfb7zxhsnm9CiTw/w2TRlw0XWHPVYAIHuMhpRBgwapsLAw+Tj55JNNNqdHmdxK3aZt2bnousemZQCQHYNM/s/37NmjoqIiDR48WBUVFaqpqdGoUaPSHtvS0qKWlpbk5/F4PFvNlHRsxKFz7UZ/Rhwy+bMyobpaqqoKZg2JV4qLeZ0AwGvGRlKmTp2qdevW6aWXXtLq1au1b98+XXTRRWpubk57fE1NjaLRaPJRUlKS1fZmcsTBxtGLoNaQAAD8y5rVPYcPH1ZpaalWrFih6jTj5+lGUkpKSoys7snUiENQV8Bg4GxY+QUAXnCzusfodE9HJ5xwgs444wzt7aYwIxKJKBKJZLlVXWVymJ8pg2DIdKBIdz8k6l4AhJE1m7kdOXJEH3/8sUaOHGm6KciCoGwrn+ml5Dat/AIA04yFlJ/+9KfaunWrGhoatG3bNv3gBz9QXl6e5s2bZ6pJyBJb9ogZKC8ChU0rvwDANGMhJRaLad68eRo7dqyuvPJKnXTSSXrrrbd0yimnmGoSsiBIIwVeBAr2rQGAY4zVpGzcuNHU/xoG9XRh91t9jpul5H2tW2lf+bVwYeJ1sWHlFwCYYk1NCsIhSCMFfV1K7nZ6i83iACDBmiXIbpm6wSAGrra260iBny/EPS0lz8QNHAEgSHy5BBl2y+Qy26DtcNvTUvIgTW8BQLYx3WMZr5bmDuTnerEaJyw73AZpegsAso2Q4jE34aC2Vho1KhEGRo3K3NLcgYSMIK3GMcHGWyAAgF9Qk+Khvuwc2j6Ncvzx0tSpUsezkZMjffrpwC5oA62JqKtLhJt0z0+b1v92hQ23QACABDfXb0ZSPNKXEYiOIxxTpqQGFCnx+fbtA2vHQPfyYLoiM8IyvQUAmURI8Uhv4aBziPHKQEMG0xUAAFMIKR7pLRykCzGd5eZKFRUDa0cmQgb7dgAATGAJchqZWG7b286h6XYrzclJPNraMjtikYklv9yxGQCQbRTOdtKXYlc3eiqYTLepWZD2DwEAoDM3129CSgcmdgdl1QcAIEzYcbafTOwOyjQKAADpUTjbActtAQCwByGlA5bbAgBgD6Z7OunrSphM3nAPAAB0xUhKGr3tDurFDfcAAEAqQopL3HAPAIDsIKS4NNB74QAAgL4hpLjECiAAALKDkOISK4AAAMgOVvf0QybuhQN/YBUXAJjDSEo/9bYCCP7HKi4AMIuQAqTBKi4AMI+QAqTBKi4AMI+QAqTBKi4AMI+QAqTBKi4AMI/VPUA3WMUFAGYRUoAeFBcTTgDAFKZ7AACAlQgpAADASoQUAABgJUIKAACwkhUhZdmyZcrJydHixYtNNwUAAFjCeEjZsWOH1qxZo4kTJ5puSuDEYlJdHVu5AwD8yWhIOXLkiK6++mo99NBDOvHEE3s8tqWlRfF4POWB7nFzPACA3xkNKYsWLdKsWbNUWVnZ67E1NTWKRqPJR0lJSRZa6E/cHA8AEATGQsrGjRv13nvvqaampk/HL1myRE1NTclHY2Ojxy30L26OBwAIAiM7zjY2NuqWW27RK6+8osGDB/fpeyKRiCKRiMctC4b2m+N1DCrcHA8A4DdGRlJ27typQ4cO6ZxzztGgQYM0aNAgbd26VQ888IAGDRqk1tZWE80KDNtvjkdBLwCgL3Icx3Gy/T9tbm7WJ598kvLc9ddfr3HjxunnP/+5zjrrrF5/RjweVzQaVVNTkwoKCrxqqq/FYvbdHK+29li9TG5uIkxVV5tuFQAgW9xcv41M9wwbNqxLEBk6dKhOOumkPgUU9I1tN8frrqC3qsqudgIA7GB8nxSEBwW9AAA3jIykpFNfX2+6CfAYBb0AADcYSUHW2F7QCwCwizUjKQiH6upEDYptBb0AAPsQUpB1thX0AgDsxHQPAACwEiEFAABYiZACAACsREgBAABWIqQAAAArEVIAAICVCCkAAMBKhBQAAGAlQgoAALASIcVHYjGpri7xEQCAoCOk+ERtrVRaKk2fnvhYW2u6RQAAeIuQ4gOxmLRggdTWlvi8rU1auJARFQBAsBFSfGDPnmMBpV1ra+JOwgAABBUhxQfKy6XcTmcqL08qKzPTHgAAsoGQ4qFMFboWF0tr1yaCiZT4uGZN4nkAAIKKkOKRTBe6VldLDQ2J0NPQkPgcAIAgy3EcxzHdiP6Ix+OKRqNqampSQUGB6eakiMUSwaRjHUleXiJcMPoBAAgzN9dvRlI8QKErAAADR0jxAIWuAAAMHCHFA34tdGVHWwCATQgpHvFboSs72gIAbEPhLCj0BQBkDYWzcIVCXwCAjQgpoNAXAGAlQgp8W+gLAAi2QaYbADtUV0tVVYkpnrIyAgoAwDxCCpKKiwknAAB7MN0DAACsREgBAABWMhZSVq9erYkTJ6qgoEAFBQWqqKjQiy++aKo5AADAMsZCSnFxsZYtW6adO3fq3Xff1fTp0zV79mx9+OGHppoEAAAsYtWOs8OHD9cvf/lLVafZQ76lpUUtLS3Jz+PxuEpKSthxFgAAH/HdjrOtra3auHGjjh49qoqKirTH1NTUKBqNJh8lJSVZbiUAAMgmoyMpu3fvVkVFhb7++msdf/zx2rBhg77//e+nPZaRFAAA/M/NSIrRfVLGjh2rXbt2qampSU8++aTmz5+vrVu36swzz+xybCQSUSQSMdBKAABgglU1KZWVlRozZozWrFnT67HcBRkAAP/xXU1Ku7a2tpQpHQAAEF7GpnuWLFmimTNnatSoUWpubtaGDRtUX1+vLVu2mGoSAACwiLGQcujQIf3whz/UgQMHFI1GNXHiRG3ZskWXXnppn76/fZYqHo972UwAAJBB7dftvlSbWFWT4kYsFmMZMgAAPtXY2KjiXu5q69uQ0tbWpv3792vYsGHKyckZ8M9rX9Lc2NgY6ELcMPSTPgZHGPoZhj5K4ehnGPooDbyfjuOoublZRUVFys3tuTTW6BLkgcjNze01gfVH+72Egi4M/aSPwRGGfoahj1I4+hmGPkoD62c0Gu3TcVat7gEAAGhHSAEAAFYipPy/SCSipUuXBn5X2zD0kz4GRxj6GYY+SuHoZxj6KGW3n74tnAUAAMHGSAoAALASIQUAAFiJkAIAAKxESAEAAFYKdEhZtWqVTjvtNA0ePFhTp07VO++80+Px//qv/6px48Zp8ODBmjBhgl544YWUrzuOozvvvFMjR47UkCFDVFlZqT179njZhV656eNDDz2kiy66SCeeeKJOPPFEVVZWdjn+uuuuU05OTsrjsssu87obvXLTz3Xr1nXpw+DBg1OO8fu5nDZtWpc+5uTkaNasWcljbDuXr7/+ui6//HIVFRUpJydHmzdv7vV76uvrdc455ygSiaisrEzr1q3rcozbf+dec9vPp59+WpdeeqlOOeUUFRQUqKKiosuNVu+6664u53LcuHEe9qJnbvtYX1+f9vf14MGDKcfZdC7d9jHdv7ecnByNHz8+eYxt57GmpkbnnXeehg0bplNPPVVz5szRRx991Ov3ZfNaGdiQ8i//8i+69dZbtXTpUr333nuaNGmSqqqqdOjQobTHb9u2TfPmzVN1dbXef/99zZkzR3PmzNEHH3yQPObee+/VAw88oAcffFBvv/22hg4dqqqqKn399dfZ6lYKt32sr6/XvHnzVFdXp+3bt6ukpEQzZszQZ599lnLcZZddpgMHDiQfjz/+eDa60y23/ZQSOyF27MMnn3yS8nW/n8unn346pX8ffPCB8vLydMUVV6QcZ9O5PHr0qCZNmqRVq1b16fh9+/Zp1qxZuuSSS7Rr1y4tXrxYN954Y8oFvD+/G15z28/XX39dl156qV544QXt3LlTl1xyiS6//HK9//77KceNHz8+5Vy+8cYbXjS/T9z2sd1HH32U0odTTz01+TXbzqXbPv7jP/5jSt8aGxs1fPjwLv8mbTqPW7du1aJFi/TWW2/plVde0bfffqsZM2bo6NGj3X5P1q+VTkBNmTLFWbRoUfLz1tZWp6ioyKmpqUl7/JVXXunMmjUr5bmpU6c6CxcudBzHcdra2pzCwkLnl7/8ZfLrhw8fdiKRiPP444970IPeue1jZ3/605+cYcOGOevXr08+N3/+fGf27NmZbuqAuO3nww8/7ESj0W5/XhDP5f333+8MGzbMOXLkSPI5G89lO0nOpk2bejzmZz/7mTN+/PiU56666iqnqqoq+flAXzev9aWf6Zx55pnO3Xffnfx86dKlzqRJkzLXsAzqSx/r6uocSc4XX3zR7TE2n8v+nMdNmzY5OTk5TkNDQ/I5m8+j4zjOoUOHHEnO1q1buz0m29fKQI6kfPPNN9q5c6cqKyuTz+Xm5qqyslLbt29P+z3bt29POV6Sqqqqksfv27dPBw8eTDkmGo1q6tSp3f5ML/Wnj519+eWX+vbbbzV8+PCU5+vr63Xqqadq7Nixuummm/THP/4xo213o7/9PHLkiEpLS1VSUqLZs2frww8/TH4tiOeytrZWc+fO1dChQ1Oet+lcutXbv8lMvG42amtrU3Nzc5d/l3v27FFRUZFOP/10XX311fr0008NtbD/zj77bI0cOVKXXnqp3nzzzeTzQTyXtbW1qqysVGlpacrzNp/HpqYmSeryu9dRtq+VgQwp//M//6PW1laNGDEi5fkRI0Z0mQNtd/DgwR6Pb//o5md6qT997OznP/+5ioqKUn6ZLrvsMj3yyCN67bXXtHz5cm3dulUzZ85Ua2trRtvfV/3p59ixY/WrX/1KzzzzjB577DG1tbXp/PPPVywWkxS8c/nOO+/ogw8+0I033pjyvG3n0q3u/k3G43F99dVXGfk3YKP77rtPR44c0ZVXXpl8burUqVq3bp1eeuklrV69Wvv27dNFF12k5uZmgy3tu5EjR+rBBx/UU089paeeekolJSWaNm2a3nvvPUmZ+Xtmk/379+vFF1/s8m/S5vPY1tamxYsX64ILLtBZZ53V7XHZvlb69i7IGJhly5Zp48aNqq+vTykqnTt3bvK/J0yYoIkTJ2rMmDGqr6/X9773PRNNda2iokIVFRXJz88//3x997vf1Zo1a3TPPfcYbJk3amtrNWHCBE2ZMiXl+SCcy7DZsGGD7r77bj3zzDMp9RozZ85M/vfEiRM1depUlZaW6oknnlB1dbWJproyduxYjR07Nvn5+eefr48//lj333+/Hn30UYMt88b69et1wgknaM6cOSnP23weFy1apA8++MBojUw6gRxJOfnkk5WXl6fPP/885fnPP/9chYWFab+nsLCwx+PbP7r5mV7qTx/b3XfffVq2bJlefvllTZw4scdjTz/9dJ188snau3fvgNvcHwPpZ7vjjjtOkydPTvYhSOfy6NGj2rhxY5/+wJk+l25192+yoKBAQ4YMycjvhk02btyoG2+8UU888USX4fTOTjjhBJ1xxhm+OZfpTJkyJdn+IJ1Lx3H0q1/9Stdee63y8/N7PNaW8/jjH/9Yzz33nOrq6lRcXNzjsdm+VgYypOTn5+vP//zP9dprryWfa2tr02uvvZbyDrujioqKlOMl6ZVXXkkeP3r0aBUWFqYcE4/H9fbbb3f7M73Unz5Kiarre+65Ry+99JLOPffcXv8/sVhMf/zjHzVy5MiMtNut/vazo9bWVu3evTvZh6CcSymxFLClpUXXXHNNr/8f0+fSrd7+TWbid8MWjz/+uK6//no9/vjjKcvIu3PkyBF9/PHHvjmX6ezatSvZ/iCdy61bt2rv3r19euNg+jw6jqMf//jH2rRpk37zm99o9OjRvX5P1q+VrkttfWLjxo1OJBJx1q1b5/znf/6ns2DBAueEE05wDh486DiO41x77bXO7bffnjz+zTffdAYNGuTcd999zu9+9ztn6dKlznHHHefs3r07ecyyZcucE044wXnmmWec//iP/3Bmz57tjB492vnqq6+y3j/Hcd/HZcuWOfn5+c6TTz7pHDhwIPlobm52HMdxmpubnZ/+9KfO9u3bnX379jmvvvqqc8455zjl5eXO119/baSPjuO+n3fffbezZcsW5+OPP3Z27tzpzJ071xk8eLDz4YcfJo/x+7lsd+GFFzpXXXVVl+dtPJfNzc3O+++/77z//vuOJGfFihXO+++/73zyySeO4zjO7bff7lx77bXJ4//whz84f/Znf+bcdtttzu9+9ztn1apVTl5envPSSy8lj+ntdTPBbT9//etfO4MGDXJWrVqV8u/y8OHDyWN+8pOfOPX19c6+ffucN99806msrHROPvlk59ChQ1nvn+O47+P999/vbN682dmzZ4+ze/du55ZbbnFyc3OdV199NXmMbefSbR/bXXPNNc7UqVPT/kzbzuNNN93kRKNRp76+PuV378svv0weY/paGdiQ4jiO80//9E/OqFGjnPz8fGfKlCnOW2+9lfzaxRdf7MyfPz/l+CeeeMI544wznPz8fGf8+PHO888/n/L1trY254477nBGjBjhRCIR53vf+57z0UcfZaMr3XLTx9LSUkdSl8fSpUsdx3GcL7/80pkxY4ZzyimnOMcdd5xTWlrq/OhHPzL6B7+dm34uXrw4eeyIESOc73//+857772X8vP8fi4dx3H+67/+y5HkvPzyy11+lo3nsn0ZaudHe7/mz5/vXHzxxV2+5+yzz3by8/Od008/3Xn44Ye7/NyeXjcT3Pbz4osv7vF4x0ksvR45cqSTn5/vfOc733GuuuoqZ+/evdntWAdu+7h8+XJnzJgxzuDBg53hw4c706ZNc37zm990+bk2ncv+/L4ePnzYGTJkiLN27dq0P9O285iuf5JS/p2Zvlbm/H9DAQAArBLImhQAAOB/hBQAAGAlQgoAALASIQUAAFiJkAIAAKxESAEAAFYipAAAACsRUgAAgJUIKQAAwEqEFAAAYCVCCgAAsBIhBYAV/vu//1uFhYX6xS9+kXxu27Ztys/P73JreADhwA0GAVjjhRde0Jw5c7Rt2zaNHTtWZ599tmbPnq0VK1aYbhoAAwgpAKyyaNEivfrqqzr33HO1e/du7dixQ5FIxHSzABhASAFgla+++kpnnXWWGhsbtXPnTk2YMMF0kwAYQk0KAKt8/PHH2r9/v9ra2tTQ0GC6OQAMYiQFgDW++eYbTZkyRWeffbbGjh2rlStXavfu3Tr11FNNNw2AAYQUANa47bbb9OSTT+q3v/2tjj/+eF188cWKRqN67rnnTDcNgAFM9wCwQn19vVauXKlHH31UBQUFys3N1aOPPqp///d/1+rVq003D4ABjKQAAAArMZICAACsREgBAABWIqQAAAArEVIAAICVCCkAAMBKhBQAAGAlQgoAALASIQUAAFiJkAIAAKxESAEAAFYipAAAACv9H24qu0yBsLvvAAAAAElFTkSuQmCC\n"
          },
          "metadata": {}
        }
      ]
    },
    {
      "cell_type": "markdown",
      "metadata": {
        "id": "ScwxpouoDDyZ"
      },
      "source": [
        "## Cost Function\n",
        "\n",
        "The equation for calculating cost function is as shown below. The cost function is only for linear regression. For other algorithms, the cost function will be different and the gradients would have to be derived from the cost functions\n",
        "\n",
        "\\begin{equation}\n",
        "J(y_{pred}) = \\frac{1}{2} m \\sum_{i=1}^{m} (h(y_{pred})^{(i)} - y^{(i)})^2\n",
        "\\end{equation}"
      ]
    },
    {
      "cell_type": "code",
      "metadata": {
        "id": "PUeTUAXH7ZaV"
      },
      "source": [
        "def cal_cost(y_pred, X, y):\n",
        "    '''\n",
        "    Calculates the cost for given X and Y.\n",
        "    y_pred = Vector of y_preds\n",
        "    X = Row of X's np.zeros((2, j))\n",
        "    y = Actual y's np.zeros((2, 1))\n",
        "\n",
        "    where:\n",
        "        j is the no of features\n",
        "    '''\n",
        "\n",
        "    m = len(y)\n",
        "\n",
        "    predictions = X.dot(y_pred)\n",
        "    cost = (1 / 2 * m) * np.sum(np.square(predictions - y))\n",
        "\n",
        "    return cost"
      ],
      "execution_count": 5,
      "outputs": []
    },
    {
      "cell_type": "markdown",
      "metadata": {
        "id": "FcXqsVNpDbKC"
      },
      "source": [
        "## Gradients\n",
        "\n",
        "\\begin{equation}\n",
        "y_{pred_0}: = y_{pred_0} -\\alpha . (1/m .\\sum_{i=1}^{m}(h(y_{pred}^{(i)} - y^{(i)}).X_0^{(i)})\n",
        "\\end{equation}\n",
        "\\begin{equation}\n",
        "y_{pred_1}: = y_{pred_1} -\\alpha . (1/m .\\sum_{i=1}^{m}(h(y_{pred}^{(i)} - y^{(i)}).X_0^{(i)})\n",
        "\\end{equation}\n",
        ".\n",
        "\n",
        ".\n",
        "\n",
        ".\n",
        "\n",
        ".\n",
        "\n",
        "\\begin{equation}\n",
        "y_{pred_j}: = y_{pred_j} -\\alpha . (1/m .\\sum_{i=1}^{m}(h(y_{pred}^{(i)} - y^{(i)}).X_0^{(i)})\n",
        "\\end{equation}"
      ]
    },
    {
      "cell_type": "code",
      "metadata": {
        "id": "fwxBFXP88NBW"
      },
      "source": [
        "def gradient_descent(X, y, y_pred, learning_rate=0.01, iterations=100):\n",
        "    '''\n",
        "    X = Matrix of X with added bias units\n",
        "    y = Vector of Y\n",
        "    y_pred = Vector of y_preds np.random.randn(j, 1)\n",
        "    learning_rate\n",
        "    iterations = no of iterations\n",
        "\n",
        "    Returns the final y_pred vector and array of cost history over no of iterations\n",
        "    '''\n",
        "\n",
        "    m = len(y)\n",
        "    cost_history = np.zeros(iterations)\n",
        "    y_pred_history = np.zeros((iterations, 2))\n",
        "\n",
        "    for it in range(iterations):\n",
        "        prediction = np.dot(X, y_pred)\n",
        "        y_pred = y_pred - (1 / m) * learning_rate * (X.T.dot((prediction - y)))\n",
        "        y_pred_history[it,:] = y_pred.T\n",
        "        cost_history[it]  = cal_cost(y_pred, X, y)\n",
        "\n",
        "    return y_pred, cost_history, y_pred_history"
      ],
      "execution_count": 6,
      "outputs": []
    },
    {
      "cell_type": "markdown",
      "metadata": {
        "id": "1iSohSB2EtK1"
      },
      "source": [
        "Let's do 1000 iterations with a learning rate of 0.01.\n",
        "We will start with a random prediction."
      ]
    },
    {
      "cell_type": "code",
      "metadata": {
        "colab": {
          "base_uri": "https://localhost:8080/"
        },
        "id": "18AX7hrU8bv5",
        "outputId": "5d3dd492-f1af-4051-89d9-7029d80d7a79"
      },
      "source": [
        "lr = 0.01\n",
        "n_iter = 1000\n",
        "\n",
        "y_pred = np.random.randn(2,1)\n",
        "X_b = np.c_[np.ones((len(X), 1)), X]\n",
        "y_pred, cost_history, y_pred_history = gradient_descent(X_b, y, y_pred, lr, n_iter)\n",
        "\n",
        "print('y_pred[0]: {:0.3f}\\ny_pred[1]: {:0.3f}'.format(y_pred[0][0], y_pred[1][0]))\n",
        "print('Final error: {:0.3f}'.format(cost_history[-1]))"
      ],
      "execution_count": 7,
      "outputs": [
        {
          "output_type": "stream",
          "name": "stdout",
          "text": [
            "y_pred[0]: 3.903\n",
            "y_pred[1]: 3.046\n",
            "Final error: 4180.375\n"
          ]
        }
      ]
    },
    {
      "cell_type": "markdown",
      "metadata": {
        "id": "m7fao2MaE216"
      },
      "source": [
        "Plotting the error vs Number of iterations"
      ]
    },
    {
      "cell_type": "code",
      "metadata": {
        "colab": {
          "base_uri": "https://localhost:8080/",
          "height": 713
        },
        "id": "DrkrAAbk8hIs",
        "outputId": "a00e9630-d94b-4d8d-efff-c613201bd589"
      },
      "source": [
        "fig, ax = plt.subplots(figsize=(12,8))\n",
        "\n",
        "ax.set_ylabel('Error')\n",
        "ax.set_xlabel('Number of iterations')\n",
        "\n",
        "ax.plot(range(n_iter), cost_history, 'b.')"
      ],
      "execution_count": 8,
      "outputs": [
        {
          "output_type": "execute_result",
          "data": {
            "text/plain": [
              "[<matplotlib.lines.Line2D at 0x799043f282b0>]"
            ]
          },
          "metadata": {},
          "execution_count": 8
        },
        {
          "output_type": "display_data",
          "data": {
            "text/plain": [
              "<Figure size 1200x800 with 1 Axes>"
            ],
            "image/png": "iVBORw0KGgoAAAANSUhEUgAABAcAAAKnCAYAAAD6CCSWAAAAOXRFWHRTb2Z0d2FyZQBNYXRwbG90bGliIHZlcnNpb24zLjcuMSwgaHR0cHM6Ly9tYXRwbG90bGliLm9yZy/bCgiHAAAACXBIWXMAAA9hAAAPYQGoP6dpAABf00lEQVR4nO3de3xU9b3v/3cmgQTBJFwkIXtCoCblohRUFOLdmMMo1G6OtAVNKdVsE7tBuYlAlUutCoJXlEvSk1M4v3orjy1sRYWTQoQqESEQEQQMLQojBrQhGaECIbN+f8zJgiEBkpDMWpP1ej4e81jOd30y85mwWp036/v9RhiGYQgAAAAAADiWy+oGAAAAAACAtQgHAAAAAABwOMIBAAAAAAAcjnAAAAAAAACHIxwAAAAAAMDhCAcAAAAAAHA4wgEAAAAAAByOcAAAAAAAAIeLsroBJ/H7/Tp48KAuvfRSRUREWN0OAAAAAKCVMwxD33//vZKSkuRynfv+AMKBEDp48KCSk5OtbgMAAAAA4DAHDhyQ2+0+53nCgRC69NJLJQX+UGJjYy3uBgAAAADQ2vl8PiUnJ5vfR8+FcCCEaqcSxMbGEg4AAAAAAELmQlPbWZAQAAAAAACHIxwAAAAAAMDhCAcAAAAAAHA4wgEAAAAAAByOcAAAAAAAAIcjHAAAAAAAwOEIBwAAAAAAcDjCAQAAAAAAHI5wAAAAAAAAhyMcAAAAAADA4QgHAAAAAABwOMIBAAAAAAAcjnAAAAAAAACHIxwAAAAAAMDhCAcAAAAAAHA4wgEAAAAAAByOcAAAAAAAAIcjHAAAAAAAwOEIBwAAAAAAcDjCAQAAAAAAHI5wAAAAAAAAhyMcAAAAAADA4QgHUIfXKxUVBY4AAAAAgNaPcABBCgqklBQpIyNwLCiwuiMAAAAAQEsjHIDJ65VyciS/P/Dc75dyc7mDAAAAAABaO8IBmMrKTgcDtWpqpL17rekHAAAAABAahAMwpaVJrrOuiMhIKTXVmn4AAAAAAKFBOACT2y3l5wcCASlwzMsLjAMAAAAAWq8oqxuAvWRnSx5PYCpBairBAAAAAAA4AeEA6nC7CQUAAAAAwEmYVgAAAAAAgMMRDgAAAAAA4HCEAwAAAAAAOBzhAAAAAAAADkc4AAAAAACAwxEOAAAAAADgcIQDAAAAAAA4HOEAAAAAAAAORzgAAAAAAIDDEQ4AAAAAAOBwhAMAAAAAADgc4QAAAAAAAA5HOAAAAAAAgMMRDgAAAAAA4HCEA6iX1ysVFQWOAAAAAIDWjXAAdRQUSCkpUkZG4FhQYHVHAAAAAICWRDiAIF6vlJMj+f2B536/lJvLHQQAAAAA0JoRDiBIWdnpYKBWTY20d681/QAAAAAAWh7hAIKkpUmus66KyEgpNdWafgAAAAAALY9wAEHcbik/PxAISIFjXl5gHAAAAADQOkVZ3QDsJztb8ngCUwlSUwkGAAAAAKC1IxxAvdxuQgEAAAAAcAqmFQAAAAAA4HCEAwAAAAAAOBzhAAAAAAAADmdpOLBhwwbdddddSkpKUkREhFauXHnO2gcffFARERF68cUXg8YrKiqUlZWl2NhYxcfHKzs7W0ePHg2q2b59u2666SbFxMQoOTlZ8+bNq/P6y5cvV+/evRUTE6N+/frpvffeCzpvGIZmzpypbt26qV27dsrMzFRZWVmTPzsAAAAAAHZhaThw7Ngx9e/fXwsXLjxv3YoVK/Txxx8rKSmpzrmsrCzt3LlThYWFWrVqlTZs2KCcnBzzvM/n05AhQ5SSkqKSkhLNnz9fs2fPVn5+vlmzceNG3XPPPcrOzta2bds0fPhwDR8+XDt27DBr5s2bpwULFmjJkiXatGmT2rdvL4/Ho+PHjzfDbwIAAAAAAOtEGIZhWN2EJEVERGjFihUaPnx40PjXX3+tQYMGac2aNRo2bJgmTJigCRMmSJJ27dqlvn37avPmzRo4cKAkafXq1Ro6dKi8Xq+SkpK0ePFiPfbYYyovL1fbtm0lSdOmTdPKlSu1e/duSdLIkSN17NgxrVq1ynzfwYMHa8CAAVqyZIkMw1BSUpImT56sRx55RJJUVVWlhIQELV26VKNGjWrQZ/T5fIqLi1NVVZViY2Mv5tcFAAAAAMAFNfR7qK3XHPD7/Ro9erSmTJmiK664os754uJixcfHm8GAJGVmZsrlcmnTpk1mzc0332wGA5Lk8Xi0Z88eHTlyxKzJzMwMem2Px6Pi4mJJ0r59+1ReXh5UExcXp0GDBpk1AAAAAACEqyirGzifZ555RlFRUXr44YfrPV9eXq6uXbsGjUVFRalTp04qLy83a3r27BlUk5CQYJ7r2LGjysvLzbEza858jTN/rr6a+pw4cUInTpwwn/t8vnPWAgAAAABgFdveOVBSUqKXXnpJS5cuVUREhNXtNMmcOXMUFxdnPpKTk61uCQAAAACAOmwbDvztb3/T4cOH1b17d0VFRSkqKkpfffWVJk+erB49ekiSEhMTdfjw4aCfO3XqlCoqKpSYmGjWHDp0KKim9vmFas48f+bP1VdTn+nTp6uqqsp8HDhwoDG/AgAAAAAAQsK24cDo0aO1fft2lZaWmo+kpCRNmTJFa9askSSlp6ersrJSJSUl5s+tW7dOfr9fgwYNMms2bNig6upqs6awsFC9evVSx44dzZq1a9cGvX9hYaHS09MlST179lRiYmJQjc/n06ZNm8ya+kRHRys2NjboAQAAAACA3Vi65sDRo0e1d+9e8/m+fftUWlqqTp06qXv37urcuXNQfZs2bZSYmKhevXpJkvr06aM77rhDDzzwgJYsWaLq6mqNGzdOo0aNMrc9vPfee/X73/9e2dnZmjp1qnbs2KGXXnpJL7zwgvm648eP1y233KLnnntOw4YN0xtvvKEtW7aY2x1GRERowoQJevLJJ5WWlqaePXtqxowZSkpKqrO7AgAAAAAA4cbScGDLli267bbbzOeTJk2SJI0ZM0ZLly5t0Gu8+uqrGjdunG6//Xa5XC6NGDFCCxYsMM/HxcXp//7f/6uxY8fqmmuuUZcuXTRz5kzl5OSYNddff71ee+01Pf744/rd736ntLQ0rVy5UldeeaVZ8+ijj+rYsWPKyclRZWWlbrzxRq1evVoxMTEX+VsAAAAAAMBaEYZhGFY34RQN3V/SLrxeqaxMSkuT3G6ruwEAAAAANFZDv4fads0BWKugQEpJkTIyAseCAqs7AgAAAAC0FMIB1OH1Sjk5kt8feO73S7m5gXEAAAAAQOtDOIA6yspOBwO1amqkM9aOBAAAAAC0IoQDqCMtTXKddWVERkqpqdb0AwAAAABoWYQDqMPtlvLzA4GAFDjm5bEoIQAAAAC0VpZuZQj7ys6WPJ7AVILUVIIBAAAAAGjNCAdwTm43oQAAAAAAOAHTCgAAAAAAcDjCAQAAAAAAHI5wAAAAAAAAhyMcAAAAAADA4QgHAAAAAABwOMIBAAAAAAAcjnAAAAAAAACHIxwAAAAAAMDhCAcAAAAAAHA4wgEAAAAAAByOcAAAAAAAAIcjHAAAAAAAwOEIBwAAAAAAcDjCAZyX1ysVFQWOAAAAAIDWiXAA51RQIKWkSBkZgWNBgdUdAQAAAABaAuEA6uX1Sjk5kt8feO73S7m53EEAAAAAAK0R4QDqVVZ2OhioVVMj7d1rTT8AAAAAgJZDOIB6paVJrrOujshIKTXVmn4AAAAAAC2HcAD1crul/PxAICAFjnl5gXEAAAAAQOsSZXUDsK/sbMnjCUwlSE0lGAAAAACA1opwAOfldhMKAAAAAEBrx7QCAAAAAAAcjnAAAAAAAACHIxwAAAAAAMDhCAcAAAAAAHA4wgEAAAAAAByOcAAAAAAAAIcjHAAAAAAAwOEIBwAAAAAAcDjCAQAAAAAAHI5wAAAAAAAAhyMcAAAAAADA4QgHAAAAAABwOMIBXJDXKxUVBY4AAAAAgNaHcADnVVAgpaRIGRmBY0GB1R0BAAAAAJob4QDOyeuVcnIkvz/w3O+XcnO5gwAAAAAAWhvCAZxTWdnpYKBWTY20d681/QAAAAAAWgbhAM4pLU1ynXWFREZKqanW9AMAAAAAaBmEAzgnt1vKzw8EAlLgmJcXGAcAAAAAtB5RVjcAe8vOljyewFSC1FSCAQAAAABojQgHcEFuN6EAAAAAALRmTCsAAAAAAMDhCAcAAAAAAHA4wgEAAAAAAByOcAAAAAAAAIcjHAAAAAAAwOEIBwAAAAAAcDjCAQAAAAAAHI5wAAAAAAAAhyMcAAAAAADA4QgHAAAAAABwOMIBXJDXKxUVBY4AAAAAgNaHcADnVVAgpaRIGRmBY0GB1R0BAAAAAJob4QDOyeuVcnIkvz/w3O+XcnO5gwAAAAAAWhvCAZxTWdnpYKBWTY20d681/QAAAAAAWgbhAM4pLU1ynXWFREZKqanW9AMAAAAAaBmEAzgnt1vKzw8EAlLgmJcXGAcAAAAAtB5RVjcAe8vOljyewFSC1FSCAQAAAABojQgHcEFuN6EAAAAAALRmTCsAAAAAAMDhLA0HNmzYoLvuuktJSUmKiIjQypUrzXPV1dWaOnWq+vXrp/bt2yspKUm//vWvdfDgwaDXqKioUFZWlmJjYxUfH6/s7GwdPXo0qGb79u266aabFBMTo+TkZM2bN69OL8uXL1fv3r0VExOjfv366b333gs6bxiGZs6cqW7duqldu3bKzMxUWVlZ8/0yAAAAAACwiKXhwLFjx9S/f38tXLiwzrl//etf2rp1q2bMmKGtW7fqrbfe0p49e/Szn/0sqC4rK0s7d+5UYWGhVq1apQ0bNignJ8c87/P5NGTIEKWkpKikpETz58/X7NmzlZ+fb9Zs3LhR99xzj7Kzs7Vt2zYNHz5cw4cP144dO8yaefPmacGCBVqyZIk2bdqk9u3by+Px6Pjx4y3wmwEAAAAAIHQiDMMwrG5CkiIiIrRixQoNHz78nDWbN2/Wddddp6+++krdu3fXrl271LdvX23evFkDBw6UJK1evVpDhw6V1+tVUlKSFi9erMcee0zl5eVq27atJGnatGlauXKldu/eLUkaOXKkjh07plWrVpnvNXjwYA0YMEBLliyRYRhKSkrS5MmT9cgjj0iSqqqqlJCQoKVLl2rUqFEN+ow+n09xcXGqqqpSbGxsU35NAAAAAAA0WEO/h4bVmgNVVVWKiIhQfHy8JKm4uFjx8fFmMCBJmZmZcrlc2rRpk1lz8803m8GAJHk8Hu3Zs0dHjhwxazIzM4Pey+PxqLi4WJK0b98+lZeXB9XExcVp0KBBZk19Tpw4IZ/PF/QAAAAAAMBuwiYcOH78uKZOnap77rnHTDvKy8vVtWvXoLqoqCh16tRJ5eXlZk1CQkJQTe3zC9Wcef7Mn6uvpj5z5sxRXFyc+UhOTm7UZwYAAAAAIBTCIhyorq7WL3/5SxmGocWLF1vdToNNnz5dVVVV5uPAgQNWtwQAAAAAQB1RVjdwIbXBwFdffaV169YFzZFITEzU4cOHg+pPnTqliooKJSYmmjWHDh0Kqql9fqGaM8/XjnXr1i2oZsCAAefsPTo6WtHR0Y35uAAAAAAAhJyt7xyoDQbKysr017/+VZ07dw46n56ersrKSpWUlJhj69atk9/v16BBg8yaDRs2qLq62qwpLCxUr1691LFjR7Nm7dq1Qa9dWFio9PR0SVLPnj2VmJgYVOPz+bRp0yazBgAAAACAcGVpOHD06FGVlpaqtLRUUmDhv9LSUu3fv1/V1dX6+c9/ri1btujVV19VTU2NysvLVV5erpMnT0qS+vTpozvuuEMPPPCAPvnkE3300UcaN26cRo0apaSkJEnSvffeq7Zt2yo7O1s7d+7Um2++qZdeekmTJk0y+xg/frxWr16t5557Trt379bs2bO1ZcsWjRs3TlJgJ4UJEyboySef1Ntvv63PPvtMv/71r5WUlHTe3RUAAAAAAAgHlm5l+MEHH+i2226rMz5mzBjNnj1bPXv2rPfnioqKdOutt0qSKioqNG7cOL3zzjtyuVwaMWKEFixYoA4dOpj127dv19ixY7V582Z16dJFDz30kKZOnRr0msuXL9fjjz+uL7/8UmlpaZo3b56GDh1qnjcMQ7NmzVJ+fr4qKyt14403atGiRfrxj3/c4M8bzlsZer1SWZmUlia53VZ3AwAAAABoiIZ+D7U0HHCacA0HCgqknBzJ75dcLik/X8rOtrorAAAAAMCFNPR7qK3XHID1vN7TwYAUOObmBsYBAAAAAK0D4QDOq6zsdDBQq6ZG2rvXmn4AAAAAAM2PcADnlZYWmEpwpshIKTXVmn4AAAAAAM2PcADn5XYH1hiIjAw8j4yU8vJYlBAAAAAAWpMoqxuA/WVnSx5PYCpBairBAAAAAAC0NoQDaBC3m1AAAAAAAForphUAAAAAAOBwhAMAAAAAADgc4QAAAAAAAA5HOAAAAAAAgMMRDgAAAAAA4HCEAwAAAAAAOBzhAAAAAAAADkc4AAAAAACAwxEOAAAAAADgcIQDAAAAAAA4HOEAGszrlYqKAkcAAAAAQOtBOIAGKSiQUlKkjIzAsaDA6o4AAAAAAM2FcAAX5PVKOTmS3x947vdLubncQQAAAAAArQXhAC6orOx0MFCrpkbau9eafgAAAAAAzYtwABeUlia5zrpSIiOl1FRr+gEAAAAANC/CAVyQ2y3l5wcCASlwzMsLjAMAAAAAwl+U1Q0gPGRnSx5PYCpBairBAAAAAAC0JoQDaDC3m1AAAAAAAFojphUAAAAAAOBwhAMAAAAAADgc4QAAAAAAAA5HOAAAAAAAgMMRDgAAAAAA4HCEAwAAAAAAOBzhAAAAAAAADkc4AAAAAACAwxEOoFG8XqmoKHAEAAAAALQOhANosIICKSVFysgIHAsKrO4IAAAAANAcCAfQIF6vlJMj+f2B536/lJvLHQQAAAAA0BoQDqBByspOBwO1amqkvXut6QcAAAAA0HwIB9AgaWmS66yrJTJSSk21ph8AAAAAQPMhHECDuN1Sfn4gEJACx7y8wDgAAAAAILxFWd0Awkd2tuTxBKYSpKYSDAAAAABAa0E4gEZxuwkFAAAAAKC1YVoBAAAAAAAORzgAAAAAAIDDEQ4AAAAAAOBwhAMAAAAAADgc4QAAAAAAAA5HOAAAAAAAgMMRDgAAAAAA4HCEAwAAAAAAOBzhABrN65WKigJHAAAAAED4IxxAoxQUSCkpUkZG4FhQYHVHAAAAAICLRTiABvN6pZwcye8PPPf7pdxc7iAAAAAAgHBHOIAGKys7HQzUqqmR9u61ph8AAAAAQPMgHECDpaVJrrOumMhIKTXVmn4AAAAAAM2DcAAN5nZL+fmBQEAKHPPyAuMAAAAAgPAVZXUDCC/Z2ZLHE5hKkJpKMAAAAAAArQHhABrN7SYUAAAAAIDWhGkFAAAAAAA4HOEAAAAAAAAORzgAAAAAAIDDEQ4AAAAAAOBwhAMAAAAAADgc4QAAAAAAAA5HOAAAAAAAgMMRDgAAAAAA4HCEA2gSr1cqKgocAQAAAADhjXAAjVZQIKWkSBkZgWNBgdUdAQAAAAAuhqXhwIYNG3TXXXcpKSlJERERWrlyZdB5wzA0c+ZMdevWTe3atVNmZqbKysqCaioqKpSVlaXY2FjFx8crOztbR48eDarZvn27brrpJsXExCg5OVnz5s2r08vy5cvVu3dvxcTEqF+/fnrvvfca3YsTeL1STo7k9wee+/1Sbi53EAAAAABAOLM0HDh27Jj69++vhQsX1nt+3rx5WrBggZYsWaJNmzapffv28ng8On78uFmTlZWlnTt3qrCwUKtWrdKGDRuUk5Njnvf5fBoyZIhSUlJUUlKi+fPna/bs2crPzzdrNm7cqHvuuUfZ2dnatm2bhg8fruHDh2vHjh2N6sUJyspOBwO1amqkvXut6QcAAAAAcPEiDMMwrG5CkiIiIrRixQoNHz5cUuBv6pOSkjR58mQ98sgjkqSqqiolJCRo6dKlGjVqlHbt2qW+fftq8+bNGjhwoCRp9erVGjp0qLxer5KSkrR48WI99thjKi8vV9u2bSVJ06ZN08qVK7V7925J0siRI3Xs2DGtWrXK7Gfw4MEaMGCAlixZ0qBeGsLn8ykuLk5VVVWKjY1tlt9bqHm9gakEZwYEkZHSl19KbrdlbQEAAAAA6tHQ76G2XXNg3759Ki8vV2ZmpjkWFxenQYMGqbi4WJJUXFys+Ph4MxiQpMzMTLlcLm3atMmsufnmm81gQJI8Ho/27NmjI0eOmDVnvk9tTe37NKSX+pw4cUI+ny/oEe7cbik/PxAISIFjXh7BAAAAAACEM9uGA+Xl5ZKkhISEoPGEhATzXHl5ubp27Rp0PioqSp06dQqqqe81znyPc9Wcef5CvdRnzpw5iouLMx/JyckX+NThITs7cKdAUVHgmJ1tdUcAAAAAgIth23CgNZg+fbqqqqrMx4EDB6xuqdm43dKtt3LHAAAAAAC0BrYNBxITEyVJhw4dCho/dOiQeS4xMVGHDx8OOn/q1ClVVFQE1dT3Gme+x7lqzjx/oV7qEx0drdjY2KAHAAAAAAB2Y9twoGfPnkpMTNTatWvNMZ/Pp02bNik9PV2SlJ6ersrKSpWUlJg169atk9/v16BBg8yaDRs2qLq62qwpLCxUr1691LFjR7PmzPepral9n4b0AgAAAABAuLI0HDh69KhKS0tVWloqKbDwX2lpqfbv36+IiAhNmDBBTz75pN5++2199tln+vWvf62kpCRzR4M+ffrojjvu0AMPPKBPPvlEH330kcaNG6dRo0YpKSlJknTvvfeqbdu2ys7O1s6dO/Xmm2/qpZde0qRJk8w+xo8fr9WrV+u5557T7t27NXv2bG3ZskXjxo2TpAb1AgAAAABAuLJ0K8MPPvhAt912W53xMWPGaOnSpTIMQ7NmzVJ+fr4qKyt14403atGiRfrxj39s1lZUVGjcuHF655135HK5NGLECC1YsEAdOnQwa7Zv366xY8dq8+bN6tKlix566CFNnTo16D2XL1+uxx9/XF9++aXS0tI0b948DR061DzfkF4upDVsZQgAAAAACB8N/R5qaTjgNIQDAAAAAIBQauj3UNuuOQAAAAAAAEKDcABN4vVKRUWBIwAAAAAgvBEOoNEKCqSUFCkjI3AsKLC6IwAAAADAxSAcQKN4vVJOjuT3B577/VJuLncQAAAAAEA4IxxAo5SVnQ4GatXUSHv3WtMPAAAAAODiEQ6gUdLSJNdZV01kpJSaak0/AAAAAICLRziARnG7pfz8QCAgBY55eYFxAAAAAEB4irK6AYSf7GzJ4wlMJUhNJRgAAAAAgHBHOIAmcbsJBQAAAACgtWBaAQAAAAAADkc4AAAAAACAwxEOAAAAAADgcIQDAAAAAAA4HOEAAAAAAAAORzgAAAAAAIDDEQ4AAAAAAOBwhANoMq9XKioKHAEAAAAA4YtwAE1SUCClpEgZGYFjQYHVHQEAAAAAmopwAI3m9Uo5OZLfH3ju90u5udxBAAAAAADhinAAjVZWdjoYqFVTI+3da00/AAAAAICLQziARktLk1xnXTmRkVJqqjX9AAAAAAAuDuEAGs3tlvLzA4GAFDjm5QXGAQAAAADhJ8rqBhCesrMljycwlSA1lWAAAAAAAMIZ4QCazO0mFAAAAACA1oBpBQAAAAAAOBzhAAAAAAAADkc4AAAAAACAwxEOAAAAAADgcIQDAAAAAAA4HOEALorXKxUVBY4AAAAAgPBEOIAmKyiQUlKkjIzAsaDA6o4AAAAAAE1BOIAm8XqlnBzJ7w889/ul3FzuIAAAAACAcEQ4gCYpKzsdDNSqqZH27rWmHwAAAABA0xEOoEnS0iTXWVdPZKSUmmpNPwAAAACApiMcQJO43VJ+fiAQkALHvLzAOAAAAAAgvERZ3QDCV3a25PEEphKkphIMAAAAAEC4IhzARXG7CQUAAAAAINwxrQAAAAAAAIcjHAAAAAAAwOEIBwAAAAAAcDjCAQAAAAAAHI5wAAAAAAAAhyMcwEXzeqWiosARAAAAABB+CAdwUQoKpJQUKSMjcCwosLojAAAAAEBjEQ6gybxeKSdH8vsDz/1+KTeXOwgAAAAAINwQDqDJyspOBwO1amqkvXut6QcAAAAA0DSEA2iytDTJddYVFBkppaZa0w8AAAAAoGkIB9BkbreUnx8IBKTAMS8vMA4AAAAACB9RVjeA8JadLXk8gakEqakEAwAAAAAQjggHcNHcbkIBAAAAAAhnTCsAAAAAAMDhCAcAAAAAAHA4wgEAAAAAAByOcAAAAAAAAIcjHECz8HqloqLAEQAAAAAQXggHcNEKCqSUFCkjI3AsKLC6IwAAAABAYxAO4KJ4vVJOjuT3B577/VJuLncQAAAAAEA4IRzARSkrOx0M1KqpkfbutaYfAAAAAEDjEQ7goqSlSa6zrqLISCk11Zp+AAAAAACNRziAi+J2S/n5gUBAChzz8gLjAAAAAIDwEGV1Awh/2dmSxxOYSpCaSjAAAAAAAOGGcADNwu0mFAAAAACAcMW0AgAAAAAAHI5wAAAAAAAAhyMcAAAAAADA4QgHAAAAAABwOFuHAzU1NZoxY4Z69uypdu3a6fLLL9cf/vAHGYZh1hiGoZkzZ6pbt25q166dMjMzVVZWFvQ6FRUVysrKUmxsrOLj45Wdna2jR48G1Wzfvl033XSTYmJilJycrHnz5tXpZ/ny5erdu7diYmLUr18/vffeey3zwcOU1ysVFQWOAAAAAIDwYetw4JlnntHixYv1yiuvaNeuXXrmmWc0b948vfzyy2bNvHnztGDBAi1ZskSbNm1S+/bt5fF4dPz4cbMmKytLO3fuVGFhoVatWqUNGzYoJyfHPO/z+TRkyBClpKSopKRE8+fP1+zZs5Wfn2/WbNy4Uffcc4+ys7O1bds2DR8+XMOHD9eOHTtC88uwuYICKSVFysgIHAsKrO4IAAAAANBQEcaZfw1vMz/96U+VkJCggjO+aY4YMULt2rXTn//8ZxmGoaSkJE2ePFmPPPKIJKmqqkoJCQlaunSpRo0apV27dqlv377avHmzBg4cKElavXq1hg4dKq/Xq6SkJC1evFiPPfaYysvL1bZtW0nStGnTtHLlSu3evVuSNHLkSB07dkyrVq0yexk8eLAGDBigJUuWNOjz+Hw+xcXFqaqqSrGxsc3yO7IDrzcQCPj9p8ciI6Uvv2R7QwAAAACwUkO/h9r6zoHrr79ea9eu1RdffCFJ+vTTT/Xhhx/qzjvvlCTt27dP5eXlyszMNH8mLi5OgwYNUnFxsSSpuLhY8fHxZjAgSZmZmXK5XNq0aZNZc/PNN5vBgCR5PB7t2bNHR44cMWvOfJ/amtr3cbKysuBgQJJqaqS9e63pBwAAAADQOFGN/YHq6mq1a9dOpaWluvLKK1uiJ9O0adPk8/nUu3dvRUZGqqamRk899ZSysrIkSeXl5ZKkhISEoJ9LSEgwz5WXl6tr165B56OiotSpU6egmp49e9Z5jdpzHTt2VHl5+Xnfpz4nTpzQiRMnzOc+n6/Bnz2cpKVJLlfdOwdSU63rCQAAAADQcI2+c6BNmzbq3r27ampqWqKfIH/5y1/06quv6rXXXtPWrVu1bNkyPfvss1q2bFmLv3dzmDNnjuLi4sxHcnKy1S21CLdbys8PBAJS4JiXx5QCAAAAAAgXTZpW8Nhjj+l3v/udKioqmrufIFOmTNG0adM0atQo9evXT6NHj9bEiRM1Z84cSVJiYqIk6dChQ0E/d+jQIfNcYmKiDh8+HHT+1KlTqqioCKqp7zXOfI9z1dSer8/06dNVVVVlPg4cONCozx9OsrMDawwUFQWO2dlWdwQAAAAAaKgmhQOvvPKKNmzYoKSkJPXq1UtXX3110KO5/Otf/5LLFdxiZGSk/P/v/vWePXsqMTFRa9euNc/7fD5t2rRJ6enpkqT09HRVVlaqpKTErFm3bp38fr8GDRpk1mzYsEHV1dVmTWFhoXr16qWOHTuaNWe+T21N7fvUJzo6WrGxsUGP1sztlm69lTsGAAAAACDcNHrNAUkaPnx4M7dRv7vuuktPPfWUunfvriuuuELbtm3T888/r/vvv1+SFBERoQkTJujJJ59UWlqaevbsqRkzZigpKcnssU+fPrrjjjv0wAMPaMmSJaqurta4ceM0atQoJSUlSZLuvfde/f73v1d2dramTp2qHTt26KWXXtILL7xg9jJ+/Hjdcssteu655zRs2DC98cYb2rJlS9B2hwAAAAAAhCNbb2X4/fffa8aMGVqxYoUOHz6spKQk3XPPPZo5c6a5s4BhGJo1a5by8/NVWVmpG2+8UYsWLdKPf/xj83UqKio0btw4vfPOO3K5XBoxYoQWLFigDh06mDXbt2/X2LFjtXnzZnXp0kUPPfSQpk6dGtTP8uXL9fjjj+vLL79UWlqa5s2bp6FDhzb487TWrQwBAAAAAPbU0O+hFxUOlJSUaNeuXZKkK664QldddVVTX8oRCAcAAAAAAKHU0O+hTZpWcPjwYY0aNUoffPCB4uPjJUmVlZW67bbb9MYbb+iyyy5rUtMAAAAAACD0mrQg4UMPPaTvv/9eO3fuVEVFhSoqKrRjxw75fD49/PDDzd0jwojXG9ixwOu1uhMAAAAAQEM1KRxYvXq1Fi1apD59+phjffv21cKFC/X+++83W3MILwUFUkqKlJEROBYUWN0RAAAAAKAhmhQO+P1+tWnTps54mzZtzG0G4Sxer5STI9X+8fv9Um4udxAAAAAAQDhoUjiQkZGh8ePH6+DBg+bY119/rYkTJ+r2229vtuYQPsrKTgcDtWpqpL17rekHAAAAANBwTQoHXnnlFfl8PvXo0UOXX365Lr/8cvXs2VM+n08vv/xyc/eIMJCWJrnOupoiI6XUVGv6AQAAAAA0XJN2K0hOTtbWrVv117/+Vbt375Yk9enTR5mZmc3aHMKH2y3l5wemEtTUBIKBvLzAOAAAAADA3iIMwzAa8wPV1dVq166dSktLdeWVV7ZUX61SQ/eXDGdeb2AqQWoqwQAAAAAAWK2h30MbfedAmzZt1L17d9XU1FxUg2id3G5CAQAAAAAIN01ac+Cxxx7T7373O1VUVDR3PwAAAAAAIMSatObAK6+8or179yopKUkpKSlq37590PmtW7c2S3MAAAAAAKDlNSkcGD58eDO3AQAAAAAArNLocODUqVOKiIjQ/fffLzeTy3EWr1cqKwtsbcjlAQAAAADhodFrDkRFRWn+/Pk6depUS/SDMFZQIKWkSBkZgWNBgdUdAQAAAAAaokkLEmZkZGj9+vXN3QvCmNcr5eRIfn/gud8v5eYGxgEAAAAA9takNQfuvPNOTZs2TZ999pmuueaaOgsS/uxnP2uW5hA+yspOBwO1amqkvXuZXgAAAAAAdhdhGIbR2B9yuc59w0FERIRqamouqqnWyufzKS4uTlVVVYqNjbW6nWbl9QamEpwZEERGSl9+STgAAAAAAFZp6PfQJk0r8Pv953wQDDiT2y3l5wcCASlwzMsjGAAAAACAcNCocGDo0KGqqqoyn8+dO1eVlZXm83/+85/q27dvszWH8JKdHbhToKgocMzOtrojAAAAAEBDNGpaQWRkpL755ht17dpVkhQbG6vS0lL96Ec/kiQdOnRISUlJ3D1wDq15WgEAAAAAwH5aZFrB2TlCE5YrAAAAAAAANtOkNQcAAAAAAEDr0ahwICIiQhEREXXGgDN5vYF1B7xeqzsBAAAAADREVGOKDcPQb37zG0VHR0uSjh8/rgcffFDt27eXJJ04caL5O0RYKSiQcnICWxq6XIEdDFiYEAAAAADsrVELEt53330NqvvTn/7U5IZas9a+IKHXK6WkBIKBWpGRgZ0L2NIQAAAAAEKvod9DG3XnAF/6cT5lZcHBgCTV1Eh79xIOAAAAAICdsSAhmk1aWmAqwZkiI6XUVGv6AQAAAAA0DOEAmo3bHVhjIDIy8DwyUsrL464BAAAAALC7Rk0rAC4kO1vyeAJTCVJTCQYAAAAAIBwQDqDZud2EAgAAAAAQTphWAAAAAACAwxEOAAAAAADgcIQDAAAAAAA4HOEAWoTXKxUVBY4AAAAAAHsjHECzKyiQUlKkjIzAsaDA6o4AAAAAAOdDOIBm5fVKOTmS3x947vdLubncQQAAAAAAdkY4gGZVVnY6GKhVUyPt3WtNPwAAAACACyMcQLNKS5NcZ11VkZFSaqo1/QAAAAAALoxwAM3K7Zby8wOBgBQ45uUFxgEAAAAA9hRldQNofbKzJY8nMJUgNZVgAAAAAADsjnAALcLtJhQAAAAAgHDBtAIAAAAAAByOcAAtxuuViorYxhAAAAAA7I5wAC2ioEBKSZEyMgLHggKrOwIAAAAAnAvhAJqd1yvl5Eh+f+C53y/l5nIHAQAAAADYFeEAml1Z2elgoFZNTWD3AgAAAACA/RAOoNmlpUmus66syMjAtoYAAAAAAPshHECzc7ul/PxAICAFjnl5bG0IAAAAAHYVZXUDaJ2ysyWPJzCVIDWVYAAAAAAA7IxwAC3G7SYUAAAAAIBwwLQCAAAAAAAcjnAAAAAAAACHIxxAi/J6paKiwBEAAAAAYE+EA2gxBQVSSoqUkRE4FhRY3REAAAAAoD6EA2gRXq+UkyP5/YHnfr+Um8sdBAAAAABgR4QDaBFlZaeDgVo1NYGtDQEAAAAA9kI4gBaRlia5zrq6IiOl1FRr+gEAAAAAnBvhAFqE2y3l5wcCASlwzMsLjAMAAAAA7CXK6gbQemVnSx5PYCpBairBAAAAAADYFeEAWpTbTSgAAAAAAHbHtAK0OK9XKipipwIAAAAAsCvCAbSoggIpJUXKyAgcCwqs7ggAAAAAcDbCAbQYr1fKyTm9paHfL+XmcgcBAAAAANgN4QBaTFnZ6WCgVk1NYIFCAAAAAIB9EA6gxaSlSa6zrrDIyMDOBQAAAAAA+yAcQItxu6X8/EAgIAWOeXnsXgAAAAAAdsNWhmhR2dmSxxOYSpCaSjAAAAAAAHZk+zsHvv76a/3qV79S586d1a5dO/Xr109btmwxzxuGoZkzZ6pbt25q166dMjMzVVZWFvQaFRUVysrKUmxsrOLj45Wdna2jR48G1Wzfvl033XSTYmJilJycrHnz5tXpZfny5erdu7diYmLUr18/vffeey3zoVsZt1u69VaCAQAAAACwK1uHA0eOHNENN9ygNm3a6P3339fnn3+u5557Th07djRr5s2bpwULFmjJkiXatGmT2rdvL4/Ho+PHj5s1WVlZ2rlzpwoLC7Vq1Spt2LBBOTk55nmfz6chQ4YoJSVFJSUlmj9/vmbPnq38/HyzZuPGjbrnnnuUnZ2tbdu2afjw4Ro+fLh27NgRml8GAAAAAAAtJMIwDMPqJs5l2rRp+uijj/S3v/2t3vOGYSgpKUmTJ0/WI488IkmqqqpSQkKCli5dqlGjRmnXrl3q27evNm/erIEDB0qSVq9eraFDh8rr9SopKUmLFy/WY489pvLycrVt29Z875UrV2r37t2SpJEjR+rYsWNatWqV+f6DBw/WgAEDtGTJkgZ9Hp/Pp7i4OFVVVSk2NrbJv5dw4/UGdi5IS+PuAQAAAAAIpYZ+D7X1nQNvv/22Bg4cqF/84hfq2rWrrrrqKv3xj380z+/bt0/l5eXKzMw0x+Li4jRo0CAVFxdLkoqLixUfH28GA5KUmZkpl8ulTZs2mTU333yzGQxIksfj0Z49e3TkyBGz5sz3qa2pfZ/6nDhxQj6fL+jhNAUFUkqKlJEROBYUWN0RAAAAAOBstg4H/vGPf2jx4sVKS0vTmjVr9Nvf/lYPP/ywli1bJkkqLy+XJCUkJAT9XEJCgnmuvLxcXbt2DTofFRWlTp06BdXU9xpnvse5amrP12fOnDmKi4szH8nJyY36/OHO65VyciS/P/Dc75dycwPjAAAAAAD7sHU44Pf7dfXVV+vpp5/WVVddpZycHD3wwAMNvo3fatOnT1dVVZX5OHDggNUthVRZ2elgoFZNTWDnAgAAAACAfdg6HOjWrZv69u0bNNanTx/t379fkpSYmChJOnToUFDNoUOHzHOJiYk6fPhw0PlTp06poqIiqKa+1zjzPc5VU3u+PtHR0YqNjQ16OElamuQ66wqLjAxsaQgAAAAAsA9bhwM33HCD9uzZEzT2xRdfKCUlRZLUs2dPJSYmau3ateZ5n8+nTZs2KT09XZKUnp6uyspKlZSUmDXr1q2T3+/XoEGDzJoNGzaourrarCksLFSvXr3MnRHS09OD3qe2pvZ9UJfbLeXnBwIBKXDMy2NRQgAAAACwG1vvVrB582Zdf/31+v3vf69f/vKX+uSTT/TAAw8oPz9fWVlZkqRnnnlGc+fO1bJly9SzZ0/NmDFD27dv1+eff66YmBhJ0p133qlDhw5pyZIlqq6u1n333aeBAwfqtddekxTY4aBXr14aMmSIpk6dqh07duj+++/XCy+8YG55uHHjRt1yyy2aO3euhg0bpjfeeENPP/20tm7dqiuvvLJBn8fJuxXs3Ru4Y4BgAAAAAABCp6HfQ20dDkjSqlWrNH36dJWVlalnz56aNGmSHnjgAfO8YRiaNWuW8vPzVVlZqRtvvFGLFi3Sj3/8Y7OmoqJC48aN0zvvvCOXy6URI0ZowYIF6tChg1mzfft2jR07Vps3b1aXLl300EMPaerUqUG9LF++XI8//ri+/PJLpaWlad68eRo6dGiDP4tTwwEAAAAAgDVaTTjQmjg5HPB6AwsUpqVx9wAAAAAAhEpDv4faes0BtA4FBVJKipSRETgWFFjdEQAAAADgTIQDaFFer5STc3pLQ79fys0NjAMAAAAA7IFwAC2qrOx0MFCrpiawQCEAAAAAwB4IB9Ci0tIk11lXWWRkYOcCAAAAAIA9EA6gRbndUn5+IBCQAse8PBYlBAAAAAA7ibK6AbR+2dmSxxOYSpCaSjAAAAAAAHZDOICQcLsJBQAAAADArphWgJDxeqWiInYqAAAAAAC7IRxASBQUSCkpUkZG4FhQYHVHAAAAAIBahANocV6vlJNzektDv1/KzeUOAgAAAACwC8IBtLiystPBQK2amsAChQAAAAAA6xEOoMWlpUmus660yMjAzgUAAAAAAOsRDqDFud1Sfn4gEJACx7w8di8AAAAAALtgK0OERHa25PEEphKkphIMAAAAAICdEA4gZNxuQgEAAAAAsCOmFSCkvF6pqIidCgAAAADATggHEDIFBVJKipSRETgWFFjdEQAAAABAIhxAiHi9Uk7O6S0N/X4pN5c7CAAAAADADggHEBJlZaeDgVo1NYEFCgEAAAAA1iIcQEikpUmus662yMjAzgUAAAAAAGsRDiAk3G4pPz8QCEiBY14euxcAAAAAgB2wlSFCJjtb8ngCUwlSUwkGAAAAAMAuCAcQUrWBQFlZ8HMAAAAAgHWYVoCQYjtDAAAAALAfwgGEDNsZAgAAAIA9EQ4gZNjOEAAAAADsiXAAIcN2hgAAAABgT4QDCBm2MwQAAAAAe2K3AoQU2xkCAAAAgP0QDiDk2M4QAAAAAOyFaQUIObYzBAAAAAB7IRxASLGdIQAAAADYD+EAQortDAEAAADAfggHEFJsZwgAAAAA9kM4gJBiO0MAAAAAsB92K0DIsZ0hAAAAANgL4QAswXaGAAAAAGAfTCuAJdjOEAAAAADsg3AAIcd2hgAAAABgL4QDCDm2MwQAAAAAeyEcQMixnSEAAAAA2AvhAEKO7QwBAAAAwF7YrQCWyM6WfvIT6cMPpRtvlK691uqOAAAAAMC5CAdgiYKC04sSulyBOwmys63uCgAAAACciWkFCDl2KwAAAAAAeyEcQMixWwEAAAAA2AvhAEKO3QoAAAAAwF4IBxBy7FYAAAAAAPbCgoSwRHa25PFIxcWSYUjXX291RwAAAADgXNw5AMusWSONGiWNHCmlpAR2MAAAAAAAhB7hACzBjgUAAAAAYB+EA7AEOxYAAAAAgH0QDsAS7FgAAAAAAPZBOABLsGMBAAAAANgHuxXAMuxYAAAAAAD2wJ0DsBQ7FgAAAACA9QgHYBl2LAAAAAAAeyAcgGXYsQAAAAAA7IFwAJZhxwIAAAAAsAfCAViGHQsAAAAAwB7YrQCWYscCAAAAALAedw7AcuxYAAAAAADWIhyApdixAAAAAACsRzgAS7FjAQAAAABYj3AAlmLHAgAAAACwHuEALHX2jgUulzRnDjsWAAAAAEAohVU4MHfuXEVERGjChAnm2PHjxzV27Fh17txZHTp00IgRI3To0KGgn9u/f7+GDRumSy65RF27dtWUKVN06tSpoJoPPvhAV199taKjo5WamqqlS5fWef+FCxeqR48eiomJ0aBBg/TJJ5+0xMd0nOxsae7cQDDg90vTprEoIQAAAACEUtiEA5s3b1ZeXp5+8pOfBI1PnDhR77zzjpYvX67169fr4MGDuvvuu83zNTU1GjZsmE6ePKmNGzdq2bJlWrp0qWbOnGnW7Nu3T8OGDdNtt92m0tJSTZgwQf/xH/+hNWvWmDVvvvmmJk2apFmzZmnr1q3q37+/PB6PDh8+3PIfvpXzeqWpU1mUEAAAAACsEmEYhmF1Exdy9OhRXX311Vq0aJGefPJJDRgwQC+++KKqqqp02WWX6bXXXtPPf/5zSdLu3bvVp08fFRcXa/DgwXr//ff105/+VAcPHlRCQoIkacmSJZo6daq+/fZbtW3bVlOnTtW7776rHTt2mO85atQoVVZWavXq1ZKkQYMG6dprr9Urr7wiSfL7/UpOTtZDDz2kadOmNehz+Hw+xcXFqaqqSrGxsc35KwprRUVSRkb947feGvJ2AAAAAKDVaOj30LC4c2Ds2LEaNmyYMjMzg8ZLSkpUXV0dNN67d291795dxcXFkqTi4mL169fPDAYkyePxyOfzaefOnWbN2a/t8XjM1zh58qRKSkqCalwulzIzM82a+pw4cUI+ny/ogbpYlBAAAAAArGX7cOCNN97Q1q1bNWfOnDrnysvL1bZtW8XHxweNJyQkqLy83Kw5MxioPV977nw1Pp9PP/zwg7777jvV1NTUW1P7GvWZM2eO4uLizEdycnLDPrTDsCghAAAAAFjL1uHAgQMHNH78eL366quKiYmxup1Gmz59uqqqqszHgQMHrG7JtliUEAAAAACsY+twoKSkRIcPH9bVV1+tqKgoRUVFaf369VqwYIGioqKUkJCgkydPqrKyMujnDh06pMTERElSYmJind0Lap9fqCY2Nlbt2rVTly5dFBkZWW9N7WvUJzo6WrGxsUEP1I9FCQEAAADAOrYOB26//XZ99tlnKi0tNR8DBw5UVlaW+c9t2rTR2rVrzZ/Zs2eP9u/fr/T0dElSenq6Pvvss6BdBQoLCxUbG6u+ffuaNWe+Rm1N7Wu0bdtW11xzTVCN3+/X2rVrzRpcnLKy08FArZoaae9ea/oBAAAAACeJsrqB87n00kt15ZVXBo21b99enTt3Nsezs7M1adIkderUSbGxsXrooYeUnp6uwYMHS5KGDBmivn37avTo0Zo3b57Ky8v1+OOPa+zYsYqOjpYkPfjgg3rllVf06KOP6v7779e6dev0l7/8Re+++675vpMmTdKYMWM0cOBAXXfddXrxxRd17Ngx3XfffSH6bbRutYsSnhkQsCghAAAAAISGrcOBhnjhhRfkcrk0YsQInThxQh6PR4sWLTLPR0ZGatWqVfrtb3+r9PR0tW/fXmPGjNETTzxh1vTs2VPvvvuuJk6cqJdeeklut1v/63/9L3k8HrNm5MiR+vbbbzVz5kyVl5drwIABWr16dZ1FCtE0tYsS5uQEAgKXS8rLY1FCAAAAAAiFCMMwDKubcIqG7i/pVAUFweFAfn5goUIAAAAAQNM09HuordccgHN4vaeDAYkFCQEAAAAglAgHYAssSAgAAAAA1iEcgC3ULkh4JhYkBAAAAIDQIByALdQuSBgZGXjucklz5rAgIQAAAACEAuEAbCM7W5o79/SWhtOmBRYpBAAAAAC0LMIB2IbXK02dyqKEAAAAABBqhAOwDRYlBAAAAABrEA7ANliUEAAAAACsQTgA22BRQgAAAACwBuEAbIVFCQEAAAAg9AgHYCssSggAAAAAoUc4AFthUUIAAAAACD3CAdhKfYsSulwsSggAAAAALYlwALZSuyhhRMTpMcOQ1qyxricAAAAAaO0IB2A7Hk/dcIB1BwAAAACg5RAOwHZYdwAAAAAAQotwALbDugMAAAAAEFqEA7Ad1h0AAAAAgNAiHIAtse4AAAAAAIQO4QBsiXUHAAAAACB0CAdgS6w7AAAAAAChQzgAW2LdAQAAAAAIHcIB2BbrDgAAAABAaBAOwLZYdwAAAAAAQoNwALbFugMAAAAAEBqEA7At1h0AAAAAgNAgHICtse4AAAAAALQ8wgHYGusOAAAAAEDLIxyArdW37kBkJOsOAAAAAEBzIhyArbnd0ujRwWO/+lVgHAAAAADQPAgHYGter/T//X/BY3/+M2sOAAAAAEBzIhyArbHmAAAAAAC0PMIB2Fp9aw64XKw5AAAAAADNiXAAtuZ2S/n5dbczXLPGup4AAAAAoLUhHIDteTx1w4HcXNYdAAAAAIDmQjgA22PdAQAAAABoWYQDsL361h2QpC1bQt8LAAAAALRGhAOwPbdbmju37vi0aUwtAAAAAIDmQDiAsDBwYN0xphYAAAAAQPMgHEBYYGoBAAAAALQcwgGEBaYWAAAAAEDLIRxA2GBqAQAAAAC0DMIBhI36pha4XFJqqjX9AAAAAEBrQTiAsOF2S/n5UkTE6THDkNassa4nAAAAAGgNCAcQVjyeuuFAbi7rDgAAAADAxSAcQFgpK5P8/uAx1h0AAAAAgItDOICwwpaGAAAAAND8CAcQVtjSEAAAAACaH+EAwg5bGgIAAABA8yIcQNhJSwtelFAKPGdLQwAAAABoGsIBAAAAAAAcjnAAYaesLLCF4ZkMQ3rpJWv6AQAAAIBwRziAsFPftAJJeuEFFiUEAAAAgKYgHEDYcbulyZPrjrMoIQAAAAA0DeEAwtL48ZKrnqt3y5bQ9wIAAAAA4Y5wAGHJ7Zbmzq07Pm0aUwsAAAAAoLEIBxC2Bg6sO8bUAgAAAABoPMIBhK20NKYWAAAAAEBzIBxA2DrX1IKpU5laAAAAAACNQTiAsFbf1AK/X3rppdD3AgAAAADhinAAYS0tTYqIqDv+/PPcPQAAAAAADUU4gLDmdkuTJ9cd5+4BAAAAAGg4wgGEvfHj67974IUXuHsAAAAAABqCcABh71x3D7CtIQAAAAA0DOEAWoXx49nWEAAAAACainAArcK5tjWcNo2pBQAAAABwIbYOB+bMmaNrr71Wl156qbp27arhw4drz549QTXHjx/X2LFj1blzZ3Xo0EEjRozQoUOHgmr279+vYcOG6ZJLLlHXrl01ZcoUnTp1Kqjmgw8+0NVXX63o6GilpqZq6dKldfpZuHChevTooZiYGA0aNEiffPJJs39mNF192xoytQAAAAAALszW4cD69es1duxYffzxxyosLFR1dbWGDBmiY8eOmTUTJ07UO++8o+XLl2v9+vU6ePCg7r77bvN8TU2Nhg0bppMnT2rjxo1atmyZli5dqpkzZ5o1+/bt07Bhw3TbbbeptLRUEyZM0H/8x39ozZo1Zs2bb76pSZMmadasWdq6dav69+8vj8ejw4cPh+aXgQvq0KH+8fbtQ9sHAAAAAISbCMMwDKubaKhvv/1WXbt21fr163XzzTerqqpKl112mV577TX9/Oc/lyTt3r1bffr0UXFxsQYPHqz3339fP/3pT3Xw4EElJCRIkpYsWaKpU6fq22+/Vdu2bTV16lS9++672rFjh/leo0aNUmVlpVavXi1JGjRokK699lq98sorkiS/36/k5GQ99NBDmjZtWoP69/l8iouLU1VVlWJjY5vzVwNJRUVSRkbd8UcekebPD30/AAAAAGC1hn4PtfWdA2erqqqSJHXq1EmSVFJSourqamVmZpo1vXv3Vvfu3VVcXCxJKi4uVr9+/cxgQJI8Ho98Pp927txp1pz5GrU1ta9x8uRJlZSUBNW4XC5lZmaaNbBeWlr9Wxo+/zzrDgAAAADA+YRNOOD3+zVhwgTdcMMNuvLKKyVJ5eXlatu2reLj44NqExISVF5ebtacGQzUnq89d74an8+nH374Qd99951qamrqral9jfqcOHFCPp8v6IGWc64tDf1+6aWXQt8PAAAAAISLsAkHxo4dqx07duiNN96wupUGmzNnjuLi4sxHcnKy1S21euPHc/cAAAAAADRWWIQD48aN06pVq1RUVCS3222OJyYm6uTJk6qsrAyqP3TokBITE82as3cvqH1+oZrY2Fi1a9dOXbp0UWRkZL01ta9Rn+nTp6uqqsp8HDhwoHEfHI3G3QMAAAAA0Hi2DgcMw9C4ceO0YsUKrVu3Tj179gw6f80116hNmzZau3atObZnzx7t379f6enpkqT09HR99tlnQbsKFBYWKjY2Vn379jVrznyN2pra12jbtq2uueaaoBq/36+1a9eaNfWJjo5WbGxs0AMtj7sHAAAAAKBxbB0OjB07Vn/+85/12muv6dJLL1V5ebnKy8v1ww8/SJLi4uKUnZ2tSZMmqaioSCUlJbrvvvuUnp6uwYMHS5KGDBmivn37avTo0fr000+1Zs0aPf744xo7dqyio6MlSQ8++KD+8Y9/6NFHH9Xu3bu1aNEi/eUvf9HEiRPNXiZNmqQ//vGPWrZsmXbt2qXf/va3OnbsmO67777Q/2JwXtw9AAAAAACNY+utDCPq++tfSX/605/0m9/8RpJ0/PhxTZ48Wa+//rpOnDghj8ejRYsWBd3u/9VXX+m3v/2tPvjgA7Vv315jxozR3LlzFRUVZdZ88MEHmjhxoj7//HO53W7NmDHDfI9ar7zyiubPn6/y8nINGDBACxYs0KBBgxr8edjKMHS8Xql7d+nsq9vlkr76KhAgAAAAAEBr19DvobYOB1obwoHQmjJFevbZuuOPPCLNnx/6fgAAAAAg1Br6PdTW0wqAi3GutQeee461BwAAAADgTIQDaLXcbiknp+64YUjFxaHvBwAAAADsinAArVpGRv3j69aFtg8AAAAAsDPCAbRq119f/3h+PlMLAAAAAKAW4QBaNbc7sADh2fx+6cknQ98PAAAAANgR4QBavXMtTJiXV/9uBgAAAADgNIQDaPXcbmny5PrPPfoo0wsAAAAAgHAAjnCuuwcMg+kFAAAAAEA4AEdwu6Vnnqn/XF6e9Pjjoe0HAAAAAOyEcACOMWWKlJtb/7mnnmL9AQAAAADORTgAR3n88fqnF0jS1KmsPwAAAADAmQgH4Cjnm17g90t794a2HwAAAACwA8IBOM6UKdJDD9V/7q23QtsLAAAAANgB4QAc6X/+z/rHX36ZxQkBAAAAOA/hABwpLe3caw889RQBAQAAAABnIRyAI51v7QGJgAAAAACAsxAOwLGmTJEee+zc59neEAAAAIBTEA7A0Z588vwBwZQpbG8IAAAAoPUjHIDjXSggmD49dL0AAAAAgBUIBwAFAoKsrPrP/fnPrD8AAAAAoHUjHAD+n7lzz32OBQoBAAAAtGaEA8D/43ZL8+ad+zwBAQAAAIDWinAAOENDdjB4+OHQ9QMAAAAAoUA4AJzlQgsUvvyy9NOfhq4fAAAAAGhphANAPS4UELz7LncQAAAAAGg9CAeAc2jIHQS/+pXk9YauJwAAAABoCYQDwHk8+aT00EPnPv/qq1Jy8vlDBAAAAACwO8IB4AIWLJCGDTt/zdNPsw4BAAAAgPBFOAA0wKpV57+DQAqsQ3D33UwzAAAAABB+CAeABlqw4MLTB1asCEwzyM0lJAAAAAAQPggHgEZ48klp/vwL1+XnsxYBAAAAgPBBOAA00iOPSAcOSIMHX7j26aelq66S/vIX7iQAAAAAYF+EA0ATuN1ScfGF1yGQpNJSaeRIphsAAAAAsC/CAeAiLFjQsGkGtWqnGxASAAAAALATwgHgItVOM3jwwYb/TG1IkJXFlAMAAAAA1oswDMOwugmn8Pl8iouLU1VVlWJjY61uBy3A65V+8Qvp448b/7PDh0v33CNdf31g2gIAAAAAXKyGfg8lHAghwgHnePxx6amnmv7z994r3Xij1LkzYQEAAACApiMcsCHCAWfxegOLFr7+urRixcW9FmEBAAAAgKYgHLAhwgHnevZZ6dFHpeb6X9vw4dKQIYF/JjAAAAAAcC6EAzZEOOBstXcSvP229OqrzRcU1Lr9dikjQ+rY8fQYwQEAAADgbIQDNkQ4gFq1QcHzzzdt8cLGqp2WIElHjkiHD0u9ekl33UVwAAAAALRmhAM2RDiA+mzeLL37rvTZZ4G1CUL9v8h775WuuCIQGHTtGnznAUECAAAAEN4IB2yIcAAX0tJTDy5WQ4KEs8eZ2gAAAABYh3DAhggH0Bi1QcE//xl4vmxZaKYgtJT6pjacGSScK1yob5zAAQAAAGgYwgEbIhzAxaqdghAdHfii/NFH0p//bHVX1jlz1wbp4kMHgggAAAC0NoQDNkQ4gJbg9UqrVklffCFddtnp0MCO0xLC2dm7QVxs6NCYWgIKAAAANBXhgA0RDiCUzp6WUOvIEamoSPrrXwkPwo2VAcX5xgkvAAAA7ItwwIYIB2AnZ4YHR45I3357+s6DWgQJaIzmXFciVLUEGwAAoLUjHLAhwgGEq4YGCWeOM7UB4WT4cCklxf5hRlNrjx8PbEd67bXN8usCAABhhHDAhggH4DTnm9pwdsDQ0NAh3HdtAKw0aJA0Zoz9wwyrayXuKgEAtB6EAzZEOAA0j7N3bah1MaFDfWNO3w0CwOnpMnYOM1pDbXO9BqEOANRFOGBDhANA+KlvNwjp4kKHxtQSUABA44XjGijUUssdUmgphAM2RDgAoCmsDijOVcu6EgAAhD/ukGr9dywRDtgQ4QCA1qYl1pUIVe3nnxNuAACAixcRIf3xj1J2ttWd1I9wwIYIBwDAXmrDjb177R9mNLW2sFB6663m+X0BAID6RUZKX35pzzsIGvo9NCqEPQEAYCtut/SLX1jdRct68MG6d3jYOcywupbpMgCApqipCfxlgx3DgYbizoEQ4s4BAADsr77pMnYNM1pL7cW+BqEOAKu1hjsHCAdCiHAAAACgZYTzGijUUssdUuHN5ZLy81lzAI1AOAAAAADAatwh1Ty1UmC3gvR0e94xUIs1BwAAAAAAdThhzR00nsvqBgAAAAAAgLUIBwAAAAAAcDjCAQAAAAAAHI5wAAAAAAAAhyMcAAAAAADA4QgHAAAAAABwOMIBAAAAAAAcjnCgkRYuXKgePXooJiZGgwYN0ieffGJ1SwAAAAAAXBTCgUZ48803NWnSJM2aNUtbt25V//795fF4dPjwYatbAwAAAACgyQgHGuH555/XAw88oPvuu099+/bVkiVLdMkll+h//+//bXVrAAAAAAA0GeFAA508eVIlJSXKzMw0x1wulzIzM1VcXFzvz5w4cUI+ny/oAQAAAACA3RAONNB3332nmpoaJSQkBI0nJCSovLy83p+ZM2eO4uLizEdycnIoWgUAAAAAoFEIB1rQ9OnTVVVVZT4OHDhgdUsAAAAAANQRZXUD4aJLly6KjIzUoUOHgsYPHTqkxMTEen8mOjpa0dHRoWgPAAAAAIAm486BBmrbtq2uueYarV271hzz+/1au3at0tPTLewMAAAAAICLw50DjTBp0iSNGTNGAwcO1HXXXacXX3xRx44d03333Wd1awAAAAAANBnhQCOMHDlS3377rWbOnKny8nINGDBAq1evrrNIIQAAAAAA4STCMAzD6iacwufzKS4uTlVVVYqNjbW6HQAAAABAK9fQ76GsOQAAAAAAgMMRDgAAAAAA4HCEAwAAAAAAOBzhAAAAAAAADsduBSFUu/ajz+ezuBMAAAAAgBPUfv+80F4EhAMh9P3330uSkpOTLe4EAAAAAOAk33//veLi4s55nq0MQ8jv9+vgwYO69NJLFRERYXU75+Tz+ZScnKwDBw6w5SJsiWsU4YDrFHbHNQq74xqF3YXLNWoYhr7//nslJSXJ5Tr3ygLcORBCLpdLbrfb6jYaLDY21tYXOcA1inDAdQq74xqF3XGNwu7C4Ro93x0DtViQEAAAAAAAhyMcAAAAAADA4QgHUEd0dLRmzZql6Ohoq1sB6sU1inDAdQq74xqF3XGNwu5a2zXKgoQAAAAAADgcdw4AAAAAAOBwhAMAAAAAADgc4QAAAAAAAA5HOAAAAAAAgMMRDqCOhQsXqkePHoqJidGgQYP0ySefWN0SHGDOnDm69tprdemll6pr164aPny49uzZE1Rz/PhxjR07Vp07d1aHDh00YsQIHTp0KKhm//79GjZsmC655BJ17dpVU6ZM0alTp0L5UeAQc+fOVUREhCZMmGCOcY3CDr7++mv96le/UufOndWuXTv169dPW7ZsMc8bhqGZM2eqW7duateunTIzM1VWVhb0GhUVFcrKylJsbKzi4+OVnZ2to0ePhvqjoBWqqanRjBkz1LNnT7Vr106XX365/vCHP+jMNdK5RhFKGzZs0F133aWkpCRFRERo5cqVQeeb63rcvn27brrpJsXExCg5OVnz5s1r6Y/WaIQDCPLmm29q0qRJmjVrlrZu3ar+/fvL4/Ho8OHDVreGVm79+vUaO3asPv74YxUWFqq6ulpDhgzRsWPHzJqJEyfqnXfe0fLly7V+/XodPHhQd999t3m+pqZGw4YN08mTJ7Vx40YtW7ZMS5cu1cyZM634SGjFNm/erLy8PP3kJz8JGucahdWOHDmiG264QW3atNH777+vzz//XM8995w6duxo1sybN08LFizQkiVLtGnTJrVv314ej0fHjx83a7KysrRz504VFhZq1apV2rBhg3Jycqz4SGhlnnnmGS1evFivvPKKdu3apWeeeUbz5s3Tyy+/bNZwjSKUjh07pv79+2vhwoX1nm+O69Hn82nIkCFKSUlRSUmJ5s+fr9mzZys/P7/FP1+jGMAZrrvuOmPs2LHm85qaGiMpKcmYM2eOhV3BiQ4fPmxIMtavX28YhmFUVlYabdq0MZYvX27W7Nq1y5BkFBcXG4ZhGO+9957hcrmM8vJys2bx4sVGbGysceLEidB+ALRa33//vZGWlmYUFhYat9xyizF+/HjDMLhGYQ9Tp041brzxxnOe9/v9RmJiojF//nxzrLKy0oiOjjZef/11wzAM4/PPPzckGZs3bzZr3n//fSMiIsL4+uuvW655OMKwYcOM+++/P2js7rvvNrKysgzD4BqFtSQZK1asMJ831/W4aNEio2PHjkH/rp86darRq1evFv5EjcOdAzCdPHlSJSUlyszMNMdcLpcyMzNVXFxsYWdwoqqqKklSp06dJEklJSWqrq4Ouj579+6t7t27m9dncXGx+vXrp4SEBLPG4/HI5/Np586dIewerdnYsWM1bNiwoGtR4hqFPbz99tsaOHCgfvGLX6hr16666qqr9Mc//tE8v2/fPpWXlwddp3FxcRo0aFDQdRofH6+BAweaNZmZmXK5XNq0aVPoPgxapeuvv15r167VF198IUn69NNP9eGHH+rOO++UxDUKe2mu67G4uFg333yz2rZta9Z4PB7t2bNHR44cCdGnubAoqxuAfXz33XeqqakJ+o9WSUpISNDu3bst6gpO5Pf7NWHCBN1www268sorJUnl5eVq27at4uPjg2oTEhJUXl5u1tR3/daeAy7WG2+8oa1bt2rz5s11znGNwg7+8Y9/aPHixZo0aZJ+97vfafPmzXr44YfVtm1bjRkzxrzO6rsOz7xOu3btGnQ+KipKnTp14jrFRZs2bZp8Pp969+6tyMhI1dTU6KmnnlJWVpYkcY3CVprreiwvL1fPnj3rvEbtuTOnflmJcACA7YwdO1Y7duzQhx9+aHUrgOnAgQMaP368CgsLFRMTY3U7QL38fr8GDhyop59+WpJ01VVXaceOHVqyZInGjBljcXeA9Je//EWvvvqqXnvtNV1xxRUqLS3VhAkTlJSUxDUKWIxpBTB16dJFkZGRdVbWPnTokBITEy3qCk4zbtw4rVq1SkVFRXK73eZ4YmKiTp48qcrKyqD6M6/PxMTEeq/f2nPAxSgpKdHhw4d19dVXKyoqSlFRUVq/fr0WLFigqKgoJSQkcI3Cct26dVPfvn2Dxvr06aP9+/dLOn2dne/f9YmJiXUWIj516pQqKiq4TnHRpkyZomnTpmnUqFHq16+fRo8erYkTJ2rOnDmSuEZhL811PYbLv/8JB2Bq27atrrnmGq1du9Yc8/v9Wrt2rdLT0y3sDE5gGIbGjRunFStWaN26dXVuvbrmmmvUpk2boOtzz5492r9/v3l9pqen67PPPgv6P+jCwkLFxsbW+Y9loLFuv/12ffbZZyotLTUfAwcOVFZWlvnPXKOw2g033FBnG9gvvvhCKSkpkqSePXsqMTEx6Dr1+XzatGlT0HVaWVmpkpISs2bdunXy+/0aNGhQCD4FWrN//etfcrmCv4JERkbK7/dL4hqFvTTX9Zienq4NGzaourrarCksLFSvXr1sM6VAErsVINgbb7xhREdHG0uXLjU+//xzIycnx4iPjw9aWRtoCb/97W+NuLg444MPPjC++eYb8/Gvf/3LrHnwwQeN7t27G+vWrTO2bNlipKenG+np6eb5U6dOGVdeeaUxZMgQo7S01Fi9erVx2WWXGdOnT7fiI8EBztytwDC4RmG9Tz75xIiKijKeeuopo6yszHj11VeNSy65xPjzn/9s1sydO9eIj483/vu//9vYvn278e///u9Gz549jR9++MGsueOOO4yrrrrK2LRpk/Hhhx8aaWlpxj333GPFR0IrM2bMGOPf/u3fjFWrVhn79u0z3nrrLaNLly7Go48+atZwjSKUvv/+e2Pbtm3Gtm3bDEnG888/b2zbts346quvDMNonuuxsrLSSEhIMEaPHm3s2LHDeOONN4xLLrnEyMvLC/nnPR/CAdTx8ssvG927dzfatm1rXHfddcbHH39sdUtwAEn1Pv70pz+ZNT/88IPxn//5n0bHjh2NSy65xPif//N/Gt98803Q63z55ZfGnXfeabRr187o0qWLMXnyZKO6ujrEnwZOcXY4wDUKO3jnnXeMK6+80oiOjjZ69+5t5OfnB533+/3GjBkzjISEBCM6Otq4/fbbjT179gTV/POf/zTuueceo0OHDkZsbKxx3333Gd9//30oPwZaKZ/PZ4wfP97o3r27ERMTY/zoRz8yHnvssaAt3rhGEUpFRUX1/jfomDFjDMNovuvx008/NW688UYjOjra+Ld/+zdj7ty5ofqIDRZhGIZhzT0LAAAAAADADlhzAAAAAAAAhyMcAAAAAADA4QgHAAAAAABwOMIBAAAAAAAcjnAAAAAAAACHIxwAAAAAAMDhCAcAAAAAAHA4wgEAAGD68ssvFRERodLSUqtbMe3evVuDBw9WTEyMBgwYUG/NrbfeqgkTJoS0r4aIiIjQypUrrW4DAIALIhwAAMBGfvOb3ygiIkJz584NGl+5cqUiIiIs6spas2bNUvv27bVnzx6tXbu23pq33npLf/jDH8znPXr00IsvvhiiDqXZs2fXG1x88803uvPOO0PWBwAATUU4AACAzcTExOiZZ57RkSNHrG6l2Zw8ebLJP/v3v/9dN954o1JSUtS5c+d6azp16qRLL720ye9xLhfTtyQlJiYqOjq6mboBAKDlEA4AAGAzmZmZSkxM1Jw5c85ZU9/fVL/44ovq0aOH+fw3v/mNhg8frqeffloJCQmKj4/XE088oVOnTmnKlCnq1KmT3G63/vSnP9V5/d27d+v6669XTEyMrrzySq1fvz7o/I4dO3TnnXeqQ4cOSkhI0OjRo/Xdd9+Z52+99VaNGzdOEyZMUJcuXeTxeOr9HH6/X0888YTcbreio6M1YMAArV692jwfERGhkpISPfHEE4qIiNDs2bPrfZ0zpxXceuut+uqrrzRx4kRFREQE3XHx4Ycf6qabblK7du2UnJyshx9+WMeOHTPP9+jRQ3/4wx/061//WrGxscrJyZEkTZ06VT/+8Y91ySWX6Ec/+pFmzJih6upqSdLSpUv1+9//Xp9++qn5fkuXLjX7P3NawWeffaaMjAy1a9dOnTt3Vk5Ojo4ePVrnz+zZZ59Vt27d1LlzZ40dO9Z8L0latGiR0tLSFBMTo4SEBP385z+v93cCAEBjEA4AAGAzkZGRevrpp/Xyyy/L6/Ve1GutW7dOBw8e1IYNG/T8889r1qxZ+ulPf6qOHTtq06ZNevDBB5Wbm1vnfaZMmaLJkydr27ZtSk9P11133aV//vOfkqTKykplZGToqquu0pYtW7R69WodOnRIv/zlL4NeY9myZWrbtq0++ugjLVmypN7+XnrpJT333HN69tlntX37dnk8Hv3sZz9TWVmZpMBt+VdccYUmT56sb775Ro888sgFP/Nbb70lt9utJ554Qt98842++eYbSYE7EO644w6NGDFC27dv15tvvqkPP/xQ48aNC/r5Z599Vv3799e2bds0Y8YMSdKll16qpUuX6vPPP9dLL72kP/7xj3rhhRckSSNHjtTkyZN1xRVXmO83cuTIOn0dO3ZMHo9HHTt21ObNm7V8+XL99a9/rfP+RUVF+vvf/66ioiItW7ZMS5cuNcOGLVu26OGHH9YTTzyhPXv2aPXq1br55psv+DsBAOCCDAAAYBtjxowx/v3f/90wDMMYPHiwcf/99xuGYRgrVqwwzvzX9qxZs4z+/fsH/ewLL7xgpKSkBL1WSkqKUVNTY4716tXLuOmmm8znp06dMtq3b2+8/vrrhmEYxr59+wxJxty5c82a6upqw+12G88884xhGIbxhz/8wRgyZEjQex84cMCQZOzZs8cwDMO45ZZbjKuuuuqCnzcpKcl46qmngsauvfZa4z//8z/N5/379zdmzZp13te55ZZbjPHjx5vPU1JSjBdeeCGoJjs728jJyQka+9vf/ma4XC7jhx9+MH9u+PDhF+x7/vz5xjXXXGM+r+/PwzAMQ5KxYsUKwzAMIz8/3+jYsaNx9OhR8/y7775ruFwuo7y83DCM039mp06dMmt+8YtfGCNHjjQMwzD+67/+y4iNjTV8Pt8FewQAoDG4cwAAAJt65plntGzZMu3atavJr3HFFVfI5Tr9r/uEhAT169fPfB4ZGanOnTvr8OHDQT+Xnp5u/nNUVJQGDhxo9vHpp5+qqKhIHTp0MB+9e/eWFPjb+VrXXHPNeXvz+Xw6ePCgbrjhhqDxG2644aI+87l8+umnWrp0aVDfHo9Hfr9f+/btM+sGDhxY52fffPNN3XDDDUpMTFSHDh30+OOPa//+/Y16/127dql///5q3769OXbDDTfI7/drz5495tgVV1yhyMhI83m3bt3MP5//8T/+h1JSUvSjH/1Io0eP1quvvqp//etfjeoDAID6EA4AAGBTN998szwej6ZPn17nnMvlkmEYQWNnzkuv1aZNm6DnERER9Y75/f4G93X06FHdddddKi0tDXqUlZUF3eJ+5pdgOzh69Khyc3ODev70009VVlamyy+/3Kw7u+/i4mJlZWVp6NChWrVqlbZt26bHHnvsohcrPJfz/flceuml2rp1q15//XV169ZNM2fOVP/+/VVZWdkivQAAnCPK6gYAAMC5zZ07VwMGDFCvXr2Cxi+77DKVl5fLMAxzwb3S0tJme9+PP/7Y/KJ/6tQplZSUmHPjr776av3Xf/2XevTooaiopv+nRGxsrJKSkvTRRx/plltuMcc/+ugjXXfddRfVf9u2bVVTUxM0dvXVV+vzzz9Xampqo15r48aNSklJ0WOPPWaOffXVVxd8v7P16dNHS5cu1bFjx8wA4qOPPpLL5arz53s+UVFRyszMVGZmpmbNmqX4+HitW7dOd999dyM+FQAAwbhzAAAAG+vXr5+ysrK0YMGCoPFbb71V3377rebNm6e///3vWrhwod5///1me9+FCxdqxYoV2r17t8aOHasjR47o/vvvlySNHTtWFRUVuueee7R582b9/e9/15o1a3Tfffdd8Avy2aZMmaJnnnlGb775pvbs2aNp06aptLRU48ePv6j+e/TooQ0bNujrr782d1GYOnWqNm7cqHHjxpl3Ovz3f/93nQUBz5aWlqb9+/frjTfe0N///nctWLBAK1asqPN++/btU2lpqb777judOHGizutkZWUpJiZGY8aM0Y4dO1RUVKSHHnpIo0ePVkJCQoM+16pVq7RgwQKVlpbqq6++0v/5P/9Hfr+/UeECAAD1IRwAAMDmnnjiiTq3/ffp00eLFi3SwoUL1b9/f33yyScNWsm/oebOnau5c+eqf//++vDDD/X222+rS5cukmT+bX9NTY2GDBmifv36acKECYqPjw9a36AhHn74YU2aNEmTJ09Wv379tHr1ar399ttKS0u7qP6feOIJffnll7r88st12WWXSZJ+8pOfaP369friiy9000036aqrrtLMmTOVlJR03tf62c9+pokTJ2rcuHEaMGCANm7caO5iUGvEiBG64447dNttt+myyy7T66+/Xud1LrnkEq1Zs0YVFRW69tpr9fOf/1y33367XnnllQZ/rvj4eL311lvKyMhQnz59tGTJEr3++uu64oorGvwaAADUJ8I4e8IiAAAAAABwFO4cAAAAAADA4QgHAAAAAABwOMIBAAAAAAAcjnAAAAAAAACHIxwAAAAAAMDhCAcAAAAAAHA4wgEAAAAAAByOcAAAAAAAAIcjHAAAAAAAwOEIBwAAAAAAcDjCAQAAAAAAHI5wAAAAAAAAh/v/Ad8t9tj2vHkAAAAAAElFTkSuQmCC\n"
          },
          "metadata": {}
        }
      ]
    },
    {
      "cell_type": "markdown",
      "metadata": {
        "id": "IG5tWAy-FCaW"
      },
      "source": [
        "Zooming in..."
      ]
    },
    {
      "cell_type": "code",
      "metadata": {
        "colab": {
          "base_uri": "https://localhost:8080/",
          "height": 693
        },
        "id": "WZ7BoFHy8kTk",
        "outputId": "73433d50-a6db-4104-ab8b-bbf8f8575b04"
      },
      "source": [
        "fig,ax = plt.subplots(figsize=(10,8))\n",
        "ax.plot(range(200), cost_history[:200], 'b.')"
      ],
      "execution_count": 9,
      "outputs": [
        {
          "output_type": "execute_result",
          "data": {
            "text/plain": [
              "[<matplotlib.lines.Line2D at 0x799043f8a710>]"
            ]
          },
          "metadata": {},
          "execution_count": 9
        },
        {
          "output_type": "display_data",
          "data": {
            "text/plain": [
              "<Figure size 1000x800 with 1 Axes>"
            ],
            "image/png": "iVBORw0KGgoAAAANSUhEUgAAA1kAAAKTCAYAAADrBcJVAAAAOXRFWHRTb2Z0d2FyZQBNYXRwbG90bGliIHZlcnNpb24zLjcuMSwgaHR0cHM6Ly9tYXRwbG90bGliLm9yZy/bCgiHAAAACXBIWXMAAA9hAAAPYQGoP6dpAABQ7ElEQVR4nO3de3RU5b3/8U8uJkE0iYAk5AzEtIkgFaGKxli1FLMYldpySltAqtROSeoBK6IWOJVLezyFhp83jgrJr9Pi+tUraxVaUeGkEKSVGCDIQVA4oQeEKU6wxWQEC4TM/v2xz2wyIVfYc3+/1pq1nb2fzDx7s5uVT59nf58kwzAMAQAAAABskRzpDgAAAABAPCFkAQAAAICNCFkAAAAAYCNCFgAAAADYiJAFAAAAADYiZAEAAACAjQhZAAAAAGCj1Eh3IJr5/X4dOXJEl156qZKSkiLdHQAAAAARYhiGPvvsM+Xl5Sk5ueuxKkJWF44cOaLBgwdHuhsAAAAAosThw4flcDi6bEPI6sKll14qybyQmZmZEe4NAAAAgEjx+XwaPHiwlRG6QsjqQmCKYGZmJiELAAAAQI8eI6LwBQAAAADYiJAFAAAAADYiZAEAAACAjQhZAAAAAGAjQhYAAAAA2IiQBQAAAAA2ImQBAAAAgI0IWQAAAABgI0IWAAAAANiIkAUAAAAANiJkAQAAAICNCFkAAAAAYCNCFgAAAADYiJAFAAAAADYiZAEAAACAjQhZAAAAAGAjQhYAAAAA2IiQBQAAAAA2ImQBAAAAgI0IWQAAAABgI0IWAAAAANiIkAUAAAAANiJkxRCPR6qpMbcAAAAAohMhK0a43VJ+vjR2rLl1uyPdIwAAAAAdIWTFAI9HKiuT/H7zvd8vlZczogUAAABEI0JWDGhoOBuwAlpbpf37I9MfAAAAAJ0jZMWAoiIpud2/VEqKVFgYmf4AAAAA6BwhKwY4HFJVlRmsJHNbWWnuBwAAABBdUiPdAfSMyyU5neYUwcJCAhYAAAAQrQhZMcThIFwBAAAA0Y7pggAAAABgI0IWAAAAANiIkAUAAAAANiJkAQAAAICNCFkAAAAAYCNCFgAAAADYiJAFAAAAADYiZAEAAACAjQhZAAAAAGAjQhYAAAAA2IiQBQAAAAA2ImQBAAAAgI0IWQAAAABgI0IWAAAAANiIkAUAAAAANiJkxSiPR6qpMbcAAAAAogchKwa53VJ+vjR2rLl1uyPdIwAAAAABhKwY4/FIZWWS32++9/ul8nJGtAAAAIBoQciKMQ0NZwNWQGurtH9/ZPoDAAAAIBghK8YUFUnJ7f7VUlKkwsLI9AcAAABAMEJWjHE4pKoqM1hJ5ray0twPAAAAIPJSI90B9J7LJTmd5hTBwkICFgAAABBNCFkxyuEgXAEAAADRiOmCAAAAAGAjQhYAAAAA2IiQBQAAAAA2ImQBAAAAgI16HbI2b96su+66S3l5eUpKStKaNWs6bfujH/1ISUlJevrpp4P2Hzt2TFOnTlVmZqays7Plcrl0/PjxoDa7du3SLbfcooyMDA0ePFgVFRXnfP6qVas0bNgwZWRkaMSIEXrzzTeDjhuGoQULFmjQoEHq06ePSktL1dDQ0NtTBgAAAIAe63XIOnHihEaOHKnnnnuuy3arV6/Wu+++q7y8vHOOTZ06VXv27FF1dbXWrl2rzZs3q6yszDru8/k0btw45efnq76+XkuXLtWiRYtUVVVltdmyZYumTJkil8ul9957TxMmTNCECRO0e/duq01FRYWWLVumFStWqK6uTn379pXT6dTJkyd7e9oAAAAA0CNJhmEY5/3DSUlavXq1JkyYELT/r3/9q4qLi7V+/XqNHz9es2bN0qxZsyRJH374oYYPH65t27Zp9OjRkqR169bpzjvvlMfjUV5enpYvX66f/vSn8nq9SktLkyTNnTtXa9as0d69eyVJkyZN0okTJ7R27Vrre2+88UaNGjVKK1askGEYysvL08MPP6xHHnlEktTc3KycnBytXLlSkydP7vb8fD6fsrKy1NzcrMzMzPO9TAAAAABiXG+yge3PZPn9ft1zzz169NFH9aUvfemc47W1tcrOzrYCliSVlpYqOTlZdXV1Vptbb73VCliS5HQ6tW/fPn366adWm9LS0qDPdjqdqq2tlSQdOHBAXq83qE1WVpaKi4utNu2dOnVKPp8v6AUAAAAAvWF7yPrlL3+p1NRU/fjHP+7wuNfr1cCBA4P2paamql+/fvJ6vVabnJycoDaB9921aXu87c911Ka9xYsXKysry3oNHjy42/MFAAAAgLZsDVn19fV65plntHLlSiUlJdn50WExb948NTc3W6/Dhw9HuksAAAAAYoytIetPf/qTjh49qiFDhig1NVWpqan66KOP9PDDD+uKK66QJOXm5uro0aNBP3fmzBkdO3ZMubm5VpvGxsagNoH33bVpe7ztz3XUpr309HRlZmYGvQAAAACgN2wNWffcc4927dqlnTt3Wq+8vDw9+uijWr9+vSSppKRETU1Nqq+vt35u48aN8vv9Ki4uttps3rxZLS0tVpvq6moNHTpUl112mdVmw4YNQd9fXV2tkpISSVJBQYFyc3OD2vh8PtXV1VltAAAAAMBuqb39gePHj2v//v3W+wMHDmjnzp3q16+fhgwZov79+we1v+iii5Sbm6uhQ4dKkq666irdfvvtmj59ulasWKGWlhbNnDlTkydPtsq933333frZz34ml8ulOXPmaPfu3XrmmWf01FNPWZ/74IMP6qtf/aqeeOIJjR8/Xq+88oq2b99ulXlPSkrSrFmz9Pjjj6uoqEgFBQWaP3++8vLyzqmGCAAAAAB26XXI2r59u772ta9Z72fPni1JmjZtmlauXNmjz3jxxRc1c+ZM3XbbbUpOTtbEiRO1bNky63hWVpb+8z//UzNmzNB1112nAQMGaMGCBUFrad1000166aWX9Nhjj+lf//VfVVRUpDVr1ujqq6+22vzkJz/RiRMnVFZWpqamJt18881at26dMjIyenvaAAAAANAjF7ROVryLpXWyPB6poUEqKpIcjkj3BgAAAIgvEV0nC+Hndkv5+dLYsebW7Y50jwAAAIDERciKcR6PVFYm+f3me79fKi839wMAAAAIP0JWjGtoOBuwAlpbpTa1SQAAAACEESErxhUVScnt/hVTUqTCwsj0BwAAAEh0hKwY53BIVVVmsJLMbWUlxS8AAACASOl1CXdEH5dLcjrNKYKFhQQsAAAAIJIIWXHC4SBcAQAAANGA6YIAAAAAYCNCFgAAAADYiJAFAAAAADYiZAEAAACAjQhZAAAAAGAjQhYAAAAA2IiQBQAAAAA2ImQBAAAAgI0IWQAAAABgI0IWAAAAANiIkAUAAAAANiJkAQAAAICNCFlxyOORamrMLQAAAIDwImTFGbdbys+Xxo41t253pHsEAAAAJBZCVhzxeKSyMsnvN9/7/VJ5OSNaAAAAQDgRsuJIQ8PZgBXQ2irt3x+Z/gAAAACJiJAVR4qKpOR2/6IpKVJhYWT6AwAAACQiQlYccTikqiozWEnmtrLS3A8AAAAgPFIj3QHYy+WSnE5zimBhIQELAAAACDdCVhxyOAhXAAAAQKQwXRAAAAAAbETIAgAAAAAbEbIAAAAAwEaELAAAAACwESELAAAAAGxEyAIAAAAAGxGyAAAAAMBGhCwAAAAAsBEhCwAAAABsRMgCAAAAABsRsgAAAADARoQsAAAAALARISsBeDxSTY25BQAAABBahKw453ZL+fnS2LHm1u2OdI8AAACA+EbIimMej1RWJvn95nu/XyovZ0QLAAAACCVCVhxraDgbsAJaW6X9+yPTHwAAACARELLiWFGRlNzuXzglRSosjEx/AAAAgERAyIpjDodUVWUGK8ncVlaa+wEAAACERmqkO4DQcrkkp9OcIlhYSMACAAAAQo2QlQAcDsIVAAAAEC5MFwQAAAAAGxGyAAAAAMBGhCwAAAAAsBEhCwAAAABsRMgCAAAAABsRsgAAAADARoQsAAAAALARIQsAAAAAbETIAgAAAAAbEbIAAAAAwEaELAAAAACwESErwXg8Uk2NuQUAAABgP0JWAnG7pfx8aexYc+t2R7pHAAAAQPwhZCUIj0cqK5P8fvO93y+VlzOiBQAAANiNkJUgGhrOBqyA1lZp//7I9AcAAACIV4SsBFFUJCW3+9dOSZEKCyPTHwAAACBeEbIShMMhVVWZwUoyt5WV5n4AAAAA9kmNdAcQPi6X5HSaUwQLCwlYAAAAQCgQshKMw0G4AgAAAEKJ6YIAAAAAYKNeh6zNmzfrrrvuUl5enpKSkrRmzRrrWEtLi+bMmaMRI0aob9++ysvL07333qsjR44EfcaxY8c0depUZWZmKjs7Wy6XS8ePHw9qs2vXLt1yyy3KyMjQ4MGDVVFRcU5fVq1apWHDhikjI0MjRozQm2++GXTcMAwtWLBAgwYNUp8+fVRaWqqGhobenjIAAAAA9FivQ9aJEyc0cuRIPffcc+cc+/zzz7Vjxw7Nnz9fO3bs0O9+9zvt27dP3/jGN4LaTZ06VXv27FF1dbXWrl2rzZs3q6yszDru8/k0btw45efnq76+XkuXLtWiRYtUVVVltdmyZYumTJkil8ul9957TxMmTNCECRO0e/duq01FRYWWLVumFStWqK6uTn379pXT6dTJkyd7e9oAAAAA0CNJhmEY5/3DSUlavXq1JkyY0Gmbbdu26YYbbtBHH32kIUOG6MMPP9Tw4cO1bds2jR49WpK0bt063XnnnfJ4PMrLy9Py5cv105/+VF6vV2lpaZKkuXPnas2aNdq7d68kadKkSTpx4oTWrl1rfdeNN96oUaNGacWKFTIMQ3l5eXr44Yf1yCOPSJKam5uVk5OjlStXavLkyd2en8/nU1ZWlpqbm5WZmXm+lwkAAABAjOtNNgj5M1nNzc1KSkpSdna2JKm2tlbZ2dlWwJKk0tJSJScnq66uzmpz6623WgFLkpxOp/bt26dPP/3UalNaWhr0XU6nU7W1tZKkAwcOyOv1BrXJyspScXGx1aa9U6dOyefzBb0AAAAAoDdCGrJOnjypOXPmaMqUKVba83q9GjhwYFC71NRU9evXT16v12qTk5MT1Cbwvrs2bY+3/bmO2rS3ePFiZWVlWa/Bgwf3+pwBAAAAJLaQhayWlhZ997vflWEYWr58eai+xlbz5s1Tc3Oz9Tp8+HCkuwQAAAAgxoRknaxAwProo4+0cePGoDmLubm5Onr0aFD7M2fO6NixY8rNzbXaNDY2BrUJvO+uTdvjgX2DBg0KajNq1KgO+52enq709PTeni4AAAAAWGwfyQoErIaGBv3xj39U//79g46XlJSoqalJ9fX11r6NGzfK7/eruLjYarN582a1tLRYbaqrqzV06FBddtllVpsNGzYEfXZ1dbVKSkokSQUFBcrNzQ1q4/P5VFdXZ7UBAAAAALv1OmQdP35cO3fu1M6dOyWZBSZ27typQ4cOqaWlRd/+9re1fft2vfjii2ptbZXX65XX69Xp06clSVdddZVuv/12TZ8+XVu3btU777yjmTNnavLkycrLy5Mk3X333UpLS5PL5dKePXv06quv6plnntHs2bOtfjz44INat26dnnjiCe3du1eLFi3S9u3bNXPmTElm5cNZs2bp8ccf1x/+8Ae9//77uvfee5WXl9dlNUQAAAAAuBC9LuG+adMmfe1rXztn/7Rp07Ro0SIVFBR0+HM1NTUaM2aMJHMx4pkzZ+r1119XcnKyJk6cqGXLlumSSy6x2u/atUszZszQtm3bNGDAAD3wwAOaM2dO0GeuWrVKjz32mA4ePKiioiJVVFTozjvvtI4bhqGFCxeqqqpKTU1Nuvnmm/X888/ryiuv7NG5UsIdAAAAgNS7bHBB62TFu0QIWR6P1NAgFRVJDkekewMAAABEp6haJwvRy+2W8vOlsWPNrdsd6R4BAAAAsY+QlaA8HqmsTPL7zfd+v1Rebu4HAAAAcP4IWQmqoeFswApobZX2749MfwAAAIB4QchKUEVFUnK7f/2UFKmwMDL9AQAAAOIFIStBORxSVZUZrCRzW1lJ8QsAAADgQqVGugOIHJdLcjrNKYKFhQQsAAAAwA6ErATncBCuAAAAADsxXRAAAAAAbETIAgAAAAAbEbIAAAAAwEaELAAAAACwESELAAAAAGxEyAIAAAAAGxGyAAAAAMBGhCwAAAAAsBEhCwAAAABsRMiCxeORamrMLQAAAIDzQ8iCJMntlvLzpbFjza3bHekeAQAAALGJkAV5PFJZmeT3m+/9fqm8nBEtAAAA4HwQsqCGhrMBK6C1Vdq/PzL9AQAAAGIZIQsqKpKS290JKSlSYWFk+gMAAADEMkIW5HBIVVVmsJLMbWWluR8AAABA76RGugOIDi6X5HSaUwQLCwlYAAAAwPkiZMHicBCuAAAAgAvFdEEAAAAAsBEhCwAAAABsRMgCAAAAABsRsgAAAADARoQsAAAAALARIQsAAAAAbETIAgAAAAAbEbIAAAAAwEaELAAAAACwESELnfJ4pJoacwsAAACgZwhZ6JDbLeXnS2PHmlu3O9I9AgAAAGIDIQvn8HiksjLJ7zff+/1SeTkjWgAAAEBPELJwjoaGswEroLVV2r8/Mv0BAAAAYgkhC+coKpKS290ZKSlSYWFk+gMAAADEEkIWzuFwSFVVZrCSzG1lpbkfAAAAQNdSI90BRCeXS3I6zSmChYUELAAAAKCnCFnolMNBuAIAAAB6i+mCAAAAAGAjQhYAAAAA2IiQBQAAAAA2ImQBAAAAgI0IWQAAAABgI0IWAAAAANiIkAUAAAAANiJkAQAAAICNCFkAAAAAYCNCFnrE45FqaswtAAAAgM4RstAtt1vKz5fGjjW3bnekewQAAABEL0IWuuTxSGVlkt9vvvf7pfJyRrQAAACAzhCy0KWGhrMBK6C1Vdq/PzL9AQAAAKIdIQtdKiqSktvdJSkpUmFhZPoDAAAARDtCFrrkcEhVVWawksxtZaW5HwAAAMC5UiPdAUQ/l0tyOs0pgoWFBCwAAACgK4Qs9IjDQbgCAAAAeoLpggAAAABgI0IWAAAAANiIkAUAAAAANiJkAQAAAICNCFkAAAAAYCNCFgAAAADYiJAFAAAAADYiZOG8eDxSTY25BQAAAHAWIQu95nZL+fnS2LHm1u2OdI8AAACA6EHIQq94PFJZmeT3m+/9fqm8nBEtAAAAIKDXIWvz5s266667lJeXp6SkJK1ZsybouGEYWrBggQYNGqQ+ffqotLRUDQ0NQW2OHTumqVOnKjMzU9nZ2XK5XDp+/HhQm127dumWW25RRkaGBg8erIqKinP6smrVKg0bNkwZGRkaMWKE3nzzzV73Bb3T0HA2YAW0tkr790emPwAAAEC06XXIOnHihEaOHKnnnnuuw+MVFRVatmyZVqxYobq6OvXt21dOp1MnT5602kydOlV79uxRdXW11q5dq82bN6usrMw67vP5NG7cOOXn56u+vl5Lly7VokWLVFVVZbXZsmWLpkyZIpfLpffee08TJkzQhAkTtHv37l71Bb1TVCQlt7trUlKkwsLI9AcAAACIOsYFkGSsXr3aeu/3+43c3Fxj6dKl1r6mpiYjPT3dePnllw3DMIwPPvjAkGRs27bNavPWW28ZSUlJxl//+lfDMAzj+eefNy677DLj1KlTVps5c+YYQ4cOtd5/97vfNcaPHx/Un+LiYqO8vLzHfelOc3OzIclobm7uUftE8atfGUZKimFI5vZXv4p0jwAAAIDQ6k02sPWZrAMHDsjr9aq0tNTal5WVpeLiYtXW1kqSamtrlZ2drdGjR1ttSktLlZycrLq6OqvNrbfeqrS0NKuN0+nUvn379Omnn1pt2n5PoE3ge3rSl/ZOnToln88X9MK5XC7p4EGzuuDBg+Z7AAAAACZbQ5bX65Uk5eTkBO3Pycmxjnm9Xg0cODDoeGpqqvr16xfUpqPPaPsdnbVpe7y7vrS3ePFiZWVlWa/Bgwf34KwTk8MhjRljbgEAAACcRXXBNubNm6fm5mbrdfjw4Uh3CQAAAECMsTVk5ebmSpIaGxuD9jc2NlrHcnNzdfTo0aDjZ86c0bFjx4LadPQZbb+jszZtj3fXl/bS09OVmZkZ9AIAAACA3rA1ZBUUFCg3N1cbNmyw9vl8PtXV1amkpESSVFJSoqamJtXX11ttNm7cKL/fr+LiYqvN5s2b1dLSYrWprq7W0KFDddlll1lt2n5PoE3ge3rSFwAAAACwW69D1vHjx7Vz507t3LlTkllgYufOnTp06JCSkpI0a9YsPf744/rDH/6g999/X/fee6/y8vI0YcIESdJVV12l22+/XdOnT9fWrVv1zjvvaObMmZo8ebLy8vIkSXfffbfS0tLkcrm0Z88evfrqq3rmmWc0e/Zsqx8PPvig1q1bpyeeeEJ79+7VokWLtH37ds2cOVOSetQXAAAAALBdb0sX1tTUGJLOeU2bNs0wDLN0+vz5842cnBwjPT3duO2224x9+/YFfcbf//53Y8qUKcYll1xiZGZmGvfdd5/x2WefBbX5r//6L+Pmm2820tPTjX/6p38ylixZck5fXnvtNePKK6800tLSjC996UvGG2+8EXS8J33pCiXcAQAAABhG77JBkmEYRgQzXlTz+XzKyspSc3Mzz2cBAAAACaw32YDqggAAAABgI0IWbOHxmIsTezyR7gkAAAAQWYQsXDC3W8rPl8aONbdud6R7BAAAAEQOIQsXxOORysokv9987/dL5eWMaAEAACBxEbJwQRoazgasgNZWaf/+yPQHAAAAiDRCFi5IUZGU3O4uSkmRCgsj0x8AAAAg0ghZuCAOh1RVZQYrydxWVpr7AQAAgESUGukOIPa5XJLTaU4RLCwkYAEAACCxEbJgC4eDcAUAAABITBcEAAAAAFsRsgAAAADARoQsAAAAALARIQsAAAAAbETIAgAAAAAbEbIAAAAAwEaELNjO45FqaswtAAAAkGgIWbCV2y3l50tjx5pbtzvSPQIAAADCi5AF23g8UlmZ5Peb7/1+qbycES0AAAAkFkIWbNPQcDZgBbS2Svv3R6Y/AAAAQCQQsmCboiIpud0dlZIiFRZGpj8AAABAJBCyYBuHQ6qqMoOVZG4rK839AAAAQKJIjXQHEF9cLsnpNKcIFhYSsAAAAJB4CFmwncNBuAIAAEDiYrogAAAAANiIkAUAAAAANiJkAQAAAICNCFkAAAAAYCNCFgAAAADYiJAFAAAAADYiZCHkPB6ppsbcAgAAAPGOkIWQcrul/Hxp7Fhz63ZHukcAAABAaBGyEDIej1RWJvn95nu/XyovZ0QLAAAA8Y2QhZBpaDgbsAJaW6X9+yPTHwAAACAcCFkImaIiKbndHZaSIhUWRqY/AAAAQDgQshAyDodUVWUGK8ncVlaa+wEAAIB4lRrpDiC+uVyS02lOESwsJGABAAAg/hGyEHIOB+EKAAAAiYPpggAAAABgI0IWAAAAANiIkAUAAAAANiJkAQAAAICNCFkAAAAAYCNCFsLK45FqaswtAAAAEI8IWQgbt1vKz5fGjjW3bnekewQAAADYj5CFsPB4pLIyye833/v9Unk5I1oAAACIP4QshEVDw9mAFdDaKu3fH5n+AAAAAKFCyEJYFBVJye3utpQUqbAwMv0BAAAAQoWQhbBwOKSqKjNYSea2stLcDwAAAMST1Eh3AInD5ZKcTnOKYGEhAQsAAADxiZCFsHI4CFcAAACIb0wXBAAAAAAbEbIAAAAAwEaELAAAAACwESELAAAAAGxEyAIAAAAAGxGyEFEej1RTY24BAACAeEDIQsS43VJ+vjR2rLl1uyPdIwAAAODCEbIQER6PVFYm+f3me79fKi9nRAsAAACxj5CFiGhoOBuwAlpbpf37I9MfAAAAwC6ELEREUZGU3O7uS0mRCgsj0x8AAADALoQsRITDIVVVmcFKMreVleZ+AAAAIJalRroDSFwul+R0mlMECwsJWAAAAIgPhCxElMNBuAIAAEB8YbogAAAAANiIkAUAAAAANiJkAQAAAICNCFkAAAAAYCPbQ1Zra6vmz5+vgoIC9enTR1/84hf1b//2bzIMw2pjGIYWLFigQYMGqU+fPiotLVVDQ0PQ5xw7dkxTp05VZmamsrOz5XK5dPz48aA2u3bt0i233KKMjAwNHjxYFRUV5/Rn1apVGjZsmDIyMjRixAi9+eabdp8ybOLxSDU15hYAAACIVbaHrF/+8pdavny5nn32WX344Yf65S9/qYqKCv3Hf/yH1aaiokLLli3TihUrVFdXp759+8rpdOrkyZNWm6lTp2rPnj2qrq7W2rVrtXnzZpWVlVnHfT6fxo0bp/z8fNXX12vp0qVatGiRqqqqrDZbtmzRlClT5HK59N5772nChAmaMGGCdu/ebfdp4wK53VJ+vjR2rLl1uyPdIwAAAOD8JBlth5hs8PWvf105OTlyt/kreeLEierTp49++9vfyjAM5eXl6eGHH9YjjzwiSWpublZOTo5WrlypyZMn68MPP9Tw4cO1bds2jR49WpK0bt063XnnnfJ4PMrLy9Py5cv105/+VF6vV2lpaZKkuXPnas2aNdq7d68kadKkSTpx4oTWrl1r9eXGG2/UqFGjtGLFim7PxefzKSsrS83NzcrMzLTtGiGYx2MGK7//7L6UFOngQcq7AwAAIDr0JhvYPpJ10003acOGDfrv//5vSdJ//dd/6c9//rPuuOMOSdKBAwfk9XpVWlpq/UxWVpaKi4tVW1srSaqtrVV2drYVsCSptLRUycnJqqurs9rceuutVsCSJKfTqX379unTTz+12rT9nkCbwPe0d+rUKfl8vqAXQq+hIThgSVJrq7lIMQAAABBrbF+MeO7cufL5fBo2bJhSUlLU2tqqf//3f9fUqVMlSV6vV5KUk5MT9HM5OTnWMa/Xq4EDBwZ3NDVV/fr1C2pTUFBwzmcEjl122WXyer1dfk97ixcv1s9+9rPzOW1cgKIiKTn53JGswsLI9QkAAAA4X7aPZL322mt68cUX9dJLL2nHjh164YUX9H/+z//RCy+8YPdX2W7evHlqbm62XocPH450lxKCwyFVVZnBSjK3lZVMFQQAAEBssn0k69FHH9XcuXM1efJkSdKIESP00UcfafHixZo2bZpyc3MlSY2NjRo0aJD1c42NjRo1apQkKTc3V0ePHg363DNnzujYsWPWz+fm5qqxsTGoTeB9d20Cx9tLT09Xenr6+Zw2LpDLJTmd5hTBwkICFgAAAGKX7SNZn3/+uZKTgz82JSVF/v+dC1ZQUKDc3Fxt2LDBOu7z+VRXV6eSkhJJUklJiZqamlRfX2+12bhxo/x+v4qLi602mzdvVktLi9WmurpaQ4cO1WWXXWa1afs9gTaB70F0cTikMWMIWAAAAIhttoesu+66S//+7/+uN954QwcPHtTq1av15JNP6p//+Z8lSUlJSZo1a5Yef/xx/eEPf9D777+ve++9V3l5eZowYYIk6aqrrtLtt9+u6dOna+vWrXrnnXc0c+ZMTZ48WXl5eZKku+++W2lpaXK5XNqzZ49effVVPfPMM5o9e7bVlwcffFDr1q3TE088ob1792rRokXavn27Zs6cafdpAwAAAICkEJRw/+yzzzR//nytXr1aR48eVV5enqZMmaIFCxZYlQANw9DChQtVVVWlpqYm3XzzzXr++ed15ZVXWp9z7NgxzZw5U6+//rqSk5M1ceJELVu2TJdcconVZteuXZoxY4a2bdumAQMG6IEHHtCcOXOC+rNq1So99thjOnjwoIqKilRRUaE777yzR+dCCXcAAAAAUu+yge0hK54QsgAAAABIEV4nCwAAAAASGSELUcvjkWpqzC0AAAAQKwhZiEput5SfL40da27d7kj3CAAAAOgZQhaijscjlZVJ/1v1X36/VF7OiBYAAABiAyELUaeh4WzACmhtNRcqBgAAAKIdIQtRp6hIareetVJSpMLCyPQHAAAA6A1CFqKOwyFVVZnBSjK3lZXmfgAAACDapUa6A0BHXC7J6TSnCBYWErAAAAAQOwhZiFoOB+EKAAAAsYfpggAAAABgI0IWAAAAANiIkAUAAAAANiJkISZ4PFJNDQsSAwAAIPoRshD13G4pP18aO9bcut2R7hEAAADQOUIWoprHI5WVSX6/+d7vl8rLGdECAABA9CJkIao1NJwNWAGtreb6WQAAAEA0ImQhqhUVScnt7tKUFHOBYgAAACAaEbIQ1RwOqarKDFaSua2sZJFiAAAARK/USHcA6I7LJTmd5hTBwkICFgAAAKIbIQsxweEgXAEAACA2MF0QAAAAAGxEyAIAAAAAGxGyEJM8HqmmhvWyAAAAEH0IWYg5breUny+NHWtu3e5I9wgAAAA4i5CFmOLxSGVlZxco9vul8nJGtAAAABA9CFmIKQ0NZwNWQGurWd4dAAAAiAaELMSUoiIpud1dm5Jirp8FAAAARANCFmKKwyFVVZnBSjK3lZWsoQUAAIDowWLEiDkul+R0mlMECwsJWAAAAIguhCzEJIeDcAUAAIDoxHRBAAAAALARIQsAAAAAbETIAgAAAAAbEbIQ8zweqaaGBYkBAAAQHQhZiGlut5SfL40da27d7kj3CAAAAImOkIWY5fFIZWWS32++9/ul8nJGtAAAABBZhCzErIaGswEroLXVXD8LAAAAiBRCFmJWUZGU3O4OTkkxFygGAAAAIoWQhZjlcEhVVWawksxtZSWLFAMAACCyUiPdAeBCuFyS02lOESwsJGABAAAg8ghZiHkOB+EKAAAA0YPpggAAAABgI0IW4g6LEwMAACCSCFmIKyxODAAAgEgjZCFusDgxAAAAogEhC3GDxYkBAAAQDQhZiBssTgwAAIBoQMhC3GBxYgAAAEQD1slCXGFxYgAAAEQaIQtxh8WJAQAAEElMFwQAAAAAGxGyAAAAAMBGhCzENY9HqqlhrSwAAACEDyELccvtlvLzpbFjza3bHekeAQAAIBEQshCXPB6prOzs4sR+v1RezogWAAAAQo+QhbjU0HA2YAW0tpql3QEAAIBQImQhLhUVScnt7u6UFHPtLAAAACCUCFmISw6HVFVlBivJ3FZWsn4WAAAAQo/FiBG3XC7J6TSnCBYWErAAAAAQHoQsxDWHg3AFAACA8GK6IAAAAADYiJCFhMLixAAAAAg1QhYSBosTAwAAIBwIWUgILE4MAACAcCFkISGwODEAAADChZCFhMDixAAAAAgXQhYSAosTAwAAIFxYJwsJg8WJAQAAEA4hGcn661//qu9973vq37+/+vTpoxEjRmj79u3WccMwtGDBAg0aNEh9+vRRaWmpGhoagj7j2LFjmjp1qjIzM5WdnS2Xy6Xjx48Htdm1a5duueUWZWRkaPDgwaqoqDinL6tWrdKwYcOUkZGhESNG6M033wzFKSNGOBzSmDEELAAAAISO7SHr008/1Ve+8hVddNFFeuutt/TBBx/oiSee0GWXXWa1qaio0LJly7RixQrV1dWpb9++cjqdOnnypNVm6tSp2rNnj6qrq7V27Vpt3rxZZWVl1nGfz6dx48YpPz9f9fX1Wrp0qRYtWqSqqiqrzZYtWzRlyhS5XC699957mjBhgiZMmKDdu3fbfdoAAAAAIElKMgzDsPMD586dq3feeUd/+tOfOjxuGIby8vL08MMP65FHHpEkNTc3KycnRytXrtTkyZP14Ycfavjw4dq2bZtGjx4tSVq3bp3uvPNOeTwe5eXlafny5frpT38qr9ertLQ067vXrFmjvXv3SpImTZqkEydOaO3atdb333jjjRo1apRWrFjR7bn4fD5lZWWpublZmZmZF3RdEJ08HrPyYFERo1sAAADoXG+yge0jWX/4wx80evRofec739HAgQP15S9/Wf/3//5f6/iBAwfk9XpVWlpq7cvKylJxcbFqa2slSbW1tcrOzrYCliSVlpYqOTlZdXV1Vptbb73VCliS5HQ6tW/fPn366adWm7bfE2gT+J72Tp06JZ/PF/RC/GJxYgAAAISC7SHrf/7nf7R8+XIVFRVp/fr1uv/++/XjH/9YL7zwgiTJ6/VKknJycoJ+Licnxzrm9Xo1cODAoOOpqanq169fUJuOPqPtd3TWJnC8vcWLFysrK8t6DR48uNfnj9jA4sQAAAAIFdtDlt/v17XXXqtf/OIX+vKXv6yysjJNnz69R9PzIm3evHlqbm62XocPH450lxAiLE4MAACAULE9ZA0aNEjDhw8P2nfVVVfp0KFDkqTc3FxJUmNjY1CbxsZG61hubq6OHj0adPzMmTM6duxYUJuOPqPtd3TWJnC8vfT0dGVmZga9EJ9YnBgAAAChYnvI+spXvqJ9+/YF7fvv//5v5efnS5IKCgqUm5urDRs2WMd9Pp/q6upUUlIiSSopKVFTU5Pq6+utNhs3bpTf71dxcbHVZvPmzWppabHaVFdXa+jQoVYlw5KSkqDvCbQJfA8SF4sTAwAAIFRsry64bds23XTTTfrZz36m7373u9q6daumT5+uqqoqTZ06VZL0y1/+UkuWLNELL7yggoICzZ8/X7t27dIHH3ygjIwMSdIdd9yhxsZGrVixQi0tLbrvvvs0evRovfTSS5LMioRDhw7VuHHjNGfOHO3evVs/+MEP9NRTT1ml3rds2aKvfvWrWrJkicaPH69XXnlFv/jFL7Rjxw5dffXV3Z4L1QXjn8fD4sQAAADoXm+yge0hS5LWrl2refPmqaGhQQUFBZo9e7amT59uHTcMQwsXLlRVVZWampp088036/nnn9eVV15ptTl27Jhmzpyp119/XcnJyZo4caKWLVumSy65xGqza9cuzZgxQ9u2bdOAAQP0wAMPaM6cOUF9WbVqlR577DEdPHhQRUVFqqio0J133tmj8yBkAQAAAJCiIGTFC0JWYmHNLAAAAHQmoutkAbGINbMAAABgF0IWEh5rZgEAAMBOhCwkPNbMAgAAgJ0IWUh4rJkFAAAAOxGykPBYMwsAAAB2So10B4Bo4HJJTidrZgEAAODCEbKA/+VwEK4AAABw4ZguCHTC45FqaqgyCAAAgN4hZAEdYN0sAAAAnC9CFtAO62YBAADgQhCygHZYNwsAAAAXgpAFtMO6WQAAALgQhCygHdbNAgAAwIWghDvQAdbNAgAAwPkiZAGdYN0sAAAAnA+mCwI9wJpZAAAA6ClCFtAN1swCAABAbxCygC6wZhYAAAB6i5AFdIE1swAAANBbhCygC6yZBQAAgN4iZAFdYM0sAAAA9BYl3IFusGYWAAAAeoOQBfRA+zWzPB7zea2iIkIXAAAAgjFdEOglSroDAACgK4QsoBco6Q4AAIDuELKAXqCkOwAAALpDyAJ6gZLuAAAA6A4hC+gFSroDAACgO1QXBHqJku4AAADoCiELOA9tS7pTzh0AAABtMV0QuACUcwcAAEB7hCzgPFHOHQAAAB0hZAHniXLuAAAA6AghCzhPlHMHAABARwhZwHminDsAAAA6QnVB4AJQzh0AAADtEbKAC9S2nLtESXcAAIBEx3RBwEaUdAcAAAAhC7AJJd0BAAAgEbIA21DSHQAAABIhC7ANJd0BAAAgEbIA21DSHQAAABLVBQFbtS/pLkk1NVQaBAAASCSMZAE2czikMWOk9eupNAgAAJCICFlACFBpEAAAIHERsoAQoNIgAABA4iJkASFApUEAAIDERcgCQoBKgwAAAImL6oJAiLSvNEjAAgAASAyELCCEHI7gcOXxmM9rUdIdAAAgfjFdEAgTt5uS7gAAAImAkAWEASXdAQAAEgchCwgDSroDAAAkDkIWEAaUdAcAAEgchCwgDCjpDgAAkDioLgiESfuS7pJUU0OlQQAAgHjDSBYQRg6HNGaMtH49lQYBAADiFSELCDMqDQIAAMQ3QhYQZlQaBAAAiG+ELCDMqDQIAAAQ3whZQJh1VmlQMgthMG0QAAAgthGygAhwuaSDB81QdfCguY9CGAAAAPEhyTAMI9KdiFY+n09ZWVlqbm5WZmZmpLuDOOXxmMGq7XNaKSlm+KK0OwAAQHToTTZgJAuIMAphAAAAxBdCFhBhFMIAAACIL4QsIMI6K4TBVEEAAIDYlBrpDgAwC2E4neYUwcAIVk2NOcpF2AIAAIgtIR/JWrJkiZKSkjRr1ixr38mTJzVjxgz1799fl1xyiSZOnKjGxsagnzt06JDGjx+viy++WAMHDtSjjz6qM2fOBLXZtGmTrr32WqWnp6uwsFArV6485/ufe+45XXHFFcrIyFBxcbG2bt0aitMELpjDIY0ZI61fT6VBAACAWBbSkLVt2zZVVlbqmmuuCdr/0EMP6fXXX9eqVav09ttv68iRI/rWt75lHW9tbdX48eN1+vRpbdmyRS+88IJWrlypBQsWWG0OHDig8ePH62tf+5p27typWbNm6Yc//KHWr19vtXn11Vc1e/ZsLVy4UDt27NDIkSPldDp19OjRUJ42cN48Hqms7GwhDL9fKi9n7SwAAIBYErIS7sePH9e1116r559/Xo8//rhGjRqlp59+Ws3Nzbr88sv10ksv6dvf/rYkae/evbrqqqtUW1urG2+8UW+99Za+/vWv68iRI8rJyZEkrVixQnPmzNEnn3yitLQ0zZkzR2+88YZ2795tfefkyZPV1NSkdevWSZKKi4t1/fXX69lnn5Uk+f1+DR48WA888IDmzp3b7TlQwh3hVlNjjmB1tH/MmLB3BwAAAP8rKkq4z5gxQ+PHj1dpaWnQ/vr6erW0tATtHzZsmIYMGaLa2lpJUm1trUaMGGEFLElyOp3y+Xzas2eP1ab9ZzudTuszTp8+rfr6+qA2ycnJKi0ttdq0d+rUKfl8vqAXEE5UGgQAAIh9IQlZr7zyinbs2KHFixefc8zr9SotLU3Z2dlB+3NycuT1eq02bQNW4HjgWFdtfD6f/vGPf+hvf/ubWltbO2wT+Iz2Fi9erKysLOs1ePDgnp80YIPOKg1K5mgW0wYBAACin+0h6/Dhw3rwwQf14osvKiMjw+6PD6l58+apubnZeh0+fDjSXUICcrmkgwfNUHXwoLmPQhgAAACxw/aQVV9fr6NHj+raa69VamqqUlNT9fbbb2vZsmVKTU1VTk6OTp8+raampqCfa2xsVG5uriQpNzf3nGqDgffdtcnMzFSfPn00YMAApaSkdNgm8BntpaenKzMzM+gFREKg0qBEIQwAAIBYY3vIuu222/T+++9r586d1mv06NGaOnWq9d8XXXSRNmzYYP3Mvn37dOjQIZWUlEiSSkpK9P777wdVAayurlZmZqaGDx9utWn7GYE2gc9IS0vTddddF9TG7/drw4YNVhsg2jU0nA1YAa2t5npaAAAAiE62L0Z86aWX6uqrrw7a17dvX/Xv39/a73K5NHv2bPXr10+ZmZl64IEHVFJSohtvvFGSNG7cOA0fPlz33HOPKioq5PV69dhjj2nGjBlKT0+XJP3oRz/Ss88+q5/85Cf6wQ9+oI0bN+q1117TG2+8YX3v7NmzNW3aNI0ePVo33HCDnn76aZ04cUL33Xef3acNhESgEEbboEUhDAAAgOhme8jqiaeeekrJycmaOHGiTp06JafTqeeff946npKSorVr1+r+++9XSUmJ+vbtq2nTpunnP/+51aagoEBvvPGGHnroIT3zzDNyOBz61a9+JafTabWZNGmSPvnkEy1YsEBer1ejRo3SunXrzimGAUSrQCGM8nJzBKt9IYyiIrMNAAAAokfI1smKB6yThWjh8ZhTBAsLpfXrzz6nlZxshjCXK9I9BAAAiG+9yQaErC4QshBtPB6zwmD76YMHDzKiBQAAEEpRsRgxAPtRCAMAACD6EbKAGBIohNEWhTAAAACiCyELiCGBQhgpKeb7lBRp8WJzhIu1swAAAKJDRKoLAjh/LpfkdJpTBLdvl+bMoQgGAABANKHwRRcofIFoRhEMAACA8KHwBZAAKIIBAAAQnQhZQIyiCAYAAEB0ImQBMaqjIhiVleZ/19RQCAMAACBSCFlADHO5zGewamrMrWQ+pzV2rLl1uyPZOwAAgMRE4YsuUPgCsYRCGAAAAKFD4QsgAVEIAwAAIDoQsoA40VEhjORk6ehRns8CAAAIJ0IWECfaF8JISpIMQ5o0ieezAAAAwomQBcSRQCGM1147G7IkcxpheTkjWgAAAOFAyALijMMhDRjA81kAAACRQsgC4hALFQMAAEQOIQuIQyxUDAAAEDmELCBOsVAxAABAZLAYcRdYjBjxgoWKAQAALgyLEQMIwkLFAAAA4UPIAhIACxUDAACEDyELSAAsVAwAABA+hCwgQbBQMQAAQHgQsoAEwkLFAAAAoUfIAhJMZwsV9+3LGloAAAB2IGQBCaajhYq/9z3pxhtZQwsAAMAOrJPVBdbJQjzzeMwpgn37mgGLNbQAAAA6xzpZALrlcEhjxkjHj3f8jNaqVUwdBAAAOB+ELCDBdfSMliTNns3UQQAAgPNByAISXPtntNqivDsAAEDvEbIAWGtoPfnkucco7w4AANA7hCwAkswRre98h/LuAAAAF4qQBcBCeXcAAIALRwn3LlDCHYmK8u4AAADBKOEO4IJ0V96dZ7QAAAA6R8gC0KmOyrsnJ0tHj/J8FgAAQGcIWQA61f4ZraQkyTCkSZN4PgsAAKAzhCwAXQqUd3/ttbMhS2INLQAAgM6kRroDAKKfwyENGNDx81m1teaxoiKKYQAAAEiMZAHooY6ez0pKkiZPprw7AABAW4QsAD3S/vmsQOAKjG4xfRAAAMBEyALQY4Hns2pqpJdfPvt8VkBg+mBNDWELAAAkLp7JAtArDof58njM0ay2z2kFpg/6/eaxqiozmAEAACQSRrIAnBemDwIAAHSMkAXgvPVk+uCqVQQtAACQWAhZAC6IwyGNGSPddNO51QclafZsKg8CAIDEQsgCYIv20wfbYuogAABIJIQsALYJTB988slzj1F5EAAAJApCFgBbORzSd77DwsUAACBxEbIA2I7KgwAAIJERsgCEBJUHAQBAoiJkAQgZKg8CAIBERMgCEHJUHgQAAImEkAUgLKg8CAAAEgUhC0DYUHkQAAAkAkIWgLDqSeXBsjLptdcY1QIAALGJkAUg7LqrPOj3S5MmMaoFAABiEyELQER0V3lQoigGAACITYQsABHVVeVBiaIYAAAg9hCyAERcYPrga69RFAMAAMQ+QhaAqBCoPEhRDAAAEOsIWQCiCkUxAABArCNkAYg6FMUAAACxjJAFIGpRFAMAAMQiQhaAqEZRDAAAEGsIWQCiHkUxAABALCFkAYgZFMUAAACxwPaQtXjxYl1//fW69NJLNXDgQE2YMEH79u0LanPy5EnNmDFD/fv31yWXXKKJEyeqsbExqM2hQ4c0fvx4XXzxxRo4cKAeffRRnTlzJqjNpk2bdO211yo9PV2FhYVauXLlOf157rnndMUVVygjI0PFxcXaunWr3acMIIx6UxRj2zae1wIAAOFne8h6++23NWPGDL377ruqrq5WS0uLxo0bpxMnTlhtHnroIb3++utatWqV3n77bR05ckTf+ta3rOOtra0aP368Tp8+rS1btuiFF17QypUrtWDBAqvNgQMHNH78eH3ta1/Tzp07NWvWLP3whz/U+vXrrTavvvqqZs+erYULF2rHjh0aOXKknE6njh49avdpAwiznhTFuPFGntcCAADhl2QY7Sfc2OuTTz7RwIED9fbbb+vWW29Vc3OzLr/8cr300kv69re/LUnau3evrrrqKtXW1urGG2/UW2+9pa9//es6cuSIcnJyJEkrVqzQnDlz9MknnygtLU1z5szRG2+8od27d1vfNXnyZDU1NWndunWSpOLiYl1//fV69tlnJUl+v1+DBw/WAw88oLlz53bbd5/Pp6ysLDU3NyszM9PuSwPABh6PWWFw8uSzz2d1JDnZnGJ4001mQAMAAOiN3mSDkD+T1dzcLEnq16+fJKm+vl4tLS0qLS212gwbNkxDhgxRbW2tJKm2tlYjRoywApYkOZ1O+Xw+7dmzx2rT9jMCbQKfcfr0adXX1we1SU5OVmlpqdWmvVOnTsnn8wW9AES3ropitMXzWgAAIFxCGrL8fr9mzZqlr3zlK7r66qslSV6vV2lpacrOzg5qm5OTI6/Xa7VpG7ACxwPHumrj8/n0j3/8Q3/729/U2traYZvAZ7S3ePFiZWVlWa/Bgwef34kDCLu2RTHefZfntQAAQOSENGTNmDFDu3fv1iuvvBLKr7HNvHnz1NzcbL0OHz4c6S4B6IVAUYzrr+d5LQAAEDkhC1kzZ87U2rVrVVNTI0ebByByc3N1+vRpNTU1BbVvbGxUbm6u1aZ9tcHA++7aZGZmqk+fPhowYIBSUlI6bBP4jPbS09OVmZkZ9AIQm7paxFgKXl+LkS0AAGAn20OWYRiaOXOmVq9erY0bN6qgoCDo+HXXXaeLLrpIGzZssPbt27dPhw4dUklJiSSppKRE77//flAVwOrqamVmZmr48OFWm7afEWgT+Iy0tDRdd911QW38fr82bNhgtQEQ33r6vBYjWwAAwE62Vxf8l3/5F7300kv6/e9/r6FDh1r7s7Ky1KdPH0nS/fffrzfffFMrV65UZmamHnjgAUnSli1bJJkl3EeNGqW8vDxVVFTI6/Xqnnvu0Q9/+EP94he/kGSWcL/66qs1Y8YM/eAHP9DGjRv14x//WG+88YacTqcks4T7tGnTVFlZqRtuuEFPP/20XnvtNe3du/ecZ7U6QnVBIH54PNL+/VLfvmagohIhAADojd5kA9tDVlJSUof7f/Ob3+j73/++JHMx4ocfflgvv/yyTp06JafTqeeffz5oGt9HH32k+++/X5s2bVLfvn01bdo0LVmyRKmpqVabTZs26aGHHtIHH3wgh8Oh+fPnW98R8Oyzz2rp0qXyer0aNWqUli1bpuLi4h6dCyELiE9utzlFsLXVDFSdBa7kZHMUzOmUGhqkoiJCFwAAiSqiISueELKA+NXTka2kJPPl958NXS5XePsKAAAirzfZILXLowAQpxyOs6NSVVVnR7baMwzzJZlBq6xMuvRSphICAIDOhXwxYgCIdt1VImyLRY0BAEB3GMkCAJ2tROjzBT+v1XYkq61A6fdrrpGOH+d5LQAAcBbPZHWBZ7KAxBR4XquwUFq/vvOphNLZwhnJydKSJdLo0QQuAADiEYUvbELIAiCZoau2Vpo8uevS7wEUyAAAIP5Q+AIAbNTZVMLOAhdTCQEASGyMZHWBkSwA7fV2UWOmEgIAEB96kw2oLggAveBwSGPGSNdfb04JTEnpvG0ggPn90k9+Io0dS1VCAAASASNZXWAkC0B3AiNb27dLc+d2P5VQMoNZbS1TCQEAiCUUvrAJIQtAbzCVEACA+EXIsgkhC8D5cru7Lv3ekUBVQqdTamggdAEAEE0IWTYhZAG4EOczlTApyXwxygUAQHQhZNmEkAXALr2ZStgR1t4CACCyWCcLAKKMw3F2JKqqqvdTCVl7CwCA2EEJdwAIM5dLOnhQqqmRli49WwY+OdmcKtiZ1lZzFKxtKXiPx/wcjycsXQcAAD3AdMEuMF0QQDgEphIWFkrr1/d8lIvntwAACB+eybIJIQtAJJxPwYy2qFIIAID9CFk2IWQBiLTzLZjBKBcAAPYiZNmEkAUgmrRdeys5WTIM89UbjHIBAHB+CFk2IWQBiDbn+/xWW92Ncnk8BDAAANojZNmEkAUg2nX2/Nb5jnLdc4/0//4f0wwBAGiPkGUTQhaAWGLHKFdnmGYIAEh0hCybELIAxDI7R7kkimkAABIbIcsmhCwA8SISo1w82wUAiCeELJsQsgDEq45GuVJSpO99T/rtby+8mAbPdgEA4g0hyyaELACJoO0oV2AEys5phm11NOolMeIFAIh+hCybELIAJLpQTDNsO+qVlGTuMwyKawAAohshyyaELAAIFspRLok1vAAA0YuQZRNCFgB0rrNRrgt5tqszPXnOiwAGAAglQpZNCFkA0HPhfLarLQIYACAcCFk2IWQBgD06G/VqPz2QAAYAiFaELJsQsgAgNNqGLil0a3h1paMA1l3lQwIZACQuQpZNCFkAEH52r+HVG11VPmREDAASGyHLJoQsAIisnjznFa4A1pneBjCJMAYAsYiQZRNCFgBEr1gMYN2tCyZ1Pj2x/TEAQHgRsmxCyAKA2HQ+ASwUhTe609PpiT1ZtJlABgChRciyCSELAOJPZwGsq8qHkR4Ra699VcZQjZbxnBkAnEXIsgkhCwAST2eVD6N1SmJ3zne07EKeM2sfzghrAOIBIcsmhCwAQGfOZ0piqNcFC4fehjO7wlpXxwhuAMKBkGUTQhYA4Hx0NSVR6v30xHgIZ+31JqyFapStq2M85wagPUKWTQhZAIBQ68n0xK7CWSIEsp6yM7iF4zk3u44BCA9Clk0IWQCAaBXK0bJof84s0ux4zs2uY72tNBnJY4RFxDpClk0IWQCAeHA+o2Xn85xZ+2OEtdDrTaXJSB6LtdHB3oRDRicTByHLJoQsAABMPR05a3/MjrDGKFv8i6bRwd6Ew3CMTkqxEw7jfcSTkGUTQhYAAPY637DW2bFQBDeec0Mk2bXsQrSOal7oiKfLFdrr3xVClk0IWQAARD+7g1s4nnOz4xiBD4kmJUU6eDByI1qELJsQsgAAQEfsCHJ2HOtppclIHyMswi41NdKYMZH5bkKWTQhZAAAg2oViJC8Ux2JhdLA34ZDRyfBjJCtOELIAAADCIxbDYahHJ2MlHIYr1FZW8kxWXCBkAQAAINxiMRyGK9RGEiHLJoQsAAAAAFLvskFymPoEAAAAAAmBkAUAAAAANiJkAQAAAICNCFkAAAAAYCNCFgAAAADYiJAFAAAAADYiZAEAAACAjQhZAAAAAGAjQhYAAAAA2IiQBQAAAAA2ImQBAAAAgI0IWQAAAABgI0IWAAAAANiIkAUAAAAANiJkAQAAAICNCFkAAAAAYKOECFnPPfecrrjiCmVkZKi4uFhbt26NdJcAAAAAxKm4D1mvvvqqZs+erYULF2rHjh0aOXKknE6njh49GumuAQAAAIhDcR+ynnzySU2fPl333Xefhg8frhUrVujiiy/Wr3/960h3DQAAAEAciuuQdfr0adXX16u0tNTal5ycrNLSUtXW1p7T/tSpU/L5fEEvAAAAAOiNuA5Zf/vb39Ta2qqcnJyg/Tk5OfJ6vee0X7x4sbKysqzX4MGDw9VVAAAAAHEirkNWb82bN0/Nzc3W6/Dhw5HuEgAAAIAYkxrpDoTSgAEDlJKSosbGxqD9jY2Nys3NPad9enq60tPTrfeGYUgS0wYBAACABBfIBIGM0JW4DllpaWm67rrrtGHDBk2YMEGS5Pf7tWHDBs2cObPbn//ss88kiWmDAAAAACSZGSErK6vLNnEdsiRp9uzZmjZtmkaPHq0bbrhBTz/9tE6cOKH77ruv25/Ny8vT4cOHdemllyopKSkMve2az+fT4MGDdfjwYWVmZka6O3GH6xt6XOPQ4vqGHtc4tLi+occ1Di2ub+hF8hobhqHPPvtMeXl53baN+5A1adIkffLJJ1qwYIG8Xq9GjRqldevWnVMMoyPJyclyOBxh6GXvZGZm8j/cEOL6hh7XOLS4vqHHNQ4trm/ocY1Di+sbepG6xt2NYAXEfciSpJkzZ/ZoeiAAAAAAXCiqCwIAAACAjQhZMSQ9PV0LFy4MqoAI+3B9Q49rHFpc39DjGocW1zf0uMahxfUNvVi5xklGT2oQAgAAAAB6hJEsAAAAALARIQsAAAAAbETIAgAAAAAbEbIAAAAAwEaELAAAAACwESErRjz33HO64oorlJGRoeLiYm3dujXSXYpJixcv1vXXX69LL71UAwcO1IQJE7Rv376gNmPGjFFSUlLQ60c/+lGEehx7Fi1adM71GzZsmHX85MmTmjFjhvr3769LLrlEEydOVGNjYwR7HFuuuOKKc65vUlKSZsyYIYn793xs3rxZd911l/Ly8pSUlKQ1a9YEHTcMQwsWLNCgQYPUp08flZaWqqGhIajNsWPHNHXqVGVmZio7O1sul0vHjx8P41lEt66ucUtLi+bMmaMRI0aob9++ysvL07333qsjR44EfUZH9/6SJUvCfCbRqbt7+Pvf//451+72228PasM93LXurnFHv5eTkpK0dOlSqw33cOd68vdZT/5+OHTokMaPH6+LL75YAwcO1KOPPqozZ86E81QshKwY8Oqrr2r27NlauHChduzYoZEjR8rpdOro0aOR7lrMefvttzVjxgy9++67qq6uVktLi8aNG6cTJ04EtZs+fbo+/vhj61VRURGhHsemL33pS0HX789//rN17KGHHtLrr7+uVatW6e2339aRI0f0rW99K4K9jS3btm0LurbV1dWSpO985ztWG+7f3jlx4oRGjhyp5557rsPjFRUVWrZsmVasWKG6ujr17dtXTqdTJ0+etNpMnTpVe/bsUXV1tdauXavNmzerrKwsXKcQ9bq6xp9//rl27Nih+fPna8eOHfrd736nffv26Rvf+MY5bX/+858H3dsPPPBAOLof9bq7hyXp9ttvD7p2L7/8ctBx7uGudXeN217bjz/+WL/+9a+VlJSkiRMnBrXjHu5YT/4+6+7vh9bWVo0fP16nT5/Wli1b9MILL2jlypVasGBBJE5JMhD1brjhBmPGjBnW+9bWViMvL89YvHhxBHsVH44ePWpIMt5++21r31e/+lXjwQcfjFynYtzChQuNkSNHdnisqanJuOiii4xVq1ZZ+z788ENDklFbWxumHsaXBx980PjiF79o+P1+wzC4fy+UJGP16tXWe7/fb+Tm5hpLly619jU1NRnp6enGyy+/bBiGYXzwwQeGJGPbtm1Wm7feestISkoy/vrXv4at77Gi/TXuyNatWw1JxkcffWTty8/PN5566qnQdi4OdHR9p02bZnzzm9/s9Ge4h3unJ/fwN7/5TWPs2LFB+7iHe67932c9+fvhzTffNJKTkw2v12u1Wb58uZGZmWmcOnUqvCdgGAYjWVHu9OnTqq+vV2lpqbUvOTlZpaWlqq2tjWDP4kNzc7MkqV+/fkH7X3zxRQ0YMEBXX3215s2bp88//zwS3YtZDQ0NysvL0xe+8AVNnTpVhw4dkiTV19erpaUl6H4eNmyYhgwZwv18Hk6fPq3f/va3+sEPfqCkpCRrP/evfQ4cOCCv1xt0z2ZlZam4uNi6Z2tra5Wdna3Ro0dbbUpLS5WcnKy6urqw9zkeNDc3KykpSdnZ2UH7lyxZov79++vLX/6yli5dGrFpQLFo06ZNGjhwoIYOHar7779ff//7361j3MP2amxs1BtvvCGXy3XOMe7hnmn/91lP/n6ora3ViBEjlJOTY7VxOp3y+Xzas2dPGHtvSg37N6JX/va3v6m1tTXohpGknJwc7d27N0K9ig9+v1+zZs3SV77yFV199dXW/rvvvlv5+fnKy8vTrl27NGfOHO3bt0+/+93vItjb2FFcXKyVK1dq6NCh+vjjj/Wzn/1Mt9xyi3bv3i2v16u0tLRz/nDKycmR1+uNTIdj2Jo1a9TU1KTvf//71j7uX3sF7suOfgcHjnm9Xg0cODDoeGpqqvr168d9fR5OnjypOXPmaMqUKcrMzLT2//jHP9a1116rfv36acuWLZo3b54+/vhjPfnkkxHsbWy4/fbb9a1vfUsFBQX6y1/+on/913/VHXfcodraWqWkpHAP2+yFF17QpZdees5UeO7hnuno77Oe/P3g9Xo7/F0dOBZuhCwkrBkzZmj37t1BzwtJCpqDPmLECA0aNEi33Xab/vKXv+iLX/xiuLsZc+644w7rv6+55hoVFxcrPz9fr732mvr06RPBnsUft9utO+64Q3l5edY+7l/EspaWFn33u9+VYRhavnx50LHZs2db/33NNdcoLS1N5eXlWrx4sdLT08Pd1ZgyefJk679HjBiha665Rl/84he1adMm3XbbbRHsWXz69a9/ralTpyojIyNoP/dwz3T291msYbpglBswYIBSUlLOqZ7S2Nio3NzcCPUq9s2cOVNr165VTU2NHA5Hl22Li4slSfv37w9H1+JOdna2rrzySu3fv1+5ubk6ffq0mpqagtpwP/feRx99pD/+8Y/64Q9/2GU77t8LE7gvu/odnJube04hojNnzujYsWPc170QCFgfffSRqqurg0axOlJcXKwzZ87o4MGD4elgHPnCF76gAQMGWL8XuIft86c//Un79u3r9nezxD3ckc7+PuvJ3w+5ubkd/q4OHAs3QlaUS0tL03XXXacNGzZY+/x+vzZs2KCSkpII9iw2GYahmTNnavXq1dq4caMKCgq6/ZmdO3dKkgYNGhTi3sWn48eP6y9/+YsGDRqk6667ThdddFHQ/bxv3z4dOnSI+7mXfvOb32jgwIEaP358l+24fy9MQUGBcnNzg+5Zn8+nuro6654tKSlRU1OT6uvrrTYbN26U3++3Qi66FghYDQ0N+uMf/6j+/ft3+zM7d+5UcnLyOdPc0D2Px6O///3v1u8F7mH7uN1uXXfddRo5cmS3bbmHz+ru77Oe/P1QUlKi999/P+j/MAj8HzbDhw8Pz4m0FfZSG+i1V155xUhPTzdWrlxpfPDBB0ZZWZmRnZ0dVD0FPXP//fcbWVlZxqZNm4yPP/7Yen3++eeGYRjG/v37jZ///OfG9u3bjQMHDhi///3vjS984QvGrbfeGuGex46HH37Y2LRpk3HgwAHjnXfeMUpLS40BAwYYR48eNQzDMH70ox8ZQ4YMMTZu3Ghs377dKCkpMUpKSiLc69jS2tpqDBkyxJgzZ07Qfu7f8/PZZ58Z7733nvHee+8Zkownn3zSeO+996zKdkuWLDGys7ON3//+98auXbuMb37zm0ZBQYHxj3/8w/qM22+/3fjyl79s1NXVGX/+85+NoqIiY8qUKZE6pajT1TU+ffq08Y1vfMNwOBzGzp07g343ByqCbdmyxXjqqaeMnTt3Gn/5y1+M3/72t8bll19u3HvvvRE+s+jQ1fX97LPPjEceecSora01Dhw4YPzxj380rr32WqOoqMg4efKk9Rncw13r7veEYRhGc3OzcfHFFxvLly8/5+e5h7vW3d9nhtH93w9nzpwxrr76amPcuHHGzp07jXXr1hmXX365MW/evEickkHIihH/8R//YQwZMsRIS0szbrjhBuPdd9+NdJdikqQOX7/5zW8MwzCMQ4cOGbfeeqvRr18/Iz093SgsLDQeffRRo7m5ObIdjyGTJk0yBg0aZKSlpRn/9E//ZEyaNMnYv3+/dfwf//iH8S//8i/GZZddZlx88cXGP//zPxsff/xxBHsce9avX29IMvbt2xe0n/v3/NTU1HT4e2HatGmGYZhl3OfPn2/k5OQY6enpxm233XbOtf/73/9uTJkyxbjkkkuMzMxM47777jM+++yzCJxNdOrqGh84cKDT3801NTWGYRhGfX29UVxcbGRlZRkZGRnGVVddZfziF78ICgmJrKvr+/nnnxvjxo0zLr/8cuOiiy4y8vPzjenTp5/zf9RyD3etu98ThmEYlZWVRp8+fYympqZzfp57uGvd/X1mGD37++HgwYPGHXfcYfTp08cYMGCA8fDDDxstLS1hPhtTkmEYRogGyQAAAAAg4fBMFgAAAADYiJAFAAAAADYiZAEAAACAjQhZAAAAAGAjQhYAAAAA2IiQBQAAAAA2ImQBAAAAgI0IWQAAAABgI0IWAAAAANiIkAUAAAAANiJkAQAAAICN/j+A/UnC+NdC5wAAAABJRU5ErkJggg==\n"
          },
          "metadata": {}
        }
      ]
    },
    {
      "cell_type": "markdown",
      "metadata": {
        "id": "JYhOp3fjnh2G"
      },
      "source": [
        "# Stochastic Gradient Descent"
      ]
    },
    {
      "cell_type": "markdown",
      "source": [
        "In Batch Gradient Descent we were considering all the examples for every step of Gradient Descent. But what if our dataset is very huge. Deep learning models crave for data. The more the data the more chances of a model to be good. Suppose our dataset has 5 million examples, then just to take one step the model will have to calculate the gradients of all the 5 million examples. This does not seem an efficient way. To tackle this problem we have Stochastic Gradient Descent. In Stochastic Gradient Descent (SGD), we consider just one example at a time to take a single step."
      ],
      "metadata": {
        "id": "10N2dcwWUctJ"
      }
    },
    {
      "cell_type": "code",
      "metadata": {
        "id": "aVwD7Cqw8m1d"
      },
      "source": [
        "def stocashtic_gradient_descent(X, y, y_pred, learning_rate=0.01, iterations=10):\n",
        "    '''\n",
        "    X = Matrix of X with added bias units\n",
        "    y = Vector of Y\n",
        "    y_pred = Vector of y_pred np.random.randn(j,1)\n",
        "    learning_rate\n",
        "    iterations = no of iterations\n",
        "\n",
        "    Returns the final y_pred vector and array of cost history over no of iterations\n",
        "    '''\n",
        "\n",
        "    m = len(y)\n",
        "    cost_history = np.zeros(iterations)\n",
        "\n",
        "    for it in range(iterations):\n",
        "        cost = 0.0\n",
        "\n",
        "        for i in range(m):\n",
        "            rand_ind = np.random.randint(0,m)\n",
        "            X_i = X[rand_ind, :].reshape(1, X.shape[1])\n",
        "            y_i = y[rand_ind].reshape(1,1)\n",
        "            prediction = np.dot(X_i, y_pred)\n",
        "\n",
        "            y_pred = y_pred - (1 / m) * learning_rate *(X_i.T.dot((prediction - y_i)))\n",
        "            cost += cal_cost(y_pred, X_i, y_i)\n",
        "\n",
        "        cost_history[it]  = cost\n",
        "\n",
        "    return y_pred, cost_history"
      ],
      "execution_count": 10,
      "outputs": []
    },
    {
      "cell_type": "code",
      "metadata": {
        "colab": {
          "base_uri": "https://localhost:8080/"
        },
        "id": "Yk6pfB5c8tPz",
        "outputId": "9540574e-8b86-47e8-9a45-5aa36ddd1cad"
      },
      "source": [
        "lr = 0.5\n",
        "n_iter = 50\n",
        "y_pred = np.random.randn(2, 1)\n",
        "X_b = np.c_[np.ones((len(X), 1)), X]\n",
        "y_pred, cost_history = stocashtic_gradient_descent(X_b, y, y_pred, lr, n_iter)\n",
        "\n",
        "print('y_pred[0]: {:0.3f}\\ny_pred[1]: {:0.3f}'.format(y_pred[0][0], y_pred[1][0]))\n",
        "print('Final error: {:0.3f}'.format(cost_history[-1]))"
      ],
      "execution_count": 11,
      "outputs": [
        {
          "output_type": "stream",
          "name": "stdout",
          "text": [
            "y_pred[0]: 4.121\n",
            "y_pred[1]: 2.755\n",
            "Final error: 41.313\n"
          ]
        }
      ]
    },
    {
      "cell_type": "code",
      "metadata": {
        "colab": {
          "base_uri": "https://localhost:8080/",
          "height": 713
        },
        "id": "YiJUgS7o8u2e",
        "outputId": "c8383692-e98c-46cc-b878-4b47c82f99bf"
      },
      "source": [
        "fig, ax = plt.subplots(figsize=(10,8))\n",
        "\n",
        "ax.set_ylabel('Error')\n",
        "ax.set_xlabel('Number of iterations')\n",
        "y_pred = np.random.randn(2,1)\n",
        "\n",
        "ax.plot(range(n_iter), cost_history, 'b.')"
      ],
      "execution_count": 12,
      "outputs": [
        {
          "output_type": "execute_result",
          "data": {
            "text/plain": [
              "[<matplotlib.lines.Line2D at 0x799040b1e8f0>]"
            ]
          },
          "metadata": {},
          "execution_count": 12
        },
        {
          "output_type": "display_data",
          "data": {
            "text/plain": [
              "<Figure size 1000x800 with 1 Axes>"
            ],
            "image/png": "iVBORw0KGgoAAAANSUhEUgAAA1IAAAKnCAYAAACMDnwZAAAAOXRFWHRTb2Z0d2FyZQBNYXRwbG90bGliIHZlcnNpb24zLjcuMSwgaHR0cHM6Ly9tYXRwbG90bGliLm9yZy/bCgiHAAAACXBIWXMAAA9hAAAPYQGoP6dpAAA780lEQVR4nO3df3TV9Z3g/1dCICCQIBQT2QDaQlWsYosKWWtrkZJaasuKu7aHtbTljNYJVsS6DlMVxdlCbbe2tjraKSvu2fpj3Cnt1DlqESluFX9h6eAvFjy2QCFgy5AI3+FX8vn+kcPVCGjeCeTemzwe59wT7uf+en+SDzd53s+vkizLsgAAAKDdSvM9AAAAgGIjpAAAABIJKQAAgERCCgAAIJGQAgAASCSkAAAAEgkpAACAREIKAAAgUVm+B1AIWlpaYvPmzTFw4MAoKSnJ93AAAIA8ybIs3nrrrRg2bFiUlh5+vZOQiojNmzfH8OHD8z0MAACgQGzcuDFqamoOe7uQioiBAwdGROs3q6KiIs+jAQAA8qWpqSmGDx+ea4TDEVIRuc35KioqhBQAAPC+u/w42AQAAEAiIQUAAJBISAEAACQSUgAAAImEFAAAQCIhBQAAkEhIAQAAJBJSAAAAiYQUAABAIiEFAACQSEgBAAAkElIAAACJhBQAAEAiIQUAAJBISAEAACQSUgAAAImEFAAAQCIhBQAAkEhIAQAAJBJSAAAAiYQUAABAIiEFAACQSEgVmE2bIpYvb/0KAAAUJiFVQBYtihg5MmLixNavixble0QAAMChCKkCsWlTxGWXRbS0tF5vaYm4/HJrpgAAoBAJqQKxbt3bEXVAc3PE+vX5GQ8AAHB4QqpAjB4dUfqun0avXhGjRuVnPAAAwOEJqQJRUxPxk5+0xlNE69e7726dDgAAFJayfA+At82cGVFX17o536hRIgoAAAqVkCowNTUCCgAACp1N+wAAABIJKQAAgER5DambbropSkpK2lxOPvnk3O27d++O+vr6GDJkSAwYMCCmTZsWW7dubfMcGzZsiClTpsQxxxwTxx13XFx77bWxf//+rp4VAACgB8n7PlKnnnpqPP7447nrZWVvD+nqq6+Of/mXf4mHHnooKisrY9asWXHRRRfFU089FRERzc3NMWXKlKiuro6nn346tmzZEl/+8pejd+/e8e1vf7vL5wUAAOgZ8h5SZWVlUV1dfdD0xsbGWLRoUdx3330xceLEiIi455574pRTTolnnnkmJkyYEL/+9a/jlVdeiccffzyqqqrijDPOiFtuuSWuu+66uOmmm6JPnz5dPTsAAEAPkPd9pNatWxfDhg2LD37wgzF9+vTYsGFDRESsWrUq9u3bF5MmTcrd9+STT44RI0bEypUrIyJi5cqVcdppp0VVVVXuPnV1ddHU1BQvv/xy184IAADQY+R1jdT48eNj8eLFcdJJJ8WWLVvi5ptvjnPPPTdeeumlaGhoiD59+sSgQYPaPKaqqioaGhoiIqKhoaFNRB24/cBth7Nnz57Ys2dP7npTU9MRmiMAAKAnyGtIXXDBBbl/n3766TF+/PgYOXJk/OM//mP069fvqL3uggUL4uabbz5qzw8AAHRved+0750GDRoUH/7wh2P9+vVRXV0de/fujR07drS5z9atW3P7VFVXVx90FL8D1w+139UBc+fOjcbGxtxl48aNR3ZGAACAbq2gQmrnzp3x+uuvx/HHHx/jxo2L3r17x7Jly3K3r127NjZs2BC1tbUREVFbWxtr1qyJbdu25e6zdOnSqKioiDFjxhz2dcrLy6OioqLNBQAAoL3yumnfN7/5zbjwwgtj5MiRsXnz5pg3b1706tUrvvSlL0VlZWXMnDkz5syZE4MHD46Kioq48soro7a2NiZMmBAREZMnT44xY8bEpZdeGrfeems0NDTE9ddfH/X19VFeXp7PWQMAALqxvIbUpk2b4ktf+lL85S9/iaFDh8bHP/7xeOaZZ2Lo0KEREXHbbbdFaWlpTJs2Lfbs2RN1dXVx55135h7fq1evePjhh+OKK66I2tra6N+/f8yYMSPmz5+fr1kCAAB6gJIsy7J8DyLfmpqaorKyMhobG23mBwAAPVh726Cg9pECAAAoBkIKAAAgkZACAABIJKQAAAASCSkAAIBEQgoAACCRkAIAAEgkpAAAABIJKQAAgERCCgAAIJGQAgAASCSkAAAAEgkpAACAREIKAAAgkZACAABIJKQAAAASCSkAAIBEQgoAACCRkAIAAEgkpAAAABIJKQAAgERCCgAAIJGQAgAASCSkAAAAEgkpAACAREIKAAAgkZACAABIJKQAAAASCSkAAIBEQgoAACCRkAIAAEgkpAAAABIJKQAAgERCCgAAIJGQAgAASCSkAAAAEgkpAACAREIKAAAgkZACAABIJKQAAAASCSkAAIBEQgoAACCRkAIAAEgkpAAAABIJKQAAgERCCgAAIJGQAgAASCSkAAAAEgkpAACAREIKAAAgkZACAABIJKQAAAASCSkAAIBEQgoAACCRkAIAAEgkpAAAABIJKQAAgERCCgAAIJGQAgAASCSkAAAAEgkpAACAREIKAAAgkZACAABIJKQAAAASCSkAAIBEQgoAACCRkAIAAEgkpAAAABIJKQAAgERCCgAAIJGQAgAASCSkAAAAEgkpAACAREIKAAAgkZACAABIJKQAAAASCSkAAIBEQgoAACCRkAIAAEgkpAAAABIJKQAAgERCCgAAIJGQAgAASCSkAAAAEgkpAACAREIKAAAgkZACAABIJKQAAAASCSkAAIBEQgoAACCRkAIAAEgkpAAAABIJKQAAgERCCgAAIJGQAgAASCSkAAAAEgkpAACAREIKAAAgkZACAABIJKQAAAASCSkAAIBEQgoAACCRkAIAAEgkpAAAABIVTEgtXLgwSkpKYvbs2blpu3fvjvr6+hgyZEgMGDAgpk2bFlu3bm3zuA0bNsSUKVPimGOOieOOOy6uvfba2L9/fxePHgAA6EkKIqSef/75uPvuu+P0009vM/3qq6+OX/3qV/HQQw/FihUrYvPmzXHRRRflbm9ubo4pU6bE3r174+mnn4577703Fi9eHDfeeGNXzwIAANCD5D2kdu7cGdOnT49/+Id/iGOPPTY3vbGxMRYtWhTf//73Y+LEiTFu3Li455574umnn45nnnkmIiJ+/etfxyuvvBL/+3//7zjjjDPiggsuiFtuuSXuuOOO2Lt3b75mCQAA6ObyHlL19fUxZcqUmDRpUpvpq1atin379rWZfvLJJ8eIESNi5cqVERGxcuXKOO2006Kqqip3n7q6umhqaoqXX365a2YAAADoccry+eIPPPBAvPjii/H8888fdFtDQ0P06dMnBg0a1GZ6VVVVNDQ05O7zzog6cPuB2w5nz549sWfPntz1pqamjs4CAADQA+VtjdTGjRvjqquuip/97GfRt2/fLn3tBQsWRGVlZe4yfPjwLn19AACguOUtpFatWhXbtm2Lj33sY1FWVhZlZWWxYsWKuP3226OsrCyqqqpi7969sWPHjjaP27p1a1RXV0dERHV19UFH8Ttw/cB9DmXu3LnR2NiYu2zcuPHIzhwAANCt5S2kzj///FizZk2sXr06dznzzDNj+vTpuX/37t07li1blnvM2rVrY8OGDVFbWxsREbW1tbFmzZrYtm1b7j5Lly6NioqKGDNmzGFfu7y8PCoqKtpcAAAA2itv+0gNHDgwPvKRj7SZ1r9//xgyZEhu+syZM2POnDkxePDgqKioiCuvvDJqa2tjwoQJERExefLkGDNmTFx66aVx6623RkNDQ1x//fVRX18f5eXlXT5PAABAz5DXg028n9tuuy1KS0tj2rRpsWfPnqirq4s777wzd3uvXr3i4YcfjiuuuCJqa2ujf//+MWPGjJg/f34eRw0AAHR3JVmWZfkeRL41NTVFZWVlNDY22swPAAB6sPa2Qd7PIwUAAFBshBQAAEAiIQUAAJBISAEAACQSUgAAAImEFAAAQCIhBQAAkEhIAQAAJBJSAAAAiYQUAABAIiEFAACQSEgBAAAkElIAAACJhBQAAEAiIQUAAJBISAEAACQSUgAAAImEFAAAQCIhBQAAkEhIAQAAJBJSAAAAiYQUAABAIiEFAACQSEgBAAAkElIAAACJhBQAAEAiIQUAAJBISAEAACQSUgAAAImEFAAAQCIhBQAAkEhIAQAAJBJSAAAAiYQUAABAIiEFAACQSEgBAAAkElIAAACJhBQAAEAiIQUAAJBISAEAACQSUgAAAImEFAAAQCIhBQAAkEhIAQAAJBJSAAAAiYQUAABAIiEFAACQSEgBAAAkElIAAACJhBQAAEAiIQUAAJBISAEAACQSUgAAAImEFAAAQCIhBQAAkEhIAQAAJBJSAAAAiYQUAABAIiEFAACQSEgBAAAkElIAAACJhBQAAEAiIQUAAJBISAEAACQSUgAAAImEFAAAQCIhBQAAkEhIAQAAJBJSAAAAiYQUAABAIiEFAACQSEgBAAAkElIAAACJhBQAAEAiIQUAAJBISAEAACQSUgAAAImEFAAAQCIhBQAAkEhIAQAAJBJSAAAAiYQUAABAIiEFAACQSEgBAAAkElIAAACJhBQAAEAiIQUAAJBISAEAACQSUgAAAImEFAAAQCIhBQAAkEhIAQAAJBJSAAAAiYQUAABAIiEFAACQSEgBAAAkElIAAACJhBQAAEAiIQUAAJBISAEAACQSUgAAAImEFAAAQCIhBQAAkEhIAQAAJBJSAAAAifIaUn//938fp59+elRUVERFRUXU1tbGI488krt99+7dUV9fH0OGDIkBAwbEtGnTYuvWrW2eY8OGDTFlypQ45phj4rjjjotrr7029u/f39WzAgAA9CB5DamamppYuHBhrFq1Kl544YWYOHFifOELX4iXX345IiKuvvrq+NWvfhUPPfRQrFixIjZv3hwXXXRR7vHNzc0xZcqU2Lt3bzz99NNx7733xuLFi+PGG2/M1ywBAAA9QEmWZVm+B/FOgwcPju9+97tx8cUXx9ChQ+O+++6Liy++OCIiXnvttTjllFNi5cqVMWHChHjkkUfic5/7XGzevDmqqqoiIuKuu+6K6667Lt58883o06dPu16zqakpKisro7GxMSoqKo7avAEAAIWtvW1QMPtINTc3xwMPPBC7du2K2traWLVqVezbty8mTZqUu8/JJ58cI0aMiJUrV0ZExMqVK+O0007LRVRERF1dXTQ1NeXWah3Knj17oqmpqc0FAACgvfIeUmvWrIkBAwZEeXl5fP3rX48lS5bEmDFjoqGhIfr06RODBg1qc/+qqqpoaGiIiIiGhoY2EXXg9gO3Hc6CBQuisrIydxk+fPiRnSkAAKBby3tInXTSSbF69ep49tln44orrogZM2bEK6+8clRfc+7cudHY2Ji7bNy48ai+HgAA0L2U5XsAffr0iVGjRkVExLhx4+L555+PH/7wh3HJJZfE3r17Y8eOHW3WSm3dujWqq6sjIqK6ujqee+65Ns934Kh+B+5zKOXl5VFeXn6E5wQAAOgp8r5G6t1aWlpiz549MW7cuOjdu3csW7Ysd9vatWtjw4YNUVtbGxERtbW1sWbNmti2bVvuPkuXLo2KiooYM2ZMl48dAADoGfK6Rmru3LlxwQUXxIgRI+Ktt96K++67L37zm9/EY489FpWVlTFz5syYM2dODB48OCoqKuLKK6+M2tramDBhQkRETJ48OcaMGROXXnpp3HrrrdHQ0BDXX3991NfXW+MEAAAcNXkNqW3btsWXv/zl2LJlS1RWVsbpp58ejz32WHz605+OiIjbbrstSktLY9q0abFnz56oq6uLO++8M/f4Xr16xcMPPxxXXHFF1NbWRv/+/WPGjBkxf/78fM0SAADQAxTceaTywXmkAACAiCI8jxQAAECxEFIAAACJhBQAAEAiIQUAAJBISAEAACQSUgAAAImEFAAAQCIhBQAAkEhIAQAAJBJSAAAAiYQUAABAIiEFAACQSEgBAAAkElIAAACJhBQAAEAiIQUAAJBISAEAACRKDql9+/ZFWVlZvPTSS0djPAAAAAUvOaR69+4dI0aMiObm5qMxHgAAgILXoU37vvWtb8Xf/u3fxvbt24/0eAAAAApeWUce9OMf/zjWr18fw4YNi5EjR0b//v3b3P7iiy8ekcEBAAAUog6F1NSpU4/wMAAAAIpHSZZlWb4HkW9NTU1RWVkZjY2NUVFRke/hAAAAedLeNujQGqkDVq1aFa+++mpERJx66qnx0Y9+tDNPBwAAUBQ6FFLbtm2LL37xi/Gb3/wmBg0aFBERO3bsiE996lPxwAMPxNChQ4/kGAEAAApKh47ad+WVV8Zbb70VL7/8cmzfvj22b98eL730UjQ1NcU3vvGNIz1GAACAgtKhfaQqKyvj8ccfj7POOqvN9Oeeey4mT54cO3bsOFLj6xL2kQIAACLa3wYdWiPV0tISvXv3Pmh67969o6WlpSNPCQAAUDQ6FFITJ06Mq666KjZv3pyb9qc//SmuvvrqOP/884/Y4AAAAApRh0Lqxz/+cTQ1NcUJJ5wQH/rQh+JDH/pQnHjiidHU1BQ/+tGPjvQYAQAACkqHjto3fPjwePHFF+Pxxx+P1157LSIiTjnllJg0adIRHRwAAEAhSg6pffv2Rb9+/WL16tXx6U9/Oj796U8fjXEBAAAUrORN+3r37h0jRoyI5ubmozEeAACAgtehfaS+9a1vxd/+7d/G9u3bj/R4AAAACl6H9pH68Y9/HOvXr49hw4bFyJEjo3///m1uf/HFF4/I4AAAAApRh0Jq6tSpR3gYAAAAxSM5pPbv3x8lJSXxta99LWpqao7GmAAAAApa8j5SZWVl8d3vfjf2799/NMYDAABQ8Dp0sImJEyfGihUrjvRYAAAAikKH9pG64IIL4m/+5m9izZo1MW7cuIMONvH5z3/+iAwOAACgEJVkWZalPqi09PArskpKSoruHFNNTU1RWVkZjY2NUVFRke/hAAAAedLeNujQGqmWlpYODwwAAKDYJe0j9dnPfjYaGxtz1xcuXBg7duzIXf/LX/4SY8aMOWKDAwAAKERJIfXYY4/Fnj17cte//e1vx/bt23PX9+/fH2vXrj1yowMAAChASSH17t2pOrB7FQAAQNHr0OHPAQAAerKkkCopKYmSkpKDpgEAAPQkSUfty7IsvvKVr0R5eXlEROzevTu+/vWv584j9c79pwAAALqrpJCaMWNGm+v/9b/+14Pu8+Uvf7lzIwIAAChwSSF1zz33HK1xAAAAFA0HmwAAAEgkpAAAABIJKQAAgERCCgAAIJGQAgAASCSkAAAAEgkpAACAREIKAAAgkZACAABIJKQAAAASCSkAAIBEQgoAACCRkAIAAEgkpAAAABIJKQAAgERCCgAAIJGQAgAASCSkAAAAEgkpAACAREIKAAAgkZACAABIJKQAAAASCSkAAIBEQgoAACCRkAIAAEgkpAAAABIJKQAAgERCCgAAIJGQAgAASCSkAAAAEgkpAACAREIKAAAgkZACAABIJKQAAAASCSkAAIBEQgoAACCRkAIAAEgkpAAAABIJKQAAgERCCgAAIJGQAgAASCSkAAAAEgkpAACAREIKAAAgkZACAABIJKQAAAASCSkAAIBEQgoAACCRkAIAAEgkpAAAABIJKQAAgERCCgAAIJGQAgAASCSkAAAAEuU1pBYsWBBnnXVWDBw4MI477riYOnVqrF27ts19du/eHfX19TFkyJAYMGBATJs2LbZu3drmPhs2bIgpU6bEMcccE8cdd1xce+21sX///q6cFQAAoAfJa0itWLEi6uvr45lnnomlS5fGvn37YvLkybFr167cfa6++ur41a9+FQ899FCsWLEiNm/eHBdddFHu9ubm5pgyZUrs3bs3nn766bj33ntj8eLFceONN+ZjlgAAgB6gJMuyLN+DOODNN9+M4447LlasWBGf+MQnorGxMYYOHRr33XdfXHzxxRER8dprr8Upp5wSK1eujAkTJsQjjzwSn/vc52Lz5s1RVVUVERF33XVXXHfddfHmm29Gnz593vd1m5qaorKyMhobG6OiouKoziMAAFC42tsGBbWPVGNjY0REDB48OCIiVq1aFfv27YtJkybl7nPyySfHiBEjYuXKlRERsXLlyjjttNNyERURUVdXF01NTfHyyy8f8nX27NkTTU1NbS4AAADtVTAh1dLSErNnz45zzjknPvKRj0RERENDQ/Tp0ycGDRrU5r5VVVXR0NCQu887I+rA7QduO5QFCxZEZWVl7jJ8+PAjPDcAAEB3VjAhVV9fHy+99FI88MADR/215s6dG42NjbnLxo0bj/prAgAA3UdZvgcQETFr1qx4+OGH48knn4yamprc9Orq6ti7d2/s2LGjzVqprVu3RnV1de4+zz33XJvnO3BUvwP3ebfy8vIoLy8/wnMBAAD0FHldI5VlWcyaNSuWLFkSTzzxRJx44oltbh83blz07t07li1blpu2du3a2LBhQ9TW1kZERG1tbaxZsya2bduWu8/SpUujoqIixowZ0zUzAgAA9Ch5XSNVX18f9913X/zyl7+MgQMH5vZpqqysjH79+kVlZWXMnDkz5syZE4MHD46Kioq48soro7a2NiZMmBAREZMnT44xY8bEpZdeGrfeems0NDTE9ddfH/X19dY6AQAAR0VeD39eUlJyyOn33HNPfOUrX4mI1hPyXnPNNXH//ffHnj17oq6uLu688842m+398Y9/jCuuuCJ+85vfRP/+/WPGjBmxcOHCKCtrXyc6/DkAABDR/jYoqPNI5YuQAgAAIor0PFIAAADFQEgBAAAkElIAAACJhBQAAEAiIQUAAJBISAEAACQSUgAAAImEFAAAQCIhBQAAkEhIAQAAJBJSAAAAiYQUAABAIiEFAACQSEgBAAAkElIAAACJhBQAAEAiIQUAAJBISAEAACQSUgAAAImEFAAAQCIhBQAAkEhIAQAAJBJSAAAAiYQUAABAIiEFAACQSEgBAAAkElIAAACJhBQAAEAiIQUAAJBISHUTmzZFLF/e+hUAADi6hFQ3sGhRxMiRERMntn5dtCjfIwIAgO5NSBW5TZsiLrssoqWl9XpLS8Tll1szBQAAR5OQKnLr1r0dUQc0N0esX5+f8QAAQE8gpIrc6NERpe/6KfbqFTFqVH7GAwAAPYGQKnI1NRE/+UlrPEW0fr377tbpAADA0VGW7wHQeTNnRtTVtW7ON2qUiAIAgKNNSHUTNTUCCgAAuopN+wAAABIJKQAAgERCCgAAIJGQAgAASCSkAAAAEgkpAACAREIKAAAgkZACAABIJKQAAAASCSkAAIBEQgoAACCRkAIAAEgkpAAAABIJKQAAgERCCgAAIJGQAgAASCSkAAAAEgkpAACAREIKAAAgkZACAABIJKQAAAASCSkAAIBEQgoAACCRkAIAAEgkpAAAABIJKQAAgERCCgAAIJGQAgAASCSkAAAAEgkpAACAREIKAAAgkZACAABIJKQAAAASCSkAAIBEQgoAACCRkAIAAEgkpAAAABIJKQAAgERCCgAAIJGQAgAASCSkAAAAEgkpAACAREIKAAAgkZACAABIJKQAAAASCSkAAIBEQgoAACCRkAIAAEgkpAAAABIJKQAAgERCCgAAIJGQAgAASCSkAAAAEgkpAACAREIKAAAgkZACAABIJKQAAAASCSkAAIBEQgoAACCRkAIAAEgkpAAAABIJKQAAgERCCgAAIJGQAgAASCSkAAAAEgkpAACAREIKAAAgUV5D6sknn4wLL7wwhg0bFiUlJfGLX/yize1ZlsWNN94Yxx9/fPTr1y8mTZoU69ata3Of7du3x/Tp06OioiIGDRoUM2fOjJ07d3bhXAAAAD1NXkNq165dMXbs2LjjjjsOefutt94at99+e9x1113x7LPPRv/+/aOuri52796du8/06dPj5ZdfjqVLl8bDDz8cTz75ZFx22WVdNQsAAEAPVJJlWZbvQURElJSUxJIlS2Lq1KkR0bo2atiwYXHNNdfEN7/5zYiIaGxsjKqqqli8eHF88YtfjFdffTXGjBkTzz//fJx55pkREfHoo4/GZz/72di0aVMMGzasXa/d1NQUlZWV0djYGBUVFUdl/gAAgMLX3jYo2H2k3njjjWhoaIhJkyblplVWVsb48eNj5cqVERGxcuXKGDRoUC6iIiImTZoUpaWl8eyzzx72uffs2RNNTU1tLgAAAO1VsCHV0NAQERFVVVVtpldVVeVua2hoiOOOO67N7WVlZTF48ODcfQ5lwYIFUVlZmbsMHz78CI8eAADozgo2pI6muXPnRmNjY+6ycePGfA8JAAAoIgUbUtXV1RERsXXr1jbTt27dmruturo6tm3b1ub2/fv3x/bt23P3OZTy8vKoqKhocwEAAGivgg2pE088Maqrq2PZsmW5aU1NTfHss89GbW1tRETU1tbGjh07YtWqVbn7PPHEE9HS0hLjx4/v8jEDAAA9Q1k+X3znzp2xfv363PU33ngjVq9eHYMHD44RI0bE7Nmz4+/+7u9i9OjRceKJJ8YNN9wQw4YNyx3Z75RTTonPfOYz8Vd/9Vdx1113xb59+2LWrFnxxS9+sd1H7AMAAEiV15B64YUX4lOf+lTu+pw5cyIiYsaMGbF48eL4b//tv8WuXbvisssuix07dsTHP/7xePTRR6Nv3765x/zsZz+LWbNmxfnnnx+lpaUxbdq0uP3227t8XgAAgJ6jYM4jlU/OIwUAAER0g/NIAQAAFCohBQAAkEhIAQAAJBJSAAAAiYQURWXTpojly1u/AgBAvggpisaiRREjR0ZMnNj6ddGifI8IAICeSkhRFDZtirjssoiWltbrLS0Rl19uzRQAAPkhpCgK69a9HVEHNDdHrF+fn/EAANCzCSk6rCv3Vxo9OqL0XUtrr14Ro0Yd/dcGAIB3E1J0SFfvr1RTE/GTn7TGU0Tr17vvbp0OAABdrSTLsizfg8i3pqamqKysjMbGxqioqMj3cArepk2t8fTOTe169Yr4wx+Ofths2tS6Od+oUSIKAIAjr71tUNaFY6KbeK/9lY523NTUCCgAAPLPpn0k6+z+SsV2LqhiGy8AAEefkCI5FDqzv1KxnQuq2MYLAEDXsI9U9Ox9pBYtevv8TKWlrYE0c2b7Hpu6v1I+963qiGIbLwAAndfeNrBGqgfr7Elua2oizjuv/VFRbOeCKrbxAgDQdYRUD9bVoVBs54IqtvECANB1hFQP1tWhUGzngiq28QIA0HXsIxX2kbr88tY1UQdCob37SHVUsZ0LqtjGCwBAx7W3DYRU9OyQihAKAABwgBPy0m5OcgsAAGnsIwUAAJBISAEAACQSUgAAAImEFAAAQCIhBQAAkEhIAQAAJBJSAAAAiYQUHCWbNkUsX976FQCA7kVIwVGwaFHEyJEREye2fl20KN8jAgDgSBJScIRt2hRx2WURLS2t11taIi6/3JopAIDuREjBEbZu3dsRdUBzc8T69fkZDwAAR56QgiNs9OiI0nf9z+rVK2LUqPyMBwCAI09IwRFWUxPxk5+0xlNE69e7726dDgBA91CW7wFAdzRzZkRdXevmfKNGiSgAgO5GSMFRUlMjoAAAuiub9kGBcf4pAIDCJ6SggHT2/FMiDACgawgpKBCdPf+UkwADAHQdIQUFojPnn3ISYACAriWk6BGKYZO3zpx/ykmAAQC6lpCi2yuWTd46c/4pJwEGAOhaJVmWZfkeRL41NTVFZWVlNDY2RkVFRb6HwxG0aVNrPL1zbU2vXhF/+EPhHpp806aOnX9q0aLWzfmam9+OsJkzj944AQC6o/a2gfNI0a291yZvhRpSHT3/lJMAAxSmTZtafx+NHu29GboTm/bRrfW0Td5qaiLOO88vaoBCUSyblwPphBTdWmf2O4IDiuFgJUDhcURV6N6EFN3ezJmt+0QtX9761X5DpPBpMtBRjqgK3ZuDTYSDTQCHVowHKwEKh/cQKE7tbQNrpAAOw6fJHAk2DX1v3fn7Y/Ny6N6EFMBh9LSDlXDk2TT0vfWE74/Ny6H7smlf2LQPODzn56KjOrtZV3c/ZLbN3qD76S7vWzbtAzgCfJpMR3Vm09CesKbGprPQvfSE9613s0YqrJGCnqK7fFJGx3+WXbkMdHSNS09ZU9NT5hN6gu72/9kaKaBb6uiO6T3xk7LuqqM/y65eBjp6oIGesqbGgRjIp+58kJN86CnvW+9mjVRYIwXF8Ol+ROsfvgdOblla2vpHWHs2tSvGT8qKae2ZtTzvbdOm1j8mRo1q/75Rxba8dkbq9wc6q6O/Szi87va+ZY0U0C75+nQ/9dPATZve/sUX0fr18svb9/hi+6SsmNaedfVYO/qzzOcyUFMTcd557f9joqetqUn9/kBndOZ3CYfX0963DrBGKqyRoufK16f7Hfk0cPny1j/WDzX9vPPe+7HF9EmZsR6d1yym7+sB+VhTU0xrQqEjOvO7hPfXXdYwWyMFvK98fLrf0U8DO3NOp3x9UtaRbfCLae1ZZ8fake9PR3+WxfhpaVevqSmmNaHQUT3p/ID52A+so+9bxbrPmpCCHqyjv1A684uoo398d/YP4a4+jHlH/ygtpl/ynRlrZ/5o7+jP0qHsD8/mThwJxfDHcDF+qNKR72sxfTBSTGM9SEbW2NiYRUTW2NiY76FAl/vpT7OsV68si2j9+tOfHt3HbdyYZaWlrY87cOnVq3V6ex+/fHn7758PnZ3Hjn5v86EjY+3s94cj74kn2v48DlyWL2/f4zdubH2O7v4z7Cnz2RE//enb/69LSwv7fSvLiuN3SZZ17PtaTO+xhTrW9raBfaTCPlLQ0W2aO/q4RYtaP+1ubn7708DutHbgSGyDX0zbmaeO1T4Khacz+5D1lCOg5Ws+i2G/tWLcB7EYdPT7WkzvsYU61va2gZAKIQX5UEyhkMofFe/N96cwdeQDjp7ys8zXfBZLpHb2j+FiiMV86Oj3tZj+XxbqWB1sAiho3fmQx/ncBt8+CnRUR/YhK6aDo3RGPuazmPZby9f+kt1dR7+vxfQeW0xjPRRrpMIaKeDo6Oq1bp359Dofnwh357WSPUWhfpp8pOVjPvO1yVNH3wus0Tw6OrMpfDG9xxbaWG3al0BIAcXOPi7kS3ff5/GArp7PzkZGR4Kos+8F9pc8OgotMnoCIZVASAHFridsS0/79JS1iz1hPjsabx0JomI6wXaxsi9Y8bCPFEAP0tFt6XvKPi49Rb72N+kpJw/u6vnsyH5rHd23Kh/vBcW+f0wK+4J1T9ZIhTVSQPdgH4Weraf8LPOxyVsxKca1091907We8n+zO7FGCqCH6cin1z3pE+HurqesXezMfPaEtQLFeKS37nwU14ji/L9ZDEeALQTWSIU1UgDd/RPhnqCnfOrd0fnsKd+fiJ5zpLdiUWzLngMQWSMFQILu/olwT9BT1i52dD6Lca1AR3Vk7fQBxfReUCxrTYrp/2Yxnb+sEFgjFdZIAdB99JQ1CqnzWWxrBXhvxbjWpBj+bzokfSuHP08gpACg++sp57zq7vJ9YIzufLASHzi0smkfAMA7dGaTNwpHvjbT7AkHKymmzRALgTVSYY0UAECxcPLgo68YNkM8mqyRAgCg28nHWpOedLCSiOI66Eg+leV7AAAAkGLmzIi6uq5ba3Lg/FzvXiP1fufnonuzRgoAgKLTlWtN7DvEoVgjBQDvo7sfqQt4f129FozCZ40UALyHnnCkLqB97DvEOwkpADiMTZvePulnROvXyy9vnQ5AzyakAOAwetqRugBoPyEFAIdx4Ehd7+RIXQBECCkAOCxH6gLgcBy1DwDegyN1AXAoQgoA3kdNjYACoC2b9gEAACQSUgAAAImEFAAAQCIhBQAAkEhIAQAAJBJSAAAAiYQUAABAIiEFAACQSEgBAAAkElIAAACJhBQAAEAiIQUAAJBISAEAACQSUgAAAImEFAAAQCIhBQAAkEhIAQAAJBJSAAAAibpNSN1xxx1xwgknRN++fWP8+PHx3HPP5XtIAABAN9UtQurBBx+MOXPmxLx58+LFF1+MsWPHRl1dXWzbti3fQwMAALqhbhFS3//+9+Ov/uqv4qtf/WqMGTMm7rrrrjjmmGPif/7P/5nvoQEAAN1Q0YfU3r17Y9WqVTFp0qTctNLS0pg0aVKsXLnykI/Zs2dPNDU1tbkAAAC0V1m+B9BZf/7zn6O5uTmqqqraTK+qqorXXnvtkI9ZsGBB3HzzzQdNF1QAANCzHWiCLMve835FH1IdMXfu3JgzZ07u+p/+9KcYM2ZMDB8+PI+jAgAACsVbb70VlZWVh7296EPqAx/4QPTq1Su2bt3aZvrWrVujurr6kI8pLy+P8vLy3PUBAwbExo0bY+DAgVFSUnJUx/t+mpqaYvjw4bFx48aoqKjI61goPpYfOsqyQ2dYfugMyw+dcTSWnyzL4q233ophw4a95/2KPqT69OkT48aNi2XLlsXUqVMjIqKlpSWWLVsWs2bNatdzlJaWRk1NzVEcZbqKigpvJnSY5YeOsuzQGZYfOsPyQ2cc6eXnvdZEHVD0IRURMWfOnJgxY0aceeaZcfbZZ8cPfvCD2LVrV3z1q1/N99AAAIBuqFuE1CWXXBJvvvlm3HjjjdHQ0BBnnHFGPProowcdgAIAAOBI6BYhFRExa9asdm/KV8jKy8tj3rx5bfbhgvay/NBRlh06w/JDZ1h+6Ix8Lj8l2fsd1w8AAIA2iv6EvAAAAF1NSAEAACQSUgAAAImEFAAAQCIhVUDuuOOOOOGEE6Jv374xfvz4eO655/I9JArQk08+GRdeeGEMGzYsSkpK4he/+EWb27MsixtvvDGOP/746NevX0yaNCnWrVuXn8FScBYsWBBnnXVWDBw4MI477riYOnVqrF27ts19du/eHfX19TFkyJAYMGBATJs2LbZu3ZqnEVNI/v7v/z5OP/303Ikva2tr45FHHsndbtmhvRYuXBglJSUxe/bs3DTLD4dz0003RUlJSZvLySefnLs9X8uOkCoQDz74YMyZMyfmzZsXL774YowdOzbq6upi27Zt+R4aBWbXrl0xduzYuOOOOw55+6233hq333573HXXXfHss89G//79o66uLnbv3t3FI6UQrVixIurr6+OZZ56JpUuXxr59+2Ly5Mmxa9eu3H2uvvrq+NWvfhUPPfRQrFixIjZv3hwXXXRRHkdNoaipqYmFCxfGqlWr4oUXXoiJEyfGF77whXj55ZcjwrJD+zz//PNx9913x+mnn95muuWH93LqqafGli1bcpff/va3udvytuxkFISzzz47q6+vz11vbm7Ohg0bli1YsCCPo6LQRUS2ZMmS3PWWlpasuro6++53v5ubtmPHjqy8vDy7//778zBCCt22bduyiMhWrFiRZVnr8tK7d+/soYceyt3n1VdfzSIiW7lyZb6GSQE79thjs5/+9KeWHdrlrbfeykaPHp0tXbo0++QnP5ldddVVWZZ57+G9zZs3Lxs7duwhb8vnsmONVAHYu3dvrFq1KiZNmpSbVlpaGpMmTYqVK1fmcWQUmzfeeCMaGhraLEuVlZUxfvx4yxKH1NjYGBERgwcPjoiIVatWxb59+9osQyeffHKMGDHCMkQbzc3N8cADD8SuXbuitrbWskO71NfXx5QpU9osJxHee3h/69ati2HDhsUHP/jBmD59emzYsCEi8rvslB3VZ6dd/vznP0dzc3NUVVW1mV5VVRWvvfZankZFMWpoaIiIOOSydOA2OKClpSVmz54d55xzTnzkIx+JiNZlqE+fPjFo0KA297UMccCaNWuitrY2du/eHQMGDIglS5bEmDFjYvXq1ZYd3tMDDzwQL774Yjz//PMH3ea9h/cyfvz4WLx4cZx00kmxZcuWuPnmm+Pcc8+Nl156Ka/LjpAC6KHq6+vjpZdearOdObyfk046KVavXh2NjY3xf/7P/4kZM2bEihUr8j0sCtzGjRvjqquuiqVLl0bfvn3zPRyKzAUXXJD79+mnnx7jx4+PkSNHxj/+4z9Gv3798jYum/YVgA984APRq1evg44usnXr1qiurs7TqChGB5YXyxLvZ9asWfHwww/H8uXLo6amJje9uro69u7dGzt27Ghzf8sQB/Tp0ydGjRoV48aNiwULFsTYsWPjhz/8oWWH97Rq1arYtm1bfOxjH4uysrIoKyuLFStWxO233x5lZWVRVVVl+aHdBg0aFB/+8Idj/fr1eX3vEVIFoE+fPjFu3LhYtmxZblpLS0ssW7Ysamtr8zgyis2JJ54Y1dXVbZalpqamePbZZy1LRETr4fFnzZoVS5YsiSeeeCJOPPHENrePGzcuevfu3WYZWrt2bWzYsMEyxCG1tLTEnj17LDu8p/PPPz/WrFkTq1evzl3OPPPMmD59eu7flh/aa+fOnfH666/H8ccfn9f3Hpv2FYg5c+bEjBkz4swzz4yzzz47fvCDH8SuXbviq1/9ar6HRoHZuXNnrF+/Pnf9jTfeiNWrV8fgwYNjxIgRMXv27Pi7v/u7GD16dJx44olxww03xLBhw2Lq1Kn5GzQFo76+Pu6777745S9/GQMHDsxtP15ZWRn9+vWLysrKmDlzZsyZMycGDx4cFRUVceWVV0ZtbW1MmDAhz6Mn3+bOnRsXXHBBjBgxIt56662477774je/+U089thjlh3e08CBA3P7Yh7Qv3//GDJkSG665YfD+eY3vxkXXnhhjBw5MjZv3hzz5s2LXr16xZe+9KX8vvcc1WMCkuRHP/pRNmLEiKxPnz7Z2WefnT3zzDP5HhIFaPny5VlEHHSZMWNGlmWth0C/4YYbsqqqqqy8vDw7//zzs7Vr1+Z30BSMQy07EZHdc889ufv8+7//e/bXf/3X2bHHHpsdc8wx2X/6T/8p27JlS/4GTcH42te+lo0cOTLr06dPNnTo0Oz888/Pfv3rX+dut+yQ4p2HP88yyw+Hd8kll2THH3981qdPn+w//If/kF1yySXZ+vXrc7fna9kpybIsO7qpBgAA0L3YRwoAACCRkAIAAEgkpAAAABIJKQAAgERCCgAAIJGQAgAASCSkAAAAEgkpALrUH/7whygpKYnVq1fneyg5r732WkyYMCH69u0bZ5xxxiHvc95558Xs2bO7dFztUVJSEr/4xS/yPQyAHkdIAfQwX/nKV6KkpCQWLlzYZvovfvGLKCkpydOo8mvevHnRv3//WLt2bSxbtuyQ9/n5z38et9xyS+76CSecED/4wQ+6aIQRN9100yEjb8uWLXHBBRd02TgAaCWkAHqgvn37xne+8534t3/7t3wP5YjZu3dvhx/7+uuvx8c//vEYOXJkDBky5JD3GTx4cAwcOLDDr3E4nRl3RER1dXWUl5cfodEA0F5CCqAHmjRpUlRXV8eCBQsOe59DrQH5wQ9+ECeccELu+le+8pWYOnVqfPvb346qqqoYNGhQzJ8/P/bv3x/XXnttDB48OGpqauKee+456Plfe+21+I//8T9G37594yMf+UisWLGize0vvfRSXHDBBTFgwICoqqqKSy+9NP785z/nbj/vvPNi1qxZMXv27PjABz4QdXV1h5yPlpaWmD9/ftTU1ER5eXmcccYZ8eijj+ZuLykpiVWrVsX8+fOjpKQkbrrppkM+zzs37TvvvPPij3/8Y1x99dVRUlLSZk3eb3/72zj33HOjX79+MXz48PjGN74Ru3btyt1+wgknxC233BJf/vKXo6KiIi677LKIiLjuuuviwx/+cBxzzDHxwQ9+MG644YbYt29fREQsXrw4br755vj973+fe73Fixfnxv/OTfvWrFkTEydOjH79+sWQIUPisssui507dx70M/ve974Xxx9/fAwZMiTq6+tzrxURceedd8bo0aOjb9++UVVVFRdffPEhvycAPZmQAuiBevXqFd/+9rfjRz/6UWzatKlTz/XEE0/E5s2b48knn4zvf//7MW/evPjc5z4Xxx57bDz77LPx9a9/PS6//PKDXufaa6+Na665Jn73u99FbW1tXHjhhfGXv/wlIiJ27NgREydOjI9+9KPxwgsvxKOPPhpbt26N//Jf/kub57j33nujT58+8dRTT8Vdd911yPH98Ic/jP/xP/5HfO9734t//dd/jbq6uvj85z8f69ati4jWTeNOPfXUuOaaa2LLli3xzW9+833n+ec//3nU1NTE/PnzY8uWLbFly5aIaF2z9ZnPfCamTZsW//qv/xoPPvhg/Pa3v41Zs2a1efz3vve9GDt2bPzud7+LG264ISIiBg4cGIsXL45XXnklfvjDH8Y//MM/xG233RYREZdccklcc801ceqpp+Ze75JLLjloXLt27Yq6uro49thj4/nnn4+HHnooHn/88YNef/ny5fH666/H8uXL4957743FixfnwuyFF16Ib3zjGzF//vxYu3ZtPProo/GJT3zifb8nAD1OBkCPMmPGjOwLX/hClmVZNmHChOxrX/talmVZtmTJkuydvxbmzZuXjR07ts1jb7vttmzkyJFtnmvkyJFZc3NzbtpJJ52UnXvuubnr+/fvz/r375/df//9WZZl2RtvvJFFRLZw4cLcffbt25fV1NRk3/nOd7Isy7Jbbrklmzx5cpvX3rhxYxYR2dq1a7Msy7JPfvKT2Uc/+tH3nd9hw4Zl//2///c2084666zsr//6r3PXx44dm82bN+89n+eTn/xkdtVVV+Wujxw5Mrvtttva3GfmzJnZZZdd1mba//2//zcrLS3N/v3f/z33uKlTp77vuL/73e9m48aNy10/1M8jy7IsIrIlS5ZkWZZlP/nJT7Jjjz0227lzZ+72f/mXf8lKS0uzhoaGLMve/pnt378/d5///J//c3bJJZdkWZZl//RP/5RVVFRkTU1N7ztGgJ7MGimAHuw73/lO3HvvvfHqq692+DlOPfXUKC19+9dJVVVVnHbaabnrvXr1iiFDhsS2bdvaPK62tjb377KysjjzzDNz4/j9738fy5cvjwEDBuQuJ598ckS0rvU5YNy4ce85tqampti8eXOcc845baafc845nZrnw/n9738fixcvbjPuurq6aGlpiTfeeCN3vzPPPPOgxz744INxzjnnRHV1dQwYMCCuv/762LBhQ9Lrv/rqqzF27Njo379/bto555wTLS0tsXbt2ty0U089NXr16pW7fvzxx+d+Pp/+9Kdj5MiR8cEPfjAuvfTS+NnPfhb/3//3/yWNA6AnEFIAPdgnPvGJqKuri7lz5x50W2lpaWRZ1mbaO/ejOaB3795trpeUlBxyWktLS7vHtXPnzrjwwgtj9erVbS7r1q1rs5nZO4OhEOzcuTMuv/zyNmP+/e9/H+vWrYsPfehDufu9e9wrV66M6dOnx2c/+9l4+OGH43e/+11861vf6vSBKA7nvX4+AwcOjBdffDHuv//+OP744+PGG2+MsWPHxo4dO47KWACKVVm+BwBAfi1cuDDOOOOMOOmkk9pMHzp0aDQ0NESWZbmDKRzJcz8988wzuSjav39/rFq1Krcvz8c+9rH4p3/6pzjhhBOirKzjv6oqKipi2LBh8dRTT8UnP/nJ3PSnnnoqzj777E6Nv0+fPtHc3Nxm2sc+9rF45ZVXYtSoUUnP9fTTT8fIkSPjW9/6Vm7aH//4x/d9vXc75ZRTYvHixbFr165crD311FNRWlp60M/3vZSVlcWkSZNi0qRJMW/evBg0aFA88cQTcdFFFyXMFUD3Zo0UQA932mmnxfTp0+P2229vM/28886LN998M2699dZ4/fXX44477ohHHnnkiL3uHXfcEUuWLInXXnst6uvr49/+7d/ia1/7WkRE1NfXx/bt2+NLX/pSPP/88/H666/HY489Fl/96lffNybe7dprr43vfOc78eCDD8batWvjb/7mb2L16tVx1VVXdWr8J5xwQjz55JPxpz/9KXc0weuuuy6efvrpmDVrVm4N2i9/+cuDDvbwbqNHj44NGzbEAw88EK+//nrcfvvtsWTJkoNe74033ojVq1fHn//859izZ89BzzN9+vTo27dvzJgxI1566aVYvnx5XHnllXHppZdGVVVVu+br4Ycfjttvvz1Wr14df/zjH+N//a//FS0tLUkhBtATCCkAYv78+QdtenfKKafEnXfeGXfccUeMHTs2nnvuuXYd0a69Fi5cGAsXLoyxY8fGb3/72/jnf/7n+MAHPhARkVuL1NzcHJMnT47TTjstZs+eHYMGDWqzP1Z7fOMb34g5c+bENddcE6eddlo8+uij8c///M8xevToTo1//vz58Yc//CE+9KEPxdChQyMi4vTTT48VK1bE//t//y/OPffc+OhHPxo33nhjDBs27D2f6/Of/3xcffXVMWvWrDjjjDPi6aefzh3N74Bp06bFZz7zmfjUpz4VQ4cOjfvvv/+g5znmmGPisccei+3bt8dZZ50VF198cZx//vnx4x//uN3zNWjQoPj5z38eEydOjFNOOSXuuuuuuP/+++PUU09t93MA9AQl2bs3gAcAAOA9WSMFAACQSEgBAAAkElIAAACJhBQAAEAiIQUAAJBISAEAACQSUgAAAImEFAAAQCIhBQAAkEhIAQAAJBJSAAAAiYQUAABAov8fKX036U69tdoAAAAASUVORK5CYII=\n"
          },
          "metadata": {}
        }
      ]
    },
    {
      "cell_type": "markdown",
      "metadata": {
        "id": "ScckWktynk1o"
      },
      "source": [
        "# Mini Batch Gradient Descent"
      ]
    },
    {
      "cell_type": "markdown",
      "source": [
        "We have seen the Batch Gradient Descent. We have also seen the Stochastic Gradient Descent. Batch Gradient Descent can be used for smoother curves. SGD can be used when the dataset is large. Batch Gradient Descent converges directly to minima. SGD converges faster for larger datasets. But, since in SGD we use only one example at a time, we cannot implement the vectorized implementation on it. This can slow down the computations. To tackle this problem, a mixture of Batch Gradient Descent and SGD is used.\n",
        "Neither we use all the dataset all at once nor we use the single example at a time. We use a batch of a fixed number of training examples which is less than the actual dataset and call it a mini-batch. Doing this helps us achieve the advantages of both the former variants we saw."
      ],
      "metadata": {
        "id": "ZTVz-QssUkuE"
      }
    },
    {
      "cell_type": "code",
      "metadata": {
        "id": "4JtxFVL78wEm"
      },
      "source": [
        "def minibatch_gradient_descent(X, y, y_pred, learning_rate=0.01, iterations=10, batch_size=20):\n",
        "    '''\n",
        "    X = Matrix of X without added bias units\n",
        "    y = Vector of Y\n",
        "    y_pred = Vector of y_preds np.random.randn(j, 1)\n",
        "    learning_rate\n",
        "    iterations = no of iterations\n",
        "\n",
        "    Returns the final theta vector and array of cost history over no of iterations\n",
        "    '''\n",
        "\n",
        "    m = len(y)\n",
        "    cost_history = np.zeros(iterations)\n",
        "    n_batches = int(m / batch_size)\n",
        "\n",
        "    for it in range(iterations):\n",
        "        cost = 0.0\n",
        "        indices = np.random.permutation(m)\n",
        "        X = X[indices]\n",
        "        y = y[indices]\n",
        "\n",
        "        for i in range(0, m, batch_size):\n",
        "            X_i = X[i: i + batch_size]\n",
        "            y_i = y[i: i + batch_size]\n",
        "\n",
        "            X_i = np.c_[np.ones(len(X_i)), X_i]\n",
        "            prediction = np.dot(X_i, y_pred)\n",
        "\n",
        "            y_pred = y_pred - (1 / m) * learning_rate * (X_i.T.dot((prediction - y_i)))\n",
        "            cost += cal_cost(y_pred, X_i, y_i)\n",
        "\n",
        "        cost_history[it]  = cost\n",
        "\n",
        "    return y_pred, cost_history"
      ],
      "execution_count": 13,
      "outputs": []
    },
    {
      "cell_type": "code",
      "metadata": {
        "colab": {
          "base_uri": "https://localhost:8080/"
        },
        "id": "SpbsVwA28znL",
        "outputId": "355a021a-6a04-4ccb-fa34-f8c00a5069aa"
      },
      "source": [
        "lr = 0.1\n",
        "n_iter = 200\n",
        "y_pred = np.random.randn(2,1)\n",
        "y_pred, cost_history = minibatch_gradient_descent(X, y, y_pred, lr, n_iter)\n",
        "\n",
        "print('y_pred[0]: {:0.3f}\\ny_pred[1]: {:0.3f}'.format(y_pred[0][0], y_pred[1][0]))\n",
        "print('Final error: {:0.3f}'.format(cost_history[-1]))"
      ],
      "execution_count": 14,
      "outputs": [
        {
          "output_type": "stream",
          "name": "stdout",
          "text": [
            "y_pred[0]: 4.148\n",
            "y_pred[1]: 2.827\n",
            "Final error: 807.488\n"
          ]
        }
      ]
    },
    {
      "cell_type": "code",
      "metadata": {
        "colab": {
          "base_uri": "https://localhost:8080/",
          "height": 713
        },
        "id": "Q_ivOYHT817C",
        "outputId": "214f6e20-8c4c-40db-c400-f665f5394fe8"
      },
      "source": [
        "fig, ax = plt.subplots(figsize=(10,8))\n",
        "\n",
        "ax.set_ylabel('Error')\n",
        "ax.set_xlabel('Number of iterations')\n",
        "y_pred = np.random.randn(2,1)\n",
        "\n",
        "ax.plot(range(n_iter), cost_history, 'b.')"
      ],
      "execution_count": 15,
      "outputs": [
        {
          "output_type": "execute_result",
          "data": {
            "text/plain": [
              "[<matplotlib.lines.Line2D at 0x799040b8fc10>]"
            ]
          },
          "metadata": {},
          "execution_count": 15
        },
        {
          "output_type": "display_data",
          "data": {
            "text/plain": [
              "<Figure size 1000x800 with 1 Axes>"
            ],
            "image/png": "iVBORw0KGgoAAAANSUhEUgAAA2QAAAKnCAYAAADp445mAAAAOXRFWHRTb2Z0d2FyZQBNYXRwbG90bGliIHZlcnNpb24zLjcuMSwgaHR0cHM6Ly9tYXRwbG90bGliLm9yZy/bCgiHAAAACXBIWXMAAA9hAAAPYQGoP6dpAABH1klEQVR4nO3deXhV5b3o8V8CMqgkqIy5wThAHRFn5FqtIiUqteXU9qilFpWKesAJ60HrgLX3FopttVbrcEvFc9ta63PUttrig4xHwQlNFRUu8KBIIWilJIIDkKz7R062bOZA4IXk83me/ezstd7s/e7Fasy3a+2VgizLsgAAAGCnK0w9AQAAgOZKkAEAACQiyAAAABIRZAAAAIkIMgAAgEQEGQAAQCKCDAAAIBFBBgAAkEjL1BNoKmpra2PJkiXRrl27KCgoSD0dAAAgkSzL4qOPPoqSkpIoLNz8MTBB1kiWLFkS3bp1Sz0NAABgF/Hee+9FaWnpZscIskbSrl27iKjb6EVFRYlnAwAApFJdXR3dunXLNcLmCLJGUn+aYlFRkSADAAC26qNMLuoBAACQiCADAABIRJABAAAkIsgAAAASEWQAAACJCDIAAIBEBBkAAEAiggwAACARQQYAAJCIIAMAAEhEkAEAACQiyAAAABIRZAAAAIkIMgAAgEQEGQAAQCKCDAAAIBFBBgAAkIggAwAASESQAQAAJCLIAAAAEhFkAAAAiQgyAACARARZE7R4ccSUKXX3AADArkuQNTHjxkWUlUX07Vt3P25c6hkBAACbIsiakMWLI4YOjaitrXtcWxtx2WWOlAEAwK5KkDUh8+Z9HmP1amoi5s9PMx8AAGDzBFkT0qNHROF6/6ItWkR0755mPgAAwOYJsiaktDTiwQfrIiyi7v6BB+qWAwAAu56WqSdA4xoyJKK8vO40xe7dxRgAAOzKBFkTVFoqxAAAYHfglEUAAIBEBBkAAEAiggwAACARQQYAAJCIIAMAAEhEkAEAACQiyAAAABIRZAAAAIkIMgAAgEQEGQAAQCKCDAAAIBFBBgAAkIggAwAASESQAQAAJCLIAAAAEhFkAAAAiQgyAACARAQZAABAIoIMAAAgEUEGAACQiCADAABIRJABAAAkIsgAAAASEWQAAACJCDIAAIBEBBkAAEAiggwAACARQQYAAJCIIAMAAEhEkAEAACSSNMhGjx4dJ5xwQrRr1y46deoUAwcOjLlz5+aNOe2006KgoCDvdvnll+eNWbRoUQwYMCD23HPP6NSpU1x//fWxdu3avDFTp06NY489Nlq3bh3du3eP8ePHbzCfe++9Nw444IBo06ZN9O7dO1566aVGf88AAAD1kgbZtGnTYtiwYfHCCy/ExIkTY82aNdG/f/9YtWpV3rhLL700li5dmruNHTs2t66mpiYGDBgQq1evjhkzZsTDDz8c48ePj1tvvTU3ZuHChTFgwIA4/fTTo6KiIq655pr47ne/G88880xuzKOPPhojRoyIUaNGxauvvhq9evWK8vLyeP/993f8hgAAAJqlgizLstSTqPfBBx9Ep06dYtq0aXHqqadGRN0RsqOPPjruuuuujX7PX//61/jKV74SS5Ysic6dO0dExP333x8jR46MDz74IFq1ahUjR46Mp59+OmbPnp37vvPPPz9WrFgREyZMiIiI3r17xwknnBD33HNPRETU1tZGt27d4sorr4wbbrhhi3Ovrq6O4uLiqKqqiqKiou3ZDAAAwG6sIW2wS32GrKqqKiIi9t1337zlv/3tb6NDhw5x5JFHxo033hgff/xxbt3MmTOjZ8+euRiLiCgvL4/q6up48803c2P69euX95zl5eUxc+bMiIhYvXp1zJo1K29MYWFh9OvXLzdmfZ999llUV1fn3QAAABqiZeoJ1KutrY1rrrkmTj755DjyyCNzy7/1rW9FWVlZlJSUxOuvvx4jR46MuXPnxuOPPx4REZWVlXkxFhG5x5WVlZsdU11dHZ988kn885//jJqamo2OmTNnzkbnO3r06PjBD36wfW8aAABo1naZIBs2bFjMnj07nnvuubzlQ4cOzX3ds2fP6Nq1a5xxxhmxYMGCOPjgg3f2NHNuvPHGGDFiRO5xdXV1dOvWLdl8AACA3c8uEWTDhw+Pp556KqZPnx6lpaWbHdu7d++IiJg/f34cfPDB0aVLlw2uhrhs2bKIiOjSpUvuvn7ZumOKioqibdu20aJFi2jRosVGx9Q/x/pat24drVu33vo3CQAAsJ6knyHLsiyGDx8eTzzxREyePDkOPPDALX5PRUVFRER07do1IiL69OkTb7zxRt7VECdOnBhFRUVx+OGH58ZMmjQp73kmTpwYffr0iYiIVq1axXHHHZc3pra2NiZNmpQbAwAA0NiSHiEbNmxY/O53v4s//vGP0a5du9xnvoqLi6Nt27axYMGC+N3vfhdnn3127LfffvH666/HtddeG6eeemocddRRERHRv3//OPzww+PCCy+MsWPHRmVlZdx8880xbNiw3BGsyy+/PO65557493//97jkkkti8uTJ8Yc//CGefvrp3FxGjBgRgwcPjuOPPz5OPPHEuOuuu2LVqlVx8cUX7/wNAwAANAtJL3tfUFCw0eUPPfRQXHTRRfHee+/Ft7/97Zg9e3asWrUqunXrFv/yL/8SN998c97lI99999244oorYurUqbHXXnvF4MGDY8yYMdGy5ee9OXXq1Lj22mvjrbfeitLS0rjlllvioosuynvde+65J+64446orKyMo48+Ou6+++7cKZJb4rL3AABARMPaYJf6O2S7M0EGAABE7MZ/hwwAAKA5EWQAAACJCDIAAIBEBBkAAEAiggwAACARQQYAAJCIIAMAAEhEkAEAACQiyAAAABIRZAAAAIkIMgAAgEQEGQAAQCKCDAAAIBFBBgAAkIggAwAASESQAQAAJCLIAAAAEhFkAAAAiQgyAACARAQZAABAIoIMAAAgEUEGAACQiCADAABIRJABAAAkIsgAAAASEWQAAACJCDIAAIBEBBkAAEAiggwAACARQQYAAJCIIAMAAEhEkAEAACQiyAAAABIRZAAAAIkIMgAAgEQEGQAAQCKCDAAAIBFBBgAAkIggAwAASESQAQAAJCLIAAAAEhFkAAAAiQgyAACARAQZAABAIoIMAAAgEUEGAACQiCADAABIRJABAAAkIsgAAAASEWQAAACJCDIAAIBEBBkAAEAiggwAACARQQYAAJCIIAMAAEhEkAEAACQiyAAAABIRZAAAAIkIMgAAgEQEGQAAQCKCDAAAIBFBBgAAkIggAwAASESQAQAAJCLIAAAAEhFkAAAAiQgyAACARAQZAABAIoIMAAAgEUEGAACQiCADAABIRJABAAAkIsgAAAASEWQAAACJCDIAAIBEBBkAAEAiggwAACARQQYAAJCIIAMAAEhEkAEAACQiyAAAABIRZAAAAIkIMgAAgEQEGQAAQCKCDAAAIBFBBgAAkIggAwAASESQAQAAJCLIAAAAEhFkAAAAiQgyAACARAQZAABAIoIMAAAgEUEGAACQiCADAABIRJABAAAkkjTIRo8eHSeccEK0a9cuOnXqFAMHDoy5c+fmjfn0009j2LBhsd9++8Xee+8d5557bixbtixvzKJFi2LAgAGx5557RqdOneL666+PtWvX5o2ZOnVqHHvssdG6devo3r17jB8/foP53HvvvXHAAQdEmzZtonfv3vHSSy81+nsGAAColzTIpk2bFsOGDYsXXnghJk6cGGvWrIn+/fvHqlWrcmOuvfba+POf/xyPPfZYTJs2LZYsWRJf//rXc+trampiwIABsXr16pgxY0Y8/PDDMX78+Lj11ltzYxYuXBgDBgyI008/PSoqKuKaa66J7373u/HMM8/kxjz66KMxYsSIGDVqVLz66qvRq1evKC8vj/fff3/nbAwAAKDZKciyLEs9iXoffPBBdOrUKaZNmxannnpqVFVVRceOHeN3v/tdfOMb34iIiDlz5sRhhx0WM2fOjJNOOin++te/xle+8pVYsmRJdO7cOSIi7r///hg5cmR88MEH0apVqxg5cmQ8/fTTMXv27NxrnX/++bFixYqYMGFCRET07t07TjjhhLjnnnsiIqK2tja6desWV155Zdxwww1bnHt1dXUUFxdHVVVVFBUVNfamAQAAdhMNaYNd6jNkVVVVERGx7777RkTErFmzYs2aNdGvX7/cmEMPPTT233//mDlzZkREzJw5M3r27JmLsYiI8vLyqK6ujjfffDM3Zt3nqB9T/xyrV6+OWbNm5Y0pLCyMfv365cas77PPPovq6uq8GwAAQEPsMkFWW1sb11xzTZx88slx5JFHRkREZWVltGrVKtq3b583tnPnzlFZWZkbs26M1a+vX7e5MdXV1fHJJ5/EP/7xj6ipqdnomPrnWN/o0aOjuLg4d+vWrdu2vXEAAKDZ2mWCbNiwYTF79uz4/e9/n3oqW+XGG2+Mqqqq3O29995LPSUAAGA30zL1BCIihg8fHk899VRMnz49SktLc8u7dOkSq1evjhUrVuQdJVu2bFl06dIlN2b9qyHWX4Vx3THrX5lx2bJlUVRUFG3bto0WLVpEixYtNjqm/jnW17p162jduvW2vWEAAIBIfIQsy7IYPnx4PPHEEzF58uQ48MAD89Yfd9xxsccee8SkSZNyy+bOnRuLFi2KPn36REREnz594o033si7GuLEiROjqKgoDj/88NyYdZ+jfkz9c7Rq1SqOO+64vDG1tbUxadKk3BgAAIDGlvQI2bBhw+J3v/td/PGPf4x27drlPq9VXFwcbdu2jeLi4hgyZEiMGDEi9t133ygqKoorr7wy+vTpEyeddFJERPTv3z8OP/zwuPDCC2Ps2LFRWVkZN998cwwbNix3BOvyyy+Pe+65J/793/89Lrnkkpg8eXL84Q9/iKeffjo3lxEjRsTgwYPj+OOPjxNPPDHuuuuuWLVqVVx88cU7f8MAAADNQtLL3hcUFGx0+UMPPRQXXXRRRNT9YejrrrsuHnnkkfjss8+ivLw8fvnLX+adSvjuu+/GFVdcEVOnTo299torBg8eHGPGjImWLT/vzalTp8a1114bb731VpSWlsYtt9ySe41699xzT9xxxx1RWVkZRx99dNx9993Ru3fvrXovLnsPAABENKwNdqm/Q7Y7E2QAAEDEbvx3yAAAAJoTQQYAAJCIIAMAAEhEkAEAACQiyAAAABIRZAAAAIkIMgAAgEQEGQAAQCKCDAAAIBFBBgAAkIggAwAASESQAQAAJCLIAAAAEhFkAAAAiQgyAACARAQZAABAIoIMAAAgEUEGAACQiCADAABIRJABAAAkIsgAAAASEWQAAACJCDIAAIBEBBkAAEAiggwAACARQQYAAJCIIAMAAEhEkAEAACQiyJqBxYsjpkypuwcAAHYdgqyJGzcuoqwsom/fuvtx41LPCAAAqCfImrDFiyOGDo2ora17XFsbcdlljpQBAMCuQpA1YfPmfR5j9WpqIubPTzMfAAAgnyBrwnr0iChc71+4RYuI7t3TzAcAAMgnyJqw0tKIBx+si7CIuvsHHqhbDgAApNcy9QTYsYYMiSgvrztNsXt3MQYAALsSQdYMlJYKMQAA2BU5ZREAACARQQYAAJCIIAMAAEhEkAEAACQiyAAAABIRZAAAAIkIMgAAgEQEGQAAQCKCDAAAIBFBBgAAkIggAwAASESQAQAAJCLIAAAAEhFkAAAAiQgyAACARAQZAABAIoIMAAAgEUEGAACQiCADAABIRJABAAAkIsgAAAASEWQAAACJCDIAAIBEBBkAAEAiggwAACARQQYAAJCIIAMAAEhEkAEAACQiyAAAABIRZAAAAIkIMgAAgEQEGQAAQCKCDAAAIBFBBgAAkIggAwAASESQAQAAJCLIAAAAEhFkAAAAiQgyAACARAQZAABAIoIMAAAgEUEGAACQiCADAABIRJABAAAkIsgAAAASEWQAAACJCDIAAIBEBBkAAEAiggwAACARQQYAAJCIIAMAAEhEkAEAACQiyAAAABIRZAAAAIkIMgAAgEQEGQAAQCKCDAAAIBFBBgAAkIggAwAASESQAQAAJCLIAAAAEkkaZNOnT49zzjknSkpKoqCgIJ588sm89RdddFEUFBTk3c4888y8McuXL49BgwZFUVFRtG/fPoYMGRIrV67MG/P666/HKaecEm3atIlu3brF2LFjN5jLY489Foceemi0adMmevbsGX/5y18a/f0CAACsK2mQrVq1Knr16hX33nvvJseceeaZsXTp0tztkUceyVs/aNCgePPNN2PixInx1FNPxfTp02Po0KG59dXV1dG/f/8oKyuLWbNmxR133BG33XZbPPjgg7kxM2bMiAsuuCCGDBkSr732WgwcODAGDhwYs2fPbvw3DQAA8N8KsizLUk8iIqKgoCCeeOKJGDhwYG7ZRRddFCtWrNjgyFm9t99+Ow4//PB4+eWX4/jjj4+IiAkTJsTZZ58dixcvjpKSkrjvvvvipptuisrKymjVqlVERNxwww3x5JNPxpw5cyIi4rzzzotVq1bFU089lXvuk046KY4++ui4//77t2r+1dXVUVxcHFVVVVFUVLQNWwAAAGgKGtIGu/xnyKZOnRqdOnWKQw45JK644or48MMPc+tmzpwZ7du3z8VYRES/fv2isLAwXnzxxdyYU089NRdjERHl5eUxd+7c+Oc//5kb069fv7zXLS8vj5kzZ25yXp999llUV1fn3QAAABpilw6yM888M/7jP/4jJk2aFD/+8Y9j2rRpcdZZZ0VNTU1ERFRWVkanTp3yvqdly5ax7777RmVlZW5M586d88bUP97SmPr1GzN69OgoLi7O3bp167Z9bxYAAGh2WqaewOacf/75ua979uwZRx11VBx88MExderUOOOMMxLOLOLGG2+MESNG5B5XV1eLMgAAoEF26SNk6zvooIOiQ4cOMX/+/IiI6NKlS7z//vt5Y9auXRvLly+PLl265MYsW7Ysb0z94y2NqV+/Ma1bt46ioqK8GwAAQEPsVkG2ePHi+PDDD6Nr164REdGnT59YsWJFzJo1Kzdm8uTJUVtbG717986NmT59eqxZsyY3ZuLEiXHIIYfEPvvskxszadKkvNeaOHFi9OnTZ0e/JQAAoBlLGmQrV66MioqKqKioiIiIhQsXRkVFRSxatChWrlwZ119/fbzwwgvxzjvvxKRJk+JrX/tadO/ePcrLyyMi4rDDDoszzzwzLr300njppZfi+eefj+HDh8f5558fJSUlERHxrW99K1q1ahVDhgyJN998Mx599NH4+c9/nne64dVXXx0TJkyIn/70pzFnzpy47bbb4pVXXonhw4fv9G0CAAA0H0kvez916tQ4/fTTN1g+ePDguO+++2LgwIHx2muvxYoVK6KkpCT69+8fP/zhD/MuwLF8+fIYPnx4/PnPf47CwsI499xz4+6774699947N+b111+PYcOGxcsvvxwdOnSIK6+8MkaOHJn3mo899ljcfPPN8c4770SPHj1i7NixcfbZZ2/1e3HZewAAIKJhbbDL/B2y3Z0gAwAAIprY3yEDAABoqgQZAABAIoIMAAAgEUEGAACQiCADAABIpMFBtmbNmmjZsmXMnj17R8wHAACg2WhwkO2xxx6x//77R01NzY6YDwAAQLOxTacs3nTTTfH9738/li9f3tjzAQAAaDZabss33XPPPTF//vwoKSmJsrKy2GuvvfLWv/rqq40yOQAAgKZsm4Js4MCBjTwNAACA5qcgy7Is9SSagurq6iguLo6qqqooKipKPR0AACCRhrTBNh0hqzdr1qx4++23IyLiiCOOiGOOOWZ7ng4AAKBZ2aYge//99+P888+PqVOnRvv27SMiYsWKFXH66afH73//++jYsWNjzhEAAKBJ2qarLF555ZXx0UcfxZtvvhnLly+P5cuXx+zZs6O6ujquuuqqxp4jAABAk7RNnyErLi6OZ599Nk444YS85S+99FL0798/VqxY0Vjz2234DBkAABDRsDbYpiNktbW1sccee2ywfI899oja2tpteUoAAIBmZ5uCrG/fvnH11VfHkiVLcsv+/ve/x7XXXhtnnHFGo00OAACgKdumILvnnnuiuro6DjjggDj44IPj4IMPjgMPPDCqq6vjF7/4RWPPEQAAoEnapqssduvWLV599dV49tlnY86cORERcdhhh0W/fv0adXIAAABNWYODbM2aNdG2bduoqKiIL3/5y/HlL395R8wLAACgyWvwKYt77LFH7L///lFTU7Mj5gMAANBsbNNnyG666ab4/ve/H8uXL2/s+QAAADQb2/QZsnvuuSfmz58fJSUlUVZWFnvttVfe+ldffbVRJgcAANCUbVOQDRw4sJGnAQAA0Pw0OMjWrl0bBQUFcckll0RpaemOmBMAAECz0ODPkLVs2TLuuOOOWLt27Y6YDwAAQLOxTRf16Nu3b0ybNq2x5wIAANCsbNNnyM4666y44YYb4o033ojjjjtug4t6fPWrX22UyQEAADRlBVmWZQ39psLCTR9YKygoaJZ/o6y6ujqKi4ujqqoqioqKUk8HAABIpCFtsE1HyGpra7dpYgAAAHyuQZ8hO/vss6Oqqir3eMyYMbFixYrc4w8//DAOP/zwRpscAABAU9agIHvmmWfis88+yz3+0Y9+FMuXL889Xrt2bcydO7fxZgcAANCENSjI1v+42TZ8/AwAAID/tk2XvQcAAGD7NSjICgoKoqCgYINlAAAANFyDrrKYZVlcdNFF0bp164iI+PTTT+Pyyy/P/R2ydT9fBgAAwOY1KMgGDx6c9/jb3/72BmO+853vbN+MAAAAmokGBdlDDz20o+YBAADQ7LioBwAAQCKCDAAAIBFBBgAAkIggAwAASESQNTOLF0dMmVJ3DwAApCXImpFx4yLKyiL69q27Hzcu9YwAAKB5E2TNxOLFEUOHRtTW1j2urY247DJHygAAICVB1kzMm/d5jNWrqYmYPz/NfAAAAEHWbPToEVG43r92ixYR3bunmQ8AACDImo3S0ogHH6yLsIi6+wceqFsOAACk0TL1BNh5hgyJKC+vO02xe3cxBgAAqQmyZqa0VIgBAMCuwimLAAAAiQgyAACARAQZAABAIoIMAAAgEUEGAACQiCADAABIRJABAAAkIsgAAAASEWQAAACJCDIAAIBEBBkAAEAiggwAACARQQYAAJCIIAMAAEhEkAEAACQiyAAAABIRZAAAAIkIMgAAgEQEGQAAQCKCDAAAIBFBBgAAkIggAwAASESQAQAAJCLIAAAAEhFkAAAAiQgyAACARAQZAABAIoIMAAAgEUEGAACQiCADAABIRJABAAAkIsgAAAASEWQAAACJCDIAAIBEBBkAAEAiggwAACARQQYAAJCIIAMAAEhEkAEAACQiyAAAABIRZAAAAIkIMgAAgEQEGQAAQCKCDAAAIBFBBgAAkIggAwAASESQAQAAJJI0yKZPnx7nnHNOlJSUREFBQTz55JN567Msi1tvvTW6du0abdu2jX79+sW8efPyxixfvjwGDRoURUVF0b59+xgyZEisXLkyb8zrr78ep5xySrRp0ya6desWY8eO3WAujz32WBx66KHRpk2b6NmzZ/zlL39p9PcLAACwrqRBtmrVqujVq1fce++9G10/duzYuPvuu+P++++PF198Mfbaa68oLy+PTz/9NDdm0KBB8eabb8bEiRPjqaeeiunTp8fQoUNz66urq6N///5RVlYWs2bNijvuuCNuu+22ePDBB3NjZsyYERdccEEMGTIkXnvttRg4cGAMHDgwZs+evePePAAA0OwVZFmWpZ5ERERBQUE88cQTMXDgwIioOzpWUlIS1113XXzve9+LiIiqqqro3LlzjB8/Ps4///x4++234/DDD4+XX345jj/++IiImDBhQpx99tmxePHiKCkpifvuuy9uuummqKysjFatWkVExA033BBPPvlkzJkzJyIizjvvvFi1alU89dRTufmcdNJJcfTRR8f999+/VfOvrq6O4uLiqKqqiqKiosbaLAAAwG6mIW2wy36GbOHChVFZWRn9+vXLLSsuLo7evXvHzJkzIyJi5syZ0b59+1yMRUT069cvCgsL48UXX8yNOfXUU3MxFhFRXl4ec+fOjX/+85+5Meu+Tv2Y+tfZmM8++yyqq6vzbgAAAA2xywZZZWVlRER07tw5b3nnzp1z6yorK6NTp05561u2bBn77rtv3piNPce6r7GpMfXrN2b06NFRXFycu3Xr1q2hbxEAAGjmdtkg29XdeOONUVVVlbu99957qacEAADsZnbZIOvSpUtERCxbtixv+bJly3LrunTpEu+//37e+rVr18by5cvzxmzsOdZ9jU2NqV+/Ma1bt46ioqK8GwAAQEPsskF24IEHRpcuXWLSpEm5ZdXV1fHiiy9Gnz59IiKiT58+sWLFipg1a1ZuzOTJk6O2tjZ69+6dGzN9+vRYs2ZNbszEiRPjkEMOiX322Sc3Zt3XqR9T/zoAAAA7QtIgW7lyZVRUVERFRUVE1F3Io6KiIhYtWhQFBQVxzTXXxP/6X/8r/vSnP8Ubb7wR3/nOd6KkpCR3JcbDDjsszjzzzLj00kvjpZdeiueffz6GDx8e559/fpSUlERExLe+9a1o1apVDBkyJN5888149NFH4+c//3mMGDEiN4+rr746JkyYED/96U9jzpw5cdttt8Urr7wSw4cP39mbBAAAaEaSXvZ+6tSpcfrpp2+wfPDgwTF+/PjIsixGjRoVDz74YKxYsSK++MUvxi9/+cv4whe+kBu7fPnyGD58ePz5z3+OwsLCOPfcc+Puu++OvffeOzfm9ddfj2HDhsXLL78cHTp0iCuvvDJGjhyZ95qPPfZY3HzzzfHOO+9Ejx49YuzYsXH22Wdv9Xtx2XsAACCiYW2wy/wdst2dIAMAACKayN8hAwAAaOoEGQAAQCKCDAAAIBFBBgAAkIggAwAASESQAQAAJCLIAAAAEhFkAAAAiQgyAACARAQZAABAIoIMAAAgEUEGAACQiCADAABIRJABAAAkIsiaucWLI6ZMqbsHAAB2LkHWjI0bF1FWFtG3b939uHGpZwQAAM2LIGumFi+OGDo0ora27nFtbcRllzlSBgAAO5Mga6bmzfs8xurV1ETMn59mPgAA0BwJsmaqR4+IwvX+9Vu0iOjePc18AACgORJkzVRpacSDD9ZFWETd/QMP1C0HAAB2jpapJ0A6Q4ZElJfXnabYvbsYAwCAnU2QNXOlpUIMAABSccoiAABAIoIMAAAgEUEGAACQiCADAABIRJABAAAkIsgAAAASEWQAAACJCDIAAIBEBBkAAEAiggwAACARQQYAAJCIIAMAAEhEkAEAACQiyAAAABIRZAAAAIkIMgAAgEQEGQAAQCKCDAAAIBFBBgAAkIggAwAASESQAQAAJCLIAAAAEhFkAAAAiQgyAACARAQZAABAIoIMAAAgEUEGAACQiCADAABIRJABAAAkIsgAAAASEWQAAACJCDIAAIBEBBkAAEAiggwAACARQQYAAJCIIAMAAEhEkAEAACQiyAAAABIRZAAAAIkIMgAAgEQEGQAAQCKCDAAAIBFBBgAAkIggAwAASESQAQAAJCLIyLN4ccSUKXX3AADAjiXIyBk3LqKsLKJv37r7ceNSzwgAAJo2QUZE1B0RGzo0ora27nFtbcRllzlSBgAAO5IgIyIi5s37PMbq1dREzJ+fZj4AANAcCDIiIqJHj4jC9faGFi0iundPMx8AAGgOBBkREVFaGvHgg3URFlF3/8ADdcsBAIAdo2XqCbDrGDIkory87jTF7t3FGAAA7GiCjDylpUIMAAB2FqcsAgAAJCLIAAAAEhFkAAAAiQgyAACARAQZAABAIoIMAAAgEUEGAACQiCADAABIRJABAAAkIsgAAAASEWQAAACJCDIAAIBEBBkAAEAiggwAACARQQYAAJCIIAMAAEhEkAEAACQiyAAAABIRZAAAAIkIMgAAgEQEGQAAQCK7dJDddtttUVBQkHc79NBDc+s//fTTGDZsWOy3336x9957x7nnnhvLli3Le45FixbFgAEDYs8994xOnTrF9ddfH2vXrs0bM3Xq1Dj22GOjdevW0b179xg/fvzOeHsAAEAzt0sHWUTEEUccEUuXLs3dnnvuudy6a6+9Nv785z/HY489FtOmTYslS5bE17/+9dz6mpqaGDBgQKxevTpmzJgRDz/8cIwfPz5uvfXW3JiFCxfGgAED4vTTT4+Kioq45ppr4rvf/W4888wzO/V9AgAAzU9BlmVZ6klsym233RZPPvlkVFRUbLCuqqoqOnbsGL/73e/iG9/4RkREzJkzJw477LCYOXNmnHTSSfHXv/41vvKVr8SSJUuic+fOERFx//33x8iRI+ODDz6IVq1axciRI+Ppp5+O2bNn5577/PPPjxUrVsSECRO2eq7V1dVRXFwcVVVVUVRUtH1vHAAA2G01pA12+SNk8+bNi5KSkjjooINi0KBBsWjRooiImDVrVqxZsyb69euXG3vooYfG/vvvHzNnzoyIiJkzZ0bPnj1zMRYRUV5eHtXV1fHmm2/mxqz7HPVj6p9jUz777LOorq7OuwEAADTELh1kvXv3jvHjx8eECRPivvvui4ULF8Ypp5wSH330UVRWVkarVq2iffv2ed/TuXPnqKysjIiIysrKvBirX1+/bnNjqqur45NPPtnk3EaPHh3FxcW5W7du3bb37QIAAM1My9QT2Jyzzjor9/VRRx0VvXv3jrKysvjDH/4Qbdu2TTiziBtvvDFGjBiRe1xdXS3KAACABtmlj5Ctr3379vGFL3wh5s+fH126dInVq1fHihUr8sYsW7YsunTpEhERXbp02eCqi/WPtzSmqKhos9HXunXrKCoqyrs1NYsXR0yZUncPAAA0vt0qyFauXBkLFiyIrl27xnHHHRd77LFHTJo0Kbd+7ty5sWjRoujTp09ERPTp0yfeeOONeP/993NjJk6cGEVFRXH44Yfnxqz7HPVj6p+juRo3LqKsLKJv37r7ceNSzwgAAJqeXfoqi9/73vfinHPOibKysliyZEmMGjUqKioq4q233oqOHTvGFVdcEX/5y19i/PjxUVRUFFdeeWVERMyYMSMi6i57f/TRR0dJSUmMHTs2Kisr48ILL4zvfve78aMf/Sgi6i57f+SRR8awYcPikksuicmTJ8dVV10VTz/9dJSXl2/1XJvSVRYXL66LsNraz5e1aBHxzjsRpaXJpgUAALuFhrTBLv0ZssWLF8cFF1wQH374YXTs2DG++MUvxgsvvBAdO3aMiIg777wzCgsL49xzz43PPvssysvL45e//GXu+1u0aBFPPfVUXHHFFdGnT5/Ya6+9YvDgwXH77bfnxhx44IHx9NNPx7XXXhs///nPo7S0NH71q181KMaamnnz8mMsIqKmJmL+fEEGAACNaZc+QrY7cYQMAACIaGJ/h4ydr7Q04sEH6yIsou7+gQfEGAAANLZd+pRF0hkyJKK8vO40xe7dxRgAAOwIgoxNKi0VYgAAsCM5ZREAACARQQYAAJCIIAMAAEhEkAEAACQiyAAAABIRZAAAAIkIMgAAgEQEGQAAQCKCDAAAIBFBBgAAkIggAwAASESQAQAAJCLIAAAAEhFkAAAAiQgyAACARAQZW23x4ogpU+ruAQCA7SfI2CrjxkWUlUX07Vt3P25c6hkBAMDuT5CxRYsXRwwdGlFbW/e4tjbissscKQMAgO0lyNiiefM+j7F6NTUR8+enmQ8AADQVgowt6tEjonC9PaVFi4ju3dPMBwAAmgpBxhaVlkY8+GBdhEXU3T/wQN1yAABg27VMPQF2D0OGRJSX152m2L27GAMAgMYgyNhqpaVCDAAAGpNTFgEAABIRZAAAAIkIMgAAgEQEGQAAQCKCDAAAIBFBBgAAkIggAwAASESQsU0WL46YMqXuHgAA2DaCjAYbNy6irCyib9+6+3HjUs8IAAB2T4KMBlm8OGLo0Ija2rrHtbURl13mSBkAAGwLQUaDzJv3eYzVq6mJmD8/zXwAAGB3JshokB49IgrX22tatIjo3j3NfAAAYHcmyGiQ0tKIBx+si7CIuvsHHqhbDgAANEzL1BNg9zNkSER5ed1pit27izEAANhWgoxtUloqxAAAYHs5ZREAACARQQYAAJCIIAMAAEhEkNEoFi+OmDLFH4gGAICGEGRst3HjIsrKIvr2rbsfNy71jAAAYPcgyNguixdHDB0aUVtb97i2NuKyyxwpAwCArSHI2C7z5n0eY/Vqaur+RhkAALB5gozt0qNHROF6e1GLFnV/MBoAANg8QcZ2KS2NePDBugiLqLt/4AF/NBoAALZGy9QTYPc3ZEhEeXndaYr1R8amTKk7eibMAABg0xwho1GUlkacdlrEM8+44iIAAGwtQUajccVFAABoGEFGo3HFRQAAaBhBRqNxxUUAAGgYQUaj2dQVFyPqLvLh1EUAAMgnyGhUQ4ZEvPNOXYC9807dMhf5AACAjSvIsixLPYmmoLq6OoqLi6OqqiqKiopST2eXsHhxXYSt+7myFi3qQs3l8AEAaKoa0gaOkLHDbOoiH4895vRFAACIEGTsQBu7yEdExIgRTl8EAIAIQcYOtP5FPtblb5QBAIAgYwerv8jHz3624bqamoiZM12BEQCA5kuQscOVlkZ885sbnr5YUBBx/vmuwAgAQPMlyNgp1j99sT7O6i/6UVsbMXRoxB/+4GgZAADNhyBjp1n3b5Q98kjE+n9wobY24rzzPj9atnix0xkBAGjaWqaeAM1LaWndbfHiuqNk618WP6Ju2aWX1p3SWFtbN27MmIjjj6+7cqO/YQYAQFPhCBlJbO4KjBF1R8/WPZ3x3/89/7Nmjp4BANAUFGTZ+ieOsS0a8te4+dzixXVXWjz//I0fLduYgoL8o2cPPhhRXl73h6h79KgbU/+1o2kAAOxsDWkDpyySVP0VGKur6/4uWU1NXWRl2YafMau37rr1T28sKPh8jFgDAGBX5whZI3GEbPstXhwxf35E9+4RzzzzeaBtj3WPpjU01hYvFm8AADRcQ9pAkDUSQdb46gPtlVcibrhh646eNcTmYu3CCyP+7//d9EVFxBoAAJsiyBIQZDvWpo6eNWagbY1tiTXxBgDQvAiyBATZzrWpQFv/gh+pY60h8RYh5AAAmgJBloAgS2vdQIvY9WJtfevH2vacMhkh5AAAdiWCLAFBtutqaKy1aBHx7W9H/OY3239RkcYm5AAAdn2CLAFBtnvaVKzVB8r6FxXZlWNtcxoacg25AuXm1gk7AKA5EmQJCLKmbd1w29pY213jLWLrr0C5o47QbW6dyAMAdnWCLAFB1nxtKtYaEm+7yymT26ohIbe9f4KgMaJv/XUAAA0hyBIQZGyNzcVbhJBriC1d1XJbo297/oD45taJRQBoPgRZAoKMHWlnhlzqK1Duahrr9M1dORY3t25HhaTTVwFoygRZAoKMXdG2hFxDrkDZnI/Q7QoaIxZThOTufPpqU49c8962o9MNeb2t/b4tjQV2bQ1qg4xGUVVVlUVEVlVVlXoq0Cjeey/Lpkypu1/3661dd8cdWdaiRd2xthYtsmzw4M8fFxRkWWFhw9e5uW3sVlhYt5/U7zfrPy4oqLvtCutSv755N85r/OpXdT/rJk+u+1m3ta+3td+3pbHrrqv/OVz/df3P4R25bme8hnnvOq+/O88tpYa0QeyE+TQLggw2tC0ht62R11jRt+66wsLPf5lyc3Nzq7+t+3NiR33f5sauu66pRK55737zTv36W5rbr37VuL/TNFRD2sApi43EKYuwc2ztVS23dFrm1q5rjNM3GzJ2a9f5rB8AbFqLFhHvvJPuFF+fIUtAkEHT1RiRt6vG4s4OycZaBwBbMmVKxGmnpXltnyFLwCmLQArbexroltbtjNfYHU5fbax1qV/fvHfsqcyN9X2bG+tUaje3rbu1aJH2s2Q+Q5aAIAPYuXbHkNwVXt+8t3/dr36VH31jxzb+921u7LrrmkLkmvfuOe/Ur7+lufkMWTPklEUAaD7WPyV5R3zf5sY29inQqU/BNu/dc96pX39Lc0vJZ8gSEGQAAEBEw9qgcCfNCQAAgPUIMgAAgEQEGQAAQCKCDAAAIBFBBgAAkIggAwAASESQAQAAJCLIAAAAEhFk67n33nvjgAMOiDZt2kTv3r3jpZdeSj0lAACgiRJk63j00UdjxIgRMWrUqHj11VejV69eUV5eHu+//37qqQEAAE2QIFvHz372s7j00kvj4osvjsMPPzzuv//+2HPPPePXv/516qkBAABNkCD7b6tXr45Zs2ZFv379cssKCwujX79+MXPmzA3Gf/bZZ1FdXZ13AwAAaAhB9t/+8Y9/RE1NTXTu3DlveefOnaOysnKD8aNHj47i4uLcrVu3bjtrqgAAQBMhyLbRjTfeGFVVVbnbe++9l3pKAADAbqZl6gnsKjp06BAtWrSIZcuW5S1ftmxZdOnSZYPxrVu3jtatW++s6QEAAE2QI2T/rVWrVnHcccfFpEmTcstqa2tj0qRJ0adPn4QzAwAAmipHyNYxYsSIGDx4cBx//PFx4oknxl133RWrVq2Kiy++OPXUAACAJkiQreO8886LDz74IG699daorKyMo48+OiZMmLDBhT4AAAAaQ0GWZVnqSTQF1dXVUVxcHFVVVVFUVJR6OgAAQCINaQOfIQMAAEhEkAEAACQiyAAAABJxUY9GUv9RvOrq6sQzAQAAUqpvgq25XIcgayQfffRRRER069Yt8UwAAIBdwUcffRTFxcWbHeMqi42ktrY2lixZEu3atYuCgoKkc6muro5u3brFe++954qPO4htvGPZvjuebbxj2b47nm28Y9m+O55tvGOl3r5ZlsVHH30UJSUlUVi4+U+JOULWSAoLC6O0tDT1NPIUFRX5H/gOZhvvWLbvjmcb71i2745nG+9Ytu+OZxvvWCm375aOjNVzUQ8AAIBEBBkAAEAigqwJat26dYwaNSpat26deipNlm28Y9m+O55tvGPZvjuebbxj2b47nm28Y+1O29dFPQAAABJxhAwAACARQQYAAJCIIAMAAEhEkAEAACQiyJqge++9Nw444IBo06ZN9O7dO1566aXUU9otjR49Ok444YRo165ddOrUKQYOHBhz587NG3PaaadFQUFB3u3yyy9PNOPdz2233bbB9jv00ENz6z/99NMYNmxY7LfffrH33nvHueeeG8uWLUs4493LAQccsMH2LSgoiGHDhkWE/XdbTJ8+Pc4555woKSmJgoKCePLJJ/PWZ1kWt956a3Tt2jXatm0b/fr1i3nz5uWNWb58eQwaNCiKioqiffv2MWTIkFi5cuVOfBe7rs1t3zVr1sTIkSOjZ8+esddee0VJSUl85zvfiSVLluQ9x8b2+zFjxuzkd7Lr2tI+fNFFF22w/c4888y8MfbhTdvS9t3Yz+SCgoK44447cmPsw5u2Nb+bbc3vDosWLYoBAwbEnnvuGZ06dYrrr78+1q5duzPfSh5B1sQ8+uijMWLEiBg1alS8+uqr0atXrygvL4/3338/9dR2O9OmTYthw4bFCy+8EBMnTow1a9ZE//79Y9WqVXnjLr300li6dGnuNnbs2EQz3j0dccQRedvvueeey6279tpr489//nM89thjMW3atFiyZEl8/etfTzjb3cvLL7+ct20nTpwYERHf/OY3c2Psvw2zatWq6NWrV9x7770bXT927Ni4++674/77748XX3wx9tprrygvL49PP/00N2bQoEHx5ptvxsSJE+Opp56K6dOnx9ChQ3fWW9ilbW77fvzxx/Hqq6/GLbfcEq+++mo8/vjjMXfu3PjqV7+6wdjbb789b7++8sord8b0dwtb2ocjIs4888y87ffII4/krbcPb9qWtu+623Xp0qXx61//OgoKCuLcc8/NG2cf3rit+d1sS7871NTUxIABA2L16tUxY8aMePjhh2P8+PFx6623pnhLdTKalBNPPDEbNmxY7nFNTU1WUlKSjR49OuGsmob3338/i4hs2rRpuWVf+tKXsquvvjrdpHZzo0aNynr16rXRdStWrMj22GOP7LHHHsste/vtt7OIyGbOnLmTZti0XH311dnBBx+c1dbWZllm/91eEZE98cQTuce1tbVZly5dsjvuuCO3bMWKFVnr1q2zRx55JMuyLHvrrbeyiMhefvnl3Ji//vWvWUFBQfb3v/99p819d7D+9t2Yl156KYuI7N13380tKysry+68884dO7kmYmPbePDgwdnXvva1TX6PfXjrbc0+/LWvfS3r27dv3jL78NZb/3ezrfnd4S9/+UtWWFiYVVZW5sbcd999WVFRUfbZZ5/t3Dfw3xwha0JWr14ds2bNin79+uWWFRYWRr9+/WLmzJkJZ9Y0VFVVRUTEvvvum7f8t7/9bXTo0CGOPPLIuPHGG+Pjjz9OMb3d1rx586KkpCQOOuigGDRoUCxatCgiImbNmhVr1qzJ258PPfTQ2H///e3P22D16tXxm9/8Ji655JIoKCjILbf/Np6FCxdGZWVl3j5bXFwcvXv3zu2zM2fOjPbt28fxxx+fG9OvX78oLCyMF198cafPeXdXVVUVBQUF0b59+7zlY8aMif322y+OOeaYuOOOO5KeirQ7mjp1anTq1CkOOeSQuOKKK+LDDz/MrbMPN55ly5bF008/HUOGDNlgnX1466z/u9nW/O4wc+bM6NmzZ3Tu3Dk3pry8PKqrq+PNN9/cibP/XMskr8oO8Y9//CNqamrydrCIiM6dO8ecOXMSzappqK2tjWuuuSZOPvnkOPLII3PLv/Wtb0VZWVmUlJTE66+/HiNHjoy5c+fG448/nnC2u4/evXvH+PHj45BDDomlS5fGD37wgzjllFNi9uzZUVlZGa1atdrgF63OnTtHZWVlmgnvxp588slYsWJFXHTRRbll9t/GVb9fbuxncP26ysrK6NSpU976li1bxr777mu/bqBPP/00Ro4cGRdccEEUFRXlll911VVx7LHHxr777hszZsyIG2+8MZYuXRo/+9nPEs5293HmmWfG17/+9TjwwANjwYIF8f3vfz/OOuusmDlzZrRo0cI+3IgefvjhaNeu3Qan4tuHt87Gfjfbmt8dKisrN/pzun5dCoIMtsKwYcNi9uzZeZ9vioi8c+Z79uwZXbt2jTPOOCMWLFgQBx988M6e5m7nrLPOyn191FFHRe/evaOsrCz+8Ic/RNu2bRPOrOkZN25cnHXWWVFSUpJbZv9ld7VmzZr413/918iyLO677768dSNGjMh9fdRRR0WrVq3isssui9GjR0fr1q139lR3O+eff37u6549e8ZRRx0VBx98cEydOjXOOOOMhDNren7961/HoEGDok2bNnnL7cNbZ1O/m+2OnLLYhHTo0CFatGixwZVkli1bFl26dEk0q93f8OHD46mnnoopU6ZEaWnpZsf27t07IiLmz5+/M6bW5LRv3z6+8IUvxPz586NLly6xevXqWLFiRd4Y+3PDvfvuu/Hss8/Gd7/73c2Os/9un/r9cnM/g7t06bLBRZbWrl0by5cvt19vpfoYe/fdd2PixIl5R8c2pnfv3rF27dp45513ds4Em5iDDjooOnTokPu5YB9uHP/1X/8Vc+fO3eLP5Qj78MZs6nezrfndoUuXLhv9OV2/LgVB1oS0atUqjjvuuJg0aVJuWW1tbUyaNCn69OmTcGa7pyzLYvjw4fHEE0/E5MmT48ADD9zi91RUVERERNeuXXfw7JqmlStXxoIFC6Jr165x3HHHxR577JG3P8+dOzcWLVpkf26ghx56KDp16hQDBgzY7Dj77/Y58MADo0uXLnn7bHV1dbz44ou5fbZPnz6xYsWKmDVrVm7M5MmTo7a2NhfEbFp9jM2bNy+effbZ2G+//bb4PRUVFVFYWLjBaXZsncWLF8eHH36Y+7lgH24c48aNi+OOOy569eq1xbH24c9t6XezrfndoU+fPvHGG2/k/R8L9f/nzuGHH75z3sj6klxKhB3m97//fda6dets/Pjx2VtvvZUNHTo0a9++fd6VZNg6V1xxRVZcXJxNnTo1W7p0ae728ccfZ1mWZfPnz89uv/327JVXXskWLlyY/fGPf8wOOuig7NRTT008893Hddddl02dOjVbuHBh9vzzz2f9+vXLOnTokL3//vtZlmXZ5Zdfnu2///7Z5MmTs1deeSXr06dP1qdPn8Sz3r3U1NRk+++/fzZy5Mi85fbfbfPRRx9lr732Wvbaa69lEZH97Gc/y1577bXcVf7GjBmTtW/fPvvjH/+Yvf7669nXvva17MADD8w++eST3HOceeaZ2THHHJO9+OKL2XPPPZf16NEju+CCC1K9pV3K5rbv6tWrs69+9atZaWlpVlFRkfdzuf7KaDNmzMjuvPPOrKKiIluwYEH2m9/8JuvYsWP2ne98J/E723Vsbht/9NFH2fe+971s5syZ2cKFC7Nnn302O/bYY7MePXpkn376ae457MObtqWfEVmWZVVVVdmee+6Z3XfffRt8v31487b0u1mWbfl3h7Vr12ZHHnlk1r9//6yioiKbMGFC1rFjx+zGG29M8ZayLMsyQdYE/eIXv8j233//rFWrVtmJJ56YvfDCC6mntFuKiI3eHnrooSzLsmzRokXZqaeemu27775Z69ats+7du2fXX399VlVVlXbiu5Hzzjsv69q1a9aqVavsf/yP/5Gdd9552fz583PrP/nkk+zf/u3fsn322Sfbc889s3/5l3/Jli5dmnDGu59nnnkmi4hs7ty5ecvtv9tmypQpG/25MHjw4CzL6i59f8stt2SdO3fOWrdunZ1xxhkbbPsPP/wwu+CCC7K99947Kyoqyi6++OLso48+SvBudj2b274LFy7c5M/lKVOmZFmWZbNmzcp69+6dFRcXZ23atMkOO+yw7Ec/+lFeTDR3m9vGH3/8cda/f/+sY8eO2R577JGVlZVll1566Qb/p659eNO29DMiy7LsgQceyNq2bZutWLFig++3D2/eln43y7Kt+93hnXfeyc4666ysbdu2WYcOHbLrrrsuW7NmzU5+N58ryLIs20EH3wAAANgMnyEDAABIRJABAAAkIsgAAAASEWQAAACJCDIAAIBEBBkAAEAiggwAACARQQbAbumdd96JgoKCqKioSD2VnDlz5sRJJ50Ubdq0iaOPPnqjY0477bS45pprduq8tkZBQUE8+eSTqacB0OwIMgC2yUUXXRQFBQUxZsyYvOVPPvlkFBQUJJpVWqNGjYq99tor5s6dG5MmTdromMcffzx++MMf5h4fcMABcdddd+2kGUbcdtttG43FpUuXxllnnbXT5gFAHUEGwDZr06ZN/PjHP45//vOfqafSaFavXr3N37tgwYL44he/GGVlZbHffvttdMy+++4b7dq12+bX2JTtmXdERJcuXaJ169aNNBsAtpYgA2Cb9evXL7p06RKjR4/e5JiNHZG566674oADDsg9vuiii2LgwIHxox/9KDp37hzt27eP22+/PdauXRvXX3997LvvvlFaWhoPPfTQBs8/Z86c+J//839GmzZt4sgjj4xp06blrZ89e3acddZZsffee0fnzp3jwgsvjH/84x+59aeddloMHz48rrnmmujQoUOUl5dv9H3U1tbG7bffHqWlpdG6des4+uijY8KECbn1BQUFMWvWrLj99tujoKAgbrvtto0+z7qnLJ522mnx7rvvxrXXXhsFBQV5Rxafe+65OOWUU6Jt27bRrVu3uOqqq2LVqlW59QcccED88Ic/jO985ztRVFQUQ4cOjYiIkSNHxhe+8IXYc88946CDDopbbrkl1qxZExER48ePjx/84Afxt7/9Lfd648ePz81/3VMW33jjjejbt2+0bds29ttvvxg6dGisXLlyg3+zn/zkJ9G1a9fYb7/9YtiwYbnXioj45S9/GT169Ig2bdpE586d4xvf+MZGtwlAcybIANhmLVq0iB/96Efxi1/8IhYvXrxdzzV58uRYsmRJTJ8+PX72s5/FqFGj4itf+Urss88+8eKLL8bll18el1122Qavc/3118d1110Xr732WvTp0yfOOeec+PDDDyMiYsWKFdG3b9845phj4pVXXokJEybEsmXL4l//9V/znuPhhx+OVq1axfPPPx/333//Ruf385//PH7605/GT37yk3j99dejvLw8vvrVr8a8efMiou6UvyOOOCKuu+66WLp0aXzve9/b4nt+/PHHo7S0NG6//fZYunRpLF26NCLqjrSdeeaZce6558brr78ejz76aDz33HMxfPjwvO//yU9+Er169YrXXnstbrnlloiIaNeuXYwfPz7eeuut+PnPfx7/5//8n7jzzjsjIuK8886L6667Lo444ojc65133nkbzGvVqlVRXl4e++yzT7z88svx2GOPxbPPPrvB60+ZMiUWLFgQU6ZMiYcffjjGjx+fC7xXXnklrrrqqrj99ttj7ty5MWHChDj11FO3uE0Amp0MALbB4MGDs6997WtZlmXZSSedlF1yySVZlmXZE088ka37n5dRo0ZlvXr1yvveO++8MysrK8t7rrKysqympia37JBDDslOOeWU3OO1a9dme+21V/bII49kWZZlCxcuzCIiGzNmTG7MmjVrstLS0uzHP/5xlmVZ9sMf/jDr379/3mu/9957WURkc+fOzbIsy770pS9lxxxzzBbfb0lJSfa///f/zlt2wgknZP/2b/+We9yrV69s1KhRm32eL33pS9nVV1+de1xWVpbdeeedeWOGDBmSDR06NG/Zf/3Xf2WFhYXZJ598kvu+gQMHbnHed9xxR3bcccflHm/s3yPLsiwisieeeCLLsix78MEHs3322SdbuXJlbv3TTz+dFRYWZpWVlVmWff5vtnbt2tyYb37zm9l5552XZVmW/ed//mdWVFSUVVdXb3GOAM2ZI2QAbLcf//jH8fDDD8fbb7+9zc9xxBFHRGHh5/9Z6ty5c/Ts2TP3uEWLFrHffvvF+++/n/d9ffr0yX3dsmXLOP7443Pz+Nvf/hZTpkyJvffeO3c79NBDI6LuKFS94447brNzq66ujiVLlsTJJ5+ct/zkk0/erve8KX/7299i/PjxefMuLy+P2traWLhwYW7c8ccfv8H3Pvroo3HyySdHly5dYu+9946bb745Fi1a1KDXf/vtt6NXr16x11575ZadfPLJUVtbG3Pnzs0tO+KII6JFixa5x127ds39+3z5y1+OsrKyOOigg+LCCy+M3/72t/Hxxx83aB4AzYEgA2C7nXrqqVFeXh433njjBusKCwsjy7K8Zet+zqjeHnvskfe4oKBgo8tqa2u3el4rV66Mc845JyoqKvJu8+bNyzt9bt3w2BWsXLkyLrvssrw5/+1vf4t58+bFwQcfnBu3/rxnzpwZgwYNirPPPjueeuqpeO211+Kmm27a7gt+bMrm/n3atWsXr776ajzyyCPRtWvXuPXWW6NXr16xYsWKHTIXgN1Vy9QTAKBpGDNmTBx99NFxyCGH5C3v2LFjVFZWRpZluYtWNObfDnvhhRdycbV27dqYNWtW7rNOxx57bPznf/5nHHDAAdGy5bb/J6+oqChKSkri+eefjy996Uu55c8//3yceOKJ2zX/Vq1aRU1NTd6yY489Nt56663o3r17g55rxowZUVZWFjfddFNu2bvvvrvF11vfYYcdFuPHj49Vq1blou/555+PwsLCDf59N6dly5bRr1+/6NevX4waNSrat28fkydPjq9//esNeFcATZsjZAA0ip49e8agQYPi7rvvzlt+2mmnxQcffBBjx46NBQsWxL333ht//etfG+1177333njiiSdizpw5MWzYsPjnP/8Zl1xySUREDBs2LJYvXx4XXHBBvPzyy7FgwYJ45pln4uKLL95ilKzv+uuvjx//+Mfx6KOPxty5c+OGG26IioqKuPrqq7dr/gcccEBMnz49/v73v+eu/jhy5MiYMWNGDB8+PHdE749//OMGF9VYX48ePWLRokXx+9//PhYsWBB33313PPHEExu83sKFC6OioiL+8Y9/xGeffbbB8wwaNCjatGkTgwcPjtmzZ8eUKVPiyiuvjAsvvDA6d+68Ve/rqaeeirvvvjsqKiri3Xffjf/4j/+I2traBgUdQHMgyABoNLfffvsGpxQedthh8ctf/jLuvffe6NWrV7z00ktbdQXCrTVmzJgYM2ZM9OrVK5577rn405/+FB06dIiIyB3Vqqmpif79+0fPnj3jmmuuifbt2+d9Xm1rXHXVVTFixIi47rrromfPnjFhwoT405/+FD169Niu+d9+++3xzjvvxMEHHxwdO3aMiIijjjoqpk2bFv/v//2/OOWUU+KYY46JW2+9NUpKSjb7XF/96lfj2muvjeHDh8fRRx8dM2bMyF19sd65554bZ555Zpx++unRsWPHeOSRRzZ4nj333DOeeeaZWL58eZxwwgnxjW98I84444y45557tvp9tW/fPh5//PHo27dvHHbYYXH//ffHI488EkccccRWPwdAc1CQrX9iPwAAADuFI2QAAACJCDIAAIBEBBkAAEAiggwAACARQQYAAJCIIAMAAEhEkAEAACQiyAAAABIRZAAAAIkIMgAAgEQEGQAAQCKCDAAAIJH/D7pJ1wgIqfDkAAAAAElFTkSuQmCC\n"
          },
          "metadata": {}
        }
      ]
    },
    {
      "cell_type": "code",
      "metadata": {
        "id": "3Sn1erIU83ck"
      },
      "source": [],
      "execution_count": 15,
      "outputs": []
    },
    {
      "cell_type": "markdown",
      "source": [
        "Things to try out:\n",
        "\n",
        "1. Change batch size in mini-batch gradient descent.\n",
        "2. Test all the three out on real datasets.\n",
        "3. Compare the effects of changing learning rate by the same amount in Batch GD, SGD and Mini-batch GD."
      ],
      "metadata": {
        "id": "0neTARjKUoP4"
      }
    },
    {
      "cell_type": "markdown",
      "source": [
        "#1.\n",
        "\n",
        "   In mini-batch gradient descent, the  batch size is a hyperparameter that determines the number of training examples used in each iteration to compute the gradient of the cost function. Changing the batch size can have an impact on the training process and the convergence of the model. Here are some considerations when changing the batch size:\n",
        "\n",
        "**Batch Size Definitions:**\n",
        "\n",
        "Batch Gradient Descent (Batch Size = Number of Training Examples): Computes the gradient of the cost function using the entire training dataset. It is computationally expensive and memory-intensive but usually provides a stable convergence.\n",
        "Mini-Batch Gradient Descent (1 < Batch Size < Number of Training Examples): Uses a subset (mini-batch) of the training dataset to compute the gradient. It strikes a balance between the efficiency of batch gradient descent and the faster convergence often achieved with stochastic gradient descent.\n",
        "Impact of Batch Size:\n",
        "\n",
        "**Smaller Batch Size:**\n",
        "\n",
        "Faster convergence as updates to the model parameters are more frequent.\n",
        "Increased noise in the parameter updates due to the randomness in the mini-batch selection.\n",
        "Can lead to a less stable convergence, and the model might oscillate around the minimum.\n",
        "\n",
        "**Larger Batch Size:**\n",
        "\n",
        "More stable convergence as the gradient estimates are based on a larger sample.\n",
        "Slower convergence due to fewer updates to the model parameters in each iteration.\n",
        "Requires more memory and computational resources.\n",
        "Choosing the Batch Size:\n",
        "\n",
        "The choice of batch size often depends on the size of the dataset, available computational resources, and the specific characteristics of the problem.\n",
        "Common choices for batch size include powers of 2 (e.g., 32, 64, 128) for efficiency in hardware utilization.\n",
        "\n",
        "**Implementation:**\n",
        "\n",
        "In deep learning frameworks like TensorFlow or PyTorch, you can typically specify the batch size when creating or configuring the data loader or iterator.\n",
        "Adjusting the learning rate is often necessary when changing the batch size to ensure appropriate updates to the model parameters."
      ],
      "metadata": {
        "id": "FzWrQkuFn5pQ"
      }
    },
    {
      "cell_type": "markdown",
      "source": [
        "#2.\n",
        "\n",
        "Assuming you have a real dataset in mind, you can follow these general steps:\n",
        "\n",
        "1. **Load your Real Dataset:**\n",
        "Replace the MNIST dataset in the previous example with your own dataset. Make sure to preprocess the data appropriately.\n",
        "\n",
        "2. **Choose Batch Sizes:**\n",
        "Define a range of batch sizes that you want to test (e.g., [32, 64, 128]).\n",
        "\n",
        "3.**Modify the Training Function:**\n",
        "\n",
        "Adjust the train_model function to accept the batch size as a parameter and use it in the training loop.\n",
        "python\n",
        "Copy code\n",
        "def train_model(batch_size):\n",
        "\n",
        "\n",
        "4.**Execute Training with Different Batch Sizes:**\n",
        "Loop through the chosen batch sizes and call the train_model function for each batch size.\n",
        "python\n",
        "Copy code\n",
        "batch_sizes = [32, 64, 128]\n",
        "\n",
        "   for batch_size in batch_sizes:\n",
        "    print(f\"\\nTraining with batch size: {batch_size}\")\n",
        "    history = train_model(batch_size)\n",
        "\n",
        "5.**Analyze Results:**\n",
        "\n",
        "Examine the generated accuracy plots to understand how different batch sizes impact training and validation performance. Look for convergence speed and final accuracy."
      ],
      "metadata": {
        "id": "u9a835kwpYd3"
      }
    },
    {
      "cell_type": "markdown",
      "source": [
        "#3.\n",
        "\n",
        "Changing the learning rate in Batch Gradient Descent (BGD), Stochastic Gradient Descent (SGD), and Mini-batch Gradient Descent (MBGD) can have varying effects on the training process. Here's a general comparison of how changing the learning rate might impact these three optimization algorithms:\n",
        "\n",
        "1. Batch Gradient Descent (BGD):\n",
        "Effect of Increasing Learning Rate:\n",
        "If the learning rate is too high, BGD may overshoot the minimum and fail to converge.\n",
        "High learning rates can lead to oscillations or divergence in the cost function.\n",
        "Effect of Decreasing Learning Rate:\n",
        "Slower convergence if the learning rate is too low.\n",
        "More stable convergence but might get stuck in local minima.\n",
        "2. Stochastic Gradient Descent (SGD):\n",
        "Effect of Increasing Learning Rate:\n",
        "SGD is more resilient to larger learning rates due to the frequent updates.\n",
        "However, too high a learning rate can still cause divergence or overshooting.\n",
        "Effect of Decreasing Learning Rate:\n",
        "A smaller learning rate may slow down the convergence but can provide more stable updates.\n",
        "3. Mini-batch Gradient Descent (MBGD):\n",
        "Effect of Increasing Learning Rate:\n",
        "Similar to SGD, MBGD can handle larger learning rates well due to frequent updates.\n",
        "However, very high learning rates may still cause instability.\n",
        "Effect of Decreasing Learning Rate:\n",
        "Smaller learning rates may slow down convergence but can lead to more stable updates.\n",
        "Comparison Summary:\n",
        "\n",
        "\n",
        "**BGD vs. SGD:**\n",
        "\n",
        "  BGD tends to be more sensitive to learning rate changes due to infrequent updates.\n",
        "SGD can handle larger learning rates due to frequent updates but may have higher variance in the updates.\n",
        "\n",
        "**SGD vs. MBGD:**\n",
        "\n",
        "Both SGD and MBGD can handle larger learning rates well.\n",
        "MBGD's intermediate approach, using mini-batches, often strikes a balance between the stability of BGD and the faster convergence of SGD.\n",
        "\n",
        "**BGD vs. MBGD:**\n",
        "\n",
        "MBGD can adapt to changes in the learning rate more quickly than BGD due to the use of mini-batches.\n",
        "Practical Considerations:\n",
        "\n",
        "**Grid Search or Hyperparameter Tuning:**\n",
        "\n",
        "It's common to perform a grid search or hyperparameter tuning to find the optimal learning rate for your specific problem.\n",
        "\n",
        "**Learning Rate Schedules:**\n",
        "\n",
        "Techniques such as learning rate schedules (e.g., reducing the learning rate over time) can be employed to improve convergence.\n",
        "\n",
        "**Validation Set:**\n",
        "\n",
        "Monitor the performance on a validation set to prevent overfitting and ensure generalization."
      ],
      "metadata": {
        "id": "R7Uo_yBHqLpS"
      }
    }
  ]
}