Module_01_Lab_03_Augmentation.ipynb

{
  "nbformat": 4,
  "nbformat_minor": 0,
  "metadata": {
    "colab": {
      "provenance": [],
      "include_colab_link": true
    },
    "kernelspec": {
      "name": "python3",
      "display_name": "Python 3"
    },
    "language_info": {
      "name": "python"
    }
  },
  "cells": [
    {
      "cell_type": "markdown",
      "metadata": {
        "id": "view-in-github",
        "colab_type": "text"
      },
      "source": [
        "<a href=\"https://colab.research.google.com/github/Harsha-070/FMML_IIITH/blob/main/Module_01_Lab_03_Augmentation.ipynb\" target=\"_parent\"><img src=\"https://colab.research.google.com/assets/colab-badge.svg\" alt=\"Open In Colab\"/></a>"
      ]
    },
    {
      "cell_type": "markdown",
      "metadata": {
        "id": "w3yfry25JgZK"
      },
      "source": [
        "# Data augmentation\n",
        "\n",
        "FMML Module 1, Lab 3<br>\n",
        "\n",
        " In this lab, we will see how augmentation of data samples help in improving the machine learning performance"
      ]
    },
    {
      "cell_type": "code",
      "metadata": {
        "id": "xZU8_elooqP0"
      },
      "source": [
        "import numpy as np\n",
        "import matplotlib.pyplot as plt\n",
        "from keras.datasets import mnist\n",
        "# set randomseed\n",
        "rng = np.random.default_rng(seed=42)\n",
        "from sklearn.utils.extmath import cartesian\n",
        "from skimage.transform import rotate, AffineTransform, warp\n",
        "import math"
      ],
      "execution_count": 1,
      "outputs": []
    },
    {
      "cell_type": "markdown",
      "metadata": {
        "id": "T5pHYogSMHiE"
      },
      "source": [
        "Augmentation is useful when we have less training data available. Augmentation allows us to 'create' a larger dataset programatically.\n",
        "\n",
        "For this lab we will use a subset of MNIST that is very small, to better understand the effect of augmentation."
      ]
    },
    {
      "cell_type": "code",
      "metadata": {
        "id": "gJvmWJ58ovx5",
        "colab": {
          "base_uri": "https://localhost:8080/"
        },
        "outputId": "339a94d2-3fec-481f-be7c-025075a8123d"
      },
      "source": [
        "#loading the dataset\n",
        "(train_X, train_y), (test_X, test_y) = mnist.load_data()\n",
        "train_X = train_X/255\n",
        "test_X = test_X/255\n",
        "\n",
        "train_X = train_X[::1200,:,:].copy() # subsample. Otherwise it will take too long!\n",
        "train_y = train_y[::1200].copy() # do the same to the labels"
      ],
      "execution_count": 2,
      "outputs": [
        {
          "output_type": "stream",
          "name": "stdout",
          "text": [
            "Downloading data from https://storage.googleapis.com/tensorflow/tf-keras-datasets/mnist.npz\n",
            "11490434/11490434 [==============================] - 0s 0us/step\n"
          ]
        }
      ]
    },
    {
      "cell_type": "markdown",
      "metadata": {
        "id": "8XamH6z1Rt7S"
      },
      "source": [
        "Let us borrow a few functions from the previous labs:"
      ]
    },
    {
      "cell_type": "code",
      "metadata": {
        "id": "Zk2W5_3BRLMS"
      },
      "source": [
        "def NN1(traindata, trainlabel, query):\n",
        "  diff  = traindata - query  # find the difference between features. Numpy automatically takes care of the size here\n",
        "  sq = diff*diff # square the differences\n",
        "  dist = sq.sum(1) # add up the squares\n",
        "  label = trainlabel[np.argmin(dist)] # our predicted label is the label of the training data which has the least distance from the query\n",
        "  return label\n",
        "\n",
        "def NN(traindata, trainlabel, testdata):\n",
        "  # we will run nearest neighbour for each sample in the test data\n",
        "  # and collect the predicted classes in an array using list comprehension\n",
        "  traindata = traindata.reshape(-1, 28*28)\n",
        "  testdata = testdata.reshape(-1,28*28)\n",
        "  predlabel = np.array([NN1(traindata, trainlabel, i) for i in testdata])\n",
        "  return predlabel\n",
        "\n",
        "def Accuracy(gtlabel, predlabel):\n",
        "  assert len(gtlabel)==len(predlabel), \"Length of the groundtruth labels and predicted labels should be the same\"\n",
        "  correct = (gtlabel==predlabel).sum() # count the number of times the groundtruth label is equal to the predicted label.\n",
        "  return correct/len(gtlabel)"
      ],
      "execution_count": 3,
      "outputs": []
    },
    {
      "cell_type": "markdown",
      "metadata": {
        "id": "eGiA3LmDSJZo"
      },
      "source": [
        "In this lab, we will use the image pixels themselves as features, instead of extracting features. Each image has 28*28 pixels, so we will flatten them to 784 pixels to use as features. Note that this is very compute intensive and will take a long time.<br>\n",
        "\n",
        "Let us check the baseline accuracy on the test set without any augmentations. We hope that adding augmentations will help us to get better results."
      ]
    },
    {
      "cell_type": "code",
      "metadata": {
        "id": "4tQvnoasRNEV",
        "colab": {
          "base_uri": "https://localhost:8080/"
        },
        "outputId": "96cc2084-0166-4cc0-fa42-3ddf870d7667"
      },
      "source": [
        "testpred = NN(train_X, train_y, test_X)\n",
        "print('Baseline accuracy without augmentation is ', Accuracy(test_y, testpred))"
      ],
      "execution_count": 4,
      "outputs": [
        {
          "output_type": "stream",
          "name": "stdout",
          "text": [
            "Baseline accuracy without augmentation is  0.6472\n"
          ]
        }
      ]
    },
    {
      "cell_type": "markdown",
      "metadata": {
        "id": "ZfkcMfhIZQ7U"
      },
      "source": [
        "Let us try to improve this accuracy using augmentations. When we create augmentations, we have to make sure that the changes reflect what will naturally occur in the dataset. For example, we should not add colour to our samples as an augmentation because they do not naturally occur. We should not also flip the images in MNIST, because flipped images have different meanings for digits.\n",
        "\n",
        "### Augmentation 1: rotation\n",
        "\n",
        "Let us try rotating the image a little. We will use skimage library for this."
      ]
    },
    {
      "cell_type": "code",
      "metadata": {
        "id": "z5WolJ9fZE7L",
        "colab": {
          "base_uri": "https://localhost:8080/",
          "height": 860
        },
        "outputId": "a9ab497f-7742-4946-8426-a2f2cee4e1d7"
      },
      "source": [
        "plt.imshow(train_X[2], cmap='gray')\n",
        "plt.show()\n",
        "plt.imshow(rotate(train_X[2],25), cmap='gray')"
      ],
      "execution_count": 5,
      "outputs": [
        {
          "output_type": "display_data",
          "data": {
            "text/plain": [
              "<Figure size 640x480 with 1 Axes>"
            ],
            "image/png": "iVBORw0KGgoAAAANSUhEUgAAAaAAAAGdCAYAAABU0qcqAAAAOXRFWHRTb2Z0d2FyZQBNYXRwbG90bGliIHZlcnNpb24zLjcuMSwgaHR0cHM6Ly9tYXRwbG90bGliLm9yZy/bCgiHAAAACXBIWXMAAA9hAAAPYQGoP6dpAAAb+UlEQVR4nO3df2xV9f3H8dctP66g7cVa++NKwYI/2EBqROkaleFoKHUhVsnir2XI/DGwuCn+WOpUdJJ0w2wal06XzFDdRMVlgJqlC1RbMldwIAx/bA3tqi3pDyaGe0uxhdDP9w++3nmlBc7l3r572+cj+SS955z3PW8OJ/fVc+/p5/qcc04AAAyyFOsGAAAjEwEEADBBAAEATBBAAAATBBAAwAQBBAAwQQABAEwQQAAAE6OtG/i6vr4+tbW1KTU1VT6fz7odAIBHzjl1dXUpGAwqJWXg65whF0BtbW3Kzc21bgMAcJpaW1s1ceLEAdcPubfgUlNTrVsAAMTByV7PExZAlZWVOv/883XGGWeooKBA77333inV8bYbAAwPJ3s9T0gAvfbaa1qxYoVWrlyp999/X/n5+SouLta+ffsSsTsAQDJyCTB79mxXVlYWeXz06FEXDAZdRUXFSWtDoZCTxGAwGIwkH6FQ6ISv93G/Ajp8+LB27NihoqKiyLKUlBQVFRWpvr7+uO17e3sVDoejBgBg+It7AH322Wc6evSosrKyopZnZWWpo6PjuO0rKioUCAQigzvgAGBkML8Lrry8XKFQKDJaW1utWwIADIK4/x1QRkaGRo0apc7OzqjlnZ2dys7OPm57v98vv98f7zYAAENc3K+Axo4dq1mzZqmmpiayrK+vTzU1NSosLIz37gAASSohMyGsWLFCixcv1uWXX67Zs2frmWeeUXd3t5YsWZKI3QEAklBCAujGG2/Uf//7Xz322GPq6OjQpZdequrq6uNuTAAAjFw+55yzbuKrwuGwAoGAdRsAgNMUCoWUlpY24Hrzu+AAACMTAQQAMEEAAQBMEEAAABMEEADABAEEADBBAAEATBBAAAATBBAAwAQBBAAwQQABAEwQQAAAEwQQAMAEAQQAMEEAAQBMEEAAABMEEADABAEEADBBAAEATBBAAAATBBAAwAQBBAAwQQABAEwQQAAAEwQQAMAEAQQAMEEAAQBMEEAAABMEEADABAEEADBBAAEATBBAAAATBBAAwAQBBAAwQQABAEwQQAAAEwQQAMAEAQQAMEEAAQBMEEAAABMEEADABAEEADBBAAEATBBAAAATBBAAwAQBBAAwQQABAEwQQAAAEwQQAMAEAQQAMEEAAQBMEEAAABOjrRtA8vL7/Z5rfvjDHyagk+Olp6fHVPfkk0/GuZP+/epXv/JcU1VV5bnmo48+8lwDDBaugAAAJgggAICJuAfQ448/Lp/PFzWmTZsW790AAJJcQj4Dmj59ujZv3vy/nYzmoyYAQLSEJMPo0aOVnZ2diKcGAAwTCfkMaM+ePQoGg5oyZYpuvfVWtbS0DLhtb2+vwuFw1AAADH9xD6CCggJVVVWpurpazz33nJqbm3X11Verq6ur3+0rKioUCAQiIzc3N94tAQCGoLgHUElJib73ve9p5syZKi4u1l/+8hcdOHBA69at63f78vJyhUKhyGhtbY13SwCAISjhdwdMmDBBF110kRobG/td7/f7Y/qDRgBAckv43wEdPHhQTU1NysnJSfSuAABJJO4B9MADD6iurk6ffPKJ/v73v+v666/XqFGjdPPNN8d7VwCAJBb3t+D27t2rm2++Wfv379e5556rq666Slu3btW5554b710BAJKYzznnrJv4qnA4rEAgYN3GiHLZZZfFVPfiiy96rvnmN78Z074gtbW1ea6J9Z2HnTt3eq7p7u6OaV8YvkKhkNLS0gZcz1xwAAATBBAAwAQBBAAwQQABAEwQQAAAEwQQAMAEAQQAMEEAAQBMEEAAABMEEADABAEEADBBAAEATCT8C+kw9JWVlcVUx8SigysYDHqu2bJlS0z7evrppz3X3H///THtCyMXV0AAABMEEADABAEEADBBAAEATBBAAAATBBAAwAQBBAAwQQABAEwQQAAAEwQQAMAEAQQAMEEAAQBMEEAAABPMhg1dfvnl1i2cUGNjo+eaP/3pTwnopH/Tp0/3XLNw4cIEdBI/paWlnmteeOEFzzUff/yx5xoMH1wBAQBMEEAAABMEEADABAEEADBBAAEATBBAAAATBBAAwAQBBAAwQQABAEwQQAAAEwQQAMAEAQQAMMFkpBjyGhoaPNf87Gc/S0An/TvzzDM916xatcpzzXXXXee55vzzz/dcE2tdbW2t55r169d7rnnjjTc81/zjH//wXCNJ+/bti6kOp4YrIACACQIIAGCCAAIAmCCAAAAmCCAAgAkCCABgggACAJgggAAAJgggAIAJAggAYIIAAgCYIIAAACZ8zjln3cRXhcNhBQIB6zZGlCVLlsRU9+yzz3quGT9+vOeaWCaEXLhwoecaSdq+fXtMdV5lZ2d7rlm9erXnmu9///ueayRpiL0snLZY/19jOfcOHTrkuebhhx/2XNPU1OS5ZrCFQiGlpaUNuJ4rIACACQIIAGDCcwBt2bJFCxcuVDAYlM/n04YNG6LWO+f02GOPKScnR+PGjVNRUZH27NkTr34BAMOE5wDq7u5Wfn6+Kisr+12/evVqPfvss3r++ee1bds2nXnmmSouLlZPT89pNwsAGD48fyNqSUmJSkpK+l3nnNMzzzyjRx55JPLtjS+99JKysrK0YcMG3XTTTafXLQBg2IjrZ0DNzc3q6OhQUVFRZFkgEFBBQYHq6+v7rent7VU4HI4aAIDhL64B1NHRIUnKysqKWp6VlRVZ93UVFRUKBAKRkZubG8+WAABDlPldcOXl5QqFQpHR2tpq3RIAYBDENYC+/OO6zs7OqOWdnZ0D/uGd3+9XWlpa1AAADH9xDaC8vDxlZ2erpqYmsiwcDmvbtm0qLCyM564AAEnO811wBw8eVGNjY+Rxc3Ozdu3apfT0dE2aNEn33nuvVq1apQsvvFB5eXl69NFHFQwGVVpaGs++AQBJznMAbd++Xddcc03k8YoVKyRJixcvVlVVlR566CF1d3frrrvu0oEDB3TVVVepurpaZ5xxRvy6BgAkPSYjRcz+8Ic/eK655ZZbEtDJ8X7/+9/HVPejH/3Ic82ll17queaNN97wXHPeeed5rvH5fJ5rpOE3GWmsqqurPde0t7d7rrnjjjs81yQDJiMFAAxJBBAAwAQBBAAwQQABAEwQQAAAEwQQAMAEAQQAMEEAAQBMEEAAABMEEADABAEEADBBAAEATBBAAAATzIaNmKWnp3uu2bx5s+ea/Px8zzV9fX2ea6Rj33fl1ejRnr/VROPHj/dcE4uhPhv2Rx995Lnmn//8ZwI66d+6des812zatMlzTU9Pj+eaZMBs2ACAIYkAAgCYIIAAACYIIACACQIIAGCCAAIAmCCAAAAmCCAAgAkCCABgggACAJgggAAAJgggAIAJ77MoAv/v888/91xTWVk5KDVjxozxXCPphBMnJqMPPvggprpVq1Z5rmlra/Ncs3fvXs81n376qecaDE1cAQEATBBAAAATBBAAwAQBBAAwQQABAEwQQAAAEwQQAMAEAQQAMEEAAQBMEEAAABMEEADABAEEADDhc8456ya+KhwOKxAIWLeBIeSTTz7xXJObmxv/RpLQqFGjrFvACBYKhU44wS9XQAAAEwQQAMAEAQQAMEEAAQBMEEAAABMEEADABAEEADBBAAEATBBAAAATBBAAwAQBBAAwQQABAEyMtm4AOJmUFO+/J/l8vgR0AiCeuAICAJgggAAAJjwH0JYtW7Rw4UIFg0H5fD5t2LAhav1tt90mn88XNRYsWBCvfgEAw4TnAOru7lZ+fr4qKysH3GbBggVqb2+PjFdeeeW0mgQADD+eb0IoKSlRSUnJCbfx+/3Kzs6OuSkAwPCXkM+AamtrlZmZqYsvvljLli3T/v37B9y2t7dX4XA4agAAhr+4B9CCBQv00ksvqaamRr/85S9VV1enkpISHT16tN/tKyoqFAgEIiM3NzfeLQEAhiCfc87FXOzzaf369SotLR1wm//85z+aOnWqNm/erHnz5h23vre3V729vZHH4XCYEEKUlpYWzzUTJ05MQCfJJ5a/oQLiJRQKKS0tbcD1CT87p0yZooyMDDU2Nva73u/3Ky0tLWoAAIa/hAfQ3r17tX//fuXk5CR6VwCAJOL5LriDBw9GXc00Nzdr165dSk9PV3p6up544gktWrRI2dnZampq0kMPPaQLLrhAxcXFcW0cAJDcPAfQ9u3bdc0110Qer1ixQpK0ePFiPffcc9q9e7defPFFHThwQMFgUPPnz9eTTz4pv98fv64BAEnPcwDNnTtXJ7pv4a9//etpNYTkMXq097lsV65c6bkmMzPTc81p3FszrHzwwQcx1T344IOea6qrq2PaF0YubpEBAJgggAAAJgggAIAJAggAYIIAAgCYIIAAACYIIACACQIIAGCCAAIAmCCAAAAmCCAAgAkCCABgggACAJg4ra/kToRwOKxAIGDdBk5BLP9Pn3/+eQI6sXXHHXd4rrn77rs918yYMcNzTaxfg1JfX++55qtf03KqDh8+7LkGycP8K7kBAOgPAQQAMEEAAQBMEEAAABMEEADABAEEADBBAAEATBBAAAATBBAAwAQBBAAwQQABAEwQQAAAE0xGipgxGekxZ599tueacDjsuaaystJzzbJlyzzXSFIsLwsvvfSS55olS5Z4rkHyYDJSAMCQRAABAEwQQAAAEwQQAMAEAQQAMEEAAQBMEEAAABMEEADABAEEADBBAAEATBBAAAATBBAAwMRo6waAZLdq1SrPNT/+8Y891zzyyCOea2KdjDQWhw4dGrR9YXjgCggAYIIAAgCYIIAAACYIIACACQIIAGCCAAIAmCCAAAAmCCAAgAkCCABgggACAJgggAAAJgggAIAJn3POWTfxVeFwWIFAwLoNnIIxY8Z4rnnqqac819x9992ea0aNGuW5JlZtbW2ea37wgx94rmlqavJc88knn3iukaRYXhZCoZDnmvT0dM81SB6hUEhpaWkDrucKCABgggACAJjwFEAVFRW64oorlJqaqszMTJWWlqqhoSFqm56eHpWVlemcc87RWWedpUWLFqmzszOuTQMAkp+nAKqrq1NZWZm2bt2qTZs26ciRI5o/f766u7sj29x3331688039frrr6uurk5tbW264YYb4t44ACC5efpG1Orq6qjHVVVVyszM1I4dOzRnzhyFQiG98MILWrt2rb7zne9IktasWaNvfOMb2rp1q771rW/Fr3MAQFI7rc+Avrzr5cs7WXbs2KEjR46oqKgoss20adM0adIk1dfX9/scvb29CofDUQMAMPzFHEB9fX269957deWVV2rGjBmSpI6ODo0dO1YTJkyI2jYrK0sdHR39Pk9FRYUCgUBk5ObmxtoSACCJxBxAZWVl+vDDD/Xqq6+eVgPl5eUKhUKR0draelrPBwBIDp4+A/rS8uXL9dZbb2nLli2aOHFiZHl2drYOHz6sAwcORF0FdXZ2Kjs7u9/n8vv98vv9sbQBAEhinq6AnHNavny51q9fr7ffflt5eXlR62fNmqUxY8aopqYmsqyhoUEtLS0qLCyMT8cAgGHB0xVQWVmZ1q5dq40bNyo1NTXyuU4gENC4ceMUCAR0++23a8WKFUpPT1daWpruueceFRYWcgccACCKpwB67rnnJElz586NWr5mzRrddtttkqSnn35aKSkpWrRokXp7e1VcXKzf/va3cWkWADB8MBkphrxYJtTkbspjfD5fTHVMRop4YDJSAMCQRAABAEwQQAAAEwQQAMAEAQQAMEEAAQBMEEAAABMEEADABAEEADBBAAEATBBAAAATBBAAwAQBBAAwEdM3ogKD6f777/dcs27dugR0knxinQ373Xff9Vxz7bXXxrQvjFxcAQEATBBAAAATBBAAwAQBBAAwQQABAEwQQAAAEwQQAMAEAQQAMEEAAQBMEEAAABMEEADABAEEADDBZKQY8t555x3PNaWlpTHt68knn/RcM336dM81KSmD87ufcy6mutbWVs81XV1dMe0LIxdXQAAAEwQQAMAEAQQAMEEAAQBMEEAAABMEEADABAEEADBBAAEATBBAAAATBBAAwAQBBAAwQQABAEz4XKyzFSZIOBxWIBCwbgM4ZUuWLPFcU15e7rlm6tSpnmuWL1/uuUaSXnvtNc81n3/+eUz7wvAVCoWUlpY24HqugAAAJgggAIAJAggAYIIAAgCYIIAAACYIIACACQIIAGCCAAIAmCCAAAAmCCAAgAkCCABgggACAJhgMlIAQEIwGSkAYEgigAAAJjwFUEVFha644gqlpqYqMzNTpaWlamhoiNpm7ty58vl8UWPp0qVxbRoAkPw8BVBdXZ3Kysq0detWbdq0SUeOHNH8+fPV3d0dtd2dd96p9vb2yFi9enVcmwYAJL/RXjaurq6OelxVVaXMzEzt2LFDc+bMiSwfP368srOz49MhAGBYOq3PgEKhkCQpPT09avnLL7+sjIwMzZgxQ+Xl5Tp06NCAz9Hb26twOBw1AAAjgIvR0aNH3Xe/+1135ZVXRi3/3e9+56qrq93u3bvdH//4R3feeee566+/fsDnWblypZPEYDAYjGE2QqHQCXMk5gBaunSpmzx5smttbT3hdjU1NU6Sa2xs7Hd9T0+PC4VCkdHa2mp+0BgMBoNx+uNkAeTpM6AvLV++XG+99Za2bNmiiRMnnnDbgoICSVJjY6OmTp163Hq/3y+/3x9LGwCAJOYpgJxzuueee7R+/XrV1tYqLy/vpDW7du2SJOXk5MTUIABgePIUQGVlZVq7dq02btyo1NRUdXR0SJICgYDGjRunpqYmrV27Vtdee63OOecc7d69W/fdd5/mzJmjmTNnJuQfAABIUl4+99EA7/OtWbPGOedcS0uLmzNnjktPT3d+v99dcMEF7sEHHzzp+4BfFQqFzN+3ZDAYDMbpj5O99jMZKQAgIZiMFAAwJBFAAAATBBAAwAQBBAAwQQABAEwQQAAAEwQQAMAEAQQAMEEAAQBMEEAAABMEEADABAEEADBBAAEATBBAAAATBBAAwAQBBAAwQQABAEwQQAAAEwQQAMAEAQQAMEEAAQBMEEAAABMEEADABAEEADBBAAEATAy5AHLOWbcAAIiDk72eD7kA6urqsm4BABAHJ3s997khdsnR19entrY2paamyufzRa0Lh8PKzc1Va2ur0tLSjDq0x3E4huNwDMfhGI7DMUPhODjn1NXVpWAwqJSUga9zRg9iT6ckJSVFEydOPOE2aWlpI/oE+xLH4RiOwzEch2M4DsdYH4dAIHDSbYbcW3AAgJGBAAIAmEiqAPL7/Vq5cqX8fr91K6Y4DsdwHI7hOBzDcTgmmY7DkLsJAQAwMiTVFRAAYPgggAAAJgggAIAJAggAYCJpAqiyslLnn3++zjjjDBUUFOi9996zbmnQPf744/L5fFFj2rRp1m0l3JYtW7Rw4UIFg0H5fD5t2LAhar1zTo899phycnI0btw4FRUVac+ePTbNJtDJjsNtt9123PmxYMECm2YTpKKiQldccYVSU1OVmZmp0tJSNTQ0RG3T09OjsrIynXPOOTrrrLO0aNEidXZ2GnWcGKdyHObOnXvc+bB06VKjjvuXFAH02muvacWKFVq5cqXef/995efnq7i4WPv27bNubdBNnz5d7e3tkfG3v/3NuqWE6+7uVn5+viorK/tdv3r1aj377LN6/vnntW3bNp155pkqLi5WT0/PIHeaWCc7DpK0YMGCqPPjlVdeGcQOE6+urk5lZWXaunWrNm3apCNHjmj+/Pnq7u6ObHPffffpzTff1Ouvv666ujq1tbXphhtuMOw6/k7lOEjSnXfeGXU+rF692qjjAbgkMHv2bFdWVhZ5fPToURcMBl1FRYVhV4Nv5cqVLj8/37oNU5Lc+vXrI4/7+vpcdna2e+qppyLLDhw44Px+v3vllVcMOhwcXz8Ozjm3ePFid91115n0Y2Xfvn1Okqurq3POHfu/HzNmjHv99dcj2/zrX/9yklx9fb1Vmwn39ePgnHPf/va33U9+8hO7pk7BkL8COnz4sHbs2KGioqLIspSUFBUVFam+vt6wMxt79uxRMBjUlClTdOutt6qlpcW6JVPNzc3q6OiIOj8CgYAKCgpG5PlRW1urzMxMXXzxxVq2bJn2799v3VJChUIhSVJ6erokaceOHTpy5EjU+TBt2jRNmjRpWJ8PXz8OX3r55ZeVkZGhGTNmqLy8XIcOHbJob0BDbjLSr/vss8909OhRZWVlRS3PysrSv//9b6OubBQUFKiqqkoXX3yx2tvb9cQTT+jqq6/Whx9+qNTUVOv2THR0dEhSv+fHl+tGigULFuiGG25QXl6empqa9PDDD6ukpET19fUaNWqUdXtx19fXp3vvvVdXXnmlZsyYIenY+TB27FhNmDAhatvhfD70dxwk6ZZbbtHkyZMVDAa1e/du/fSnP1VDQ4P+/Oc/G3YbbcgHEP6npKQk8vPMmTNVUFCgyZMna926dbr99tsNO8NQcNNNN0V+vuSSSzRz5kxNnTpVtbW1mjdvnmFniVFWVqYPP/xwRHwOeiIDHYe77ror8vMll1yinJwczZs3T01NTZo6depgt9mvIf8WXEZGhkaNGnXcXSydnZ3Kzs426mpomDBhgi666CI1NjZat2Lmy3OA8+N4U6ZMUUZGxrA8P5YvX6633npL77zzTtTXt2RnZ+vw4cM6cOBA1PbD9XwY6Dj0p6CgQJKG1Pkw5ANo7NixmjVrlmpqaiLL+vr6VFNTo8LCQsPO7B08eFBNTU3KycmxbsVMXl6esrOzo86PcDisbdu2jfjzY+/evdq/f/+wOj+cc1q+fLnWr1+vt99+W3l5eVHrZ82apTFjxkSdDw0NDWppaRlW58PJjkN/du3aJUlD63ywvgviVLz66qvO7/e7qqoq9/HHH7u77rrLTZgwwXV0dFi3Nqjuv/9+V1tb65qbm927777rioqKXEZGhtu3b591awnV1dXldu7c6Xbu3OkkuV//+tdu586d7tNPP3XOOfeLX/zCTZgwwW3cuNHt3r3bXXfddS4vL8998cUXxp3H14mOQ1dXl3vggQdcfX29a25udps3b3aXXXaZu/DCC11PT49163GzbNkyFwgEXG1trWtvb4+MQ4cORbZZunSpmzRpknv77bfd9u3bXWFhoSssLDTsOv5OdhwaGxvdz3/+c7d9+3bX3NzsNm7c6KZMmeLmzJlj3Hm0pAgg55z7zW9+4yZNmuTGjh3rZs+e7bZu3Wrd0qC78cYbXU5Ojhs7dqw777zz3I033ugaGxut20q4d955x0k6bixevNg5d+xW7EcffdRlZWU5v9/v5s2b5xoaGmybToATHYdDhw65+fPnu3PPPdeNGTPGTZ482d15553D7pe0/v79ktyaNWsi23zxxRfu7rvvdmeffbYbP368u/766117e7td0wlwsuPQ0tLi5syZ49LT053f73cXXHCBe/DBB10oFLJt/Gv4OgYAgIkh/xkQAGB4IoAAACYIIACACQIIAGCCAAIAmCCAAAAmCCAAgAkCCABgggACAJgggAAAJgggAIAJAggAYOL/AC0/Gz+zK6mJAAAAAElFTkSuQmCC\n"
          },
          "metadata": {}
        },
        {
          "output_type": "execute_result",
          "data": {
            "text/plain": [
              "<matplotlib.image.AxesImage at 0x7e9516ea9a20>"
            ]
          },
          "metadata": {},
          "execution_count": 5
        },
        {
          "output_type": "display_data",
          "data": {
            "text/plain": [
              "<Figure size 640x480 with 1 Axes>"
            ],
            "image/png": "iVBORw0KGgoAAAANSUhEUgAAAaAAAAGdCAYAAABU0qcqAAAAOXRFWHRTb2Z0d2FyZQBNYXRwbG90bGliIHZlcnNpb24zLjcuMSwgaHR0cHM6Ly9tYXRwbG90bGliLm9yZy/bCgiHAAAACXBIWXMAAA9hAAAPYQGoP6dpAAAd0UlEQVR4nO3de3BU9f3/8dcGyAIx2TSE3ORiAAXKJa1UIkURJQNE6wgyHW/tYMfiYIMVEWmpVdR2Ji2dsY4ORZ12oE7FC22Baju0GCS0NeBwG6rVSDJRQEhQLLshQIjJ5/cHP/fryvWz7OadhOdj5jNjds8r++ZwzIuzuzkbcM45AQDQzlKsBwAAXJgoIACACQoIAGCCAgIAmKCAAAAmKCAAgAkKCABgggICAJjobj3Al7W1tWnfvn1KT09XIBCwHgcA4Mk5p8bGRhUUFCgl5fTnOR2ugPbt26f+/ftbjwEAOE979uxRv379Tnt/h3sKLj093XoEAEACnO3nedIKaMmSJbrkkkvUs2dPFRcX66233jqnHE+7AUDXcLaf50kpoJdfflnz5s3TokWLtG3bNhUVFWnKlCk6cOBAMh4OANAZuSQYO3asKysri37d2trqCgoKXHl5+Vmz4XDYSWKxWCxWJ1/hcPiMP+8TfgZ0/Phxbd26VSUlJdHbUlJSVFJSoqqqqpO2b25uViQSiVkAgK4v4QX0ySefqLW1Vbm5uTG35+bmqr6+/qTty8vLFQqFoot3wAHAhcH8XXALFy5UOByOrj179liPBABoBwn/PaDs7Gx169ZNDQ0NMbc3NDQoLy/vpO2DwaCCwWCixwAAdHAJPwNKTU3VmDFjVFFREb2tra1NFRUVGjduXKIfDgDQSSXlSgjz5s3TzJkz9Y1vfENjx47Vk08+qaamJn3ve99LxsMBADqhpBTQLbfcoo8//liPPPKI6uvr9bWvfU1r16496Y0JAIALV8A556yH+KJIJKJQKGQ9BgDgPIXDYWVkZJz2fvN3wQEALkwUEADABAUEADBBAQEATFBAAAATFBAAwAQFBAAwQQEBAExQQAAAExQQAMAEBQQAMEEBAQBMUEAAABMUEADABAUEADBBAQEATFBAAAATFBAAwAQFBAAwQQEBAExQQAAAExQQAMAEBQQAMEEBAQBMUEAAABMUEADABAUEADBBAQEATFBAAAATFBAAwAQFBAAwQQEBAExQQAAAExQQAMAEBQQAMEEBAQBMUEAAABMUEADABAUEADBBAQEATFBAAAATFBAAwAQFBAAwQQEBAExQQAAAExQQAMAEBQQAMEEBAQBMUEAAABMUEADABAUEADBBAQEATHS3HgA4m6KiIu9Ma2trXI/16aefemeOHj3qnTl8+LB3pqWlxTuD85OS4v9v9La2tiRM0jVxBgQAMEEBAQBMJLyAHn30UQUCgZg1bNiwRD8MAKCTS8prQCNGjNDrr7/+fw/SnZeaAACxktIM3bt3V15eXjK+NQCgi0jKa0C7du1SQUGBBg0apDvuuEO7d+8+7bbNzc2KRCIxCwDQ9SW8gIqLi7V8+XKtXbtWS5cuVV1dna6++mo1Njaecvvy8nKFQqHo6t+/f6JHAgB0QAHnnEvmAxw6dEgDBw7UE088obvuuuuk+5ubm9Xc3Bz9OhKJUEKIwe8BncDvAbU/fg/o/ITDYWVkZJz2/qS/OyAzM1OXXXaZampqTnl/MBhUMBhM9hgAgA4m6b8HdPjwYdXW1io/Pz/ZDwUA6EQSXkDz589XZWWlPvjgA7355puaPn26unXrpttuuy3RDwUA6MQS/hTc3r17ddttt+ngwYPq27evrrrqKm3atEl9+/ZN9EMBADqxpL8JwVckElEoFLIeA+dg+vTp3pmHHnrIO5Oamuqd+d///uedkaSmpibvzDvvvOOd2bdvn3fmt7/9rXcmnj+PJKWnp3tn4vl7iucNHNddd513Jt6fKX/5y1+8M+FwOK7H6orO9iYErgUHADBBAQEATFBAAAATFBAAwAQFBAAwQQEBAExQQAAAExQQAMAEBQQAMEEBAQBMUEAAABMUEADARNI/kA7xCwQC3plRo0Z5Zy6//HLvjKS4PmJjzJgx3pkOdr3ck5SWlnpn6urqvDMDBw70zsRzsU9J+uyzz7wzWVlZ3pkhQ4Z4Z774Ccrnav/+/d4ZSXr55ZfjyuHccAYEADBBAQEATFBAAAATFBAAwAQFBAAwQQEBAExQQAAAExQQAMAEBQQAMEEBAQBMUEAAABMUEADABAUEADDB1bA7sMzMTO/Mfffd55255pprvDOSNGjQIO9MPFe2bm1t9c4cPnzYOyNJoVDIO3P8+HHvTDxXti4rK/POHDt2zDsjSWlpaXHlfMVzZeu///3v3plXX33VOyPFd+zh3HEGBAAwQQEBAExQQAAAExQQAMAEBQQAMEEBAQBMUEAAABMUEADABAUEADBBAQEATFBAAAATFBAAwAQXI20n8VxY9Mc//rF35tvf/rZ35qKLLvLOSPFdSHLTpk3emXfffdc78/HHH3tnJGn48OHemSFDhnhnhg4d6p3p1auXd6a9LioqxbfPd+7c6Z355z//6Z3Ztm2bd0biYqTJxhkQAMAEBQQAMEEBAQBMUEAAABMUEADABAUEADBBAQEATFBAAAATFBAAwAQFBAAwQQEBAExQQAAAE1yMtJ2EQiHvTF5enncmnguLtrW1eWckac+ePd6Z5557zjuzbt0670w4HPbOSFJhYaF35oYbbvDOLFiwwDvTu3dv74xzzjsjSS0tLd6Z9evXe2eefvpp78ybb77pnUHHxBkQAMAEBQQAMOFdQBs3btSNN96ogoICBQIBrV69OuZ+55weeeQR5efnq1evXiopKdGuXbsSNS8AoIvwLqCmpiYVFRVpyZIlp7x/8eLFeuqpp/TMM89o8+bNSktL05QpU3Ts2LHzHhYA0HV4vwmhtLRUpaWlp7zPOacnn3xSP/3pT3XTTTdJkp5//nnl5uZq9erVuvXWW89vWgBAl5HQ14Dq6upUX1+vkpKS6G2hUEjFxcWqqqo6Zaa5uVmRSCRmAQC6voQWUH19vSQpNzc35vbc3NzofV9WXl6uUCgUXf3790/kSACADsr8XXALFy5UOByOrnh+twQA0PkktIA+/8XJhoaGmNsbGhpO+0uVwWBQGRkZMQsA0PUltIAKCwuVl5enioqK6G2RSESbN2/WuHHjEvlQAIBOzvtdcIcPH1ZNTU3067q6Ou3YsUNZWVkaMGCA5s6dq5///Oe69NJLVVhYqIcfflgFBQWaNm1aIucGAHRy3gW0ZcsWXXvttdGv582bJ0maOXOmli9frgULFqipqUl33323Dh06pKuuukpr165Vz549Ezc1AKDT8y6giRMnnvECh4FAQI8//rgef/zx8xqsq/nwww+9Mykp/s+QNjU1eWfS0tK8M1J8F6zMycnxznTv7n/N3NbWVu+MpLheg3zwwQe9M19+p2iyvPPOO3HlXnjhBe/MypUrvTO86ejCZv4uOADAhYkCAgCYoIAAACYoIACACQoIAGCCAgIAmKCAAAAmKCAAgAkKCABgggICAJiggAAAJiggAIAJCggAYML/MsNoN3/729+8MxMmTPDOxPtRGUOGDPHOzJ8/3zuTnp7unXnllVe8M5J0++23e2cyMzPjeixfn376qXfmT3/6U1yPFc/VsLmyNXxxBgQAMEEBAQBMUEAAABMUEADABAUEADBBAQEATFBAAAATFBAAwAQFBAAwQQEBAExQQAAAExQQAMBEwDnnrIf4okgkolAoZD1GhxAIBLwz3//+970z3/nOd7wzkjRixAjvTDwX7jxw4IB3Jt5jKCXF/99kqamp3pl4/m4bGxu9M9nZ2d4ZSWppaYkrB3xROBxWRkbGae/nDAgAYIICAgCYoIAAACYoIACACQoIAGCCAgIAmKCAAAAmKCAAgAkKCABgggICAJiggAAAJiggAIAJLkbaxfTu3ds789WvfjWux7ruuuu8M9/97ne9M8OHD/fOxHNR0Y4unguYzp8/P67H2rJli3emvr7eO/P+++97ZzrYjyycARcjBQB0SBQQAMAEBQQAMEEBAQBMUEAAABMUEADABAUEADBBAQEATFBAAAATFBAAwAQFBAAwQQEBAExwMVLELRgMemeuv/5678zs2bO9M9/85je9M5KUlpbmnYnnIqHt5ejRo3Hl4rmw6LZt27wzTz75pHempqbGOxPPnwfnj4uRAgA6JAoIAGDCu4A2btyoG2+8UQUFBQoEAlq9enXM/XfeeacCgUDMmjp1aqLmBQB0Ed4F1NTUpKKiIi1ZsuS020ydOlX79++PrhdffPG8hgQAdD3dfQOlpaUqLS094zbBYFB5eXlxDwUA6PqS8hrQhg0blJOTo6FDh+qee+7RwYMHT7ttc3OzIpFIzAIAdH0JL6CpU6fq+eefV0VFhX75y1+qsrJSpaWlam1tPeX25eXlCoVC0dW/f/9EjwQA6IC8n4I7m1tvvTX636NGjdLo0aM1ePBgbdiwQZMmTTpp+4ULF2revHnRryORCCUEABeApL8Ne9CgQcrOzj7tL48Fg0FlZGTELABA15f0Atq7d68OHjyo/Pz8ZD8UAKAT8X4K7vDhwzFnM3V1ddqxY4eysrKUlZWlxx57TDNmzFBeXp5qa2u1YMECDRkyRFOmTEno4ACAzs27gLZs2aJrr702+vXnr9/MnDlTS5cu1c6dO/X73/9ehw4dUkFBgSZPnqyf/exncV03DADQdXExUrSr7t393/dy5ZVXemf++Mc/emckKTc3N66cr3j+t9u7d693Jj093Tsjxff3lJLi/4z+Bx984J156qmnvDPPPvusdwbnj4uRAgA6JAoIAGCCAgIAmKCAAAAmKCAAgAkKCABgggICAJiggAAAJiggAIAJCggAYIICAgCYoIAAACYoIACAiYR/JDdwJp999pl3prq62jvz4YcfemckqW/fvt6ZQCDgnXnjjTe8MytWrPDOHDlyxDsjKeYjV87VrFmzvDPDhw/3ztx+++3emcrKSu+MJL333ntx5XBuOAMCAJiggAAAJiggAIAJCggAYIICAgCYoIAAACYoIACACQoIAGCCAgIAmKCAAAAmKCAAgAkKCABggouRol3Fc+HOESNGeGfS09O9M1J888Vj7Nix3pkFCxZ4Z95//33vjCT99a9/9c7k5+d7Z771rW95Z+LZd7Nnz/bOSNLcuXPjyuHccAYEADBBAQEATFBAAAATFBAAwAQFBAAwQQEBAExQQAAAExQQAMAEBQQAMEEBAQBMUEAAABMUEADABBcjRYe3fft270yfPn2SMEnipKWleWeWLl3qnVm1apV3RpL+8Y9/eGc++uijuB7L15EjR7wz27Zti+ux4rk4rXMurse6EHEGBAAwQQEBAExQQAAAExQQAMAEBQQAMEEBAQBMUEAAABMUEADABAUEADBBAQEATFBAAAATFBAAwAQXI0W7iudCjc3Nzd6Z5557zjsjSQ899FBcufbw9a9/3TszZMiQuB7rhz/8oXcmNzc3rsfydeDAAe9MPBcwlbiwaLJxBgQAMEEBAQBMeBVQeXm5rrjiCqWnpysnJ0fTpk1TdXV1zDbHjh1TWVmZ+vTpo4suukgzZsxQQ0NDQocGAHR+XgVUWVmpsrIybdq0SevWrVNLS4smT56spqam6Db333+/Xn31Va1cuVKVlZXat2+fbr755oQPDgDo3LzehLB27dqYr5cvX66cnBxt3bpVEyZMUDgc1u9+9zutWLFC1113nSRp2bJlGj58uDZt2qQrr7wycZMDADq183oNKBwOS5KysrIkSVu3blVLS4tKSkqi2wwbNkwDBgxQVVXVKb9Hc3OzIpFIzAIAdH1xF1BbW5vmzp2r8ePHa+TIkZKk+vp6paamKjMzM2bb3Nxc1dfXn/L7lJeXKxQKRVf//v3jHQkA0InEXUBlZWV6++239dJLL53XAAsXLlQ4HI6uPXv2nNf3AwB0DnH9IuqcOXP02muvaePGjerXr1/09ry8PB0/flyHDh2KOQtqaGhQXl7eKb9XMBhUMBiMZwwAQCfmdQbknNOcOXO0atUqrV+/XoWFhTH3jxkzRj169FBFRUX0turqau3evVvjxo1LzMQAgC7B6wyorKxMK1as0Jo1a5Senh59XScUCqlXr14KhUK66667NG/ePGVlZSkjI0P33nuvxo0bxzvgAAAxvApo6dKlkqSJEyfG3L5s2TLdeeedkqRf//rXSklJ0YwZM9Tc3KwpU6boN7/5TUKGBQB0HV4FdC4X5uvZs6eWLFmiJUuWxD0U8EXHjx/3znz+jyVfo0aN8s6MHz/eO/Pld4qei27durXL43R0GRkZ3plAIJCESXC+uBYcAMAEBQQAMEEBAQBMUEAAABMUEADABAUEADBBAQEATFBAAAATFBAAwAQFBAAwQQEBAExQQAAAExQQAMBEXJ+ICrSntrY278znn1Xl64EHHvDOlJSUeGdmzpzpnYnnM7Wampq8M5LUu3dv70xra6t3JiXF/9/AL730knfmzTff9M5I8V1F+1w+NQAncAYEADBBAQEATFBAAAATFBAAwAQFBAAwQQEBAExQQAAAExQQAMAEBQQAMEEBAQBMUEAAABMUEADABBcjRZcUzwVMJam2ttY788EHH3hn/vOf/3hnxowZ45257LLLvDOSdPjwYe9Mv379vDO7d+/2zixdutQ789FHH3lnkHycAQEATFBAAAATFBAAwAQFBAAwQQEBAExQQAAAExQQAMAEBQQAMEEBAQBMUEAAABMUEADABAUEADARcM456yG+KBKJKBQKWY8BJFUwGPTO9OzZ0zsTDoe9M5KUlpbmnYnnArBHjx71zqDzCIfDysjIOO39nAEBAExQQAAAExQQAMAEBQQAMEEBAQBMUEAAABMUEADABAUEADBBAQEATFBAAAATFBAAwAQFBAAw0d16AOBC1Nzc3C6ZeDU1NbXbY+HCxRkQAMAEBQQAMOFVQOXl5briiiuUnp6unJwcTZs2TdXV1THbTJw4UYFAIGbNnj07oUMDADo/rwKqrKxUWVmZNm3apHXr1qmlpUWTJ08+6fniWbNmaf/+/dG1ePHihA4NAOj8vN6EsHbt2pivly9frpycHG3dulUTJkyI3t67d2/l5eUlZkIAQJd0Xq8Bff5xv1lZWTG3v/DCC8rOztbIkSO1cOFCHTly5LTfo7m5WZFIJGYBAC4ALk6tra3uhhtucOPHj4+5/dlnn3Vr1651O3fudH/4wx/cxRdf7KZPn37a77No0SInicVisVhdbIXD4TP2SNwFNHv2bDdw4EC3Z8+eM25XUVHhJLmamppT3n/s2DEXDoeja8+ePeY7jcVisVjnv85WQHH9IuqcOXP02muvaePGjerXr98Zty0uLpYk1dTUaPDgwSfdHwwGFQwG4xkDANCJeRWQc0733nuvVq1apQ0bNqiwsPCsmR07dkiS8vPz4xoQANA1eRVQWVmZVqxYoTVr1ig9PV319fWSpFAopF69eqm2tlYrVqzQ9ddfrz59+mjnzp26//77NWHCBI0ePTopfwAAQCfl87qPTvM837Jly5xzzu3evdtNmDDBZWVluWAw6IYMGeIefPDBsz4P+EXhcNj8eUsWi8Vinf8628/+wP8vlg4jEokoFApZjwEAOE/hcFgZGRmnvZ9rwQEATFBAAAATFBAAwAQFBAAwQQEBAExQQAAAExQQAMAEBQQAMEEBAQBMUEAAABMUEADABAUEADBBAQEATFBAAAATFBAAwAQFBAAwQQEBAExQQAAAExQQAMAEBQQAMEEBAQBMUEAAABMUEADABAUEADBBAQEATHS4AnLOWY8AAEiAs/0873AF1NjYaD0CACABzvbzPOA62ClHW1ub9u3bp/T0dAUCgZj7IpGI+vfvrz179igjI8NoQnvshxPYDyewH05gP5zQEfaDc06NjY0qKChQSsrpz3O6t+NM5yQlJUX9+vU74zYZGRkX9AH2OfbDCeyHE9gPJ7AfTrDeD6FQ6KzbdLin4AAAFwYKCABgolMVUDAY1KJFixQMBq1HMcV+OIH9cAL74QT2wwmdaT90uDchAAAuDJ3qDAgA0HVQQAAAExQQAMAEBQQAMNFpCmjJkiW65JJL1LNnTxUXF+utt96yHqndPfroowoEAjFr2LBh1mMl3caNG3XjjTeqoKBAgUBAq1evjrnfOadHHnlE+fn56tWrl0pKSrRr1y6bYZPobPvhzjvvPOn4mDp1qs2wSVJeXq4rrrhC6enpysnJ0bRp01RdXR2zzbFjx1RWVqY+ffrooosu0owZM9TQ0GA0cXKcy36YOHHiScfD7NmzjSY+tU5RQC+//LLmzZunRYsWadu2bSoqKtKUKVN04MAB69Ha3YgRI7R///7o+te//mU9UtI1NTWpqKhIS5YsOeX9ixcv1lNPPaVnnnlGmzdvVlpamqZMmaJjx46186TJdbb9IElTp06NOT5efPHFdpww+SorK1VWVqZNmzZp3bp1amlp0eTJk9XU1BTd5v7779err76qlStXqrKyUvv27dPNN99sOHXinct+kKRZs2bFHA+LFy82mvg0XCcwduxYV1ZWFv26tbXVFRQUuPLycsOp2t+iRYtcUVGR9RimJLlVq1ZFv25ra3N5eXnuV7/6VfS2Q4cOuWAw6F588UWDCdvHl/eDc87NnDnT3XTTTSbzWDlw4ICT5CorK51zJ/7ue/To4VauXBnd5t1333WSXFVVldWYSffl/eCcc9dcc42777777IY6Bx3+DOj48ePaunWrSkpKorelpKSopKREVVVVhpPZ2LVrlwoKCjRo0CDdcccd2r17t/VIpurq6lRfXx9zfIRCIRUXF1+Qx8eGDRuUk5OjoUOH6p577tHBgwetR0qqcDgsScrKypIkbd26VS0tLTHHw7BhwzRgwIAufTx8eT987oUXXlB2drZGjhyphQsX6siRIxbjnVaHuxjpl33yySdqbW1Vbm5uzO25ubl67733jKayUVxcrOXLl2vo0KHav3+/HnvsMV199dV6++23lZ6ebj2eifr6ekk65fHx+X0XiqlTp+rmm29WYWGhamtr9ZOf/ESlpaWqqqpSt27drMdLuLa2Ns2dO1fjx4/XyJEjJZ04HlJTU5WZmRmzbVc+Hk61HyTp9ttv18CBA1VQUKCdO3fqRz/6kaqrq/XnP//ZcNpYHb6A8H9KS0uj/z169GgVFxdr4MCBeuWVV3TXXXcZToaO4NZbb43+96hRozR69GgNHjxYGzZs0KRJkwwnS46ysjK9/fbbF8TroGdyuv1w9913R/971KhRys/P16RJk1RbW6vBgwe395in1OGfgsvOzla3bt1OehdLQ0OD8vLyjKbqGDIzM3XZZZeppqbGehQznx8DHB8nGzRokLKzs7vk8TFnzhy99tpreuONN2I+viUvL0/Hjx/XoUOHYrbvqsfD6fbDqRQXF0tShzoeOnwBpaamasyYMaqoqIje1tbWpoqKCo0bN85wMnuHDx9WbW2t8vPzrUcxU1hYqLy8vJjjIxKJaPPmzRf88bF3714dPHiwSx0fzjnNmTNHq1at0vr161VYWBhz/5gxY9SjR4+Y46G6ulq7d+/uUsfD2fbDqezYsUOSOtbxYP0uiHPx0ksvuWAw6JYvX+7++9//urvvvttlZma6+vp669Ha1QMPPOA2bNjg6urq3L///W9XUlLisrOz3YEDB6xHS6rGxka3fft2t337difJPfHEE2779u3uww8/dM4594tf/MJlZma6NWvWuJ07d7qbbrrJFRYWuqNHjxpPnlhn2g+NjY1u/vz5rqqqytXV1bnXX3/dXX755e7SSy91x44dsx49Ye655x4XCoXchg0b3P79+6PryJEj0W1mz57tBgwY4NavX++2bNnixo0b58aNG2c4deKdbT/U1NS4xx9/3G3ZssXV1dW5NWvWuEGDBrkJEyYYTx6rUxSQc849/fTTbsCAAS41NdWNHTvWbdq0yXqkdnfLLbe4/Px8l5qa6i6++GJ3yy23uJqaGuuxku6NN95wkk5aM2fOdM6deCv2ww8/7HJzc10wGHSTJk1y1dXVtkMnwZn2w5EjR9zkyZNd3759XY8ePdzAgQPdrFmzutw/0k7155fkli1bFt3m6NGj7gc/+IH7yle+4nr37u2mT5/u9u/fbzd0EpxtP+zevdtNmDDBZWVluWAw6IYMGeIefPBBFw6HbQf/Ej6OAQBgosO/BgQA6JooIACACQoIAGCCAgIAmKCAAAAmKCAAgAkKCABgggICAJiggAAAJiggAIAJCggAYIICAgCY+H/DSIQ17Yr0ZQAAAABJRU5ErkJggg==\n"
          },
          "metadata": {}
        }
      ]
    },
    {
      "cell_type": "markdown",
      "metadata": {
        "id": "KE33Yxgggu0c"
      },
      "source": [
        "After rotating, the the class of the image is still the same. Let us make a function to rotate multiple images by random angles. We want a slightly different image every time we run this function. So, we generate a random number between 0 and 1 and change it so that it lies between -constraint/2 and +constraint/2"
      ]
    },
    {
      "cell_type": "code",
      "metadata": {
        "id": "vyM7pUV7Reze"
      },
      "source": [
        "def augRotate(sample, angleconstraint):\n",
        "  if angleconstraint==0:\n",
        "    return sample\n",
        "  if len(sample.shape)==2:\n",
        "    sample = np.expand_dims(sample, 0)  # make sure the sample is 3 dimensional\n",
        "  angle = rng.random(len(sample)) # generate random numbers for angles\n",
        "  angle = (angle-0.5)*angleconstraint # make the random angle constrained\n",
        "  nsample = sample.copy() # preallocate the augmented array to make it faster\n",
        "  for ii in range(len(sample)):\n",
        "    nsample[ii] = rotate(sample[ii], angle[ii])\n",
        "  return np.squeeze(nsample) # take care if the input had only one sample."
      ],
      "execution_count": 6,
      "outputs": []
    },
    {
      "cell_type": "markdown",
      "metadata": {
        "id": "kDk-N5VNjar9"
      },
      "source": [
        "This function returns a slightly different image each time we call it. So we can increase the number of images in the sample by any multiple."
      ]
    },
    {
      "cell_type": "code",
      "metadata": {
        "id": "Vw3O9zGFgI8K",
        "colab": {
          "base_uri": "https://localhost:8080/",
          "height": 639
        },
        "outputId": "38e65a7d-6f25-417c-def1-dd9966cb2d99"
      },
      "source": [
        "sample = train_X[20]\n",
        "angleconstraint = 70\n",
        "# show the original image\n",
        "plt.imshow(sample, cmap='gray')\n",
        "plt.show()\n",
        "plt.subplot(1,3,1)\n",
        "plt.imshow(augRotate(sample, angleconstraint), cmap='gray') # show an augmented image\n",
        "plt.subplot(1,3,2)\n",
        "plt.imshow(augRotate(sample, angleconstraint), cmap='gray') # show another augmented image from the same sample\n",
        "plt.subplot(1,3,3)\n",
        "plt.imshow(augRotate(sample, angleconstraint), cmap='gray') # one more image from the same sample"
      ],
      "execution_count": 7,
      "outputs": [
        {
          "output_type": "display_data",
          "data": {
            "text/plain": [
              "<Figure size 640x480 with 1 Axes>"
            ],
            "image/png": "iVBORw0KGgoAAAANSUhEUgAAAaAAAAGdCAYAAABU0qcqAAAAOXRFWHRTb2Z0d2FyZQBNYXRwbG90bGliIHZlcnNpb24zLjcuMSwgaHR0cHM6Ly9tYXRwbG90bGliLm9yZy/bCgiHAAAACXBIWXMAAA9hAAAPYQGoP6dpAAAblElEQVR4nO3df2yV9fn/8dcp0kPV9nSltqeVHxZQWURYxqQ2aEXpKNUQUebAOYOL08CKGzB16TJAN5duLNkMruL+MHRu4g82gcm2LlBsyWaLoUKI2dbQrht10IIsnAPFFkLf3z/4ej4eKT/u03N6nbbPR/JOeu77vnpf3t6cV+9z7vM+PuecEwAAAyzFugEAwPBEAAEATBBAAAATBBAAwAQBBAAwQQABAEwQQAAAEwQQAMDEFdYNfFZvb68OHTqk9PR0+Xw+63YAAB4553TixAnl5+crJeXC1zlJF0CHDh3S2LFjrdsAAPRTe3u7xowZc8H1SfcSXHp6unULAIA4uNTzecICqKqqStddd51GjRqlwsJCvffee5dVx8tuADA0XOr5PCEB9MYbb2jlypVas2aN3n//fU2bNk2lpaU6cuRIInYHABiMXALMmDHDlZeXRx6fPXvW5efnu8rKykvWhkIhJ4nBYDAYg3yEQqGLPt/H/Qro9OnTampqUklJSWRZSkqKSkpK1NDQcN72PT09CofDUQMAMPTFPYA++ugjnT17Vrm5uVHLc3Nz1dHRcd72lZWVCgQCkcEdcAAwPJjfBVdRUaFQKBQZ7e3t1i0BAAZA3D8HlJ2drREjRqizszNqeWdnp4LB4Hnb+/1++f3+eLcBAEhycb8CSk1N1fTp01VbWxtZ1tvbq9raWhUVFcV7dwCAQSohMyGsXLlSixcv1pe+9CXNmDFDzz//vLq6uvSNb3wjEbsDAAxCCQmghQsX6ujRo1q9erU6Ojr0hS98QTU1NefdmAAAGL58zjln3cSnhcNhBQIB6zYAAP0UCoWUkZFxwfXmd8EBAIYnAggAYIIAAgCYIIAAACYIIACACQIIAGCCAAIAmCCAAAAmCCAAgAkCCABgggACAJgggAAAJgggAIAJAggAYIIAAgCYIIAAACYIIACACQIIAGCCAAIAmCCAAAAmCCAAgAkCCABgggACAJgggAAAJgggAIAJAggAYIIAAgCYIIAAACYIIACACQIIAGCCAAIAmCCAAAAmCCAAgAkCCABgggACAJgggAAAJgggAIAJAggAYIIAAgCYuMK6AWCwKy8v91yzbt06zzUpKd7/Xuzt7fVcI0kbNmzwXNPU1OS55je/+Y3nmpMnT3quQXLiCggAYIIAAgCYIIAAACYIIACACQIIAGCCAAIAmCCAAAAmCCAAgAkCCABgggACAJgggAAAJgggAIAJn3POWTfxaeFwWIFAwLoNDFPf/va3Pdc899xznmuuvPJKzzU+n89zTZL98z7PAw884Lmmvr7ec83//vc/zzXov1AopIyMjAuu5woIAGCCAAIAmIh7AD3zzDPy+XxRY/LkyfHeDQBgkEvIF9LddNNN2rFjx//t5Aq+9w4AEC0hyXDFFVcoGAwm4lcDAIaIhLwHdODAAeXn52vChAl66KGHdPDgwQtu29PTo3A4HDUAAENf3AOosLBQ1dXVqqmp0fr169XW1qbbb79dJ06c6HP7yspKBQKByBg7dmy8WwIAJKG4B1BZWZkeeOABTZ06VaWlpfrTn/6k48eP68033+xz+4qKCoVCochob2+Pd0sAgCSU8LsDMjMzdcMNN6ilpaXP9X6/X36/P9FtAACSTMI/B3Ty5Em1trYqLy8v0bsCAAwicQ+gJ598UvX19fr3v/+td999V/fdd59GjBihBx98MN67AgAMYnF/Ce7DDz/Ugw8+qGPHjumaa67RbbfdpsbGRl1zzTXx3hUAYBBjMlIkvZQU7xfqDz/8cEz7eumllzzXjBw5MqZ9eRXLZKSxfqyht7c3pjqv0tPTPdds2bLFc81Xv/pVzzXoPyYjBQAkJQIIAGCCAAIAmCCAAAAmCCAAgAkCCABgggACAJgggAAAJgggAIAJAggAYIIAAgCYIIAAACYS/oV0QH89+uijnmvWr1+fgE76duTIEc81L7/8sueaWCZlXbdunecaSers7Iypzqu2tjbPNXfddZfnmltvvdVzjSQ1NjbGVIfLwxUQAMAEAQQAMEEAAQBMEEAAABMEEADABAEEADBBAAEATBBAAAATBBAAwAQBBAAwQQABAEwQQAAAEwQQAMAEs2Ej6c2bN8+6hYuKZeboVatWJaCT4SEzM9NzTXFxcUz7YjbsxOIKCABgggACAJgggAAAJgggAIAJAggAYIIAAgCYIIAAACYIIACACQIIAGCCAAIAmCCAAAAmCCAAgAkmIwUQFw888IDnmuzsbM81R44c8Vzz+9//3nMNEo8rIACACQIIAGCCAAIAmCCAAAAmCCAAgAkCCABgggACAJgggAAAJgggAIAJAggAYIIAAgCYIIAAACaYjBRJ7w9/+IPnmrvvvjsBnfQtGAx6riksLPRcs3v3bs81sRo1apTnmiVLlniuSUtL81xz9OhRzzWtra2ea5B4XAEBAEwQQAAAE54DaNeuXZo3b57y8/Pl8/m0ZcuWqPXOOa1evVp5eXlKS0tTSUmJDhw4EK9+AQBDhOcA6urq0rRp01RVVdXn+rVr12rdunV66aWXtHv3bl111VUqLS1Vd3d3v5sFAAwdnm9CKCsrU1lZWZ/rnHN6/vnn9YMf/ED33nuvJOmVV15Rbm6utmzZokWLFvWvWwDAkBHX94Da2trU0dGhkpKSyLJAIKDCwkI1NDT0WdPT06NwOBw1AABDX1wDqKOjQ5KUm5sbtTw3Nzey7rMqKysVCAQiY+zYsfFsCQCQpMzvgquoqFAoFIqM9vZ265YAAAMgrgH0yQfyOjs7o5Z3dnZe8MN6fr9fGRkZUQMAMPTFNYAKCgoUDAZVW1sbWRYOh7V7924VFRXFc1cAgEHO811wJ0+eVEtLS+RxW1ub9u3bp6ysLI0bN07Lly/Xc889p+uvv14FBQVatWqV8vPzNX/+/Hj2DQAY5DwH0J49e3TnnXdGHq9cuVKStHjxYlVXV+vpp59WV1eXHn/8cR0/fly33XabampqYppbCgAwdPmcc866iU8Lh8MKBALWbSCJpKR4f6V47dq1Me1r+fLlMdV59fHHH3uu+fQffpdrz549nmskRX2U4nLV1NR4rvnvf//ruWbhwoWeaxobGz3XoP9CodBF39c3vwsOADA8EUAAABMEEADABAEEADBBAAEATBBAAAATBBAAwAQBBAAwQQABAEwQQAAAEwQQAMAEAQQAMEEAAQBMMBs2hqTx48fHVBfLjM7XX399TPvyatOmTZ5r3nzzzZj29eKLL3quieWp5Ctf+YrnmnfffddzDWwwGzYAICkRQAAAEwQQAMAEAQQAMEEAAQBMEEAAABMEEADABAEEADBBAAEATBBAAAATBBAAwAQBBAAwwWSkwKdMmjTJc80f//hHzzUTJ070XOPz+TzXDOQ/78rKSs81q1atSkAnSBZMRgoASEoEEADABAEEADBBAAEATBBAAAATBBAAwAQBBAAwQQABAEwQQAAAEwQQAMAEAQQAMEEAAQBMMBkp0E9Tp071XLN3717PNSkp3v9e7O3t9VwjSV//+tc917z22msx7QtDF5ORAgCSEgEEADBBAAEATBBAAAATBBAAwAQBBAAwQQABAEwQQAAAEwQQAMAEAQQAMEEAAQBMEEAAABNXWDcADHYTJ070XBPLHMCxTCwa61zDSTZHMYYoroAAACYIIACACc8BtGvXLs2bN0/5+fny+XzasmVL1PpHHnlEPp8vasydOzde/QIAhgjPAdTV1aVp06apqqrqgtvMnTtXhw8fjgy+qAoA8Fmeb0IoKytTWVnZRbfx+/0KBoMxNwUAGPoS8h5QXV2dcnJydOONN2rp0qU6duzYBbft6elROByOGgCAoS/uATR37ly98sorqq2t1U9/+lPV19errKxMZ8+e7XP7yspKBQKByBg7dmy8WwIAJKG4fw5o0aJFkZ9vvvlmTZ06VRMnTlRdXZ1mz5593vYVFRVauXJl5HE4HCaEAGAYSPht2BMmTFB2drZaWlr6XO/3+5WRkRE1AABDX8ID6MMPP9SxY8eUl5eX6F0BAAYRzy/BnTx5Mupqpq2tTfv27VNWVpaysrL07LPPasGCBQoGg2ptbdXTTz+tSZMmqbS0NK6NAwAGN88BtGfPHt15552Rx5+8f7N48WKtX79e+/fv169//WsdP35c+fn5mjNnjn70ox/J7/fHr2sAwKDnOYBmzZp10YkK//KXv/SrISAeRo0aFVPdL3/5S88199xzT0z78urUqVOea9LS0mLaV6x1gBfMBQcAMEEAAQBMEEAAABMEEADABAEEADBBAAEATBBAAAATBBAAwAQBBAAwQQABAEwQQAAAEwQQAMAEAQQAMBH3r+QG4i0QCHiueeutt2La1x133BFTnVexzBq/bds2zzUvvPCC5xpJWrduneeaf/3rX55r6uvrPddg6OAKCABgggACAJgggAAAJgggAIAJAggAYIIAAgCYIIAAACYIIACACQIIAGCCAAIAmCCAAAAmCCAAgAkmI8WAys3N9VxTXV3tuWagJhWVpN/97neea775zW96rnn44Yc918QqLS3Nc01JSYnnGiYjHd64AgIAmCCAAAAmCCAAgAkCCABgggACAJgggAAAJgggAIAJAggAYIIAAgCYIIAAACYIIACACQIIAGCCyUgRs0Ag4LkmlolFv/zlL3uuidVbb73luWbp0qWea06ePOm5BhhquAICAJgggAAAJgggAIAJAggAYIIAAgCYIIAAACYIIACACQIIAGCCAAIAmCCAAAAmCCAAgAkCCABggslIEbNYJuEcqIlF//znP8dU9/DDD3uu6enpiWlfwHDHFRAAwAQBBAAw4SmAKisrdcsttyg9PV05OTmaP3++mpubo7bp7u5WeXm5Ro8erauvvloLFixQZ2dnXJsGAAx+ngKovr5e5eXlamxs1Pbt23XmzBnNmTNHXV1dkW1WrFiht99+W5s2bVJ9fb0OHTqk+++/P+6NAwAGN083IdTU1EQ9rq6uVk5OjpqamlRcXKxQKKSXX35ZGzdu1F133SVJ2rBhgz7/+c+rsbFRt956a/w6BwAMav16DygUCkmSsrKyJElNTU06c+aMSkpKIttMnjxZ48aNU0NDQ5+/o6enR+FwOGoAAIa+mAOot7dXy5cv18yZMzVlyhRJUkdHh1JTU5WZmRm1bW5urjo6Ovr8PZWVlQoEApExduzYWFsCAAwiMQdQeXm5PvjgA73++uv9aqCiokKhUCgy2tvb+/X7AACDQ0wfRF22bJm2bdumXbt2acyYMZHlwWBQp0+f1vHjx6Ougjo7OxUMBvv8XX6/X36/P5Y2AACDmKcrIOecli1bps2bN2vnzp0qKCiIWj99+nSNHDlStbW1kWXNzc06ePCgioqK4tMxAGBI8HQFVF5ero0bN2rr1q1KT0+PvK8TCASUlpamQCCgRx99VCtXrlRWVpYyMjL0xBNPqKioiDvgAABRPAXQ+vXrJUmzZs2KWr5hwwY98sgjkqRf/OIXSklJ0YIFC9TT06PS0lK9+OKLcWkWADB0eAog59wltxk1apSqqqpUVVUVc1NAf8V6O/9AfWh60aJFnms+/fGGRNuxY4fnmh//+McJ6ARDGXPBAQBMEEAAABMEEADABAEEADBBAAEATBBAAAATBBAAwAQBBAAwQQABAEwQQAAAEwQQAMAEAQQAMEEAAQBMxPSNqIAkHT161HPN6dOnPdekpqZ6rlm4cKHnmv7UDQSfz+e55nJmsO9LdXW155ru7u6Y9oXhiysgAIAJAggAYIIAAgCYIIAAACYIIACACQIIAGCCAAIAmCCAAAAmCCAAgAkCCABgggACAJgggAAAJpiMFDF7+eWXPddMmjTJc81TTz3luWYo2rVrl+ea119/PaZ97dmzJ6Y6wAuugAAAJgggAIAJAggAYIIAAgCYIIAAACYIIACACQIIAGCCAAIAmCCAAAAmCCAAgAkCCABgggACAJjwOeecdROfFg6HFQgErNsAAPRTKBRSRkbGBddzBQQAMEEAAQBMEEAAABMEEADABAEEADBBAAEATBBAAAATBBAAwAQBBAAwQQABAEwQQAAAEwQQAMAEAQQAMEEAAQBMEEAAABOeAqiyslK33HKL0tPTlZOTo/nz56u5uTlqm1mzZsnn80WNJUuWxLVpAMDg5ymA6uvrVV5ersbGRm3fvl1nzpzRnDlz1NXVFbXdY489psOHD0fG2rVr49o0AGDwu8LLxjU1NVGPq6urlZOTo6amJhUXF0eWX3nllQoGg/HpEAAwJPXrPaBQKCRJysrKilr+6quvKjs7W1OmTFFFRYVOnTp1wd/R09OjcDgcNQAAw4CL0dmzZ90999zjZs6cGbX8V7/6laupqXH79+93v/3tb921117r7rvvvgv+njVr1jhJDAaDwRhiIxQKXTRHYg6gJUuWuPHjx7v29vaLbldbW+skuZaWlj7Xd3d3u1AoFBnt7e3mB43BYDAY/R+XCiBP7wF9YtmyZdq2bZt27dqlMWPGXHTbwsJCSVJLS4smTpx43nq/3y+/3x9LGwCAQcxTADnn9MQTT2jz5s2qq6tTQUHBJWv27dsnScrLy4upQQDA0OQpgMrLy7Vx40Zt3bpV6enp6ujokCQFAgGlpaWptbVVGzdu1N13363Ro0dr//79WrFihYqLizV16tSE/AcAAAYpL+/76AKv823YsME559zBgwddcXGxy8rKcn6/302aNMk99dRTl3wd8NNCoZD565YMBoPB6P+41HO/7/8HS9IIh8MKBALWbQAA+ikUCikjI+OC65kLDgBgggACAJgggAAAJgggAIAJAggAYIIAAgCYIIAAACYIIACACQIIAGCCAAIAmCCAAAAmCCAAgAkCCABgggACAJgggAAAJgggAIAJAggAYIIAAgCYIIAAACYIIACACQIIAGCCAAIAmCCAAAAmCCAAgAkCCABgIukCyDln3QIAIA4u9XyedAF04sQJ6xYAAHFwqedzn0uyS47e3l4dOnRI6enp8vl8UevC4bDGjh2r9vZ2ZWRkGHVoj+NwDsfhHI7DORyHc5LhODjndOLECeXn5ysl5cLXOVcMYE+XJSUlRWPGjLnoNhkZGcP6BPsEx+EcjsM5HIdzOA7nWB+HQCBwyW2S7iU4AMDwQAABAEwMqgDy+/1as2aN/H6/dSumOA7ncBzO4Ticw3E4ZzAdh6S7CQEAMDwMqisgAMDQQQABAEwQQAAAEwQQAMDEoAmgqqoqXXfddRo1apQKCwv13nvvWbc04J555hn5fL6oMXnyZOu2Em7Xrl2aN2+e8vPz5fP5tGXLlqj1zjmtXr1aeXl5SktLU0lJiQ4cOGDTbAJd6jg88sgj550fc+fOtWk2QSorK3XLLbcoPT1dOTk5mj9/vpqbm6O26e7uVnl5uUaPHq2rr75aCxYsUGdnp1HHiXE5x2HWrFnnnQ9Lliwx6rhvgyKA3njjDa1cuVJr1qzR+++/r2nTpqm0tFRHjhyxbm3A3XTTTTp8+HBk/PWvf7VuKeG6uro0bdo0VVVV9bl+7dq1WrdunV566SXt3r1bV111lUpLS9Xd3T3AnSbWpY6DJM2dOzfq/HjttdcGsMPEq6+vV3l5uRobG7V9+3adOXNGc+bMUVdXV2SbFStW6O2339amTZtUX1+vQ4cO6f777zfsOv4u5zhI0mOPPRZ1Pqxdu9ao4wtwg8CMGTNceXl55PHZs2ddfn6+q6ysNOxq4K1Zs8ZNmzbNug1TktzmzZsjj3t7e10wGHQ/+9nPIsuOHz/u/H6/e+211ww6HBifPQ7OObd48WJ37733mvRj5ciRI06Sq6+vd86d+38/cuRIt2nTpsg2//jHP5wk19DQYNVmwn32ODjn3B133OG+853v2DV1GZL+Cuj06dNqampSSUlJZFlKSopKSkrU0NBg2JmNAwcOKD8/XxMmTNBDDz2kgwcPWrdkqq2tTR0dHVHnRyAQUGFh4bA8P+rq6pSTk6Mbb7xRS5cu1bFjx6xbSqhQKCRJysrKkiQ1NTXpzJkzUefD5MmTNW7cuCF9Pnz2OHzi1VdfVXZ2tqZMmaKKigqdOnXKor0LSrrJSD/ro48+0tmzZ5Wbmxu1PDc3V//85z+NurJRWFio6upq3XjjjTp8+LCeffZZ3X777frggw+Unp5u3Z6Jjo4OSerz/Phk3XAxd+5c3X///SooKFBra6u+//3vq6ysTA0NDRoxYoR1e3HX29ur5cuXa+bMmZoyZYqkc+dDamqqMjMzo7YdyudDX8dBkr72ta9p/Pjxys/P1/79+/W9731Pzc3Neuuttwy7jZb0AYT/U1ZWFvl56tSpKiws1Pjx4/Xmm2/q0UcfNewMyWDRokWRn2+++WZNnTpVEydOVF1dnWbPnm3YWWKUl5frgw8+GBbvg17MhY7D448/Hvn55ptvVl5enmbPnq3W1lZNnDhxoNvsU9K/BJedna0RI0acdxdLZ2engsGgUVfJITMzUzfccINaWlqsWzHzyTnA+XG+CRMmKDs7e0ieH8uWLdO2bdv0zjvvRH19SzAY1OnTp3X8+PGo7Yfq+XCh49CXwsJCSUqq8yHpAyg1NVXTp09XbW1tZFlvb69qa2tVVFRk2Jm9kydPqrW1VXl5edatmCkoKFAwGIw6P8LhsHbv3j3sz48PP/xQx44dG1Lnh3NOy5Yt0+bNm7Vz504VFBRErZ8+fbpGjhwZdT40Nzfr4MGDQ+p8uNRx6Mu+ffskKbnOB+u7IC7H66+/7vx+v6uurnZ///vf3eOPP+4yMzNdR0eHdWsD6rvf/a6rq6tzbW1t7m9/+5srKSlx2dnZ7siRI9atJdSJEyfc3r173d69e50k9/Of/9zt3bvX/ec//3HOOfeTn/zEZWZmuq1bt7r9+/e7e++91xUUFLiPP/7YuPP4uthxOHHihHvyySddQ0ODa2trczt27HBf/OIX3fXXX++6u7utW4+bpUuXukAg4Orq6tzhw4cj49SpU5FtlixZ4saNG+d27tzp9uzZ44qKilxRUZFh1/F3qePQ0tLifvjDH7o9e/a4trY2t3XrVjdhwgRXXFxs3Hm0QRFAzjn3wgsvuHHjxrnU1FQ3Y8YM19jYaN3SgFu4cKHLy8tzqamp7tprr3ULFy50LS0t1m0l3DvvvOMknTcWL17snDt3K/aqVatcbm6u8/v9bvbs2a65udm26QS42HE4deqUmzNnjrvmmmvcyJEj3fjx491jjz025P5I6+u/X5LbsGFDZJuPP/7Yfetb33Kf+9zn3JVXXunuu+8+d/jwYbumE+BSx+HgwYOuuLjYZWVlOb/f7yZNmuSeeuopFwqFbBv/DL6OAQBgIunfAwIADE0EEADABAEEADBBAAEATBBAAAATBBAAwAQBBAAwQQABAEwQQAAAEwQQAMAEAQQAMEEAAQBM/D9E7dVnW1sBsQAAAABJRU5ErkJggg==\n"
          },
          "metadata": {}
        },
        {
          "output_type": "execute_result",
          "data": {
            "text/plain": [
              "<matplotlib.image.AxesImage at 0x7e9516f138e0>"
            ]
          },
          "metadata": {},
          "execution_count": 7
        },
        {
          "output_type": "display_data",
          "data": {
            "text/plain": [
              "<Figure size 640x480 with 3 Axes>"
            ],
            "image/png": "iVBORw0KGgoAAAANSUhEUgAAAh8AAADACAYAAACkqgECAAAAOXRFWHRTb2Z0d2FyZQBNYXRwbG90bGliIHZlcnNpb24zLjcuMSwgaHR0cHM6Ly9tYXRwbG90bGliLm9yZy/bCgiHAAAACXBIWXMAAA9hAAAPYQGoP6dpAAAbyUlEQVR4nO3de3BU9f3/8XeC2SVA2HBNSCEYEAtIuZgSjLcRm0KpMFAYKVNntNWWCokdZKZ0cIrOOJ2m1altoahUC2hbLtJyURhgnHArTIByFQQiVJRUSLiUbELIjeR8/3DMz/h+748NLGf3bJ6PmfOHL87uOQvvrG8O7/M5CY7jOAIAAOCSxGifAAAAaFtoPgAAgKtoPgAAgKtoPgAAgKtoPgAAgKtoPgAAgKtoPgAAgKtoPgAAgKtoPgAAgKtoPgAAgKtuu1VvvHDhQnn55ZelrKxMhg0bJgsWLJCcnJzrvq6pqUnOnj0rKSkpkpCQcKtOD3HOcRypqqqSjIwMSUxsXY9N7SKaqF14Vatq17kFVqxY4fh8Pmfx4sXOhx9+6PzkJz9xUlNTnfLy8uu+trS01BERNraIbKWlpdQumyc3apfNq1s4tXtLmo+cnBwnPz+/+b8bGxudjIwMp7Cw8LqvraioiPpvHFv8bBUVFdQumyc3apfNq1s4tRvxmY/6+nrZv3+/5OXlNWeJiYmSl5cnxcXFav+6ujqprKxs3qqqqiJ9SmjDWnMJmdpFLKF24VXh1G7Em4+LFy9KY2OjpKWltcjT0tKkrKxM7V9YWCiBQKB569OnT6RPCQgLtQuvonbhNVG/22Xu3LkSDAabt9LS0mifEhAWahdeRe0i2iJ+t0v37t2lXbt2Ul5e3iIvLy+X9PR0tb/f7xe/3x/p0wBajdqFV1G78JqIX/nw+XySnZ0tRUVFzVlTU5MUFRVJbm5upA8HRAy1C6+iduE5rRqnDtOKFSscv9/vLF261Dl27Jgzffp0JzU11SkrK7vua4PBYNQnddniZwsGg9Qumyc3apfNq1s4tXtLmg/HcZwFCxY4mZmZjs/nc3Jycpzdu3eH9Tp+CNgiubX2C5zaZYuVjdpl8+oWTu0mOI7jSAyprKyUQCAQ7dNAnAgGg9K5c2dXjkXtIpKoXXhVOLV7y5ZXh/uSkpJU1tjYaO7b1NR0q08HAOJK+/btVVZfX68yvl+vL+q32gIAgLaF5gMAALiK5gMAALiK5gMAALiK5gMAALiKu11iTM+ePVU2evRolVm3MV24cEFlBw4cMI9j7VtTUxPOKQJhs55uGWN39wPy7W9/W2VZWVkqq66uVtnBgwdVdv78efM4Fy9evIGzi09c+QAAAK6i+QAAAK6i+QAAAK6i+QAAAK5i4DTCrAE765HWc+fONV8/fvx4lV25ckVl1sBoenq6yhYtWmQep6KiQmWvvfaayi5duqSyUEu2o2277Tb9ddKvXz+VPfPMMyqbOnWqyqzaExE5fvy4ytavX68yaxDw6NGjKrt27Zp5HHjbjBkzVGbVnohI7969VWYNh2ZmZqps48aNKktMtP9e/6c//Ullu3fvVtnly5fN18cTrnwAAABX0XwAAABX0XwAAABX0XwAAABXMXAaYe3atVPZY489prIxY8aYr7dWf/T7/Srr0aOHympra1X23e9+1zyONRyYnJyssjfffFNlp06dMt8TbYNVjyIiw4cPV1lBQYHKJk+erLL27durzKpxEZG+ffuqbMiQISrbvHmzyv74xz+q7MyZM+ZxGhoazBzR5fP5VPb444+r7JVXXlGZVWci9hB9WlqayqxBUOu71KpHEZHnnnsurPNcs2aN+fp4wpUPAADgKpoPAADgKpoPAADgKpoPAADgKgZOI8x6NPODDz6oslAr4FmOHDmisiVLlqhs5cqVKsvOzjbfc/HixSp79NFHVbZv3z6VMXDadqSkpKjsnnvuMfedOXOmyh555BGVWUPZllADn9ZwYK9evVQ2btw4lVmrmf797383j7N///7rnSKiwBoEffjhh1VmfcdaA/0iIlVVVSqbP3++yqyB5atXr6rs3//+t3kc6/vYWl21Y8eOKquurjbf06u48gEAAFxF8wEAAFxF8wEAAFxF8wEAAFzFwOlNsAaf8vLyVHbnnXeq7Ny5c+Z7rlu3TmV79+5V2T/+8Q+V1dTUqKy4uNg8Tl1dnco6d+6ssgsXLqjMWtGPx5LHJ2tA7he/+IW57+jRo1V25coVle3cuVNlJ0+eVFlJSYl5nBMnTqisQ4cOKps2bZrKJk6cqLLS0lLzOMePH1eZNVwId1kr3I4YMSKs1xYVFZm5NVy8aNEilVkrnHbq1EllVo2L2MOl1srAbaHOuPIBAABcRfMBAABcRfMBAABcRfMBAABcRfMBAABcxd0uYQi1HPTgwYNVNnXqVJVZd4eEWjramrA+evTo9U4xpLvvvtvML168qDJrKe2vf/3rKtu+ffsNnw+85a677lJZ//79w359MBhU2dtvv62yDRs2qCzUUtjWXV1NTU0qGzlypMqsJdcHDhxoHic9PV1lH3/8sbkvIi8hIcHMrTtbevbsqTJrOfJQS+YXFhaqzKpdi9/vV1lSUpK5r3VHofX/F2u/cM/HK7jyAQAAXEXzAQAAXEXzAQAAXEXzAQAAXMXAaRgaGxvNPNSgWjisIVQRkWHDhqns2LFjKrMG7BITdS/Zvn178zjdu3dX2enTp1W2a9cu8/VoG6w6aw3rZ8daotpahr017rjjDpVZw7LWcF+oQb5QP/dwR79+/cz8gQceUFlqaqrKrCXKrZsERG5umNMabA01cFpRUaGyPXv2qMx6/EW84coHAABwFc0HAABwFc0HAABwVaubjx07dsiECRMkIyNDEhISZO3atS1+3XEcef7556VXr16SnJwseXl55uOyAbdRu/AqahfxptUDp9XV1TJs2DB58sknZfLkyerXX3rpJZk/f7689dZbkpWVJfPmzZOxY8fKsWPHQg4/elVZWZnKLl26pLKMjAyVdenSxXzPH/zgByo7fvy4yqyVFquqqlQWapVI6/iLFy8O6zheRe223oEDB1R25swZc1+rzgOBgMoyMzNVZq2ua9WziD2sPWHCBJVZq/vW19erbOfOneZxzp07Z+bR0BZr98KFC2aenJysMuvPtUOHDiqzBvpFRAoKClT2/vvvq6y8vFxlPp9PZaG+d63/Z2zdulVltbW15uvjSaubj3HjxplLFIt8/hv+hz/8QX75y1/KxIkTReTzpZTT0tJk7dq1Mm3atJs7W+AmULvwKmoX8SaiMx+nT5+WsrIyycvLa84CgYCMGjVKiouLzdfU1dVJZWVliw1wG7ULr6J24UURbT6+uKSUlpbWIk9LSzMvN4l8/kCfQCDQvPXp0yeSpwSEhdqFV1G78KKo3+0yd+5cCQaDzVtpaWm0TwkIC7ULr6J2EW0RXeH0i0dQl5eXS69evZrz8vJyGT58uPkav99vPpLYC3bs2KGyTZs2qewb3/iGyqxhKBGR0aNHq2zbtm0qsx4BbT0ueubMmeZxrL8RnThxQmVtYfBJpO3Vbrj27t2rsqVLl5r7ZmVlqcwaQv3d736nMmtQe+PGjeZxrJUvrSFM67Hk7777rsoOHTpkHscaYoxF8Vq7of4paP369Sr76lUfEZHs7Oyw9hP5vBn7qp///Ocq+/Wvf62ys2fPquz+++83j3Pq1Ckzb4sieuUjKytL0tPTpaioqDmrrKyUPXv2SG5ubiQPBUQUtQuvonbhRa2+8nHlypUW3dvp06fl0KFD0rVrV8nMzJRZs2bJr371KxkwYEDzLV8ZGRkyadKkSJ430GrULryK2kW8aXXzsW/fvhb/NDB79mwREXniiSdk6dKlMmfOHKmurpbp06dLRUWF3H///bJp0ybP3muO+EHtwquoXcSbVjcfDz30UMgFVEREEhIS5MUXX5QXX3zxpk4MiDRqF15F7SLeRP1uFwAA0LZE9G6Xtsaa0H/11VdV9sknn6jsscceM9/Tmk7v2LGjyqZPn64yaznigQMHmsf517/+pTJrGff/39+2EP+sP/8NGzaY+1rzBT169FBZp06dVPboo4+qrKGhwTzOk08+qbL+/fur7K233lLZm2++qTLrbgXErlWrVqns8uXLKrNWdn3ggQfM9+zevbvKrGX858yZE9Z+1mMFRETeeOMNlVl3GbYFXPkAAACuovkAAACuovkAAACuovkAAACuSnBibKKwsrIy5LCOV6Wmpqps1KhR5r7jx49XmTXIZy1bnZioe8lQQ3s//vGPVbZy5UqV1dXVma/3imAwaC6zfSvEY+22Rt++fVW2evVqlQ0dOlRl1lLm5eXl5nGsJbKtJbet4cAzZ86oLMa+AptRu+FLSEhQ2eDBg1X2ne98x3y9NYhqfUdbtWcdO1RN/exnP1PZsmXLVPa///3PfL1XhFO7XPkAAACuovkAAACuovkAAACuovkAAACuYuA0xnTp0kVl3/zmN1U2f/58lVmrmYb64124cKHKFixYoLKPPvrIfL1XMLTnHmvgeeTIkSorLi5WmTW0F2pYurCwUGXvvPOOyj788EPz9V5B7d4cq6batWtn7tutWzeV5eTkqGzevHkqs2o81PeutTrwb3/7W5UdPnxYZVVVVeZ7xiIGTgEAQMyh+QAAAK6i+QAAAK6i+QAAAK7SzwJGVFmPhv70009VduTIEZUNGDBAZY2NjeZx7r//fpVZjxa3Hl996tQp8z3RNliPEBcR6dmzp8ruvvtulYU7415ZWWnmBw4cUJnXh0sReVadXbt2zdzXWk334MGDKtu1a5fKrBoPNSw9duxYlfXr109l1g0F7733nsqs72yv4MoHAABwFc0HAABwFc0HAABwFc0HAABwFQOnMcZa4fT73/++ykaPHq2yzz77TGWbN282j/OjH/1IZYMGDVKZz+dT2aJFi1RWVlZmHgfelpKSojJrxV0RkdmzZ6ssNzdXZU1NTSqzBgGvXLliHifGFmVGHLBW57Vqd8KECSo7d+6cyt59913zOEOHDg3rOE899ZTKrAHs5cuXm8fxAq58AAAAV9F8AAAAV9F8AAAAV9F8AAAAV9F8AAAAV3G3S4y56667VDZ58mSVWXfFrF69WmXFxcXmcYYPH66y7OxslVnLsO/cuVNl3O3iLdZ0v7U8+rhx41Q2Z84c8z379++vskuXLqnMWrL/6tWrKrPu8gqVHz58WGXWYwkAi9/vV1leXp7KMjIyVLZnzx6VhbrbpaKiQmX33nuvyqzv4lGjRqls27Zt5nGsO3BiDVc+AACAq2g+AACAq2g+AACAq2g+AACAqxg4jZLk5GQzt4bp+vXrp7KTJ0+qbMOGDSr7z3/+Yx7n0KFDKrOGnAYOHKiyIUOGqKyoqMg8DqLLGiwVEbn99ttVNnXqVJVZyzz36dPHfM9Tp06p7I033lDZ1q1bVTZz5kyVhfoZsR4DMHjwYJUxcIqvCvXzYA36W4+wuO02/b/MgwcPqizUoL9V09brR4wYobJu3bqpzHpUgVdw5QMAALiK5gMAALiK5gMAALiK5gMAALiKgVMXWENK1qp2IiIPP/ywyqwhKWugacuWLSqrq6szj2MNOVn7NjQ0qMxa0S8hIcE8juM4Zg539OrVy8ytweYZM2aozBpys2pHROTPf/6zylasWKEya4C6Y8eOKgtVU9aQnc/nM/cFvixUnUycOFFlPXr0UNknn3yish07dqjMWrFXROTIkSMqu3Dhgspqa2tVVlNTozJrxVSv4MoHAABwFc0HAABwFc0HAABwVauaj8LCQhk5cqSkpKRIz549ZdKkSVJSUtJin9raWsnPz5du3bpJp06dZMqUKVJeXh7RkwZai9qFV1G7iEetGjjdvn275Ofny8iRI+XatWvy3HPPyZgxY+TYsWPNA2PPPvusbNiwQVatWiWBQEAKCgpk8uTJsmvXrlvyAbygffv2Kps2bZq574ABA1T23//+V2Vvv/22yqwBPWuVPhF7NcukpCSVWatEWkOksT5Y2lZrNysry8ytuvja176mso8++khlixcvNt/znXfeUZk1xGz9PFgDp6FYg3ehBvziQVut3VshMzPTzIcPH64yqyZ3796tsk2bNqnsjjvuMI+Tn5+vstzcXJVZPyOXL19WWffu3c3jfPbZZ2YeS1rVfHz1N3np0qXSs2dP2b9/vzz44IMSDAblL3/5iyxbtqz5ro0lS5bIoEGDZPfu3XLPPfdE7syBVqB24VXULuLRTc18BINBERHp2rWriIjs379fGhoaJC8vr3mfgQMHSmZmZsi17uvq6qSysrLFBtxq1C68itpFPLjh5qOpqUlmzZol9913X/ODxsrKysTn80lqamqLfdPS0qSsrMx8n8LCQgkEAs1bqIdWAZFC7cKrqF3EixtuPvLz8+Xo0aPmIkKtMXfuXAkGg81baWnpTb0fcD3ULryK2kW8uKEVTgsKCmT9+vWyY8cO6d27d3Oenp4u9fX1UlFR0aILLy8vl/T0dPO9/H6/+P3+GzkNz7AeXx5qIKldu3Yqq6qqUllOTo7K7rzzTpU98sgj5nGsAavGxkaVnT17VmVfXPb1orZWu9bquiL2So/hDg1bjx8XEXn88cdVZtW5Nezav39/lZ05c8Y8ztq1a1X2wQcfmPvGk7ZWuzfLGsDv0KGDue8XV5G+zPoutgZBf/rTn6os1Pe7tZJqcnKyyvbu3auyffv2qay6uto8jhe06sqH4zhSUFAga9askS1btqhJ+uzsbElKSpKioqLmrKSkRM6cOWNO9AJuoXbhVdQu4lGrrnzk5+fLsmXLZN26dZKSktL874mBQECSk5MlEAjIU089JbNnz5auXbtK586d5ZlnnpHc3FwmrhFV1C68itpFPGpV8/Haa6+JiMhDDz3UIl+yZIn88Ic/FBGR3//+95KYmChTpkyRuro6GTt2rLz66qsROVngRlG78CpqF/GoVc1HOP8m3L59e1m4cKEsXLjwhk8KiDRqF15F7SIe8WwXAADgqhu62wWtY91xcO7cubBf37dvX5U9/fTTYe0XivW3KWsp5r/97W8qa825I7pC3THy1WeDiIgMHTpUZdYdVFbWGk1NTSpLTNR/D1q3bp35+s2bN6vs/PnzN3VOiD/Wd9yX7xL6MusxEtZS7NZdMVb2xQJw4bCWQrf+ycy6y6uhoSHs48QarnwAAABX0XwAAABX0XwAAABX0XwAAABXMXDqghMnTqgs1NMmx48frzJr+d1wh0s3bNhg5v/85z9VtmrVKpVdvXo1rOMgNn388cdm/te//lVl1oDeoEGDVBZqmK579+4q69atm8qsYdetW7eqbOPGjeZxrOeQhLs0PNo2a4lyEZEtW7aozFqgLRAIqMxahv3YsWPmcaxboV9//XVz33jHlQ8AAOAqmg8AAOAqmg8AAOAqmg8AAOAqBk5dUFtbq7LVq1eb+waDQZXde++9Kjt8+LDKrCFSVn6EZefOnWFlt0JKSorKrFWA6+vrzdfX1NRE/JzQNnzxROCvsr47rX1HjBihsh07dqhs+fLlN3B2bQtXPgAAgKtoPgAAgKtoPgAAgKtoPgAAgKsSnBhbGrCystJcRa6tSEpKUpmXH5scbcFgUDp37uzKsdp67SKyqN3o8vl8Kgs1BI2WwqldrnwAAABX0XwAAABX0XwAAABX0XwAAABX0XwAAABXsbx6jOHOFgCIPu5subW48gEAAFxF8wEAAFxF8wEAAFxF8wEAAFxF8wEAAFxF8wEAAFxF8wEAAFxF8wEAAFwVc82H4zjRPgXEETfridpFJFG78Kpw6inmmo+qqqponwLiiJv1RO0ikqhdeFU49ZTgxFjL29TUJGfPnpWUlBSpqqqSPn36SGlpqXTu3Dnap3bTKisr+TwucRxHqqqqJCMjQxIT3emxqV3viOXPQ+1GViz/Wd+IWP48randmHu2S2JiovTu3VtERBISEkREpHPnzjH3m3wz+DzuCAQCrh6P2vWeWP081G7k8XncEW7txtw/uwAAgPhG8wEAAFwV082H3++XF154Qfx+f7RPJSL4PG1HvP3e8Hnajnj7veHzxKaYGzgFAADxLaavfAAAgPhD8wEAAFxF8wEAAFxF8wEAAFwVs83HwoUL5fbbb5f27dvLqFGjZO/evdE+pbDt2LFDJkyYIBkZGZKQkCBr165t8euO48jzzz8vvXr1kuTkZMnLy5OTJ09G52Svo7CwUEaOHCkpKSnSs2dPmTRpkpSUlLTYp7a2VvLz86Vbt27SqVMnmTJlipSXl0fpjGODV+uX2qV2qd3YEO/1G5PNx8qVK2X27NnywgsvyIEDB2TYsGEyduxYOX/+fLRPLSzV1dUybNgwWbhwofnrL730ksyfP19ef/112bNnj3Ts2FHGjh0rtbW1Lp/p9W3fvl3y8/Nl9+7d8v7770tDQ4OMGTNGqqurm/d59tln5b333pNVq1bJ9u3b5ezZszJ58uQonnV0ebl+qV1ql9qNDXFfv04MysnJcfLz85v/u7Gx0cnIyHAKCwujeFY3RkScNWvWNP93U1OTk56e7rz88svNWUVFheP3+53ly5dH4Qxb5/z5846IONu3b3cc5/NzT0pKclatWtW8z/Hjxx0RcYqLi6N1mlEVL/VL7bY91G7sirf6jbkrH/X19bJ//37Jy8trzhITEyUvL0+Ki4ujeGaRcfr0aSkrK2vx+QKBgIwaNcoTny8YDIqISNeuXUVEZP/+/dLQ0NDi8wwcOFAyMzM98XkiLZ7rl9qNb9RubIu3+o255uPixYvS2NgoaWlpLfK0tDQpKyuL0llFzhefwYufr6mpSWbNmiX33XefDBkyREQ+/zw+n09SU1Nb7OuFz3MrxHP9UrvxjdqNXfFYvzH3VFvErvz8fDl69Kjs3Lkz2qcCtAq1Cy+Lx/qNuSsf3bt3l3bt2qmJ3fLycklPT4/SWUXOF5/Ba5+voKBA1q9fL1u3bm1+9LbI55+nvr5eKioqWuwf65/nVonn+qV24xu1G5vitX5jrvnw+XySnZ0tRUVFzVlTU5MUFRVJbm5uFM8sMrKysiQ9Pb3F56usrJQ9e/bE5OdzHEcKCgpkzZo1smXLFsnKymrx69nZ2ZKUlNTi85SUlMiZM2di8vPcavFcv9RufKN2Y0vc12+UB15NK1ascPx+v7N06VLn2LFjzvTp053U1FSnrKws2qcWlqqqKufgwYPOwYMHHRFxXnnlFefgwYPOp59+6jiO4/zmN79xUlNTnXXr1jkffPCBM3HiRCcrK8upqamJ8plrM2bMcAKBgLNt2zbn3LlzzdvVq1eb93n66aedzMxMZ8uWLc6+ffuc3NxcJzc3N4pnHV1erl9ql9qldmNDvNdvTDYfjuM4CxYscDIzMx2fz+fk5OQ4u3fvjvYphW3r1q2OiKjtiSeecBzn89u+5s2b56SlpTl+v9/51re+5ZSUlET3pEOwPoeIOEuWLGnep6amxpk5c6bTpUsXp0OHDs73vvc959y5c9E76Rjg1fqldqldajc2xHv9JjiO49zaaysAAAD/T8zNfAAAgPhG8wEAAFxF8wEAAFxF8wEAAFxF8wEAAFxF8wEAAFxF8wEAAFxF8wEAAFxF8wEAAFxF8wEAAFxF8wEAAFxF8wEAAFz1fz8F/pMD4ty4AAAAAElFTkSuQmCC\n"
          },
          "metadata": {}
        }
      ]
    },
    {
      "cell_type": "markdown",
      "metadata": {
        "id": "ytv3NxF-kgxN"
      },
      "source": [
        "Let us augment the whole dataset and see if this improves the test accuracy"
      ]
    },
    {
      "cell_type": "code",
      "metadata": {
        "id": "iNzNAoDBkRzj",
        "colab": {
          "base_uri": "https://localhost:8080/"
        },
        "outputId": "a3ccb918-3693-4818-d6b7-6e630e71e4e0"
      },
      "source": [
        "# hyperparameters\n",
        "angleconstraint = 60\n",
        "naugmentations = 5\n",
        "\n",
        "# augment\n",
        "augdata = train_X # we include the original images also in the augmented dataset\n",
        "auglabel = train_y\n",
        "for ii in range(naugmentations):\n",
        "  augdata = np.concatenate((augdata, augRotate(train_X, angleconstraint))) # concatenate the augmented data to the set\n",
        "  auglabel = np.concatenate((auglabel, train_y))  # the labels don't change when we augment\n",
        "\n",
        "# check the test accuracy\n",
        "testpred = NN(augdata, auglabel, test_X)\n",
        "print('Accuracy after rotation augmentation is ', Accuracy(test_y, testpred))"
      ],
      "execution_count": 8,
      "outputs": [
        {
          "output_type": "stream",
          "name": "stdout",
          "text": [
            "Accuracy after rotation augmentation is  0.6616\n"
          ]
        }
      ]
    },
    {
      "cell_type": "markdown",
      "metadata": {
        "id": "E88Nt9s1p5R6"
      },
      "source": [
        "The angle constraint is a hyperparameter which we have to tune using a validation set. (Here we are not doing that for time constraints). Let us try a grid search to find the best angle constraint."
      ]
    },
    {
      "cell_type": "code",
      "metadata": {
        "id": "aiaFRLREmGp6",
        "colab": {
          "base_uri": "https://localhost:8080/"
        },
        "outputId": "20a397fa-d291-487a-89c9-3bc1ef7609b2"
      },
      "source": [
        "angleconstraints = [0,10,20,30,40,50,60,70,80,90] # the values we want to test\n",
        "accuracies = np.zeros(len(angleconstraints), dtype=np.float) # we will save the values here\n",
        "\n",
        "for ii in range(len(angleconstraints)):\n",
        "  # create the augmented dataset\n",
        "  augdata = train_X # we include the original images also in the augmented dataset\n",
        "  auglabel = train_y\n",
        "  for jj in range(naugmentations):\n",
        "    augdata = np.concatenate((augdata, augRotate(train_X, angleconstraints[ii]))) # concatenate the augmented data to the set\n",
        "    auglabel = np.concatenate((auglabel, train_y))  # the labels don't change when we augment\n",
        "\n",
        "  # check the test accuracy\n",
        "  testpred = NN(augdata, auglabel, test_X)\n",
        "  accuracies[ii] = Accuracy(test_y, testpred)\n",
        "  print('Accuracy after rotation augmentation constrained by ',angleconstraints[ii], ' is ', accuracies[ii], flush=True)"
      ],
      "execution_count": 9,
      "outputs": [
        {
          "output_type": "stream",
          "name": "stderr",
          "text": [
            "<ipython-input-9-2f14d3721156>:2: DeprecationWarning: `np.float` is a deprecated alias for the builtin `float`. To silence this warning, use `float` by itself. Doing this will not modify any behavior and is safe. If you specifically wanted the numpy scalar type, use `np.float64` here.\n",
            "Deprecated in NumPy 1.20; for more details and guidance: https://numpy.org/devdocs/release/1.20.0-notes.html#deprecations\n",
            "  accuracies = np.zeros(len(angleconstraints), dtype=np.float) # we will save the values here\n"
          ]
        },
        {
          "output_type": "stream",
          "name": "stdout",
          "text": [
            "Accuracy after rotation augmentation constrained by  0  is  0.6472\n",
            "Accuracy after rotation augmentation constrained by  10  is  0.6727\n",
            "Accuracy after rotation augmentation constrained by  20  is  0.673\n",
            "Accuracy after rotation augmentation constrained by  30  is  0.6737\n",
            "Accuracy after rotation augmentation constrained by  40  is  0.6731\n",
            "Accuracy after rotation augmentation constrained by  50  is  0.6701\n",
            "Accuracy after rotation augmentation constrained by  60  is  0.6675\n",
            "Accuracy after rotation augmentation constrained by  70  is  0.661\n",
            "Accuracy after rotation augmentation constrained by  80  is  0.6462\n",
            "Accuracy after rotation augmentation constrained by  90  is  0.6469\n"
          ]
        }
      ]
    },
    {
      "cell_type": "markdown",
      "metadata": {
        "id": "2oVDRYP2rxob"
      },
      "source": [
        "Let us see the best value for angle constraint: (Ideally this should be done on validation set, not test set)"
      ]
    },
    {
      "cell_type": "code",
      "metadata": {
        "id": "LqthJa_pmMHz",
        "colab": {
          "base_uri": "https://localhost:8080/",
          "height": 485
        },
        "outputId": "3bc561c8-a641-472c-f308-7a47dbd079a7"
      },
      "source": [
        "fig = plt.figure()\n",
        "ax = fig.add_axes([0.1, 0.1, 0.8, 0.8]) # main axes\n",
        "# plot the variation of accuracy\n",
        "ax.plot(angleconstraints, accuracies)\n",
        "ax.set_xlabel('angle')\n",
        "ax.set_ylabel('accuracy')\n",
        "# plot the maximum accuracy\n",
        "maxind = np.argmax(accuracies)\n",
        "plt.scatter(angleconstraints[maxind], accuracies[maxind], c='red')"
      ],
      "execution_count": 10,
      "outputs": [
        {
          "output_type": "execute_result",
          "data": {
            "text/plain": [
              "<matplotlib.collections.PathCollection at 0x7e9516cec2b0>"
            ]
          },
          "metadata": {},
          "execution_count": 10
        },
        {
          "output_type": "display_data",
          "data": {
            "text/plain": [
              "<Figure size 640x480 with 1 Axes>"
            ],
            "image/png": "iVBORw0KGgoAAAANSUhEUgAAAlgAAAHDCAYAAAD4L2TBAAAAOXRFWHRTb2Z0d2FyZQBNYXRwbG90bGliIHZlcnNpb24zLjcuMSwgaHR0cHM6Ly9tYXRwbG90bGliLm9yZy/bCgiHAAAACXBIWXMAAA9hAAAPYQGoP6dpAABeOElEQVR4nO3deVzUdf4H8NfcMNwyHAMil6ImHoVJpGKtGFnblrmua7aatbYpJcJuKR26mWHH6lrZ5mqrududPytT1FUs79v1SuVG8OCWWxiY+f7+AEYnQBEGvnO8no/HPIgvn5l5j1+LV9/P5/v+SARBEEBEREREZiMVuwAiIiIiW8OARURERGRmDFhEREREZsaARURERGRmDFhEREREZsaARURERGRmDFhEREREZsaARURERGRmDFhEREREZsaARURERGRmFhGwPvzwQwQFBcHBwQGRkZE4fPjwTceXl5cjLi4OWq0WKpUKYWFhSElJMf48KCgIEomk1SMuLs445r777mv18+eee67bPiMRERHZD7nYBXz11VdITEzEypUrERkZieXLlyM2NhZpaWnw9vZuNV6n02HcuHHw9vbG+vXr4e/vjwsXLsDd3d045siRI9Dr9cbvz5w5g3HjxmHSpEkmrzVz5kwsWrTI+L1arTb/ByQiIiK7I3rAWrZsGWbOnIkZM2YAAFauXInNmzdjzZo1mD9/fqvxa9asQVlZGfbv3w+FQgGg6YrVjby8vEy+f+uttxAaGooxY8aYHFer1fD19e1U3QaDAZcvX4aLiwskEkmnXoOIiIisiyAIqKqqgp+fH6TSm0wECiKqr68XZDKZ8O2335ocnzZtmvCb3/ymzeeMHz9emDp1qjBz5kzB29tbGDRokPDmm28KjY2N7b6Hp6en8Oabb5ocHzNmjKDRaARPT09h0KBBwvz584Wampp2a62rqxMqKiqMj7NnzwoA+OCDDz744IMPO3zk5+ffNOOIegWrpKQEer0ePj4+Jsd9fHxw/vz5Np+TnZ2NnTt3YurUqUhJSUFmZiZmz56NhoYGLFy4sNX47777DuXl5XjqqadMjj/xxBMIDAyEn58fTp06hXnz5iEtLQ0bNmxo832XLFmC119/vdXx/Px8uLq6dvATExERkTWrrKxEQEAAXFxcbjpOIgiC0EM1tXL58mX4+/tj//79iIqKMh5/6aWXsGvXLhw6dKjVc8LCwlBXV4ecnBzIZDIATdOM7777Lq5cudJqfGxsLJRKJX744Yeb1rJz506MHTsWmZmZCA0NbfXz+vp61NfXG79v+QOuqKhgwCIiIrITlZWVcHNzu+Xvf1GvYGk0GshkMhQWFpocLywsbHdtlFarhUKhMIYrABg4cCAKCgqg0+mgVCqNxy9cuIAdO3a0e1XqRpGRkQDQbsBSqVRQqVQd+lxERERk30Rt06BUKhEREYHU1FTjMYPBgNTUVJMrWjcaOXIkMjMzYTAYjMfS09Oh1WpNwhUArF27Ft7e3nj44YdvWcuJEycANAU4IiIioq4QvQ9WYmIiVq9ejXXr1uHcuXOYNWsWampqjHcVTps2DUlJScbxs2bNQllZGeLj45Geno7NmzcjOTnZpMcV0BTU1q5di+nTp0MuN71Ql5WVhTfeeAPHjh1Dbm4uNm7ciGnTpiE6OhpDhgzp/g9NRERENk30Ng2TJ09GcXExFixYgIKCAgwbNgxbt241LnzPy8szuQ0yICAA27ZtQ0JCAoYMGQJ/f3/Ex8dj3rx5Jq+7Y8cO5OXl4emnn271nkqlEjt27MDy5ctRU1ODgIAATJw4Ea+++mr3flgiIiKyC6IucrdmHV3kRkRERLajo7//RZ8iJCIiIrI1DFhEREREZsaARURERGRmDFhEREREZib6XYREJBK9HtizB7hyBdBqgdGjgRsa+BIRUecxYBHZow0bgPh44OLF68d69wbeew94/HHx6iIishGcIiSyNxs2AL/9rWm4AoBLl5qOd2BrKSIiujkGLCJ7otc3Xblqq/1dy7G5c5vGERFRp3GKkMiOVO7cjQzBGWlDY5GuCUSaVyCKnTwQUFGI4LJLCL56uenrf3dBG3s/pFKJ2CUTEVkldnLvJHZyJ0tW16BHZlE10gqqkF5YhbTCKqQXVOFyRV2HX0MllyLI0wnBGicEezkh2LP5q8YJnk5KSCQMX0Rkfzr6+59XsIisWIPegNySGmOASiusQnphNS6U1sDQzv86aSuLEVZyAf2LLyCs5AK8q6/iops3cnr5I8fDH9m9/JCvCUB9owFpzeHsl1xUcmPYuvERpHGCq4Oimz81EZHlY8AisgIGg4CLV681B6gq45WprOJqNOjbTlIeagX6+7qgv48Lwpq/9tOo4TawX9OC9rYuXkskQO/eaMzMwqUqHbJLapBbUoOc5kd2cQ0uV1xDVX0jTl2swKmLFa1eQuOsNAlcIRonBGucEeiphoOCbSCIyD4wYBFZEEEQUFRVf31qr/lremE1rjW0vfDcSSkzBqgwHxf09236qnFuZxrvvfea7haUSExDVsvY5cshVyoQ6KlAoKcT0N/06XUNeuSV1SK7uAa5pTXIKW4OYKU1KK6qR0m1DiXVOhzJvWryPIkE8HNzbCN8OaG3hyPkMt5zQ0S2g2uwOolrsKirrtbomsNTyxqpaqQVVqHiWkOb45VyKfp6ORsDVH9fZ4T5uMDPzfH2F6O31QcrIABYvrxLfbCq6hqQW1KLHGPwqm668lVSg6q6xnafJ5dK0KeXunX48nKCj4sDF9sTkcXo6O9/BqxOYsCijqqpb0RGUfUNa6SarkwVVdW3OV4qAYI1TteDVPMUX2AvtXmv8vRgJ3dBEFBWozNONd74yC2tQV2Dod3nOiiaFtuHeDkZF92HeDVNO3qoFVxsT0Q9igGrmzFg0S/VN+qRXVxjMrWXVliF/LJr7T6nt4ejyRqpMB8XhHg52dVaJYNBQEFlXevgVVKDvLJaNLa3Wh+Aq4McwV7OCNE0hy8vJ9zVxx29PdQ9+AmIyJ4wYHUzBiz7pTcIuFDaEqSqjUEqp6QG+nbCgJeL6oY1Uk1Te/18XOCs4jLIm2nQG3Dx6rXmqcZa45RjbkktLpW3HVyVMimSHhqAp+4N4tUtIjI7BqxuxoBlG3SNBpRf06G8tgFXa3S4WtuAimtNX6/W6lBe0/y15ftrDSiv1bV7556rg/yGNVJNX8N8XNDLSdnDn8z2XdPpcaGs6UpXdknTmq+zVyrx8+VKAEDMQB+8+9sh8OCfPRGZEQNWN2PAsiyCIKCyrhHltTeEI2MwajAeb/radLy8tgHV9e0vvL4ZB4XUGJ5unOLzcVXxqomIBEHAvw9cwJubz0GnN0Dr5oD3fn8nRgT3Ers0IrIRDFjdjAGr+9Q16I1XjG4MQy2h6cbAdLVWh4raBpRfa2h3eu5WJBLAzVEBD7US7urrX90dlfBQK+Du1PT1xp/7uvLONkt25lIFXvjif8gpqYFUAiTEhGH2/X0h4zkjoi5iwOpmDFgdV1HbgMziapNwdD0wtQSp68fb6/fUEY4KWVMoUivh4dT8tTkUtYSo68ebfubioOAvXhtUXd+IBd+dwYb/XQIA3BvqieWTh8Hb1UHkyojImjFgdTMGrI6pqG3AqHd23rQHUltkUgk81Iobriw1ByWnX1xdag5MLQHKnu6+o475v2MX8dr3Z1Cr08PTSYmlvxuK+/p7i10WEVkp7kVIFuH0pQpU1TVCJW9as9QyxWYMR81frx9Xwt1JAReVnGuZyCwmRvTGsD7ueP7z/+HclUo8tfYI/hQdgr/E9oeC3eOJqJswYFG3yixq2ig4OswLq6cNF7kaslehXs74dva9SE45h38fuIB/7s7GoZwyfDDlTgT0Ys8sIjI//u8bdavM4moAQF9vZ5ErIXvnoJBh0aPhWPlkBFwd5DiRX46H3tuDzaeuiF0aEdkgBizqVplFzQHLiwGLLMOD4b5IiR+NiEAPVNU3Iu7z43j529Oo68LNFUREv8SARd0qs6gGABDKK1hkQXp7qPHls/dg9n2hkEiAzw/l4dEV+5BRWCV2aURkIxiwqNtU1DagpLppQ+NQLyeRqyEypZBJ8dKDA/Dvp0dA46xCWmEVHlmxF18fyQdvriairmLAom6TWdx0NcDX1QEuDgqRqyFq2+h+XtgSPxqj+2lQ12DAS/93CvFfnkBVXYPYpRGRFWPAom5jXH/F6UGycF4uKqybMQIvPdgfMqkEG09exq8/2ItTF8vFLo2IrBQDFnUbBiyyJlKpBLPv64uv/xQFf3dHXCitxcSP9uNfe3M4ZUhEt40Bi7pNVjEXuJP1iQj0QMqc0XhwkC8a9ALe2HQWf1x3FGU1OrFLIyIrwoBF3YYtGshauakV+OjJu/DGo4OglEuRer4ID723B4eyS8UujYisBAMWdYu6Bj3yr9YC4BQhWSeJRII/RAXhu9kjEeLlhILKOkxZfRDv7ciA3sApQyK6OQYs6hbZxTUQBMDNUQGNs1Lscog67Q4/V2x6YRR+G9EbBgH4+450TP34IAor68QujYgsGAMWdYsbt8jhps1k7dRKOf42aSj+Pnko1EoZDmaXYfx7e/Dj+SKxSyMiC8WARd2iZf0VG4ySLZlwZ29semEUBvm5oqxGhxmfHMGbm89C12gQuzQisjAMWNQtstiigWxUiJczNsy+F0/dGwQAWL0nB5NW7kdeaa24hRGRRWHAom7BHlhky1RyGf76m0FY9YcIuDkqcPJiBR5+fw9+OHlZ7NKIyEIwYJHZNeoNyClp6oHV18tF5GqIus8Dg3yxJX40hgd6oKq+ES988T8kbTiFazq92KURkcgYsMjs8q9eg05vgEouhb+Ho9jlEHUrP3dHfPnsPXjhV30hkQBfHM7Hox/uRXphldilEZGIGLDI7FrWX4V4OUMm5R2EZPvkMin+/EB/fPpMJLxcVEgvrMZvVuzFF4fzuM0OkZ1iwCKzu7FFA5E9GdlXgy3xoxEd5oW6BgOSNpzGC1/8D5V1DWKXRkQ9jAGLzI5b5JA90zir8MlTdyNp/ADIpRJsOnUFv35/L07ml4tdGhH1IAYsMjveQUj2TiqV4E9jQvH1c1Ho7eGIvLJaTPxoP1bvzoaB2+wQ2QUGLDIrQRDYA4uo2V19PLB5zmg8NNgXjQYBb6acwzPrjqC0ul7s0oiomzFgkVkVVdWjqr4RUgkQpFGLXQ6R6NwcFfjwibvw5oRwqORS/JhWjIfe34MDWaVil0ZE3YgBi8yqZXqwTy81VHKZyNUQWQaJRIKpkYH4/vmR6OvtjMLKejzx8UEs256ORj232SGyRQxYZFZcf0XUvgG+rtj4/Ej8bnhvCALwfmoGnvj4EK5UXBO7NCIyMwYsMivjJs8MWERtUivleOe3Q/He74fBSSnD4ZwyPPTeHqSeKxS7NCIyIwYsMiu2aCDqmEeH+WPznNEI93fF1doGPLPuKBb9cBb1jdxmh8gWMGCRWWWxyShRhwVpnPB/s+7F0yODAQBr9uXgtx8dQG7zXp5EZL0sImB9+OGHCAoKgoODAyIjI3H48OGbji8vL0dcXBy0Wi1UKhXCwsKQkpJi/HlQUBAkEkmrR1xcnHFMXV0d4uLi4OnpCWdnZ0ycOBGFhbxE3xWVdQ0oqmq6/ZxThEQdo5LLsOCRO/DxtOFwVytw+lIFfv3BXnx/4pLYpRFRF4gesL766iskJiZi4cKFOH78OIYOHYrY2FgUFRW1OV6n02HcuHHIzc3F+vXrkZaWhtWrV8Pf39845siRI7hy5YrxsX37dgDApEmTjGMSEhLwww8/4JtvvsGuXbtw+fJlPP744937YW1cy/Sgj6sKrg4Kkashsi4xd/hgS/xojAjqher6RsR/eQIvrT+JWl2j2KURUSdIBJF3Io2MjMTdd9+NFStWAAAMBgMCAgLwwgsvYP78+a3Gr1y5Eu+++y7Onz8PhaJjv8Tnzp2LTZs2ISMjAxKJBBUVFfDy8sLnn3+O3/72twCA8+fPY+DAgThw4ADuueeeW75mZWUl3NzcUFFRAVdX19v4xLbr66P5eGn9KYzs64nP/njrP0Miaq1Rb8D7OzPxwc4MCAIQ5KlG8oTBuLevRuzSiAgd//0v6hUsnU6HY8eOISYmxnhMKpUiJiYGBw4caPM5GzduRFRUFOLi4uDj44Pw8HAkJydDr297YahOp8Onn36Kp59+GhKJBABw7NgxNDQ0mLzvgAED0KdPn3bft76+HpWVlSYPMpXFBe5EXSaXSZE4Lgyf/TESvq4OyC2txRMfH8JL60+ivFYndnlE1EGiBqySkhLo9Xr4+PiYHPfx8UFBQUGbz8nOzsb69euh1+uRkpKC1157DUuXLsXixYvbHP/dd9+hvLwcTz31lPFYQUEBlEol3N3dO/y+S5YsgZubm/EREBDQ8Q9qJ9gDi8h87g3VYHtiNP5wTyAA4OujFxGzbBd+OHkZIk88EFEHiL4G63YZDAZ4e3tj1apViIiIwOTJk/HKK69g5cqVbY7/17/+hfHjx8PPz69L75uUlISKigrjIz8/v0uvZ4sym+8gDOUVLCKzcHFQ4I3HwrH+uSj09XZGSbUOL3zxPzyz7igulbM5KZElEzVgaTQayGSyVnfvFRYWwtfXt83naLVahIWFQSa7vg3LwIEDUVBQAJ3O9PL5hQsXsGPHDvzxj380Oe7r6wudTofy8vIOv69KpYKrq6vJg66ra9Ajv6wWAK9gEZnb8KBe2DxnFObG9INCJsHO80V4YNkufLIvB3oDr2YRWSJRA5ZSqURERARSU1ONxwwGA1JTUxEVFdXmc0aOHInMzEwYDNf370pPT4dWq4VSqTQZu3btWnh7e+Phhx82OR4REQGFQmHyvmlpacjLy2v3fenmckpqYBAAFwc5vFxUYpdDZHNUchnmxoQhZc5oRAR6oEanx19/OIvfrtyPtIIqscsjol8QfYowMTERq1evxrp163Du3DnMmjULNTU1mDFjBgBg2rRpSEpKMo6fNWsWysrKEB8fj/T0dGzevBnJyckmPa6ApqC2du1aTJ8+HXK53ORnbm5ueOaZZ5CYmIgff/wRx44dw4wZMxAVFdWhOwiptRvXX7XcTEBE5tfPxwXf/CkKbzw6CM4qOf6XV45ff7AHy/6bhroGdoEnshTyWw/pXpMnT0ZxcTEWLFiAgoICDBs2DFu3bjUufM/Ly4NUej0HBgQEYNu2bUhISMCQIUPg7++P+Ph4zJs3z+R1d+zYgby8PDz99NNtvu/f//53SKVSTJw4EfX19YiNjcU//vGP7vugNs7YwZ3rr4i6nVQqwR+ighBzhw8WfP8ztp8txPs7M7Hp9BW89fgQjAjuJXaJRHZP9D5Y1op9sEw9//lxbDp1BUnjB+BPY0LFLofIbgiCgK1nCrBg488obt5J4YnIPpg/fgAb/hJ1A6vog0W2gy0aiMQhkUgwfrAWOxLGYMqIpvYxnx/KQ8zSXdh6pu22M0TU/RiwqMv0BgHZzZvTMmARicNNrcCSx4fgy2fvQbDGCUVV9Xju02P403+OorCyTuzyiOwOAxZ12cWrtdA1GqCUS9HbQy12OUR27Z4QT2yJH424+0Mhl0qw7edCxCzdhU8PXoCBLR2IegwDFnVZy/RgiMYJMinvICQSm4NChhdjB+CHF0ZhaIA7quob8ep3ZzB51QHjv69E1L0YsKjLWv6DHcrpQSKLMlDrig2z7sWCX98BtVKGI7lX8dB7e/B+agZ0jYZbvwARdRoDFnVZJjd5JrJYMqkET48Kxn8TonF/fy/o9AYs256OX3+wB8cuXBW7PCKbxYBFXdayByEXuBNZrt4eaqx56m689/th8HRSIr2wGr9duR8Lvz+D6vpGscsjsjkMWNQlgiCwRQORlZBIJHh0mD92JI7BbyN6QxCAdQcuYNyyXUg9V3jrFyCiDmPAoi4prq5HVV0jpBIgWOMkdjlE1AEeTkr8bdJQfPpMJPr0UuNKRR2eWXcUcZ8fNzYrJaKuYcCiLmm5ehXQSw0HhUzkaojodozqp8G2udH4U3QIZFIJNp+6gphlu/D1kXxwkw+irmHAoi7J4gJ3IqvmqJQh6aGB+D5uJAb5uaLiWgNe+r9TmPrxIeQ2NxAmotvHgEVdwvVXRLYh3N8N38eNxMsPDYCDQor9WaWIXb4b//gpEw16tnQgul0MWNQlLXcQsgcWkfWTy6R4NjoU/507BqP6alDfaMA7W9PwmxX7cDK/XOzyiKwKAxZ1Ca9gEdmePp5q/OeZEVg6aSjc1Qqcu1KJCf/Yhzc2nUWtji0diDqCAYs6rbKuAYWVTXcchXINFpFNkUgkmBjRGzsSx+DRYX4wCMC/9ubggb/vxq70YrHLI7J4DFjUaS0L3L1cVHBzVIhcDRF1B42zCu/9/k6snXE3/N0dcfHqNUxfcxhzv/wfSqvZ0oGoPQxY1GncIofIftzf3xv/TYjG0yODIZUA3524jJhlu7Dh+EW2dCBqAwMWdVpWcdMt3Fx/RWQfnFRyLHjkDmyYPRIDfF1wtbYBiV+fxLQ1h5FfVit2eUQWhQGLOo0L3Ins07AAd/zwwii8GNsfSrkUezJK8MDfd+PjPdloZEsHIgAMWNQFWdzkmchuKWRSxN3fF1vjR+OekF641qDH4s3nMOEf+/Hz5QqxyyMSHQMWdUp9ox4XSjlFSGTvQryc8cXMe/D2xMFwdZDj9KUK/GbFPizZcg7XdHqxyyMSDQMWdUpuSS0MAuCiksPbRSV2OUQkIolEgsl398GOP4/Bw4O10BsE/HNXNh58bzf2ZZaIXR6RKBiwqFNa1l+FejtDIpGIXA0RWQJvFwd8OPUurJ42HL6uDrhQWoupHx/Ci9+cRHmtTuzyiHoUAxZ1Che4E1F7xt3hg+2J0ZgWFQiJBPjm2EXELNuFjScvs6UD2Q0GLOoU4x6E7IFFRG1wcVBg0aPhWP9cFPp5O6OkWoc5X/wPH+/JEbs0oh7BgEWdwitYRNQREYG9sGnOKDw3JhQAsHR7GvJK2TOLbB8DFt02vUFANls0EFEHqeQyzHuwP+4N9URdgwGvfn+GU4Vk8xiw6LZdLr+G+kYDlDIpAjwcxS6HiKyARCLB4sfCoZRLsTu9GBtPXha7JKJuxYBFt61lejBY4wS5jH+FiKhjQryc8fz9fQEAb2w6i4raBpErIuo+/O1It43rr4ios/40JgR9mxe9v7X1nNjlEHUbBiy6bTf2wCIiuh0quQzJEwYDAL44nI8juWUiV0TUPRiw6LZlcoE7EXXBiOBe+P3dAQCAlzechq6RG0ST7WHAotsiCML1KUL2wCKiTpo/fgA0zkpkFFVj1e4sscshMjsGLLotJdU6VFxrgEQChHg5iV0OEVkpd7USr/36DgDA+zszkVtSI3JFRObFgEW3peXqVYCHGg4KmcjVEJE1+81QP4zup4Gu0YBXvjvN3lhkUxiw6LZc3yKHV6+IqGtaemOp5FLsyyzFdycuiV0SkdkwYNFtyWKLBiIyo0BPJ8wZ2w8A8Mamc7haoxO5IiLzYMCi25LFOwiJyMxmjg5BmI8zymp0WLKFvbHINjBg0W1hk1EiMjelXIoljzf1xvr66EUczC4VuSKirmPAog6rrm/ElYo6AEBfLxeRqyEiWxIR2AtPRPYBALz87WnUN+pFroioaxiwqMNa1l9pnFVwUytEroaIbM28BwdA46xCdnENPvqJvbHIujFgUYddnx7kHYREZH5ujgosfKSpN9Y/fswyrvkkskYMWNRh3CKHiLrbr4docV9/L+j0BrzyLXtjkfViwKIO4xY5RNTdJBIJ3ng0HA4KKQ5ml2H9sYtil0TUKQxY1GHXe2BxgTsRdZ+AXmokxIQBAN5MOYfS6nqRKyK6fQxY1CG6RgMulNUCAEK5BouIutnTo4IxwNcF5bUNeDOFvbHI+jBgUYfkltZAbxDgrJLD19VB7HKIyMYpZE29sSQSYMPxS9ifWSJ2SUS3hQGLOqRlejDUywkSiUTkaojIHtzZxwN/uCcQAPDKd2dQ18DeWGQ9GLCoQ1oWuIfyDkIi6kF/ie0PbxcVckpq8I8fM8Uuh6jDGLCoQ9iigYjE4OqgwOu/GQQA+GhXFjIKq0SuiKhjGLCoQ9iigYjE8mC4L8YO8EaDXsDL356GwcDeWGT5GLDolgwGwdhRmVewiKinSSQSLHosHGqlDEdyr+Lro/lil0R0SwxYdEuXyq+hrsEApUyKPr3UYpdDRHbI390RieOaemMlp5xDcRV7Y5FlEz1gffjhhwgKCoKDgwMiIyNx+PDhm44vLy9HXFwctFotVCoVwsLCkJKSYjLm0qVLePLJJ+Hp6QlHR0cMHjwYR48eNf78qaeegkQiMXk8+OCD3fL5bEHL+qsgjRpymeh/ZYjITj11bxAG+bmisq4RizefFbscopsS9bflV199hcTERCxcuBDHjx/H0KFDERsbi6KiojbH63Q6jBs3Drm5uVi/fj3S0tKwevVq+Pv7G8dcvXoVI0eOhEKhwJYtW3D27FksXboUHh4eJq/14IMP4sqVK8bHF1980a2f1Zpd7+DO6UEiEo+8uTeWVAJ8f+IydqcXi10SUbvkYr75smXLMHPmTMyYMQMAsHLlSmzevBlr1qzB/PnzW41fs2YNysrKsH//figUCgBAUFCQyZi3334bAQEBWLt2rfFYcHBwq9dSqVTw9fU146exXcYWDVzgTkQiG9LbHdPvDcLafbl49bsz2DY3Go5KmdhlEbUi2hUsnU6HY8eOISYm5noxUiliYmJw4MCBNp+zceNGREVFIS4uDj4+PggPD0dycjL0er3JmOHDh2PSpEnw9vbGnXfeidWrV7d6rZ9++gne3t7o378/Zs2ahdLS0pvWW19fj8rKSpOHvcjkFSwisiB/fqA/tG4OyCurxQc7M8Quh6hNogWskpIS6PV6+Pj4mBz38fFBQUFBm8/Jzs7G+vXrodfrkZKSgtdeew1Lly7F4sWLTcZ89NFH6NevH7Zt24ZZs2Zhzpw5WLdunXHMgw8+iH//+99ITU3F22+/jV27dmH8+PEmQe2XlixZAjc3N+MjICCgi38C1qPlDkJewSIiS+Cskht7Y63anY3zBfbzP7xkPSSCIIjSUOTy5cvw9/fH/v37ERUVZTz+0ksvYdeuXTh06FCr54SFhaGurg45OTmQyZouCS9btgzvvvsurly5AgBQKpUYPnw49u/fb3zenDlzcOTIkXavjGVnZyM0NBQ7duzA2LFj2xxTX1+P+vrrd61UVlYiICAAFRUVcHV1vf0/ACtRWl2PiMU7IJEAZ19/kJfiichiPPvvo/jv2ULc1ccd65+7F1Ipt/Gi7ldZWQk3N7db/v4X7QqWRqOBTCZDYWGhyfHCwsJ210ZptVqEhYUZwxUADBw4EAUFBdDpdMYxd9xxh8nzBg4ciLy8vHZrCQkJgUajQWZm+9swqFQquLq6mjzsQcv0oL+7I8MVEVmU1x8dBCelDMfzyvHFkfb/G08kBtECllKpREREBFJTU43HDAYDUlNTTa5o3WjkyJHIzMyEwWAwHktPT4dWq4VSqTSOSUtLM3leeno6AgMD263l4sWLKC0thVar7cpHskncIoeILJXWzRF/ie0PAHhry3kUVdaJXBHRdaK2aUhMTMTq1auxbt06nDt3DrNmzUJNTY3xrsJp06YhKSnJOH7WrFkoKytDfHw80tPTsXnzZiQnJyMuLs44JiEhAQcPHkRycjIyMzPx+eefY9WqVcYx1dXVePHFF3Hw4EHk5uYiNTUVjz76KPr27YvY2Nie/QOwAtwih4gs2bSoIAzp7YaqukYs2sTeWGQ5RG3TMHnyZBQXF2PBggUoKCjAsGHDsHXrVuPC97y8PEil1zNgQEAAtm3bhoSEBAwZMgT+/v6Ij4/HvHnzjGPuvvtufPvtt0hKSsKiRYsQHByM5cuXY+rUqQAAmUyGU6dOYd26dSgvL4efnx8eeOABvPHGG1CpVD37B2AFeAchEVkymVSC5AmD8ZsVe7Hp1BVMjCjC/f29xS6LSLxF7tauo4vcrN29S1JxuaIO65+LwvCgXmKXQ0TUpsWbzuLjvTnwd3fE9sRoqJWiXj8gG2bxi9zJ8tXUN+JyRdOaBrZoICJLljAuDP7ujrhUfg3v7WBvLBIfAxa1q6X/laeTEh5OSpGrISJqn5NKjkWPNvXG+nhvDs5eZm8sEhcDFrXLuEUO118RkRUYO9AHDw32hd4gIOnb09AbuAKGxMOARe3KYosGIrIyCx8ZBBeVHCfzy/HZoQtil0N2jAGL2sUWDURkbXxcHfDSg029sd7ZmoaCCvbGInEwYFG72KKBiKzRE5GBGBbgjur6Rrz+w89il0N2igGL2tSgN+BCaS0ABiwisi4yqQRLHh8MmVSCLWcKsONs4a2fRGRmDFjUpgulNWg0CHBSyqB1cxC7HCKi2zJQ64o/jg4GACz4/gxq6htFrojsDQMWtenGOwglEu5QT0TWJ35sP/T2cMTlijr8fXu62OWQnWHAojZxgTsRWTu1Uo43HgsHAKzZl4MzlypErojsCQMWtYk9sIjIFtzf3xu/HqKFQQCSNrA3FvUcBixqU2ZzDyxukUNE1m7BI3fAxUGO05cqsG5/rtjlkJ1gwKJWDAYBWUU1AHgHIRFZP28XB8wfPwAAsPS/abhcfk3kisgeMGBRK1cq63CtQQ+5VIJAT7XY5RARddmUu/sgItADNTo9Fm5kbyzqfgxY1ErL+qsgjRMUMv4VISLrJ5VKkDxhMORSCbafLcS2nwvELolsHH97Uiu8g5CIbFF/Xxf8aUwIAGDh9z+jqq5B5IrIljFgUSvcIoeIbNULv+qHQE81CirrsPS/7I1F3YcBi1rJYsAiIhvloJBhcXNvrHUHcnEyv1zcgshmMWBRKy0tGhiwiMgWje7nhceG+UFo7o3VqDeIXRLZIAYsMlFWo0NZjQ4AEOLlJHI1RETd49Vf3wE3RwXOXqnE2n25YpdDNogBi0y0rL/yd3eEWikXuRoiou6hcVbh5YeaemMt256Oi1drRa6IbA0DFpngAncisheTIgIwIqgXrjXoseD7nyEI3EaHzIcBi0xkcYscIrITUqkEyY+HQyGTYOf5Imw5w95YZD4MWGSCV7CIyJ709XbBrDGhAIC/bvwZleyNRWbCgEUmGLCIyN7Mvr8vgjVOKKqqx7tb08Quh2wEAxYZ1eoacal5E1QGLCKyFw4KGd5s7o316aELOJ53VeSKyBYwYJFRdnENAKCXkxK9nJQiV0NE1HPu7avBxLt6QxCAlzecRgN7Y1EXMWCREfcgJCJ79srDA+GhVuB8QRX+tTdH7HLIynUqYP3444/mroMsQEvACuX0IBHZoV5OSrzy8B0AgOU70pFfxt5Y1HmdClgPPvggQkNDsXjxYuTn55u7JhIJF7gTkb2beJc/okI8UddgwCvfnWFvLOq0TgWsS5cu4fnnn8f69esREhKC2NhYfP3119DpdOauj3oQ9yAkInsnkUjw5oRwKGVS7E4vxg+nrohdElmpTgUsjUaDhIQEnDhxAocOHUJYWBhmz54NPz8/zJkzBydPnjR3ndTNGvQG5JY0LXJnwCIiexbi5Yy4+/sCABb9cBYVteyNRbevy4vc77rrLiQlJeH5559HdXU11qxZg4iICIwePRo///yzOWqkHpBXVotGgwBHhQxaVwexyyEiEtVz94Ug1MsJJdX1eGvrebHLISvU6YDV0NCA9evX46GHHkJgYCC2bduGFStWoLCwEJmZmQgMDMSkSZPMWSt1o+sL3J0glUpEroaISFwquQzJEwYDAL44nIejuWUiV0TWplMB64UXXoBWq8Wf/vQnhIWF4X//+x8OHDiAP/7xj3ByckJQUBD+9re/4fx5pn5rwRYNRESmIkM88bvhvQEASRtOQ9fI3ljUcZ0KWGfPnsUHH3yAy5cvY/ny5QgPD281RqPRsJ2DFcniHYRERK28/NBAeDopkVFUjdV7ssUuh6xIpwJWamoqpkyZApVK1e4YuVyOMWPGdLow6lm8g5CIqDV3tRKv/nogAOD91AzjzUBEt9KpgLVkyRKsWbOm1fE1a9bg7bff7nJR1LMEQeAVLCKidjw2zB+j+mpQ32jAq+yNRR3UqYD1z3/+EwMGDGh1fNCgQVi5cmWXi6KedaWiDjU6PeRSCQI9ncQuh4jIokgkEix+LBwquRR7M0vw/YnLYpdEVqBTAaugoABarbbVcS8vL1y5wqZs1qZlgXugpxoKGbenJCL6pSCNE+aM7QcAeGPTWZTXsrE23VynfpsGBARg3759rY7v27cPfn5+XS6Keha3yCEiurWZo0MQ5uOM0hodlqTwLnm6uU4FrJkzZ2Lu3LlYu3YtLly4gAsXLmDNmjVISEjAzJkzzV0jdTMucCciujWlXGrsjfXV0Xwcyi4VuSKyZPLOPOnFF19EaWkpZs+ebdx/0MHBAfPmzUNSUpJZC6Tu17LAPZQ9sIiIbmp4UC9MGdEHXxzOw8vfnsaW+Ggo5VxaQa116m+FRCLB22+/jeLiYhw8eBAnT55EWVkZFixYYO76qAdk8QoWEVGHzX9wADydlMgqrsGu9GKxyyEL1aXY7ezsjLvvvhvh4eE37YlFlqu8VoeS6qarkLyCRUR0a25qBcYP9gUA7MlgwKK2dWqKEACOHj2Kr7/+Gnl5ecZpwhYbNmzocmHUM1oWuPu5OcBJ1em/DkREdiW6nxc+PZiH3byCRe3o1BWsL7/8Evfeey/OnTuHb7/9Fg0NDfj555+xc+dOuLm5mbtG6kbXN3nm1Ssioo6KCvWEXCpBbmkt8kprxS6HLFCnAlZycjL+/ve/44cffoBSqcR7772H8+fP43e/+x369Olj7hqpG7FFAxHR7XNxUOCuPh4AgN2cJqQ2dCpgZWVl4eGHHwYAKJVK1NTUQCKRICEhAatWrTJrgdS92KKBiKhzRvfTAOA6LGpbpwKWh4cHqqqqAAD+/v44c+YMAKC8vBy1tbxUak2MV7C4wJ2I6LZEh3kBAPZnlqJBbxC5GrI0nQpY0dHR2L59OwBg0qRJiI+Px8yZMzFlyhSMHTvWrAVS97mm0+NS+TUAvIJFRHS7wv3d4K5WoKq+ESfzy8UuhyxMp24bW7FiBerq6gAAr7zyChQKBfbv34+JEyfi1VdfNWuB1H2yiqshCICHWgFPZ7bZICK6HTKpBKP6arDp1BXszijB8KBeYpdEFuS2r2A1NjZi06ZNkMlkTS8glWL+/PnYuHEjli5dCg8Pj9t6vQ8//BBBQUFwcHBAZGQkDh8+fNPx5eXliIuLg1arhUqlQlhYGFJSUkzGXLp0CU8++SQ8PT3h6OiIwYMH4+jRo8afC4KABQsWQKvVwtHRETExMcjIyLitum1BS4NR9r8iIuqc6H5N04Rs10C/dNsBSy6X47nnnjNeweqKr776ComJiVi4cCGOHz+OoUOHIjY2FkVFRW2O1+l0GDduHHJzc7F+/XqkpaVh9erV8Pf3N465evUqRo4cCYVCgS1btuDs2bOtgt8777yD999/HytXrsShQ4fg5OSE2NhYs3wma5LFOwiJiLpkdFjTQvdTF8tRXqu7xWiyJ52aIhwxYgROnDiBwMDALr35smXLMHPmTMyYMQMAsHLlSmzevBlr1qzB/PnzW41fs2YNysrKsH//figUCgBAUFCQyZi3334bAQEBWLt2rfFYcHCw8Z8FQcDy5cvx6quv4tFHHwUA/Pvf/4aPjw++++47/P73v+/SZ7ImvIOQiKhrtG6O6OftjIyiauzLLMXDQ7Ril0QWolOL3GfPno3ExESsWLECBw4cwKlTp0weHaHT6XDs2DHExMRcL0YqRUxMDA4cONDmczZu3IioqCjExcXBx8cH4eHhSE5Ohl6vNxkzfPhwTJo0Cd7e3rjzzjuxevVq489zcnJQUFBg8r5ubm6IjIxs930BoL6+HpWVlSYPa8cmo0REXddyNyHbNdCNOnUFq+Uqz5w5c4zHJBIJBEGARCIxCTztKSkpgV6vh4+Pj8lxHx8fnD9/vs3nZGdnY+fOnZg6dSpSUlKQmZmJ2bNno6GhAQsXLjSO+eijj5CYmIiXX34ZR44cwZw5c6BUKjF9+nQUFBQY3+eX79vys7YsWbIEr7/++i0/l7Vo1BuQU1IDgC0aiIi6YnQ/Df61Nwe704uNvweJOhWwcnJyzF1HhxgMBnh7e2PVqlWQyWSIiIjApUuX8O677xoDlsFgwPDhw5GcnAwAuPPOO3HmzBmsXLkS06dP7/R7JyUlITEx0fh9ZWUlAgICuvaBRJRXVosGvQBHhQz+7o5il0NEZLUigz2hlEtxuaIOWcU1XHZBADoZsLq69goANBoNZDIZCgsLTY4XFhbC19e3zedotVooFArjHYwAMHDgQBQUFECn00GpVEKr1eKOO+4wed7AgQPxf//3fwBgfO3CwkJotdfnygsLCzFs2LB261WpVFCpbKeVQcv0YIiXE6RS/t8WEVFnOSplGBHUC3szS7A7vZgBiwB0MmD9+9//vunPp02bdsvXUCqViIiIQGpqKh577DEATVefUlNT8fzzz7f5nJEjR+Lzzz+HwWCAVNq0fCw9PR1arRZKpdI4Ji0tzeR56enpxlAYHBwMX19fpKamGgNVZWUlDh06hFmzZt2yblvBBe5EROYTHabB3swS7MkoxtOjgm/9BLJ5nQpY8fHxJt83NDSgtrYWSqUSarW6QwELABITEzF9+nQMHz4cI0aMwPLly1FTU2O8q3DatGnw9/fHkiVLAACzZs3CihUrEB8fjxdeeAEZGRlITk42WQuWkJCAe++9F8nJyfjd736Hw4cPY9WqVcY9EiUSCebOnYvFixejX79+CA4OxmuvvQY/Pz9j0LMH3CKHiMh8RvfzAnAeB7PLUN+oh0ouu+VzyLZ1KmBdvXq11bGMjAzMmjULL774YodfZ/LkySguLsaCBQtQUFCAYcOGYevWrcYF6Hl5ecYrVQAQEBCAbdu2ISEhAUOGDIG/vz/i4+Mxb94845i7774b3377LZKSkrBo0SIEBwdj+fLlmDp1qnHMSy+9hJqaGjz77LMoLy/HqFGjsHXrVjg4OHTmj8MqsQcWEZH5DPB1gZeLCsVV9TiWexX39tWIXRKJTCIIgmCuFzt69CiefPLJdu8CtCWVlZVwc3NDRUUFXF1dxS7ntgiCgMF//S+q6xuxPSEa/XxcxC6JiMjqJX59AhuOX8KfxoQgafxAscuhbtLR3/+d6oPVHrlcjsuXL5vzJakbFFbWo7q+ETKpBIGeTmKXQ0RkE1q2zdmTXiJyJWQJOjVFuHHjRpPvBUHAlStXsGLFCowcOdIshVH3aVl/FdhLDaXcrBmbiMhujerXNC149koliqvq4eViO3ee0+3rVMD65WJwiUQCLy8v/OpXv8LSpUvNURd1o8yiKgDs4E5EZE4aZxUG+bni58uV2JtZjAl39ha7JBJRpwKWwWAwdx3Ug9iigYioe0SHeeHny5XYnV7CgGXnOD9kh9iigYioe4xunibck1ECg8Fs95CRFepUwJo4cSLefvvtVsffeecdTJo0qctFUffKLGreg5BXsIiIzCoi0ANqpQwl1fU4V1Apdjkkok4FrN27d+Ohhx5qdXz8+PHYvXt3l4ui7lNR24CS6noAXINFRGRuKrkM94R4Ami6ikX2q1MBq7q62rg1zY0UCgUqK5nYLVlmcdMCd62bA5xVnVqCR0RENxFtnCYsFrkSElOnAtbgwYPx1VdftTr+5ZdfttpomSxLJju4ExF1q9FhTf2wjuRcRa2uUeRqSCyduoTx2muv4fHHH0dWVhZ+9atfAQBSU1PxxRdf4JtvvjFrgWReWcVN669CucCdiKhbhGic4O/uiEvl13Aopwz39/cWuyQSQaeuYD3yyCP47rvvkJmZidmzZ+PPf/4zLl68iB07dtjVhsnWqOUKFtdfERF1D4lEguiwpmnC3emcJrRXnV6E8/DDD+Phhx82Zy3UA9iigYio+0X388IXh/O50N2OdeoK1pEjR3Do0KFWxw8dOoSjR492uSjqHnUNeuRfrQXANVhERN3p3lANpJKm/6m9XH5N7HJIBJ0KWHFxccjPz291/NKlS4iLi+tyUdQ9sotrIAiAm6MCGufWd4ESEZF5uKkVGBbgDoB3E9qrTgWss2fP4q677mp1/M4778TZs2e7XBR1jxu3yJFIJCJXQ0Rk20b3a7qbcHc6pwntUacClkqlQmFhYavjV65cgVzO3kqWiuuviIh6TnRzu4a9mSXQc9scu9OpgPXAAw8gKSkJFRUVxmPl5eV4+eWXMW7cOLMVR+aVxR5YREQ9ZmhvN7g4yFFxrQGnLpaLXQ71sE4FrL/97W/Iz89HYGAg7r//ftx///0IDg5GQUEBli5dau4ayUzYZJSIqOfIZVKM6nt982eyL50KWP7+/jh16hTeeecd3HHHHYiIiMB7772H06dPIyAgwNw1khk06g3IKeEmz0REPen6OiwudLc3nV4w5eTkhFGjRqFPnz7Q6XQAgC1btgAAfvOb35inOjKbi1evQac3QCWXwt/dUexyiIjswujmfQn/l1+OyroGuDooRK6IekqnAlZ2djYmTJiA06dPQyKRQBAEk7vS9Hq92Qok82iZHgzxcoZUyjsIiYh6QkAvNUI0TsguqcH+zFI8GO4rdknUQzo1RRgfH4/g4GAUFRVBrVbjzJkz2LVrF4YPH46ffvrJzCWSOdzYooGIiHpOy1Us9sOyL50KWAcOHMCiRYug0WgglUohk8kwatQoLFmyBHPmzDF3jWQGbNFARCSOlnYNuzOKIQhs12AvOhWw9Ho9XFxcAAAajQaXL18GAAQGBiItLc181ZHZ8A5CIiJx3BPiCYVMgvyya7hQWit2OdRDOhWwwsPDcfLkSQBAZGQk3nnnHezbtw+LFi1CSEiIWQukrhMEgT2wiIhE4qSSIyLQA0DTVSyyD50KWK+++ioMBgMAYNGiRcjJycHo0aORkpKC999/36wFUtcVVdWjqr4RUgkQpFGLXQ4Rkd3htjn2p1N3EcbGxhr/uW/fvjh//jzKysrg4eHBPe4sUMv0YKCnE1RymcjVEBHZnzFhXnh3WxoOZJWgQW+AQtap6xtkRcx2hnv16sVwZaFaAlYoF7gTEYniDq0rPJ2UqNHpcfzCVbHLoR7ACG0HstiigYhIVFKpBKP6cdsce8KAZQd4ByERkfiM67C40N0uMGDZgetThE4iV0JEZL+im69gnb5UgbIancjVUHdjwLJxlXUNKKqqBwCE8goWEZFovF0dMMDXBYIA7M3kNKGtY8CycS1Xr3xcVdxklIhIZC1d3fekc5rQ1jFg2TiuvyIishwt+xJy2xzbx4Bl47K4ByERkcW4O6gXVHIpCivrkdH832eyTQxYNo5XsIiILIeDQobIEE8AwG5OE9o0Biwbl9ncA4sL3ImILEO0cZqQC91tGQOWDatr0CO/rGnndl7BIiKyDC0L3Q9ll6KuQS9yNdRdGLBsWG5pDQwC4Oogh5ezSuxyiIgIQD9vZ/i6OqC+0YAjuWVil0PdhAHLht24/or7RBIRWQaJRHL9bkKuw7JZDFg2jAvciYgs0+iWflhch2WzGLBs2PUtchiwiIgsyai+GkgkwPmCKhRW1oldDnUDBiwbxitYRESWqZeTEoP93QDwKpatYsCyUXqDgOySGgAMWERElii6X9M0Iddh2SYGLBt18WotdI0GKOVS9PZQi10OERH9QstC972ZJTAYuG2OrWHAslEt04MhGifIpLyDkIjI0twV6AEnpQxlNTqcvVIpdjlkZgxYNorrr4iILJtCJkVUaNNVrF2cJrQ5DFg2igGLiMjyjQlrClh7MhiwbA0Dlo3KKmbAIiKydKObF7ofu3AVNfWNIldD5sSAZYMEQeAVLCIiKxCkcUKfXmo06AUczC4VuxwyIwYsG1RcXY/KukZIJUCQp5PY5RAR0U1w2xzbxIBlg1quXgX0UsNBIRO5GiIiuplobptjkxiwbFBWy/Qgt8ghIrJ4UaGekEklyC6pQX5ZrdjlkJlYRMD68MMPERQUBAcHB0RGRuLw4cM3HV9eXo64uDhotVqoVCqEhYUhJSXF+PO//vWvkEgkJo8BAwaYvMZ9993Xasxzzz3XLZ+vp3H9FRGR9XB1UOCuPu4AeBXLlsjFLuCrr75CYmIiVq5cicjISCxfvhyxsbFIS0uDt7d3q/E6nQ7jxo2Dt7c31q9fD39/f1y4cAHu7u4m4wYNGoQdO3YYv5fLW3/UmTNnYtGiRcbv1Wrb6Hie2XwHYSgDFhGRVRjdzwtHcq9id3oxnojsI3Y5ZAaiB6xly5Zh5syZmDFjBgBg5cqV2Lx5M9asWYP58+e3Gr9mzRqUlZVh//79UCgUAICgoKBW4+RyOXx9fW/63mq1+pZjrBGvYBERWZfoMC8s256OfVklaNQbIJdZxAQTdYGoZ1Cn0+HYsWOIiYkxHpNKpYiJicGBAwfafM7GjRsRFRWFuLg4+Pj4IDw8HMnJydDr9SbjMjIy4Ofnh5CQEEydOhV5eXmtXuuzzz6DRqNBeHg4kpKSUFvb/tx3fX09KisrTR6WqLKuAYWV9QAYsIiIrMVgfze4qxWoqmvEyYvlYpdDZiBqwCopKYFer4ePj4/JcR8fHxQUFLT5nOzsbKxfvx56vR4pKSl47bXXsHTpUixevNg4JjIyEp988gm2bt2Kjz76CDk5ORg9ejSqqqqMY5544gl8+umn+PHHH5GUlIT//Oc/ePLJJ9utdcmSJXBzczM+AgICuvjpu0fLAndvFxVcHRQiV0NERB0hk0owsm9Luwauw7IFok8R3i6DwQBvb2+sWrUKMpkMERERuHTpEt59910sXLgQADB+/Hjj+CFDhiAyMhKBgYH4+uuv8cwzzwAAnn32WeOYwYMHQ6vVYuzYscjKykJoaGir901KSkJiYqLx+8rKSosMWVnFNQB49YqIyNpE99Ng86kr2J1RjIRxYWKXQ10kasDSaDSQyWQoLCw0OV5YWNju2iitVguFQgGZ7Hp/p4EDB6KgoAA6nQ5KpbLVc9zd3REWFobMzMx2a4mMjAQAZGZmthmwVCoVVCpVhz6XmLj+iojIOrVsm3MyvxwVtQ1wU3MWwpqJOkWoVCoRERGB1NRU4zGDwYDU1FRERUW1+ZyRI0ciMzMTBoPBeCw9PR1arbbNcAUA1dXVyMrKglarbbeWEydOAMBNx1gDBiwiIuvk5+6Ivt7OMAjAvixOE1o70W9TSExMxOrVq7Fu3TqcO3cOs2bNQk1NjfGuwmnTpiEpKck4ftasWSgrK0N8fDzS09OxefNmJCcnIy4uzjjmL3/5C3bt2oXc3Fzs378fEyZMgEwmw5QpUwAAWVlZeOONN3Ds2DHk5uZi48aNmDZtGqKjozFkyJCe/QMws5ZNnkPZZJSIyOq0bJuzJ4Pb5lg70ddgTZ48GcXFxViwYAEKCgowbNgwbN261bjwPS8vD1Lp9RwYEBCAbdu2ISEhAUOGDIG/vz/i4+Mxb94845iLFy9iypQpKC0thZeXF0aNGoWDBw/Cy6vp8qtSqcSOHTuwfPly1NTUICAgABMnTsSrr77asx/ezOob9bhQyjVYRETWKjrMC2v35WJ3egkEQYBEIhG7JOokiSAIgthFWKPKykq4ubmhoqICrq6uYpcDAEgrqELs8t1wUclx6q8P8F9MIiIrU6trxLDXt0OnNyD1z2M4G2GBOvr7X/QpQjKflvVXod7ODFdERFZIrZTj7mAPAMCedE4TWjMGLBvCBe5ERNav5W7C3dyX0KoxYNmQlj0IGbCIiKxXdHPAOpBVivpG/S1Gk6ViwLIhxitYnLMnIrJaA3xdoHFW4VqDHscuXBW7HOokBiwbYTAIyOYVLCIiqyeVShBtbNfAaUJrxYBlIy6VX0N9owFKuRQBvdRil0NERF0wOqxlX0IudLdWDFg2omV6METjBJmUdxASEVmzUX2b1mH9fLkSJdX1IldDncGAZSOMLRq4/oqIyOp5uahwh7apx9JeThNaJQYsG3FjDywiIrJ+0WEt7Ro4TWiNGLBsBFs0EBHZlhsXunPTFevDgGUDBEFgiwYiIhsTEeQBR4UMxVX1OF9QJXY5dJsYsGxASbUOFdcaIJEAIV5OYpdDRERmoJLLcE9ILwC8m9AaMWDZgJarVwEeajgoZCJXQ0RE5tKyDov9sKwPA5YN4PorIiLb1LIv4eHcMlzTcdsca8KAZQOyuMkzEZFNCvVygp+bA3SNBhzKKRW7HLoNDFg2IKuYC9yJiGyRRCK53q4hndOE1oQBywawBxYRke1qmSbcw35YVoUBy8pV1zfiSkUdAF7BIiKyRSP7ekIqATKKqnG5/JrY5VAHMWBZuZb1VxpnFdzUCpGrISIic3NXKzGktzsAbptjTRiwrJyxwag3+18REdkqbptjfRiwrBxbNBAR2b6WbXP2ZpZAb+C2OdaAAcvKcYscIiLbNyzAHS4qOcprG3DmUoXY5VAHMGBZues9sFxEroSIiLqLXCbFvX09AXDbHGvBgGXFdI0GXCirBcApQiIiW8dtc6wLA5YVu1BaA71BgLNKDh9XldjlEBFRN4pu7od1PO8qquoaRK6GboUBy4rd2GBUIpGIXA0REXWngF5qBGuc0GgQcCCL2+ZYOgYsK8YF7kRE9mV0892EbNdg+RiwrBhbNBAR2ZfoflyHZS0YsKyYcYrQi01GiYjswT2hnpBLJbhQWosLpTVil0M3wYBlpQwGAVm8gkVEZFecVXJEBHoAAHbzKpZFY8CyUpfKr6GuwQClTIo+vdRil0NERD3EuG0O+2FZNAYsK9Wy/ipIo4ZcxtNIRGQvWtZhHcgqRYPeIHI11B7+ZrZS1zu4c3qQiMieDPJzRS8nJarrG/G/vHKxy6F2MGBZKeP6K7ZoICKyK1KpBKP6NrVr2MN2DRaLActK3dhklIiI7IuxHxbXYVksBiwrlckpQiIiuzW6eR3WqUsVuFqjE7kaagsDlhUqra7H1doGSCRAKKcIiYjsjq+bA/r7uEAQgH1ZbNdgiRiwrFDL1aveHo5wUMhEroaIiMTAaULLxoBlhVpaNPDqFRGR/Wrph7UnowSCIIhcDf0SA5YV4ibPREQ0IrgXVHIprlTUGX8vkOVgwLJCXOBOREQOChlGBPcCwG1zLBEDlhVik1EiIgKud3XnOizLw4BlZWrqG3G5og4AAxYRkb1rWYd1KKcUdQ16kauhGzFgWZns4hoAgMZZCXe1UuRqiIhITGE+zvBxVaGuwYCjuVfFLoduwIBlZTKLqwDwDkIiIgIkEomx6Si3zbEsDFhWhgvciYjoRi39sHZxHZZFYcCyMgxYRER0o9H9vCCRAOcLqlBUWSd2OdSMAcvKMGAREdGNejkpEe7nBqCp6ShZBgYsK9KgN+BCaS0ArsEiIqLrosOapgm5DstyMGBZkQulNWg0CHBSyqB1cxC7HCIishDXF7qXwGDgtjmWgAHLirRMD4Z6O0MikYhcDRERWYq7+njASSlDaY0OZ69Uil0OgQHLqnAPQiIiaotSLkVUqCcAYDenCS2CRQSsDz/8EEFBQXBwcEBkZCQOHz580/Hl5eWIi4uDVquFSqVCWFgYUlJSjD//61//ColEYvIYMGCAyWvU1dUhLi4Onp6ecHZ2xsSJE1FYWNgtn89cbryCRUREdKOWru570rnQ3RKIHrC++uorJCYmYuHChTh+/DiGDh2K2NhYFBUVtTlep9Nh3LhxyM3Nxfr165GWlobVq1fD39/fZNygQYNw5coV42Pv3r0mP09ISMAPP/yAb775Brt27cLly5fx+OOPd9vnNIes5i7uvIOQiIh+qWUd1tELZaipbxS5GpKLXcCyZcswc+ZMzJgxAwCwcuVKbN68GWvWrMH8+fNbjV+zZg3Kysqwf/9+KBQKAEBQUFCrcXK5HL6+vm2+Z0VFBf71r3/h888/x69+9SsAwNq1azFw4EAcPHgQ99xzj5k+nfkYDAKyitmigYiI2hbkqUZvD0dcvHoNh3JK8asBPmKXZNdEvYKl0+lw7NgxxMTEGI9JpVLExMTgwIEDbT5n48aNiIqKQlxcHHx8fBAeHo7k5GTo9aabXGZkZMDPzw8hISGYOnUq8vLyjD87duwYGhoaTN53wIAB6NOnT7vvW19fj8rKSpNHT7pSWYdanR4KmQSBvdQ9+t5ERGT5JBKJcZpwN6cJRSdqwCopKYFer4ePj2nK9vHxQUFBQZvPyc7Oxvr166HX65GSkoLXXnsNS5cuxeLFi41jIiMj8cknn2Dr1q346KOPkJOTg9GjR6Oqqmkfv4KCAiiVSri7u3f4fZcsWQI3NzfjIyAgoAuf/Pa1rL8K8nSCXCb6zC4REVmg6OZtc7jQXXxW95vaYDDA29sbq1atQkREBCZPnoxXXnkFK1euNI4ZP348Jk2ahCFDhiA2NhYpKSkoLy/H119/3en3TUpKQkVFhfGRn59vjo/TYezgTkREtxIVqoFMKkF2cQ0uXq0Vuxy7JmrA0mg0kMlkre7eKywsbHf9lFarRVhYGGQymfHYwIEDUVBQAJ1O1+Zz3N3dERYWhszMTACAr68vdDodysvLO/y+KpUKrq6uJo+exIBFRES34uaowLAAdwDcNkdsogYspVKJiIgIpKamGo8ZDAakpqYiKiqqzeeMHDkSmZmZMBgMxmPp6enQarVQKpVtPqe6uhpZWVnQarUAgIiICCgUCpP3TUtLQ15eXrvvK7aslhYN7IFFREQ3EW3s6s5pQjGJPkWYmJiI1atXY926dTh37hxmzZqFmpoa412F06ZNQ1JSknH8rFmzUFZWhvj4eKSnp2Pz5s1ITk5GXFycccxf/vIX7Nq1C7m5udi/fz8mTJgAmUyGKVOmAADc3NzwzDPPIDExET/++COOHTuGGTNmICoqyiLvIASATN5BSEREHTC6eV/CvRklaNQbbjGauovobRomT56M4uJiLFiwAAUFBRg2bBi2bt1qXPiel5cHqfR6DgwICMC2bduQkJCAIUOGwN/fH/Hx8Zg3b55xzMWLFzFlyhSUlpbCy8sLo0aNwsGDB+Hl5WUc8/e//x1SqRQTJ05EfX09YmNj8Y9//KPnPvhtKKvRoaymafozxMtJ5GqIiMiSDe3tDlcHOSrrGnHqUgXu6uMhdkl2SSIIAneF7ITKykq4ubmhoqKi29djHc4pw+/+eQD+7o7YN/9X3fpeRERk/WZ/dgwppwswN6Yf5saEiV2OTeno73/Rpwjp1thglIiIbsf1dVhc6C4WBiwrwDsIiYjodoxubjh6Ir8cFdcaRK7GPjFgWQEGLCIiuh3+7o4I9XKC3iDgQBavYomBAcsKMGAREdHtatn8eRe3zREFA5aFq9U14lL5NQBAX/bAIiKiDhpj3JewGLyfrecxYFm47OIaAICnkxIeTm03UiUiIvqlyJBeUMqkuFR+DTklNWKXY3cYsCxcJju4ExFRJ6iVcgwPauqBxbsJex4DloUzBiyuvyIiotvUsg5rdzq3zelpDFgWjgvciYios6Kbt805kF0KXSO3zelJDFgWjnsQEhFRZw30dYXGWYlanR7HLlwVuxy7woBlwRr0BlwobVqYyIBFRES3SyqVGKcJ92RwmrAnMWBZsLyyWjToBaiVMvi5OYhdDhERWaHR/ZqmCbnQvWcxYFmwG+8glEgkIldDRETWaFRzwDpzuQKl1fUiV2M/GLAsGBe4ExFRV3m7OGCg1hWCAOzN5FWsnsKAZcGyGLCIiMgMopuvYu3mtjk9hgHLgrXcQcgmo0RE1BXRYdcXunPbnJ7BgGWhBEG44QqWk8jVEBGRNYsI9ICDQoqiqnqkFVaJXY5dYMCyUFcq6lCj00MulSDQkwGLiIg6z0Ehwz0hngCAPZwm7BEMWBaqZYF7oKcaChlPExERdY1x2xz2w+oR/M1toXgHIRERmdOY5m1zDuWUoa5BL3I1to8By0JlcYscIiIyo1AvZ2jdHKBrNOBQTpnY5dg8BiwLxStYRERkThKJBNEt2+akc5qwuzFgWSjjFSwvF5ErISIiWzG6eZqQ67C6HwOWBSqv1aGkWgcACGWLBiIiMpNRfTWQSID0wmoUVNSJXY5NY8CyQC3Tg/7ujlAr5SJXQ0REtsJdrcSQ3u4AeBWruzFgWSDjJs9cf0VERGY2pnnbnD0Z7IfVnRiwLJAxYHlxepCIiMxrdPO2OXsziqE3cNuc7sKAZYEy2aKBiIi6ybAAd7io5Lha24CfL1eIXY7NYsCyQMYWDdzkmYiIzEwhkyIqtGnbnN1s19BtGLAsjCAImHCnP2IH+aCfD1s0EBGR+UWHtWybw3VY3YW3qFkYiUSCPz/QX+wyiIjIhrU0HD1+4Sqq6xvhrGIcMDf+iRIREdmZPp5qBHqqcaG0FgeySjHuDh+xS+oSvUFAflkt0gqrkFFYhbTCatwZ4I6nRwWLVhMDFhERkR2K7ueF/5RewJ6MYqsJWIIgoKCyDmkFVUgvrEJaQTXSC6uQUVSFugaDydjqugYGLCIiIupZo/tp8J+DFyx2oXtZje56kCqsQnpB09equsY2x6vkUvTzcUaYjwv6+7hgaIB7zxb8CwxYREREdigq1BNyqQS5pbXIK61FH0+1KHVU1zci/YYA1XJlqqS6vs3xMqkEIRonhPk2BakwHxf093VBn15qyKSSHq6+fQxYREREdsjFQYG7+njgcG4ZdmcU40nPwG59v7oGPbKKq02m9tIKqnCp/Fq7z+nTS90coJyNQSpY4wSVXNattZoDAxYREZGdig7T4HBuGfZkFOPJe8wTsBr1BuSW1hoDVMsUX25JDdprHO/jqjJO7bVcmerr7QwnK7670XorJyIioi4Z3c8Lf/tvOvZnlqJBb4BC1vH2mAaDgEvl136xRqoaWUXV0OkNbT7HXa1oFaTCfJzhrlaa6yNZDAYsIiIiOxXu7wYPtQJXaxtwMr8cw4N6tRojCAKKq+uRXlBtstg8o7AKNTp9m6+rVsrQz8cF/X2uT+3193GBl4sKEonlrJPqTgxYREREdkomlWBkXw02nbqC3enF6OftgvSiG6b2mr9erW1o8/lKmRQhXk7o7+tivDLV39cF/u6OkFrQgnMxMGARERHZsegwL2w6dQUf/pSF93dmtjlGKgGCPJ0QdsPUXn9fZwR6Ot3WtKI9YcAiIiKyY/eFecFBITU26vR3d7x+Rar57r1QL2c4KCz/zj1LwoBFRERkx7xdHbAlPhplNTqE+TjDxUEhdkk2gQGLiIjIzgVrnBCscRK7DJvCiVMiIiIiM2PAIiIiIjIzBiwiIiIiM2PAIiIiIjIzBiwiIiIiM2PAIiIiIjIzBiwiIiIiM2PAIiIiIjIzBiwiIiIiM7OIgPXhhx8iKCgIDg4OiIyMxOHDh286vry8HHFxcdBqtVCpVAgLC0NKSkqbY9966y1IJBLMnTvX5Ph9990HiURi8njuuefM9ZGIiIjIjom+Vc5XX32FxMRErFy5EpGRkVi+fDliY2ORlpYGb2/vVuN1Oh3GjRsHb29vrF+/Hv7+/rhw4QLc3d1bjT1y5Aj++c9/YsiQIW2+98yZM7Fo0SLj92q12myfi4iIiOyX6Fewli1bhpkzZ2LGjBm44447sHLlSqjVaqxZs6bN8WvWrEFZWRm+++47jBw5EkFBQRgzZgyGDh1qMq66uhpTp07F6tWr4eHh0eZrqdVq+Pr6Gh+urq5m/3xERERkf0QNWDqdDseOHUNMTIzxmFQqRUxMDA4cONDmczZu3IioqCjExcXBx8cH4eHhSE5Ohl6vNxkXFxeHhx9+2OS1f+mzzz6DRqNBeHg4kpKSUFtb2+7Y+vp6VFZWmjyIiIiI2iLqFGFJSQn0ej18fHxMjvv4+OD8+fNtPic7Oxs7d+7E1KlTkZKSgszMTMyePRsNDQ1YuHAhAODLL7/E8ePHceTIkXbf+4knnkBgYCD8/Pxw6tQpzJs3D2lpadiwYUOb45csWYLXX3+9k5+UiIiI7Inoa7Bul8FggLe3N1atWgWZTIaIiAhcunQJ7777LhYuXIj8/HzEx8dj+/btcHBwaPd1nn32WeM/Dx48GFqtFmPHjkVWVhZCQ0NbjU9KSkJiYqLx+4qKCvTp04dXsoiIiOxIy+99QRBuOk7UgKXRaCCTyVBYWGhyvLCwEL6+vm0+R6vVQqFQQCaTGY8NHDgQBQUFxinHoqIi3HXXXcaf6/V67N69GytWrEB9fb3Jc1tERkYCADIzM9sMWCqVCiqVyvh9yx9wQEDAbXxiIiIisgVVVVVwc3Nr9+eiBiylUomIiAikpqbiscceA9B0hSo1NRXPP/98m88ZOXIkPv/8cxgMBkilTUvI0tPTodVqoVQqMXbsWJw+fdrkOTNmzMCAAQMwb968NsMVAJw4cQJAU4DrCD8/P+Tn58PFxQUSiaRDz+moyspKBAQEID8/nwvvLRDPj+XiubFcPDeWjeen4wRBQFVVFfz8/G46TvQpwsTEREyfPh3Dhw/HiBEjsHz5ctTU1GDGjBkAgGnTpsHf3x9LliwBAMyaNQsrVqxAfHw8XnjhBWRkZCA5ORlz5swBALi4uCA8PNzkPZycnODp6Wk8npWVhc8//xwPPfQQPD09cerUKSQkJCA6Orrdlg6/JJVK0bt3b3P9MbTJ1dWVf9EtGM+P5eK5sVw8N5aN56djbnblqoXoAWvy5MkoLi7GggULUFBQgGHDhmHr1q3Ghe95eXnGK1VA05Tctm3bkJCQgCFDhsDf3x/x8fGYN29eh99TqVRix44dxjAXEBCAiRMn4tVXXzX75yMiIiL7IxFutUqLelxlZSXc3NxQUVHB/5OwQDw/lovnxnLx3Fg2nh/zE73RKLWmUqmwcOFCk0X1ZDl4fiwXz43l4rmxbDw/5scrWERERERmxitYRERERGbGgEVERERkZgxYRERERGbGgEVERERkZgxYFujDDz9EUFAQHBwcEBkZicOHD4tdkt1ZsmQJ7r77bri4uMDb2xuPPfYY0tLSTMbU1dUhLi4Onp6ecHZ2xsSJE1tt+0Td76233oJEIsHcuXONx3huxHXp0iU8+eST8PT0hKOjIwYPHoyjR48afy4IAhYsWACtVgtHR0fExMQgIyNDxIrtg16vx2uvvYbg4GA4OjoiNDQUb7zxhsmeejw35sOAZWG++uorJCYmYuHChTh+/DiGDh2K2NhYFBUViV2aXdm1axfi4uJw8OBBbN++HQ0NDXjggQdQU1NjHJOQkIAffvgB33zzDXbt2oXLly/j8ccfF7Fq+3PkyBH885//bLUDA8+NeK5evYqRI0dCoVBgy5YtOHv2LJYuXQoPDw/jmHfeeQfvv/8+Vq5ciUOHDsHJyQmxsbGoq6sTsXLb9/bbb+Ojjz7CihUrcO7cObz99tt455138MEHHxjH8NyYkUAWZcSIEUJcXJzxe71eL/j5+QlLliwRsSoqKioSAAi7du0SBEEQysvLBYVCIXzzzTfGMefOnRMACAcOHBCrTLtSVVUl9OvXT9i+fbswZswYIT4+XhAEnhuxzZs3Txg1alS7PzcYDIKvr6/w7rvvGo+Vl5cLKpVK+OKLL3qiRLv18MMPC08//bTJsccff1yYOnWqIAg8N+bGK1gWRKfT4dixY4iJiTEek0qliImJwYEDB0SsjCoqKgAAvXr1AgAcO3YMDQ0NJudqwIAB6NOnD89VD4mLi8PDDz9scg4Anhuxbdy4EcOHD8ekSZPg7e2NO++8E6tXrzb+PCcnBwUFBSbnx83NDZGRkTw/3ezee+9Famoq0tPTAQAnT57E3r17MX78eAA8N+Ym+l6EdF1JSQn0er1xH8YWPj4+OH/+vEhVkcFgwNy5czFy5EjjhuEFBQVQKpVwd3c3Gevj44OCggIRqrQvX375JY4fP44jR460+hnPjbiys7Px0UcfITExES+//DKOHDmCOXPmQKlUYvr06cZz0NZ/53h+utf8+fNRWVmJAQMGQCaTQa/X480338TUqVMBgOfGzBiwiG4hLi4OZ86cwd69e8UuhQDk5+cjPj4e27dvh4ODg9jl0C8YDAYMHz4cycnJAIA777wTZ86cwcqVKzF9+nSRq7NvX3/9NT777DN8/vnnGDRoEE6cOIG5c+fCz8+P56YbcIrQgmg0GshkslZ3OxUWFsLX11ekquzb888/j02bNuHHH39E7969jcd9fX2h0+lQXl5uMp7nqvsdO3YMRUVFuOuuuyCXyyGXy7Fr1y68//77kMvl8PHx4bkRkVarxR133GFybODAgcjLywMA4zngf+d63osvvoj58+fj97//PQYPHow//OEPSEhIwJIlSwDw3JgbA5YFUSqViIiIQGpqqvGYwWBAamoqoqKiRKzM/giCgOeffx7ffvstdu7cieDgYJOfR0REQKFQmJyrtLQ05OXl8Vx1s7Fjx+L06dM4ceKE8TF8+HBMnTrV+M88N+IZOXJkq5Ym6enpCAwMBAAEBwfD19fX5PxUVlbi0KFDPD/drLa2FlKp6a99mUwGg8EAgOfG7MReZU+mvvzyS0GlUgmffPKJcPbsWeHZZ58V3N3dhYKCArFLsyuzZs0S3NzchJ9++km4cuWK8VFbW2sc89xzzwl9+vQRdu7cKRw9elSIiooSoqKiRKzaft14F6Eg8NyI6fDhw4JcLhfefPNNISMjQ/jss88EtVotfPrpp8Yxb731luDu7i58//33wqlTp4RHH31UCA4OFq5duyZi5bZv+vTpgr+/v7Bp0yYhJydH2LBhg6DRaISXXnrJOIbnxnwYsCzQBx98IPTp00dQKpXCiBEjhIMHD4pdkt0B0OZj7dq1xjHXrl0TZs+eLXh4eAhqtVqYMGGCcOXKFfGKtmO/DFg8N+L64YcfhPDwcEGlUgkDBgwQVq1aZfJzg8EgvPbaa4KPj4+gUqmEsWPHCmlpaSJVaz8qKyuF+Ph4oU+fPoKDg4MQEhIivPLKK0J9fb1xDM+N+UgE4YYWrkRERETUZVyDRURERGRmDFhEREREZsaARURERGRmDFhEREREZsaARURERGRmDFhEREREZsaARURERGRmDFhEREREZsaARURkZn/9618xbNgwscsgIhExYBERERGZGQMWERERkZkxYBGRXdm6dStGjRoFd3d3eHp64te//jWysrIAALm5uZBIJNiwYQPuv/9+qNVqDB06FAcOHDB5jdWrVyMgIABqtRoTJkzAsmXL4O7uftP3/fjjjzFw4EA4ODhgwIAB+Mc//tFdH5GILAADFhHZlZqaGiQmJuLo0aNITU2FVCrFhAkTYDAYjGNeeeUV/OUvf8GJEycQFhaGKVOmoLGxEQCwb98+PPfcc4iPj8eJEycwbtw4vPnmmzd9z88++wwLFizAm2++iXPnziE5ORmvvfYa1q1b162flYjEIxEEQRC7CCIisZSUlMDLywunT5+Gs7MzgoOD8fHHH+OZZ54BAJw9exaDBg3CuXPnMGDAAPz+979HdXU1Nm3aZHyNJ598Eps2bUJ5eTmApkXu3333HU6cOAEA6Nu3L9544w1MmTLF+JzFixcjJSUF+/fv77HPSkQ9h1ewiMiuZGRkYMqUKQgJCYGrqyuCgoIAAHl5ecYxQ4YMMf6zVqsFABQVFQEA0tLSMGLECJPX/OX3N6qpqUFWVhaeeeYZODs7Gx+LFy82Tk0Ske2Ri10AEVFPeuSRRxAYGIjVq1fDz88PBoMB4eHh0Ol0xjEKhcL4zxKJBABMphBvR3V1NYCmdVuRkZEmP5PJZJ16TSKyfAxYRGQ3SktLkZaWhtWrV2P06NEAgL17997Wa/Tv3x9HjhwxOfbL72/k4+MDPz8/ZGdnY+rUqbdfNBFZJQYsIrIbHh4e8PT0xKpVq6DVapGXl4f58+ff1mu88MILiI6OxrJly/DII49g586d2LJli/FKV1tef/11zJkzB25ubnjwwQdRX1+Po0eP4urVq0hMTOzqxyIiC8Q1WERkN6RSKb788kscO3YM4eHhSEhIwLvvvntbrzFy5EisXLkSy5Ytw9ChQ7F161YkJCTAwcGh3ef88Y9/xMcff4y1a9di8ODBGDNmDD755BMEBwd39SMRkYXiXYRERF00c+ZMnD9/Hnv27BG7FCKyEJwiJCK6TX/7298wbtw4ODk5YcuWLVi3bh0bhxKRCV7BIiK6Tb/73e/w008/oaqqCiEhIXjhhRfw3HPPiV0WEVkQBiwiIiIiM+MidyIiIiIzY8AiIiIiMjMGLCIiIiIzY8AiIiIiMjMGLCIiIiIzY8AiIiIiMjMGLCIiIiIzY8AiIiIiMrP/B9GEDIVpzb9VAAAAAElFTkSuQmCC\n"
          },
          "metadata": {}
        }
      ]
    },
    {
      "cell_type": "markdown",
      "metadata": {
        "id": "eJ8YuVfCuGTj"
      },
      "source": [
        "Let us try one more augmentation: shear. Here is what this looks like:"
      ]
    },
    {
      "cell_type": "code",
      "metadata": {
        "id": "pMiw46NLwssK"
      },
      "source": [
        "def shear(sample, amount):\n",
        "  tform = AffineTransform(shear = amount) # create the shear transform\n",
        "  img = warp(sample, tform) # apply the shear\n",
        "  # this makes the digit off-center. Since all the images in the test set are centralized, we will do the same here\n",
        "  col = img.sum(0).nonzero()[0]\n",
        "  row = img.sum(1).nonzero()[0]\n",
        "  if len(col)>0 and len(row)>0:\n",
        "    xshift = int(sample.shape[0]/2 - (row[0]+row[-1])/2)\n",
        "    yshift = int(sample.shape[1]/2 - (col[0]+col[-1])/2)\n",
        "    img = np.roll(img, (xshift, yshift),(0,1))\n",
        "  return img"
      ],
      "execution_count": 11,
      "outputs": []
    },
    {
      "cell_type": "code",
      "metadata": {
        "id": "4_u_EYpmnABK",
        "colab": {
          "base_uri": "https://localhost:8080/",
          "height": 861
        },
        "outputId": "03d804ac-97ad-4318-dc0c-1e881c5c294f"
      },
      "source": [
        "sample = train_X[2]\n",
        "plt.imshow(sample, cmap='gray')\n",
        "plt.show()\n",
        "\n",
        "# apply shear\n",
        "plt.imshow(shear(sample, 0.4), cmap='gray')"
      ],
      "execution_count": null,
      "outputs": [
        {
          "output_type": "display_data",
          "data": {
            "text/plain": [
              "<Figure size 640x480 with 1 Axes>"
            ],
            "image/png": "iVBORw0KGgoAAAANSUhEUgAAAaAAAAGdCAYAAABU0qcqAAAAOXRFWHRTb2Z0d2FyZQBNYXRwbG90bGliIHZlcnNpb24zLjcuMSwgaHR0cHM6Ly9tYXRwbG90bGliLm9yZy/bCgiHAAAACXBIWXMAAA9hAAAPYQGoP6dpAAAb+UlEQVR4nO3df2xV9f3H8dctP66g7cVa++NKwYI/2EBqROkaleFoKHUhVsnir2XI/DGwuCn+WOpUdJJ0w2wal06XzFDdRMVlgJqlC1RbMldwIAx/bA3tqi3pDyaGe0uxhdDP9w++3nmlBc7l3r572+cj+SS955z3PW8OJ/fVc+/p5/qcc04AAAyyFOsGAAAjEwEEADBBAAEATBBAAAATBBAAwAQBBAAwQQABAEwQQAAAE6OtG/i6vr4+tbW1KTU1VT6fz7odAIBHzjl1dXUpGAwqJWXg65whF0BtbW3Kzc21bgMAcJpaW1s1ceLEAdcPubfgUlNTrVsAAMTByV7PExZAlZWVOv/883XGGWeooKBA77333inV8bYbAAwPJ3s9T0gAvfbaa1qxYoVWrlyp999/X/n5+SouLta+ffsSsTsAQDJyCTB79mxXVlYWeXz06FEXDAZdRUXFSWtDoZCTxGAwGIwkH6FQ6ISv93G/Ajp8+LB27NihoqKiyLKUlBQVFRWpvr7+uO17e3sVDoejBgBg+It7AH322Wc6evSosrKyopZnZWWpo6PjuO0rKioUCAQigzvgAGBkML8Lrry8XKFQKDJaW1utWwIADIK4/x1QRkaGRo0apc7OzqjlnZ2dys7OPm57v98vv98f7zYAAENc3K+Axo4dq1mzZqmmpiayrK+vTzU1NSosLIz37gAASSohMyGsWLFCixcv1uWXX67Zs2frmWeeUXd3t5YsWZKI3QEAklBCAujGG2/Uf//7Xz322GPq6OjQpZdequrq6uNuTAAAjFw+55yzbuKrwuGwAoGAdRsAgNMUCoWUlpY24Hrzu+AAACMTAQQAMEEAAQBMEEAAABMEEADABAEEADBBAAEATBBAAAATBBAAwAQBBAAwQQABAEwQQAAAEwQQAMAEAQQAMEEAAQBMEEAAABMEEADABAEEADBBAAEATBBAAAATBBAAwAQBBAAwQQABAEwQQAAAEwQQAMAEAQQAMEEAAQBMEEAAABMEEADABAEEADBBAAEATBBAAAATBBAAwAQBBAAwQQABAEwQQAAAEwQQAMAEAQQAMEEAAQBMEEAAABMEEADABAEEADBBAAEATBBAAAATBBAAwAQBBAAwQQABAEwQQAAAEwQQAMAEAQQAMEEAAQBMEEAAABOjrRtA8vL7/Z5rfvjDHyagk+Olp6fHVPfkk0/GuZP+/epXv/JcU1VV5bnmo48+8lwDDBaugAAAJgggAICJuAfQ448/Lp/PFzWmTZsW790AAJJcQj4Dmj59ujZv3vy/nYzmoyYAQLSEJMPo0aOVnZ2diKcGAAwTCfkMaM+ePQoGg5oyZYpuvfVWtbS0DLhtb2+vwuFw1AAADH9xD6CCggJVVVWpurpazz33nJqbm3X11Verq6ur3+0rKioUCAQiIzc3N94tAQCGoLgHUElJib73ve9p5syZKi4u1l/+8hcdOHBA69at63f78vJyhUKhyGhtbY13SwCAISjhdwdMmDBBF110kRobG/td7/f7Y/qDRgBAckv43wEdPHhQTU1NysnJSfSuAABJJO4B9MADD6iurk6ffPKJ/v73v+v666/XqFGjdPPNN8d7VwCAJBb3t+D27t2rm2++Wfv379e5556rq666Slu3btW5554b710BAJKYzznnrJv4qnA4rEAgYN3GiHLZZZfFVPfiiy96rvnmN78Z074gtbW1ea6J9Z2HnTt3eq7p7u6OaV8YvkKhkNLS0gZcz1xwAAATBBAAwAQBBAAwQQABAEwQQAAAEwQQAMAEAQQAMEEAAQBMEEAAABMEEADABAEEADBBAAEATCT8C+kw9JWVlcVUx8SigysYDHqu2bJlS0z7evrppz3X3H///THtCyMXV0AAABMEEADABAEEADBBAAEATBBAAAATBBAAwAQBBAAwQQABAEwQQAAAEwQQAMAEAQQAMEEAAQBMEEAAABPMhg1dfvnl1i2cUGNjo+eaP/3pTwnopH/Tp0/3XLNw4cIEdBI/paWlnmteeOEFzzUff/yx5xoMH1wBAQBMEEAAABMEEADABAEEADBBAAEATBBAAAATBBAAwAQBBAAwQQABAEwQQAAAEwQQAMAEAQQAMMFkpBjyGhoaPNf87Gc/S0An/TvzzDM916xatcpzzXXXXee55vzzz/dcE2tdbW2t55r169d7rnnjjTc81/zjH//wXCNJ+/bti6kOp4YrIACACQIIAGCCAAIAmCCAAAAmCCAAgAkCCABgggACAJgggAAAJgggAIAJAggAYIIAAgCYIIAAACZ8zjln3cRXhcNhBQIB6zZGlCVLlsRU9+yzz3quGT9+vOeaWCaEXLhwoecaSdq+fXtMdV5lZ2d7rlm9erXnmu9///ueayRpiL0snLZY/19jOfcOHTrkuebhhx/2XNPU1OS5ZrCFQiGlpaUNuJ4rIACACQIIAGDCcwBt2bJFCxcuVDAYlM/n04YNG6LWO+f02GOPKScnR+PGjVNRUZH27NkTr34BAMOE5wDq7u5Wfn6+Kisr+12/evVqPfvss3r++ee1bds2nXnmmSouLlZPT89pNwsAGD48fyNqSUmJSkpK+l3nnNMzzzyjRx55JPLtjS+99JKysrK0YcMG3XTTTafXLQBg2IjrZ0DNzc3q6OhQUVFRZFkgEFBBQYHq6+v7rent7VU4HI4aAIDhL64B1NHRIUnKysqKWp6VlRVZ93UVFRUKBAKRkZubG8+WAABDlPldcOXl5QqFQpHR2tpq3RIAYBDENYC+/OO6zs7OqOWdnZ0D/uGd3+9XWlpa1AAADH9xDaC8vDxlZ2erpqYmsiwcDmvbtm0qLCyM564AAEnO811wBw8eVGNjY+Rxc3Ozdu3apfT0dE2aNEn33nuvVq1apQsvvFB5eXl69NFHFQwGVVpaGs++AQBJznMAbd++Xddcc03k8YoVKyRJixcvVlVVlR566CF1d3frrrvu0oEDB3TVVVepurpaZ5xxRvy6BgAkPSYjRcz+8Ic/eK655ZZbEtDJ8X7/+9/HVPejH/3Ic82ll17queaNN97wXHPeeed5rvH5fJ5rpOE3GWmsqqurPde0t7d7rrnjjjs81yQDJiMFAAxJBBAAwAQBBAAwQQABAEwQQAAAEwQQAMAEAQQAMEEAAQBMEEAAABMEEADABAEEADBBAAEATBBAAAATzIaNmKWnp3uu2bx5s+ea/Px8zzV9fX2ea6Rj33fl1ejRnr/VROPHj/dcE4uhPhv2Rx995Lnmn//8ZwI66d+6des812zatMlzTU9Pj+eaZMBs2ACAIYkAAgCYIIAAACYIIACACQIIAGCCAAIAmCCAAAAmCCAAgAkCCABgggACAJgggAAAJgggAIAJ77MoAv/v888/91xTWVk5KDVjxozxXCPphBMnJqMPPvggprpVq1Z5rmlra/Ncs3fvXs81n376qecaDE1cAQEATBBAAAATBBAAwAQBBAAwQQABAEwQQAAAEwQQAMAEAQQAMEEAAQBMEEAAABMEEADABAEEADDhc8456ya+KhwOKxAIWLeBIeSTTz7xXJObmxv/RpLQqFGjrFvACBYKhU44wS9XQAAAEwQQAMAEAQQAMEEAAQBMEEAAABMEEADABAEEADBBAAEATBBAAAATBBAAwAQBBAAwQQABAEyMtm4AOJmUFO+/J/l8vgR0AiCeuAICAJgggAAAJjwH0JYtW7Rw4UIFg0H5fD5t2LAhav1tt90mn88XNRYsWBCvfgEAw4TnAOru7lZ+fr4qKysH3GbBggVqb2+PjFdeeeW0mgQADD+eb0IoKSlRSUnJCbfx+/3Kzs6OuSkAwPCXkM+AamtrlZmZqYsvvljLli3T/v37B9y2t7dX4XA4agAAhr+4B9CCBQv00ksvqaamRr/85S9VV1enkpISHT16tN/tKyoqFAgEIiM3NzfeLQEAhiCfc87FXOzzaf369SotLR1wm//85z+aOnWqNm/erHnz5h23vre3V729vZHH4XCYEEKUlpYWzzUTJ05MQCfJJ5a/oQLiJRQKKS0tbcD1CT87p0yZooyMDDU2Nva73u/3Ky0tLWoAAIa/hAfQ3r17tX//fuXk5CR6VwCAJOL5LriDBw9GXc00Nzdr165dSk9PV3p6up544gktWrRI2dnZampq0kMPPaQLLrhAxcXFcW0cAJDcPAfQ9u3bdc0110Qer1ixQpK0ePFiPffcc9q9e7defPFFHThwQMFgUPPnz9eTTz4pv98fv64BAEnPcwDNnTtXJ7pv4a9//etpNYTkMXq097lsV65c6bkmMzPTc81p3FszrHzwwQcx1T344IOea6qrq2PaF0YubpEBAJgggAAAJgggAIAJAggAYIIAAgCYIIAAACYIIACACQIIAGCCAAIAmCCAAAAmCCAAgAkCCABgggACAJg4ra/kToRwOKxAIGDdBk5BLP9Pn3/+eQI6sXXHHXd4rrn77rs918yYMcNzTaxfg1JfX++55qtf03KqDh8+7LkGycP8K7kBAOgPAQQAMEEAAQBMEEAAABMEEADABAEEADBBAAEATBBAAAATBBAAwAQBBAAwQQABAEwQQAAAE0xGipgxGekxZ599tueacDjsuaaystJzzbJlyzzXSFIsLwsvvfSS55olS5Z4rkHyYDJSAMCQRAABAEwQQAAAEwQQAMAEAQQAMEEAAQBMEEAAABMEEADABAEEADBBAAEATBBAAAATBBAAwMRo6waAZLdq1SrPNT/+8Y891zzyyCOea2KdjDQWhw4dGrR9YXjgCggAYIIAAgCYIIAAACYIIACACQIIAGCCAAIAmCCAAAAmCCAAgAkCCABgggACAJgggAAAJgggAIAJn3POWTfxVeFwWIFAwLoNnIIxY8Z4rnnqqac819x9992ea0aNGuW5JlZtbW2ea37wgx94rmlqavJc88knn3iukaRYXhZCoZDnmvT0dM81SB6hUEhpaWkDrucKCABgggACAJjwFEAVFRW64oorlJqaqszMTJWWlqqhoSFqm56eHpWVlemcc87RWWedpUWLFqmzszOuTQMAkp+nAKqrq1NZWZm2bt2qTZs26ciRI5o/f766u7sj29x3331688039frrr6uurk5tbW264YYb4t44ACC5efpG1Orq6qjHVVVVyszM1I4dOzRnzhyFQiG98MILWrt2rb7zne9IktasWaNvfOMb2rp1q771rW/Fr3MAQFI7rc+Avrzr5cs7WXbs2KEjR46oqKgoss20adM0adIk1dfX9/scvb29CofDUQMAMPzFHEB9fX269957deWVV2rGjBmSpI6ODo0dO1YTJkyI2jYrK0sdHR39Pk9FRYUCgUBk5ObmxtoSACCJxBxAZWVl+vDDD/Xqq6+eVgPl5eUKhUKR0draelrPBwBIDp4+A/rS8uXL9dZbb2nLli2aOHFiZHl2drYOHz6sAwcORF0FdXZ2Kjs7u9/n8vv98vv9sbQBAEhinq6AnHNavny51q9fr7ffflt5eXlR62fNmqUxY8aopqYmsqyhoUEtLS0qLCyMT8cAgGHB0xVQWVmZ1q5dq40bNyo1NTXyuU4gENC4ceMUCAR0++23a8WKFUpPT1daWpruueceFRYWcgccACCKpwB67rnnJElz586NWr5mzRrddtttkqSnn35aKSkpWrRokXp7e1VcXKzf/va3cWkWADB8MBkphrxYJtTkbspjfD5fTHVMRop4YDJSAMCQRAABAEwQQAAAEwQQAMAEAQQAMEEAAQBMEEAAABMEEADABAEEADBBAAEATBBAAAATBBAAwAQBBAAwEdM3ogKD6f777/dcs27dugR0knxinQ373Xff9Vxz7bXXxrQvjFxcAQEATBBAAAATBBAAwAQBBAAwQQABAEwQQAAAEwQQAMAEAQQAMEEAAQBMEEAAABMEEADABAEEADDBZKQY8t555x3PNaWlpTHt68knn/RcM336dM81KSmD87ufcy6mutbWVs81XV1dMe0LIxdXQAAAEwQQAMAEAQQAMEEAAQBMEEAAABMEEADABAEEADBBAAEATBBAAAATBBAAwAQBBAAwQQABAEz4XKyzFSZIOBxWIBCwbgM4ZUuWLPFcU15e7rlm6tSpnmuWL1/uuUaSXnvtNc81n3/+eUz7wvAVCoWUlpY24HqugAAAJgggAIAJAggAYIIAAgCYIIAAACYIIACACQIIAGCCAAIAmCCAAAAmCCAAgAkCCABgggACAJhgMlIAQEIwGSkAYEgigAAAJjwFUEVFha644gqlpqYqMzNTpaWlamhoiNpm7ty58vl8UWPp0qVxbRoAkPw8BVBdXZ3Kysq0detWbdq0SUeOHNH8+fPV3d0dtd2dd96p9vb2yFi9enVcmwYAJL/RXjaurq6OelxVVaXMzEzt2LFDc+bMiSwfP368srOz49MhAGBYOq3PgEKhkCQpPT09avnLL7+sjIwMzZgxQ+Xl5Tp06NCAz9Hb26twOBw1AAAjgIvR0aNH3Xe/+1135ZVXRi3/3e9+56qrq93u3bvdH//4R3feeee566+/fsDnWblypZPEYDAYjGE2QqHQCXMk5gBaunSpmzx5smttbT3hdjU1NU6Sa2xs7Hd9T0+PC4VCkdHa2mp+0BgMBoNx+uNkAeTpM6AvLV++XG+99Za2bNmiiRMnnnDbgoICSVJjY6OmTp163Hq/3y+/3x9LGwCAJOYpgJxzuueee7R+/XrV1tYqLy/vpDW7du2SJOXk5MTUIABgePIUQGVlZVq7dq02btyo1NRUdXR0SJICgYDGjRunpqYmrV27Vtdee63OOecc7d69W/fdd5/mzJmjmTNnJuQfAABIUl4+99EA7/OtWbPGOedcS0uLmzNnjktPT3d+v99dcMEF7sEHHzzp+4BfFQqFzN+3ZDAYDMbpj5O99jMZKQAgIZiMFAAwJBFAAAATBBAAwAQBBAAwQQABAEwQQAAAEwQQAMAEAQQAMEEAAQBMEEAAABMEEADABAEEADBBAAEATBBAAAATBBAAwAQBBAAwQQABAEwQQAAAEwQQAMAEAQQAMEEAAQBMEEAAABMEEADABAEEADBBAAEATAy5AHLOWbcAAIiDk72eD7kA6urqsm4BABAHJ3s997khdsnR19entrY2paamyufzRa0Lh8PKzc1Va2ur0tLSjDq0x3E4huNwDMfhGI7DMUPhODjn1NXVpWAwqJSUga9zRg9iT6ckJSVFEydOPOE2aWlpI/oE+xLH4RiOwzEch2M4DsdYH4dAIHDSbYbcW3AAgJGBAAIAmEiqAPL7/Vq5cqX8fr91K6Y4DsdwHI7hOBzDcTgmmY7DkLsJAQAwMiTVFRAAYPgggAAAJgggAIAJAggAYCJpAqiyslLnn3++zjjjDBUUFOi9996zbmnQPf744/L5fFFj2rRp1m0l3JYtW7Rw4UIFg0H5fD5t2LAhar1zTo899phycnI0btw4FRUVac+ePTbNJtDJjsNtt9123PmxYMECm2YTpKKiQldccYVSU1OVmZmp0tJSNTQ0RG3T09OjsrIynXPOOTrrrLO0aNEidXZ2GnWcGKdyHObOnXvc+bB06VKjjvuXFAH02muvacWKFVq5cqXef/995efnq7i4WPv27bNubdBNnz5d7e3tkfG3v/3NuqWE6+7uVn5+viorK/tdv3r1aj377LN6/vnntW3bNp155pkqLi5WT0/PIHeaWCc7DpK0YMGCqPPjlVdeGcQOE6+urk5lZWXaunWrNm3apCNHjmj+/Pnq7u6ObHPffffpzTff1Ouvv666ujq1tbXphhtuMOw6/k7lOEjSnXfeGXU+rF692qjjAbgkMHv2bFdWVhZ5fPToURcMBl1FRYVhV4Nv5cqVLj8/37oNU5Lc+vXrI4/7+vpcdna2e+qppyLLDhw44Px+v3vllVcMOhwcXz8Ozjm3ePFid91115n0Y2Xfvn1Okqurq3POHfu/HzNmjHv99dcj2/zrX/9yklx9fb1Vmwn39ePgnHPf/va33U9+8hO7pk7BkL8COnz4sHbs2KGioqLIspSUFBUVFam+vt6wMxt79uxRMBjUlClTdOutt6qlpcW6JVPNzc3q6OiIOj8CgYAKCgpG5PlRW1urzMxMXXzxxVq2bJn2799v3VJChUIhSVJ6erokaceOHTpy5EjU+TBt2jRNmjRpWJ8PXz8OX3r55ZeVkZGhGTNmqLy8XIcOHbJob0BDbjLSr/vss8909OhRZWVlRS3PysrSv//9b6OubBQUFKiqqkoXX3yx2tvb9cQTT+jqq6/Whx9+qNTUVOv2THR0dEhSv+fHl+tGigULFuiGG25QXl6empqa9PDDD6ukpET19fUaNWqUdXtx19fXp3vvvVdXXnmlZsyYIenY+TB27FhNmDAhatvhfD70dxwk6ZZbbtHkyZMVDAa1e/du/fSnP1VDQ4P+/Oc/G3YbbcgHEP6npKQk8vPMmTNVUFCgyZMna926dbr99tsNO8NQcNNNN0V+vuSSSzRz5kxNnTpVtbW1mjdvnmFniVFWVqYPP/xwRHwOeiIDHYe77ror8vMll1yinJwczZs3T01NTZo6depgt9mvIf8WXEZGhkaNGnXcXSydnZ3Kzs426mpomDBhgi666CI1NjZat2Lmy3OA8+N4U6ZMUUZGxrA8P5YvX6633npL77zzTtTXt2RnZ+vw4cM6cOBA1PbD9XwY6Dj0p6CgQJKG1Pkw5ANo7NixmjVrlmpqaiLL+vr6VFNTo8LCQsPO7B08eFBNTU3KycmxbsVMXl6esrOzo86PcDisbdu2jfjzY+/evdq/f/+wOj+cc1q+fLnWr1+vt99+W3l5eVHrZ82apTFjxkSdDw0NDWppaRlW58PJjkN/du3aJUlD63ywvgviVLz66qvO7/e7qqoq9/HHH7u77rrLTZgwwXV0dFi3Nqjuv/9+V1tb65qbm927777rioqKXEZGhtu3b591awnV1dXldu7c6Xbu3OkkuV//+tdu586d7tNPP3XOOfeLX/zCTZgwwW3cuNHt3r3bXXfddS4vL8998cUXxp3H14mOQ1dXl3vggQdcfX29a25udps3b3aXXXaZu/DCC11PT49163GzbNkyFwgEXG1trWtvb4+MQ4cORbZZunSpmzRpknv77bfd9u3bXWFhoSssLDTsOv5OdhwaGxvdz3/+c7d9+3bX3NzsNm7c6KZMmeLmzJlj3Hm0pAgg55z7zW9+4yZNmuTGjh3rZs+e7bZu3Wrd0qC78cYbXU5Ojhs7dqw777zz3I033ugaGxut20q4d955x0k6bixevNg5d+xW7EcffdRlZWU5v9/v5s2b5xoaGmybToATHYdDhw65+fPnu3PPPdeNGTPGTZ482d15553D7pe0/v79ktyaNWsi23zxxRfu7rvvdmeffbYbP368u/766117e7td0wlwsuPQ0tLi5syZ49LT053f73cXXHCBe/DBB10oFLJt/Gv4OgYAgIkh/xkQAGB4IoAAACYIIACACQIIAGCCAAIAmCCAAAAmCCAAgAkCCABgggACAJgggAAAJgggAIAJAggAYOL/AC0/Gz+zK6mJAAAAAElFTkSuQmCC\n"
          },
          "metadata": {}
        },
        {
          "output_type": "execute_result",
          "data": {
            "text/plain": [
              "<matplotlib.image.AxesImage at 0x7f7ffd26e140>"
            ]
          },
          "metadata": {},
          "execution_count": 12
        },
        {
          "output_type": "display_data",
          "data": {
            "text/plain": [
              "<Figure size 640x480 with 1 Axes>"
            ],
            "image/png": "iVBORw0KGgoAAAANSUhEUgAAAaAAAAGdCAYAAABU0qcqAAAAOXRFWHRTb2Z0d2FyZQBNYXRwbG90bGliIHZlcnNpb24zLjcuMSwgaHR0cHM6Ly9tYXRwbG90bGliLm9yZy/bCgiHAAAACXBIWXMAAA9hAAAPYQGoP6dpAAAeG0lEQVR4nO3de2zV9f3H8dcplwNoe7AUepGCBS8sAt1kUBuQoTSUqkSUGbxtsBmMWNgAnaaLiswt/cmWzbgxXbIFNBNUFoHIlEwLLV4KjiohxK2hWKQKLcLGOVBswfbz+4PYeeTm53ja9znl+Ui+CT3n++r37ZevffHtOf004JxzAgCgi6VYDwAAOD9RQAAAExQQAMAEBQQAMEEBAQBMUEAAABMUEADABAUEADDR03qAr2pvb9e+ffuUmpqqQCBgPQ4AwJNzTkeOHFFOTo5SUs58n5NwBbRv3z7l5uZajwEA+IYaGho0ePDgMz6fcN+CS01NtR4BABAH5/p63mkFtGzZMl1yySXq06ePCgoK9O67736tHN92A4Du4VxfzzulgF588UUtWrRIixcv1nvvvaf8/HwVFxfrwIEDnXE4AEAycp1g3LhxrrS0tOPjtrY2l5OT48rLy8+ZDYfDThIbGxsbW5Jv4XD4rF/v434HdPz4cdXU1KioqKjjsZSUFBUVFam6uvqU/VtbWxWJRKI2AED3F/cCOnjwoNra2pSZmRn1eGZmphobG0/Zv7y8XKFQqGPjHXAAcH4wfxdcWVmZwuFwx9bQ0GA9EgCgC8T954AyMjLUo0cPNTU1RT3e1NSkrKysU/YPBoMKBoPxHgMAkODifgfUu3dvjRkzRhUVFR2Ptbe3q6KiQoWFhfE+HAAgSXXKSgiLFi3SrFmz9N3vflfjxo3Tk08+qebmZv3oRz/qjMMBAJJQpxTQzJkz9emnn+rRRx9VY2Ojvv3tb2vDhg2nvDEBAHD+CjjnnPUQXxaJRBQKhazHAAB8Q+FwWGlpaWd83vxdcACA8xMFBAAwQQEBAExQQAAAExQQAMAEBQQAMEEBAQBMUEAAABMUEADABAUEADBBAQEATFBAAAATFBAAwAQFBAAwQQEBAExQQAAAExQQAMAEBQQAMEEBAQBMUEAAABMUEADABAUEADBBAQEATFBAAAATFBAAwAQFBAAw0dN6AMRXMBj0zlx99dUxHSsQCHhnQqGQd+aGG27wzrS1tXlnJOkf//iHd6a6uto709jY6J0BuhvugAAAJiggAIAJCggAYIICAgCYoIAAACYoIACACQoIAGCCAgIAmKCAAAAmKCAAgAkKCABgggICAJhgMdIEdtFFF3lnioqKvDM33nijd0aSMjMzvTO9evXyzowaNco7M3DgQO+MJE2ePNk7884773hn/vrXv3pn3n33Xe/MkSNHvDOS5JyLKQf44A4IAGCCAgIAmKCAAAAmKCAAgAkKCABgggICAJiggAAAJiggAIAJCggAYIICAgCYoIAAACYoIACACRYjTWD9+vXzzlx11VXemWnTpnlnJKl///4x5bpCrItpXnrppd6ZSy65xDtTUFDgnVm1apV3Zv369d4ZSXrvvfdiygE+uAMCAJiggAAAJuJeQI899pgCgUDUNmLEiHgfBgCQ5DrlNaArr7xSb7zxxv8O0pOXmgAA0TqlGXr27KmsrKzO+NQAgG6iU14D2rVrl3JycjRs2DDdeeed2rt37xn3bW1tVSQSidoAAN1f3AuooKBAK1as0IYNG/T000+rvr5e11xzzRl/N315eblCoVDHlpubG++RAAAJKO4FVFJSoltvvVWjR49WcXGxXn31VR0+fFgvvfTSafcvKytTOBzu2BoaGuI9EgAgAXX6uwP69++vyy+/XHV1dad9PhgMKhgMdvYYAIAE0+k/B3T06FHt3r1b2dnZnX0oAEASiXsBPfDAA6qqqtKePXv0zjvv6Oabb1aPHj10++23x/tQAIAkFvdvwX388ce6/fbbdejQIQ0cOFATJkzQli1bNHDgwHgfCgCQxAIu1lUbO0kkElEoFLIeI2n9+c9/9s7Eenfat29f78zRo0e9M7G8MaWtrc07I0lpaWnemYsvvtg706NHD+9MIBDwzrz66qveGUkqLS31znz00UfemQT78oM4C4fDZ/1/irXgAAAmKCAAgAkKCABgggICAJiggAAAJiggAIAJCggAYIICAgCYoIAAACYoIACACQoIAGCCAgIAmOj0X0iHrrVt2zbvzI033hjTsWJZjLS5udk7s3nzZu/Mli1bvDPSyV+g6GvSpEnemfHjx3tnYplt4sSJ3hlJ+tWvfuWdee6557wztbW13pn//Oc/3plYrjsp9kVt8fVwBwQAMEEBAQBMUEAAABMUEADABAUEADBBAQEATFBAAAATFBAAwAQFBAAwQQEBAExQQAAAExQQAMAEBQQAMBFwzjnrIb4sEokoFApZj5G0BgwY4J15+eWXYzpWLCs6nzhxwjuzfv1678ysWbO8M5J07Ngx70xGRoZ3Zt68ed6ZmTNnemdGjBjhnZGk9vZ278ynn37qnamsrPTOvPnmm96Zmpoa74wkHTx40Dvz+eefe2f27NnjnUkG4XBYaWlpZ3yeOyAAgAkKCABgggICAJiggAAAJiggAIAJCggAYIICAgCYoIAAACYoIACACQoIAGCCAgIAmKCAAAAmWIwUmjBhQky5xx9/3DtTWFjonWlpafHOLFmyxDsjSc8++6x3JhKJeGeGDRvmnZkzZ4535gc/+IF3RpIyMzO9Mwn2pSRKrIt9fvjhh96Z//73v96ZW2+91TuTDFiMFACQkCggAIAJCggAYIICAgCYoIAAACYoIACACQoIAGCCAgIAmKCAAAAmKCAAgAkKCABgggICAJjoaT0A7L311lsx5R544AHvzBNPPOGdufbaa70zP/nJT7wzktTe3u6d+dvf/uadueGGG7wzP/7xj70z6enp3plE19bW5p25+OKLYzpWz57+XyLffvvtmI51PuIOCABgggICAJjwLqDNmzdr2rRpysnJUSAQ0Nq1a6Oed87p0UcfVXZ2tvr27auioiLt2rUrXvMCALoJ7wJqbm5Wfn6+li1bdtrnly5dqqeeekrPPPOMtm7dqgsuuEDFxcUx/VIxAED35f0KW0lJiUpKSk77nHNOTz75pB5++GHddNNNkqTnnntOmZmZWrt2rW677bZvNi0AoNuI62tA9fX1amxsVFFRUcdjoVBIBQUFqq6uPm2mtbVVkUgkagMAdH9xLaDGxkZJp/4++czMzI7nvqq8vFyhUKhjy83NjedIAIAEZf4uuLKyMoXD4Y6toaHBeiQAQBeIawFlZWVJkpqamqIeb2pq6njuq4LBoNLS0qI2AED3F9cCysvLU1ZWlioqKjoei0Qi2rp1qwoLC+N5KABAkvN+F9zRo0dVV1fX8XF9fb22b9+u9PR0DRkyRAsWLNAvf/lLXXbZZcrLy9MjjzyinJwcTZ8+PZ5zAwCSnHcBbdu2LWptrkWLFkmSZs2apRUrVujBBx9Uc3Oz7rnnHh0+fFgTJkzQhg0b1KdPn/hNDQBIegHnnLMe4ssikYhCoZD1GOgkGRkZ3pl169Z5Z8aNG+edkU7+LJuvcDjsnenVq5d3JjU11TsTCAS8M7HmuupLSW1trXdmx44dMR0rloVP16xZ451ZvXq1dyYZhMPhs76ub/4uOADA+YkCAgCYoIAAACYoIACACQoIAGCCAgIAmKCAAAAmKCAAgAkKCABgggICAJiggAAAJiggAIAJCggAYILVsNGlYlllefz48d6ZxYsXe2ckRf2qka8r1hWnE/U4kvTpp596Z9auXeudqays9M7s27fPO/PRRx95Z6TYVjo/ceKEd+bo0aPemWTAatgAgIREAQEATFBAAAATFBAAwAQFBAAwQQEBAExQQAAAExQQAMAEBQQAMEEBAQBMUEAAABMUEADABIuRIuGlpPj/O+k73/lOTMd69dVXvTMDBw6M6ViJ7MMPP/TOFBcXe2diWfT0888/984cP37cOxPrsfA/LEYKAEhIFBAAwAQFBAAwQQEBAExQQAAAExQQAMAEBQQAMEEBAQBMUEAAABMUEADABAUEADBBAQEATPS0HgA4l/b2du9MTU1NTMd6++23vTPXX3+9dyYYDHpnutLw4cO9M5mZmd6Z3bt3e2fQfXAHBAAwQQEBAExQQAAAExQQAMAEBQQAMEEBAQBMUEAAABMUEADABAUEADBBAQEATFBAAAATFBAAwASLkSLhBQIB70xeXl5Mx2poaPDOOOdiOlZ386c//ck7s2zZMu/M3//+d+9MLH+v6HzcAQEATFBAAAAT3gW0efNmTZs2TTk5OQoEAlq7dm3U87Nnz1YgEIjapk6dGq95AQDdhHcBNTc3Kz8//6zfu506dar279/fsa1ateobDQkA6H6834RQUlKikpKSs+4TDAaVlZUV81AAgO6vU14Dqqys1KBBg3TFFVdo7ty5OnTo0Bn3bW1tVSQSidoAAN1f3Ato6tSpeu6551RRUaEnnnhCVVVVKikpUVtb22n3Ly8vVygU6thyc3PjPRIAIAHF/eeAbrvtto4/jxo1SqNHj9bw4cNVWVmpyZMnn7J/WVmZFi1a1PFxJBKhhADgPNDpb8MeNmyYMjIyVFdXd9rng8Gg0tLSojYAQPfX6QX08ccf69ChQ8rOzu7sQwEAkoj3t+COHj0adTdTX1+v7du3Kz09Xenp6VqyZIlmzJihrKws7d69Ww8++KAuvfRSFRcXx3VwAEBy8y6gbdu26dprr+34+IvXb2bNmqWnn35aO3bs0LPPPqvDhw8rJydHU6ZM0eOPP65gMBi/qQEASS/gEmwlxUgkolAoZD0Gkly/fv1iym3atMk7M2bMGO9MSkrXrIIV6w+BT5gwwTuTk5PjnYnlxy6ef/5578yX3+jk4/PPP48ph5PC4fBZX9dnLTgAgAkKCABgggICAJiggAAAJiggAIAJCggAYIICAgCYoIAAACYoIACACQoIAGCCAgIAmKCAAAAmKCAAgIm4/0puIBHEusj7Bx984J0ZO3ZsTMfqCnv37o0pF8vq0dOmTfPOlJSUeGfuuusu78zx48e9M5L0hz/8wTuzZ8+emI51PuIOCABgggICAJiggAAAJiggAIAJCggAYIICAgCYoIAAACYoIACACQoIAGCCAgIAmKCAAAAmKCAAgImAi3XVxk4SiUQUCoWsx0CS69GjR0y5iRMnemcqKipiOlZXqK2tjSk3e/Zs70xdXZ13Zv78+d6ZxYsXe2cOHDjgnZGkJ5980jtTXl4e07G6o3A4rLS0tDM+zx0QAMAEBQQAMEEBAQBMUEAAABMUEADABAUEADBBAQEATFBAAAATFBAAwAQFBAAwQQEBAExQQAAAEyxGCnxJ3759vTMvvfSSd6aoqMg7EwwGvTOx2rRpk3fm/vvv98588skn3pmqqirvzIgRI7wzkvTmm296Z2bNmuWd2bNnj3cmGbAYKQAgIVFAAAATFBAAwAQFBAAwQQEBAExQQAAAExQQAMAEBQQAMEEBAQBMUEAAABMUEADABAUEADDBYqTAlwQCAe9Mfn6+d+bBBx/0zsyYMcM706tXL++MJLW0tHhnYlkkdN26dd6ZwsJC78wPf/hD74wktba2emd+97vfeWfKysq8M8mAxUgBAAmJAgIAmPAqoPLyco0dO1apqakaNGiQpk+frtra2qh9WlpaVFpaqgEDBujCCy/UjBkz1NTUFNehAQDJz6uAqqqqVFpaqi1btuj111/XiRMnNGXKFDU3N3fss3DhQr3yyitavXq1qqqqtG/fPt1yyy1xHxwAkNx6+uy8YcOGqI9XrFihQYMGqaamRhMnTlQ4HNZf/vIXrVy5Utddd50kafny5frWt76lLVu26Oqrr47f5ACApPaNXgMKh8OSpPT0dElSTU2NTpw4EfXrhkeMGKEhQ4aourr6tJ+jtbVVkUgkagMAdH8xF1B7e7sWLFig8ePHa+TIkZKkxsZG9e7dW/3794/aNzMzU42Njaf9POXl5QqFQh1bbm5urCMBAJJIzAVUWlqqnTt36oUXXvhGA5SVlSkcDndsDQ0N3+jzAQCSg9drQF+YN2+e1q9fr82bN2vw4MEdj2dlZen48eM6fPhw1F1QU1OTsrKyTvu5gsGggsFgLGMAAJKY1x2Qc07z5s3TmjVrtHHjRuXl5UU9P2bMGPXq1UsVFRUdj9XW1mrv3r0x/fQyAKD78roDKi0t1cqVK7Vu3TqlpqZ2vK4TCoXUt29fhUIh3X333Vq0aJHS09OVlpam+fPnq7CwkHfAAQCieBXQ008/LUmaNGlS1OPLly/X7NmzJZ1cByklJUUzZsxQa2uriouL9cc//jEuwwIAug8WIwUM9OnTxzsTy2KfY8eO9c50R7EsMitJR44c8c7885//9M5MnjzZO5MMWIwUAJCQKCAAgAkKCABgggICAJiggAAAJiggAIAJCggAYIICAgCYoIAAACYoIACACQoIAGCCAgIAmKCAAAAmYvqNqAC+mZaWFu/M3LlzvTNLlizxzkjSdddd552J5Tcbp6Qk9r+BP/nkE+/Mb37zm06YpHtK7L99AEC3RQEBAExQQAAAExQQAMAEBQQAMEEBAQBMUEAAABMUEADABAUEADBBAQEATFBAAAATFBAAwASLkQJJYseOHd6Z+fPnx3SsadOmeWfuu+8+78zw4cO9Mz17+n/Zam9v985I0oUXXuidyc/P98689tpr3pnugDsgAIAJCggAYIICAgCYoIAAACYoIACACQoIAGCCAgIAmKCAAAAmKCAAgAkKCABgggICAJiggAAAJgLOOWc9xJdFIhGFQiHrMYDzWo8ePbwzwWDQO/P973/fO7Nw4ULvzJVXXumdkaQnnnjCO/Pss896Z+rq6rwzySAcDistLe2Mz3MHBAAwQQEBAExQQAAAExQQAMAEBQQAMEEBAQBMUEAAABMUEADABAUEADBBAQEATFBAAAATFBAAwASLkQIw07t3b+/M2Ra3PJMLLrjAOyNJLS0t3plwONwlx0kGLEYKAEhIFBAAwIRXAZWXl2vs2LFKTU3VoEGDNH36dNXW1kbtM2nSJAUCgajt3nvvjevQAIDk51VAVVVVKi0t1ZYtW/T666/rxIkTmjJlipqbm6P2mzNnjvbv39+xLV26NK5DAwCSX0+fnTds2BD18YoVKzRo0CDV1NRo4sSJHY/369dPWVlZ8ZkQANAtfaPXgL54t0d6enrU488//7wyMjI0cuRIlZWV6dixY2f8HK2trYpEIlEbAKD787oD+rL29nYtWLBA48eP18iRIzsev+OOOzR06FDl5ORox44deuihh1RbW6uXX375tJ+nvLxcS5YsiXUMAECSivnngObOnavXXntNb731lgYPHnzG/TZu3KjJkyerrq5Ow4cPP+X51tZWtba2dnwciUSUm5sby0gAkgw/BxT7cZLBuX4OKKY7oHnz5mn9+vXavHnzWctHkgoKCiTpjAUUDAYVDAZjGQMAkMS8Csg5p/nz52vNmjWqrKxUXl7eOTPbt2+XJGVnZ8c0IACge/IqoNLSUq1cuVLr1q1TamqqGhsbJUmhUEh9+/bV7t27tXLlSl1//fUaMGCAduzYoYULF2rixIkaPXp0p/wHAACSk9drQIFA4LSPL1++XLNnz1ZDQ4Puuusu7dy5U83NzcrNzdXNN9+shx9++Gt/35a14IDzB68BxX6cZBDX14DO1VW5ubmqqqry+ZQAgPNUzG/DBoBv6vjx496ZgwcPdkkGnY/FSAEAJiggAIAJCggAYIICAgCYoIAAACYoIACACQoIAGCCAgIAmKCAAAAmKCAAgAkKCABgggICAJiggAAAJiggAIAJCggAYIICAgCYoIAAACYoIACACQoIAGCCAgIAmKCAAAAmKCAAgAkKCABgggICAJhIuAJyzlmPAACIg3N9PU+4Ajpy5Ij1CACAODjX1/OAS7Bbjvb2du3bt0+pqakKBAJRz0UiEeXm5qqhoUFpaWlGE9rjPJzEeTiJ83AS5+GkRDgPzjkdOXJEOTk5Skk5831Ozy6c6WtJSUnR4MGDz7pPWlraeX2BfYHzcBLn4STOw0mch5Osz0MoFDrnPgn3LTgAwPmBAgIAmEiqAgoGg1q8eLGCwaD1KKY4DydxHk7iPJzEeTgpmc5Dwr0JAQBwfkiqOyAAQPdBAQEATFBAAAATFBAAwETSFNCyZct0ySWXqE+fPiooKNC7775rPVKXe+yxxxQIBKK2ESNGWI/V6TZv3qxp06YpJydHgUBAa9eujXreOadHH31U2dnZ6tu3r4qKirRr1y6bYTvRuc7D7NmzT7k+pk6dajNsJykvL9fYsWOVmpqqQYMGafr06aqtrY3ap6WlRaWlpRowYIAuvPBCzZgxQ01NTUYTd46vcx4mTZp0yvVw7733Gk18eklRQC+++KIWLVqkxYsX67333lN+fr6Ki4t14MAB69G63JVXXqn9+/d3bG+99Zb1SJ2uublZ+fn5WrZs2WmfX7p0qZ566ik988wz2rp1qy644AIVFxerpaWliyftXOc6D5I0derUqOtj1apVXThh56uqqlJpaam2bNmi119/XSdOnNCUKVPU3Nzcsc/ChQv1yiuvaPXq1aqqqtK+fft0yy23GE4df1/nPEjSnDlzoq6HpUuXGk18Bi4JjBs3zpWWlnZ83NbW5nJyclx5ebnhVF1v8eLFLj8/33oMU5LcmjVrOj5ub293WVlZ7te//nXHY4cPH3bBYNCtWrXKYMKu8dXz4Jxzs2bNcjfddJPJPFYOHDjgJLmqqirn3Mm/+169ernVq1d37POvf/3LSXLV1dVWY3a6r54H55z73ve+537605/aDfU1JPwd0PHjx1VTU6OioqKOx1JSUlRUVKTq6mrDyWzs2rVLOTk5GjZsmO68807t3bvXeiRT9fX1amxsjLo+QqGQCgoKzsvro7KyUoMGDdIVV1yhuXPn6tChQ9YjdapwOCxJSk9PlyTV1NToxIkTUdfDiBEjNGTIkG59PXz1PHzh+eefV0ZGhkaOHKmysjIdO3bMYrwzSrjFSL/q4MGDamtrU2ZmZtTjmZmZ+ve//200lY2CggKtWLFCV1xxhfbv368lS5bommuu0c6dO5Wammo9nonGxkZJOu318cVz54upU6fqlltuUV5ennbv3q2f//znKikpUXV1tXr06GE9Xty1t7drwYIFGj9+vEaOHCnp5PXQu3dv9e/fP2rf7nw9nO48SNIdd9yhoUOHKicnRzt27NBDDz2k2tpavfzyy4bTRkv4AsL/lJSUdPx59OjRKigo0NChQ/XSSy/p7rvvNpwMieC2227r+POoUaM0evRoDR8+XJWVlZo8ebLhZJ2jtLRUO3fuPC9eBz2bM52He+65p+PPo0aNUnZ2tiZPnqzdu3dr+PDhXT3maSX8t+AyMjLUo0ePU97F0tTUpKysLKOpEkP//v11+eWXq66uznoUM19cA1wfpxo2bJgyMjK65fUxb948rV+/Xps2bYr69S1ZWVk6fvy4Dh8+HLV/d70eznQeTqegoECSEup6SPgC6t27t8aMGaOKioqOx9rb21VRUaHCwkLDyewdPXpUu3fvVnZ2tvUoZvLy8pSVlRV1fUQiEW3duvW8vz4+/vhjHTp0qFtdH845zZs3T2vWrNHGjRuVl5cX9fyYMWPUq1evqOuhtrZWe/fu7VbXw7nOw+ls375dkhLrerB+F8TX8cILL7hgMOhWrFjhPvjgA3fPPfe4/v37u8bGRuvRutT999/vKisrXX19vXv77bddUVGRy8jIcAcOHLAerVMdOXLEvf/+++799993ktxvf/tb9/7777uPPvrIOefc//3f/7n+/fu7devWuR07dribbrrJ5eXluc8++8x48vg623k4cuSIe+CBB1x1dbWrr693b7zxhrvqqqvcZZdd5lpaWqxHj5u5c+e6UCjkKisr3f79+zu2Y8eOdexz7733uiFDhriNGze6bdu2ucLCQldYWGg4dfyd6zzU1dW5X/ziF27btm2uvr7erVu3zg0bNsxNnDjRePJoSVFAzjn3+9//3g0ZMsT17t3bjRs3zm3ZssV6pC43c+ZMl52d7Xr37u0uvvhiN3PmTFdXV2c9VqfbtGmTk3TKNmvWLOfcybdiP/LIIy4zM9MFg0E3efJkV1tbazt0JzjbeTh27JibMmWKGzhwoOvVq5cbOnSomzNnTrf7R9rp/vslueXLl3fs89lnn7n77rvPXXTRRa5fv37u5ptvdvv377cbuhOc6zzs3bvXTZw40aWnp7tgMOguvfRS97Of/cyFw2Hbwb+CX8cAADCR8K8BAQC6JwoIAGCCAgIAmKCAAAAmKCAAgAkKCABgggICAJiggAAAJiggAIAJCggAYIICAgCYoIAAACb+H2nB5KWJYkRWAAAAAElFTkSuQmCC\n"
          },
          "metadata": {}
        }
      ]
    },
    {
      "cell_type": "markdown",
      "metadata": {
        "id": "lGnWMoyM2pK4"
      },
      "source": [
        "Create an augmentation function which applies a random shear according to the constraint we provide:"
      ]
    },
    {
      "cell_type": "code",
      "metadata": {
        "id": "-qLDJyGytwP5"
      },
      "source": [
        "def augShear(sample, shearconstraint):\n",
        "  if shearconstraint==0:\n",
        "    return sample\n",
        "  if len(sample.shape)==2:\n",
        "    sample = np.expand_dims(sample, 0)  # make sure the sample is 3 dimensional\n",
        "  amt = rng.random(len(sample)) # generate random numbers for shear\n",
        "  amt = (amt-0.5)*shearconstraint # make the random shear constrained\n",
        "  nsample = sample.copy() # preallocate the augmented array to make it faster\n",
        "  for ii in range(len(sample)):\n",
        "    nsample[ii] = shear(sample[ii], amt[ii])\n",
        "  return np.squeeze(nsample) # take care if the input had only one sample."
      ],
      "execution_count": 12,
      "outputs": []
    },
    {
      "cell_type": "markdown",
      "metadata": {
        "id": "s6lQcWW93suJ"
      },
      "source": [
        "Let us do a grid search to find the best shear constraint."
      ]
    },
    {
      "cell_type": "code",
      "metadata": {
        "id": "l_wrqPkrzBb_",
        "colab": {
          "base_uri": "https://localhost:8080/"
        },
        "outputId": "fc27e531-4748-4409-eb94-ced079e6c6c4"
      },
      "source": [
        "shearconstraints = [0, 0.2,0.4,0.6,0.8,1.0,1.2,1.4,1.6,1.8,2.0] # the values we want to test\n",
        "accuracies = np.zeros(len(shearconstraints), dtype=np.float) # we will save the values here\n",
        "\n",
        "for ii in range(len(shearconstraints)):\n",
        "  # create the augmented dataset\n",
        "  augdata = train_X # we include the original images also in the augmented dataset\n",
        "  auglabel = train_y\n",
        "  for jj in range(naugmentations):\n",
        "    augdata = np.concatenate((augdata, augShear(train_X, shearconstraints[ii]))) # concatenate the augmented data to the set\n",
        "    auglabel = np.concatenate((auglabel, train_y))  # the labels don't change when we augment\n",
        "\n",
        "  # check the test accuracy\n",
        "  testpred = NN(augdata, auglabel, test_X)\n",
        "  accuracies[ii] = Accuracy(test_y, testpred)\n",
        "  print('Accuracy after shear augmentation constrained by ',shearconstraints[ii], ' is ', accuracies[ii], flush=True)"
      ],
      "execution_count": 13,
      "outputs": [
        {
          "output_type": "stream",
          "name": "stderr",
          "text": [
            "<ipython-input-13-8a89fef4983d>:2: DeprecationWarning: `np.float` is a deprecated alias for the builtin `float`. To silence this warning, use `float` by itself. Doing this will not modify any behavior and is safe. If you specifically wanted the numpy scalar type, use `np.float64` here.\n",
            "Deprecated in NumPy 1.20; for more details and guidance: https://numpy.org/devdocs/release/1.20.0-notes.html#deprecations\n",
            "  accuracies = np.zeros(len(shearconstraints), dtype=np.float) # we will save the values here\n"
          ]
        },
        {
          "output_type": "stream",
          "name": "stdout",
          "text": [
            "Accuracy after shear augmentation constrained by  0  is  0.6472\n",
            "Accuracy after shear augmentation constrained by  0.2  is  0.6147\n",
            "Accuracy after shear augmentation constrained by  0.4  is  0.632\n",
            "Accuracy after shear augmentation constrained by  0.6  is  0.6351\n",
            "Accuracy after shear augmentation constrained by  0.8  is  0.6636\n",
            "Accuracy after shear augmentation constrained by  1.0  is  0.6652\n",
            "Accuracy after shear augmentation constrained by  1.2  is  0.6495\n",
            "Accuracy after shear augmentation constrained by  1.4  is  0.6181\n",
            "Accuracy after shear augmentation constrained by  1.6  is  0.6323\n",
            "Accuracy after shear augmentation constrained by  1.8  is  0.6432\n",
            "Accuracy after shear augmentation constrained by  2.0  is  0.6277\n"
          ]
        }
      ]
    },
    {
      "cell_type": "code",
      "metadata": {
        "id": "EKaH-YR-zVnA",
        "colab": {
          "base_uri": "https://localhost:8080/",
          "height": 481
        },
        "outputId": "dd452753-080f-489c-8224-1c75e13d8969"
      },
      "source": [
        "fig = plt.figure()\n",
        "ax = fig.add_axes([0.1, 0.1, 0.8, 0.8]) # main axes\n",
        "# plot the variation of accuracy\n",
        "ax.plot(shearconstraints, accuracies)\n",
        "ax.set_xlabel('angle')\n",
        "ax.set_ylabel('accuracy')\n",
        "# plot the maximum accuracy\n",
        "maxind = np.argmax(accuracies)\n",
        "plt.scatter(shearconstraints[maxind], accuracies[maxind], c='red')"
      ],
      "execution_count": 14,
      "outputs": [
        {
          "output_type": "execute_result",
          "data": {
            "text/plain": [
              "<matplotlib.collections.PathCollection at 0x7e9516b79d50>"
            ]
          },
          "metadata": {},
          "execution_count": 14
        },
        {
          "output_type": "display_data",
          "data": {
            "text/plain": [
              "<Figure size 640x480 with 1 Axes>"
            ],
            "image/png": "iVBORw0KGgoAAAANSUhEUgAAAlAAAAG/CAYAAACXGOmPAAAAOXRFWHRTb2Z0d2FyZQBNYXRwbG90bGliIHZlcnNpb24zLjcuMSwgaHR0cHM6Ly9tYXRwbG90bGliLm9yZy/bCgiHAAAACXBIWXMAAA9hAAAPYQGoP6dpAABs0ElEQVR4nO3deXhTZfo38O9J0iRd05buC12AsnVjtxREFMRlGNEZBcYNRpkRywzCzLzK+FPUUVBHnU1HhJHFmVFAFHUEQUUWZS/72lK6l6606d6mTc77RxaoLdC0SU+W7+e6cimnT5L7kJLcec793I8giqIIIiIiIuo2mdQBEBERETkbJlBEREREVmICRURERGQlJlBEREREVmICRURERGQlJlBEREREVmICRURERGQlJlBEREREVlJIHYAjMhgMuHTpEnx9fSEIgtThEBERUR8QRRH19fWIiIiATHb9OSYmUF24dOkSoqOjpQ6DiIiIJFBUVISoqKjrjmEC1QVfX18Axr9APz8/iaMhIiKivlBXV4fo6GhLHnA9TKC6YL5s5+fnxwSKiIjIzXSnfIdF5ERERERWYgJFREREZCUmUERERERWYgJFREREZCUmUERERERWYgJFREREZCUmUERERERWYgJFREREZCUmUERERERWYidyIiIzvR74/nugtBQIDwcmTgTkcqmjIiIHxASKiAgAPv0UWLgQKC6+ciwqCvjb34D77pMuLiJySLyER0T06afAz3/eMXkCgJIS4/FPP5UmLiJyWEygiMi96fXGmSdR7Pwz87GnnjKOIyIy4SU8InJv33+PxvIqHItJwaHo4ciMHA51eyv+8cXr8G5rMSZRRUXG2qhbbpE6WiJyEEygiMjtXG5oxeH8GhzOr8bhI3U489QG6GUdi8U3JU3Bo0e/vHKgtLSPoyQiR8YEiohcmiiKKK5pxqG8ahzOr8ah/GrkVjZeNUIOyIDI2gqMKT4DmSji08RbsW7kT/Dw0S2QwXQZLzxckviJyDExgSIil2IwiMgqrzfOLuXX4HBeNcrqWjqNSwj1wZjYQIyN8cfo2Xcj8sJpQBTRoPTEN4PGIbdfFPbEjcAt+ceMq/EmTpTgbIjIUTGBIiKn1tqux+mSWhzKM16Sy8yvRl1Le4cxCpmApCgNxsQGYkxsIEbHBCDAW3llwPIXjKvtBAE+umbcf/IbrB4zA2tH/dSYQP31r+wHRUQdCKLY1dIT91ZXVweNRoPa2lr4+flJHQ4RXaW+pQ1HC7U4nGe8HHeiSIvWdkOHMV5KOUb2DzAmTHEBGBEdAE/lDRKgq/pA5fuHY/Kv3oMoyLAzWYe4X9xrxzMiIkdhzec/Z6CIyKFV1rcaa5fyqpFZUI2zl+pg+NHXvn7eSoyONSZMY+MCMSzcDwq5lV1a7rsPuOce4PvvEVtaismXPPBdpR7rfBLwgs3OhohcBRMoInIYoiii4HKTqX7JWMOUV9XYaVx0oKcxWYoNxOjYQAwI9oYgCL0PQC63tCqYk12J71YfwqYjxfjd7QnwVXv0/vGJyGUwgSIiyegNIs6X1eFwXrWlrUBFfWuHMYIADA71NV2OMyZNYRq13WObOCgIA4K9cbGyEZ8cKcac9Di7PycROQ8mUETUZ1ra9DhZXGu5JHe0oAb1rR0Lvj3kApKj/E2X4wIwqn8gNF59P/sjCALmjI/Fc5+fwbr9BXgkLRYymQ1muYjIJTCBIiK7qW1uw9GCGhwyrY47UVQLnb5jwbePSoGRMQEYa6phSon2h9rDMVa83TcyCq9vy0JeVSP2XKjELYNDpA6JiBwEEygisrn8qkYsXH8MJ0tqO20xF+Sjwti4AEtLgSFhvtYXfPcRb5UC94+Oxuq9eVi7L58JFBFZMIEiIpvbkFmEE8W1AIDYfl6WZGlMXCBi+3nZpuC7jzySFoM1+/KwK6sSuZUNiA/2kTokInIATKCIyOYO51UDAJbdm4RfjOsvcTS9ExvkjVsHh2DH+Qp8sL8AL/x0uNQhEZEDcMx5cyJyWuZCcQAYP6CfxNHYxpz0WADAx5lFqG9pkzYYInIITKCIyKZOFhsLxYN8VIjp5yV1ODYxYaCxpUGjTo9PjhRLHQ4ROQAmUERkU4fzjZfvxsYFOFWt0/WYWxoAwLr9BTD8uBU6EbkdJlBEZFPmBGp0TKDEkdjWfSOj4KtSIK+qEbsvVEodDhFJjAkUEdmM3iDiSEENAGBsnGslUN4qBR4YEw0AWLs3X9pgiEhyTKCIyGayyupR39IOb6UcQ8J8pQ7H5h5Ji4EgALuzK3GxskHqcIhIQkygiMhmMguMl+9GxgQ4bHPM3ojpZ2xpAAD/3l8gcTREJCXXe4cjIskcMvV/GhPrWpfvrsaWBkQEMIEiIhsRRdFSQO7KCdSEgUEYGOKDRp0em9jSgMhtMYEiIpsormlGeV0rPOQCUqP9pQ7HbgRBwKPmlgb78tnSgMhNMYEiIpswzz4lRmrgqZRLHI193TciEr5qBfIvN7GlAZGbYgJFRDZxON/YvsCVL9+ZeasUeGA0WxoQuTMmUERkE+5Q/3Q1tjQgcm9MoIio16obdcipMCYRo2MCJI6mb8T088ZtQ4wtDT7Yly9tMETU55hAEVGvZZpmnwaF+CDAWylxNH1nzvg4AMCmI8VsaUDkZphAEVGvZZq2bxntJpfvzNIH9mNLAyI3xQSKiHrN3EBzbJx7XL4zY0sDIvfFBIqIeqVZp8fpkloAwOgY95qBAn7U0iCbLQ2I3AUTKCLqlWNFNWg3iAjXqBEV4Cl1OH3OW6XATFNLgzUsJidyG5InUO+88w5iY2OhVqsxbtw4HDp06LrjtVotMjIyEB4eDpVKhYSEBGzdurXDmJKSEjz00EPo168fPD09kZSUhMzMTHueBpHbysy/Uv8kCILE0UjjkbRYCAKwhy0NiNyGpAnUhg0bsHjxYixduhRHjx5FSkoKpk2bhoqKii7H63Q6TJ06Ffn5+di0aROysrKwatUqREZGWsbU1NQgPT0dHh4e+Oqrr3D27Fm8+eabCAhwr9oMor5ypf+T+/4b69/Piy0NiNyMQsonf+uttzBv3jzMnTsXALBixQps2bIFq1evxjPPPNNp/OrVq1FdXY19+/bBw8MDABAbG9thzGuvvYbo6GisWbPGciwuLs5+J0Hkxtr1BhwtcJ8O5NczZ3wcvj1XgU1HivG7aYPhp/aQOiQisiPJZqB0Oh2OHDmCKVOmXAlGJsOUKVOwf//+Lu/zxRdfIC0tDRkZGQgNDUViYiKWLVsGvV7fYczo0aNx//33IyQkBCNGjMCqVauuG0trayvq6uo63Ijoxs6V1qNRp4evWoGEUF+pw5FUh5YGmWxpQOTqJEugqqqqoNfrERoa2uF4aGgoysrKurxPbm4uNm3aBL1ej61bt+K5557Dm2++iZdffrnDmHfffReDBg3C9u3bMX/+fPz2t7/FunXrrhnL8uXLodFoLLfo6GjbnCSRizNfvhsdEwC5zD3rn8wEQcAcc0uD/WxpQOTqJC8it4bBYEBISAhWrlyJUaNGYebMmXj22WexYsWKDmNGjhyJZcuWYcSIEfjVr36FefPmdRjzY0uWLEFtba3lVlRU1BenQ+T0LAmUm1++M7tvpLGlQcHlJuzK7rqWk4hcg2QJVFBQEORyOcrLyzscLy8vR1hYWJf3CQ8PR0JCAuRyueXY0KFDUVZWBp1OZxkzbNiwDvcbOnQoCgsLrxmLSqWCn59fhxsRXZ8oijhsWoE3No4JFAB4Ka+0NFi7r0DiaIjIniRLoJRKJUaNGoUdO3ZYjhkMBuzYsQNpaWld3ic9PR05OTkwGAyWY9nZ2QgPD4dSqbSMycrK6nC/7OxsxMTE2OEsiNxX/uUmVDW0QimXISlSI3U4DuPqlgbmDZaJyPVIeglv8eLFWLVqFdatW4dz585h/vz5aGxstKzKe+SRR7BkyRLL+Pnz56O6uhoLFy5EdnY2tmzZgmXLliEjI8MyZtGiRThw4ACWLVuGnJwcfPjhh1i5cmWHMUTUe+bLdynRGqg95DcY7T6MLQ2MtZ0f7M+XNhgishtJ2xjMnDkTlZWVeP7551FWVobU1FRs27bNUlheWFgImexKjhcdHY3t27dj0aJFSE5ORmRkJBYuXIinn37aMmbMmDHYvHkzlixZgpdeeglxcXH461//igcffLDPz4/IlR3OY/3TtcxNj8W358qx6Ugxfs+WBkQuSRBFkUtFfqSurg4ajQa1tbWshyK6hslv7EJeVSPWzBmDyaYmkmQkiiJu/8seXKhowHM/GYbHJrAXHZEzsObz36lW4RGRY6iob0FeVSMEARgZ474dyK9FEAQ8ampp8AFbGhC5JCZQRGS1I6bVd4NDfaHx5OWprrClAZFrYwJFRFY7ZNn/jvVP1+KlVGDWGGNLgzV786UNhohsjgkUEVkt0zQDNYb9n67L3NLg+wtVyKmolzocIrIhJlBEZJWG1nacuVQLABgTy/qn64kOvNLSYB0baxK5FCZQRGSVY4U1MIhAVIAnwjWeUofj8OamxwIAPjlajLqWNmmDISKbYQJFRFYx939i/VP3jB/QD4NCfNCk0+PjzGKpwyEiG2ECRURWMe9/xwSqewRBwBzTLNS6ffnQs6UBkUtgAkVE3aZrN+BYkTmBYv1Td907IhJ+agUKq5uwK4stDYhcARMoIuq2M5dq0dJmQICXBwaG+EgdjtPwUiow09TSYO2+fGmDISKbYAJFRN1mbl8wKiYQgiBIHI1zYUsDItfCBIqIus3cQHNsHC/fWSs60AtThrKlAZGrYAJFRN0iiiIyTQnUaBaQ98hc0/54nxwtRm0zWxoQOTMmUETULRcrG1DT1Aa1hwyJERqpw3FKaQP6ISHU3NKgSOpwiKgXmEARUbeY2xekRvtDqeBbR08IgoBHTbNQH+wvYEsDIifGd0Ei6hZzA82xvHzXK2xpQOQamEARUbccLmD9ky14KRWYNbY/ALY0IHJmTKCI6IbKaltQVN0MmQCMjOEKvN56+KYYyEwtDS6Us6UBkTNiAkVEN3TYtPpuWIQffFQKiaNxfh1aGuzPlzYYIuoRJlBEdEPmBGp0DC/f2cocUzH5p0dL2NKAyAkxgSKiGzKvwBsbxwTKVtjSgMi5MYEiouuqbW7D+bI6AMBobiBsM4IgYM74OABsaUDkjJhAEdF1HS2sgSgCsf28EOKrljoclzJjRAQ0nh4orG7CzvNsaUDkTJhAEdF1mfs/sX2B7XkpFZg1JhoAWxoQORsmUER0XZnm+icmUHbxkKmlwQ85bGlA5EyYQBHRNbW263G8WAuA9U/2wpYGRM6JCRQRXdOp4lro2g0I8lEiLshb6nBc1pz0WADAJ0fY0oDIWTCBIqJrOnRV/ydBECSOxnWlxffD4FBfNLexpQGRs2ACRUTXZK5/GsP+T3YlCIJlFmrd/ny2NCByAkygiKhLBoOITNMM1BjWP9ndjNRIaDw9UFTdzJYGRE6ACRQRdSm7oh51Le3wUsoxLNxP6nBcnqdSzpYGRE6ECRQRdcnc/2lk/wAo5Hyr6AtsaUDkPPiuSERdMu9/N4b9n/pMdKAXpg4ztjTgLBSRY2MCRUSdiKKIw6x/koR5f7xPj5agtoktDYgcFRMoIuqkRNuM0toWKGQCUvv7Sx2OW7kpPvBKS4MjbGlA5KiYQBFRJ+bZp+GRGngpFRJH417Y0oDIOTCBIqJODlv2v+PlOylc3dLgO7Y0IHJITKCIqBPzCrzRLCCXhKdSjlljzS0N8iSOhoi6wgSKiDqoadThQkUDAGB0DGegpPKwqaXB3pzLbGlA5ICYQBFRB0cKjJfvBgR7o5+PSuJo3FdUAFsaEDkyJlBE1IG5gHws97+THFsaEDkuJlBE1IE5gRodwwRKajfFB2JImLGlwcZMtjQgciRMoIjIoqVNj1MltQA4A+UIBEHAnPGxANjSgMjRMIEiIovjRVq06UWE+qkQFeApdTgE4B5TS4PiGrY0IHIkTKCIyMLcvmBMbCAEQZA4GgLY0oDIUTGBIiKLwwXcQNgRXd3SIJstDYgcAhMoIgIA6A0ijjKBckhRAV64fVgYALY0IHIUTKCICABwrrQODa3t8FUpMDjMV+pw6EfM++NtZksDIofABIqIAFxpXzAyJgByGeufHM24OLY0IHIkTKCICACQad5AmO0LHBJbGhA5FiZQRARRFHHI0kCT+985qntSI+HvZWxpsONcudThELk1JlBEhMLqJlTWt0IplyEl2l/qcOgaPJVyzBrTHwCLyYmkxgSKiHDI1P8pKUoDtYdc4mjoeh5OM7Y02HfxMrLK2NKASCpMoIjIUv/E9gWOL9Lf09LSYN3+fGmDIXJjTKCIyLICb0ws65+cgbmlwadHi9nSgEgiTKCI3FxVQytyqxoBAKNjOAPlDMwtDVraDNiQWSh1OERuiQkUkZvLNM0+DQ71hcbLQ+JoqDsEQcBc0yzUun0FbGlAJAEmUERu7rC5/imOl++cibmlQYm2Gd+ypQFRn3OIBOqdd95BbGws1Go1xo0bh0OHDl13vFarRUZGBsLDw6FSqZCQkICtW7dafv7CCy9AEIQOtyFDhtj7NIic0pX6J16+cyZqjystDdaxpQFRn5M8gdqwYQMWL16MpUuX4ujRo0hJScG0adNQUVHR5XidToepU6ciPz8fmzZtQlZWFlatWoXIyMgO44YPH47S0lLL7YcffuiL0yFyKo2t7ThzqQ4AEyhnxJYGRNKRPIF66623MG/ePMydOxfDhg3DihUr4OXlhdWrV3c5fvXq1aiursZnn32G9PR0xMbGYtKkSUhJSekwTqFQICwszHILCgrqi9MhcirHCrXQG0RE+nsiwt9T6nDISpH+npg23NjSgI01ifqWpAmUTqfDkSNHMGXKFMsxmUyGKVOmYP/+/V3e54svvkBaWhoyMjIQGhqKxMRELFu2DHq9vsO4CxcuICIiAvHx8XjwwQdRWHjtlSqtra2oq6vrcCNyB2xf4PzM++NtPlYMbZNO2mCI3IikCVRVVRX0ej1CQ0M7HA8NDUVZWVmX98nNzcWmTZug1+uxdetWPPfcc3jzzTfx8ssvW8aMGzcOa9euxbZt2/Duu+8iLy8PEydORH1911Pcy5cvh0ajsdyio6Ntd5JEDsycQI3m5TunNTYuEEPD/YwtDQ4XSR0OkduQ/BKetQwGA0JCQrBy5UqMGjUKM2fOxLPPPosVK1ZYxtx55524//77kZycjGnTpmHr1q3QarXYuHFjl4+5ZMkS1NbWWm5FRXwTItfXpjfgWKEWgPFDmJyTIAiYMz4GAPDBfrY0IOorkiZQQUFBkMvlKC/vuAS3vLwcYWFhXd4nPDwcCQkJkMuv7Nc1dOhQlJWVQafrevra398fCQkJyMnJ6fLnKpUKfn5+HW5Eru7MpTo0t+mh8fTAwGAfqcOhXmBLA6K+J2kCpVQqMWrUKOzYscNyzGAwYMeOHUhLS+vyPunp6cjJyYHBYLAcy87ORnh4OJRKZZf3aWhowMWLFxEeHm7bEyByYplX1T/JZILE0VBvqD3kmD3W2NKAl/GI+obkl/AWL16MVatWYd26dTh37hzmz5+PxsZGzJ07FwDwyCOPYMmSJZbx8+fPR3V1NRYuXIjs7Gxs2bIFy5YtQ0ZGhmXM73//e+zevRv5+fnYt28f7r33XsjlcsyePbvPz4/IUR3KY/2TK7kz0Thrf6SgBqLIy3hE9qaQOoCZM2eisrISzz//PMrKypCamopt27ZZCssLCwshk13J86Kjo7F9+3YsWrQIycnJiIyMxMKFC/H0009bxhQXF2P27Nm4fPkygoODMWHCBBw4cADBwcF9fn5EjkgURWQWmDqQM4FyCUPC/KBUyFDb3Ib8y02IC/KWOiQilyaI/KrSSV1dHTQaDWpra1kPRS7pYmUDbntzN1QKGU69MA1KheST0WQD9/1zL44WavGXmSm4d0SU1OEQOR1rPv/5rknkhg6bLt+lRvszeXIhqdHGfl7HTasrich++M5J5IYsGwjz8p1LSe3vDwA4XqSVNA4id8AEisgNWTqQs/+TSxkR7Q8AOFtah5Y2/fUHE1GvMIEicjPldS0orG6CTABGmmYsyDVEBXiin7cSbXoRZ0u5JRWRPTGBInIz5tmnoeF+8FV7SBwN2ZIgCEg1zUKxDorIvphAEbmZTNY/ubQUcwLFOigiu2ICReRmzA00mUC5plQmUER9ggkUkRupa2nD+TJjbczo2ACJoyF7MM9AFVY3obqx6/1Biaj3mEARuZGjBTUwiED/QC+E+qmlDofsQOPpgfhgYxfyE5yFIrIbJlBEboT1T+7BfBnvGBMoIrthAkXkRg6Z+z/x8p1LG8E6KCK7YwLVxzLzq/HOzhycvcQeLdS3Wtv1lks6bKDp2sxbupwo0oLbnRLZh0LqANzNmr352HKqFIIADIvgRsXUd06X1KK13YB+3krEB3lLHQ7Z0ZBwX6gUMtQ2tyGvqhHxwT5Sh0TkcjgD1ceSozQAgJNFtRJHQu7GvP/d6NgACIIgcTRkTx5yGRIjje81vIxHZB9MoPpYcpQ/AOBksVbSOMj9HGb/J7fCflBE9sUEqo8lRWkgCMCl2hZU1rdKHQ65CYNBRGYBV+C5E3YkJ7IvJlB9zEelwABTPQJnoaivXKhoQG1zGzw95Ky9cxPmlXjnSuvQ0qaXNhgiF8QESgLmOqgTxayDor5h3kB4ZIw/POT8Z+8OogI80c9biTa9iLOlXPVLZGt8J5WAuTaBM1DUV8wJ1OgYXr5zF4IgXKmDKtRKGguRK2ICJYErheS17NFCfcLcgXws+z+5FRaSE9kPEygJDA33hYdcQHWjDsU1zVKHQy6uRNuMEm0z5LIrMxLkHlL7+wNgAkVkD0ygJKBSyDEkzFjIe5J1UGRnmabLd4kRfvBWsXeuOzHPdhdWN+FyA1f9EtkSEyiJWBpqsg6K7OyQqf/TaLYvcDsaTw8MCDZ2nT/B9xoim2ICJZEU0zdDvqmRvZnrn9j/yT2Z98VjITmRbTGBkkhytHEG6nRJHQwGFpKTfWibdMgqrwdg3MKF3E+q6b3mGOugiGyKCZREBgb7wNNDjobWduRWNUgdDrmoI6bu4/HB3gjyUUkcDUnBPAN1okjLL2tENsQESiIKuQyJkcZC8hPcWJjs5JCpgHwM+z+5rSHhvlApZKhraUfe5UapwyFyGUygJMSNhcneLPVP7P/ktjzkMiRGmnY/4GU8IpthAiUhbulC9tTSprck52NY/+TW2FCTyPaYQEnIvBLvbGkddO0GaYMhl3OiSIs2vYgQXxX6B3pJHQ5JiAkUke0xgZJQTD8vaDw9oGs3INu0UorIVjILrrQvEARB4mhISuYE6lxpHVra9NIGQ+QimEBJSBCEqy7jaaUNhlyOuYEmL99RVIAngnyUaNOLOHOpTupwiFwCEyiJWTqScyUe2ZDeIOKoaQaKHchJEARexiOysR4lUDt37rR1HG4rmR3JyQ7Ol9WhvrUdPioFhob7SR0OOQAmUES21aME6o477sCAAQPw8ssvo6ioyNYxuRVzIfmFigY06dqlDYZchrl9wciYAMhlrH8iIMWSQNVIGwiRi+hRAlVSUoIFCxZg06ZNiI+Px7Rp07Bx40bodDpbx+fywjRqhPiqoDewNoFsx9xAcyzrn8jEPNtdVN2Myw2t0gZD5AJ6lEAFBQVh0aJFOH78OA4ePIiEhAQ8+eSTiIiIwG9/+1ucOHHC1nG6NMtlPE6tkw2IoohMUwLF+icy03h6YECwNwCWDBDZQq+LyEeOHIklS5ZgwYIFaGhowOrVqzFq1ChMnDgRZ86csUWMLi/FXEjOhppkA0XVzSiva4WH/ErhMBFwZV+844VaaQMhcgE9TqDa2tqwadMm3HXXXYiJicH27dvx9ttvo7y8HDk5OYiJicH9999vy1hdVrLpQ45bupAtHDbNPiVFaqD2kEscDTmS1P7+AIBjnO0m6jVFT+70m9/8Bh999BFEUcTDDz+M119/HYmJiZafe3t744033kBERITNAnVlyaZ9qvIvN6G2qQ0aLw+JIyJnZk6gxvDyHf3ICNOXtRNFWhgMImRcYOC2vj5Thk1HivHyjESE+KmlDscp9SiBOnv2LP7xj3/gvvvug0ql6nJMUFAQ2x10U4C3Ev0DvVBY3YSTJVpMHBQsdUjkxJhA0bUMDvOFSiFDXUs78i43YkCwj9QhkQTK61qweOMJNLS2I1yjxov3JN74TtRJjy7h7dixA7Nnz75m8gQACoUCkyZN6nFg7iaZdVBkA5cbWnGxshEAMCqGK/CoIw+5DEmmGW/WQbmvl748i4ZWY9ucj48Uo7a5TeKInFOPEqjly5dj9erVnY6vXr0ar732Wq+DckcpXIlHNmDe/y4h1AcB3kqJoyFHxIaa7m1XVgW2nCyFXCYgXKNGk06PjzPZz7EnepRAvffeexgyZEin48OHD8eKFSt6HZQ74gwU2cLhPLYvoOszF5IzgXI/LW16PP+5cXX83PGx+M2tgwAAa/flQ28QpQzNKfUogSorK0N4eHin48HBwSgtLe11UO4oMVIDmQCU1bWgoq5F6nDISR02zUCNZQJF12Ce7T5XWoeWNr20wVCfevu7HBRWNyFco8ZTUxNw74hI+Ht5oLimGd+eK5c6PKfTowQqOjoae/fu7XR87969XHnXQ94qBQaGGAs6T3AWinqgSdeOMyXG353R7EBO1xAV4IkgHyXaDSLOXOJ7jbvIqajHe3suAgCWTh8OH5UCnko5Zo/tDwBYszdPyvCcUo8SqHnz5uGpp57CmjVrUFBQgIKCAqxevRqLFi3CvHnzbB2j2zB3JGc/KOqJ44VatBtERGjUiArwkjocclCCIFxVB8UEyh2IoohnN59Gm17EbUNCMG14qOVnD98UA7lMwIHcaibUVupRG4M//OEPuHz5Mp588knL/ndqtRpPP/00lixZYtMA3UlKtD82HSnmDBT1yCFu30LdlBrtj2/PVbAOyk18erQEB/OqofaQ4YWfDocgXOn/FeHviTsSw7DlZCnW7s3Hn+9PkTBS59KjGShBEPDaa6+hsrISBw4cwIkTJ1BdXY3nn3/e1vG5lStbumghiizoI+tk5hvrn8bEMYGi67Ns6VJUI3EkZG/aJh1e2XoOALDwtgREB3aenf5leiwA4PMTl7jRtBV6tReej48PxowZg8TExOv2hKLuGRLmB6VcBm1TG4qqm6UOh5xIu96Ao4WmBIr1T3QDydEaCIJx30R+YLq217adR3WjDgmhPnh8YlyXY0b2D0BKlAa6dgM+PFjYxxE6rx5dwgOAzMxMbNy4EYWFhZbLeGaffvpprwNzR0qFDEPDfXGiuBYnirXo3491LNQ9Z0vr0KTTw0+tQEKIr9ThkIPzU3tgQLAPcioacLxIi9uGht74TuR0jhRU46NDxh5Pr9ybBA9513MmgiBgbnocntpwHP8+UIBfTxoApaJX8ytuoUd/Q+vXr8f48eNx7tw5bN68GW1tbThz5gy+++47aDQaW8foVlhITj1x6Kr+T9zfjLqDDTVdW5vegGc3nwYAzBwdfcOtne5KCkeIrwoV9a3YeortiLqjRwnUsmXL8Je//AX/+9//oFQq8be//Q3nz5/HAw88gP79+9s6RrdibqjJQnKyhqX+iQXk1E1MoFzb6h/ycL6sHgFeHnjmzs6Nr39MqZDhoZtiABhbGrAO98Z6lEBdvHgRd999NwBAqVSisbERgiBg0aJFWLlypU0DdDcppje10yW17AxL3SKK4lUbCLP+ibrn6gTKwPcal1Jc04S/fnsBAPDHu4Z2e1unX4zrD6VchhPFtTjKvRJvqEcJVEBAAOrr6wEAkZGROH3aOE2o1WrR1NRku+jc0IBgH3gp5WjS6XGxskHqcMgJ5FU14nKjDkqFDElRvIRO3TM4zBcqhQz1Le3IrWqUOhyyoRe+OIvmNj3GxgXi56Oiun2/IB8V7kk1NsNmY80b61ECdfPNN+Obb74BANx///1YuHAh5s2bh9mzZ+O2226zaYDuRi4TkGjaLZ0bC1N3mC/fpUb5Q6WQSxwNOQsPuQxJpvcaXsZzHV+fKcO358qhkAl4ZUZih55P3TE33bhS76vTZSit5Wrw6+lRAvX2229j1qxZAIBnn30WixcvRnl5OX72s5/h/ffft2mA7iiFGwuTFcwNNMfE8fIdWcd8GY9f1lxDY2s7XvjCuFnwr26Ox6BQ61fkDovww7i4QOgNIj7YX2DrEF2K1W0M2tvb8eWXX2LatGkAAJlMhmeeecbmgbkzrsQja2SyAzn1UGp/fwCcgXIVf9txAZdqWxAd6Inf3Dqox48zNz0OB/Oq8dGhQvz21kHwVHJmuytWz0ApFAo88cQTaGlpsVkQ77zzDmJjY6FWqzFu3DgcOnTouuO1Wi0yMjIQHh4OlUqFhIQEbN26tcuxr776KgRBwFNPPWWzeO3tym7p9dC1G6QNhhxaRX0L8i83QRCAUTGcgSLrmGegzpXWoaVNL20w1CvnSuvw/g/GuqWXfprYq6Rn6rBQRAV4QtvUhs+Ol9gqRJfTo0t4Y8eOxfHjx20SwIYNG7B48WIsXboUR48eRUpKCqZNm4aKiooux+t0OkydOhX5+fnYtGkTsrKysGrVKkRGRnYae/jwYbz33ntITk62Sax9JTrQEwFeHtDpDThfVid1OOTAzPVPQ8L84Kf2kDgacjaR/p4I8lGh3SByI1knZjCI+OPmU9AbRNyVFIbJQ0J69XhymYA542MBsKXB9fQogXryySexePFivP3229i/fz9OnjzZ4WaNt956C/PmzcPcuXMxbNgwrFixAl5eXli9enWX41evXo3q6mp89tlnSE9PR2xsLCZNmoSUlI4bIDY0NODBBx/EqlWrEBDgXN/MBUFAkmkWirUJdD3mBppj2b6AekAQBMss1DEuW3da6w8X4VihFt5KOZ7/yXCbPOb9o6PhpZQju7wBe3Mu2+QxXU2PEqhZs2YhLy8Pv/3tb5Geno7U1FSMGDHC8t/u0ul0OHLkCKZMmXIlIJkMU6ZMwf79+7u8zxdffIG0tDRkZGQgNDQUiYmJWLZsGfT6jtPPGRkZuPvuuzs89rW0trairq6uw01qKWyoSd2QWcD6J+qdEayDcmpVDa149SvjZsG/u30wwjRqmzyuxtPD0gKBLQ261qO98PLybPOXWVVVBb1ej9DQjvswhYaG4vz5813eJzc3F9999x0efPBBbN26FTk5OXjyySfR1taGpUuXAjBuNXP06FEcPny4W3EsX74cL774Yu9OxsZYSE43Ut/ShrOXjMk+O5BTT7EjuXNbtuUc6lraMTzCD4+kxdj0sR8dH4sP9hfgu6wK5Fc1IjbI26aP7+x6lEDFxNj2RbKGwWBASEgIVq5cCblcjlGjRqGkpAR//vOfsXTpUhQVFWHhwoX45ptvoFZ3LxNfsmQJFi9ebPlzXV0doqOj7XUK3WKegcqpaEBjazu8VT3e95lc1LFCLQyisWbOVt86yf0kRWkgCEBxTTOqGloR5KOSOiTqpn0Xq/DpsRIIgnGzYMU1NgvuqQHBPpg8OBg7syqxdl8+XvipbS4PuooefSp/8MEH1/35I4880q3HCQoKglwuR3l5eYfj5eXlCAsL6/I+4eHh8PDwgFx+ZYXB0KFDUVZWZrkkWFFRgZEjR1p+rtfrsWfPHrz99ttobW3tcF8AUKlUUKkc600jxE+NMD81yupacLqkFuPi+0kdEjmYK9u3cPaJes5P7YEBwT7IqWjA8UItpgwLvfGdSHKt7Xr832fGXUAeGhdjmUm0tbnpcdiZVYlNR4rxu9sT4MvFKhY9SqAWLlzY4c9tbW1oamqCUqmEl5dXtxMopVKJUaNGYceOHZgxYwYA4wzTjh07sGDBgi7vk56ejg8//BAGgwEymTHbzs7ORnh4OJRKJW677TacOnWqw33mzp2LIUOG4Omnn+6UPDmy5CgNys624GQxEyjqjAkU2UpqtD9yKhpwopgJlLNYuTsXuZWNCPZV4Q93DLbb80wcFISBIcYEe2NmMR6bEGe353I2PZrvq6mp6XBraGhAVlYWJkyYgI8++siqx1q8eDFWrVqFdevW4dy5c5g/fz4aGxsxd+5cAMbZrCVLlljGz58/H9XV1Vi4cCGys7OxZcsWLFu2DBkZGQAAX19fJCYmdrh5e3ujX79+SExM7MnpSsa8sfAJ1kHRj+jaDZZVU9xAmHqLdVDOJb+qEf/YmQMAeO4nw+zawkQQrrQ0WLcvn5vcX8VmhTWDBg3Cq6++ioceeuiaBeBdmTlzJiorK/H888+jrKwMqamp2LZtm6WwvLCw0DLTBADR0dHYvn07Fi1ahOTkZERGRmLhwoV4+umnbXUqDiOZW7rQNZy+VIvWdgMCvIyXX4h64+oEymAQIZNZt38a9R1RFPHc56ehazdg4qAgTE8Ot/tz3jcyEq9vO4/C6iZ8d74CUzlLCcCGCRRg7FJ+6dIlq++3YMGCa16y27VrV6djaWlpOHDgQLcfv6vHcAbJkf4AgMLqJtQ06hDgrZQ2IHIYh/OutC+wdrNQoh8bEuYLtYcM9S3tyK1qxMAQJuWO6suTpfj+QhWUChleusf6zYJ7wkupwOxx/fHe7lys2ZvHBMqkRwnUF1980eHPoiiitLQUb7/9NtLT020SGAEaLw/E9vNC/uUmnCypxaSEYKlDIgdx2NSBfCzrn8gGFHIZkiI1OJxfg+NFWiZQDqqupQ0vfXkWAJBxy0DE9WFbgUfSYvGv7/Ow7+JlnC+rw5Awvz57bkfVowTKXPBtJggCgoODceutt+LNN9+0RVxkkhzlb0ygirRMoAiAcduGKw00Wf9EtpEa7W9KoGosDRTJsby5PQuV9a2ID/LGE7fE9+lzR/p7YtrwUGw9VYY1P+TjtZ871xZp9tCjInKDwdDhptfrUVZWhg8//BDh4fa/HutOktmRnH7kYmUDtE1tUHvIkBipkTocchGp0cZknIXkjulksRYfHCgAAPxpRiJUir5fUT433bgC77PjJahu1PX58zsa23bdIpszr8RjR3IyO2RqXzAiOgAeNm6cR+4r1bSly/nSerS06a8/mPqU3rRZsCgCM1IjkD4wSJI4RscEIClSg9Z2Az46VChJDI6kR+++P/vZz/Daa691Ov7666/j/vvv73VQdMXwCD/IBKCivhVltS1Sh0MOINNU/zQmjvVPZDsRGjWCfVVoN4g4XcIZb0fywf58nC6pg59agWfvHiZZHIIgYG56LADg3/sL0KY3SBaLI+hRArVnzx7cddddnY7feeed2LNnT6+Doiu8lAokhPoCYD8oMjqUZ26gyfonsh1BEJBi2oOTl/EcR1ltC978OhsA8PSdQxDsK+2uGXcnhyPIR4WyuhZ8dbpM0lik1qMEqqGhAUpl5yX1Hh4eqKur63VQ1NGVflBaaQMhyV3SNqNE2wy5TMCI/kygyLZGmC7jMYFyHH/68iwaWtsxor8/Zo/pL3U4UCnkeOgmYxyrf8iTOBpp9SiBSkpKwoYNGzodX79+PYYNk2560VVdqYPitLq7M2/fMizcDz7cYJpsjB3JHcvOrApsOVUKuUzAKzOSHKbB6YPjYqCUy3C8SItjhTVShyOZHr0DP/fcc7jvvvtw8eJF3HrrrQCAHTt24KOPPsLHH39s0wAJlmn1k8W1EEWRjRPdmKX+if2fyA6SozQQBKC4phlVDa0I8nGsTdbdSUubHs9/btwseO74WAyLcJy+S8G+KkxPicAnR4uxZm++286G92gGavr06fjss8+Qk5ODJ598Er/73e9QXFyMb7/9tlOPKOq9wWG+UCpkqG1uQ8HlJqnDIQld2UDYPd+wyL581R4YaNoa6Lhpr0WSxtvf5aCouhnhGjWempogdTidmIvJt54qddsFTj1eA3333Xdj7969aGxsRFVVFb777jtMmjTJlrGRiYdchmHhxm8fLCR3X7VNbcgqrwdg3MKFyB54GU96ORX1eG/PRQDA0unDHfJyfWKkBmNjA9FuEPHvA/lShyOJHiVQhw8fxsGDBzsdP3jwIDIzM3sdFHWWwo2F3d6RwmqIIhAX5C35ShxyXaksJJeUKIp4dvNptOlFTBkagmnDHXffOfMs1IcHC92yd1iPEqiMjAwUFRV1Ol5SUoKMjIxeB0WdJVvqoLSSxkHSOWypf+LlO7If8wzUiSItDAZR2mDc0CdHS3AwrxqeHnK88NPhDl3zOnVYKCL9PVHT1IbPj5dIHU6f61ECdfbsWYwcObLT8REjRuDs2bO9Doo6S4k2zkCdLqlDu5s3L3NXh/PM+9/x8h3Zz+BQX3h6yFHf2o7cqgapw3ErNY06LNt6DgCwcMogRAV4SRzR9SnkMjw6PgYAsGZvPkTRvRLuHiVQKpUK5eXlnY6XlpZCoXC8a7WuID7IBz4qBZrb9Mip5Juau2lp01su345lAkV2pJDLkGTaY/EYC8n71GvbzqO6UYeEUB88NiFO6nC6Zebo/vD0kON8WT32516WOpw+1aME6vbbb8eSJUtQW3ulHker1eKPf/wjpk6darPg6AqZTEBipLGQ/GQR66DczamSWuj0BgT5qBDTz7G/lZLzM894sw6q72TmV2P9YWNpzCv3JjnNPpcaLw/8bFQkAGD1D/nSBtPHevQKvfHGGygqKkJMTAwmT56MyZMnIy4uDmVlZXjzzTdtHSOZmPtBcSWe+zFv3zI2LsChayLINaRGG+vs+F7TN9r0Bjy72djzaeboaKfr8zZnvHG2bMf5chRcbpQ4mr7TowQqMjISJ0+exOuvv45hw4Zh1KhR+Nvf/oZTp04hOjra1jGSSTITKLeVaer/NDrGud5YyTmZV+KdL613y9VVfW31D3nIKq9HoLcSz9w5ROpwrDYwxAc3JwRDFIF1+wqkDqfP9HiO0NvbGxMmTMD06dNx8803w9/fH1999RW++OILW8ZHVzHvicc3NfeiN4jILDCuwBsbxwSK7C9Co0awrwrtBhGnS1gyYE/FNU3467cXAAB/vGsoArw77zPrDH5pamnwcWYRGlrbpQ2mj/So4js3Nxf33nsvTp06BUEQOm0votfzw90eogI8EeitRHWjDudK69y2fb67yS6vR31LO7yVcgwJ85U6HHIDgiAgNdof35wtx/EiLVd+2okoinjhizNobtNjXFwgfjYyUuqQeuzmQcGID/ZGbmUjNmUWYU66cxTB90aPZqAWLlyIuLg4VFRUwMvLC6dPn8bu3bsxevRo7Nq1y8YhkpkgCJZZKDbUdB/m7VtGxgRA4SSFpeT8zP2gjrGQ3G6+PluOb89VwEMu4JV7E526vlEmEzB3fCwAYO2+fLfoIdajd+P9+/fjpZdeQlBQEGQyGeRyOSZMmIDly5fjt7/9ra1jpKuwDsr9HOYGwiSBEeYtXdjKwC4aW9vxwhdnAAC/ujkeA0Ocf3b5vpFR8FUrkH+5CTuzKqQOx+56lEDp9Xr4+hpf7KCgIFy6dAkAEBMTg6ysLNtFR51wSxf3IoqipYEmEyjqS0lRGggCUKJtRmV9q9ThuJy/fpuN0toWRAd6YsHkQVKHYxPeKgVmj+0PwNhY09X1KIFKTEzEiRMnAADjxo3D66+/jr179+Kll15CfHy8TQOkjswzUBcrG9ymUM+dFdc0o6yuBR5ywXJJhagv+Ko9MCjEBwD7Qdna2Ut1WG1KMF76aSI8lXJpA7KhR9JiIBOAH3KqkG3a/NxV9SiB+r//+z8YDMbtRF566SXk5eVh4sSJ2Lp1K/7+97/bNEDqKNhXhQiNGqIInOIslMsz1z8lRmpc6k2WnIM5aT9eVCNtIC7EYBDx7GenoDeIuCspDJOHhEgdkk1FBXjh9mFhAFx/FqpHCdS0adNw3333AQAGDhyI8+fPo6qqChUVFbj11lttGiB1xo2F3Qfrn0hKKZYESitpHK5k/eEiHCvUwkelwPM/GS51OHYx19TS4NOjxahp1EkbjB3ZbElPYGCgU68gcCbJ0ayDcgdnL9VhT3YlAGB0DFtWUN8zz0CdLKp1i1VV9lZZ34pXvzJuFvy72xMQplFLHJF9jI0LxLBwP7S2G/DR4UKpw7Ebrol2QtzSxXWJooi9OVV4+P2DuOvv36NE2wyNpwcbaJIkBof6wtNDjvrWduRWcRPz3lq29RzqWtqRGOmHh2+KkTocuxEEAb80bYb87/0FaNMbJI7IPphAOaFE007pxTXNuNzA1TGuoF1vwP9OXML0t3/Ag/86iO8vVEEmANNTIrDx12nw93LO7sTk3BRyGZJM7zfH2M6gV/blVGHzsRIIAvDKjCSX7+k2PSUcQT5KlNa2YPuZMqnDsQvXfgVdlMbTA/FB3gCAk9xmwak16/T4YH8+Jr+5C7/56BhOl9RB7SHDnPGx2P2HyfjH7BEYzO7jJCHzvnisg+q51nY9/u8z42bBD98UY6ktc2UqhRy/GGecZVv9Q57E0dhHj7ZyIeklR2mQW9WIk0W1mDzYtVZxuIPqRh3W7cvHB/vzUdPUBgAI9Fbi0bRYPJwWg0An3Q+LXE8qC8l77b3ducitakSwrwq/nzZY6nD6zEM39ce7u3JwtFCLE0Val0scmUA5qeQof3x2/BJX4jmZwstN+NcPudiYWYSWNmNdQP9AL8ybGIefj4pmqwJyOOYE6nxZPZp1ev6OWim/qhFv78wBADz3k2HwU3tIHFHfCfFV4yfJEdh8rARr9ubhr7NGSB2STTGBclIpppV4J4prO23mTI7nVHEt3ttzEVtPlcK8mCkpUoNfT4rHHcPDXL4egpxXuEaNEF8VKupbcfpSLVtqWEEURTz3+Wno2g2YOCgI05PDpQ6pz/0yPQ6bj5Vgy6lS/PGuoQjxc52Vh0ygnNSwcA3kMgFVDa0orW1BhL+n1CHRj4iiiD0XqrByz0XszblsOT4pIRi/nhSPtPh+THzJ4QmCsQv+12fLcbxQywTKCv87WYrvL1RBqZDhT/c492bBPZUUpcHomABkFtTgPwcKsPh217mEyQTKSXkq5UgI9cW50jqcLNYygXIgbXoDtpwsxXt7cnGutA4AIJcJ+GlKBOZNjMewCD+JIySyTmp/UwLFOqhuq21uw5++PAsAWDB5IGJNC3/c0dz0OGQW1OC/Bwvx5OSBUHu4xmVgJlBOLCVKg3OldThRXIs7Et1vatjRNLa2Y8PhIrz/Qx5KtM0AAC+lHLPG9McvJ8QiKsBL4giJeibV1HuOCVT3vfl1FirrWxEf5I1fT3LvPWKnDQ9FhEaNS7Ut+OLEJTwwOlrqkGyCCZQTS4n2x/rDRSwkl1hlfSvW7cvHvw8UoLbZuKIuyEeJOeNj8dBNMezhRE4vKUoDQQBKtM2orG9FsK9K6pAc2okiLf59oAAA8PKMRKgUrjHj0lMKuQyPjI/Fq1+dx5q9+bh/VJRLXM5kAuXEkqOubOliMIiQyZz/F9KZ5FU1YtX3udh0pBi6duOKurggb8ybGI/7Rka6zDQ1ka/aA4NCfJBd3oDjRVpMHRYqdUgOq11vwB83n4IoAveOiMT4gUFSh+QQZo2Jxl+/zca50joczKvGTfH9pA6p15hAObGEUF+oFDLUt7Qj/3Ij4oN9pA7JLRwrrMHKPbnYdqYMomlFXUq0P+ZPisfUYWGQM5ElF5Qa7W9KoGqYQF3Hvw8U4MylOvipFfjjXUOlDsdh+Hspcd/IKHx4sBBr9uYxgSJpechlGB7hh6OFWpwsrmUCZUcGg4hd2RVYsTsXh/KqLcdvHRKCX98cj7Fx3EybXFtqdAA2ZhazDuo6ympb8ObX2QCAp+8cwkudPzJ3fCw+PFiIr8+Wo6i6CdGBzl0XygTKySVH+Ru7vBZrMWNEpNThuBxduwFfnLiElXsuIrvcuJmqh1zAPamR+NXN8UgI5TYr5B7MDTVPFrFk4Fpe+vIMGlrbMaK/P2aP6S91OA5nUKgvJg4KwvcXqrBuXz7+7yfDpA6pV5hAOTlzQ82TxdwTz5bqW9qw/pBxRV1ZXQsAwEelwC/G9cfc9FiEa9g2gtxLQqgPPD3kqG9tx8XKBgzil4cOdp6vwNZTZZDLBLwyI4kJ5jX8Mj0O31+owobMIiyamgBvlfOmIc4bOQEwzkABwJlLtWjTG+DBjta9UlHXgjX78vGfAwWob2kHAAT7qvDL9Dj8Ylx/aDzdZxsGoqsp5DIkRWlwKK8ax4q0TKCu0qzT4/kvjJsF/zI9lr3ermNSQjDig7yRW9WIT44W45G0WKlD6jF+2jq5uH7e8FUp0NJmQHZ5vdThOK2cigY8vekkJry2E+/uuoj6lnYMCPbGaz9Lwg9PT8b8WwYweSK3N4IbC3fp7Z0XUFTdjHCNGk9NSZA6HIcmkwl4dHwsAGDt3nwYzHtbOSHOQDk5mUxAUpQG+y5exsniWgyP0EgdklM5UlCNFbtz8c3Zcsux0TEB+PWkAbhtSAin4YmuYq6DOl6olTQOR3KhvB4r9+QCAF746XCnviTVV342KgpvbM9CblUjdmdXYvKQEKlD6hG+0i4gOcrflEBpMXssCxdvxGAQseN8Bd7bfRGZBTWW41OHheLXN8djNPf6IupSiimByiqvR7NOD0+le/c6E0URz352Gm16EVOGhuB2tnfoFh+VAg+Micb7P+Rh9d48JlAknRRTQ80TRSwkv57Wdj0+O1aClXtycbGyEQCglMtw74hIzLs5HgND2AaC6HrCNWqE+KpQUd+K05dq3X5j4e1nynEorxqeHnK88NPhbGVihTnjY7Fmbx6+v1CFnIp6DAxxvpo6JlAuIPmqb4UtbXp2wP6RNr0B//o+D2v25qGivhUA4KtW4KGbYjB3fCxC/NQSR0jkHARBQGq0aWPhQq1bJ1CiKOKfu3IAgHtd9kB0oBemDA3F12fLsWZvPl65N0nqkKzGInIXEKFRI8hHCb1BxJlLdVKH43D+seMCXtt2HhX1rQjzU+PZu4Zi3zO34uk7hjB5IrJSan9/ACwk35tjrDtVe8jwy/Q4qcNxSnNNf2+fHi2BtkkncTTWYwLlAgRBsLQz4MbCHbXpDfjwUBEA4A/TBmPP/5uMeTfHw1fNFXVEPZHKlXgAgHd2GmefZo3pj34+7DjeEzfFB2JImC+a2/RYf7hI6nCsxgTKRVy9sTBd8d35ClQ1tCLIR4lf3RwPpYK/8kS9kRzlD0EASrTNqKhvkTocSRwrrMH+3MtQyATMuzle6nCcliAI+OUE4yzUB/vy0a43SByRdfhp4iJSTDNQJzgD1cFG07ean42KYpNRIhvwUSmQYCr4ddd2Bv/cdREAMGNEJCL9uStBb/w0JQL9vJW4VNuCr69qJ+MM+IniIswzULmVjahraZM4GsdQVtuCnVkVAIAHRkdLHA2R63Dny3hZZfX45mw5BAF4YtIAqcNxemoPOX4xzth+Z83ePImjsQ4TKBfRz0dl+SZ0mpfxAACbjhTBIAJjYwMxIJgtCohsxZ0LyVfsNs4+3TE8jK1PbOShm2KgkAk4nF+DU070+cUEyoWYNxY+4US/gPZiMIjYmFkMAJg5hrNPRLZknoE6WVwLvRNvxWGtouomfHHiEgDgyVsGShyN6wj1U+Pu5HAAzjULxQTKhXAl3hUHci+jsLoJvioF7koKlzocIpcyKMQHnh5yNLS2I7eyQepw+sx7ey5CbxAxcVAQkqK4bZYtmVsa/O/kJadZnMAEyoVwJd4V5iWxP02NcPvtJohsTSGXWRKIY25yGa+ivsUyq83ZJ9tLjfbHyP7+aNOL+O+BQqnD6RYmUC4kKVJjWV5c1dAqdTiS0TbpsO1MGQBjjxYisr0RblZI/v4PedC1GzCyvz9uinffDuz2ZJ6F+u/BArS26yWO5sYcIoF65513EBsbC7VajXHjxuHQoUPXHa/VapGRkYHw8HCoVCokJCRg69atlp+/++67SE5Ohp+fH/z8/JCWloavvvrK3qchOV+1B+KDvAG492W8z46VQNduwNBwPyRG+kkdDpFLsqzEc4NWBrVNbZZZkSdvGcg97+zkjsQwhPmpUdWgw/9OlEodzg1JnkBt2LABixcvxtKlS3H06FGkpKRg2rRpqKio6HK8TqfD1KlTkZ+fj02bNiErKwurVq1CZGSkZUxUVBReffVVHDlyBJmZmbj11ltxzz334MyZM311WpKx9INy042FRVG0XL6bNSaab3REdmJeiZdVXo9mnePPFvTGB/vz0dDajsGhvrh1SIjU4bgsD7kMD6fFADAWk4uiYy9QkDyBeuuttzBv3jzMnTsXw4YNw4oVK+Dl5YXVq1d3OX716tWorq7GZ599hvT0dMTGxmLSpElISUmxjJk+fTruuusuDBo0CAkJCXjllVfg4+ODAwcO9NVpSeZKHZRW2kAkcqqkFufL6qFUyDAjNfLGdyCiHgnXeCLUTwW9QcSpEtf9wtas02PNvnwAwJOTB0Am45cye/rF2P5QKWQ4c6kOh/NrpA7nuiRNoHQ6HY4cOYIpU6ZYjslkMkyZMgX79+/v8j5ffPEF0tLSkJGRgdDQUCQmJmLZsmXQ67v+BqTX67F+/Xo0NjYiLS3NLufhSJKvWl7s6Nm7PZhnn+5MDIPGi/vdEdnTlYaajv1B1xvrDxeiulGH/oFeuJsreu0uwFuJ+0Yav/w6eksDSROoqqoq6PV6hIaGdjgeGhqKsrKyLu+Tm5uLTZs2Qa/XY+vWrXjuuefw5ptv4uWXX+4w7tSpU/Dx8YFKpcITTzyBzZs3Y9iwYV0+ZmtrK+rq6jrcnNWwcD8oZAIuN+pQom2WOpw+1aRrx/+OG3u0sPcTkf2lRgcAcN1Ccl27ASv35AIAfnVzPBTcDqpPzBlvLCbffqYMxTVNEkdzbU7322AwGBASEoKVK1di1KhRmDlzJp599lmsWLGiw7jBgwfj+PHjOHjwIObPn49HH30UZ8+e7fIxly9fDo1GY7lFRzvvh6/aQ47BYcZ9qtytncHWU2Wob21H/0Av3BTXT+pwiFyeqxeSf3a8BKW1LQj2VeHno6KkDsdtDA7zRfrAfjCIwAf7C6QO55okTaCCgoIgl8tRXt5xA8Hy8nKEhYV1eZ/w8HAkJCRALr/S22fo0KEoKyuDTqezHFMqlRg4cCBGjRqF5cuXIyUlBX/729+6fMwlS5agtrbWcisqKrLB2Ukn2U03Ft5w2LhKZuaYaNYpEPWB5CgNZAJwqbYFFXXO0fywu/QGEStMmwY/PiEOag/2k+tLvzS1NFh/qBBNunaJo+mapAmUUqnEqFGjsGPHDssxg8GAHTt2XLNeKT09HTk5OTAYDJZj2dnZCA8Ph1KpvOZzGQwGtLZ23RtJpVJZWh6Yb84s1bSly0k3Wol3sbIBh/NrIBPAb4pEfcRbpcCgEOOMt6tdxtt+pgy5VY3wUyvw4E0xUofjdiYPDkFsPy/UtbTjk6MlUofTJckv4S1evBirVq3CunXrcO7cOcyfPx+NjY2YO3cuAOCRRx7BkiVLLOPnz5+P6upqLFy4ENnZ2diyZQuWLVuGjIwMy5glS5Zgz549yM/Px6lTp7BkyRLs2rULDz74YJ+fnxTMM1CnS2phcJN9qjaaiscnDw5BqJ9a4miI3EeqCzbUFEUR/9yVAwCYMz4WPiqFxBG5H5lMwKPjYwEAa/fmOeRnmeS/FTNnzkRlZSWef/55lJWVITU1Fdu2bbMUlhcWFkImu5LnRUdHY/v27Vi0aBGSk5MRGRmJhQsX4umnn7aMqaiowCOPPILS0lJoNBokJydj+/btmDp1ap+fnxQGhfhA7SFDfWs7cqsaXX7H8Da9AZ8c5cbBRFJI7e+PDZlFLpVA7blQhdMldfD0kGOO6VIS9b2fj4rCm19n42JlI77PqcKkhGCpQ+pA8gQKABYsWIAFCxZ0+bNdu3Z1OpaWlnbdnk7vv/++rUJzSgq5DIkRGmQW1OBksdblE6gd5ypQ1aBDsK8Kk9nkjqhPpV7VOkVvECF3gfrDf+40zj7NHtsfgd7XLg0h+/JVe+D+0VFYszcfq3/Ic7gESvJLeGQflkJyF/pWeC3m4vGfjYyCB5cZE/WphFBfeCnlaGhtx8XKBqnD6bUjBdU4mFcND7mAeTdz9klqc8bHQhCA3dmVyKlwrN8vftq4qBRTIfkJF29lUFrbjN3ZlQB4+Y5ICnKZgKRI4/uNK7Qz+OdO48q7+0ZEIVzjKXE0FNPPG7cNMZb0rDN1hHcUTKBclHkG6mxpHXTthusPdmKbMothEIFxcYGIM22kTER9y7wv3jEnn/E+V1qHHecrIAjAryfFSx0OmfwyPRYAsOlIMWqb2qQN5ipMoFxUbD8v+KkV0LUbkF1eL3U4dmEwiNiQaVx9x9knIumMcJGVeO+a+j7dlRiO+GDXrh11JmkD+mFwqC+a2/TYkFkodTgWTKBclCAILt9Qc9/FyyiuaYavWoE7E7lHFZFUzFu6ZJXVOWzTwxspuNyIL08at4Kaf8sAiaOhqwmCgLmmWah1+wrQrneMqypMoFxYcpRrN9Q0zz7dkxoBTyW7BBNJJUyjRpifGgYROF3inHuJvrcnFwYRmJQQjERTTRc5jhkjIhHg5YESbTO+PVd+4zv0ASZQLsyVZ6BqGnXYftq44fSsMf0ljoaIzAtXjhfVSByJ9crrWrAp09hLLmPyQImjoa6oPeT4xTjje/3qvfnSBmPCBMqFmd/QLlQ0oFmnlzga2/rseAl0egOGR/jx2yKRAzBfxnPGOqj3f8iDTm/A6JgAjI0LlDocuoaHb4qFQibgUF41TpdIf2WFCZQLC/NTI9hXBb1BxJlL0v+y2Yooilh/iMXjRI7EsqWLk7Uy0Dbp8J8DBQCAJyez9smRhWnUuDPJWO+6xgFmoZhAuTBBEJAS5Xr9oE4U1yKrvB4qhQz3pERKHQ4RwVhzKROAS7UtqKhrkTqcblu3rwBNOj2GhPli8mDuZODozMXk/ztxCVUNrZLGwgTKxZnroE66UB3UBtPGwXclhUPj5SFxNEQEAN4qBRJCfQE4Tz+oxtZ2rNmXBwB4cvJACILzb0Pj6kb2D8C8iXFYPWcM+km8zQ4TKBdnWYnnIjNQja3t+OJ4CQDggdG8fEfkSFKdrB/UR4cKoW1qQ2w/L9ydxFYozuLZu4dhwqAgyRNeJlAuzjwDlVfViNpmx+ng2lNbTpWiUadHbD8v3BTPYk8iR+JMdVCt7Xr863vj7NOvJw1wiU2QqW8xgXJxgd5KRAca93M65QKzUBtNl+8eGBMt+bcPIurIvKXLyWIt9AZR2mBuYPPREpTVtSDUT4X7RrKWkqzHBMoNuEo/qJyKemQW1EAuE/DzkVFSh0NEPzIoxBfeSjkadXrkVDRIHc416Q0i3tuTCwCYNzEeKgUb8ZL1mEC5gRRLHZRW2kB6yVw8PnlwCEL81BJHQ0Q/JpcJSDKv/HXgOqivTpcir6oR/l4emD2WjXipZ5hAuYErK/Gc9xKert2AT48ai8dnsfcTkcNKMdVBOepKPFEU8c5O46bBc8bHwlulkDgiclZMoNxAYqQGggCU1ragot55+rNcbce5clxu1CHEV4VbBgdLHQ4RXcMIB1+Jtyu7EudK6+CllGPO+FipwyEnxgTKDfioFBgY7APAeTcWNm8c/PNRUVDI+WtL5KjMW7pkldWhSdcucTSd/XNnDgDgF2P7w99L2j5C5Nz4SeQmnLmh5iVtM3ZnVwJg7yciRxemUSPMTw2D6Hgrfw/nV+Nwfg2UchkenxgvdTjk5JhAuQnzxsLOuKXLx5nFEEXgpvhAxAZ5Sx0OEd2AozbUNM8+/WxUJMI0XIhCvcMEyk1cPQMlio7dn+VqBoOIjabLd7PGcLUMkTMw94NypATqzKVa7MyqhEwAfn0zNw2m3mMC5SaGhvvCQy6gpqkNxTXNUofTbXsvVqFE2wxftQJ3JIZJHQ4RdYMjzkC9u8u48u7u5AjOZJNNMIFyEyqFHEPC/AA4V0PN9abeT/eOiITag83uiJxBUqQGMtPK3/I66Vf+5lU1YuupUgDA/EmcfSLbYALlRpxtY+HqRh2+OVMOAJjJ3k9ETsNbpUBCqC8A4JgD7Iv33u6LMIjArUNCMCzCT+pwyEUwgXIj5gZ3jtwh+Gqbj5VApzcgMdIPwyM0UodDRFYwX8aTesa7rLYFnxwtBgA8eQtnn8h2mEC5kRRTIfmpklqH3+hTFEVsOFwIAJjJ4nEip2Opg5J4BmrV97lo04sYGxeI0bGBksZCroUJlBsZGOIDL6UcTTo9LlY67kafgLH4NLu8AWoPGX6aEiF1OERkJfNKvJPFWsm+sNU06vDRIeMXMc4+ka0xgXIjcpmAxAjH3+gTuLJx8F2J4dB4ekgcDRFZa1CIL7yVcjTq9MipkOYL29p9+WjS6TE8wg+TErgFFNkWEyg34wyF5I2t7fjfiUsAWDxO5KzkMgFJpveb40U1ff78Da3tWLsvHwDw5C0DIQhCn8dAro0JlJtJNtUlOPKWLltOlqJRp0dckDfGxrFmgchZmffFk6If1EcHC1Hb3Ib4IG/2kCO7YALlZlJM3wjPldZD126QOJqurTcVjz8wOprfGomcmLmQvK9bGbS267Hq+1wAwBOTBkAu4/sI2R4TKDfTP9AL/l4e0OkNOF9WJ3U4nVwor8fRQi3kMgE/GxUpdThE1AsjTIXk2eX1aGxt77Pn/eRICSrqWxGuUWPGCL6PkH0wgXIzgiAgKdJxNxY2F4/fOiQEIb7c7JPImYX6qRGuUcMgGtun9IV2vQHv7TFu2/L4xHgoFfyYI/vgb5YbMveDOulgK/F07QZ8eqwEADCLxeNELiG1jxv4bjlVioLLTQjw8sDssXwfIfthAuWGHHUl3rfnylHdqEOon4pLjolcREofbiwsiqJl0+C56XHwUirs/pzkvphAuSHzG9qFino06fquLuFGzBsH/3xUFBRy/moSuYLUPkygvjtfgfNl9fBWyvFoWqzdn4/cGz+l3FConxqhfioYROB0iWMUkhfXNOH7C5UAjKvviMg1JEVqIBOA0toWlNe12O15RFHEP02zTw/dFAONFxvwkn0xgXJTyeY6KAfpB7XpSDFEERg/oB9i+nlLHQ4R2Yi3SoGEUF8A9m1ncCivGkcKaqBUyPDYhDi7PQ+RGRMoN2XuB+UIK/H0BhEfZxp3S2fncSLXY25nYM/LeO+YZp/uHxWFED+u4CX7YwLlphxpBuqHnCqUaJuh8fTAtOHsGEzkaq7UQdlnS5fTJbXYk10JmQD8+mZuGkx9gwmUmzKvxCu43ARtk07SWDaaisfvHREJtYdc0liIyPbMW7qcKq6F3iDa/PHNK++mp0Sgfz8vmz8+UVeYQLkpfy8lYkxvNFK2M7jc0Iqvz5YBYPE4kasaGOIDb6UcjTo9LlTU2/SxL1Y2YOvpUgDA/Fs4+0R9hwmUG3OEy3ibj5WgTS8iOUqDYRF+ksVBRPYjlwmW95vjNi4kf2/3RYgiMGVoCIaE8T2E+g4TKDcmdSG5KIqWrVtYPE7k2lJNheQnbPiF7ZK2GZ8eNe5e8OTkgTZ7XKLuYALlxqSegTpaqMWFigaoPWSYnhIhSQxE1DfMW0jZspXBqu9z0W4QcVN8IEb2D7DZ4xJ1BxMoN5YY6QeZAJTXtdq1wd21mIvH706KgJ+aTe+IXJm5lUF2eT0aW3u/A8LlhlasP2R8D3nyFs4+Ud9jAuXGvJQKDAoxNrjrq40+zRpa2/G/k5cA8PIdkTsI9VMjXKOGQQROlfS+bGDtvnw0t+mRFKnBxEFBNoiQyDpMoNycVBsLf3niEpp0esQHeWNMLKfeidyBrfbFq29pw7p9+QCAJ28ZAEEQehcYUQ8wgXJzyaY3NFsWdnbHhswrxeN88yNyD5YEqpd1UP89WIi6lnYMCPZm812SDBMoN2deiXeqpBaiaPsGd13JKqvHsUItFDIB942M6pPnJCLp2WIGqqVNj/d/yAMAPDFpAGQyfgEjaTCBcnNDwvyglMugbWpDYXVTnzynuXXBbUNDEOyr6pPnJCLpJUVpIJcJKKtrQVltzxaubDpSjMr6VkRo1LgnNdLGERJ1HxMoN6dUyDA03FhIbs+NPs1a2/XYfMy4cfCsMf3t/nxE5Di8lAokhJrfb6zfF69db8B7e4zbtvzq5ngoFfwII+nwt4+u6gdl/0Lyb86Wo6apDWF+atycEGz35yMix2K+jHesB1/YvjxZiqLqZvTzVmImv4CRxJhAEVJMb2h90VDTfPnu/tFRkLN2gcjtjDAvXLEygTIYRPxzVw4A4JcT4uCp5MbjJC0mUGQpJD9dUod2vcFuz1NU3YQfcqoAcONgIndl/sJ2qrgWekP3F67sOF+B7PIG+KgUeOimGDtFR9R9TKAI8cHGndKb2/TIqWyw2/N8fKQYogikD+yH6EAvuz0PETmugSHG95tGnR4XKuq7dR9RFPHOTuPs00M3xUDjyZ0LSHpMoAhymYDESFNDzSL71EHpDSI2WXo/sXaByF3JZYKl7rK7/aD2517G8SItVAoZHpsQZ7/giKzgEAnUO++8g9jYWKjVaowbNw6HDh267nitVouMjAyEh4dDpVIhISEBW7dutfx8+fLlGDNmDHx9fRESEoIZM2YgKyvL3qfh1FLs3FDz+wuVuFTbAn8vD9w+LNQuz0FEziHVtC9ed1f+vrvLuPLugdHRbH1CDkPyBGrDhg1YvHgxli5diqNHjyIlJQXTpk1DRUVFl+N1Oh2mTp2K/Px8bNq0CVlZWVi1ahUiI6/0A9m9ezcyMjJw4MABfPPNN2hra8Ptt9+OxsbGvjotp2PvLV3MxeMzUiOh9mDxJ5E7s6ah5sliLb6/UAW5TMCvbo63b2BEVlBIHcBbb72FefPmYe7cuQCAFStWYMuWLVi9ejWeeeaZTuNXr16N6upq7Nu3Dx4exuvgsbGxHcZs27atw5/Xrl2LkJAQHDlyBDfffLN9TsTJpZim1M+X1aG1XQ+VwnZJTlVDK749Vw6AGwcT0ZWVeNnl9WhsbYe36tofRf/caZx9uiclgrWT5FAknYHS6XQ4cuQIpkyZYjkmk8kwZcoU7N+/v8v7fPHFF0hLS0NGRgZCQ0ORmJiIZcuWQa/XX/N5amuNsyqBgYG2PQEXEhXgiQAvD7TpRZwr7V5hZ3dtPlqCNr2IlCgNhob72fSxicj5hPipEaFRwyBef9Y7p6Ie28+WAQCeuGVAX4VH1C2SJlBVVVXQ6/UIDe1YExMaGoqysrIu75Obm4tNmzZBr9dj69ateO655/Dmm2/i5Zdf7nK8wWDAU089hfT0dCQmJnY5prW1FXV1dR1u7kYQhKsaampt9riiKGL94UIALB4noiu6Uwf17q5ciCJw+7BQSwdzIkcheQ2UtQwGA0JCQrBy5UqMGjUKM2fOxLPPPosVK1Z0OT4jIwOnT5/G+vXrr/mYy5cvh0ajsdyio93zMpO5H9QJG67EO1pYg4uVjfD0kGN6SrjNHpeInNuVOqiut3Qp0Tbj8+MlAIAnJw/sq7CIuk3SBCooKAhyuRzl5eUdjpeXlyMsLKzL+4SHhyMhIQFy+ZUanaFDh6KsrAw6na7D2AULFuDLL7/Ezp07ERUVdc04lixZgtraWsutqKioF2flvOwxA7X+kPHv8u7kcPiq2buFiIxSowMAXPsL26o9uWg3iEgf2M+SbBE5EkkTKKVSiVGjRmHHjh2WYwaDATt27EBaWlqX90lPT0dOTg4Mhisds7OzsxEeHg6lUgnAeNlowYIF2Lx5M7777jvExV2/b4hKpYKfn1+HmztKjjbOQOVUNqChtb3Xj1ff0oYvT5YCAGaxeJyIrpIY6Qe5TEBZXQvKals6/KyqoRUfHTJe+n/yFs4+kWOS/BLe4sWLsWrVKqxbtw7nzp3D/Pnz0djYaFmV98gjj2DJkiWW8fPnz0d1dTUWLlyI7OxsbNmyBcuWLUNGRoZlTEZGBv7zn//gww8/hK+vL8rKylBWVobm5uY+Pz9nEuKrRrhGDVEETpf0/jLelydL0dymx4Bgb4yKCbBBhETkKryUCktd048v463Zm4fWdgNSojQYP6CfFOER3ZDkbQxmzpyJyspKPP/88ygrK0Nqaiq2bdtmKSwvLCyETHYlz4uOjsb27duxaNEiJCcnIzIyEgsXLsTTTz9tGfPuu+8CAG655ZYOz7VmzRrMmTPH7ufkzJKjNCitbcHJYi1uiu/dG9f6w+bO49EQBG4cTEQdpUb741xpHY4VaXFHorFGsq6lDR/sKwBgrH3iewc5KskTKMBYq7RgwYIuf7Zr165Ox9LS0nDgwIFrPp4odn+DSuooOcof28+U40QvG2qeL6vDiSItFDIB9428dv0ZEbmvEdH++OhQYYctXf5zoAD1re0YFOKDqUO5awE5LodIoMhxpNiokNzceXzqsFAE+XDrBSLqzNzK4FRJLfQGEW16A1b/kAcAmH/LAMhknH0ix8UEijpIMrUyKKpuRnWjDoHeSqsfo7Vdj83HjMuPH2DxOBFdw4BgH/ioFGhobUd2eT0O51ejqkGHSH9PTE+JkDo8ouuSvIicHIvG0wNxQd4Aej4L9fWZcmib2hCuUePmQcE2jI6IXIlcJlj24czMr8Z7u3MBAL+eFA8POT+eyLHxN5Q66e3GwubLd/ePjoacU/BEdB3mHk9//y4HJdpmBPko8cBozlyT42MCRZ30pqFmUXUTfsipgiAA949i8TgRXZ85gaqsbwUA/HJCHNQettvMnMhemEBRJ5YtXYprrV7R+HGmcfZpwsAg7pxORDd0dZdxX5UCD90UI10wRFZgAkWdDI/QQC4TUFnfirK6lhvfwURvELExsxgAOAVPRN0S4qdGhEYNAHhkfAz8uOUTOQkmUNSJp1KOQSE+AKzbWHhPdiXK6lrg7+WB24ezfwsRdc//u2MIfpIcjl9NHCB1KETdxgSKutSTflDm4vH7RkRBpWANAxF1z4wRkXj7FyOh8eLsEzkPJlDUJfPGwie6mUBV1rfi23PlAIxbtxAREbkyJlDUpSszULUwGG5cSP7p0WK0G0SkRvtjcJivnaMjIiKSFhMo6tLgMF8oFTLUt7Qj/3LjdceKoogNptV3szj7REREboAJFHXJQy7DsHA/ADduqJlZUIPcykZ4KeX4CbdfICIiN8AEiq7pSj8o7XXHmYvHf5IcDh8Vt1ckIiLXxwSKrinF1ODuejNQdS1t2HKyFAAwc0z/vgiLiIhIckyg6JrMW7qcuVSLdr2hyzH/O3EJzW16DAzxwcj+/n0XHBERkYSYQNE1xQd5w1elQEubAdnlDV2O2Xj4SvG4IHDjYCIicg9MoOiaZDIBiZHGOqiuGmqevVSHE8W18JALuHdEZB9HR0REJB0mUHRdVxpqdq6D2mhqXTB1WCj6+aj6NC4iIiIpMYGi67rWli4tbXpsPlYCgMXjRETkfphA0XUlm1oZZJXVo6VNbzm+/UwZapvbEKFRY8LAIKnCIyIikgQTKLquSH9P9PNWot0g4mxpneW4uffT/aOjIZexeJyIiNwLEyi6LkEQLLNQJ4u0AIDCy03Yd/EyBAG4f3SUhNERERFJgwkU3VDyVRsLA1eKxycMDEJUgJdUYREREUmGCRTdUEr0lS1d2vUGfHzE3PuJxeNEROSemEDRDZlnoHKrGrHlVCnK61oR6K3ElGEh0gZGREQkESZQdENBPipE+ntCFIHlW88DAO4dEQmVQi5xZERERNJgAkXdYi4kL6trAQDMHBMtZThERESSYgJF3WK+jAcAI/v7IyHUV7pgiIiIJMYEirolxTQDBXD2iYiISCF1AOQckqP94e/lAYVMwN3JEVKHQ0REJCkmUNQtPioFvvzNBMhlAnxU/LUhIiL3xk9C6jY2zSQiIjJiDRQRERGRlZhAEREREVmJCRQRERGRlZhAEREREVmJCRQRERGRlZhAEREREVmJCRQRERGRlZhAEREREVmJCRQRERGRlZhAEREREVmJCRQRERGRlZhAEREREVmJCRQRERGRlZhAEREREVlJIXUAjkgURQBAXV2dxJEQERFRXzF/7pvzgOthAtWF+vp6AEB0dLTEkRAREVFfq6+vh0ajue4YQexOmuVmDAYDLl26BF9fXwiCYNPHrqurQ3R0NIqKiuDn52fTx3Y07nKuPE/XwvN0Pe5yrjzP3hNFEfX19YiIiIBMdv0qJ85AdUEmkyEqKsquz+Hn5+fSv+BXc5dz5Xm6Fp6n63GXc+V59s6NZp7MWEROREREZCUmUERERERWYgLVx1QqFZYuXQqVSiV1KHbnLufK83QtPE/X4y7nyvPsWywiJyIiIrISZ6CIiIiIrMQEioiIiMhKTKCIiIiIrMQEioiIiMhKTKBs4J133kFsbCzUajXGjRuHQ4cOXXf8xx9/jCFDhkCtViMpKQlbt27t8HNRFPH8888jPDwcnp6emDJlCi5cuGDPU+gWa85z1apVmDhxIgICAhAQEIApU6Z0Gj9nzhwIgtDhdscdd9j7NG7ImvNcu3Ztp3NQq9Udxjjq6wlYd6633HJLp3MVBAF33323ZYyjvaZ79uzB9OnTERERAUEQ8Nlnn93wPrt27cLIkSOhUqkwcOBArF27ttMYa//N9wVrz/XTTz/F1KlTERwcDD8/P6SlpWH79u0dxrzwwgudXs8hQ4bY8SxuzNrz3LVrV5e/t2VlZR3GOdprau15dvVvTxAEDB8+3DLGEV/P5cuXY8yYMfD19UVISAhmzJiBrKysG97PET5HmUD10oYNG7B48WIsXboUR48eRUpKCqZNm4aKiooux+/btw+zZ8/GY489hmPHjmHGjBmYMWMGTp8+bRnz+uuv4+9//ztWrFiBgwcPwtvbG9OmTUNLS0tfnVYn1p7nrl27MHv2bOzcuRP79+9HdHQ0br/9dpSUlHQYd8cdd6C0tNRy++ijj/ridK7J2vMEjN1wrz6HgoKCDj93xNcTsP5cP/300w7nefr0acjlctx///0dxjnSa9rY2IiUlBS888473Rqfl5eHu+++G5MnT8bx48fx1FNP4fHHH++QWPTkd6QvWHuue/bswdSpU7F161YcOXIEkydPxvTp03Hs2LEO44YPH97h9fzhhx/sEX63WXueZllZWR3OIyQkxPIzR3xNrT3Pv/3tbx3Or6ioCIGBgZ3+fTra67l7925kZGTgwIED+Oabb9DW1obbb78djY2N17yPw3yOitQrY8eOFTMyMix/1uv1YkREhLh8+fIuxz/wwAPi3Xff3eHYuHHjxF//+teiKIqiwWAQw8LCxD//+c+Wn2u1WlGlUokfffSRHc6ge6w9zx9rb28XfX19xXXr1lmOPfroo+I999xj61B7xdrzXLNmjajRaK75eI76eopi71/Tv/zlL6Kvr6/Y0NBgOeaIr6kZAHHz5s3XHfP//t//E4cPH97h2MyZM8Vp06ZZ/tzbv7e+0J1z7cqwYcPEF1980fLnpUuXiikpKbYLzMa6c547d+4UAYg1NTXXHOPor2lPXs/NmzeLgiCI+fn5lmOO/nqKoihWVFSIAMTdu3dfc4yjfI5yBqoXdDodjhw5gilTpliOyWQyTJkyBfv37+/yPvv37+8wHgCmTZtmGZ+Xl4eysrIOYzQaDcaNG3fNx7S3npznjzU1NaGtrQ2BgYEdju/atQshISEYPHgw5s+fj8uXL9s0dmv09DwbGhoQExOD6Oho3HPPPThz5ozlZ474egK2eU3ff/99zJo1C97e3h2OO9Jraq0b/fu0xd+bozIYDKivr+/0b/TChQuIiIhAfHw8HnzwQRQWFkoUYe+kpqYiPDwcU6dOxd69ey3HXfU1ff/99zFlyhTExMR0OO7or2dtbS0AdPo9vJqjfI4ygeqFqqoq6PV6hIaGdjgeGhra6fq6WVlZ2XXHm/9rzWPaW0/O88eefvppREREdPiFvuOOO/DBBx9gx44deO2117B7927ceeed0Ov1No2/u3pynoMHD8bq1avx+eef4z//+Q8MBgPGjx+P4uJiAI75egK9f00PHTqE06dP4/HHH+9w3NFeU2td699nXV0dmpubbfJvwVG98cYbaGhowAMPPGA5Nm7cOKxduxbbtm3Du+++i7y8PEycOBH19fUSRmqd8PBwrFixAp988gk++eQTREdH45ZbbsHRo0cB2Ob9zdFcunQJX331Vad/n47+ehoMBjz11FNIT09HYmLiNcc5yueowmaPRHQNr776KtavX49du3Z1KLCeNWuW5f+TkpKQnJyMAQMGYNeuXbjtttukCNVqaWlpSEtLs/x5/PjxGDp0KN577z386U9/kjAy+3r//feRlJSEsWPHdjjuCq+pO/rwww/x4osv4vPPP+9QG3TnnXda/j85ORnjxo1DTEwMNm7ciMcee0yKUK02ePBgDB482PLn8ePH4+LFi/jLX/6Cf//73xJGZj/r1q2Dv78/ZsyY0eG4o7+eGRkZOH36tOR1Wd3FGaheCAoKglwuR3l5eYfj5eXlCAsL6/I+YWFh1x1v/q81j2lvPTlPszfeeAOvvvoqvv76ayQnJ193bHx8PIKCgpCTk9PrmHuiN+dp5uHhgREjRljOwRFfT6B359rY2Ij169d36w1X6tfUWtf69+nn5wdPT0+b/I44mvXr1+Pxxx/Hxo0bO10W+TF/f38kJCQ4zet5LWPHjrWcg6u9pqIoYvXq1Xj44YehVCqvO9aRXs8FCxbgyy+/xM6dOxEVFXXdsY7yOcoEqheUSiVGjRqFHTt2WI4ZDAbs2LGjw6zE1dLS0jqMB4BvvvnGMj4uLg5hYWEdxtTV1eHgwYPXfEx768l5AsZVEH/605+wbds2jB49+obPU1xcjMuXLyM8PNwmcVurp+d5Nb1ej1OnTlnOwRFfT6B35/rxxx+jtbUVDz300A2fR+rX1Fo3+vdpi98RR/LRRx9h7ty5+Oijjzq0o7iWhoYGXLx40Wlez2s5fvy45Rxc7TXdvXs3cnJyuvUFxxFeT1EUsWDBAmzevBnfffcd4uLibngfh/kctVk5uptav369qFKpxLVr14pnz54Vf/WrX4n+/v5iWVmZKIqi+PDDD4vPPPOMZfzevXtFhUIhvvHGG+K5c+fEpUuXih4eHuKpU6csY1599VXR399f/Pzzz8WTJ0+K99xzjxgXFyc2Nzf3+fmZWXuer776qqhUKsVNmzaJpaWlllt9fb0oiqJYX18v/v73vxf3798v5uXlid9++604cuRIcdCgQWJLS4sk5yiK1p/niy++KG7fvl28ePGieOTIEXHWrFmiWq0Wz5w5YxnjiK+nKFp/rmYTJkwQZ86c2em4I76m9fX14rFjx8Rjx46JAMS33npLPHbsmFhQUCCKoig+88wz4sMPP2wZn5ubK3p5eYl/+MMfxHPnzonvvPOOKJfLxW3btlnG3OjvTSrWnut///tfUaFQiO+8806Hf6NardYy5ne/+524a9cuMS8vT9y7d684ZcoUMSgoSKyoqOjz8zOz9jz/8pe/iJ999pl44cIF8dSpU+LChQtFmUwmfvvtt5YxjviaWnueZg899JA4bty4Lh/TEV/P+fPnixqNRty1a1eH38OmpibLGEf9HGUCZQP/+Mc/xP79+4tKpVIcO3aseODAAcvPJk2aJD766KMdxm/cuFFMSEgQlUqlOHz4cHHLli0dfm4wGMTnnntODA0NFVUqlXjbbbeJWVlZfXEq12XNecbExIgAOt2WLl0qiqIoNjU1ibfffrsYHBwsenh4iDExMeK8efMk/xASRevO86mnnrKMDQ0NFe+66y7x6NGjHR7PUV9PUbT+d/f8+fMiAPHrr7/u9FiO+Jqal7D/+GY+r0cffVScNGlSp/ukpqaKSqVSjI+PF9esWdPpca/39yYVa8910qRJ1x0visYWDuHh4aJSqRQjIyPFmTNnijk5OX17Yj9i7Xm+9tpr4oABA0S1Wi0GBgaKt9xyi/jdd991elxHe0178rur1WpFT09PceXKlV0+piO+nl2dI4AO/+4c9XNUMJ0AEREREXUTa6CIiIiIrMQEioiIiMhKTKCIiIiIrMQEioiIiMhKTKCIiIiIrMQEioiIiMhKTKCIiIiIrMQEiojISi+88AJSU1OlDoOIJMQEioiIiMhKTKCIiIiIrMQEiohcyrZt2zBhwgT4+/ujX79++MlPfoKLFy8CAPLz8yEIAj799FNMnjwZXl5eSElJwf79+zs8xqpVqxAdHQ0vLy/ce++9eOutt+Dv73/d5/3Xv/6FoUOHQq1WY8iQIfjnP/9pr1MkIgfABIqIXEpjYyMWL16MzMxM7NixAzKZDPfeey8MBoNlzLPPPovf//73OH78OBISEjB79my0t7cDAPbu3YsnnngCCxcuxPHjxzF16lS88sor133O//73v3j++efxyiuv4Ny5c1i2bBmee+45rFu3zq7nSkTS4WbCROTSqqqqEBwcjFOnTsHHxwdxcXH417/+hcceewwAcPbsWQwfPhznzp3DkCFDMGvWLDQ0NODLL7+0PMZDDz2EL7/8ElqtFoCxiPyzzz7D8ePHAQADBw7En/70J8yePdtyn5dffhlbt27Fvn37+uxciajvcAaKiFzKhQsXMHv2bMTHx8PPzw+xsbEAgMLCQsuY5ORky/+Hh4cDACoqKgAAWVlZGDt2bIfH/PGfr9bY2IiLFy/iscceg4+Pj+X28ssvWy4dEpHrUUgdABGRLU2fPh0xMTFYtWoVIiIiYDAYkJiYCJ1OZxnj4eFh+X9BEACgwyU+azQ0NAAw1k2NGzeuw8/kcnmPHpOIHB8TKCJyGZcvX0ZWVhZWrVqFiRMnAgB++OEHqx5j8ODBOHz4cIdjP/7z1UJDQxEREYHc3Fw8+OCD1gdNRE6JCRQRuYyAgAD069cPK1euRHh4OAoLC/HMM89Y9Ri/+c1vcPPNN+Ott97C9OnT8d133+Grr76yzFR15cUXX8Rvf/tbaDQa3HHHHWhtbUVmZiZqamqwePHi3p4WETkg1kARkcuQyWRYv349jhw5gsTERCxatAh//vOfrXqM9PR0rFixAm+99RZSUlKwbds2LFq0CGq1+pr3efzxx/Gvf/0La9asQVJSEiZNmoS1a9ciLi6ut6dERA6Kq/CIiG5g3rx5OH/+PL7//nupQyEiB8FLeEREP/LGG29g6tSp8Pb2xldffYV169axMSYRdcAZKCKiH3nggQewa9cu1NfXIz4+Hr/5zW/wxBNPSB0WETkQJlBEREREVmIROREREZGVmEARERERWYkJFBEREZGVmEARERERWYkJFBEREZGVmEARERERWYkJFBEREZGVmEARERERWYkJFBEREZGV/j8nOl6WsUCOPwAAAABJRU5ErkJggg==\n"
          },
          "metadata": {}
        }
      ]
    },
    {
      "cell_type": "markdown",
      "metadata": {
        "id": "ccfdbRcQ7Zgg"
      },
      "source": [
        "We can do multiple augmentations at the same time. Here is a function to do both shear and rotation to the sample. In this case, we will have two hyperparameters."
      ]
    },
    {
      "cell_type": "code",
      "metadata": {
        "id": "sh8S_Pxa0XCv"
      },
      "source": [
        "def augRotateShear(sample, angleconstraint, shearconstraint):\n",
        "  if len(sample.shape)==2:\n",
        "    sample = np.expand_dims(sample, 0)  # make sure the sample is 3 dimensional\n",
        "  amt = rng.random(len(sample)) # generate random numbers for shear\n",
        "  amt = (amt-0.5)*shearconstraint # make the random shear constrained\n",
        "  angle = rng.random(len(sample)) # generate random numbers for angles\n",
        "  angle = (angle-0.5)*angleconstraint # make the random angle constrained\n",
        "  nsample = sample.copy() # preallocate the augmented array to make it faster\n",
        "  for ii in range(len(sample)):\n",
        "    nsample[ii] = rotate(shear(sample[ii], amt[ii]), angle[ii]) # first apply shear, then rotate\n",
        "  return np.squeeze(nsample) # take care if the input had only one sample."
      ],
      "execution_count": 19,
      "outputs": []
    },
    {
      "cell_type": "markdown",
      "metadata": {
        "id": "OGKyjjNx-NQ4"
      },
      "source": [
        "Since we have two hyperparameters, we have to do the grid search on a 2 dimensional matrix. We can use our previous experience to inform where to search for the best hyperparameters."
      ]
    },
    {
      "cell_type": "code",
      "metadata": {
        "id": "TJC45WRg0pOP",
        "colab": {
          "base_uri": "https://localhost:8080/"
        },
        "outputId": "cf72a178-3785-4b7f-fe04-f3ccc526fb18"
      },
      "source": [
        "shearconstraints = [0, 0.2,0.4,0.6,0.8,1.0,1.2,1.4,1.6] # the values we want to test\n",
        "angleconstraints = [0,10,20,30,40,50,60] # the values we want to test\n",
        "hyp = cartesian((shearconstraints, angleconstraints)) # cartesian product of both\n",
        "\n",
        "accuracies = np.zeros(len(hyp), dtype=np.float) # we will save the values here\n",
        "\n",
        "for ii in range(len(hyp)):\n",
        "  # create the augmented dataset\n",
        "  augdata = train_X # we include the original images also in the augmented dataset\n",
        "  auglabel = train_y\n",
        "  for jj in range(naugmentations):\n",
        "    augdata = np.concatenate((augdata, augRotateShear(train_X, hyp[ii][0], hyp[ii][1]))) # concatenate the augmented data to the set\n",
        "    auglabel = np.concatenate((auglabel, train_y))  # the labels don't change when we augment\n",
        "\n",
        "  # check the test accuracy\n",
        "  testpred = NN(augdata, auglabel, test_X)\n",
        "  accuracies[ii] = Accuracy(test_y, testpred)\n",
        "  print('Accuracy after augmentation shear:',hyp[ii][0], 'angle:',hyp[ii][1], ' is ', accuracies[ii], flush=True)"
      ],
      "execution_count": 21,
      "outputs": [
        {
          "output_type": "stream",
          "name": "stderr",
          "text": [
            "<ipython-input-21-09de780976d0>:5: DeprecationWarning: `np.float` is a deprecated alias for the builtin `float`. To silence this warning, use `float` by itself. Doing this will not modify any behavior and is safe. If you specifically wanted the numpy scalar type, use `np.float64` here.\n",
            "Deprecated in NumPy 1.20; for more details and guidance: https://numpy.org/devdocs/release/1.20.0-notes.html#deprecations\n",
            "  accuracies = np.zeros(len(hyp), dtype=np.float) # we will save the values here\n"
          ]
        },
        {
          "output_type": "stream",
          "name": "stdout",
          "text": [
            "Accuracy after augmentation shear: 0.0 angle: 0.0  is  0.6332\n",
            "Accuracy after augmentation shear: 0.0 angle: 10.0  is  0.5942\n",
            "Accuracy after augmentation shear: 0.0 angle: 20.0  is  0.6024\n",
            "Accuracy after augmentation shear: 0.0 angle: 30.0  is  0.5797\n",
            "Accuracy after augmentation shear: 0.0 angle: 40.0  is  0.583\n",
            "Accuracy after augmentation shear: 0.0 angle: 50.0  is  0.6271\n",
            "Accuracy after augmentation shear: 0.0 angle: 60.0  is  0.6088\n",
            "Accuracy after augmentation shear: 0.2 angle: 0.0  is  0.6338\n",
            "Accuracy after augmentation shear: 0.2 angle: 10.0  is  0.5931\n",
            "Accuracy after augmentation shear: 0.2 angle: 20.0  is  0.5802\n",
            "Accuracy after augmentation shear: 0.2 angle: 30.0  is  0.6063\n",
            "Accuracy after augmentation shear: 0.2 angle: 40.0  is  0.6026\n",
            "Accuracy after augmentation shear: 0.2 angle: 50.0  is  0.5791\n",
            "Accuracy after augmentation shear: 0.2 angle: 60.0  is  0.6205\n",
            "Accuracy after augmentation shear: 0.4 angle: 0.0  is  0.6344\n",
            "Accuracy after augmentation shear: 0.4 angle: 10.0  is  0.6096\n",
            "Accuracy after augmentation shear: 0.4 angle: 20.0  is  0.5902\n",
            "Accuracy after augmentation shear: 0.4 angle: 30.0  is  0.6228\n",
            "Accuracy after augmentation shear: 0.4 angle: 40.0  is  0.5872\n",
            "Accuracy after augmentation shear: 0.4 angle: 50.0  is  0.6278\n",
            "Accuracy after augmentation shear: 0.4 angle: 60.0  is  0.5818\n",
            "Accuracy after augmentation shear: 0.6 angle: 0.0  is  0.6344\n",
            "Accuracy after augmentation shear: 0.6 angle: 10.0  is  0.598\n",
            "Accuracy after augmentation shear: 0.6 angle: 20.0  is  0.596\n",
            "Accuracy after augmentation shear: 0.6 angle: 30.0  is  0.5984\n",
            "Accuracy after augmentation shear: 0.6 angle: 40.0  is  0.6057\n",
            "Accuracy after augmentation shear: 0.6 angle: 50.0  is  0.5947\n",
            "Accuracy after augmentation shear: 0.6 angle: 60.0  is  0.6053\n",
            "Accuracy after augmentation shear: 0.8 angle: 0.0  is  0.6342\n",
            "Accuracy after augmentation shear: 0.8 angle: 10.0  is  0.6056\n",
            "Accuracy after augmentation shear: 0.8 angle: 20.0  is  0.6143\n",
            "Accuracy after augmentation shear: 0.8 angle: 30.0  is  0.5948\n",
            "Accuracy after augmentation shear: 0.8 angle: 40.0  is  0.6209\n",
            "Accuracy after augmentation shear: 0.8 angle: 50.0  is  0.5965\n",
            "Accuracy after augmentation shear: 0.8 angle: 60.0  is  0.5821\n",
            "Accuracy after augmentation shear: 1.0 angle: 0.0  is  0.6342\n",
            "Accuracy after augmentation shear: 1.0 angle: 10.0  is  0.6276\n",
            "Accuracy after augmentation shear: 1.0 angle: 20.0  is  0.5801\n",
            "Accuracy after augmentation shear: 1.0 angle: 30.0  is  0.5852\n",
            "Accuracy after augmentation shear: 1.0 angle: 40.0  is  0.5993\n",
            "Accuracy after augmentation shear: 1.0 angle: 50.0  is  0.6171\n",
            "Accuracy after augmentation shear: 1.0 angle: 60.0  is  0.5948\n",
            "Accuracy after augmentation shear: 1.2 angle: 0.0  is  0.6339\n",
            "Accuracy after augmentation shear: 1.2 angle: 10.0  is  0.594\n",
            "Accuracy after augmentation shear: 1.2 angle: 20.0  is  0.6032\n",
            "Accuracy after augmentation shear: 1.2 angle: 30.0  is  0.6069\n",
            "Accuracy after augmentation shear: 1.2 angle: 40.0  is  0.6273\n",
            "Accuracy after augmentation shear: 1.2 angle: 50.0  is  0.5793\n",
            "Accuracy after augmentation shear: 1.2 angle: 60.0  is  0.5882\n",
            "Accuracy after augmentation shear: 1.4 angle: 0.0  is  0.6349\n",
            "Accuracy after augmentation shear: 1.4 angle: 10.0  is  0.6184\n",
            "Accuracy after augmentation shear: 1.4 angle: 20.0  is  0.6158\n",
            "Accuracy after augmentation shear: 1.4 angle: 30.0  is  0.5866\n",
            "Accuracy after augmentation shear: 1.4 angle: 40.0  is  0.613\n",
            "Accuracy after augmentation shear: 1.4 angle: 50.0  is  0.6131\n",
            "Accuracy after augmentation shear: 1.4 angle: 60.0  is  0.6012\n",
            "Accuracy after augmentation shear: 1.6 angle: 0.0  is  0.6329\n",
            "Accuracy after augmentation shear: 1.6 angle: 10.0  is  0.6126\n",
            "Accuracy after augmentation shear: 1.6 angle: 20.0  is  0.6145\n",
            "Accuracy after augmentation shear: 1.6 angle: 30.0  is  0.6194\n",
            "Accuracy after augmentation shear: 1.6 angle: 40.0  is  0.5819\n",
            "Accuracy after augmentation shear: 1.6 angle: 50.0  is  0.5971\n",
            "Accuracy after augmentation shear: 1.6 angle: 60.0  is  0.6106\n"
          ]
        }
      ]
    },
    {
      "cell_type": "markdown",
      "metadata": {
        "id": "PT6CnvSDEX7a"
      },
      "source": [
        "Let us plot it two dimensionally to see which is the best value for the hyperparameters:"
      ]
    },
    {
      "cell_type": "code",
      "metadata": {
        "id": "jD2i7msI_cLd",
        "colab": {
          "base_uri": "https://localhost:8080/",
          "height": 481
        },
        "outputId": "093b39a5-4ad6-46d7-e1d9-d2105ced77ae"
      },
      "source": [
        "fig = plt.figure()\n",
        "ax = fig.add_axes([0.1, 0.1, 0.8, 0.8]) # main axes\n",
        "im = ax.imshow(accuracies.reshape((len(shearconstraints), len(angleconstraints))), cmap='inferno')\n",
        "ax.set_xlabel('angle')\n",
        "ax.set_ylabel('shear')\n",
        "ax.set_xticks(np.arange(len(angleconstraints)));\n",
        "ax.set_xticklabels(angleconstraints);\n",
        "ax.set_yticks(np.arange(len(shearconstraints)));\n",
        "ax.set_yticklabels(shearconstraints);\n",
        "plt.colorbar(im)"
      ],
      "execution_count": 22,
      "outputs": [
        {
          "output_type": "execute_result",
          "data": {
            "text/plain": [
              "<matplotlib.colorbar.Colorbar at 0x7e95169618d0>"
            ]
          },
          "metadata": {},
          "execution_count": 22
        },
        {
          "output_type": "display_data",
          "data": {
            "text/plain": [
              "<Figure size 640x480 with 2 Axes>"
            ],
            "image/png": "iVBORw0KGgoAAAANSUhEUgAAAccAAAG/CAYAAAAtjRweAAAAOXRFWHRTb2Z0d2FyZQBNYXRwbG90bGliIHZlcnNpb24zLjcuMSwgaHR0cHM6Ly9tYXRwbG90bGliLm9yZy/bCgiHAAAACXBIWXMAAA9hAAAPYQGoP6dpAABAMklEQVR4nO3dfVxUZd4/8M8MMoOkoKYMA6LYbT6Q8bCQNNKDrSi53qzV7uaWu7C02U+FXXK2vZUtwZ7AzWStvUmSQuzuQcuXmiViLoreJmag3FkpSKiQMSBrgJAyOuf8/lCOzQA2wwycmeHz7nW9Xs6Z61zzHXrh1+t7rnMdhSiKIoiIiEiilDsAIiIiZ8PkSEREZIHJkYiIyAKTIxERkQUmRyIiIgtMjkRERBaYHImIiCwwORIREVkYJHcA/U0QBHz33XcYOnQoFAqF3OEQEXVLFEVcuHABAQEBUCr7dh5z6dIlGI1Gh4ylUqng5eXlkLHkNOCS43fffYegoCC5wyAiskpdXR1Gjx7dZ+NfunQJ48b5w2Bocch4/v7+OHXqlMsnyAGXHIcOHQoAOHEyDUOHOv//vKcmfyd3CFaJHnlR7hCs9uTJ9+QOwWreqjFyh2CVM9td56+S/3suWO4QrNJ+5QrmHi6W/s7qK0ajEQZDC2rO/AM+PoPtGqu19SJuGbsERqORydHVdJZShw71go+P8//PUynUcodglcEeJrlDsIHrlNMVCg+5Q7CKz02uEScA3DTIU+4QbNJfl398fAbbnRzdyYBLjkRE1JUoXoEoXrF7DHfB5EhERBBFE0TRvgqQvec7E97KQUREZIEzRyIigiBegWBnWdTe850JkyMREfGaowWWVYmIiCxw5khERNcW5Ng7c3SfBTlMjkREBFG4AlGwMznaeb4zYVmViIjIAmeOREQEiFeuNnvHcBNMjkRExNWqFlhWJSIissCZIxERAcIVQLhs/xhugsmRiIiulVXte7oKy6pOICcnB8HBwfDy8kJ0dDQOHz4sd0hEROQmXDI5btq0CXq9HhkZGThy5AjCwsIQFxeHxsZGuUMjInJNwhXHNDfhkskxOzsbCxYsQFJSEkJCQpCbmwtvb2/k5+fLHRoRkWticjTjcsnRaDSivLwcsbGx0jGlUonY2FiUlpZ26d/R0YHW1lazRkREdCMulxybmppgMpmg0WjMjms0GhgMhi79s7Ky4OvrK7WgoKD+CpWIyIWYrm8E0NsG99lb1eWSo63S0tLQ0tIitbq6OrlDIiJyOgrhikOau3C5WzlGjhwJDw8PNDQ0mB1vaGiAv79/l/5qtRpqtbq/wiMiIjfgcjNHlUqFyMhIFBcXS8cEQUBxcTF0Op2MkRERuTAuyDHjcjNHANDr9UhMTERUVBSmTp2KNWvWoL29HUlJSXKHRkTkmoQrgGDnfInJUV7z5s3DuXPnkJ6eDoPBgPDwcBQVFXVZpENERNQbLpkcASAlJQUpKSlyh0FE5BYU4hUoRPtmjgo32j7OZZMjERE5kCAAgp23YgiCY2JxAi63IIeIiNyHrftkNzc3Izk5GVqtFmq1GhMmTEBhYaH0/tq1axEaGgofHx/4+PhAp9Nh586dNsfFmSMREV27T1Fh9xi26NwnOzc3F9HR0VizZg3i4uJQWVkJPz+/Lv2NRiNmzpwJPz8/bN68GYGBgThz5gyGDRsm9Rk9ejRWrlyJW2+9FaIoYsOGDZg7dy6OHj2K2267zerYmByJiOhqSdXu1aq2lWV/vE82AOTm5mLHjh3Iz8/HsmXLuvTPz8/H+fPncfDgQXh6egIAgoODzfrEx8ebvX7xxRexdu1aHDp0yKbkyLIqERE5lOV+1h0dHV362LpPNgBs374dOp0OycnJ0Gg0mDJlCjIzM2EydZ+UTSYTNm7ciPb2dpvvg2dyJCIih24CEBQUZLandVZWVpePs3WfbACoqanB5s2bYTKZUFhYiOXLl2P16tV44YUXzPodO3YMQ4YMgVqtxsKFC7F161aEhITY9ONgWZWIiKAQTFDYWVZVXCur1tXVwcfHRzruqC08BUGAn58f1q1bBw8PD0RGRuLs2bNYtWoVMjIypH4TJ05ERUUFWlpasHnzZiQmJmLfvn02JUgmRyIicqjOlaI3Yus+2QCg1Wrh6ekJDw8P6djkyZNhMBhgNBqhUqkAXN1mdPz48QCAyMhIfP7553jllVfw+uuvW/0dWFYlIiJANF1blGNHE61fkNObfbJjYmJQXV0N4Uf3U1ZVVUGr1UqJsTuCIHR73fNGmByJiAgKQbhWWrWn2bYJgF6vR15eHjZs2IDjx49j0aJFZvtkJyQkIC0tTeq/aNEinD9/HqmpqaiqqsKOHTuQmZmJ5ORkqU9aWhr279+P06dP49ixY0hLS0NJSQnmz59vU2wsqxIRkSx+ap/s2tpaKJXX53BBQUHYtWsXlixZgtDQUAQGBiI1NRVLly6V+jQ2NiIhIQH19fXw9fVFaGgodu3ahZkzZ9oUG5MjERFdK43atwlAb7afu9E+2SUlJV2O6XQ6HDp0qMfx3nzzTZtj6A6TIxERXSuL2rtDjp17szoRXnMkIiKywJkjERHJVlZ1VgM2OSqVXlAqveQO4yddMolyh2CVhZXvyR2C1XZEPih3CFb7rn2o3CFYRX33erlDsNqllbVyh2CV1ksicLD/Po9lVXMsqxIREVkYsDNHIiL6EZZVzTA5EhERFIJo80383Y3hLlhWJSIissCZIxERXSurOmAMN8HkSERE1zYed8AYboJlVSIiIgucORIRERSiAIVo532Oor1TT+fB5EhERLzmaIFlVSIiIgucORIRESAIDtgEgGVVIiJyJ0yOZlhWJSIissCZIxERQSEIUNg58bN3+zlnwuRIRETXyqoOGMNNsKxKRERkwSmSY05ODoKDg+Hl5YXo6GgcPny4x755eXm4++67MXz4cAwfPhyxsbE37E9ERFYQBMc0NyF7cty0aRP0ej0yMjJw5MgRhIWFIS4uDo2Njd32LykpwSOPPIK9e/eitLQUQUFBmDVrFs6ePdvPkRMRuREmRzOyJ8fs7GwsWLAASUlJCAkJQW5uLry9vZGfn99t/3feeQeLFy9GeHg4Jk2ahDfeeAOCIKC4uLifIyciIncl64Ico9GI8vJypKWlSceUSiViY2NRWlpq1Rg//PADLl++jBEjRnT7fkdHBzo6OqTXra2t9gVNROSORBNg78OK3WhvVVlnjk1NTTCZTNBoNGbHNRoNDAaDVWMsXboUAQEBiI2N7fb9rKws+Pr6Si0oKMjuuImI3M3VWznsb+5C9rKqPVauXImNGzdi69at8PLy6rZPWloaWlpapFZXV9fPURIRkauRtaw6cuRIeHh4oKGhwex4Q0MD/P39b3juyy+/jJUrV+Jf//oXQkNDe+ynVquhVqsdEi8RkdvifY5mZJ05qlQqREZGmi2m6Vxco9PpejzvpZdewvPPP4+ioiJERUX1R6hERO6Nq1XNyL5Djl6vR2JiIqKiojB16lSsWbMG7e3tSEpKAgAkJCQgMDAQWVlZAIC///3vSE9Px7vvvovg4GDp2uSQIUMwZMgQ2b4HERG5D9mT47x583Du3Dmkp6fDYDAgPDwcRUVF0iKd2tpaKJXXJ7hr166F0WjEr3/9a7NxMjIysGLFiv4MnYjIfQii/TM/e1e7OhHZkyMApKSkICUlpdv3SkpKzF6fPn267wMiIhpoBNEB1xzdJzm69GpVIiKivuAUM0ciIpKZQx527D4zRyZHIiJicrTAsioREZEFzhyJiIgLciwwORIR0dVNw0U7y6qi+yRHllWJiIgscOZIRERXZ332llXdaObI5EhERLzmaIFlVSIiIgucORIREWeOFpgciYjo6mJVO5Ojvec7E5ZViYiILHDmSERELKtaGLDJcdCgmzBo0GC5w/hJT0VUyx2CVRo+my93CFabtbhA7hCsFvrnX8odglUufRYqdwhW84lplDsEq4iiAKC9/z5QgAOSoyMCcQ4sqxIREVkYsDNHIiL6Ec4czTA5EhERIF5r9o7hJlhWJSIissCZIxERQRQUEO182LE73efI5EhERLzmaIFlVSIiIgucORIR0dUHHdtZVnWnBTlMjkRExGuOFlhWJSIissCZIxERXS2p2ltWdaOZI5MjERFdveYo8ppjJ5ZViYhINjk5OQgODoaXlxeio6Nx+PDhG/Zvbm5GcnIytFot1Go1JkyYgMLCQun9rKws3HHHHRg6dCj8/PzwwAMPoLKy0ua4mByJiEhakGNvs8WmTZug1+uRkZGBI0eOICwsDHFxcWhs7P7JKUajETNnzsTp06exefNmVFZWIi8vD4GBgVKfffv2ITk5GYcOHcLu3btx+fJlzJo1C+3ttj3hhGVVIiICBKUDrjnaVlfNzs7GggULkJSUBADIzc3Fjh07kJ+fj2XLlnXpn5+fj/Pnz+PgwYPw9PQEAAQHB5v1KSoqMntdUFAAPz8/lJeX45577rE6Ns4ciYjIoVpbW81aR0dHlz5GoxHl5eWIjY2VjimVSsTGxqK0tLTbcbdv3w6dTofk5GRoNBpMmTIFmZmZMJlMPcbS0tICABgxYoRN38EpkqOtNedOGzduhEKhwAMPPNC3ARIRubvO1ar2NgBBQUHw9fWVWlZWVpePa2pqgslkgkajMTuu0WhgMBi6DbGmpgabN2+GyWRCYWEhli9fjtWrV+OFF17o/isJAp588knExMRgypQpNv04ZC+rdtacc3NzER0djTVr1iAuLg6VlZXw8/Pr8bzTp0/jqaeewt13392P0RIRuSdRVEC0c7WqeK2qWldXBx8fH+m4Wq22a9xOgiDAz88P69atg4eHByIjI3H27FmsWrUKGRkZXfonJyfjyy+/xIEDB2z+LNlnjj+uOYeEhCA3Nxfe3t7Iz8/v8RyTyYT58+fj2WefxS233NKP0RIR0U/x8fExa90lx5EjR8LDwwMNDQ1mxxsaGuDv79/tuFqtFhMmTICHh4d0bPLkyTAYDDAajWZ9U1JS8PHHH2Pv3r0YPXq0zd9B1uTYm5ozADz33HPw8/PDH//4x5/8jI6Oji71byIisiAoHdOspFKpEBkZieLi4ushCAKKi4uh0+m6PScmJgbV1dUQhOu7DVRVVUGr1UKlUgEARFFESkoKtm7dij179mDcuHG9+nHImhx7U3M+cOAA3nzzTeTl5Vn1GVlZWWa176CgILvjJiJyN6LgiNs5bPtMvV6PvLw8bNiwAcePH8eiRYvQ3t4urV5NSEhAWlqa1H/RokU4f/48UlNTUVVVhR07diAzMxPJyclSn+TkZLz99tt49913MXToUBgMBhgMBly8eNGm2GS/5miLCxcu4Pe//z3y8vIwcuRIq85JS0uDXq+XXre2tjJBEhE5gXnz5uHcuXNIT0+HwWBAeHg4ioqKpAlTbW0tlMrrc7igoCDs2rULS5YsQWhoKAIDA5GamoqlS5dKfdauXQsAmD59utlnrV+/Hn/4wx+sjk3W5Ghrzfmbb77B6dOnER8fLx3rnF4PGjQIlZWV+I//+A+zc9RqtcMuBhMRuS2HPLLK9vNTUlKQkpLS7XslJSVdjul0Ohw6dKjnEETH7GEna1nV1przpEmTcOzYMVRUVEjtl7/8Je677z5UVFRwRkhE1Eudq1Xtbe5C9rKqXq9HYmIioqKiMHXqVKxZs6ZLzTkwMBBZWVnw8vLqcq/KsGHDAMDme1iIiIh6IntytLXmTEREfcDG1abdj+GYUJyB7MkRsL3m/GMFBQWOD4iIaIDpzcbh3Y3hLjglIyIisuAUM0ciIpKXY7aPc5+ZI5MjERHxmqMFllWJiIgscOZIRERckGOByZGIiHjN0QLLqkRERBY4cyQiIi7IscDkSEREvOZogWVVIiIiC5w5EhERF+RYYHIkIiJAdMA1R8c8StEpsKxKRERkgTNHIiLighwLTI5ERARRtP+aoehGZVUmRye3tXq83CFY5TYf1/kX43MrnpA7BKv9feK/5Q7BKovn3C13CFbbdLtr/Ex/MF3G/C/elzuMAYvJkYiIAAeUVcGyKhERuRNRVEIU7VujKbpRXZWrVYmIiCxw5khERFdLoiyrSpgciYiIO+RYYFmViIjIAmeORETETQAsMDkSERFXq1pgWZWIiMgCZ45ERMSyqgUmRyIi4mpVCyyrEhERWeDMkYiIOHO0wORIRERXk6O91xzdKDmyrEpERGSBM0ciIuJ9jhacYuaYk5OD4OBgeHl5ITo6GocPH75h/+bmZiQnJ0Or1UKtVmPChAkoLCzsp2iJiNxP560c9jZ3IfvMcdOmTdDr9cjNzUV0dDTWrFmDuLg4VFZWws/Pr0t/o9GImTNnws/PD5s3b0ZgYCDOnDmDYcOG9X/wRETklmRPjtnZ2ViwYAGSkpIAALm5udixYwfy8/OxbNmyLv3z8/Nx/vx5HDx4EJ6engCA4ODg/gyZiMjtcLWqOVnLqkajEeXl5YiNjZWOKZVKxMbGorS0tNtztm/fDp1Oh+TkZGg0GkyZMgWZmZkwmUzd9u/o6EBra6tZIyIic53J0d7mLmRNjk1NTTCZTNBoNGbHNRoNDAZDt+fU1NRg8+bNMJlMKCwsxPLly7F69Wq88MIL3fbPysqCr6+v1IKCghz+PYiIyL04xYIcWwiCAD8/P6xbtw6RkZGYN28enn76aeTm5nbbPy0tDS0tLVKrq6vr54iJiJyfKDhiUY7c38JxZL3mOHLkSHh4eKChocHseENDA/z9/bs9R6vVwtPTEx4eHtKxyZMnw2AwwGg0QqVSmfVXq9VQq9WOD56IyI3wmqM5WWeOKpUKkZGRKC4ulo4JgoDi4mLodLpuz4mJiUF1dTUE4fo/UaqqqqDVarskRiIiot6Qvayq1+uRl5eHDRs24Pjx41i0aBHa29ul1asJCQlIS0uT+i9atAjnz59HamoqqqqqsGPHDmRmZiI5OVmur0BE5PI6NwGwt7kL2W/lmDdvHs6dO4f09HQYDAaEh4ejqKhIWqRTW1sLpfL6DzwoKAi7du3CkiVLEBoaisDAQKSmpmLp0qVyfQUiIpcniAoIdpZF7T3fmcieHAEgJSUFKSkp3b5XUlLS5ZhOp8OhQ4f6OCoiIhqonCI5EhGRzByx/Ru3jyMiInfC1arm3OfqKRERkYNw5khERJw5WmByJCIiJkcLLKsSERFZ4MyRiIggiEoIdt7Eb+/5zoTJkYiIrpZV7bwVg2VVIiIiN8aZIxERcUGOBSZHIiJicrTAsioREckmJycHwcHB8PLyQnR0NA4fPnzD/s3NzUhOToZWq4VarcaECRNQWFgovb9//37Ex8cjICAACoUC27Zt61VcTI5ERCQ9lcPeZotNmzZBr9cjIyMDR44cQVhYGOLi4tDY2Nhtf6PRiJkzZ+L06dPYvHkzKisrkZeXh8DAQKlPe3s7wsLCkJOTY9fPg2VVIiKSpayanZ2NBQsWSM/vzc3NxY4dO5Cfn49ly5Z16Z+fn4/z58/j4MGD8PT0BAAEBweb9Zk9ezZmz57duy/wI5w5EhGRQ7W2tpq1jo6OLn2MRiPKy8sRGxsrHVMqlYiNjUVpaWm3427fvh06nQ7JycnQaDSYMmUKMjMzYTKZHP4dBuzMURCvQBCvyB3GT4oe9W+5Q7BKVMjXcodgtfRd98gdgtXiHt0mdwhW+dfLC+QOwWq//XK/3CFYRRSFfv48x80cg4KCzI5nZGRgxYoVZseamppgMpmkB9t30mg0OHHiRLfj19TUYM+ePZg/fz4KCwtRXV2NxYsX4/Lly8jIyLArdksDNjkSEdF1vblm2N0YAFBXVwcfHx/puFqttmtcaXxBgJ+fH9atWwcPDw9ERkbi7NmzWLVqFZMjERE5Nx8fH7Pk2J2RI0fCw8MDDQ0NZscbGhrg7+/f7TlarRaenp7w8PCQjk2ePBkGgwFGoxEqlcr+4K+x+Zrj5cuXMWPGDJw8edJhQRARkbxE8XpptffN+s9TqVSIjIxEcXGxdEwQBBQXF0On03V7TkxMDKqrqyEI10vOVVVV0Gq1Dk2MQC+So6enJ7744guHBkFERPKyPzHafs1Sr9cjLy8PGzZswPHjx7Fo0SK0t7dLq1cTEhKQlpYm9V+0aBHOnz+P1NRUVFVVYceOHcjMzERycrLUp62tDRUVFaioqAAAnDp1ChUVFaitrbUptl6VVX/3u9/hzTffxMqVK3tzOhEREebNm4dz584hPT0dBoMB4eHhKCoqkhbp1NbWQqm8PocLCgrCrl27sGTJEoSGhiIwMBCpqalYunSp1KesrAz33Xef9Fqv1wMAEhMTUVBQYHVsvUqOV65cQX5+Pv71r38hMjISN910k9n72dnZvRmWiIhkIjpgQU5vVrumpKQgJSWl2/dKSkq6HNPpdDh06FCP402fPh2iLfXdHvQqOX755Zf42c9+BuBqvffHFAr32VuPiGig4N6q5nqVHPfu3evoOIiIiJwGb+UgIiLOHC30OjmWlZXh/fffR21tLYxGo9l7W7ZssTswIiLqP47cBMAd9Gpv1Y0bN2LatGk4fvw4tm7disuXL+Orr77Cnj174Ovr6+gYiYiI+lWvkmNmZib+8Y9/4KOPPoJKpcIrr7yCEydO4OGHH8aYMWMcHSMREfUxOe5zdGa9So7ffPMN5syZA+DqLgft7e1QKBRYsmQJ1q1b59AAiYio78nxPEdn1qvkOHz4cFy4cAEAEBgYiC+//BLA1Sc0//DDD46LjoiISAa9WpBzzz33YPfu3bj99tvxm9/8BqmpqdizZw92796NGTNmODpGIiLqYyIUEGHnalU7z3cmvUqO//3f/41Lly4BAJ5++ml4enri4MGD+NWvfoVnnnnGoQESEVHf460c5npVVh0xYgQCAgKuDqBUYtmyZdi+fTtWr16N4cOH2zxeTk4OgoOD4eXlhejoaBw+fPiG/desWYOJEydi8ODBCAoKwpIlS6RkTUREZK9eJUfg6qKcZ555Bo888ggaGxsBADt37sRXX31l0zibNm2CXq9HRkYGjhw5grCwMMTFxUljWnr33XexbNkyZGRk4Pjx43jzzTexadMm/O1vf+vtVyEiGvC4IMdcr5Ljvn37cPvtt+Ozzz7Dli1b0NbWBgD4v//7P5ufxpydnY0FCxYgKSkJISEhyM3Nhbe3N/Lz87vtf/DgQcTExODRRx9FcHAwZs2ahUceeeQnZ5tERNQz3sphrlfJcdmyZXjhhRewe/duswdM/vznP7/hbumWjEYjysvLERsbez0gpRKxsbEoLS3t9pxp06ahvLxcSoY1NTUoLCzEL37xi277d3R0oLW11awRERHdSK8W5Bw7dgzvvvtul+N+fn5oamqyepympiaYTCbp2V2dNBoNTpw40e05jz76KJqamnDXXXdBFEVcuXIFCxcu7LGsmpWVhWeffdbqmIiIBiIBDtg+zo1Wq/Zq5jhs2DDU19d3OX706FEEBgbaHdSNlJSUIDMzE6+99hqOHDmCLVu2YMeOHXj++ee77Z+WloaWlhap1dXV9Wl8RESuiGVVc72aOf72t7/F0qVL8cEHH0ChUEAQBHz66ad46qmnkJCQYPU4I0eOhIeHBxoaGsyONzQ0wN/fv9tzli9fjt///vd4/PHHAQC333472tvb8cQTT+Dpp582e2o0AKjVaqjVahu/IRERDWS93lt10qRJCAoKQltbG0JCQnDPPfdg2rRpNt3nqFKpEBkZieLiYumYIAgoLi6GTqfr9pwffvihSwL08PAAAIc8/ZmIaCASoHBIcxe9mjmqVCrk5eVh+fLl+PLLL9HW1oaIiAjceuutNo+l1+uRmJiIqKgoTJ06FWvWrEF7ezuSkpIAAAkJCQgMDERWVhYAID4+HtnZ2YiIiEB0dDSqq6uxfPlyxMfHS0mSiIhs5Iiy6EAvq3YaM2aM3U/hmDdvHs6dO4f09HQYDAaEh4ejqKhIWqRTW1trNlN85plnoFAo8Mwzz+Ds2bMYNWoU4uPj8eKLL9oVBxERUadeJUeTyYSCggIUFxejsbERgiCYvb9nzx6bxktJSUFKSkq375WUlJi9HjRoEDIyMmy+n5KIiHrGhx2b61VyTE1NRUFBAebMmYMpU6ZAoXCfHwgR0UDEvVXN9So5bty4Ee+//36PN94TERG5sl4vyBk/fryjYyEiIpkI15q9Y7iLXt3K8Ze//AWvvPIKb50gInIT3ATAnNUzx4ceesjs9Z49e7Bz507cdttt8PT0NHtvy5YtjomOiIhIBlYnR19fX7PXDz74oMODISIieQii/atNBTcqJlqdHNevXy/9+eLFixAEATfddBMA4PTp09i2bRsmT56MuLg4x0dJRER9SoQCop073Nh7vjPp1TXHuXPn4n/+538AAM3NzbjzzjuxevVqPPDAA1i7dq1DAyQiIupvvUqOR44cwd133w0A2Lx5MzQaDc6cOYO33noLr776qkMDJCKivte5CYC9zV306laOH374AUOHDgUAfPLJJ3jooYegVCpx55134syZMw4NkIiI+t7Va472j+EuejVzHD9+PLZt24a6ujrs2rULs2bNAgA0NjbCx8fHoQESERH1t14lx/T0dDz11FMIDg5GdHS09HipTz75BBEREQ4NkIiI+l7nghx7m7voVVn117/+Ne666y7U19cjLCxMOj5jxgze4kFE5IK48bi5Xj+yyt/fH/7+/mbHpk6dandAREREcrPreY7U9+5/7Su5Q7CK17QquUOwmr+361wXf+m/H5M7BKs8N+9DuUOwWvs7rlHdMgod+J/zr/fb54ni1WbvGO6CyZGIiCBCAYGbAEh6tSCHiIjInXHmSEREfNixBSZHIiLialULLKsSERFZ4MyRiIggXmv2juEumByJiIhlVQssqxIREVngzJGIiCBca/aO4S6YHImIiLdyWGBZlYiIyAJnjkRExAU5FpgciYiIt3JYYFmViIjIAmeORETEsqoFJkciIuKtHBZkLavu378f8fHxCAgIgEKhwLZt237ynJKSEvzsZz+DWq3G+PHjUVBQ0OdxEhHRwCJrcmxvb0dYWBhycnKs6n/q1CnMmTMH9913HyoqKvDkk0/i8ccfx65du/o4UiIi99Z5n6O9zV3IWladPXs2Zs+ebXX/3NxcjBs3DqtXrwYATJ48GQcOHMA//vEPxMXF9VWYRERuT4T9ZVGuVpVJaWkpYmNjzY7FxcWhtLS0x3M6OjrQ2tpq1oiIiG7EpZKjwWCARqMxO6bRaNDa2oqLFy92e05WVhZ8fX2lFhQU1B+hEhG5FBEOKKvCfcqqLpUceyMtLQ0tLS1Sq6urkzskIiKnI4iOae7CpZKjv78/GhoazI41NDTAx8cHgwcP7vYctVoNHx8fs0ZERM4hJycHwcHB8PLyQnR0NA4fPnzD/s3NzUhOToZWq4VarcaECRNQWFho15jdcankqNPpUFxcbHZs9+7d0Ol0MkVEROQeRAc1W2zatAl6vR4ZGRk4cuQIwsLCEBcXh8bGxm77G41GzJw5E6dPn8bmzZtRWVmJvLw8BAYG9nrMnsiaHNva2lBRUYGKigoAV2/VqKioQG1tLYCrJdGEhASp/8KFC1FTU4P/+q//wokTJ/Daa6/h/fffx5IlS+QIn4jIbXTukGNvs0V2djYWLFiApKQkhISEIDc3F97e3sjPz++2f35+Ps6fP49t27YhJiYGwcHBuPfeexEWFtbrMXsia3IsKytDREQEIiIiAAB6vR4RERFIT08HANTX10uJEgDGjRuHHTt2YPfu3QgLC8Pq1avxxhtv8DYOIiInYnmHQEdHR5c+RqMR5eXlZncgKJVKxMbG9ngHwvbt26HT6ZCcnAyNRoMpU6YgMzMTJpOp12P2RNb7HKdPnw5R7Hki3t3uN9OnT8fRo0f7MCoiooHHkdvHWd4VkJGRgRUrVpgda2pqgslk6vYOhBMnTnQ7fk1NDfbs2YP58+ejsLAQ1dXVWLx4MS5fvoyMjIxejdkT7q1KREQO2eGm8/y6ujqzxY9qtdqucTsJggA/Pz+sW7cOHh4eiIyMxNmzZ7Fq1SpkZGQ45DM6MTkSEZFDWXNnwMiRI+Hh4dHtHQj+/v7dnqPVauHp6QkPDw/p2OTJk2EwGGA0Gns1Zk9carUqERH1DcFBzVoqlQqRkZFmdyAIgoDi4uIe70CIiYlBdXU1BOH6J1VVVUGr1UKlUvVqzJ4wORIREUTRMc0Wer0eeXl52LBhA44fP45Fixahvb0dSUlJAICEhASkpaVJ/RctWoTz588jNTUVVVVV2LFjBzIzM5GcnGz1mNZiWZWIiGQxb948nDt3Dunp6TAYDAgPD0dRUZG0oKa2thZK5fU5XFBQEHbt2oUlS5YgNDQUgYGBSE1NxdKlS60e01pMjkREBAEKCHbujdqb81NSUpCSktLteyUlJV2O6XQ6HDp0qNdjWovJkYiIHLI3KvdWJSIicmOcORIREdCLBTXdjeEumByJiEi2a47OimVVIiIiC5w5EhFRr+5T7G4Md8HkSEREDt143B2wrEpERGSBM0ciIuJ9jhYGbHIUxSsQxStyh/GTHp9zr9whWGX95Ei5Q7BawJALcodgtXv/sU/uEKyijvlW7hCsFu59Xu4QrGISjf36eSLsvxPDjXIjy6pERESWBuzMkYiIrrtaVrXzPkc3mjoyORIREW/lsMCyKhERkQXOHImIiPc5WmByJCIillUtsKxKRERkgTNHIiJiWdUCkyMREUF0wA45LKsSERG5Mc4ciYiI28dZYHIkIiJuPG6BZVUiIiILnDkSERHvc7TA5EhERLyVwwLLqkRERBZkTY779+9HfHw8AgICoFAosG3bthv237JlC2bOnIlRo0bBx8cHOp0Ou3bt6p9giYjcWOeCHHubu5A1Oba3tyMsLAw5OTlW9d+/fz9mzpyJwsJClJeX47777kN8fDyOHj3ax5ESEbk30UHNXch6zXH27NmYPXu21f3XrFlj9jozMxMffvghPvroI0RERDg4OiIiGqhcekGOIAi4cOECRowY0WOfjo4OdHR0SK9bW1v7IzQiIpfC+xzNufSCnJdffhltbW14+OGHe+yTlZUFX19fqQUFBfVjhERErqHzVg57m7tw2eT47rvv4tlnn8X7778PPz+/HvulpaWhpaVFanV1df0YJRERuSKXLKtu3LgRjz/+OD744APExsbesK9arYZare6nyIiIXBPvczTncsnxvffew2OPPYaNGzdizpw5codDROQWBDjgmqNDInEOsibHtrY2VFdXS69PnTqFiooKjBgxAmPGjEFaWhrOnj2Lt956C8DVUmpiYiJeeeUVREdHw2AwAAAGDx4MX19fWb4DERG5H1mvOZaVlSEiIkK6DUOv1yMiIgLp6ekAgPr6etTW1kr9161bhytXriA5ORlarVZqqampssRPROQueJ+jOVlnjtOnT4d4g+VNBQUFZq9LSkr6NiAiogFKFO0vi3K1KhERkRtzuQU5RETkeKJof1nUnWaOTI5ERMRbOSywrEpERGSBM0ciIrq6t6qdhVV32luVyZGIiBxyK4Yb5UaWVYmIiCxx5khERNfKqvaP4S6YHImICOK1/+wdw12wrEpERGSBM0ciImJZ1QKTIxERcRMACyyrEhERWeDMkYiIIIoOWJDjRpurMjkSERHLqhYGbHJUDfKBapC33GH8pNf+3/tyh2CVS02+codgNd27E+UOwWqfZN8kdwhWqflVlNwhWO2jL1zjZ3rJ5ImlNXJHMXAN2ORIRETXsaxqjsmRiIggwv6yqPukRq5WJSIi6oIzRyIigiCKDnhklfvMHTlzJCIiaW9Ve/+zVU5ODoKDg+Hl5YXo6GgcPny4x74FBQVQKBRmzcvLy6xPQ0MD/vCHPyAgIADe3t64//77cfLkSZvjYnIkIiJZbNq0CXq9HhkZGThy5AjCwsIQFxeHxsbGHs/x8fFBfX291M6cOSO9J4oiHnjgAdTU1ODDDz/E0aNHMXbsWMTGxqK9vd2m2JgciYhIus/R3maL7OxsLFiwAElJSQgJCUFubi68vb2Rn5/f4zkKhQL+/v5S02g00nsnT57EoUOHsHbtWtxxxx2YOHEi1q5di4sXL+K9996zKTYmRyIiggDRIc1aRqMR5eXliI2NlY4plUrExsaitLS0x/Pa2towduxYBAUFYe7cufjqq6+k9zo6OgDArNSqVCqhVqtx4MABW34cTI5ERORYra2tZq0zaf1YU1MTTCaT2cwPADQaDQwGQ7fjTpw4Efn5+fjwww/x9ttvQxAETJs2Dd9++y0AYNKkSRgzZgzS0tLw/fffw2g04u9//zu+/fZb1NfX2/QdmByJiOjqalUHNAAICgqCr6+v1LKyshwSo06nQ0JCAsLDw3Hvvfdiy5YtGDVqFF5//XUAgKenJ7Zs2YKqqiqMGDEC3t7e2Lt3L2bPng2l0rZ0x1s5iIio16tNLccAgLq6Ovj4+EjH1Wp1l74jR46Eh4cHGhoazI43NDTA39/fqs/z9PREREQEqqurpWORkZGoqKhAS0sLjEYjRo0ahejoaERF2bbFIWeORETkUD4+Pmatu+SoUqkQGRmJ4uJi6ZggCCguLoZOp7Pqc0wmE44dOwatVtvlPV9fX4waNQonT55EWVkZ5s6da9N34MyRiIhsXlDT0xi20Ov1SExMRFRUFKZOnYo1a9agvb0dSUlJAICEhAQEBgZKZdnnnnsOd955J8aPH4/m5masWrUKZ86cweOPPy6N+cEHH2DUqFEYM2YMjh07htTUVDzwwAOYNWuWTbExORIRkSzJcd68eTh37hzS09NhMBgQHh6OoqIiaZFObW2t2bXC77//HgsWLIDBYMDw4cMRGRmJgwcPIiQkROpTX18PvV6PhoYGaLVaJCQkYPny5TZ/F4XoTtuoW6G1tRW+vr4437wOPj7O/8iqi08/JXcIVuEjq/rGJ/c3/HQnsslHX4TLHYJVLpmMWFqzHi0tLWbX7xyt8+/EO7x+j0EKlV1jXRGN+PzS//R5zP2BM0ciInLoghx3IOuCnP379yM+Ph4BAQFQKBTYtm2b1ed++umnGDRoEMLDw/ssPiKigUJ0wAYATI4O0t7ejrCwMOTk5Nh0XnNzMxISEjBjxow+ioyIiAYyWcuqs2fPxuzZs20+b+HChXj00Ufh4eFh02yTiIi6JygEKBT2Pe5YsPtxyc7D5e5zXL9+PWpqapCRkWFV/46Oji5bGRERkbn+3lvV2blUcjx58iSWLVuGt99+G4MGWTfpzcrKMtvGKCgoqI+jJCIiV+cyydFkMuHRRx/Fs88+iwkTJlh9XlpaGlpaWqRWV1fXh1ESEbkmx8wb3aes6jK3cly4cAFlZWU4evQoUlJSAFzdakgURQwaNAiffPIJfv7zn3c5T61Wd7t1ERERXScAUNi9CYD7cJnk6OPjg2PHjpkde+2117Bnzx5s3rwZ48aNkykyIiJyN7Imx7a2NrPd1E+dOoWKigqMGDFCeibX2bNn8dZbb0GpVGLKlClm5/v5+cHLy6vLcSIisg1Xq5qTNTmWlZXhvvvuk17r9XoAQGJiIgoKClBfX4/a2lq5wiMiGjAECFDYmdyYHB1k+vTpuNHWrgUFBTc8f8WKFVixYoVjgyIiogHPZa45EhFR3+HM0RyTIxEROeRWDHe6lcNl7nMkIiLqL5w5EhERV6taYHIkIiKIEOxObiyrEhERuTHOHImICCJMEO2cL4kwOSga+TE5EhHRtZIqrzl2YlmViIjIAmeORER07UHF9s4c3edhx0yORER07Zqjwu4x3AXLqkRERBY4cyQiIi7IscDkSERE3FvVwoBNjj+0n8IgDy+5w/hJjcfHyR2CVbyHtMsdgtW++OsBuUOw2rAXBssdglX+5PeQ3CFYLWHK13KHYJW2K1eAGrmjGLgGbHIkIqLrBJgAOxfkCG60IIfJkYiIWFa1wNWqREREFjhzJCIiCKIDyqoiy6pERORGWFY1x7IqERGRBc4ciYjo2szRvrKoO80cmRyJiAiiKECwd29V0X2SI8uqREREFjhzJCKiayVRe5/K4T4zRyZHIiKC6IDbMBwxhrNgWZWIiMgCZ45ERISry3FYVu3E5EhERNdWmnK1aieWVYmIiCxw5khERHZvAOCoMZyFrDPH/fv3Iz4+HgEBAVAoFNi2bdtPntPR0YGnn34aY8eOhVqtRnBwMPLz8/s+WCIiNyaKIkRRsLOJcn8Nh5F15tje3o6wsDA89thjeOgh654k/vDDD6OhoQFvvvkmxo8fj/r6egiC+9S5iYhIfrImx9mzZ2P27NlW9y8qKsK+fftQU1ODESNGAACCg4P7KDoiooHDEStN3Wm1qkstyNm+fTuioqLw0ksvITAwEBMmTMBTTz2Fixcv9nhOR0cHWltbzRoREZkTRZNDmrtwqQU5NTU1OHDgALy8vLB161Y0NTVh8eLF+Pe//43169d3e05WVhaeffbZfo6UiIhcmUvNHAVBgEKhwDvvvIOpU6fiF7/4BbKzs7Fhw4YeZ49paWloaWmRWl1dXT9HTUTk/OxfjCO41X2OLjVz1Gq1CAwMhK+vr3Rs8uTJEEUR3377LW699dYu56jVaqjV6v4Mk4jI5fCaozmXmjnGxMTgu+++Q1tbm3SsqqoKSqUSo0ePljEyIiJyJ7Imx7a2NlRUVKCiogIAcOrUKVRUVKC2thbA1ZJoQkKC1P/RRx/FzTffjKSkJHz99dfYv38//vrXv+Kxxx7D4MGD5fgKRERugWVVc7Imx7KyMkRERCAiIgIAoNfrERERgfT0dABAfX29lCgBYMiQIdi9ezeam5sRFRWF+fPnIz4+Hq+++qos8RMRuQsRgkOau5D1muP06dNvuKNCQUFBl2OTJk3C7t27+zAqIiIa6FxqQQ4REfWNq/co2rf9mzuVVZkciYgIVxOjvcnNffZWdanVqkRERP2BM0ciInLQw445cyQiIjci12rVnJwcBAcHw8vLC9HR0Th8+HCPfQsKCqBQKMyal5eXWZ+2tjakpKRg9OjRGDx4MEJCQpCbm2tzXJw5EhGRLDZt2gS9Xo/c3FxER0djzZo1iIuLQ2VlJfz8/Lo9x8fHB5WVldJrhcJ8tqvX67Fnzx68/fbbCA4OxieffILFixcjICAAv/zlL62OjTNHIiLC1cU4jmjWy87OxoIFC5CUlCTN8Ly9vW/4AHuFQgF/f3+paTQas/cPHjyIxMRETJ8+HcHBwXjiiScQFhZ2wxlpd5gciYgIEAXHNCsZjUaUl5cjNjZWOqZUKhEbG4vS0tIez2tra8PYsWMRFBSEuXPn4quvvjJ7f9q0adi+fTvOnj0LURSxd+9eVFVVYdasWTb9OJgciYjIoSyfodvR0dGlT1NTE0wmU5eZn0ajgcFg6HbciRMnIj8/Hx9++CHefvttCIKAadOm4dtvv5X6/POf/0RISAhGjx4NlUqF+++/Hzk5Objnnnts+g685khERNcW09i5WvXafY5BQUFmxzMyMrBixQq7xgYAnU4HnU4nvZ42bRomT56M119/Hc8//zyAq8nx0KFD2L59O8aOHYv9+/cjOTkZAQEBZrPUn8LkSEREgAOSY+cmAHV1dfDx8ZGOdvfYwJEjR8LDwwMNDQ1mxxsaGuDv72/Vp3l6eiIiIgLV1dUAgIsXL+Jvf/sbtm7dijlz5gAAQkNDUVFRgZdfftmm5MiyKhEROZSPj49Z6y45qlQqREZGori4WDomCAKKi4vNZoc3YjKZcOzYMWi1WgDA5cuXcfnyZSiV5qnNw8MDgmDbYiHOHImICIDogN3fbBtAr9cjMTERUVFRmDp1KtasWYP29nYkJSUBABISEhAYGIisrCwAwHPPPYc777wT48ePR3NzM1atWoUzZ87g8ccfB3A1Kd97773461//isGDB2Ps2LHYt28f3nrrLWRnZ9sUG5MjERHh+hVD+8awxbx583Du3Dmkp6fDYDAgPDwcRUVF0iKd2tpas1ng999/jwULFsBgMGD48OGIjIzEwYMHERISIvXZuHEj0tLSMH/+fJw/fx5jx47Fiy++iIULF9oUm0J0p/1+rNDS0oJhw4bh+Ak9hg7tOtV3Nuee+JfcIVjF+6Yf5A7Bar63fCd3CFYLWuX1052cwP8bZf3N1XJ7JOSE3CFYpf3KFcza/ymam5vh6+vbZ5/T2tp6bXwPB41oQktLi9k1R1c04JLjt99+22UlFRGRs6qrq8Po0aP7bPxLly5h3LhxPd4+YSt/f3+cOnWqy7ZurmbAJUdBEPDdd99h6NChXbYdskdrayuCgoK6rNJyNq4SJ8BY+4KrxAkwVlEUceHCBQQEBHRZYOJoly5dgtFodMhYKpXK5RMjMACvOSqVyj79V1jn6ixn5ypxAoy1L7hKnMDAjrUvy6k/5uXl5RYJzZF4KwcREZEFJkciIiILTI4OolarkZGR0e3Nrs7EVeIEGGtfcJU4AcZK8hpwC3KIiIh+CmeOREREFpgciYiILDA5EhERWWByJCIissDk6AA5OTkIDg6Gl5cXoqOjcfjwYblDwv79+xEfH4+AgAAoFAps27bN7H1RFJGeng6tVovBgwcjNjYWJ0+e7Pc4s7KycMcdd2Do0KHw8/PDAw88gMrKSrM+ly5dQnJyMm6++WYMGTIEv/rVr7o8A64/rF27FqGhodKN3jqdDjt37nS6OC2tXLkSCoUCTz75pHTMWWJdsWIFFAqFWZs0aZLTxdnp7Nmz+N3vfoebb74ZgwcPxu23346ysjLpfWf5vSL7MTnaadOmTdDr9cjIyMCRI0cQFhaGuLg4NDY2yhpXe3s7wsLCkJOT0+37L730El599VXk5ubis88+w0033YS4uDhcunSpX+Pct28fkpOTcejQIezevRuXL1/GrFmz0N7eLvVZsmQJPvroI3zwwQfYt28fvvvuOzz00EP9GicAjB49GitXrkR5eTnKysrw85//HHPnzsVXX33lVHH+2Oeff47XX38doaGhZsedKdbbbrsN9fX1Ujtw4IBTxvn9998jJiYGnp6e2LlzJ77++musXr0aw4cPl/o4y+8VOYBIdpk6daqYnJwsvTaZTGJAQICYlZUlY1TmAIhbt26VXguCIPr7+4urVq2SjjU3N4tqtVp87733ZIjwusbGRhGAuG/fPikuT09P8YMPPpD6HD9+XAQglpaWyhWmZPjw4eIbb7zhlHFeuHBBvPXWW8Xdu3eL9957r5iamiqKonP9TDMyMsSwsLBu33OmOEVRFJcuXSreddddPb7vzL9XZDvOHO1gNBpRXl6O2NhY6ZhSqURsbCxKS0tljOzGTp06BYPBYBa3r68voqOjZY+7paUFADBixAgAQHl5OS5fvmwW66RJkzBmzBhZYzWZTNi4cSPa29uh0+mcMs7k5GTMmTPHLCbA+X6mJ0+eREBAAG655RbMnz8ftbW1Thnn9u3bERUVhd/85jfw8/NDREQE8vLypPed+feKbMfkaIempiaYTCbpwZydNBqNwx7/0hc6Y3O2uAVBwJNPPomYmBhMmTIFwNVYVSoVhg0bZtZXrliPHTuGIUOGQK1WY+HChdi6dStCQkKcLs6NGzfiyJEj0hPUf8yZYo2OjkZBQQGKioqwdu1anDp1CnfffTcuXLjgVHECQE1NDdauXYtbb70Vu3btwqJFi/DnP/8ZGzZsAOC8v1fUOwPuqRzkvJKTk/Hll1+aXXNyNhMnTkRFRQVaWlqwefNmJCYmYt++fXKHZaaurg6pqanYvXu30z9pYfbs2dKfQ0NDER0djbFjx+L999/H4MGDZYysK0EQEBUVhczMTABAREQEvvzyS+Tm5iIxMVHm6MjROHO0w8iRI+Hh4dFl9VxDQwP8/f1liuqndcbmTHGnpKTg448/xt69e80eKebv7w+j0Yjm5maz/nLFqlKpMH78eERGRiIrKwthYWF45ZVXnCrO8vJyNDY24mc/+xkGDRqEQYMGYd++fXj11VcxaNAgaDQap4nV0rBhwzBhwgRUV1c71c8UALRaLUJCQsyOTZ48WSoDO+PvFfUek6MdVCoVIiMjUVxcLB0TBAHFxcXQ6XQyRnZj48aNg7+/v1ncra2t+Oyzz/o9blEUkZKSgq1bt2LPnj0YN26c2fuRkZHw9PQ0i7WyshK1tbVO8TMWBAEdHR1OFeeMGTNw7NgxVFRUSC0qKgrz58+X/uwssVpqa2vDN998A61W61Q/UwCIiYnpcptRVVUVxo4dC8C5fq/IAeReEeTqNm7cKKrVarGgoED8+uuvxSeeeEIcNmyYaDAYZI3rwoUL4tGjR8WjR4+KAMTs7Gzx6NGj4pkzZ0RRFMWVK1eKw4YNEz/88EPxiy++EOfOnSuOGzdOvHjxYr/GuWjRItHX11csKSkR6+vrpfbDDz9IfRYuXCiOGTNG3LNnj1hWVibqdDpRp9P1a5yiKIrLli0T9+3bJ546dUr84osvxGXLlokKhUL85JNPnCrO7vx4taooOk+sf/nLX8SSkhLx1KlT4qeffirGxsaKI0eOFBsbG50qTlEUxcOHD4uDBg0SX3zxRfHkyZPiO++8I3p7e4tvv/221MdZfq/IfkyODvDPf/5THDNmjKhSqcSpU6eKhw4dkjskce/evSKALi0xMVEUxavLzpcvXy5qNBpRrVaLM2bMECsrK/s9zu5iBCCuX79e6nPx4kVx8eLF4vDhw0Vvb2/xwQcfFOvr6/s91scee0wcO3asqFKpxFGjRokzZsyQEqMzxdkdy+ToLLHOmzdP1Gq1okqlEgMDA8V58+aJ1dXVThdnp48++kicMmWKqFarxUmTJonr1q0ze99Zfq/IfnxkFRERkQVecyQiIrLA5EhERGSByZGIiMgCkyMREZEFJkciIiILTI5EREQWmByJiIgsMDkS9YMVK1YgPDxc7jCIyEpMjkRERBaYHImIiCwwOdKAV1RUhLvuugvDhg3DzTffjP/8z//EN998AwA4ffo0FAoFtmzZgvvuuw/e3t4ICwvr8mT3vLw8BAUFwdvbGw8++CCys7O7PKTX0htvvIHJkyfDy8sLkyZNwmuvvdZXX5GIbMTkSANee3s79Ho9ysrKUFxcDKVSiQcffBCCIEh9nn76aTz11FOoqKjAhAkT8Mgjj+DKlSsAgE8//RQLFy5EamoqKioqMHPmTLz44os3/Mx33nkH6enpePHFF3H8+HFkZmZi+fLl0lPliUhe3HicyEJTUxNGjRqFY8eOYciQIRg3bhzeeOMN/PGPfwQAfP3117jttttw/PhxTJo0Cb/97W/R1taGjz/+WBrjd7/7HT7++GPpQb0rVqzAtm3bUFFRAQAYP348nn/+eTzyyCPSOS+88AIKCwtx8ODBfvuuRNQ9zhxpwDt58iQeeeQR3HLLLfDx8UFwcDAASE94B4DQ0FDpz1qtFgDQ2NgI4OoDeKdOnWo2puXrH2tvb8c333yDP/7xjxgyZIjUXnjhBamcS0TyGiR3AERyi4+Px9ixY5GXl4eAgAAIgoApU6bAaDRKfTw9PaU/KxQKADAru9qira0NwNXrlNHR0WbveXh49GpMInIsJkca0P7973+jsrISeXl5uPvuuwEABw4csGmMiRMn4vPPPzc7Zvn6xzQaDQICAlBTU4P58+fbHjQR9TkmRxrQhg8fjptvvhnr1q2DVqtFbW0tli1bZtMYf/rTn3DPPfcgOzsb8fHx2LNnD3bu3CnNMLvz7LPP4s9//jN8fX1x//33o6OjA2VlZfj++++h1+vt/VpEZCdec6QBTalUYuPGjSgvL8eUKVOwZMkSrFq1yqYxYmJikJubi+zsbISFhaGoqAhLliyBl5dXj+c8/vjjeOONN7B+/XrcfvvtuPfee1FQUIBx48bZ+5WIyAG4WpWoDyxYsAAnTpzA//7v/8odChH1AsuqRA7w8ssvY+bMmbjpppuwc+dObNiwgTf1E7kwzhyJHODhhx9GSUkJLly4gFtuuQV/+tOfsHDhQrnDIqJeYnIkIiKywAU5REREFpgciYiILDA5EhERWWByJCIissDkSEREZIHJkYiIyAKTIxERkQUmRyIiIgtMjkRERBb+P6LkZzjHE1teAAAAAElFTkSuQmCC\n"
          },
          "metadata": {}
        }
      ]
    },
    {
      "cell_type": "markdown",
      "metadata": {
        "id": "OHcZWJiFJDMh"
      },
      "source": [
        "It seems that rotation and shear don't mix! The best accuracy is when rotation is zero."
      ]
    },
    {
      "cell_type": "markdown",
      "metadata": {
        "id": "PAasQo1C3x4A"
      },
      "source": [
        "## Questions\n",
        "Try these questions for better understanding. You may not be able to solve all of them.\n",
        "1. What is the best value for angle constraint and shear constraint you got? How much did the accuracy improve as compared to not using augmentations?\n",
        "2. Can you increase the accuracy by increasing the number of augmentations from each sample?\n",
        "3. Try implementing a few augmentations of your own and experimenting with them. A good reference is <a href=https://www.analyticsvidhya.com/blog/2019/12/image-augmentation-deep-learning-pytorch/>here. </a>\n",
        "4. Try combining various augmentations. What is the highest accuracy you can get? What is the smallest training dataset you can take and still get accuracy above 50%?\n",
        "\n",
        "Whenever you do any experiment, a good practice is to vary the hyperparameters gradually and create a graph of your results, like we did for gridsearch."
      ]
    },
    {
      "cell_type": "markdown",
      "source": [
        "Best Value for Angle and Shear Constraints:\n",
        "\n",
        "The best values for angle and shear constraints in data augmentation depend on your dataset and the task you are working on. There is no one-size-fits-all answer.\n",
        "You can find the optimal values through experimentation. Start with a range of values and use cross-validation to evaluate which values result in the highest accuracy improvement. Compare this to not using augmentations to measure the improvement.\n",
        "Increasing Accuracy by Increasing Augmentations:\n",
        "\n",
        "Increasing the number of augmentations per sample can improve accuracy up to a certain point. Beyond that point, adding more augmentations may lead to overfitting.\n",
        "You can experiment by gradually increasing the number of augmentations and monitoring the validation accuracy. Once you notice diminishing returns or overfitting, you've likely reached the optimal number.\n",
        "Implementing Custom Augmentations:\n",
        "\n",
        "Custom augmentations can be very effective depending on your data and task. Some common custom augmentations include random cropping, color jittering, and cutout.\n",
        "Implement these augmentations using libraries like OpenCV or Pillow, and then evaluate their impact on your model's accuracy through experimentation.\n",
        "Combining Various Augmentations:\n",
        "\n",
        "Combining multiple augmentations can result in higher accuracy, but there's a trade-off between complexity and effectiveness.\n",
        "You can create augmentation pipelines by stacking different augmentations (e.g., rotation, scaling, and cropping) and evaluate their impact on accuracy.\n",
        "The highest accuracy will depend on your dataset, model architecture, and the specific combinations of augmentations you use.\n",
        "The smallest training dataset that still achieves accuracy above 50% will vary based on your task and the complexity of the model. Regularization techniques (e.g., dropout) can help mitigate the need for larger datasets."
      ],
      "metadata": {
        "id": "XQ7fwc5-IMuc"
      }
    }
  ]
}