BiT-L(ResNet).ipynb

{
 "cells": [
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "# <a href='https://github.com/google-research/big_transfer'> BiT-L </a>:\n",
    "Means Big Transform Large : aims to simplify & create a pre-trained model, or transfer pipeline which uses a minimal no. of necessary tricks to attain very strong performances across a broad spectrum of popular classification tasks.\n",
    "It's <a href='https://paperswithcode.com/paper/large-scale-learning-of-general-visual'> the state of art</a> of multiple datasets such as Cifar-10, Cifar-100, Imagenet ...\n",
    "\n",
    "https://www.youtube.com/watch?v=lK5rm2_OPGo&t=1218s\n",
    "\n",
    "\n",
    "https://medium.com/analytics-vidhya/enhance-learning-by-transfering-resnet-architecture-into-big-transform-architecture-44603f537fcf\n",
    "\n"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "## Load Data "
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 1,
   "metadata": {},
   "outputs": [],
   "source": [
    "from data import Images_load"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 2,
   "metadata": {},
   "outputs": [
    {
     "name": "stdout",
     "output_type": "stream",
     "text": [
      "Data has apparently already been downloaded and unpacked.\n"
     ]
    },
    {
     "name": "stderr",
     "output_type": "stream",
     "text": [
      "  2%|▏         | 61/4006 [00:00<00:06, 609.91it/s]"
     ]
    },
    {
     "name": "stdout",
     "output_type": "stream",
     "text": [
      "Data has apparently already been downloaded and unpacked.\n"
     ]
    },
    {
     "name": "stderr",
     "output_type": "stream",
     "text": [
      "100%|██████████| 4006/4006 [00:06<00:00, 586.48it/s]\n",
      "100%|██████████| 4192/4192 [00:07<00:00, 587.69it/s]\n"
     ]
    }
   ],
   "source": [
    "train, validation, test = Images_load.load_data()"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "## Train a BiT-L ResNet 50x1"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 3,
   "metadata": {},
   "outputs": [
    {
     "name": "stderr",
     "output_type": "stream",
     "text": [
      "Using TensorFlow backend.\n"
     ]
    }
   ],
   "source": [
    "from models.Image_classifier import BiT_L_ResNet"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 4,
   "metadata": {},
   "outputs": [
    {
     "name": "stdout",
     "output_type": "stream",
     "text": [
      "WARNING:tensorflow:From /opt/conda/lib/python3.7/site-packages/tensorflow_core/python/ops/resource_variable_ops.py:1781: calling BaseResourceVariable.__init__ (from tensorflow.python.ops.resource_variable_ops) with constraint is deprecated and will be removed in a future version.\n",
      "Instructions for updating:\n",
      "If using Keras pass *_constraint arguments to layers.\n"
     ]
    },
    {
     "name": "stderr",
     "output_type": "stream",
     "text": [
      "WARNING:tensorflow:From /opt/conda/lib/python3.7/site-packages/tensorflow_core/python/ops/resource_variable_ops.py:1781: calling BaseResourceVariable.__init__ (from tensorflow.python.ops.resource_variable_ops) with constraint is deprecated and will be removed in a future version.\n",
      "Instructions for updating:\n",
      "If using Keras pass *_constraint arguments to layers.\n"
     ]
    },
    {
     "name": "stdout",
     "output_type": "stream",
     "text": [
      "Model sucessfuly built\n",
      "callbacks and data okay\n",
      "Train on 10 steps, validate on 4 steps\n",
      "WARNING:tensorflow:From /opt/conda/lib/python3.7/site-packages/keras/callbacks/tensorboard_v1.py:200: The name tf.summary.merge_all is deprecated. Please use tf.compat.v1.summary.merge_all instead.\n",
      "\n"
     ]
    },
    {
     "name": "stderr",
     "output_type": "stream",
     "text": [
      "WARNING:tensorflow:From /opt/conda/lib/python3.7/site-packages/keras/callbacks/tensorboard_v1.py:200: The name tf.summary.merge_all is deprecated. Please use tf.compat.v1.summary.merge_all instead.\n",
      "\n"
     ]
    },
    {
     "name": "stdout",
     "output_type": "stream",
     "text": [
      "WARNING:tensorflow:From /opt/conda/lib/python3.7/site-packages/keras/callbacks/tensorboard_v1.py:203: The name tf.summary.FileWriter is deprecated. Please use tf.compat.v1.summary.FileWriter instead.\n",
      "\n"
     ]
    },
    {
     "name": "stderr",
     "output_type": "stream",
     "text": [
      "WARNING:tensorflow:From /opt/conda/lib/python3.7/site-packages/keras/callbacks/tensorboard_v1.py:203: The name tf.summary.FileWriter is deprecated. Please use tf.compat.v1.summary.FileWriter instead.\n",
      "\n"
     ]
    },
    {
     "name": "stdout",
     "output_type": "stream",
     "text": [
      "WARNING:tensorflow:Issue encountered when serializing variables.\n",
      "Type is unsupported, or the types of the items don't match field type in CollectionDef. Note this is a warning and probably safe to ignore.\n",
      "'list' object has no attribute 'name'\n"
     ]
    },
    {
     "name": "stderr",
     "output_type": "stream",
     "text": [
      "WARNING:tensorflow:Issue encountered when serializing variables.\n",
      "Type is unsupported, or the types of the items don't match field type in CollectionDef. Note this is a warning and probably safe to ignore.\n",
      "'list' object has no attribute 'name'\n"
     ]
    },
    {
     "name": "stdout",
     "output_type": "stream",
     "text": [
      "Epoch 1/10\n",
      " 9/10 [==========================>...] - ETA: 18s - loss: 1.6471 - acc: 0.6593WARNING:tensorflow:From /opt/conda/lib/python3.7/site-packages/keras/callbacks/tensorboard_v1.py:343: The name tf.Summary is deprecated. Please use tf.compat.v1.Summary instead.\n",
      "\n"
     ]
    },
    {
     "name": "stderr",
     "output_type": "stream",
     "text": [
      "WARNING:tensorflow:From /opt/conda/lib/python3.7/site-packages/keras/callbacks/tensorboard_v1.py:343: The name tf.Summary is deprecated. Please use tf.compat.v1.Summary instead.\n",
      "\n"
     ]
    },
    {
     "name": "stdout",
     "output_type": "stream",
     "text": [
      "10/10 [==============================] - 262s 26s/step - loss: 1.6352 - acc: 0.6709 - val_loss: 0.7856 - val_acc: 0.8296\n",
      "Epoch 2/10\n",
      "10/10 [==============================] - 223s 22s/step - loss: 1.3669 - acc: 0.7711 - val_loss: 0.6076 - val_acc: 0.7867\n",
      "Epoch 3/10\n",
      "10/10 [==============================] - 223s 22s/step - loss: 1.0590 - acc: 0.7660 - val_loss: 0.8387 - val_acc: 0.7477\n",
      "Epoch 4/10\n",
      "10/10 [==============================] - 223s 22s/step - loss: 0.6530 - acc: 0.8145 - val_loss: 0.4639 - val_acc: 0.8004\n",
      "Epoch 5/10\n",
      "10/10 [==============================] - 225s 22s/step - loss: 0.4437 - acc: 0.8369 - val_loss: 0.4778 - val_acc: 0.7887\n",
      "Epoch 6/10\n",
      "10/10 [==============================] - 224s 22s/step - loss: 0.4354 - acc: 0.8320 - val_loss: 0.5371 - val_acc: 0.7822\n",
      "Epoch 7/10\n",
      "10/10 [==============================] - 224s 22s/step - loss: 0.5133 - acc: 0.8043 - val_loss: 0.4019 - val_acc: 0.8283\n",
      "Epoch 8/10\n",
      "10/10 [==============================] - 224s 22s/step - loss: 0.3897 - acc: 0.8494 - val_loss: 0.5294 - val_acc: 0.7529\n",
      "Epoch 9/10\n",
      "10/10 [==============================] - 224s 22s/step - loss: 0.5063 - acc: 0.8047 - val_loss: 0.5368 - val_acc: 0.8062\n",
      "Epoch 10/10\n",
      "10/10 [==============================] - 222s 22s/step - loss: 0.5450 - acc: 0.8266 - val_loss: 0.4535 - val_acc: 0.8114\n",
      "Training model is finished\n",
      "5/5 [==============================] - 102s 20s/step - loss: 0.4722 - acc: 0.8117\n",
      "Test accuracy: 0.8117073\n",
      "CPU times: user 3h 23min 13s, sys: 44min 31s, total: 4h 7min 45s\n",
      "Wall time: 41min 30s\n"
     ]
    }
   ],
   "source": [
    "%%time \n",
    "history,model = BiT_L_ResNet.train_Bit_L(train,validation,test)"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 5,
   "metadata": {},
   "outputs": [
    {
     "data": {
      "image/png": "iVBORw0KGgoAAAANSUhEUgAAAYIAAAEWCAYAAABrDZDcAAAABHNCSVQICAgIfAhkiAAAAAlwSFlzAAALEgAACxIB0t1+/AAAADh0RVh0U29mdHdhcmUAbWF0cGxvdGxpYiB2ZXJzaW9uMy4yLjEsIGh0dHA6Ly9tYXRwbG90bGliLm9yZy+j8jraAAAgAElEQVR4nO3dd3gU1frA8e+bTkICAUJLgIQWIBBICIhIFaQooCAq1osN67Veu17s158i9oZdQRQRVBRpAgIKSEIJJfQaaqhJIIGU8/tjFgwhjWQ3k2Tfz/Pkye7OzJl3JzDvzDlnzhFjDEoppdyXh90BKKWUspcmAqWUcnOaCJRSys1pIlBKKTeniUAppdycJgKllHJzmgiU04jIFyLyYgnX3S4ifV0Yy/UiMstV5buSiDwrIuMdrxuLSLqIeBa3bin3tVZEepV2+yLKnS8itzm7XOUaXnYHoFR+IvIFkGyMebq0ZRhjJgATnBaUTYwxO4HqziiroONqjIlyRtmqctM7AlXpiIhewCjlRJoI3IyjSuYREUkUkeMi8qmI1BOR30QkTUTmiEhwnvWHOKoPjjpu91vnWRYjIssd230H+OXb1yARWenY9i8RiS5BfKOA64FHHVUi0/LE/ZiIJALHRcRLRB4XkS2O/a8TkaF5yhkpIovyvDcicqeIbBKRIyLynohIAftvKCIZIlIr3/c8KCLeItJcRP4QkWOOz74r5HvMEJF78322SkSGOV6/JSK7RCRVRBJEpHsh5YQ7YvdyvI9w7D9NRGYDdfKt/72I7HPEt0BEokpwXPs6XvuKyJsissfx86aI+DqW9RKRZBF5WEQOiMheEbm54L/iOd/BQ0SeFpEdjm2/EpEajmV+IjJeRA45/p0sE5F6jmUjRWSr47tuE5HrS7I/VQrGGP1xox9gO7AEqAeEAgeA5UAM4AvMBUY71m0JHAcuAbyBR4HNgI/jZwfwoGPZcCALeNGxbayj7AsAT+Bfjn375omjbyExfnG6nHxxrwQaAdUcn10FNMS6oLnGEWsDx7KRwKI82xvgF6Am0BhIAQYUsv+5wO153r8GfOh4PRF4yrFPP6BbIWXcBPyZ530b4Gie738DUBurevZhYB/g51j2LDDe8TrcEbuX4/1iYKzjb9UDSDu9rmP5LUCgY/mbwMoSHNe+jtfPO/5t1AVCgL+AFxzLegHZjnW8gUuBE0BwId9/PnBbnpg2A02xqrmmAF87lt0BTAP8Hf9OOgJBQACQCkQ61msARNn9/6eq/ugdgXt6xxiz3xizG1gILDXGrDDGnASmYiUFsE6uvxpjZhtjsoAxQDWgK9AF64TwpjEmyxgzGViWZx+3Ax8ZY5YaY3KMMV8CJx3bldbbxphdxpgMAGPM98aYPcaYXGPMd8AmoHMR279ijDlqrHr3eUCHQtb7BrgWwHHXMMLxGVjJrgnQ0BiTaYxZVHARTAU6iEgTx/vrgSmOY4wxZrwx5pAxJtsY8zrWiTuyqC8vIo2BTsAzxpiTxpgFWCfRM4wxnxlj0hz7eRZof/rquwSuB543xhwwxqQAzwE35lme5VieZYyZDqQXF3OecscaY7YaY9KBJ4ARjrucLKyE2Nzx7yTBGJPq2C4XaCsi1Ywxe40xa0v4PdR50kTgnvbneZ1RwPvTjZMNsa76ATDG5AK7sO4kGgK7jTF5Ry3cked1E+Bhx+3+URE5inU137AMce/K+0ZEbspT9XQUaEu+qpJ89uV5fYLCG2EnAxeKSEOsq26DlTDBuisS4G9HldktBRVgjEkDfsVKIjh+n2m8dlSxJDmqcI4CNYqJHaxjd8QYczzPZ2eOuYh4isgrjuqyVKyrfUpQbt7y8/4Nd3D23+uQMSY7z/uijmFx5Xph3ZV+DcwEvnVUR70qIt6O73gNcCewV0R+FZFWJfwe6jxpIlBF2YN1QgfOXB03AnYDe4HQfPXsjfO83gW8ZIypmefH3xgzsQT7LWxI3DOfO660PwbuBWobY2oCa7BO0mVijDkKzAKuBq4DJp5OeMaYfcaY240xDbGqNd4XkeaFFDURuFZELsS6k5rniL078Jij/GBH7MdKEPteIFhEAvJ8lveYXwdcDvTFSizhjs9Pl1vcUMNn/b0dZe8pZpuSKKjcbGC/4+7iOWNMG6w7zUFY1WoYY2YaYy7BqhZaj/X3Vi6giUAVZRJwmYj0ERFvrLrsk1h1x4ux/jPf52i4HcbZ1TIfA3eKyAViCRCRy0QksAT73Y9Vn1yUAKwTWwqAo+Gy7fl8uWJ8g3VCupJ/qoUQkatEJMzx9ogjhpxCypiOdQJ8HvjOcUcFVh1+tiN2LxH5L1a9eJGMMTuAeOA5EfERkW7A4DyrBGL9fQ5h1bm/nK+I4o7rROBpEQkRkTrAf4FSP6OQr9wHHQ3d1R1xfWeMyRaR3iLSTqznJFKxqopyxOrAMMSR9E5iVUMVdpxVGWkiUIUyxmzAatR8BziIddIZbIw5ZYw5BQzDapQ9gnUbPyXPtvFY7QTvOpZvdqxbEp8CbRxVPj8WEts64HWshLQfaAf8eX7fsEg/Ay2wrlpX5fm8E7BURNId69xvjNlWSIwnsY5JX/IkE6yqkN+AjVjVJJnkq/YqwnVYDfCHgdHAV3mWfeUobzewDqvhN6/ijuuLWIkmEViN1YmgRA8IFuMzrCqgBcA2rO/7b8ey+lhVcalAEvAHVvLxwLrw2IP1XXsCdzshFlUAObuKVymllLvROwKllHJzmgiUUsrNaSJQSik3p4lAKaXcXKUbvKtOnTomPDzc7jCUUqpSSUhIOGiMCSloWaVLBOHh4cTHx9sdhlJKVSoisqOwZVo1pJRSbk4TgVJKuTlNBEop5eYqXRuBUqpqycrKIjk5mczMTLtDqRL8/PwICwvD29u7xNtoIlBK2So5OZnAwEDCw8ORcyeNU+fBGMOhQ4dITk4mIiKixNtp1ZBSylaZmZnUrl1bk4ATiAi1a9c+77srTQRKKdtpEnCe0hxLt0kEB9IyeW7aWk5l5xa/slJKuRG3SQQJ24/w+Z/befHXdXaHopSqQI4ePcr7779/3ttdeumlHD161AURlT+3SQQD2zVgVI+mfLV4B9/Hl3QOEKVUVVdYIsjJKXpCtOnTp1OzZk1XhVWu3CYRADzaP5KLmtfmqR/XkJhcNTK5UqpsHn/8cbZs2UKHDh3o1KkTvXv35rrrrqNdu3YAXHHFFXTs2JGoqCjGjRt3Zrvw8HAOHjzI9u3bad26NbfffjtRUVH069ePjIwMu75OqbhshjIR+QxrIuoDxpgC55IVkV7Am4A3cNAY07O4cuPi4kxZxho6fPwUg99ZhDGGaf/uRu3qvqUuSylVdklJSbRu3RqA56atZd2eVKeW36ZhEKMHRxW6fPv27QwaNIg1a9Ywf/58LrvsMtasWXOm++Xhw4epVasWGRkZdOrUiT/++IPatWufGfcsPT2d5s2bEx8fT4cOHbj66qsZMmQIN9xwg1O/x/nIe0xPE5EEY0xcQeu78o7gC2BAYQtFpCbwPjDEGBMFXOXCWM6oFeDDRzd25NDxU9z7zQqyc7TxWCn1j86dO5/VB//tt9+mffv2dOnShV27drFp06ZztomIiKBDhw4AdOzYke3bt5dXuE7hsgfKjDELRCS8iFWuA6YYY3Y61j/gqljyaxtag5eHtuPh71fxym/reXpQm/LatVKqCEVduZeXgICAM6/nz5/PnDlzWLx4Mf7+/vTq1avAPvq+vv/ULHh6ela6qiE72whaAsEiMl9EEkTkpsJWFJFRIhIvIvEpKSlO2fmVHcP414VN+GTRNn5audspZSqlKp/AwEDS0tIKXHbs2DGCg4Px9/dn/fr1LFmypJyjKx92DjHhBXQE+gDVgMUissQYszH/isaYccA4sNoInBXA04PakLQ3jcd+SKRF3UDaNAxyVtFKqUqidu3aXHTRRbRt25Zq1apRr169M8sGDBjAhx9+SHR0NJGRkXTp0sXGSF3HZY3FAI6qoV8KaiwWkccBP2PMs473nwIzjDHfF1VmWRuL8zuQlsngdxbh4+XBtHu7UdPfx2llK6WKV1DDpiqbitRYXJyfgO4i4iUi/sAFQFJ5B1E30I8PbujIvmOZ3PftSnJyXZcYlVKqInJZIhCRicBiIFJEkkXkVhG5U0TuBDDGJAEzgETgb+ATY8waV8VTlNjGwTx/eVsWbEzh9Vkb7AhBKaVs48peQ9eWYJ3XgNdcFcP5uLZzYxKTj/L+/C1Eh9VgQNsGdoeklFLlwq2eLC7Os0Oi6NCoJg9PWsWm/QX3IlBKqapGE0Eevl6efHhDR6r5eDHq6wRSM7PsDkkppVxOE0E+9Wv48f71sew6fIKHvltJrjYeK6WqOE0EBegcUYtnBrVhTtIB3p577uPkSin3Vb16dQD27NnD8OHDC1ynV69eFNfN/c033+TEiRNn3ts5rLUmgkLcdGEThsWG8uacTfyetN/ucJRSFUzDhg2ZPHlyqbfPnwjsHNZaE0EhRISXh7ajbWgQD3y7kq0p6XaHpJRygccee+ys+QieffZZnnvuOfr06UNsbCzt2rXjp59+Ome77du307at9axsRkYGI0aMIDo6mmuuueassYbuuusu4uLiiIqKYvTo0YA1kN2ePXvo3bs3vXv3Bv4Z1hpg7NixtG3blrZt2/Lmm2+e2Z+rhru2c4iJCs/P22o8HvzOIu74OoGp91xEdV89ZEq5zG+Pw77Vzi2zfjsY+Eqhi0eMGMEDDzzA3XffDcCkSZOYMWMGDz74IEFBQRw8eJAuXbowZMiQQucD/uCDD/D39ycxMZHExERiY2PPLHvppZeoVasWOTk59OnTh8TERO677z7Gjh3LvHnzqFOnzlllJSQk8Pnnn7N06VKMMVxwwQX07NmT4OBgNm3axMSJE/n444+5+uqr+eGHH5wy3LXeERQjLNif966LZUtKOo98vwpXDsmhlCp/MTExHDhwgD179rBq1SqCg4Np0KABTz75JNHR0fTt25fdu3ezf3/hVcQLFiw4c0KOjo4mOjr6zLJJkyYRGxtLTEwMa9euZd26oqfLXbRoEUOHDiUgIIDq1aszbNgwFi5cCLhuuGu9vC2Brs3r8PjAVrw8fT0f/LGFu3s1tzskpaqmIq7cXWn48OFMnjyZffv2MWLECCZMmEBKSgoJCQl4e3sTHh5e4PDTeRV0t7Bt2zbGjBnDsmXLCA4OZuTIkcWWU9TFpquGu9Y7ghK6vXtTBkU3YMzMDSzY6JyhsJVSFcOIESP49ttvmTx5MsOHD+fYsWPUrVsXb29v5s2bx44dO4rcvkePHkyYMAGANWvWkJiYCEBqaioBAQHUqFGD/fv389tvv53ZprDhr3v06MGPP/7IiRMnOH78OFOnTqV79+5O/Lbn0kRQQiLCq8OjaVkvkH9PXMHOQyeK30gpVSlERUWRlpZGaGgoDRo04Prrryc+Pp64uDgmTJhAq1atitz+rrvuIj09nejoaF599VU6d+4MQPv27YmJiSEqKopbbrmFiy666Mw2o0aNYuDAgWcai0+LjY1l5MiRdO7cmQsuuIDbbruNmJgY53/pPFw6DLUrOHsY6vO149BxBr+ziNBgf6bc1ZVqPp62xaJUVaDDUDtfZRqGulJqUjuAt66NYf2+VB6fkqiNx0qpSk8TQSn0jqzLw5e05KeVe/jsz+12h6OUUmWiiaCU7u7VnP5R9Xh5ehJ/bTlodzhKVWp6Z+08pTmWmghKycNDGHNVe8Jr+/Pvb1aw+6hzunEp5W78/Pw4dOiQJgMnMMZw6NAh/Pz8zms7bSwuoy0p6Vz+7p80DQlg0h0X4uetjcdKnY+srCySk5OL7V+vSsbPz4+wsDC8vb3P+ryoxmJ9oKyMmoVUZ+zV7Rn1dQLP/LiGV4dHF/oYulLqXN7e3kRERNgdhlvTqiEn6BdVn/subs73CcmMX7rT7nCUUuq8aCJwkgf6tqR3ZAjP/byW+O2H7Q5HKaVKTBOBk3h4CG+OiCEsuBp3TVjO/lSt71RKVQ6aCJyoRjVvProxjuMns7lrfAKnsnPtDkkppYrlskQgIp+JyAERWVPMep1EJEdECp7zrZKJrB/Iq8OjWb7zKM9NW2t3OEopVSxX3hF8AQwoagUR8QT+D5jpwjjK3aDohtzRsykTlu7ku2XaeKyUqthclgiMMQuA4lpN/w38ABxwVRx2eaRfJN2a1+GZH9eycpc9E1IrpVRJ2NZGICKhwFDgQ7ticCUvTw/euTaGkEBf7hqfwMH0k3aHpJRSBbKzsfhN4DFjTE5xK4rIKBGJF5H4lJTKMylMcIAPH93YkcPHT3HPhOVk5WjjsVKq4rEzEcQB34rIdmA48L6IXFHQisaYccaYOGNMXEhISHnGWGZtQ2vwypXtWLrtMC9PT7I7HKWUOodtQ0wYY848Uy4iXwC/GGN+tCseVxoaE0Zi8jE+/3M70WE1GBoTZndISil1hssSgYhMBHoBdUQkGRgNeAMYY6pku0BRnry0Nev2pPLElNW0qBtI29AadoeklFKAjj5arlLSTjL4nUV4eQrT7+9OkJ938RsppZQT6FSVFURIoC/vXR/LnqMZPPfzOrvDUUopQBNBuevYJJi7ezXnh+XJzFizz+5wlFJKE4Ed7uvTgrahQTw5dTUH0nRwOqWUvTQR2MDHy4M3ru5A+slsnvhhtU7Rp5SylSYCm7SoF8hjA1rx+/oDfLdsl93hKKXcmCYCG93cNZyuzWrz/C/r2HHouN3hKKXclCYCG3l4CK9d1R5PER6etIqcXK0iUkqVP00ENgutWY3nLo8ifscRPlqwxe5wlFJuSBNBBTA0JpSBbevzxuyNrN1zzO5wlFJuRhNBBSAivDS0HTX9fXjou1VkZhU7IKtSSjmNJoIKolaAD68Oj2bD/jTGzt5odzhKKTeiiaAC6R1Zl+suaMzHC7eyZOshu8NRSrkJTQQVzFOXtqZxLX8enrSKtMwsu8NRSrkBTQQVTICvF2Ov7sDeYxk8N00HplNKuZ4mggqoY5Ng7urVjMkJycxcqwPTKaVcSxNBBXV/n5ZENQziiSmrSUnTie+VUq6jiaCC8vHy4I1rHAPTTUnUgemUUi6jiaACa1kvkEf7RzIn6QCT4nVgOqWUa2giqOBuuSiCC5vW5vlp69h56ITd4SilqiBNBBWch4cw5ur2eIjw8PcrdWA6pZTTaSKoBEJrVuPZIVEs236EcQu22h2OUqqK0URQSQyLDWVAVH3Gzt7Auj2pdoejlKpCNBFUEiLCy8PaUaOaDw9NWsnJbB2YTinlHC5LBCLymYgcEJE1hSy/XkQSHT9/iUh7V8VSVVgD07Vj/b40xs7SgemUUs7hyjuCL4ABRSzfBvQ0xkQDLwDjXBhLlXFxq3pc27kx4xZuZakOTKeUcgKXJQJjzALgcBHL/zLGHHG8XQKEuSqWqubpyxwD032vA9MppcquorQR3Ar8VthCERklIvEiEp+SklKOYVVM1sB07dlzNIPndWA6pVQZ2Z4IRKQ3ViJ4rLB1jDHjjDFxxpi4kJCQ8guuAuvYpBZ39mzG9wnJzNKB6ZRSZWBrIhCRaOAT4HJjjFZ4n6cH+rakTQNrYLqD6TownVKqdGxLBCLSGJgC3GiM0S4wpXB6YLq0k9k8/sNqHZhOKVUqruw+OhFYDESKSLKI3Coid4rInY5V/gvUBt4XkZUiEu+qWKqyyPqnB6bbz/fxyXaHo5SqhLxcVbAx5tpilt8G3Oaq/buTWy6KYE7Sfp6btpYLm9WmUS1/u0NSSlUitjcWq7Lz8BDGXOUYmG7SKh2YTil1XjQRVBFhwf6MHhLF39sP8/FCHZhOKVVymgiqkCtjQ+kfVY+xszaStFcHplNKlYwmgipERHh5aDuCqnnz4Hc6MJ1SqmQ0EVQxtav78n9XOgamm629cpVSxdNEUAX1aV2Pazs3YtyCrfy9rdDhnpRSCtBEUGU9fVkbGgX78/D3K0k/mW13OEqpCkwTQRV1emC63UcyeEEHplNKFUETQRUWF16LO3o247v4XTownVKqUJoIqrgH+7aktQ5Mp5QqgiaCKs7Hy4M3r+lAWmY2T0zRgemUUufSROAGIusH8kj/SGav28/3CTownVLqbJoI3MSt3SK4IKIWz09bx67DJ+wORylVgWgicBMeHsLrV7cH0IHplFJn0UTgRsKC/Rk9uA1/bz/MJzownVLKQROBmxneMYx+berxug5Mp5Ry0ETgZkSE/w1rR1A1Lx6dnEiuVhEp5fY0Ebih2tV9eXxga1bvPsYMfdBMKbdXokQgIveLSJBYPhWR5SLSz9XBKdcZGhNK87rVeX3WBrJzcu0ORyllo5LeEdxijEkF+gEhwM3AKy6LSrmcp4fwn34t2ZJynCkrdtsdjlLKRiVNBOL4fSnwuTFmVZ7PVCXVP6o+0WE1eGvOJp3ERik3VtJEkCAis7ASwUwRCQS0PqGSExEe6R/J7qMZTFy60+5wlFI2KWkiuBV4HOhkjDkBeGNVD6lKrlvzOnRpWot3523muM5boJRbKmkiuBDYYIw5KiI3AE8Dx4raQEQ+E5EDIrKmkOUiIm+LyGYRSRSR2PMLXTmDdVfQioPpp/jir+12h6OUskFJE8EHwAkRaQ88CuwAvipmmy+AAUUsHwi0cPyMcuxD2aBjk2D6tq7Lh39s4eiJU3aHo5QqZyVNBNnGGr/4cuAtY8xbQGBRGxhjFgBFTZh7OfCVsSwBaopIgxLGo5zs4X6RpJ/M5qMFOvSEUu6mpIkgTUSeAG4EfhURT6x2grIIBXbleZ/s+OwcIjJKROJFJD4lJaWMu1UFad0giCHtG/L5n9s4kJZpdzhKqXJU0kRwDXAS63mCfVgn7NfKuO+Cup8WON6BMWacMSbOGBMXEhJSxt2qwjzYtyVZOYZ35262OxSlVDkqUSJwnPwnADVEZBCQaYwpro2gOMlAozzvw4A9ZSxTlUF4nQCujmvExL936pwFSrmRkg4xcTXwN3AVcDWwVESGl3HfPwM3OXoPdQGOGWP2lrFMVUb392mBiPDmnE12h6KUKideJVzvKaxnCA4AiEgIMAeYXNgGIjIR6AXUEZFkYDSOdgVjzIfAdKwH1DYDJ9DnEiqE+jX8+NeFTfh00Tbu7NmUFvWK7BOglKoCSpoIPE4nAYdDFHM3YYy5tpjlBrinhPtX5eiuXs2Z+PcuXp+1kQ9v7Gh3OEopFytpY/EMEZkpIiNFZCTwK9YVvaqCagX4cFv3CGas3ceqXUftDkcp5WIlbSx+BBgHRAPtgXHGmMdcGZiy163dIgj292bMrA12h6KUcrEST0xjjPnBGPOQMeZBY8xUVwal7Bfo5809vZuzcNNB/tpy0O5wlFIuVGQiEJE0EUkt4CdNRHTC2yruhi5NqB/kx5iZG7CadJRSVVFxDb6BxpigAn4CjTFB5RWksoeftyf39WnB8p1H+T3pQPEbKKUqJZ2zWBXpqrgwwmv7M2bWBp3oXqkqShOBKpK3pwcPXtKS9fvSmJaoD34rVRVpIlDFGhzdkFb1Axk7eyNZOtG9UlWOJgJVLA8Pa0rLHYdO8H18st3hKKWcTBOBKpGLW9UltnFN3v59E5lZOtG9UlWJJgJVIiLCowNasS81k68X77A7HKWUE2kiUCXWpWltureow/vzN5OWmWV3OEopJ9FEoM7LI/0jOXIii08WbrM7FKWUk2giUOclOqwmA9vW55OFWzl8XCe6V6oq0ESgzttDl7QkIyuHD+brlJZKVQWaCNR5a1EvkKExYXy5eAd7j2XYHY5Sqow0EahSeaBvC4wxvP273hUoVdlpIlCl0qiWP9d1bsyk+F1sP3jc7nCUUmWgicBdGQM5ZesCes/FzfHx9GDs7I1OCkopZQdNBO7oZDp8NgA+6w+5pX9KuG6gHzdfFM7Pq/awbo9OT6FUZaWJwN1kn4TvboBdS2B3Aiz/qkzF3dGjGYF+XoydrVNaKlVZaSJwJ7k5MGUUbJ0Hl78PjbvC3Bch81ipi6zh782dPZsxJ+kACTsOOzFYpVR50UTgLoyBXx+GdT9Cv5cg5noY8DKcOAQLXy9T0TdfFE6d6j68OkOntFSqMnJpIhCRASKyQUQ2i8jjBSyvISLTRGSViKwVkZtdGY9bm/siJHwO3R6CrvdanzWMgQ7XwZIP4HDph4zw9/Hi3t7NWbrtMIs260T3SlU2LksEIuIJvAcMBNoA14pIm3yr3QOsM8a0B3oBr4uIj6ticluL34OFYyD2X9Dnv2cvu/gZ8PCG2f8teNsSuvaCxoTWrMZrOtG9UpWOK+8IOgObjTFbjTGngG+By/OtY4BAERGgOnAYyHZhTO5n5USY+SS0HgKD3gCRs5cHNYBuD0LSz7B9Ual34+vlyQN9W5CYfIyZa/eVMWilVHlyZSIIBXbleZ/s+Cyvd4HWwB5gNXC/MeacuRBFZJSIxItIfEpKiqvirXo2/AY/3QMRPeHKT8DDs+D1ut4LQWEw44kydScdGhNKs5AAxszaSI5OdK9UpeHKRCAFfJb/7NAfWAk0BDoA74pI0DkbGTPOGBNnjIkLCQkpXTQHN8NXV0Cqm0zAvv1P+H4kNGgPIyaAl2/h63pXg0ueg32JsGpiqXfp5enBf/pFsvlAOlNX7C51OUqp8uXKRJAMNMrzPgzryj+vm4EpxrIZ2Aa0ckk0x3ZC8jL4+GLYs8Ilu6gw9ibCxBFQszFcPxl8A4vfpu2VENYJfn/eeuCslAa0rU+70Bq8MXsjJ7N1SkulKgNXJoJlQAsRiXA0AI8Afs63zk6gD4CI1AMiga0uiabZxXDLTPDwgs8GwtqpLtmN7Q5tgfHDwDcIbpwKAbVLtp0I9P8fpO+HRW+Uevciwn/6R7L7aAbfLdtV/AZKKdu5LBEYY7KBe4GZQBIwyRizVkTuFJE7Hau9AHQVkdXA78BjxhjX9T+s3xZunwcNoq1qkz9etfrXVxWpe+HrK8DkWkmgRtj5bd+oE7S7Cha/C0d3ljqMHi3qcEFELd7+fTMnTmnbv1IVnUufIzDGTDfGtDTGNDPGvOT47ENjzIeO13uMMf2MMe2MMW2NMeNdGQ8A1UPgpp8hegTMewl+uBWyqsCY+sqtN7sAAB4HSURBVCcOW3cCJw5b1UEhLUtXTt9nAYE5z5Y6FBHhkf6RHEw/yRd/bS91OUqp8uGeTxZ7+8HQD6HPaFgzBb64DNIqcZfHU8fhm2vg0GYY8Q2Expa+rBph0PXfsOYH2Lm01MXEhdfi4lZ1+XD+Fo6d0InularI3DMRgFUn3v0huGY8HEiyGpH3rrI7qvOXfQom3QS74+HKT6Fpz7KXedH9ENgAZj4Buef05i2x//SLJDUzm3ELt5Q9JqWUy7hvIjit9SCrERmxhmZel789uwLLzYUf74TNc2DwW9BmiHPK9a1uPYG8OwHWTC51MW0aBjG4fUM+W7SdlLSTzolNKeV0mgjAajy+fS7UbQOTboQFYyp+I7Ix8NujVhVO3+cg9ibnlh89Ahp0gNmjraqnUnrokpacysnlvXk6paVSFZUmgtMC68HIX61eM3NfsIZrzsq0O6rCzX8Fln0MXe+Dbg84v3wPDxjwCqTtgb/eKXUxEXUCuDoujAlLd5B85IQTA1RKOYsmgry8/WDYx3Dx07B6Enw5CNIP2B3VuZZ+BH+8AjE3wCXPu24/TS6ENlfAn2/BsdI/KXxfnxaICG/N2eTE4JRSzqKJID8R6PEIXP0V7FsD43rDvtV2R/WPxElWlVCrQTDorXMHkXO2S56zxh/6vfQJp0GNatzUpQk/LE9m84E0JwanlHIGTQSFaXM53DLDejjr0/6w/le7I4KNM+HHuyC8u9VDyNPL9fsMDocL74bEb63G41K6q1czqnl76kT3SlVAmgiK0rADjJoHIZHw7fXW0At2NSLvWGx1E60XZT0r4O1Xfvvu9hAE1IUZT5b6+9eu7sut3ZsyffU+VieXfmpMpZTzaSIoTmB9uHk6RA21nrb98S5rAvjytG+N9cBYjTC4/gfwO2eAVtfyC7LaTXYtKdMYTbd3j6CmvzevzdKJ7pWqSDQRlIR3NRj+GfR60hqm+cshkF5O8yIc3moNHeETYI0fVL2Uw3CXVcwNUK+d1Z20lL2pAv28ubtXMxZsTGHJ1kNODlApVVqaCEpKBHo9Bld9YT2B/PHFsH+ta/eZtg++Hgo5p6wkULOxa/dXFA9Pa7L7YzthyXulLuamC8OpF+TLGJ3SUqkKQxPB+YoaalUV5WbBp/1gwwzX7CfjCHw9zLrzuH4y1HXNNA3nJaKH1Vtp4VhI21+qIvy8PbmvTwvidxxh3oYK2DVXKTekiaA0QmOtJ5FrN7MmgPnzbec2Ip86Ad+MgIMbYcR4CItzXtlldcnzVhvJ3BdKXcTVcY1oXMuf12ZuJFentFTKdpoISiuoIdw8wxrfZ/Yz8NO91gBwZZWTBd//C3YthSs/tibUqUhqN4ML7oAV40s9SJ+3pwcPXdKSpL2p/Lp6r5MDVEqdL00EZeHjD8O/gJ6Pwcrx8NXlcLwMjaC5ufDj3bBpFgwaa1VDVUQ9HgH/WmXqTjqkfUNa1Q9k7OyNZOWUfoRTpVTZaSIoKw8P6P2k9YDX7gT4uLc1rPX5MgZmPG4NbXHxMxB3i/NjdZZqNa3vvGMRrP+lVEV4eAgP94tk28HjfLO09LOhKaXKThOBs7QbbjUiZ2fCJ5fAptnnt/2C1+Dvj6DLPdD9YdfE6EyxIyGkNcx6ptTPVfRtXZfuLerw/C/r+HnVHufGp5QqMU0EzhQWZzUi1wqHb66Gxe+VrOrk74+taTPbXwv9XnT9+EHO4OkF/V+CI9usQfBKQUT48IaOdAoP5oFvVzB1RbKTg1RKlYQmAmerEWZNdNPqMpj5JEy7v+hG5NWTYfoj0HIgDHnHqmqqLJr3gRb9rbuZ4wdLVUSArxefj+zMhc1q89CkVXwfv8vJQSqlilOJzjqViE8AXPWVVcWz/EvrobATh89db9McmHoHNL4QrvocPL3LP9ay6vciZJ2w7mhKqZqPJ5/+qxPdmtfh0R8S+fZvbTNQqjxpInAVDw9ruseh4yB5mfUkckqeMXZ2/W3NhhbSGq771hrGojIKaQmdboOEL2D/ulIX4+ftycc3xdGrZQiPT1nN+CU7nBejUqpILk0EIjJARDaIyGYRebyQdXqJyEoRWSsif7gyHlu0vwZG/gKn0uGTvtb8wvvXwYSrrAHtbpwCfjXsjrJsej4GvkHWZPdleLDOz9uTD2/sSN/WdXn6xzV8+dd258WolCqUyxKBiHgC7wEDgTbAtSLSJt86NYH3gSHGmCjgKlfFY6tGna1G5JqNrQTwxWXg5ecYRK6u3dGVnX8t6PUEbJ1vzZlQBr5enrx/fUf6R9Vj9M9r+WThVufEqJQqlCvvCDoDm40xW40xp4BvgcvzrXMdMMUYsxPAGFN1B5+p2dhqRI681OoVdONUa9KXqqLTrVC7Bcx6yno6ugx8vDx497pYLmvXgBd/TeKjP7Y4KUilVEFcmQhCgbxdQJIdn+XVEggWkfkikiAiN7kwHvv5VocRE+DhDVCvTfHrVyae3lZ30kObYdknZS7O29ODt0Z0YHD7hvzvt/W8N2+zE4JUShXElXMdFtQZPn8FshfQEegDVAMWi8gSY8xZ8xmKyChgFEDjxjYOxewslbF3UEm06AdNe8P8VyD6GqvKqAy8PD144+r2eHkIr83cQHaO4f6+LZwUrFLqNFfeESQDjfK8DwPyPz6aDMwwxhw3xhwEFgDt8xdkjBlnjIkzxsSFhNg0MYsqngj0fxlOplrJwAm8PD0Yc1V7hncM4405G3l9ls5joJSzuTIRLANaiEiEiPgAI4Cf863zE9BdRLxExB+4ACjFQD2qwqjXBjqOtKqHUpwzJaWnh/DqldGM6NSId+Zu5lWd1EYpp3JZIjDGZAP3AjOxTu6TjDFrReROEbnTsU4SMANIBP4GPjHGrHFVTKqc9H7Keqhu1tNOK9LDQ3h5aDtu6NKYD+Zv4eXpSZoMlHISV7YRYIyZDkzP99mH+d6/BrzmyjhUOQuoYw1VPfsZ67mJ5n2dUqyHh/DC5W3x8vDg44XbyM41/HdQG6QyjM2kVAWmTxYr17jgDgiOgJlPQU6204oVEUYPbsOt3SL4/M/t/PentTrLmVJlpIlAuYaXrzUOUcp6WP6FU4sWEZ6+rDV39GzK10t28NSPazQZKFUGLq0aUm6u1WUQ3h3mvgRth1sT2jiJiPD4gFZ4eQjvzdtCTm4u/xsWjaeHVhOpquVQ+knW70sjaW8qbRoG0bVZHafvQxOBcp3T3Uk/6mENVd2/9COUFly88J9+kXh5ePDW75vIzjW8Nrx95U0G2SfhyA6o06JyzElRFRlj/R2yMyAr85/fQQ2deiFTkKycXLamHCdpbypJ+1JJ2pvG+r2pHEj7Z+KnO3o01USgKqEG0RBzgzV5TdwtULuZU4sXER68pCVeHsLrszeSk2t4/ar2eHlWglpPY+DQFtjyO2z+HbYvtIb0Du8Og99y+rGqlHKyICvDmvmvpL+zM88+iZ/5Xdy2J63X5zz3Cnj6QpvLIfYmCO9W5kR9KP2kdaLfl8q6vams35vG5gPpnHLM3+3j6UHzutXp1qIObRoE0ap+EK0aBFKnum+Z9lsYqWxd8OLi4kx8fLzdYajzkbYf3omFpr2sITZc5P35m3l1xgYui27Am9d0wLsiJoPMVNi2wOpNteV3OOqYe6FWM2uin8AGsOhN64TU81G46P6q+yR6UQ4kwW+PWseqtLyqgbdfEb/9rOHfz/z2LXhdLx/YsRgSJ8HJY1CrKcTcCB2us0YQLkJWTi5bUtJZvzfNcaVv/U7Jc5VfN9CXVg2CaN0gkNb1g2jdIIimIQFO//crIgnGmLgCl2kiUOVi4evw+/Pwr2kQ0cNlu/l4wVZemp7EgKj6vH1tDD5eNieD3FzYu9Jx1T8Xkv+G3GzwqQ4RPaH5xdCsD9SK+GebtH3WSXDdT1C3DQx+Gxp1su87lKfMVPjj/2Dph9YxirvFGqrknJN2Yb8dJ3MvX+dXr2VlwLqfYflXsGMRiCe0HGDdJTTvy8GMnDwnfKtqZ/OBNLJyrHPs6av81qdP+g2CaFU/kNouusrPTxOBsl9WJrzbyZp74Y4/wMPTZbv6bNE2nv9lHZe0qce718Xg6+W6fRUobT9smWud/LfMgxOOaTwbtLdO+s37WkOTF3elv346TP8PpO6BzrdbEx35Bro0dGMMicnH2HbwOAPa1sfPu5yOnTGw+nuY9Qyk77dOrn1GQ0Dt8tn/ecjKyWXnpkRy4r+i4Y6pVM86zAFq8V12Dybl9GSXqUfdQF/rRN8g8EzVjiuu8s+HJgJVMayZApNvtq5wO/7Lpbv6arH1jMHFrery/vWxrj2hZZ+CXUus6p7Nc2H/auvzgBBodrF14m/aG6qXYpysk2nw+wvw9zirwfLSMdDqUufGD+w+msGPK3bzw/JktqYcB6BRrWo8dWkb+kfVc+1De/vXWvN27/gTGsbCZWMgtKPr9neekvamsmjTwTNVO3mv8v09c7k2OIkrmUur9KV4kMupxt3x6TQSWg2yqpcqCE0EqmIwBj4bAIe3wr8TwC/Ipbv7ZulOnpy6mp4tQ/joxo7OTQaHtlgNvFt+h20LIes4eHhBoy5WXX/zPlCvnTVlqTMkx8PP98GBtdB6CFz6WrH108VJy8zitzX7mLI8mSVbrTm1O0fU4srYUOoG+fHK9PVs2J9G12a1GT04isj6Tr4byTwG8/5nJTm/GtB3NMTc5LxjVka7Dp9gzKwN/LTSGiuzXpAvrRx1+KerdiLq5LnKP7YbVn4DK76y2n6qBUP0COvupgIMO6+JQFUcuxOs+Zu7PQh9n3X57iYt28VjUxK5qFkdPr4pjmo+pUwGJ9McjbyOk/+R7dbnweHWFX+zPhDR3bVVNzlZ8NfbMP//rDrxS56F2JHndeLMzsll0eaDTFm+m1nr9pGZlUtEnQCGxYRyRUwojWr5n7XuN3/v5PVZG0nLzOKGLk146JKW1PT3Kdv3MAZWfQuz/wvHUyDuZrj4mTIPW+4sR0+c4t25m/lq8Q5E4LbuEYzsGkFIYAnr8nNzYdsfVlvC+l8g5xSEdbISQtQwa14SG2giUBXL1DthzQ9w77JymaVtckIyj0xeRZeI2nw6Mg5/nxL0ms7NhX2J/3Tt3LXUauT1DrAau5v3sap97OjieWgL/PKAlZgaX2h1NQ2JLHR1Ywzr9qYydflufly5h4PpJ6np783g6IYMiw2lQ6OaRVb9HDl+ijfmbGT8kh0EVfPmoUtacl3nxqXrortvNfz6H6sqLTTOurMJjT3/clwgMyuHrxZv5925m0k7mc1VHcN48JKWNKhRrfSFHj8Eid/B8i+tp+x9qkPbYRD7L6v6qxyfF9FEoCqW1D3wTkdrIpurvyyXXf60cjcPfreSuCa1+OzmTlT3LSAZpB+wGnk3/w5b51lXqwD12+Vp5L3A6k5oN2OsaohZT8HJdOj+MHR/yOot47A/NZMfV+xm6ordrN+XhrencHGrugyLDaN3ZN3z7lG1fl8qz09bx19bDhFZL5DRg9vQtXkJH27KOArzXoZlH1tVJn2fhQ43VIhqoNxcw0+rdjNm5kZ2H82gd2QIjw1sRav6Tqy6NAaSl1kJYc0U63mRum2suwQnTOJUEpoIVMUz//9g/stw82/QpKvzy8/KhMyj1gko8yhkHmP5hm1MW7qOyBo5DGsdgE92+j/Lj6fAQcfEeP61z27kDazn/PicJT0FZj5h9bip05LMAWP5LS2CKct38+fmg+QaiGlck2GxYQxq14DggLIlMWMMM9fu48Vfk0g+ksGAqPo8dVnrs6qUzpKbC6smwpzRcOKQ1R2091MVphpo0aaDvDw9iXV7U2kbGsSTA1uXPLmVVmYqrJ1iVR3tTgBPH2g92PGwWg+XJUdNBKriOXUC3o2zetbcPu/cf/zGwKnj55zMz7wu7rPszCJ3nyHV8K1eCw//YKuhslowNIyxqnzqt68QV6ollZNrWL9oCvUXPkntrH18k30xX1W/hX6xLbkiJpSmIc6vk87MyuGThVutcZ6MYVT3ptzdu9nZ1W57V1nVQMl/Q1hnqzdQg3MmILTFuj2pvDJjPQs2phAWXI1H+kcyOLohHuU9PMm+NbDia6vNJPMo1GwCsTdCh+utXmJOpIlAVUyJk2DK7dbVN3LuyTy3qOGrxep15FfTGgPGr0ae1wV9Fnzms9lbM7j729W0bhDE17dcQA3/yvnk7qb9afywfDc/rdzN3mOZ1PXNZkzIdLofmgQBIcjAV61hEVxYD73vWCav/JbEjyv3UD/Ij8cHtuLySH9k3osQ/xlUqwWXPA/tr60QyXXP0Qxen7WRKSuSCfLz5t8XN+fGC5uU/7Mm+WVlWg3Ly7+02n7Ew6o6jb3J+u2Ep8s1EaiKKTcXvr8J9q8r4ck8z2e+Ncp0Yvk9aT93jV9Oi3rVGX/rBWWuMikvB9NP8vPKPUxdsZvVu4/h6SH0bBnCsNhQ+rauZ3WR3bPC6mq6LxEiL7UaZGuEuTSuhB2Hee6nNbTa/zNP+XxHEOlIp9uh95MuH6ytJI5lZPHB/C189uc2AG6+KJy7ezavmBcBh7fCivGwYgKk74Pq9azhLGJuLFPnBE0EShVg/oYDjPo6gaZ1Aphw2wXl9qj/+crMymFO0n6mLN/NHxtTyMk1tA0NYlhMGIPbNyy4W2NONiz9wBoC3MPTeiq5022ue6J7zwrMr/9BdsezklY8eeom2sZ245H+rUre7dIFTmbnMH7JTt6Zu4ljGVkMjQnl4X6RhNYsQ0+g8pKTDZtnW20JG2eCybE6BfT5b6mK00SgVCEWbkrhti/jaVLbn1suiqCajyd+3p74+3hSzdt6Xc3xuprjta+Xh8unx8zNNcTvOMKU5cn8unovaZnZ1A/y44qYUIbFhtKyXgmfVziyHX55yOoGGxoHQ96GelHOC/TEYWsMqYQvrPaeS54nNfJK3p23hc//3Iavlyf39WnOyK4R5TruU26u4ZfVe3lt5np2Hc6gW/M6PD6wFW1Da5RbDE6VuhdWfWP9DZv2LFURmgiUKsJfmw8y6usE0k+WfErNankShJ+3R57XnvmWnZ1I/M5KKh5n1vf38aKatycZWTn8mriHqSt3s+twBv4+ngxoW58rY8Po0rR26eZaOD2Oz4zHrbaXrvdZI5t6l+GqODfXeoJ2znNWmZ1HQe8nrGo7h60p6bz4axJz1x8gok4AzwxqzcWtXN8Da/GWQ/zvtyQSk4/Rqn4gT17amh4tSzG8RxWjiUCpYmScyuHIiVNkZOWQcSqHzKycM68zshzvT+WQkZVLxqlsa1lWDhmncgteN9/702PTlIQIdGteh6ExofSPqk9AQc88lMaJwzDraVg5wRpKedCbpbu63J1g9Qbasxwad7V6AxVxlzFvwwFemLaOrQeP0ysyhKcva0Pzus7vybRxfxqv/LaeuesP0LCGHw/3i+SKmNDKO1GRk2kiUMpmWTn/JIzMU7lnEsmJU9mOJGN9lmsMPVqEUL+GCwcr2zofpj0AR7ZZD3X1e6Fk/fqPH4Lfn7PqrKvXteakbndViXolncrO5avF23lrziYysnIY2TWc+/q2IMiv7I21+1MzeWP2RibF7yLA14t7ejdnZNfw8hs5tZLQRKCUOltWBvzxqjV2kV9NGPAKtBte8Ek9N8dqA5j7gvUwVJe7oOdjpRo0MCXtJK/P2sB38buo5e/DI/0juSquUamu2tMysxi3YCsfL9xKTq7hpgvDubd380rTA6y82ZYIRGQA8BbgCXxijHmlkPU6AUuAa4wxk4sqUxOBUk60bw1Mu8+q7mneFy4bC8FN/lm+a5k1J8LeldYUmpe+BnVbl3m3q5OP8dy0tcTvOELb0CBGD46iU3jJnjbOysll4t87eWvOJg4dP8WQ9g35T79IGtcu5OlmBdiUCETEE9gIXAIkA8uAa40x6wpYbzaQCXymiUCpcpabA39/bPX+wVh9/9tdZd0BrBhvTZ/Z70Voe6VTH04zxvDzqj288tt69h7LZEj7hjw+sBUNC+naaYxhxpp9vDpzA9sOHqdL01o8eWlrosPsf06hMrArEVwIPGuM6e94/wSAMeZ/+dZ7AMgCOgG/aCJQyiZHd1lX/xtnWE+2isc/1UAuHF77xKlsPpy/hY8WbMVDhLt6NWNUj6Zn1fEv236Yl6cnsWLnUVrWq84TA1vTKzLE5d14q5KiEoGTuiMUKBTYled9MnBBvsBCgaHAxViJoEAiMgoYBdC4cWOnB6qUAmo2gmu/hXU/Wg8wXfQA1G3l8t36+3jxUD+rreB/vyUxdvZGvlu2i6cua03LeoG8OmM9s9btp16QL/93ZTuGdyxdm4IqnCsTQUF/qfy3H28CjxljcorK7MaYccA4sO4InBahUupsIhA11PopZ41q+fP+9R35a8tBnp+2jrsnLAeguq8X/+nXklu6RZRsLgl13lx5VJOBRnnehwF78q0TB3zrSAJ1gEtFJNsY86ML41JKVWBdm9Xhl3934/uEZPYezeCmruHUqaDDf1QVrkwEy4AWIhIB7AZGANflXcEYE3H6tYh8gdVGoElAKTfn5enBtZ21Gri8uCwRGGOyReReYCZW99HPjDFrReROx/IPXbVvpZRSJefSCjdjzHRger7PCkwAxpiRroxFKaVUweyfKUIppZStNBEopZSb00SglFJuThOBUkq5OU0ESinl5jQRKKWUm6t08xGISAqwo5Sb1wEOOjGcyk6Px9n0ePxDj8XZqsLxaGKMKXDOzkqXCMpCROILG33PHenxOJsej3/osThbVT8eWjWklFJuThOBUkq5OXdLBOPsDqCC0eNxNj0e/9BjcbYqfTzcqo1AKaXUudztjkAppVQ+mgiUUsrNuU0iEJEBIrJBRDaLyON2x2MnEWkkIvNEJElE1orI/XbHZDcR8RSRFSLyi92x2E1EaorIZBFZ7/g3cqHdMdlFRB50/B9ZIyITRcTP7phcwS0SgYh4Au8BA4E2wLUi0sbeqGyVDTxsjGkNdAHucfPjAXA/kGR3EBXEW8AMY0wroD1uelxEJBS4D4gzxrTFmmBrhL1RuYZbJAKgM7DZGLPVGHMK+Ba43OaYbGOM2WuMWe54nYb1Hz3U3qjsIyJhwGXAJ3bHYjcRCQJ6AJ8CGGNOGWOO2huVrbyAaiLiBfhz7rzrVYK7JIJQYFee98m48YkvLxEJB2KApfZGYqs3gUeBXLsDqQCaAinA546qsk9EJMDuoOxgjNkNjAF2AnuBY8aYWfZG5RrukgikgM/cvt+siFQHfgAeMMak2h2PHURkEHDAGJNgdywVhBcQC3xgjIkBjgNu2aYmIsFYNQcRQEMgQERusDcq13CXRJAMNMrzPowqeotXUiLijZUEJhhjptgdj40uAoaIyHasKsOLRWS8vSHZKhlINsacvkOcjJUY3FFfYJsxJsUYkwVMAbraHJNLuEsiWAa0EJEIEfHBavD52eaYbCMiglUHnGSMGWt3PHYyxjxhjAkzxoRj/buYa4ypkld9JWGM2QfsEpFIx0d9gHU2hmSnnUAXEfF3/J/pQxVtOPeyO4DyYIzJFpF7gZlYLf+fGWPW2hyWnS4CbgRWi8hKx2dPGmOm2xiTqjj+DUxwXDRtBW62OR5bGGOWishkYDlWT7sVVNGhJnSICaWUcnPuUjWklFKqEJoIlFLKzWkiUEopN6eJQCml3JwmAqWUcnOaCJQqRyLSS0c4VRWNJgKllHJzmgiUKoCI3CAif4vIShH5yDFfQbqIvC4iy0XkdxEJcazbQUSWiEiiiEx1jFGDiDQXkTkissqxTTNH8dXzjPc/wfHUqlK20USgVD4i0hq4BrjIGNMByAGuBwKA5caYWOAPYLRjk6+Ax4wx0cDqPJ9PAN4zxrTHGqNmr+PzGOABrLkxmmI96a2UbdxiiAmlzlMfoCOwzHGxXg04gDVM9XeOdcYDU0SkBlDTGPOH4/Mvge9FJBAINcZMBTDGZAI4yvvbGJPseL8SCAcWuf5rKVUwTQRKnUuAL40xT5z1ocgz+dYranyWoqp7TuZ5nYP+P1Q206ohpc71OzBcROoCiEgtEWmC9f9luGOd64BFxphjwBER6e74/EbgD8f8DskicoWjDF8R8S/Xb6FUCemViFL5GGPWicjTwCwR8QCygHuwJmmJEpEE4BhWOwLAv4APHSf6vKN13gh8JCLPO8q4qhy/hlIlpqOPKlVCIpJujKludxxKOZtWDSmllJvTOwKllHJzekeglFJuThOBUkq5OU0ESinl5jQRKKWUm9NEoJRSbu7/AbJs48hOUwwkAAAAAElFTkSuQmCC\n",
      "text/plain": [
       "<Figure size 432x288 with 1 Axes>"
      ]
     },
     "metadata": {
      "needs_background": "light"
     },
     "output_type": "display_data"
    }
   ],
   "source": [
    "from matplotlib import pyplot\n",
    "# plot train and validation loss\n",
    "pyplot.plot(history.history['loss'])\n",
    "pyplot.plot(history.history['val_loss'])\n",
    "pyplot.title('model train vs validation loss')\n",
    "pyplot.ylabel('loss')\n",
    "pyplot.xlabel('epoch')\n",
    "pyplot.legend(['train', 'validation'], loc='upper right')\n",
    "pyplot.show() "
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 6,
   "metadata": {},
   "outputs": [
    {
     "name": "stdout",
     "output_type": "stream",
     "text": [
      "INFO:tensorflow:Assets written to: saved_model/model1_10epochs_bit/assets\n"
     ]
    },
    {
     "name": "stderr",
     "output_type": "stream",
     "text": [
      "INFO:tensorflow:Assets written to: saved_model/model1_10epochs_bit/assets\n"
     ]
    }
   ],
   "source": [
    "import tensorflow as tf \n",
    "# Save fine-tuned model as SavedModel\n",
    "export_module_dir = 'saved_model/model1_10epochs_bit'\n",
    "tf.saved_model.save(model, export_module_dir)"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "### Experiment 2:"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 4,
   "metadata": {},
   "outputs": [
    {
     "name": "stdout",
     "output_type": "stream",
     "text": [
      " 9/10 [==========================>...] - ETA: 16s - loss: 1.0949 - acc: 0.6892WARNING:tensorflow:From /opt/conda/lib/python3.7/site-packages/keras/callbacks/tensorboard_v1.py:343: The name tf.Summary is deprecated. Please use tf.compat.v1.Summary instead.\n",
      "\n"
     ]
    },
    {
     "name": "stderr",
     "output_type": "stream",
     "text": [
      "WARNING:tensorflow:From /opt/conda/lib/python3.7/site-packages/keras/callbacks/tensorboard_v1.py:343: The name tf.Summary is deprecated. Please use tf.compat.v1.Summary instead.\n",
      "\n"
     ]
    },
    {
     "name": "stdout",
     "output_type": "stream",
     "text": [
      "10/10 [==============================] - 251s 25s/step - loss: 1.0787 - acc: 0.7014 - val_loss: 0.9322 - val_acc: 0.7646\n",
      "Epoch 2/50\n",
      "10/10 [==============================] - 227s 23s/step - loss: 1.0234 - acc: 0.7902 - val_loss: 0.9646 - val_acc: 0.7022\n",
      "Epoch 3/50\n",
      "10/10 [==============================] - 224s 22s/step - loss: 0.8034 - acc: 0.7773 - val_loss: 0.5256 - val_acc: 0.7757\n",
      "Epoch 4/50\n",
      "10/10 [==============================] - 225s 22s/step - loss: 0.5346 - acc: 0.8131 - val_loss: 0.4115 - val_acc: 0.8231\n",
      "Epoch 5/50\n",
      "10/10 [==============================] - 224s 22s/step - loss: 0.3881 - acc: 0.8379 - val_loss: 0.3881 - val_acc: 0.8166\n",
      "Epoch 6/50\n",
      "10/10 [==============================] - 225s 22s/step - loss: 0.3340 - acc: 0.8586 - val_loss: 0.5155 - val_acc: 0.7757\n",
      "Epoch 7/50\n",
      "10/10 [==============================] - 225s 23s/step - loss: 0.4281 - acc: 0.8205 - val_loss: 0.3953 - val_acc: 0.8238\n",
      "Epoch 8/50\n",
      "10/10 [==============================] - 226s 23s/step - loss: 0.5008 - acc: 0.8162 - val_loss: 0.4478 - val_acc: 0.7971\n",
      "Epoch 9/50\n",
      "10/10 [==============================] - 224s 22s/step - loss: 0.4305 - acc: 0.8338 - val_loss: 0.5024 - val_acc: 0.7802\n",
      "Epoch 10/50\n",
      "10/10 [==============================] - 224s 22s/step - loss: 0.4371 - acc: 0.8285 - val_loss: 0.4469 - val_acc: 0.8069\n",
      "Epoch 11/50\n",
      "10/10 [==============================] - 225s 23s/step - loss: 0.3511 - acc: 0.8576 - val_loss: 0.4218 - val_acc: 0.8075\n",
      "Epoch 12/50\n",
      "10/10 [==============================] - 225s 22s/step - loss: 0.3040 - acc: 0.8717 - val_loss: 0.4763 - val_acc: 0.7932\n",
      "Epoch 13/50\n",
      "10/10 [==============================] - 222s 22s/step - loss: 0.3133 - acc: 0.8656 - val_loss: 0.4696 - val_acc: 0.7724\n",
      "Epoch 14/50\n",
      "10/10 [==============================] - 225s 23s/step - loss: 0.3681 - acc: 0.8486 - val_loss: 0.6499 - val_acc: 0.7744\n",
      "Epoch 15/50\n",
      " 9/10 [==========================>...] - ETA: 15s - loss: 0.4027 - acc: 0.8379Restoring model weights from the end of the best epoch\n",
      "10/10 [==============================] - 227s 23s/step - loss: 0.4255 - acc: 0.8305 - val_loss: 0.7518 - val_acc: 0.7178\n",
      "Epoch 00015: early stopping\n",
      "Training model is finished\n",
      "5/5 [==============================] - 101s 20s/step - loss: 0.3931 - acc: 0.8127\n",
      "Test accuracy: 0.8126829\n",
      "CPU times: user 5h 31s, sys: 1h 5min 33s, total: 6h 6min 5s\n",
      "Wall time: 1h 12s\n"
     ]
    }
   ],
   "source": [
    "%%time\n",
    "history,model = BiT_L_ResNet.train_Bit_L(train,validation,test,epochs=50)"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": null,
   "metadata": {},
   "outputs": [
    {
     "data": {
      "image/png": "iVBORw0KGgoAAAANSUhEUgAAAYIAAAEWCAYAAABrDZDcAAAABHNCSVQICAgIfAhkiAAAAAlwSFlzAAALEgAACxIB0t1+/AAAADh0RVh0U29mdHdhcmUAbWF0cGxvdGxpYiB2ZXJzaW9uMy4yLjEsIGh0dHA6Ly9tYXRwbG90bGliLm9yZy+j8jraAAAgAElEQVR4nOzdd1yVdfvA8c/FEkFUVARc4RZQcODKXLkzM83SxtPOrGz3e7Ke9lM9Tdtltqdlao40NcuRmeVWcOJGHIjiQmR9f398j4bK5hwOyPV+vXhxzrnv+3uuc9D7uu/vFGMMSimlKi4PdweglFLKvTQRKKVUBaeJQCmlKjhNBEopVcFpIlBKqQpOE4FSSlVwmgiU04jI5yLyfCH33SEivV0Yy/UiMtdV5buSiDwjIl87HjcQkeMi4lnQvsV8rzgR6VHc4/Mpd4GI3O7scpVreLk7AKXOJSKfAwnGmCeKW4Yx5hvgG6cF5SbGmF1AFWeUldv3aoyJdEbZqnzTOwJV7oiIXsAo5USaCCoYR5XM/4nIWhE5ISKfiEiwiPwsIsdEZJ6IBObY/wpH9UGK43Y/PMe2NiKy0nHc94DvOe91uYisdhy7RESiChHfSOB64N+OKpEZOeJ+VETWAidExEtExojIVsf7rxeRITnKuVlEFud4bkRklIhsEZHDIvKeiEgu719HRE6KSI1zPudBEfEWkSYislBEjjhe+z6PzzFbREaf89oaERnqePyWiOwWkaMiskJEuuZRTpgjdi/H84aO9z8mIr8Atc7Z/wcR2eeIb5GIRBbie+3teFxJRN4UkUTHz5siUsmxrYeIJIjIwyJyQET2isgtuf8Vz/sMHiLyhIjsdBz7pYhUc2zzFZGvRSTZ8e9kmYgEO7bdLCLbHJ91u4hcX5j3U8VgjNGfCvQD7ACWAsFAXeAAsBJoA1QCfgOeduzbDDgB9AG8gX8D8YCP42cn8KBj2zAgA3jecWxbR9kdAU/gJsd7V8oRR+88Yvz8dDnnxL0aqA9Udrx2NVAHe0Ez3BFrqGPbzcDiHMcb4CegOtAASAL65/H+vwF35Hj+KjDO8XgC8B/He/oCl+RRxo3AHzmeRwApOT7/DUBNbPXsw8A+wNex7Rnga8fjMEfsXo7nfwJjHX+rbsCx0/s6tt8KBDi2vwmsLsT32tvx+DnHv43aQBCwBPivY1sPINOxjzdwGZAKBObx+RcAt+eIKR5ohK3mmgJ85dh2JzAD8HP8O2kHVAX8gaNAc8d+oUCku///XKg/ekdQMb1jjNlvjNkD/A78ZYxZZYw5BfyITQpgT64zjTG/GGMygNeAysDFQCfsCeFNY0yGMWYSsCzHe9wBfGiM+csYk2WM+QI45TiuuN42xuw2xpwEMMb8YIxJNMZkG2O+B7YAHfI5/iVjTIqx9e7zgdZ57PctcC2A465hhOM1sMnuIqCOMSbNGLM49yL4EWgtIhc5nl8PTHF8xxhjvjbGJBtjMo0xr2NP3M3z+/Ai0gBoDzxpjDlljFmEPYmeYYz51BhzzPE+zwDRp6++C+F64DljzAFjTBLwLPCvHNszHNszjDGzgOMFxZyj3LHGmG3GmOPAY8AIx11OBjYhNnH8O1lhjDnqOC4baCkilY0xe40xcYX8HKqINBFUTPtzPD6Zy/PTjZN1sFf9ABhjsoHd2DuJOsAeY0zOWQt35nh8EfCw43Y/RURSsFfzdUoQ9+6cT0TkxhxVTylAS86pKjnHvhyPU8m7EXYS0FlE6mCvug02YYK9KxLgb0eV2a25FWCMOQbMxCYRHL/PNF47qlg2OKpwUoBqBcQO9rs7bIw5keO1M9+5iHiKyEuO6rKj2Kt9ClFuzvJz/g13cvbfK9kYk5njeX7fYUHlemHvSr8C5gDfOaqjXhERb8dnHA6MAvaKyEwRaVHIz6GKSBOByk8i9oQOnLk6rg/sAfYCdc+pZ2+Q4/Fu4AVjTPUcP37GmAmFeN+8psQ987rjSvsjYDRQ0xhTHYjFnqRLxBiTAswFrgGuAyacTnjGmH3GmDuMMXWw1Rrvi0iTPIqaAFwrIp2xd1LzHbF3BR51lB/oiP1IIWLfCwSKiH+O13J+59cBg4He2MQS5nj9dLkFTTV81t/bUXZiAccURm7lZgL7HXcXzxpjIrB3mpdjq9UwxswxxvTBVgttxP69lQtoIlD5mQgMFJFeIuKNrcs+ha07/hP7n/k+R8PtUM6ulvkIGCUiHcXyF5GBIhJQiPfdj61Pzo8/9sSWBOBouGxZlA9XgG+xJ6Sr+KdaCBG5WkTqOZ4edsSQlUcZs7AnwOeA7x13VGDr8DMdsXuJyFPYevF8GWN2AsuBZ0XER0QuAQbl2CUA+/dJxta5v3hOEQV9rxOAJ0QkSERqAU8BxR6jcE65Dzoauqs44vreGJMpIj1FpJXYcRJHsVVFWWI7MFzhSHqnsNVQeX3PqoQ0Eag8GWM2YRs13wEOYk86g4wx6caYdGAotlH2MPY2fkqOY5dj2wnedWyPd+xbGJ8AEY4qn6l5xLYeeB2bkPYDrYA/ivYJ8zUdaIq9al2T4/X2wF8ictyxz/3GmO15xHgK+530JkcywVaF/AxsxlaTpHFOtVc+rsM2wB8Cnga+zLHtS0d5e4D12IbfnAr6Xp/HJpq1wDpsJ4JCDRAswKfYKqBFwHbs573XsS0EWxV3FNgALMQmHw/shUci9rN2B+52QiwqF3J2Fa9SSqmKRu8IlFKqgtNEoJRSFZwmAqWUquA0ESilVAVX7ibvqlWrlgkLC3N3GEopVa6sWLHioDEmKLdt5S4RhIWFsXz5cneHoZRS5YqI7Mxrm8uqhkTkU8dMg7F5bG8hIn+KyCkRecRVcSillMqfK9sIPgf657P9EHAfdiIzpZRSbuKyROCYGfFQPtsPGGOWYYeUK6WUcpNy0UbgWFRjJECDBg0K2FspVZ5kZGSQkJBAWlqau0O5IPj6+lKvXj28vb0LfUy5SATGmPHAeICYmBidE0OpC0hCQgIBAQGEhYUh5y8ap4rAGENycjIJCQk0bNiw0MfpOAKllFulpaVRs2ZNTQJOICLUrFmzyHdXmgiUUm6nScB5ivNdurL76ATsFMHNHYte3yZ28fBRju0hIpIAPISdAz1BRAqck724ko+f4rkZ6zmapm3TSimVkyt7DV1rjAk1xngbY+oZYz4xxowzxoxzbN/neL2qY/WqejnWKnW6P7Ym8/mS7fQZu5B56/cXfIBSqkJISUnh/fffL/Jxl112GSkpKS6IqPRVmKqhK6LrMPWeLgT6+XD7l8u5d8IqDh4/5e6wlFJullciyMrKf0G0WbNmUb16dVeFVaoqTCIAiKpXnemjL+HhPs2YE7uPPmMXMnXVHnRxHqUqrjFjxrB161Zat25N+/bt6dmzJ9dddx2tWrUC4Morr6Rdu3ZERkYyfvz4M8eFhYVx8OBBduzYQXh4OHfccQeRkZH07duXkydPuuvjFEu5W6EsJibGOGOuoS37j/Ho5LWs3JVCz+ZBvDCkFXWqV3ZChEqpotiwYQPh4eEAPDsjjvWJzq0hjqhTlacHRea5fceOHVx++eXExsayYMECBg4cSGxs7Jnul4cOHaJGjRqcPHmS9u3bs3DhQmrWrHlm3rPjx4/TpEkTli9fTuvWrbnmmmu44ooruOGGG5z6OYoi53d6moisMMbE5LZ/hbojyKlpcAA/jLqYpwdFsHTbIfqMXchXf+4gO7t8JUallHN16NDhrD74b7/9NtHR0XTq1Indu3ezZcuW845p2LAhrVu3BqBdu3bs2LGjtMJ1inIxoMxVPD2EW7o0pHd4MI//uI4np8UxY81eXrqqFY2Cqrg7PKUqnPyu3EuLv7//mccLFixg3rx5/Pnnn/j5+dGjR49c++hXqlTpzGNPT89yVzVUYe8Icqpfw48vb+3Aq8Oi2LjvKP3f+p0PFmwlMyvb3aEppVwsICCAY8eO5brtyJEjBAYG4ufnx8aNG1m6dGkpR1c6KvQdQU4iwtUx9enePIinp8Xx8uyNzFyXyMtXRRFZp5q7w1NKuUjNmjXp0qULLVu2pHLlygQHB5/Z1r9/f8aNG0dUVBTNmzenU6dObozUdSpsY3FBfl63lyenxXE4NZ1R3Rtx76VN8fX2dPn7KlXR5NawqUpGG4udZECrUH59qDtD29Tlvflbuezt31m2I89ZtZVSqtzSRJCPan7evHp1NF/d1oH0zGyuHvcnT02L5fipTHeHppRSTqOJoBC6Ng1izgPduLVLQ75aupO+Yxcyf9MBd4ellFJOoYmgkPwrefHUoAgmjboY/0pe3PLZMh76fjWHT6S7OzSllCoRTQRF1O6iQH667xLu69WU6WsS6T12IXPj9rk7LKWUKjZNBMVQycuTh/o0Y8a9lxBc1Zd7J6ziyEmd3lopVT5pIiiB8NCqvHxVFKcys5m+eo+7w1FKlYIqVeysA4mJiQwbNizXfXr06EFB3dzffPNNUlNTzzx357TWmghKqGXdqoSHVuX75bvdHYpSqhTVqVOHSZMmFfv4cxOBO6e1duUKZZ+KyAERic1ju4jI2yISLyJrRaStq2JxJRFhRPv6xO45SlziEXeHo5QqokcfffSs9QieeeYZnn32WXr16kXbtm1p1aoV06ZNO++4HTt20LJlSwBOnjzJiBEjiIqKYvjw4WfNNXTXXXcRExNDZGQkTz/9NGAnsktMTKRnz5707NkT+Gdaa4CxY8fSsmVLWrZsyZtvvnnm/Vw13bUrp5j4HHgX+DKP7QOApo6fjsAHjt/lzpWt6/LCrA1MXLabZwfrdBRKFdvPY2DfOueWGdIKBryU5+YRI0bwwAMPcPfddwMwceJEZs+ezYMPPkjVqlU5ePAgnTp14oorrshzPeAPPvgAPz8/1q5dy9q1a2nb9p/r2hdeeIEaNWqQlZVFr169WLt2Lffddx9jx45l/vz51KpV66yyVqxYwWeffcZff/2FMYaOHTvSvXt3AgMD2bJlCxMmTOCjjz7immuuYfLkyU6Z7tqVS1UuAvIbijsY+NJYS4HqIhLqqnhcqZqfN/0jQ/hx1R7SMvJf1UgpVba0adOGAwcOkJiYyJo1awgMDCQ0NJTHH3+cqKgoevfuzZ49e9i/P+8lbhctWnTmhBwVFUVUVNSZbRMnTqRt27a0adOGuLg41q9fn288ixcvZsiQIfj7+1OlShWGDh3K77//Drhuumt3TjpXF8hZsZ7geG3vuTuKyEhgJECDBg1KJbiiGt6+PtPXJDInbh+DW9d1dzhKlU/5XLm70rBhw5g0aRL79u1jxIgRfPPNNyQlJbFixQq8vb0JCwvLdfrpnHK7W9i+fTuvvfYay5YtIzAwkJtvvrnAcvKb/81V0127s7E4t3usXL8BY8x4Y0yMMSYmKCjIxWEVT+dGNalfozLfL9NGY6XKmxEjRvDdd98xadIkhg0bxpEjR6hduzbe3t7Mnz+fnTt35nt8t27d+OabbwCIjY1l7dq1ABw9ehR/f3+qVavG/v37+fnnn88ck9f01926dWPq1KmkpqZy4sQJfvzxR7p27erET3s+d94RJAD1czyvByS6KZYS8/AQrm5Xn7G/bGZXcioNavq5OySlVCFFRkZy7Ngx6tatS2hoKNdffz2DBg0iJiaG1q1b06JFi3yPv+uuu7jllluIioqidevWdOjQAYDo6GjatGlDZGQkjRo1okuXLmeOGTlyJAMGDCA0NJT58+efeb1t27bcfPPNZ8q4/fbbadOmjUtXPXPpNNQiEgb8ZIxpmcu2gcBo4DJsI/HbxpgOBZVZWtNQF0diykm6vPwbo3s24eG+zd0djlLlgk5D7XxlZhpqEZkA/Ak0F5EEEblNREaJyCjHLrOAbUA88BFwt6tiKS11qlemW9MgJq1IIEvXPlZKlRMuqxoyxlxbwHYD3OOq93eX4e3rc/c3K1m0JYmezWu7OxyllCqQjix2st7hwdTw92GiNhorVWjlbaXEsqw436UmAifz8fJgaJu6zNuwn+Tjp9wdjlJlnq+vL8nJyZoMnMAYQ3JyMr6+vkU6ThevLwljYO8awECdNmdeHt6+Ph8v3s6Pq/Zwe9dG7otPqXKgXr16JCQkkJSU5O5QLgi+vr7Uq1evSMdoIiiOEwdh7URY9TUciAPfavDvHeBhb7CaBgfQpkF1vlu2m9suaZjnsHSlFHh7e9OwYUN3h1GhadVQYWVlwqbZ8N318HoLmPMYeFWCyCGQdgQObz9r9+Ex9Yk/cJyVu9wzraxSShWWJoKCJG2GX56CNyJgwnDYtRQ63gl3/Qkj58MlD9r99q4567DLo+vg5+OpjcZKqTJPq4Zyk3YU4qbYqp+EZSCe0KwftL7e/vb0/mffoHDw8LaJoOXQMy9XqeTFwFah/LQ2kacGReBfSb9qpVTZpGen07KzYediWPUNrJ8GmSchqAX0+S9EDYeA4NyP8/KB4Ijz7gjANhr/sCKBmWv3ck37+rkcrJRS7qeJIGUXrJ4Aq7+BlJ1QqSpEj4A2N0DddlCYht6QKNg0y/YiyrF/u4sCaRzkz/fLd2siUEqVWRUzEWSchA0zbNXP9kWAgYbd4dInIfxy8K5ctPJCo2HVV3B0D1T7p9uWiDC8fX1enLWR+APHaFI7wLmfQymlnKDiJAJjYM9Ke8KOnQKnjkD1BtDjMWh9rX1cXKF2oQj2rjkrEQAMbVuPV2ZvYuLyBB6/TCfWUkqVPRUnEaz6GqaPBq/KEHGFrfq56JIzff9LJDgSxAP2roUWA8/aVKtKJXqF12bKygT+r19zvD21o5ZSqmypOImg+WUw6C3b79/XyesK+/hBrWa5NhiDbTSeE7efXzccoH/LEOe+t1JKlVDFuTz1rwntbnZ+EjgtNDrPRNCtaRDBVSvx/bJdrnlvpZQqgYqTCFwtJAqOJcLx8+dL8fL0YFi7eizcnMS+I/mvV6qUUqVNE4GzhEbb3/tyvyu4JqY+2QYmrdCRxkqpskUTgbOEtLK/86geuqimP50a1WDi8gSydfUypVQZ4tJEICL9RWSTiMSLyJhctgeKyI8islZE/haR89Y2LjcqV4fAMNtzKA8j2jdg16FUlm5PLr24lFKqAK5cs9gTeA8YAEQA14pIxDm7PQ6sNsZEATcCb7kqnlKRT4MxQP+WIQT4eulEdEqpMsWVdwQdgHhjzDZjTDrwHTD4nH0igF8BjDEbgTARyWNSn3IgNNpOR512JNfNvt6eXNm6Lj/H7uPIyYxSDk4ppXLnykRQF8h56ZvgeC2nNcBQABHpAFwEnLe0joiMFJHlIrK8TK9idKbBeF2euwxvX59TmdlMX72nlIJSSqn8uTIR5DZb27mtpC8BgSKyGrgXWAVknneQMeONMTHGmJigoCDnR+osIY5EkE/1UMu61YgIrcr3y7V6SClVNrgyESQAOafcrAck5tzBGHPUGHOLMaY1to0gCDh7qa/ypEoQBNTJNxGAvSuI3XOU2D25VyEppVRpcmUiWAY0FZGGIuIDjACm59xBRKo7tgHcDiwyxhx1YUyuV0CDMcCVrevi4+XBRL0rUEqVAS5LBMaYTGA0MAfYAEw0xsSJyCgRGeXYLRyIE5GN2N5F97sqnlITGgUHN0N6ap67VPPzpn9kCFNX7SEtI6sUg1NKqfO5dNI5Y8wsYNY5r43L8fhPoKkrYyh1odFgsmF/HNRvn+duI9rXZ/qaRObE7WNw63Pb0JVSqvToyGJnO91zaO/qfHfr1Kgm9WtU5nsdU6CUcjNNBM5WtS5UrgH78h5hDODhIVzTrj5LtiazKznvaiSllHI1TQTOJlKoBmOAYTH18BD4QSeiU0q5kSYCVwiNhv3rITM9/92qVaZbsyAmrUggSyeiU0q5iSYCVwiNguwMSNpY4K7DY+qz90gai7aU4RHTSqkLmiYCV8i5mH0BeoUHU9PfRyeiU0q5jSYCVwhsCD4BhUoEPl4eDGlTl1/W7+fg8VOlEJxSSp1NE4EreHjYhWoK6Dl02vD29cnMNvy4UieiU0qVPk0ErhIabWchzS545HDT4ADaNqjO98t3Y4w2GiulSpcmAlcJjYaMVEiOL9Tuw9vXJ/7AcVbuSnFxYEopdTZNBK4SGmV/57N0ZU4Do+rg5+OpjcZKqVKnicBVajUHL98Cp5o4rUolLy6PCuWntYmcOHXekgxKKeUymghcxdMLgiML1XPotOHt63MiPYuZa/e6MDCllDqbJgJXCo22PYcK2QDctkEgjYP8dfUypVSp0kTgSiFRdiH7lJ2F2l1EGN6+Pit2Hib+wDEXB6eUUpYmAlcKLXgN43MNbVsPLw/R6amVUqXGpYlARPqLyCYRiReRMblsryYiM0RkjYjEicgtroyn1NWOAA+vIiWCWlUq0Ts8mCkr95Ceme3C4JRSynJZIhART+A97BKUEcC1IhJxzm73AOuNMdFAD+D1HGsYl3/evhDUotBdSE8b3r4+ySfS+W3jfhcFppRS/3DlHUEHIN4Ys80Ykw58Bww+Zx8DBIiIAFWAQ8CF1XcyNNp2IS3CiOFuzYIIrlqJyTrlhFKqFLgyEdQFclZ0Jzhey+ld7AL2icA64H5jzHn1ISIyUkSWi8jypKRyNl1zaDScSIJj+wp9iKeHMKBlKIs2J5GafmHlRaVU2ePKRCC5vHbuZXE/YDVQB2gNvCsiVc87yJjxxpgYY0xMUFCQ8yN1pRDHCONCTkB3Wt/IYE5lZrNoczlLfEqpcseViSABqJ/jeT3slX9OtwBTjBUPbAdauDCm0hfSEpAiNRgDdAirQaCfN7NjC38noZRSxeHKRLAMaCoiDR0NwCOA6efsswvoBSAiwUBzYJsLYyp9lQKgZpMiJwIvTw96hwfz68YD2ntIKeVSLksExphMYDQwB9gATDTGxInIKBEZ5djtv8DFIrIO+BV41Bhz0FUxuU1oVJF7DgH0iwzhWFomS7cluyAopZSyvFxZuDFmFjDrnNfG5XicCPR1ZQxlQmg0xE6G1EPgV6PQh13StBZ+Pp7MidtHt2blrG1EKVVu6Mji0lCMEcYAvt6e9GgexC/r95OdrQvWKKVcQxNBaShmzyGw1UMHjp1i1W5dsEYp5RqaCEqDXw2o1qDIdwQAPVvUxttTmBOnvYeUUq6hiaC0hEYVKxFU9fXm4sa1mBO3T9czVkq5hCaC0hIaDclb4VTRp5fuFxnCzuRUNu3XqamVUs6niaC0hEYDBvbFFvnQPhHBiMCcWJ2ETqkKK2kTZLlmyhlNBKWlmD2HAIICKtGuQaC2EyhVUR2Mh0/6wpzHXFK8JoLSEhACVYKLlQjAVg+t33uU3YdSnRyYUqpMO3kYJgwHD0/ofI9L3kITQWkKiSpWF1KwiQDQuwKlKpKsDJh4ExzeCcO/gcAwl7yNJoLSFBoNBzZARlqRD21Q04/w0KqaCJSqSGaPge0LYdBbcFFnl72NJoLSFBoNJgsOxBXr8H6RwSzfeZikY6ecHJhSqsz5+yNY9jFcfB+0ud6lb6WJoDSFOkYYF2MCOrDVQ8bAvA3ae0ipC1r8r/Dzo9BsAPR+xuVvp4mgNFW/CHyrFbvBuEVIAA1q+Gn1kFIXsqTN8MMtdr3zqz6yjcQupomgNIk41jAuXiIQEfpFBrMkPpljaRlODk4p5Xaph+Dba8DLB677zq5nUgo0EZS2kCjYH2d7AxRDv8gQ0rOymb9Jl7BU6oKSmQ4Tb4Sje2wPoeoNSu2tNRGUttDWkHUKDm4u1uFtGwQSFFCJObqEpVIXDmNg1iOw43e44h1o0LFU375QiUBE7heRqmJ9IiIrReTCX1DGFUowwhjAw0PoExHMgk0HSMvIcmJgSim3+WscrPwCLnkIokeU+tsX9o7gVmPMUexqYkHYRedfKuggEekvIptEJF5ExuSy/f9EZLXjJ1ZEskSk8Et4lUc1G4O3X7F7DoGtHjqRnsUf8Rfeqp5KVThbfoE5j0OLy+HSJ90SQmETgTh+XwZ8ZoxZk+O13A8Q8QTeAwYAEcC1IhKRcx9jzKvGmNbGmNbAY8BCY8yhonyAcsfDE0JaFfuOAKBzo5oE+Hpp7yGlyrsDG2wPoeBIGPIheLintr6w77pCROZiE8EcEQkAsgs4pgMQb4zZZoxJB74DBuez/7XAhELGU76FRtupJrIL+gpz5+PlwaUtajNvwwEys4pXhlLKzU4kw7fDwbsyXPsdVKritlAKmwhuA8YA7Y0xqYA3tnooP3WB3TmeJzheO4+I+AH9gcl5bB8pIstFZHlS0gXQWyYkCtKPw+HtxS6iX2QIh06ks3znYScGppQqFZnp8P0NcGwfXDsBqtVzaziFTQSdgU3GmBQRuQF4AjhSwDG5VR3ltcTWIOCPvKqFjDHjjTExxpiYoKCgQoZchp1pMF5d7CK6NwvCx8tDq4eUKm+MgZ8ehF1L4Mr3oV6MuyMqdCL4AEgVkWjg38BO4MsCjkkA6ud4Xg9IzGPfEVSUaiGwIwY9fUrUTuBfyYtuTYOYG7dfl7BUqjz5811Y/TV0+ze0GubuaIDCJ4JMY882g4G3jDFvAQUNeVsGNBWRhiLigz3ZTz93JxGpBnQHphU+7HLOywdqh5eo5xDYSej2pJwkds9RJwWmlHKpTbNh7pMQMRh6uGaRmeIobCI4JiKPAf8CZjp6BHnnd4AxJhMYDcwBNgATjTFxIjJKREbl2HUIMNcYc6Lo4Zdjp6eaKMHVfO/wYDw9RKuHlCoP9q+HybfZ//tXjnNbD6HcFDaS4cAp7HiCfdhG31cLOsgYM8sY08wY09gY84LjtXHGmHE59vncGFP6IyjcLTQaTh6CIwnFLiLQ34cOYTU0EShV1h1PsquM+VSxjcM+fu6O6CyFSgSOk/83QDURuRxIM8YU1Eag8hPa2v4uQTsB2OqhLQeOsy3puBOCUko5XeYp20Po+AGbBKrWcXdE5ynsFBPXAH8DVwPXAH+JSNlo5SivakeAeBR76crT+p5ZwlLXKFCqzDEGZtwPu5fClR9A3bbujihXha0a+g92DMFNxpgbsYPF3DMW+kLh4we1mpf4jqBO9cpE16vGbK0eUqrs+eMtWDPBNgy3HOruaPJU2ETgYYw5kON5chGOVXkpwdoEOfWNDGHN7q1DZOcAACAASURBVBT2HSn6WshKKRfZOBPmPQORQ6H7o+6OJl+FPZnPFpE5InKziNwMzARmuS6sCiI0Co7ttXWHJdDPUT00d73eFShVJuxbB5PvgDpt7KAxyXdqNrcrbGPx/wHjgSggGhhvjCnbKa48ODPCuGTtBE1qV6FxkL/2HlKqLDi2H74dYZelHfGtnUuojPMq7I7GmMnkMReQKqaQVvb33tXQtHeJiuoXGcKHi7aRkppOdT8fJwSnlCqW6fdCajLcOhuqhro7mkLJ945ARI6JyNFcfo6JiA5nLSnfahDYsMQ9h8Amgqxsw68bSlbNpJQqgb1rYMsc6PYI1Gnt7mgKLd9EYIwJMMZUzeUnwBhTtbSCvKA5qcE4ql41Qqv5au8hpdxp8RtQqSq0v93dkRSJ9vxxt9BoOLwDTqaUqBgRoV9kCIs2J5Ganumc2JRShXcwHuKmQvvboHJ1d0dTJJoI3C00yv7et67ERfWNDOZUZjaLNl8AazYoVd788SZ4VYJOd7s7kiLTROBuISVbzD6nDmE1CPTz1lHGSpW2I3tgzXfQ5gaoUtvd0RSZJgJ3qxIEVes6JRF4eXrQKzyYXzfsJ0OXsFSq9Pz5HphsuPg+d0dSLJoIyoKQKKf0HALbe+hoWiZLtyU7pTylVAFSD8GKz+0iM4EXuTuaYtFEUBaERsPBzZBe8iUZujathZ+Ppw4uU6q0/PUhZJyASx50dyTFpomgLAiNtreV++NKXJSvtyc9mtslLLOzdQlLpVzq1HH4axw0v8yuOlhOuTQRiEh/EdkkIvEiMiaPfXqIyGoRiRORha6Mp8wKdV6DMdjqoQPHTrFqd8m6pCqlCrDic0hLgUsecnckJeKyROBYzvI9YAAQAVwrIhHn7FMdeB+4whgTiV3voOKpWgf8ajotEfRsURtvT2GuVg8p5TqZp+xC9GFdoX57d0dTIq68I+gAxBtjthlj0oHvgMHn7HMdMMUYswvgnKmuKw4Rp40wBqjq603nxrWYE7cPU4I1kZVS+VjznZ09uBy3DZzmykRQF9id43mC47WcmgGBIrJARFaIyI25FSQiI0VkuYgsT0q6QAdLhUbDgQ2Qme6U4vpFBrMjOZXN+3UJS6WcLjvLLjoTGg2NL3V3NCXmykSQ2wTc516eegHtgIFAP+BJEWl23kHGjDfGxBhjYoKCgpwfaVkQEgXZGZC0wSnF9YkIRgTtPaSUK6yfBoe22raBMr7WQGG4MhEkAPVzPK8HJOayz2xjzAljzEFgEXa9g4rHyQ3GtQN8adcgkNmxmgiUcipjYPFYqNkEwge5OxqncGUiWAY0FZGGIuIDjACmn7PPNKCriHiJiB/QEXDOJXF5E9jQzlropEQAtvfQ+r1H2X0o1WllKlXhxf9q5wbr8gB4eLo7GqdwWSIwxmQCo4E52JP7RGNMnIiMEpFRjn02ALOBtcDfwMfGmFhXxVSmeXjYhWpKuFpZTqeXsNTqIaWcaPFYOy1M1HB3R+I0hV6hrDiMMbM4Z21jY8y4c56/CrzqyjjKjdBoWP6ZbYhywpVGg5p+tAgJYG7cfm7v2sgJASpVwe36C3b+Af3+B14XzkqAOrK4LAmNhsyTcHCL04rsFxnCsp2HOHj8lNPKVKrCWjwWKteAdje5OxKn0kRQloScXpvAudVDxsC89To1tVIlsj8ONs+GjqPAx9/d0TiVJoKypFYz8PJ1aoNxeGgA9WtU1nYCpUpq8RvgUwU63OHuSJxOE0FZ4ukFwS2dmghEhP6RIfwRn8yxtAynlatUhXJoO8ROhnY3g18Nd0fjdJoIyprQKNtzyIlTQ/SLDCE9K5v5my7QUdlKudqSt8HDCzqPdnckLqGJoKwJjYZTR+yC9k7StkEgtapU0uohpYrj2H5Y9Q1EXwtVQ90djUtoIihrnDzCGMDDQ+gTEcyCjQdIy8hyWrlKVQhL37PTv3S5392RuIwmgrKmdoS9BXVizyGwk9CdSM9iydaDTi1XqQvayRRY9ilEXAk1G7s7GpfRRFDWeFWCoHCn3hEAXNy4FgGVvJgTq91IlSq0ZR9B+rELYqrp/GgiKItCoyFxtVMbjH28PLg0vDa/bNhPZla208pV6oKVngpLP4AmfWwnjguYJoKyKDQaUg/aRS+cqF9kCIdOpPP3jkNOLVepC9KqryA1GbqW72UoC0MTQVl0+urDiRPQAfRoHkQNfx/e/GWLrlymVH6yMmDJO1C/E1x0sbujcTlNBGVRcEtAnN5O4OfjxSN9m/P3jkP8tNa5dxtKXVDW/QBHdleIuwHQRFA2VaoCIS1h9de214ITDW9fn8g6VfnfrA2cTNeupEqdJzvbTicR3BKa9nV3NKVCE0FZNfANOJoI00c7tdHY00N4elAkiUfS+GDhVqeVq9QFY+NPcHCz7Sl0ASxDWRiaCMqq+u2h19OwYQb8Pd6pRXdoWINB0XX4cOFWEg6X4dXLVn4FP49xaiJUKl+nl6EMDLNjByoITQRlWefR0LQfzH0CElc5tejHBrRABP43a6NTy3WafbHw04Pw1wdOT4RK5WnbAvt/rcv9dhLICsKliUBE+ovIJhGJF5ExuWzvISJHRGS14+cpV8ZT7nh4wJBx4B8EP9wMaUecVnSd6pW5q3sTZq7by59bk51WrlNkZcDUu6ByIDTqCXOfhP3r3R2VqggWj4UqwRB9nbsjKVUuSwQi4gm8BwwAIoBrRSQil11/N8a0dvw856p4yi2/GjDsM0jZDdPvc2o1yZ3dG1G3emWenRFXtgaZ/T7WTrFx+Rsw9CPwrQpT7oCMNHdHlq/sbBdUYaXsslVkmbrCnMslrIDti+yduLevu6MpVa68I+gAxBtjthlj0oHvgMEufL8LV4OO0OtJWD8Vln/itGJ9vT35z8BwNu47xoRlu51WbonsXQuLXoFWV0P45VAlCAa/D/tj4deye50wbfUeop+dy8uzNzonIWRn2VGt73WyHQbGdYXdf5e8XJW3xWPBtzrE3OLuSEqdKxNBXSDn2SXB8dq5OovIGhH5WUQicytIREaKyHIRWZ6UVEHn1L/4fjvUffZjTh1fMKBlCJ0a1eD1uZtISU13WrnFkpkOU++2a8IOeOWf15v1hfZ32Fkgt/7mvvhyYYzhrXlbuP+71QT4evHBgq2M/Gp5yRYB2h8Hn/SB2WMgrAsM/RjST8Anfe3fP/2E8z6Asg5stL2FOoyESgHujqbUuTIR5Nbv6txLpZXARcaYaOAdYGpuBRljxhtjYowxMUFBQU4Os5w43V7gV9PRXnDUKcWK2O6kR09m8MYvm51SZrH9/jrsXweD3jx/Fai+/4VazeHHu+BE2WjTOJWZxUMT1/DGvM1c1bYe8/+vB88NjmT+piSu+mAJu5KL2CMrIw1+ex4+7AaHd8JVn8B1EyHqarhnKbS/DZa+Dx9cDNsWuuZDVVR/vAnefnY94grIlYkgAaif43k9IDHnDsaYo8aY447HswBvEanlwpjKN/9aMOxTu2jNTw84rb0gPLQq13e8iK//2sWmfcecUmaR7V0Lv78GUcOhxcDzt3tXhqs+tnO/zHBuW0lxHDqRzg0f/8WPq/bwSN9mvHZ1FJW8PLmxcxhf3tqB/UdPMfi9xYVviN+5BMZdAotetdVio5dBq2H/9GOvFAADX4ebZ4F4wJdX2DYjJ3YgqLBSdtmRxG1vAv+a7o7GLVyZCJYBTUWkoYj4ACOA6Tl3EJEQEfsvXUQ6OOIpG5d7ZdVFF0PP/9j1U1d87rRiH+rTjCqVvHh2Rlzpz0OUmW57CfnVhP4v5b1faBT0esrewq/8svTiO8fWpOMMef8P1iQc4Z1r2zD60qZIjoFHXZrUYto9Xajh78O/PvmLr5fuzLuwtCO2m+xnAyDrFNwwxXHnl8e6uGFd4K4lcPF9dlK09zrBptlO/oQVzJJ3AIGLL8xlKAvDZYnAGJMJjAbmABuAicaYOBEZJSKn77+GAbEisgZ4GxhhdDa0gl3yEDS+1NYh74t1SpGB/j483LcZS7Yml/6SlotetY3Bg94qeGHwzqOhYTf72ZNLf2T0kq0HGfLeHxxPy2TCHZ0YFF0n1/3Cavnz4z1d6Nq0Fk9MjeXJqbFknNsza+NMeK+jTeidR8PdS6FJr4KD8K5sq8punweVq8OE4TD59jJTZVauHE+yFxVRw6FaPXdH4zZS3s67MTExZvny5e4Ow/2OJ9mqhEpVYOQCpzRwZWZlM/DtxZxIz2TeQ93x9fYscZkFSlwNH11qq0OGfli4Y47ssfXkNRrBbXPB09u1MTpMXL6bx6esI6yWP5/d3J76NfwKPCYr2/Dy7I2MX7SNzo1q8v71bQnMPgw//x+sn2bns7nibajbrnhBZabb3i6LXrPdbAe8Ai2vurCmRti2wLYfiae9azzzU8Pxk+O1yjWK1vXz1+dsd+XRy6BWU5d9hLJARFYYY2Jy3aaJoBzb/rutK245DIaOd8p//iXxB7nu4794uE8z7u3l4v8Ymekwvoet979nqR1AVljrp8HEG6HrI7ZrrQtlZxtem7uJ9xdspWvTWrx7XVuqVS5a8pm0IoHHp6zlNv/FPMLXeGalQY9HbRWPMxLZ/jiYNhoSV0Lzy2Dg2PK/0HrqITuqfvU3UK0BBATbfyupyfm3jfhU+SdBVK6RS/JwPK5UBb4YDI26w/CvSu9zuUl+iaDijKG+EDXsCt3HwIIX7eO2N5a4yIub1KJ/ZAjvL9jKsJh6hFar7IRA87DoFTgQB9d+X7QkABAxGFrfYK+Gm/Ry2ZzxaRlZPDRxNbPW7ePaDg14bnAk3p5Fr1EdFnaKvvXfpuq+P1lmIsgY+AYXd+jkvECDI+G2X2yvovkv2Cqnfs9Dm3+Vv7sDY2wb2M+PQloKdH0Yuv2frRI7LSsDTh7+JzGkJtvEcdZvx09yvH0tPY+OEBf4MpSFoXcE5V12Fnw1xA42uuM3CM5t8HbR7D6USq+xCxnQMoS3RrRxQpC5SFwFH/WydbNDPiheGaeO2YFW2Vlw12LwrebUEJOOneL2L5ezNiGFxweEc3vXhmc1ChdKVgb8+S4seAk8K5FyyZP8a1UzYvce59H+LbizW6Oil1mQ5K0w/V7Y+Qc06mHbXgLDnPserpKyG2Y+DFvmQJ22ttospJVzys5Mh5PnJAmfAGja2znll3FaNXShO34APuhir6pHzgcf/xIX+frcTbzzWzyTRnUmJqyABtyiyjwFH3a3V3t3L7UNnsWVsNwOtGp5FVz1kdNC3LTvGLd+voxDJ9J5c0Rr+kWGFL2QxFX2hLxvHYQPggGvQtVQTqZn8cikNcxcu5ehbery4tBWzm+Pyc6GFZ/CL0/bK+xeT9nBUh5ldJ7J7CxY9rGtszfZcOmT0PFO8CiFdqoKIr9EUEb/VagiqVLbngQPboaZjzilyLt6NCa0mi/PzIgjy9lz6Cx8GZI2wKC3S5YEAOrFQPdHYd1EWPuDc8LbnMSwD5aQkZXNxDs7Fz0JpKfauu2PLrWN+td8BcO/PlNnX9nHk3evbcNDfZoxZdUeRoxfyoGjTp5HycMD2t9uE+1FnWH2o/BZf0hy86DB3Oxfb5P5z/+GBp1szJ3v1iRQijQRXCga9YDu/4Y138Kqb0pcnJ+PF2MGtCB2z1F+WO7EeYj2rITFb0Lr6+3UEc7Q9WGo3xFmPmRH5JbA10t3cuvny6hXw4+p93ShVb0iVjdtnQ8fdLZ909veCPf8BRFXnLebiHBfr6aMu6Etm/Yd44p3/2BtgnNXowOgen24fhJcOQ6SNtmeZr+PhaxM579XUWWkwW8vwIdd4fB2O5XG9ZMg8CJ3R1bhaNXQhSQ7C74cDHtWwB3zoXaLEhVnjOGaD/9kW9IJfnukR5F7ypznTJXQEbj7z5LfDeR0eAd8cImtT775pyJfTWZlG16ctYFPFm/n0ha1efvaNlSpVIS+FCeS4ZcnbQ+XGo1t3XbYJYU6dH3iUe74cjkHj5/itauj8xybUGLH9sOsR2DDdAiNtl1N63VwT3XRziV2ZHTyFogaAf1erLCjekuLVg1VFB6edhoGH387H1F6yVYfOz0P0aHUdN7+dUvJ41vwkq0SusIJVULnCgyDga/BriV2vdkiOHEqkzu/WsEni7dz88VhfHRjTOGTQHY2rPoa3o2Btd/bwX53/VHoJAAQUacq00Z3IapeNe6dsIrX5mxyzZTWAcG2m+Q1X8LRvfBpPxjbAqbeA+un28Z3V0s7AjMeOHsk9dAPNQm4md4RXIi2/gZfDYU218Pg90pc3JjJa5m0IoHZD3SlSe1iDlxLWAGf9IbW1zklplwZA5NutVe8t80t1CCtfUfSuO2LZWzYe5SnB0Vy08VhhX+/Axvgp4ds8qnfya6fUIJeW6cys3hyaiwTlyfQNyKYN4a3xr8odyVFkXYENs6yvXPif4NTR8DD23bDbdYPmvWHmo2d+54bZtg2rBMHoNPd0PNxp3RsUIWjvYYqol//aydxGzIeooeXqKiDx0/R87UFtGkQyBe3tC96d8eMNDujZvpxWyXk5G6eZzl52FYReVWCOxfZQUN5iN1zhNu+WMbxtEzeva4tPVvULtx7pKfaMRBL3rEjuvv817Z5OKGKxRjDZ3/s4PmZ62kWHMBHN8YUagTzadnZhmNpmRxOTedQajqHT6Rz6EQ6KakZ5z0PDw3gicsj8CYLdv8Fm+fYn4ObbGE1Gtuk0LQvXNQFvHyK96GO7rUjqTfMgOBWjpHUbYtXlio2TQQVUVYmfDHIrl0wcgEENStRcR//vo3nZ27g4xtj6B0RXLSDf3naTvN7w2RoUgp9trf/bj972xvtSScX89bv577vVlG9sjef3Nye8NCqhSt78xxbz56yyw5o6/OcS6o1Fm5OYvS3K/H29ODVYVEE+vuQkprOoRMZ9mSemu54ns7hExlnnh9Ozcizl5eXhxDo70MNPx/8K3myclcKl7WyY0XOGiR3eAdsnmvvFrb/bqtwfAKgcQ+7hnbTvraaqSDZ2bDyC/v3zzoFPcbYOZVKaUoQdTZNBBXV0UTbS6RKCNzx69kjM4soIyub/m8uIjPbMPfBblTyKmRjbMJyu8hK6+th8LvFfv8iO518hn9t+/Dn8MWSHTwzI45Wdavx8Y0x1K5aiLlpjuyxXTA3zICgFnYKh7AuLgre2pp0nDu+WM62g+cvRJPzpF7dz5sa/j4E+vsQ6OdNoJ9Pjud2n0B/b6pU8jrrbu50ch8YFcpbw1vjlduI6fQTdu2DLXNscjjmmEm+ThubFJr1hdA2598NHdwCM+63g9rCutpBbc6uaqpgMrOy8fSQYg9A1ERQkW2ZB99cZedaz+PquLAWbk7ipk//5tH+LbirRyH+U2ek2a6B6alw9xLXVgmdKzPdtkmk7LbTNjv68H+/bBePTl5H34hg3hrRhso+BSS0rEz4+0OY/6LtldX93/aqtrjVJEV0NC2DRZuT8Pfx+ufE7+9NwDkn9eIav2grL87ayKDoOrxxTXTuyeA0Y+wssaerkBKWAQb8a9u7hGZ97Ul/+Sew8FV74dHvBXsRUN6muShj4g8c4+Ef1nJNTD2u71i87rU611BF1rS3nUtl8Rt2+uZWw4pdVPdmQfQOr827v23hqrZ1C76Snv+CHeR2w5TSTQJgT9RDP7ZtE1Pvghum8MvGJB6bso7uzYJ47/q2Bc8ZtHuZXStg/zp7orvs1VKfqqGqrzeXR7moOykwsltjsg289PNGPATGXtMaT488TtoitntuSCvo9ojtMhs/z94tbJwBq7/+Z9/IoTDgZTvYURVbVrbhk8XbeG3uZvx8PAn0c80FiN4RVARZmfD5QHs1N3Ih1GpS7KJ2HDxB3zcWMSi6Dq9fE533jruXwad97aRnJbwTKZFln8DMh9jV/kn6/BlJi9CqfHt7x/x745w8DPOetesEBITaE1r4oAv6qvb9BfG8MnsTQ9rU5bWro/NOBnnJyrQNztsXQt0Y5w0WrMC2HzzBIz+sYcXOw/QOD+bFoS2pHVCEKbbPoXcEFZ2nFwz7xLYX/HAz3DS94AVg8hBWy59bL2nIuIVbuaFTA9o0yGXW0IyT9iq8al3o+3zJYi+pmFs5HvszIcv+R9eqr/PKzb3yTgLG2CUL5zxuJyTrdDf0fKxCLGZ+d48mGAOvztmEAK8WNRl4etk2Exe3m1QE2dmGz5fs4JU5G/Hx9OCN4dFc2bqu8ycnzMGlA8pEpL+IbBKReBEZk89+7UUkS0SKX2+h8letHgz50FZzvNLIzoPz2/Ow8087Q2YRjL60CUEBlXhmxvrcBz7Nf8GOGL3ibbtYihvtOZLGsMTrOEYVPqj8ATV8snLf8eAWu7bDlDugegPb06r/ixUiCZx2T88mPOyY/+jfk9Y6f44pVaBdyamM+Ggpz/20ns6NavLLQ90Z0qaeS5MAuDARiIgn8B4wAIgArhWR80bbOPZ7GbukpXKlZv1s3/oeY+xqT7+/bicie7khTLjOzv54aFuBxVSp5MWj/VuwZncKU1btOXvjrr9gybvQ7ha7nKYbHT6Rzk2f/s2eDH9SL3sb7+SNMO+Zs3fKSLMNwR9cDIlrbG+g236xUzBUQPf2asoDvZsyeWUCYyavdc0IZ3We7GzDV0t30v+tRWxIPMorw6L49Ob2BBemR5sTuLJqqAMQb4zZBiAi3wGDgfXn7HcvMBlo78JY1Gmh0fanxxg4mQLbF8HWX+3o0k0z7T6BYdC4lz2RN+yaa0Pv0DZ1+WrpTl6evZH+LUPslAwZJ2Ha3VCtvl1T141S0zO59Ytl7DqUyle3dqB+o5pw8E74axw06WMb0eN/tXPfH94Ora6xPVy0cZMHejcj28Dbv27B00N4cUgrPIraZqAKLeFwKo9OXssf8cl0bVqLl66Kom51Fy4IlQtXJoK6QM5pKxOAjjl3EJG6wBDgUvJJBCIyEhgJ0KBBA6cHWmFVrm5nxoy4wtaPH9pmT45bf7Pz5iz/xN451O9gk0LjS23/cQ9PPDyEZwZFMOT9Jbzz2xYeGxBuq5qS4+HGaW6tUsnIymb0t6tYszuFD25oR8dGjgFffZ61iW/qXbYuO+5HqNnExtuoh9viLYse7N0UYwzv/BaPCLxwpSYDZzPG8P2y3Tw/cwPZxvDCkJZc16GBy6uBcuPKRJDbpzn3PvNN4FFjTFZ+H94YMx4YD7bXkNMiVP8QsQN+ajaGjiNtP/yEv21S2PqbrT6Z/wL4VrcnzcaX0qZJL65qW49PF2/npnr7qPPnexBzq1tPqsYYxkxex28bD/DikFZnryXgXdmu2/DRpXaenZ7/gS732+ko1FlEhIf6NCPbGN6bvxUR4fnBLTUZOMneIycZM3kdCzcn0blRTV4ZFlWkqUSczZWJIAGon+N5PSDxnH1igO8cSaAWcJmIZBpjprowLlUYXj52Bs2wS+zqVieSYdt8O9/+1l9hvf0TvVyjKa29muA5bS2mWj2kz3NuDfvl2ZuYvDKBh/o047qOudw9hrSybQCVq5ef5RvdRER4pG9zsg18sGArHgL/HdzSLVesFwpjDFNW7uGZGXFkZhmevSKSf3W6yO0J1pWJYBnQVEQaAnuAEcB1OXcwxjQ8/VhEPgd+0iRQRvnXtIPRWg2z1UhJG2Hrb3jF/8q1Kb/hkZHOi94vccNxDy5y0wX2J4u3M27hVv7V6SLuvTSfsRJ1WpdeUOWciPDvfs3JzjZ8uGgbniI8c0WkJoNiOHAsjcenrGPehgO0Dwvk1WHRhNUqG7OvuiwRGGMyRWQ0tjeQJ/CpMSZOREY5to9z1XsrFxOB2uH2p/M9eKSfZOofq5mw4ChfvrGIB3o34/auDQseuetE01bv4b8/rWdAyxA9UTmZiDBmQAuyjeGj37c71qmI0O+4kIwxTF+TyNPT4ziZnsUTA8O5pUvDog/acyEdWaycZt+RNJ6aFsvc9fsJD63Ky1e1IqqekxegycWizUnc9sUy2l0UyOe3dHD+QvAKsCe052faVdxu6RLGU5drMijIweOneHJqLD/H7qNNg+q8dnU0jYPynhrdlXRksSoVIdV8GX9jDLNj9/HUtFiufO8PbunSkIf6NHPZAitrdqcw6usVNKkdwPgbYzQJuJCI8MTAcLIdayZ4ivCfgeGaDPIwa91enpgay/G0TMYMaMEdXRuVqbuAnDQRKKfr3zKEi5vU5JXZG/lk8XZmx+7j+SEt6dncuX30tyUd55bPl1Gzig9f3NKeqr46z72riQhPXR6BMfDx4u14eAiPDWihySCHwyfSeWp6HDPWJBJVrxqvXx1N0+CyPUJdE4Fyiaq+3jx/ZSuubF2XMVPWcctnyxgUXYenLo8gKKDkrckHjqZx46d/I8CXt3Ys3JoCyilOtxFkZRvGL9qGCIzpr8nAGMNPa/fy7Iw4jpzM4JG+zRjVvXH+U3uXEZoIlEvFhNVg5n2XMG7BNt6bH8+izUn8Z2A4V7cr/vwpR9MyuOmzZRw6kc53IzvRsIz0vKhIRITnBkdiMHy4cBsejt5FFTUZ7D1ykienxjJvwwGi61Xjq9s6Fn7VuzJAE4FyuUpentzfuykDo0J4bMo6/j1pLT+u3MOLQ1sV+SSelpHFyC+XE3/gGJ/e3L5UGqNV7kSE565oeWacgacID/dtVqGSQXa24Zu/d/HyzxvJyjZlskdQYWgiUKWmSe0Avh/Zme+W7eZ/P2+g35uLuL9XU0Z2a1SorqZZ2YYHv1/N0m2HeGtEa7o2DSqFqFV+PDzsiGNjDO/Oj8fDw45IrgjiDxznsSlrWbbjMJc0qcX/hrZy6+jgktBEoEqVh4dwXccG9A6vzTMz4nh1ziZmrEnkf0Nb5b62gYMxhqem2W54T10eweDWdUsxapUfDw/hhStbkZ1tJ6rbm3KS/+vX/IJtt0nPzObDhVt557d4Kvt48uqwKIaVoKqzLNBxBMqtflm/nyenxrL/WBo3dQ7jyQnirQAACq9JREFUkX7N7Uym53hr3hbemLeZUd0bM2ZACzdEqgqSnW14Zc4mPv59Gz5eHtzZrTF3dGuIn8+Fc725ZncKj05ey8Z9xxgYFcozgyKd0vmhNOji9apMO5aWwWtzNvHl0p2EVvXlv1e2pFd48Jnt3/y1k//8GMtVbevx2tVR5frKqyLYcfAEr8zZyKx1+6gdUImH+zZjWLv65a7ePKfU9Exen7uZz/7YTlBAJZ6/shV9IoILPrAM0USgyoUVOw/z2JS1bN5/nIGtQnn6ighW7kzh7m9W0KN5bT78V7tSnbZClcyKnYd4fuYGVu1KoXlwAI8PDKd7s/LXrvP7liQe/3Eduw+d5PqODXh0QItyOWZFE4EqN9Izsxm/aCtv/xaPr5cHaZnZRNapyre3d6Kyj44aLm+MMfwcu4+Xft7IrkOpdG1ai8cGhBNRp+x3rUxJTee/P21g8soEGtXy539DW/2ztkU5pIlAlTvbko7zxNRYjpzM4OvbOhLo7+PukFQJnMrM4uulu3j71y0cTctgWNt6PNy3OSHVyl6DsjGGmev28sz0OFJSM7izeyPuvbRpuZ++RBOBUqpMOJKawXsL4vn8jx14eMAdXRtxZ/fGuXYQcIecA8Na1a3Gy1dFlYu7l8LQRKCUKlN2H0rlFUfX4VpVfHigdzNGtK/vtukYsrMN3/69i5d+3khmdjYP92nOLV3CysX0EIWliUApVSat3p3CizM38PeOQzSpXYXHBrTg0ha1S7Vn2Nak4zw2eR1/7zjExY1r8r+hrbio5oU3bYkmAqVUmWWMYe76/bz880a2HTxB50Y1+c/AcFrWreay90zPzObAsTSmrU7krV+34OvlwRMDI7g6pnwPDMuPJgKlVJmXkZXNhL938ea8LRw6kc6QNnV5pF9z6lavXKRy0jKy2Hckjb1H0th39KT9ffq54/fB46fO7H9ZK7uqXe2Astdw7UxuSwQi0h94C7tU5cfGmJfO2T6Y/2/v/mOtrus4jj9f8eteuHCpAV26/BSc0xgCsrKoVpEblQObNjQlFm2uZSWtljr7sdVWbKXVlhOdGDSYZQTLtZUQFY0t0kQRgUzT1EsY9EMQFeTKuz++H/R47z3nXuEevud7v6/HdnfO+d7v+d7XPTvf7/v745zPG74FnAA6geURsa3WMl0IzAa2w0ePc+sf/s6qbU8CsGzeVD77gWmMahrCkWOdPHso27i/fgOfNviHj/Lci8e7LbO1eQhto5poa21ifOtrt9PHjeSCydWHNhlIcikEkgYBfwMuAjrImtlfERF7KuZpAV6IiJA0E7g7ImqOH+BCYFYO+557iZvufZQND+6jZdhgBDx/rLPbfGNahtLW2lSxoW+mbdRrG/y21qYBNczFqcqrVeU7gMcj4okU4qfAIuDVQhARRyrmHwEU6zyVmdVN++hmbl48i2Xvmcra7U/RNGRQxd58M+Nbmxg3ahjDBhf78/2NoJ6FoB14puJxB/DOrjNJ+hjwHWAc8NGeFiTpauBqgEmTJvV7UDNrXDPaW1lx6cy8Ywxo9fyQbE+X3rvt8UfExnQ66BKy6wXdnxRxe0TMjYi5Y8cWb6wSM7NGVs9C0AFMrHg8AfhntZkj4o/ANElj6pjJzMy6qGchuB84W9JUSUOBy4F7KmeQNF3pQ7uS5gBDgf/UMZOZmXVRt2sEEdEp6XPAvWQfH70zInZL+kz6/UrgUuCTko4DLwGLo2hfbDAzKzh/oczMrARqfXx04IyoZGZmp8SFwMys5FwIzMxKrnDXCCQdBJ46xaePAf7dj3HqrUh5i5QVipW3SFmhWHmLlBVOL+/kiOjxi1iFKwSnQ9Jfql0saURFylukrFCsvEXKCsXKW6SsUL+8PjVkZlZyLgRmZiVXtkJwe94B3qAi5S1SVihW3iJlhWLlLVJWqFPeUl0jMDOz7sp2RGBmZl24EJiZlVxpCoGkBZIelfS4pOvzzlONpImSfi9pr6Tdkq7NO1NfSBok6UFJv8o7Sy2SRktaL+mv6TV+V96ZapH0xfQ+eETSXZIaqsO6pDslHZD0SMW0t0jaLOmxdNsQTYGrZP1uei88LGmjpNF5ZqzUU96K331ZUvTXsP2lKASpf/ItwIeB84ArJJ2Xb6qqOoEvRcS5wIXANQ2ctdK1wN68Q/TBD4HfpGZI59PAmSW1A18A5kbEDLJRfC/PN1U3q4EFXaZdD2yJiLOBLelxI1hN96ybgRkRMZOsx/oNZzpUDavpnhdJE8l6wT/dX3+oFIWAiv7JEfEycLJ/csOJiP0RsSPdf55sQ9Web6raJE0gazN6R95ZapE0CngfsAogIl6OiOfyTdWrwUCzpMHAcGo0d8pDaij13y6TFwFr0v01ZN0Hc9dT1ojYFBGd6eF2sgZaDaHKawvwfeAr9GOP97IUgp76Jzf0xhVA0hRgNvDnfJP06gdkb8wTeQfpxVnAQeDH6TTWHZJG5B2qmojYB3yPbM9vP3AoIjblm6pP3hoR+yHbsSHrR14Ey4Bf5x2iFkkLgX0RsbM/l1uWQtCn/smNRFIL8AtgeUQczjtPNZIuBg5ExAN5Z+mDwcAc4NaImA28QOOctugmnVtfBEwF3gaMkHRVvqkGJkk3kp2WXZd3lmokDQduBL7e38suSyF4Q/2T8yZpCFkRWBcRG/LO04t5wEJJ/yA75fZBSWvzjVRVB9ARESePsNaTFYZG9SHgyYg4GBHHgQ3Au3PO1Bf/kjQeIN0eyDlPTZKWAhcDVzZ4h8RpZDsFO9P6NgHYIantdBdclkLQa//kRpF6OK8C9kbEzXnn6U1E3BAREyJiCtnr+ruIaMi91oh4FnhG0jlp0nxgT46RevM0cKGk4el9MZ8Gvrhd4R5gabq/FPhljllqkrQAuA5YGBEv5p2nlojYFRHjImJKWt86gDnpfX1aSlEI0sWgk/2T9wJ3R8TufFNVNQ9YQrZn/VD6+UjeoQaQzwPrJD0MzAK+nXOeqtKRy3pgB7CLbH1tqCERJN0F/Ak4R1KHpE8DK4CLJD1G9umWFXlmPKlK1h8BI4HNaV1bmWvIClXy1udvNfaRkJmZ1VspjgjMzKw6FwIzs5JzITAzKzkXAjOzknMhMDMrORcCszNI0vsbfYRWKx8XAjOzknMhMOuBpKsk3Ze+ZHRb6rdwRNJNknZI2iJpbJp3lqTtFWPavzlNny7pt5J2pudMS4tvqeiJsC59a9gsNy4EZl1IOhdYDMyLiFnAK8CVwAhgR0TMAbYC30hP+QlwXRrTflfF9HXALRFxPtkYQfvT9NnAcrLeGGeRfZvcLDeD8w5g1oDmAxcA96ed9WaygdNOAD9L86wFNkhqBUZHxNY0fQ3wc0kjgfaI2AgQEUcB0vLui4iO9PghYAqwrf7/llnPXAjMuhOwJiJe161K0te6zFdrfJZap3uOVdx/Ba+HljOfGjLrbgtwmaRx8GoP3slk68tlaZ5PANsi4hDwP0nvTdOXAFtTD4kOSZekZQxL48mbNRzviZh1ERF7JH0V2CTpTcBx4BqyRjZvl/QAcIjsOgJkQy2vTBv6J4BPpelLgNskfTMt4+Nn8N8w6zOPPmrWR5KORERL3jnM+ptPDZmZlZyPCMzMSs5HBGZmJedCYGZWci4EZmYl50JgZlZyLgRmZiX3fzwsJaqqbKaNAAAAAElFTkSuQmCC\n",
      "text/plain": [
       "<Figure size 432x288 with 1 Axes>"
      ]
     },
     "metadata": {
      "needs_background": "light"
     },
     "output_type": "display_data"
    }
   ],
   "source": [
    "from matplotlib import pyplot\n",
    "# plot train and validation loss\n",
    "pyplot.plot(history.history['loss'])\n",
    "pyplot.plot(history.history['val_loss'])\n",
    "pyplot.title('model train vs validation loss')\n",
    "pyplot.ylabel('loss')\n",
    "pyplot.xlabel('epoch')\n",
    "pyplot.legend(['train', 'validation'], loc='upper right')\n",
    "pyplot.show() "
   ]
  },
  {
   "cell_type": "code",
   "execution_count": null,
   "metadata": {},
   "outputs": [
    {
     "name": "stdout",
     "output_type": "stream",
     "text": [
      "INFO:tensorflow:Assets written to: saved_model/model1_50epochs_bit/assets\n"
     ]
    },
    {
     "name": "stderr",
     "output_type": "stream",
     "text": [
      "INFO:tensorflow:Assets written to: saved_model/model1_50epochs_bit/assets\n"
     ]
    }
   ],
   "source": [
    "import tensorflow as tf \n",
    "# Save fine-tuned model as SavedModel\n",
    "export_module_dir = 'saved_model/model1_50epochs_bit'\n",
    "tf.saved_model.save(model, export_module_dir)"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "## Train a BiT-L ResNet R50x30"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 4,
   "metadata": {},
   "outputs": [
    {
     "name": "stdout",
     "output_type": "stream",
     "text": [
      "WARNING:tensorflow:From /opt/conda/lib/python3.7/site-packages/tensorflow_core/python/ops/resource_variable_ops.py:1781: calling BaseResourceVariable.__init__ (from tensorflow.python.ops.resource_variable_ops) with constraint is deprecated and will be removed in a future version.\n",
      "Instructions for updating:\n",
      "If using Keras pass *_constraint arguments to layers.\n"
     ]
    },
    {
     "name": "stderr",
     "output_type": "stream",
     "text": [
      "WARNING:tensorflow:From /opt/conda/lib/python3.7/site-packages/tensorflow_core/python/ops/resource_variable_ops.py:1781: calling BaseResourceVariable.__init__ (from tensorflow.python.ops.resource_variable_ops) with constraint is deprecated and will be removed in a future version.\n",
      "Instructions for updating:\n",
      "If using Keras pass *_constraint arguments to layers.\n"
     ]
    },
    {
     "name": "stdout",
     "output_type": "stream",
     "text": [
      "Model sucessfuly built\n",
      "callbacks and data okay\n",
      "Train on 10 steps, validate on 4 steps\n",
      "WARNING:tensorflow:From /opt/conda/lib/python3.7/site-packages/keras/callbacks/tensorboard_v1.py:200: The name tf.summary.merge_all is deprecated. Please use tf.compat.v1.summary.merge_all instead.\n",
      "\n"
     ]
    },
    {
     "name": "stderr",
     "output_type": "stream",
     "text": [
      "WARNING:tensorflow:From /opt/conda/lib/python3.7/site-packages/keras/callbacks/tensorboard_v1.py:200: The name tf.summary.merge_all is deprecated. Please use tf.compat.v1.summary.merge_all instead.\n",
      "\n"
     ]
    },
    {
     "name": "stdout",
     "output_type": "stream",
     "text": [
      "WARNING:tensorflow:From /opt/conda/lib/python3.7/site-packages/keras/callbacks/tensorboard_v1.py:203: The name tf.summary.FileWriter is deprecated. Please use tf.compat.v1.summary.FileWriter instead.\n",
      "\n"
     ]
    },
    {
     "name": "stderr",
     "output_type": "stream",
     "text": [
      "WARNING:tensorflow:From /opt/conda/lib/python3.7/site-packages/keras/callbacks/tensorboard_v1.py:203: The name tf.summary.FileWriter is deprecated. Please use tf.compat.v1.summary.FileWriter instead.\n",
      "\n"
     ]
    },
    {
     "name": "stdout",
     "output_type": "stream",
     "text": [
      "WARNING:tensorflow:Issue encountered when serializing variables.\n",
      "Type is unsupported, or the types of the items don't match field type in CollectionDef. Note this is a warning and probably safe to ignore.\n",
      "'list' object has no attribute 'name'\n"
     ]
    },
    {
     "name": "stderr",
     "output_type": "stream",
     "text": [
      "WARNING:tensorflow:Issue encountered when serializing variables.\n",
      "Type is unsupported, or the types of the items don't match field type in CollectionDef. Note this is a warning and probably safe to ignore.\n",
      "'list' object has no attribute 'name'\n"
     ]
    },
    {
     "name": "stdout",
     "output_type": "stream",
     "text": [
      "Epoch 1/10\n",
      " 9/10 [==========================>...] - ETA: 1:13 - loss: 0.9638 - acc: 0.7209WARNING:tensorflow:From /opt/conda/lib/python3.7/site-packages/keras/callbacks/tensorboard_v1.py:343: The name tf.Summary is deprecated. Please use tf.compat.v1.Summary instead.\n",
      "\n"
     ]
    },
    {
     "name": "stderr",
     "output_type": "stream",
     "text": [
      "WARNING:tensorflow:From /opt/conda/lib/python3.7/site-packages/keras/callbacks/tensorboard_v1.py:343: The name tf.Summary is deprecated. Please use tf.compat.v1.Summary instead.\n",
      "\n"
     ]
    },
    {
     "name": "stdout",
     "output_type": "stream",
     "text": [
      "10/10 [==============================] - 1075s 107s/step - loss: 0.9496 - acc: 0.7309 - val_loss: 1.1943 - val_acc: 0.7295\n",
      "Epoch 2/10\n",
      "10/10 [==============================] - 1033s 103s/step - loss: 0.9338 - acc: 0.8000 - val_loss: 1.4265 - val_acc: 0.6495\n",
      "Epoch 3/10\n",
      "10/10 [==============================] - 1032s 103s/step - loss: 0.7882 - acc: 0.8092 - val_loss: 0.6230 - val_acc: 0.7724\n",
      "Epoch 4/10\n",
      "10/10 [==============================] - 1031s 103s/step - loss: 0.4280 - acc: 0.8479 - val_loss: 1.0048 - val_acc: 0.6834\n",
      "Epoch 5/10\n",
      "10/10 [==============================] - 1031s 103s/step - loss: 1.1533 - acc: 0.7668 - val_loss: 0.8569 - val_acc: 0.7965\n",
      "Epoch 6/10\n",
      "10/10 [==============================] - 1033s 103s/step - loss: 0.7142 - acc: 0.8430 - val_loss: 0.9698 - val_acc: 0.7406\n",
      "Epoch 7/10\n",
      "10/10 [==============================] - 1029s 103s/step - loss: 0.4672 - acc: 0.8512 - val_loss: 0.5891 - val_acc: 0.8017\n",
      "Epoch 8/10\n",
      "10/10 [==============================] - 1031s 103s/step - loss: 0.5656 - acc: 0.8352 - val_loss: 0.5398 - val_acc: 0.8225\n",
      "Epoch 9/10\n",
      "10/10 [==============================] - 1033s 103s/step - loss: 0.3729 - acc: 0.8846 - val_loss: 0.6636 - val_acc: 0.7536\n",
      "Epoch 10/10\n",
      "10/10 [==============================] - 1031s 103s/step - loss: 0.3861 - acc: 0.8697 - val_loss: 0.5315 - val_acc: 0.8153\n",
      "Training model is finished\n",
      "5/5 [==============================] - 437s 87s/step - loss: 0.5675 - acc: 0.8180\n",
      "Test accuracy: 0.8180488\n",
      "CPU times: user 14h 41min 57s, sys: 1h 14min 41s, total: 15h 56min 39s\n",
      "Wall time: 3h 2min 8s\n"
     ]
    }
   ],
   "source": [
    "%%time\n",
    "history,model = BiT_L_ResNet.train_Bit_L(train,validation,test,model_name=\"r50x3\")"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 5,
   "metadata": {},
   "outputs": [
    {
     "data": {
      "image/png": "iVBORw0KGgoAAAANSUhEUgAAAYIAAAEWCAYAAABrDZDcAAAABHNCSVQICAgIfAhkiAAAAAlwSFlzAAALEgAACxIB0t1+/AAAADh0RVh0U29mdHdhcmUAbWF0cGxvdGxpYiB2ZXJzaW9uMy4yLjEsIGh0dHA6Ly9tYXRwbG90bGliLm9yZy+j8jraAAAgAElEQVR4nOzdd3iVRfbA8e9JJ4UQkgCBJFTphJYmTbCDIquiVBUUsfd1dX+6rq66q2tDxYYFVBRELKCCskoXEor0DqEkhJaEdNLn98fc4AXSc0vKfJ6HJ8m97zvvuQHuuTPzzhlRSmEYhmE0Xi7ODsAwDMNwLpMIDMMwGjmTCAzDMBo5kwgMwzAaOZMIDMMwGjmTCAzDMBo5kwgMmxGRWSLyQhWPPSQil9sxlgkissRe7duTiDwrIrMt34eLSLaIuFZ2bA2vtUNEhtb0/AraXS4iU2zdrmEfbs4OwDDOJyKzgCSl1NM1bUMp9QXwhc2CchKl1BHA1xZtlfV7VUr1sEXbRv1megRGvSMi5gOMYdiQSQSNjGVI5nER2SoiOSLysYi0FJHFIpIlIr+KSIDV8ddZhg/SLd39blbP9RWRPyznfQV4nXeta0Vks+XcNSISUYX4pgITgL9ZhkR+sIr7CRHZCuSIiJuIPCkiByzX3yki11u1M0lEVlv9rETkbhHZJyKnReQdEZEyrt9aRM6ISPPzXmeKiLiLSCcRWSEiGZbHvirndfwsIvef99gWEbnB8v2bIpIoIpkislFEBpfTTjtL7G6Wn9tbrp8lIv8Dgs47/msROW6Jb6WI9KjC7/Vyy/eeIjJNRJItf6aJiKfluaEikiQij4nISRE5JiKTy/5bvOA1uIjI0yJy2HLuZyLib3nOS0Rmi0iq5d/JehFpaXlukogkWF7rQRGZUJXrGTWglDJ/GtEf4BAQB7QE2gAngT+AvoAnsBT4p+XYzkAOcAXgDvwN2A94WP4cBh6xPDcaKAResJzbz9J2DOAK3Ga5tqdVHJeXE+Os0nbOi3szEAY0sTx2E9Aa/YFmjCXWEMtzk4DVVucr4EegGRAOnAKuLuf6S4E7rX5+BXjf8v0c4CnLNb2AQeW0cSvwu9XP3YF0q9c/EQhED88+BhwHvCzPPQvMtnzfzhK7m+XntcDrlr+rIUBW6bGW528H/CzPTwM2V+H3ernl+39Z/m20AIKBNcDzlueGAkWWY9yBEUAuEFDO618OTLGKaT/QAT3M9S3wueW5u4AfAG/Lv5P+QFPAB8gEuliOCwF6OPv/T0P9Y3oEjdPbSqkTSqmjwCogXim1SSmVD3yHTgqg31x/Ukr9TylVCLwKNAEGALHoN4RpSqlCpdR8YL3VNe4EPlBKxSulipVSnwL5lvNq6i2lVKJS6gyAUuprpVSyUqpEKfUVsA+IruD8l5RS6UqPuy8D+pRz3JfAOABLr2Gs5THQya4t0FoplaeUWl12E3wH9BGRtpafJwDfWn7HKKVmK6VSlVJFSqnX0G/cXSp68SISDkQB/1BK5SulVqLfRM9SSn2ilMqyXOdZoHfpp+8qmAD8Syl1Uil1CngOuMXq+ULL84VKqUVAdmUxW7X7ulIqQSmVDfwdGGvp5RSiE2Iny7+TjUqpTMt5JUBPEWmilDqmlNpRxddhVJNJBI3TCavvz5Txc+nkZGv0p34AlFIlQCK6J9EaOKqUsq5aeNjq+7bAY5bufrqIpKM/zbeuRdyJ1j+IyK1WQ0/pQE/OGyo5z3Gr73MpfxJ2PnCxiLRGf+pW6IQJulckwDrLkNntZTWglMoCfkInESxfz05eW4ZYdlmGcNIB/0piB/27O62UyrF67OzvXERcReQly3BZJvrTPlVo17p967/Dw5z795WqlCqy+rmi32Fl7bqhe6WfA78Acy3DUf8VEXfLaxwD3A0cE5GfRKRrFV+HUU0mERgVSUa/oQNnPx2HAUeBY0Cb88bZw62+TwReVEo1s/rjrZSaU4XrllcS9+zjlk/aHwL3A4FKqWbAdvSbdK0opdKBJcDNwHhgTmnCU0odV0rdqZRqjR7WeFdEOpXT1BxgnIhcjO5JLbPEPhh4wtJ+gCX2jCrEfgwIEBEfq8esf+fjgVHA5ejE0s7yeGm7lZUaPufv29J2ciXnVEVZ7RYBJyy9i+eUUt3RPc1r0cNqKKV+UUpdgR4W2o3++zbswCQCoyLzgGtE5DIRcUePZeejx47Xov8zP2iZuL2Bc4dlPgTuFpEY0XxE5BoR8avCdU+gx5Mr4oN+YzsFYJm47FmdF1eJL9FvSDfy57AQInKTiIRafjxtiaG4nDYWod8A/wV8ZelRgR7DL7LE7iYiz6DHxSuklDoMbACeExEPERkEjLQ6xA/995OKHnP/93lNVPZ7nQM8LSLBIhIEPAPUeI3Cee0+Ypno9rXE9ZVSqkhEholIL9HrJDLRQ0XFom9guM6S9PLRw1Dl/Z6NWjKJwCiXUmoPelLzbSAF/aYzUilVoJQqAG5AT8qeRnfjv7U6dwN6nmC65fn9lmOr4mOgu2XI5/tyYtsJvIZOSCeAXsDv1XuFFVoIXIT+1LrF6vEoIF5Esi3HPKSUOlhOjPno38nlWCUT9FDIYmAvepgkj/OGvSowHj0Bnwb8E/jM6rnPLO0dBXaiJ36tVfZ7fQGdaLYC29A3EVRpgWAlPkEPAa0EDqJf7wOW51qhh+IygV3ACnTycUF/8EhGv9ZLgHttEItRBjl3iNcwDMNobEyPwDAMo5EzicAwDKORM4nAMAyjkTOJwDAMo5Grd8W7goKCVLt27ZwdhmEYRr2ycePGFKVUcFnP1btE0K5dOzZs2ODsMAzDMOoVETlc3nNmaMgwDKORM4nAMAyjkTOJwDAMo5Grd3MEhmE0LIWFhSQlJZGXl+fsUBoELy8vQkNDcXd3r/I5JhEYhuFUSUlJ+Pn50a5dO+TCTeOMalBKkZqaSlJSEu3bt6/yeWZoyDAMp8rLyyMwMNAkARsQEQIDA6vduzKJwDAMpzNJwHZq8rs0icDR0g7C9m+cHYVhGMZZJhE40pl0mH0DzL8dctOcHY1hGEB6ejrvvvtutc8bMWIE6enpdojI8UwicJSSEvj+XkhL0D8f3ejceAzDAMpPBMXFFW+ItmjRIpo1a2avsBzKbolARD4RkZMisr2S46JEpFhERtsrljrh92mw5ye49B8gLpAY7+yIDMMAnnzySQ4cOECfPn2Iiopi2LBhjB8/nl69egHwl7/8hf79+9OjRw9mzJhx9rx27dqRkpLCoUOH6NatG3feeSc9evTgyiuv5MyZM856OTViz9tHZ6G3KfysvAMs+5S+jN66r+FKWAFLn4ceN8Dgx2DnAkhc5+yoDKPOee6HHexMzrRpm91bN+WfI3uU+/xLL73E9u3b2bx5M8uXL+eaa65h+/btZ2+//OSTT2jevDlnzpwhKiqKG2+8kcDAwHPa2LdvH3PmzOHDDz/k5ptv5ptvvmHixIk2fR32ZLcegVJqJXqv0Yo8AHwDnLRXHE6XcVTPCQR1huveBhEIi9ZDQyVmL27DqGuio6PPuQf/rbfeonfv3sTGxpKYmMi+ffsuOKd9+/b06dMHgP79+3Po0CFHhWsTTltQJiJtgOuBS9Ebgld07FRgKkB4eLj9g7OVogL4+jYoyoObPwdPX/14aDSs/whO7oRWvZwbo2HUIRV9cncUHx+fs98vX76cX3/9lbVr1+Lt7c3QoUPLvEff09Pz7Peurq71bmjImZPF04AnlFKVfixWSs1QSkUqpSKDg8ssp103LXkKktbDqHcguPOfj4dF669mnsAwnM7Pz4+srKwyn8vIyCAgIABvb292795NXFycg6NzDGeWmIgE5loWPwQBI0SkSCn1vRNjsp2t82DdDLj4fujxl3OfC2gHPsGQuB6ipjglPMMwtMDAQAYOHEjPnj1p0qQJLVu2PPvc1Vdfzfvvv09ERARdunQhNjbWiZHaj9MSgVLq7CCciMwCfmwwSeDETvjhIQgfAJc/e+HzIhAWA0lmwtgw6oIvv/yyzMc9PT1ZvHhxmc+VzgMEBQWxffufN0f+9a9/tXl89mbP20fnAGuBLiKSJCJ3iMjdInK3va5ZJ+RlwrxbwNMPbpoJruVUAAyN0msKsk85Nj7DMIzz2K1HoJQaV41jJ9krDodSChbcq8tITPoR/FqVf2xYjP6atB66jnBMfIZhGGUwK4ttac1bsOsHuOJf0HZAxce27gMubmbC2DAMpzOJwFYOroJfn4Xuo+Di+yo/3r0JhPTWPQLDMAwnMonAFjKPwfzJENhJ3ypa1TKwodFw9A8oLrRvfIZhGBUwiaC2igvh60lQkAtjZutJ4qoKi4KiM3B8m93CMwzDqIxJBLW15B+QGAej3obgLtU713rC2DCMesHXV1cISE5OZvTosmtlDh06lA0bNlTYzrRp08jNzT37szPLWptEUBvb5kP8exBzD/S8sfrn+4eCX2szYWwY9VDr1q2ZP39+jc8/PxE4s6y1SQQ1dXI3LHwQwmLhyudr3k5YtF5hbBiGUzzxxBPn7Efw7LPP8txzz3HZZZfRr18/evXqxYIFCy4479ChQ/Ts2ROAM2fOMHbsWCIiIhgzZsw5tYbuueceIiMj6dGjB//85z8BXcguOTmZYcOGMWzYMODPstYAr7/+Oj179qRnz55Mmzbt7PXsVe7amSUm6q/8LPhqInj4wE2zyl80VhVh0bDzez3h3DTEZiEaRr20+Enbz5m16gXDXyr36bFjx/Lwww9z7733AjBv3jx+/vlnHnnkEZo2bUpKSgqxsbFcd9115e4H/N577+Ht7c3WrVvZunUr/fr1O/vciy++SPPmzSkuLuayyy5j69atPPjgg7z++ussW7aMoKCgc9rauHEjM2fOJD4+HqUUMTExXHLJJQQEBNit3LXpEVSXUrDgPr0q+KaZtX/zPjtPYMpNGIYz9O3bl5MnT5KcnMyWLVsICAggJCSE//u//yMiIoLLL7+co0ePcuLEiXLbWLly5dk35IiICCIiIs4+N2/ePPr160ffvn3ZsWMHO3furDCe1atXc/311+Pj44Ovry833HADq1atAuxX7tr0CKpr7Tt6Y5krnod2g2rfXqsIcPXUG9V0H1X79gyjPqvgk7s9jR49mvnz53P8+HHGjh3LF198walTp9i4cSPu7u60a9euzPLT1srqLRw8eJBXX32V9evXExAQwKRJkyptRylV7nP2KndtegTVceh3+N8z0G0kDHjANm26eehVxmbHMsNwmrFjxzJ37lzmz5/P6NGjycjIoEWLFri7u7Ns2TIOHz5c4flDhgzhiy++AGD79u1s3boVgMzMTHx8fPD39+fEiRPnFLArr/z1kCFD+P7778nNzSUnJ4fvvvuOwYMH2/DVXsj0CKoq67heNNa8PYx6t+qLxqoiLBriP4CifHDzrPx4wzBsqkePHmRlZdGmTRtCQkKYMGECI0eOJDIykj59+tC1a9cKz7/nnnuYPHkyERER9OnTh+hovedI79696du3Lz169KBDhw4MHDjw7DlTp05l+PDhhISEsGzZsrOP9+vXj0mTJp1tY8qUKfTt29euu55JRd2QuigyMlJVdn+uzRUXwqcj4dgWuHMptOhm2/Z3LtQVS+/435+b1hhGI7Fr1y66dbPx/6lGrqzfqYhsVEpFlnW8GRqqil+fhSNr9Z7Dtk4CYLVjmRkeMgzD8UwiqMyO72DtdIi+C3qVvYqw1vxaQbNws7DMMAynMImgIqf2wIL7dXG4K1+w77XCYnSpiXo2VGcYtlDfhqjrspr8Lk0iKE9+Nnx1C7h56UVjbh72vV5oNGQdg4xE+17HMOoYLy8vUlNTTTKwAaUUqampeHl5Ves8c9dQWZSChQ9A6j645Xvwb2P/a1rPEzQLt//1DKOOCA0NJSkpiVOnzLattuDl5UVoaGi1zjGJoCzx78OOb/XG8x0uccw1W/YEd289PGSvuQjDqIPc3d1p3769s8No1MzQ0PmOxMGSp6HrtTDwYcdd19UNWvczE8aGYTicSQTWsk7AvNv00MxfbLxorCrConXBrYLcyo81DMOwEZMIShUXwfzbIS8Dbv4cvPwdH0NYNJQUQfImx1/bMIxGyySCUr89B4dXw8g3oVVP58QQapkwNpVIDcNwIJMIQJd4WPMWRE2B3mOcF4dPIDTvaFYYG4bhUCYRpOyD7++FNpFw1b+dHY1eWJa4ziwsMwzDYRp3IijIsSwa84CbP60blT/DoiA3BU4fdHYkhmE0Eo03ESil9xxO2QM3fqw3kq8LQk0BOsMwHKvxJoJ1H8L2+TDsKeg4zNnR/KlFN/DwM4nAMAyHaZyJIHEd/PJ/0Hk4DHrU2dGcy8UVQvubRGAYhsPYLRGIyCciclJEtpfz/AQR2Wr5s0ZEetsrlnNkn9KLxvzbwPXvg0sdzIVhMXByB+RfuI2dYRiGrdnzXXAWcHUFzx8ELlFKRQDPAzPsGItWXKS3mzyTpheNNWlm90vWSGg0qBI4utHZkRiG0QjYLREopVYCaRU8v0YpddryYxxg/9naZS/AoVVw7RsQEmH3y9VYqGU3ucT1zo3DMIxGoa6Mi9wBLC7vSRGZKiIbRGRDjUvV7l4Eq9+A/pOhz/gahukgTZpBcFezwtgwDIdweiIQkWHoRPBEeccopWYopSKVUpHBwcE1u1BIBPSfBMNfrtn5jhYWrSeMS0qcHYlhGA2cUxOBiEQAHwGjlFKpdr2Yf6iuI1QXFo1VRWg05KVD6n5nR2IYRgPntEQgIuHAt8AtSqm9zoqjzjq7Y5nZn8AwDPuy5+2jc4C1QBcRSRKRO0TkbhG523LIM0Ag8K6IbBaRDfaKpV4KvAi8mpl5AsMw7M5uW1UqpcZV8vwUYIq9rl/vubhAaJRZWGYYht05fbLYqEBYDJzaDWfSnR2JYRgNmEkEdVlYlP561Iya2duR1FyKS0zpb6NxMomgLmvTH8TFDA/ZWUp2Ppe/voJ3l5k7tIzGySSCuszTD1r0MInAzuIT0igoLmF2/GEKi826DaPxMYmgrguLgqQNUFLs7EgarPiDegnLicx8ftt1wsnRGIbjmURQ14XFQEGWnjQ27CIuIZVBnYJo7e/F7Lgjzg7HMBzOJIK6LtQyYWwWltlFanY+e09kM6BTIONjwlm9P4WDKTnODsswHMokgrqueQfwDjKVSO1k3UFdIDemfSA3R4Xh5iJ8EXfYyVEZhmOZRFDXiehyE2aFsV3EJaTSxN2ViFB/Wvh5cVXPVny9MYm8QjMnYzQeJhHUB2HRuvhcjn3r8jVG8QfTiGwXgLur/q8wMaYtGWcK+XHrMSdHZhiOYxJBfRBqKUCXZIaHbCktp4Ddx7OI7RB49rHYDs3p1MKX2WZ4yGhETCKoD1r3BRc3M2FsY+sst43Gdmh+9jERYUJMOJsT09l+NMNZoRmGQ5lEUB94eEOrXqZHYGNxCWl4ubvQq825e1ff0C+UJu6upldgNBomEdQXodF6M/viImdH0mDEJaQS2bY5Hm7n/jfwb+LOdb1bs2BzMpl5hU6KzjAcxySC+iIsGgpz4cR2Z0fSIJw+Oz/QvMznJ8a25UxhMd9uTHJwZIbheCYR1BdhZsLYltYdsqwfsJoottYr1J/eof7Mjj+CUqYqqdGwmURQX/iHgV+ImTC2kbiEVLzcXYgI9S/3mImxbdl/Mpt4y6Izw2ioTCKoL0TMjmU2FJ+QRv+2AXi6uZZ7zMjerfFv4m4mjY0GzySC+iQsBtIPQ5apkFkb6bkF7DqeSUz7soeFSnm5uzK6fyi/7DjOqax8B0VnGI5nEkF9cnaewPQKamPdwTSU4pyFZOWZEBNOYbFi3oZEB0RmGM5hEkF9EtIbXD3MPEEtxR9Mw9PNhd5h5c8PlOoQ7MvAToF8GX/EbGVpNFgmEdQnbp4Q0sdUIq2luIRU+oVXPD9gbWJMW46mn2HZ7pN2jswwnMMkgvomLBqSN0FRgbMjqZcycgvZeSyzSsNCpS7v3pIWfp7MjjeTxkbDZBJBfRMWDcX5cHyrsyOpl9Yf0vMDMeUsJCuLu6sLY6PDWbH3FIlpuXaMzjCcwySC+qa0Eqm5jbRG4hJS8XBzoU9Ys8oPtjIuOgwXEb6IN1tZGg2PSQT1TdMQ8A83E8Y1FHcwlb5hzfByr9r8QKkQ/yZc3q0F8zYkkl9kNq0xGhaTCOqjsChTaqIGMs4UsjO5evMD1ibGtiUtp4Cftx+3cWSG4VwmEdRHodGQeRQyTEG06thwKI2SKq4fKMvAjkG0C/Q2K42NBsduiUBEPhGRkyJSZrlM0d4Skf0islVE+tkrlgYnzMwT1ET8wTQ8XF3oG169+YFSLi7ChJi2rD90mt3HM20cnWE4jz17BLOAqyt4fjhwkeXPVOA9O8bSsLTqBW5NGkYiSN4Mpx3zCTsuIZU+4dWfH7A2un8oHm4ufBFnJo2NhsNuiUAptRKoqGzjKOAzpcUBzUQkxF7xNCiu7tCmX/0vNZGbBjOHw3sDYe8vdr1UZl4h249m1HhYqFSAjwfXRoTw3aajZOebTYKMhsGZcwRtAOsCLkmWxy4gIlNFZIOIbDh16pRDgqvzQqPg2FYoPOPsSGou/gO92Y5/G/hyDKyeBnaq/b/x0Gk9P9C+6usHyjMxti3Z+UV8v+moDSIzDOdzZiKQMh4r811AKTVDKRWplIoMDg62c1j1RFgMlBTqoZX6KD8b4t+HLtfAncugx1/g13/Ct1PtktziElIt8wMBtW6rb1gzuoc0ZXbcYbNpjdEgODMRJAFhVj+HAslOiqX+CY3SX+vr8NDGWZCXDoMeAQ9vGD0TLn0ats2DmSMg07b/FOISUukd5k8Tj5rPD5QSESbGtmX38Sz+OHLaBtEZhnM5MxEsBG613D0UC2QopY45MZ76xTcYmneonxPGRfmwdjq0G6zXRIDeeGfI4zD2S0jZCzOGQdJGm1wuK6+Q7bVYP1CWUX1a4+vpxmwzaWw0APa8fXQOsBboIiJJInKHiNwtIndbDlkEJAD7gQ+Be+0VS4MVGq0TQX0bntgyF7KO6d7A+bpeA3cs0ZVWZw6HLV/V+nIbDp+muETZNBH4eLpxY782/LT1GGk5pgCgUb/Z866hcUqpEKWUu1IqVCn1sVLqfaXU+5bnlVLqPqVUR6VUL6XUBnvF0mCFRUHOSTh9yNmRVF1JMfz+pt5boeOlZR/TsoeeNwiLhu+mwpJ/6PNqKD4hDXdXoZ8N5gesTYhtS0FxCV/bYtMapWDHd/DZX+rX36fRIJiVxfVZWIz+Wp/KTexcAGkHYNCjejioPD6BcMt3EDUF1rwFc8ZCXkaNLhmXkErv0GY2mR+w1rmlH9Htm/PluiOU1GbTmtOH4cub4etJkLAMVr5isxgNoypMIqjPWnQHD9/6M0+gFKx+AwI7QbeRlR/v6g7XvAbXvA4HlsJHl0PqgWpdMju/iG02WD9QnomxbTmcmsuq/SnVP7m4EH5/C96NhUO/w1X/gcjb9XBYhrk11XCcRpMICopKyCtsYFUjXVyhTf/6U4n0wG96H4WBD+vYqyrqDrh1AeSkwIfDdFKooo2W+YHq7D9QHVf3aEWQr0f16w8lbdQT4v/7B7S/BO6Lh4vv1b8bVaIn0w3DQdycHYCjLN19krtnb8TP041AXw+CfD2tvnoS7OtBoK8ngT4eBPl5EuTjSdMmbkhFwxd1QVg0rHpd35fv6evsaCq26g3waw0RY6p/brtBMHUZzBkPs2+Eq/4NMXdXPLyEHhZycxH6t7Xt/EApDzcXbo4M4/0VBziafoY2zZpUfEJeJix9HtZ9CH4hMGY2dL32z9cR0BZ63aRvrx38Vz1EZhh21mgSQacWPjx+VRdSsvNJyS4gNTufgyk5bDh0mrTcgjJvvHF3FQJ9PAny89BffT0JskoigVY/N/fxwN3VCR2ssBhQxZD8B7Qf4vjrV1XiOji8Wr+Bu3nUrI2AdvqOou/ugp+fhBPb9bCRm2e5p8QlpBIR6o+3h/3+qY+LDue9FQeYu+4Ij13ZpeyDlIJdC2HxE5B1HKKn6nUTXk0vPHbQw7B1rl5wd+lTdovbMEo1okTgR6cWfmU+V1RcwuncQlKy80nNLiA1J59TWfmk5hSQUvo1O5/9J7M5lZ1PQVFJme0083bXSeJsr+LPHkdp7yPI14OWTb1qVfjsHKGR+mviurqdCFa9Dk0CoN9ttWvH0xdu/hxWvAQrXoaUffpTtW+LCw7NyS9iW1IGU4d0qN01KxHW3JthXVowd30iD1520YUfCNITYdHjsHexLhg45gsI7V9+gy266V7Cug9g4IPgWfa/W8OwlUaTCCri5upCsJ8nwX7lf7IspZQiO7+I1OyCs72L0gSSkp1Pak4+KVkF7DqWSUpWPpl5FxYma+btzltj+zKksw3KZTQJgKDOdXvC+MRO/SY49O+2Gb5ycYFh/6ffML+7B2YM1QvRWvc557CNh09TZOP1A+WZGBvO7bM2sGTHCa6JsNROLC7Sb+ZLXwQUXPkCxNwDrlX4bzfoUdj9I2yYqZOBYdhRlRKBiDwEzASygI+AvsCTSqkldoytThIR/Lzc8fNyp12QT6XHFxSVkJqTf07i+GhVApNmruPJ4V25c3CH2s9DhEXD7kV6+KEuzmn8Pg3cffRwiC31uF6vrp4zHj65Gv7yLvS84ezT8QdTcbXj/IC1Szq3IDSgCbPjDutEcPQP+OEhPTl+0VVwzavQLLzqDYb21z28te/o35u7l/2CNxq9qg5q366UygSuBIKBycBLdouqAfFwcyHEvwk92/gztEsLRvcP5dt7BzC8Zwj/XrSbh7/azJmCWt7NFBoNZ9KqfWulQ5w+BNvmQ+Rk8LbDnTshvfUkckhvmD8Zlr4AJXroLi4hjYhQf3w87d/xdXURxseEszUhifRvHoWPLoPsk3DTpzD+q+olgVKDHoXs47DlS9sHbBhWqpoISj9mjgBmKqW2UHb1UKMKvD3cmD6+L49f1YWFW5IZ/f4ajqbXouJm6cKyungb6Zq3QVzg4vvsdw3fFnDbQuh7i16M9dVEcrNOsyUxnZj2jrvrZqL/Nn7zfBz/bZ/o9QD3r9NVVWvaS+swFFr31ThtoaUAACAASURBVCuxi83eB4b9VDURbBSRJehE8IuI+AFlz5gaVSIi3DesEx/fFsmR1Fyue3s18QmpNWssqDN4+de9SqTZJ2HTbOg9Fpq2tu+13Dzhurdh+H9h78+oj66klTpBrJ3WD5wjIwnmTqDpgkkUewVwC89z5or/6r+T2hCBwY/pXtXO720SqmGUpaqJ4A7gSSBKKZULuKOHh4xaurRrS76/fyD+3u5M+Ciez9Yeqn6NexcXXZa6rk0Yx72rK40OfNgx1xOBmLtg4je4ZB9nocfTxMhO+12vpBji3oN3YmD/b3DFv0i+eTGr8zrwwxYbldHucg0EddErsutbcUGj3qhqIrgY2KOUSheRicDTQM0KvxgX6Bjsy/f3DeSSzsE8s2AHT36zjfyias4bhEbDyV01rsdjc3kZsP5j6D4Kgjo59todh/F4s9fJcQugydwbYf1Htr9G8mb48FK9niE8Fu6Lg4EPEdmhBZ1b+jI73kb7MLu46HUFJ7bDvkZ3b4bhIFVNBO8BuSLSG/gbcBj4zG5RNUJNvdz58NZI7h/Wia82JDJuRhwnM/Oq3kBYFKAgqY4UcV3/EeRnwuBHHX7pMwXF/HLch6/7zNQVTn96DH58VNf2qa38bPjlKV3qIjNZb6gzYb5e7Mafm9ZsTcpgS2J67a8HeqWxfxises30Cgy7qGoiKFJ6vGIU8KZS6k3ArHKxMRcX4a9XdeHdCf3YfTyLa99ezaaq7oDVJhKQulGJtPCMHjLpeJm+m8fB/jhymsJiRd/ObWHcXBj4EGz4GD6/HnJqOA8DsGexHgZaOx36T4L71+vbVc+bDL6+bxu8PVyrX3+oPK7uMOBBfTPA4TW2adMwrFQ1EWSJyN+BW4CfRMQVPU9g2MGIXiF8e+8APN1dGPNBHPOqUu/eq6muRloX5gk2zYacU07pDQDEJ6TiIhDZNkAXt7viX3D9DP27+XAonNhRvQYzk+GriboUtldTuH0JXPsGNGlW5uF+Xu6M6tOGH7Ymk5Frg14IQN+J4B0Eq1+3TXuGYaWqiWAMkI9eT3AcaAOYoul21LVVUxbeN4io9gH8bf5Wnl24g8LiSm7UCovWQ0MlTryhq7hI7x8QGg1tBzolhLiENHq18cfPy+qzSu8xMHkxFBXAR1fArh8rb6ikGOJnwPRo2Pc/uOyfcNdKCI+p9NSJseHkFZYw/4+kWrwSKx7eEHsP7P8Vjm2xTZuGYVGlRGB58/8C8BeRa4E8pZSZI7CzAB8PPp0czZRB7Zm15hC3fBxPanZ++SeERUN+BqTscVyQ59v+DaQf0dtQOmGVc15hMZsT04kpq6xEaH+YuhyCu8BXE2DFK+WPuR/bCh9fAYsf1/Mv98bpHo5r1TrCPVr70y+8GV/EH67+XWDliZoCHn76DiLDsKEqJQIRuRlYB9wE3AzEi8hoewZmaG6uLjx9bXdev7k3fxxJ57rpv7MjuZw7g5y9sKykRL9JBXeDzlc7JYQ/jpymoLik/PUDTUNg8iJdCnvZC3o1ckHun88X5MCSp3X9ovQjcOPHMPFbaN6+2rFMjG1Lwqkc1h6oxbyEtSbN9N4MO76HlP22adMwqPrQ0FPoNQS3KaVuBaKBf9gvLON8N/QLZf7dF1OiFDe+t4aFZd2n3rwDeAdCopMmjPf+DKd26d6Ai3P2PIpLSNPzA+0qWEjm3gSu/0DPHez4Hj65Si8K27sE3onVq6H7TtSTwb1G17hnM6JXCAHe7ra7lRT0Cm03T12/yTBspKr/W12UUietfk6txrmGjUSENmPh/YPo1cafB+ds4j+Ld1FsvVeuiB6bd0aPQCk9kdksHHre6PjrW8QlpNKjtT9NvSoZwhHRdxONn6dX7k6Pgi9v0mPxk3+G697SlV1rwcvdlZsiw1iy40T1bgWuiG8LnaS2zDXbWRo2U9U3859F5BcRmSQik4CfgEX2C8soT7CfJ19MiWVCTDgfrEhg8qz1596ZEhYFqfsgN82xgR3+Xd+6OuDBqpVZtoPS+YFqlZXofCVM+VVv+Xnp03DXKmh7sc1iGh8dTlGJYu76Ktz5VVUDHrRsZ/mO7do0GrWqThY/DswAIoDewAyl1BP2DMwon4ebCy9e34t/X9+LtQdSGPXOavaeyNJPhkbrr45eWLbqdfAJ1p9WnWTTkXQKikqqv/9AcBeY9CMMebzmu6eVo12QD4MvCmLOuiMUVXbXV1UFtNVDVhtn1m5dhGFYVHl4Ryn1jVLqUaXUI0qp7+wZlFE142PCmXNnLNn5xVz/zu/8suM4tOkH4urY4aHkzXpj+th79fi7k8QfTEUqmx9wgomxbTmWkcdvu09WfnBVDXoECnP1xjeGUUsVJgIRyRKRzDL+ZIlIpqOCNMoX2a45PzwwkE4tfLnr8428seIoqlVPx1YiXf0GeDbVd7Q4kZ4faIp/k7q11vGyri0I8fey3Upj0LuzdbkG4j+A/CzbtWs0ShUmAqWUn1KqaRl//JRSZey6bThDiH8TvrrrYm7o14Y3f9vHspz2qKSNjqlhn7Ifdi7QSaC2ZZdrIa+wmD+OOHb/gapyc3VhbFQ4q/alcCglx3YND34U8tJh4yzbtWk0SubOnwbCy92V127qzTPXdmdhWihSmEPy3o32v/Dv0/TtjLH32v9aFdiSWMP5AQcZGx2Gq4vw5bojtms0NBLaDYY103W5b8OoIZMIGhAR4fZB7Zk4+iYAZn71Fcv32HBc+nwZR/VtjH0n6tsanSguIQ0RiK5j8wOlWjb14qoeLfl6QyJ5hbXcmtTaYMt2lpvNdpZGzZlE0ABF9u5NsXcLYt0PcPus9by/4oDtyhxYi3tX38Y44EHbt13dUBJS6daqKf7edWt+wNrEmLaczi1k0bZjtmu0wzCznaVRa3ZNBCJytYjsEZH9IvJkGc/7i8gPIrJFRHaIiNn1zBZEcA2PZpjvYYb3DOGlxbt5aO5mzhTY8JNobhpsmKlvYwxoa7t2ayC/qJg/jpyus8NCpS7uGEiHYB/bThqL6E3uTx8021kaNWa3RGApVf0OMBzoDowTke7nHXYfsFMp1RsYCrwmIra9kbuxCovB5fRBpo8K5W9Xd+GHrcnc+N4akk7nVn5uVaybAYU5jtuGsgJbEjPILyohxhH7E9eCiDAhpi1/HElnZ7INb7rreq3et3r1NLNxjVEj9uwRRAP7lVIJSqkCYC56YxtrCvATEQF8gTTA9G9tIUwvLJOk9dw7tBOf3BZF4ulcrpv+e+2LoOVnQ/z70Hk4tDw/tztefIJePxDTvm4nAoDR/ULxcnexbf0hFxedkE9s0+WyDaOa7JkI2gDW6+qTLI9Zmw50A5KBbcBDSqkLll+KyFQR2SAiG06dOmWveBuWkD7g4n52Ydmwri1YcN9AArzdmfhxPLN+P1jzeYM/PoUzp5228cz54g6m0rVVU5p51/3OpL+3OyMjWvP9pqNk5dlo0xrQ21k2DdXbWRpGNdkzEZRVsvH8d56rgM1Aa6APMF1ELlifoJSaoZSKVEpFBgcH2z7ShsjdS28TaVWJtEOwL9/fN5BhXYJ59oedvPjTruq3W5Svb1dsO+hsr8OZCopK2Hj4dL3oDZSaGNuW3IJivttkw6Jxbh4w8EFIjDPbWRrVZs9EkASEWf0civ7kb20y8K3S9gMHga52jKlxCYuB5D/0rlwWfl7uzLglkrFRYXz8+0H2n6zmqtSt8yArGQY/YuNga2ZrUjp5hXV3/UBZeoc1o1cbf2bH2XDTGoC+t+jtLFeZ7SyN6rFnIlgPXCQi7S0TwGOBhecdcwS4DEBEWgJdgAQ7xtS4hEVBUZ4eO7bi4iI8flUXmri78tZv1djgpKRYLyBrFaE3pq8D4hL0fEd96hGA3spy74ls1h86bbtGPbwh9m7Y/z+9w5phVJHdEoFSqgi4H/gF2AXMU0rtEJG7ReRuy2HPAwNEZBvwG/CEUirFXjE1OqWVSMvYqCbQ15NbL27HD1uTq94r2PUDpO7XcwNO2IayLHEJaXRt5UeAT92fH7B2Xe82+Hm52fZWUoCoOy3bWZpegVF1dl1HoJRapJTqrJTqqJR60fLY+0qp9y3fJyulrlRK9VJK9VRKzbZnPI2Ofxs9gVhOJdI7B7eveq+gdOOZ5h2h23U2DrRmSucH6tOwUKkmHq6M7h/K4u3HSKloH+pqN9wMom7X9Z9SD9iuXaNBMyuLG7qwKL1hTBmq1Ss4sBSObdG7erm42iHQ6tt2NJ0zhcX1blio1ISYthQWK+ZtsOGmNQCx9+k7xsx2lkYVmUTQ0IXFQEYiZJaxxzHV6BWsfgP8QqD3WDsEWTNxCXoXtuh6mgg6tfDl4g6BfBl/5NwtR2vLr6Wu/7R5Trl/74ZhzSSChu7sPEHZ+xNUqVeQuB4OrYKL79eVRuuIuIRUurT0I9C37sRUXRNj25J0+gwr99p4fcxAs52lUXUmETR0rXqBm1e5iQBg6pAOFfcKVr+hN3LvP8k+MdZAYbFl/UAdLytRmSt7tCTYz9P2k8YB7aDnjboelKP3rzbqHZMIGjo3D12dsoIdy5r7eHDbgHJ6BSd3wZ6fIPou8PS1c7BVt+1oBrkFxfVyotiau6sLY6PCWLrnJIlpNqoDVWrQI7oeVLzZztKomEkEjUFYtJ7oLcwr95A7B5fTK1g9Ddy9IeYuOwdZPaXrB+rr/IC1cdHhCDDHlpvWgK4D1WWErguVn23bto0GxSSCxiA0GooLdDIoR5m9gtOHYdvXekjIu2694cYlpHFRC1+C6vH8QKnWzZpwadeWzNuQSEHRBaW2ameQ2c7SqJxJBI1BaU2gSja0v6BXsOZtEBc9SVyHFBaXsPFQWr0fFrI2MTaclOwCft5x3LYNh0Xp7SzXmu0sjfKZRNAY+LbQk4flLCwrZd0rOHgoATZ9Dr3H6IVpdcj2oxnkFBTX+4lia0MuCia8ubftJ41BzxVkHYMtc2zfttEgmETQWIRG6zuHKilyVtor2LfwVf0Jsg5sPHO++IP6LpiY9g2nR+DiIkyICWfdwTS2JKbbtvGOl+qy5L+/qetFGcZ5TCJoLMKiIfsEpFc8Idncx4M7o4OITf2WrA4jIOgiBwVYdXEJqXRq4UuwX/2fH7A2Niqclk09eeSrzeTk23B/JhFdHyotwWxnaZTJJILG4uw8QdnlJqzd5b2cpnKG94rrRk0ha0XFJaw/mFZvy0pUxN/bnTfG9OFgag7/XLjDto13HQmBF8GqN8x2lsYFTCJoLFr0AHefCheWAVB4Bu+NH3DQP5r39vqx70Q19yuwsx3JmeQ0gPUD5RnQMYj7h3Vi/sYkFmy24cY1Li4wyGxnaZTNJILGwtUN2vSrdMKYzV9AzkkCr/67voNoaTX2K3CAs/sPNKCJ4vM9dNlFRLYN4KnvtnM4Ncd2Dfe6GZq2MSWqjQuYRNCYhMXA8W1QUM6bS3ER/P4WtImkaddh3DagHT9uTa5TvYK4hFQ6BPvQws/L2aHYjZurC9PG9sFF4ME5m2y3tsDNAwY8CEfWwuG1tmnTaBBMImhMwqJBFUPyprKf3/EtpB8+u/HMnYM74F2HegVFxSVsOFQ/9x+ortAAb16+MYItSRm8tmSP7Rrudyt4B5pegXEOkwgak9Ao/bWseYKSEl1cLrgrdB4O/LmuoK70CnYeyyQrv6hBThSXZXivEMbHhPPBygRW2Ko6qYc3xNwD+5aY7SyNs0wiaEy8m+s7R8pKBPuWwMmdevGRy5//LKbUoV5BvGX/gcbQIyj1zLXd6dzSl8fmbeZkVvm1oqolegp4+OrEbxiYRND4hEXrUhPWtxCWbkPpH65LF1upS72CuIRUOgT50LJpw50fOJ+XuyvTx/cjK6+Ix+ZtocQWG9g0CYCoO/SaArOdZeWOb4NTe50dhV2ZRNDYhEVDbqpeXFTq8Bp9N9GAB8DV/YJT6kKvoLhEse5gWoO+W6g8nVv68czI7qzal8KHqxIqP6Eqzm5n+aZt2muIivLh12fhgyHw/kBdzruBrsEwiaCxKWvHstWvg3eQ3t6wDHWhV7DLMj/QmIaFrI2PDmd4z1a88sseNtuiBIVfS+g7QdcfyjxW+/YamuPb4cNL9fBZnwm6TMfiv8Hc8Q1yox+TCBqb4K7g2fTPSqTHtsD+XyH2Hj2RWA5n9wrOrh9oQPWFqkNEeOmGCFo29eLBOZvIyiusfaMDHoSSIl2Z1NBKimHV6zBjKGSfhPHzYNR0GDcXrn5JL8Z7fxAc+t3ZkdqUSQSNjYsLhEb+2SNY/QZ4+EHUlApPc3avIC4hlXaB3rTybzzzA+fz93bnzbF9OJp+hqe+246q7TBF8/ZmO0trqQdg5nD47TnoOgLujYPOV+nnRPSHpSm/6q1fP70Wlr/UYIr4mUTQGIVG6zuEkjfDzgV64rBJs0pPc1avoHR+oLEOC1mLbNechy+7iIVbkpm/Man2DZZuZ7luRu3bqq+UgvUf60/6p3bDDR/BTZ+CTxn/3lr3gbtW6FXay/8Dn14HGTYsBeIkJhE0RmHRoErgu7v0hGHsvVU6zbpXsNeBvYJdxzLJzCtqlBPFZbl3WCdiOzTnmQU7OHCqlltQtuyh14001u0sM5Nh9o3w06MQHqt7ARE36R5AeTz94IYP4C/v68WZ7w+CPT87LmY7MImgMQqNBER/+uk7UU8cVtHZXsFv++wX33ka4v4DteHqIkwb0xcvdxce+HIT+UW1HJ4Y/CicOQ1/fGqbAOuLbfPh3VhdcuOa12Dit9C0ddXP7zMO7lqpN26aMwYWP1lvd4EziaAx8vKHFt30NpQDHqjWqaW9gp+2HXNYryAuIZW2gd60btbEIderD1r5e/HK6N7sPJbJfxbtrl1jYdHQdpDemrSevpFVS24afD0JvrkDgjrD3av1HFlFvYDyBHWCKb9BzN0Q/x58dDmkOH/xZXWZRNBYDXoErnheTxhW050O7BWUlK4faCRlJarj8u4tmTSgHbPWHOLXnSdq19jg0u0s59omuLpq7xLdC9j1I1z2DEz+GQI71q5NN08Y/jKMnQMZiXrdQT37Pdo1EYjI1SKyR0T2i8iT5RwzVEQ2i8gOEVlhz3gMKxE3w4CabUof4OPBpIGO6RXsPp5FxplCM1Fcjr+P6Er3kKY8Pn8LxzNqUYKi42UQ0rvhbmeZnw0/PARf3qSL7t25FAY/psuz20rXEXD373pC+bu74Nu7IN/5Nbqqwm6JQERcgXeA4UB3YJyIdD/vmGbAu8B1SqkewE32isewrSmDHNMr+HP/AZMIyuLp5srb4/uSV1jCw19torimJShEYNCjkHZA30nWkBxeq1cGb/wUBj4EU5dDSIR9ruXfBm77AYb+HbbNgw8u0Xfn1XH27BFEA/uVUglKqQJgLjDqvGPGA98qpY4AKKVO2jEew4Yc1SuIS0glrHkT2pj5gXJ1DPbluVE9iEtI491ltRif7jYSAjvpleYNoZRCYR4s+YdeGwAweTFc8S89lGNPLq4w9EmdEArPwMdXQNx7dfp3as9E0AZItPo5yfKYtc5AgIgsF5GNInJrWQ2JyFQR2SAiG06dslE5XqPW7N0rKClRrDuURqy5W6hSN/UP5brerZn22z42HKrh4jAXVxj4sC6ytv9X2wboaMe2wofDYM1b0H+SHrJpe7FjY2g3SE9Ed7wMfn4S5oyDnFTHxlBF9kwEZU3Bn58S3YD+wDXAVcA/RKTzBScpNUMpFamUigwODrZ9pEaN2LtXsOdEFum5hWZYqApEhBev70nrZl48NHczGbk1LEERMUZvZ7niZciwwYI1RysugpWv6jpBuWkwYT6MnAaevs6JxycQxs2Bq1+GA7/pIapDq50TSwXsmQiSgDCrn0OB5DKO+VkplaOUSgFWAr3tGJNhY/bsFcSfrS9k7hiqCj8vd94e148TmXk8+e3WmpWgcPOAS/4GSevhjR4wPVrfH793SflbnNYVqQdg5tWw9Hk9zHXvWrjoCmdHZSlPcbcuT+HuDZ+OhGX/0UmrjrBnIlgPXCQi7UXEAxgLLDzvmAXAYBFxExFvIAbYZceYDBuzZ68gLiGNNs2aENa8/GJ4xrn6hDXjr1d1YfH243y57kjNGuk/Ce5ZC1e+CP6hsHGmvtvmpbYw61pdlC15s97Vri4oKYF1H8J7AyFlH9z4Mdw0U2/EVJeE9NblKSLGwIqX4LO6U55Cal24qqLGRUYA0wBX4BOl1IsicjeAUup9yzGPA5OBEuAjpdS0itqMjIxUGzZssFvMRvWdzilg0MtLGda1BdPH97NJmyUliv4v/I9Lu7bktZtNJ7E6SkoUt81cx7qDaSy8fxBdWvnVrsHCPL369sBSOLAMTmzTj3sHQodh0Oky/bVpSO2Dr66Mo7DgPkhYBp0uh+umOyeO6to8B356TPfARr2rbz21MxHZqJSKLPM5eyYCezCJoG565ZfdvLv8AL88PITOLWv5xgPsOZ7FVdNW8sroCG6KDKv8BOMcJ7PyGPHmKpr7eLDw/kF4ubvarvGs45Cw3JIYlkKO5QaOFt113f6OwyB8QIVlzWtNKdj2Nfz0VygphKtehP6Ta7Y62FlS9sP8yXB8K0Tfpe9ocrdfdd2KEoFZWWzYhK3nCkrXD5iFZDXTws+L127uw94T2Tz/407bNu7XCnqPhRtmwGN79Z0xV/wLfFvoIZrZN8LL7eCzUXqB2vFt1b51cktiOuM/jGPEm6tIyT6v7EVOKsy7Fb69E1p01dePvL1+JQGwlKf4VRd9XPcBfHy5HtpyAtMjMGzGlr2Ce2ZvZGtSBqufGIbUt//gdci/F+1ixsoE3pvQj+G9HDBkUpALR9boIaQDS3W5cwCfFpbewqXQYWi5hQ6PpObyypI9/LAlmUAfD3IKiugY7MucqbE09XKHPYth4YO6SN6lT+nNdVxs2Ntxlj2L4ft7da2na16F3uNsntjM0JDhEKdzChj832Vc0iWYd2oxV6CUov8LvzK0SzCv39zHhhE2PgVFJdz0/hoOpuSw6KHBhAY4eOI9M/nPpJCwTO+XDdCylx5C6ngphF9MWoELby/dx+y4w7i5uDBlcHumDunAxsOnufOzDVzcxp1PQr7HbctsaNkTrv8AWvV07Guxt4yjupdz+Hc9oXzNa7rktY2YRGA4zKu/7OGd5fv5+aEhNZ6k3HsiiyvfWMl/b4zg5igzP1Bbh1NzuOat1XRt5cfcqbG4uTppRLikRI+Hl84tHImDkkIKXTyJL+7G8uKe+HS/kvHXXElL/z9Xkq/9bQGhKx+jjaSiBjyE66V/t//qYGcpKdbrIFa8BAHtYPQn0LqvTZo2cwSGw9wxqD0+Hm68tbTmY53xZn7AptoG+vDi9T3ZcPi0Q/eRuICLiy7INvhRim/9gW+v/J3H3P6P2QVD6eiRxtNus3lk7620/KivHibZNh9+eYqLV91GMx8vRuc/w4Mp11Hs4uG812BvLq4w9Am47Uc9TPTRFbD2HbuXp7Bh6T3DsKwrGNCOd5bvZ8/xrBr1CuIS0gjx9yKsuakvZCuj+rRh5d4U3l62n4s7BnFxR+ckWaUUy/ee4uXFu9l9PIveoQO5acQUQjoEQnqiHj46sBT2LILNX+iTIu/A74p/MTz+JC8u2oWfpxv/uaFXw547ajdQT4IvuA9++T9IWAF/eRd8guxyOTM0ZNhcbeYKlFJEvfgrgy8K5o0xZn7AlnLyixj59mpyCopY/NAQmvs49pP1tqQM/rN4F2sO6I2G/nZVV0b0alX2G3pJsV605uKqexEWr/6yh+nL9jN1SAf+Prxrw04GoHsC62bAkqf1uo0bPoT2g2vUlBkaMhyqtFewaNsx9hyv3mrjA6eySckuINbsT2xzPp5uvDWuL6dzCvnb/C01K0FRA4lpuTw0dxMjp69m9/Esnh3Znf89cgnXRISU/0bu4gqh/c9JAgCPXdmZWy9uy4yVCby7/IADoncyEYi5S99m6uGjS3/YgUkEhl3UdK5gbYLZn9ieerbx58nhXfl110k+XXPIrtc6nVPACz/u5LLXVvDLjuPcN6wjyx8fyqSB7fFwq9lbj4jw7MgeXN+3Da/8sofP1x6yacx1VkhvvT/ywIft0ryZIzDsoqZzBXEJqbRq6kXbQFNfyF4mD2zH6v0p/HvRbqLaN6dHa3+btp9XWMysNYd4Z9l+cvKLGN0/lEev6EIrf9usmnVxEf47OoKsvCL+sWAHfl7u/KXv+RXuGyAPH7s1bXoEht1Ut1eglCI+IY3YDs0b/tivE4kIr4yOoJm3Ow/M2URugW2qYBaXKL7ZmMSlry7npcW7iWrXnMUPDeG/o3vbLAmUcnd1Yfr4vlzcIZDHvt5S+z2bGzmTCAy7qe5cwYFTOaRk55v9Bxwg0NeTaWP7cDAlh38u2FHr9lbuPcW1b6/msa+3EOTnyZd3xvDJpKjaF7yrgJe7Kx/eFknP1k2598s/WHMgxW7XauhMIjDsqjq9gviDZv2AIw3oGMR9Qzvx9cYkFmyuWTnk7UczuOXjeG79ZB3Z+YW8Na4v3987kAEd7XOb4/l8Pd2YNTmats29ufPTDWxJTHfIdRsakwgMu6pOryAuIY0Wfp60M/MDDvPw5RfRL7wZT323nSOpuVU+L+l0Lo98tZmR01ez7WgG/7i2O78+egnX9W6Ni4tjh/UCfDyYPSWG5r4e3DZznV330G6oTCIw7G7K4Mp7BUop4hJSie0QaOYHHMjN1YU3x/ZFBB6Yu4nC4oo3m8nILeTfi3Zx6asrWLTtGHdf0pEVjw/jjkHt8XRzXvG3lk29+OKOWDxcXbjl43gS06qe1AyTCAwHaObtweSBFfcKDqbkcCor3wwLOUFYc29evjGCLYnpvLpkT5nH5BUW8+HKBIa8sowPVyVwXZ/WLPvrUJ64uiv+TdwdHHHZwgO9+fyOGPIKS5jwUTwnM/OcHVK9YRKB4RCVzRXEla4fMAvJs/cU+wAACk9JREFUnGJErxDGRYfzwYoEVu49dfbxkhLFd5uSuOy1Fby4aBd9w5ux6MHBvHpTb1o3q3slQLq08uPT26NJzc7nlo/XkZ5b4OyQ6gWTCAyHqKxXEJeQSrCfJx2C7HevtFGxZ67tTueWvjw6bwunsvJZvS+FkdNX88hXWwjwceeLKTHMmhxNt5Cmzg61Qn3CmvHhrZEcTM3htpnryc6vO5vE11UmERgOc7ZXcF4FTKUU8QdTiWlv1g84UxMPV94e14+svEKumraSiR/Hk55byJtj+7DwvkEM7OSYO4FsYUCnIKaP68v2oxlM/WwDeYXFzg6pTjOJwHCY0l7BT+f1Cg6l5nIi08wP1AVdWvnx/KieeLq58PQ13Vj610sY1aeNw+8EsoUre7TildERrDmQygNzNlFUyUR4Y2YSgeFQdwxqj6/nub0Cs/9A3XJzVBhr/34ZUwZ3cOqdQLZwQ79QnruuB//beYK/zd9KSUn9qrbsKCYRGA5VVq8gLiGVIF9POgab+QHD9m4b0I7HrujMt5uO8q8fdzqs6mp9YhKB4XDWvQK9fiCNGFNfyLCj+y/txJRB7Zm15hBv/G+vs8Opc0z1UcPhSnsFby/dz6idrTmemWeGhQy7EhGeuqYbmXmFvLV0P02buDNlcAdnh1VnmB6B4RSlvYK/fr0FgNj2Zv2AYV8iwn9uiGBEr1a88NMu5q1PdHZIdYZJBIZTlPYKMvOKCPTxoFMLX2eHZDQCri7CG2P6MPiiIJ78diuLth1zdkh1gkkEhtPcMag9fp5uDOgUZOYHDIfxdHPlg1v60zc8gIfmbmKF1UrqxsokAsNpmnl78P39A3l2ZHdnh2I0Mt4ebnwyKYpOLfy4+/ONbDyc5uyQnMokAsOpOgb7Eujr6ewwjEbIv4k7n90eTSt/LybNXM/O5Exnh+Q0dk0EInK1iOwRkf0i8mQFx0WJSLGIjLZnPIZhGNaC/Tz5/I5ofD3duPWTeBJOZTs7JKewWyIQEVfgHWA40B0YJyIXjAFYjnsZ+MVesRiGYZQnNECXry5RcMvH60hOP+PskBzOnj2CaGC/UipBKVUAzAVGlXHcA8A3wEk7xmIYhlGuTi18+ez2aDLPFDLx43hSsvOdHZJD2TMRtAGsb9RNsjx2loi0Aa4H3q+oIRGZKiIbRGTDqVNmht8wDNvr2cafjydFcfT0GW77ZB2ZeYXODslh7JkIyrof8PwiH9OAJ5RSFdaIVUrNUEpFKqUig4ODbRagYRiGtej2zXl/Yn/2HM9iyqwNnClwXvnqkhJFem4BB1Ny+OPIaZbuPlHpvt81Zc8SE0lAmNXPoUDyecdEAnMt95AHASNEpEgp9b0d4zIMwyjXsK4teGNMHx6cu4l7vtjIjFsi8XCr+WdmpRQ5BcWk5xaQnlvIacvX9NwCTlt+zrB8PZ1bSMYZy2NnCjm/Pt5dQzrw9xHdavkKL2TPRLAeuEhE2gNHgbHAeOsDlFLtS78XkVnAjyYJGIbhbCN7tyY7v4i/f7uNR+dt5s2xfXF1EfKLii1v2qVv6KVv7oUXvtGfKTj7eGFx+RVPfT3d8G/iToCPOwHeHoQ196ZZE3cCvP+/vXuLsauq4zj+/TEX6JkqUwI+MIW2XAIFIrQSUmwEYnnQaJSHGrm0AZ4gAQVCgmIgJr74okQfiEIqBEMDxFoSY6oSgTTpA7QwFEopJKRKGayhEig3tRd/PuxNOjMdJ1Pp6ZrO+n1e5uw15+z8zsrZ+7/32ues3cdgp5/BTtM+2OljaE53bg/atUJge5+km2m+DdQDPGB7q6Qb2/9Pel0gIqKkqy46lff/uZcf/+FVntn+Dh/v2c/HkwwV9fccM2anveDEARZ3+hns9Lc79b72cbN8fKePwVn9n+ps43Dp6uyjttcB68a1TVgAbF/XzSwREYfqhktPZ+DYXoZ3vDtqB978ndPpb4/km+VZfT1H7VQpmYY6ImISK5bMY8WSeaVjdFX5c5KIiCgqhSAionIpBBERlUshiIioXApBRETlUggiIiqXQhARUbkUgoiIysnjZzWa5iTtAt74P19+IvCPwxjnaJf+GCv9cUD6YqyZ0B/zbE84ffNRVwg+DUnP2b6wdI7pIv0xVvrjgPTFWDO9PzI0FBFRuRSCiIjK1VYI7i8dYJpJf4yV/jggfTHWjO6Pqq4RRETEwWo7I4iIiHFSCCIiKldNIZD0FUmvSXpd0vdL5ylJ0imSnpa0TdJWSbeUzlSapB5JL0j6fekspUkalLRG0qvtZ+Ti0plKkXRbu428LOkRSceVztQNVRQCST3AvcBXgXOAqySdUzZVUfuA220vBJYAN1XeHwC3ANtKh5gmfg780fbZwPlU2i+ShoDvAhfaPo/m3utXlk3VHVUUAuAi4HXb223vAR4Fvlk4UzG2d9oebh9/QLOhD5VNVY6kucDXgFWls5Qm6bPAJcCvAGzvsf1e2VRF9QKzJPUCHeBvhfN0RS2FYAh4c9TyCBXv+EaTNB9YBDxbNklRPwPuAP5TOsg0cBqwC3iwHSpbJWmgdKgSbL8F/ATYAewEdtt+omyq7qilEGiCtuq/NytpNvBb4Fbb75fOU4KkrwNv236+dJZpohdYDPzC9iLgI6DKa2qS5tCMHCwATgYGJK0om6o7aikEI8Apo5bnMkNP8aZKUh9NEVhte23pPAUtBb4h6a80Q4ZflvRw2UhFjQAjtj85Q1xDUxhqdDnwF9u7bO8F1gJfLJypK2opBJuAMyUtkNRPc8Hnd4UzFSNJNGPA22zfUzpPSbbvtD3X9nyaz8VTtmfkUd9U2P478Kaks9qmZcArBSOVtANYIqnTbjPLmKEXzntLBzgSbO+TdDPwJ5or/w/Y3lo4VklLgZXAFkmb27Yf2F5XMFNMH98BVrcHTduB6wvnKcL2s5LWAMM037R7gRk61USmmIiIqFwtQ0MREfE/pBBERFQuhSAionIpBBERlUshiIioXApBxBEk6bLMcBrTTQpBRETlUggiJiBphaSNkjZLuq+9X8GHkn4qaVjSk5JOap97gaRnJL0k6fF2jhoknSHpz5JebF9zerv62aPm+1/d/mo1opgUgohxJC0Evg0stX0BsB+4BhgAhm0vBtYDP2xf8mvge7Y/D2wZ1b4auNf2+TRz1Oxs2xcBt9LcG+M0ml96RxRTxRQTEYdoGfAFYFN7sD4LeJtmmurH2uc8DKyVdDwwaHt92/4Q8BtJnwGGbD8OYPtfAO36NtoeaZc3A/OBDd1/WxETSyGIOJiAh2zfOaZRunvc8yabn2Wy4Z5/j3q8n2yHUViGhiIO9iSwXNLnACSdIGkezfayvH3O1cAG27uBdyV9qW1fCaxv7+8wIumKdh3HSuoc0XcRMUU5EokYx/Yrku4CnpB0DLAXuInmJi3nSnoe2E1zHQHgWuCX7Y5+9GydK4H7JP2oXce3juDbiJiyzD4aMUWSPrQ9u3SOiMMtQ0MREZXLGUFEROVyRhARUbkUgoiIyqUQRERULoUgIqJyKQQREZX7LwLEezIUEjnQAAAAAElFTkSuQmCC\n",
      "text/plain": [
       "<Figure size 432x288 with 1 Axes>"
      ]
     },
     "metadata": {
      "needs_background": "light"
     },
     "output_type": "display_data"
    }
   ],
   "source": [
    "from matplotlib import pyplot\n",
    "# plot train and validation loss\n",
    "pyplot.plot(history.history['loss'])\n",
    "pyplot.plot(history.history['val_loss'])\n",
    "pyplot.title('model train vs validation loss')\n",
    "pyplot.ylabel('loss')\n",
    "pyplot.xlabel('epoch')\n",
    "pyplot.legend(['train', 'validation'], loc='upper right')\n",
    "pyplot.show() "
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 6,
   "metadata": {},
   "outputs": [
    {
     "name": "stdout",
     "output_type": "stream",
     "text": [
      "INFO:tensorflow:Assets written to: saved_model/model2_10epochs_bit/assets\n"
     ]
    },
    {
     "name": "stderr",
     "output_type": "stream",
     "text": [
      "INFO:tensorflow:Assets written to: saved_model/model2_10epochs_bit/assets\n"
     ]
    }
   ],
   "source": [
    "import tensorflow as tf \n",
    "# Save fine-tuned model as SavedModel\n",
    "export_module_dir = 'saved_model/model2_10epochs_bit'\n",
    "tf.saved_model.save(model, export_module_dir)"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "## Model Based on ResNet 101x1\n",
    "### Experiment 1:"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 4,
   "metadata": {},
   "outputs": [
    {
     "name": "stdout",
     "output_type": "stream",
     "text": [
      "WARNING:tensorflow:From /opt/conda/lib/python3.7/site-packages/tensorflow_core/python/ops/resource_variable_ops.py:1781: calling BaseResourceVariable.__init__ (from tensorflow.python.ops.resource_variable_ops) with constraint is deprecated and will be removed in a future version.\n",
      "Instructions for updating:\n",
      "If using Keras pass *_constraint arguments to layers.\n"
     ]
    },
    {
     "name": "stderr",
     "output_type": "stream",
     "text": [
      "WARNING:tensorflow:From /opt/conda/lib/python3.7/site-packages/tensorflow_core/python/ops/resource_variable_ops.py:1781: calling BaseResourceVariable.__init__ (from tensorflow.python.ops.resource_variable_ops) with constraint is deprecated and will be removed in a future version.\n",
      "Instructions for updating:\n",
      "If using Keras pass *_constraint arguments to layers.\n"
     ]
    },
    {
     "name": "stdout",
     "output_type": "stream",
     "text": [
      "Model sucessfuly built\n",
      "callbacks and data okay\n",
      "Train on 10 steps, validate on 4 steps\n",
      "WARNING:tensorflow:From /opt/conda/lib/python3.7/site-packages/keras/callbacks/tensorboard_v1.py:200: The name tf.summary.merge_all is deprecated. Please use tf.compat.v1.summary.merge_all instead.\n",
      "\n"
     ]
    },
    {
     "name": "stderr",
     "output_type": "stream",
     "text": [
      "WARNING:tensorflow:From /opt/conda/lib/python3.7/site-packages/keras/callbacks/tensorboard_v1.py:200: The name tf.summary.merge_all is deprecated. Please use tf.compat.v1.summary.merge_all instead.\n",
      "\n"
     ]
    },
    {
     "name": "stdout",
     "output_type": "stream",
     "text": [
      "WARNING:tensorflow:From /opt/conda/lib/python3.7/site-packages/keras/callbacks/tensorboard_v1.py:203: The name tf.summary.FileWriter is deprecated. Please use tf.compat.v1.summary.FileWriter instead.\n",
      "\n"
     ]
    },
    {
     "name": "stderr",
     "output_type": "stream",
     "text": [
      "WARNING:tensorflow:From /opt/conda/lib/python3.7/site-packages/keras/callbacks/tensorboard_v1.py:203: The name tf.summary.FileWriter is deprecated. Please use tf.compat.v1.summary.FileWriter instead.\n",
      "\n"
     ]
    },
    {
     "name": "stdout",
     "output_type": "stream",
     "text": [
      "WARNING:tensorflow:Issue encountered when serializing variables.\n",
      "Type is unsupported, or the types of the items don't match field type in CollectionDef. Note this is a warning and probably safe to ignore.\n",
      "'list' object has no attribute 'name'\n"
     ]
    },
    {
     "name": "stderr",
     "output_type": "stream",
     "text": [
      "WARNING:tensorflow:Issue encountered when serializing variables.\n",
      "Type is unsupported, or the types of the items don't match field type in CollectionDef. Note this is a warning and probably safe to ignore.\n",
      "'list' object has no attribute 'name'\n"
     ]
    },
    {
     "name": "stdout",
     "output_type": "stream",
     "text": [
      "Epoch 1/10\n",
      " 9/10 [==========================>...] - ETA: 30s - loss: 1.3831 - acc: 0.6595 WARNING:tensorflow:From /opt/conda/lib/python3.7/site-packages/keras/callbacks/tensorboard_v1.py:343: The name tf.Summary is deprecated. Please use tf.compat.v1.Summary instead.\n",
      "\n"
     ]
    },
    {
     "name": "stderr",
     "output_type": "stream",
     "text": [
      "WARNING:tensorflow:From /opt/conda/lib/python3.7/site-packages/keras/callbacks/tensorboard_v1.py:343: The name tf.Summary is deprecated. Please use tf.compat.v1.Summary instead.\n",
      "\n"
     ]
    },
    {
     "name": "stdout",
     "output_type": "stream",
     "text": [
      "10/10 [==============================] - 447s 45s/step - loss: 1.3627 - acc: 0.6736 - val_loss: 0.9334 - val_acc: 0.7945\n",
      "Epoch 2/10\n",
      "10/10 [==============================] - 380s 38s/step - loss: 1.2109 - acc: 0.7762 - val_loss: 0.6339 - val_acc: 0.7965\n",
      "Epoch 3/10\n",
      "10/10 [==============================] - 389s 39s/step - loss: 0.8082 - acc: 0.7980 - val_loss: 1.1226 - val_acc: 0.6489\n",
      "Epoch 4/10\n",
      "10/10 [==============================] - 389s 39s/step - loss: 0.7577 - acc: 0.7727 - val_loss: 0.5446 - val_acc: 0.7958\n",
      "Epoch 5/10\n",
      "10/10 [==============================] - 392s 39s/step - loss: 0.5631 - acc: 0.8357 - val_loss: 0.5664 - val_acc: 0.7633\n",
      "Epoch 6/10\n",
      "10/10 [==============================] - 383s 38s/step - loss: 0.3768 - acc: 0.8531 - val_loss: 0.4053 - val_acc: 0.8114\n",
      "Epoch 7/10\n",
      "10/10 [==============================] - 390s 39s/step - loss: 0.5045 - acc: 0.8166 - val_loss: 0.5470 - val_acc: 0.7750\n",
      "Epoch 8/10\n",
      "10/10 [==============================] - 382s 38s/step - loss: 0.5766 - acc: 0.8105 - val_loss: 0.4007 - val_acc: 0.8231\n",
      "Epoch 9/10\n",
      "10/10 [==============================] - 388s 39s/step - loss: 0.4892 - acc: 0.8252 - val_loss: 0.6981 - val_acc: 0.7334\n",
      "Epoch 10/10\n",
      "10/10 [==============================] - 383s 38s/step - loss: 0.3975 - acc: 0.8475 - val_loss: 0.5720 - val_acc: 0.7588\n",
      "Training model is finished\n",
      "5/5 [==============================] - 183s 37s/step - loss: 0.6319 - acc: 0.7673\n",
      "Test accuracy: 0.76731706\n",
      "CPU times: user 5h 42min 15s, sys: 1h 20min 55s, total: 7h 3min 11s\n",
      "Wall time: 1h 11min\n"
     ]
    }
   ],
   "source": [
    "%%time\n",
    "history,model = BiT_L_ResNet.train_Bit_L(train,validation,test,model_name=\"r101x1\",epochs=10)"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 5,
   "metadata": {},
   "outputs": [
    {
     "data": {
      "image/png": "iVBORw0KGgoAAAANSUhEUgAAAYIAAAEWCAYAAABrDZDcAAAABHNCSVQICAgIfAhkiAAAAAlwSFlzAAALEgAACxIB0t1+/AAAADh0RVh0U29mdHdhcmUAbWF0cGxvdGxpYiB2ZXJzaW9uMy4yLjEsIGh0dHA6Ly9tYXRwbG90bGliLm9yZy+j8jraAAAgAElEQVR4nOzdd3hUVfrA8e+bTiC0QGiBhJJQAqGFLoJSAvaC0hEbi65919Wt6uruurv+XNS1YUOWJgL2goCAIIiEHmooAUILoaaQfn5/nAEDJCFlJpNk3s/z8CQz995z3xlg3rnnnvMeMcaglFLKc3m5OwCllFLupYlAKaU8nCYCpZTycJoIlFLKw2kiUEopD6eJQCmlPJwmAuU0IjJNRF4o4b6JIjLYhbGMFZHvXNW+K4nIsyIyw/F7CxFJExHvK+1bxnNtFZGBZT2+mHaXich9zm5XuYaPuwNQ6lIiMg1IMsb8qaxtGGNmAjOdFpSbGGMOALWc0VZh76sxJsoZbauqTa8IVJUjIvoFRikn0kTgYRxdMk+KyGYRSReR90SkkYh8IyKpIrJYROoV2P8mR/fBacflfvsC27qKyHrHcR8BAZec6wYR2eg4dpWIRJcgvknAWOB3ji6RLwrE/ZSIbAbSRcRHRJ4WkT2O828TkVsLtDNRRFYWeGxEZLKIJIjIKRF5XUSkkPM3FZFzIlL/kteZIiK+ItJGRJaLyBnHcx8V8Tq+FZGHLnluk4jc5vj9FRE5KCJnRWSdiPQvop1wR+w+jsctHedPFZFFQINL9v9YRI464vtBRKJK8L4OdvzuLyJTROSw488UEfF3bBsoIkki8hsRSRaRIyJyd+F/i5e9Bi8R+ZOI7HccO11E6ji2BYjIDBE54fh3slZEGjm2TRSRvY7Xuk9ExpbkfKoMjDH6x4P+AInAT0AjoBmQDKwHugL+wPfAM459I4F0YAjgC/wO2A34Of7sBx53bBsB5AAvOI7t5mi7F+AN3OU4t3+BOAYXEeO08+1cEvdGoDlQw/HcHUBT7BeakY5Ymzi2TQRWFjjeAF8CdYEWwHFgWBHn/x64v8DjfwNvOX6fDfzRcc4A4Koi2pgA/FjgcQfgdIHXPw4IxnbP/gY4CgQ4tj0LzHD8Hu6I3cfxeDXwsuPv6mog9fy+ju33AEGO7VOAjSV4Xwc7fv+r499GCNAQWAU879g2EMh17OMLXAdkAPWKeP3LgPsKxLQbaIXt5loA/M+x7VfAF0Cg499Jd6A2UBM4C7R17NcEiHL3/5/q+kevCDzTa8aYY8aYQ8AKYI0xZoMxJgv4BJsUwH64fmWMWWSMyQFeAmoAfYHe2A+EKcaYHGPMPGBtgXPcD7xtjFljjMkzxnwIZDmOK6tXjTEHjTHnAIwxHxtjDhtj8o0xHwEJQM9ijn/RGHPa2H73pUCXIvabBYwGcFw1jHI8BzbZhQFNjTGZxpiVhTfBJ0AXEQlzPB4LLHC8xxhjZhhjThhjco0x/4f94G5b3IsXkRZAD+DPxpgsY8wP2A/RC4wx7xtjUh3neRbofP7bdwmMBf5qjEk2xhwHngPGF9ie49ieY4z5Gki7UswF2n3ZGLPXGJMG/B4Y5bjKycEmxDaOfyfrjDFnHcflAx1FpIYx5ogxZmsJX4cqJU0EnulYgd/PFfL4/M3Jpthv/QAYY/KBg9griabAIWNMwaqF+wv8Hgb8xnG5f1pETmO/zTctR9wHCz4QkQkFup5OAx25pKvkEkcL/J5B0Tdh5wF9RKQp9lu3wSZMsFdFAvzs6DK7p7AGjDGpwFfYJILj54Wb144ulu2OLpzTQJ0rxA72vTtljEkv8NyF91xEvEXkRUd32Vnst31K0G7B9gv+He7n4r+vE8aY3AKPi3sPr9SuD/aq9H/AQmCOozvqXyLi63iNI4HJwBER+UpE2pXwdahS0kSginMY+4EOXPh23Bw4BBwBml3Sz96iwO8Hgb8ZY+oW+BNojJldgvMWVRL3wvOOb9rvAA8BwcaYukA89kO6XIwxp4HvgDuBMcDs8wnPGHPUGHO/MaYptlvjDRFpU0RTs4HRItIHeyW11BF7f+ApR/v1HLGfKUHsR4B6IlKzwHMF3/MxwM3AYGxiCXc8f77dK5Uavujv29H24SscUxKFtZsLHHNcXTxnjOmAvdK8AduthjFmoTFmCLZbaAf271u5gCYCVZy5wPUiMkhEfLF92VnYvuPV2P/Mjzhu3N7Gxd0y7wCTRaSXWDVF5HoRCSrBeY9h+5OLUxP7wXYcwHHjsmNpXtwVzMJ+IN3OL91CiMgdIhLqeHjKEUNeEW18jf0A/CvwkeOKCmwffq4jdh8R+Qu2X7xYxpj9QBzwnIj4ichVwI0FdgnC/v2cwPa5//2SJq70vs4G/iQiDUWkAfAXoMxzFC5p93HHje5ajrg+Msbkisg1ItJJ7DyJs9iuojyxAxhuciS9LGw3VFHvsyonTQSqSMaYndibmq8BKdgPnRuNMdnGmGzgNuxN2VPYy/gFBY6Nw94n+K9j+27HviXxHtDB0eXzaRGxbQP+D5uQjgGdgB9L9wqL9TkQgf3WuqnA8z2ANSKS5tjnUWPMviJizMK+J4MpkEywXSHfALuw3SSZXNLtVYwx2BvwJ4FngOkFtk13tHcI2Ia98VvQld7XF7CJZjOwBTuIoEQTBK/gfWwX0A/APuzrfdixrTG2K+4ssB1Yjk0+XtgvHoexr3UA8KATYlGFkIu7eJVSSnkavSJQSikPp4lAKaU8nCYCpZTycJoIlFLKw7mseJeIvI8dE5xsjClyWJ+I9MCObhjpmJ1arAYNGpjw8HCnxamUUp5g3bp1KcaYhoVtc2UVx2nYoYPTi9rBMXb4n9jhdCUSHh5OXFxcuYNTSilPIiL7i9rmsq4hRx2Uk1fY7WFgPrY4mVJKKTdw2z0CEWkG3Aq8VYJ9J4lInIjEHT9+3PXBKaWUB3HnzeIpwFPGmCtOGzfGTDXGxBhjYho2LLSLSymlVBm5c6WnGGzFQbDVEa8TkVxjTKElBZRS1VNOTg5JSUlkZma6O5RqISAggNDQUHx9fUt8jNsSgTGm5fnfxa6l+qUmAaU8T1JSEkFBQYSHhyOXLxqnSsEYw4kTJ0hKSqJly5ZXPsDBlcNHZ2NXNWogIknYAlm+AMaYK94XUEp5hszMTE0CTiIiBAcHU9p7qS5LBMaY0aXYd6Kr4lBKVX6aBJynLO+lx8wsPpmezXNfbCUzR0uaK6VUQR6TCH7cncK0VYmMf28NZzJy3B2OUqqSOH36NG+88Uapj7vuuus4ffq0CyKqeB6TCG7s3JRXR3Vl08Ez3PH2Kg6fPufukJRSlUBRiSAvr/jeg6+//pq6deu6KqwK5TGJAGwymHZ3Dw6fzuT2N1eRcCzV3SEppdzs6aefZs+ePXTp0oUePXpwzTXXMGbMGDp16gTALbfcQvfu3YmKimLq1KkXjgsPDyclJYXExETat2/P/fffT1RUFEOHDuXcuar1RbPKrVAWExNjyltraOvhM0z8YC3Zufm8d1cMMeH1nRSdUqq0tm/fTvv27QF47outbDt81qntd2ham2dujCpye2JiIjfccAPx8fEsW7aM66+/nvj4+AvDL0+ePEn9+vU5d+4cPXr0YPny5QQHB1+oe5aWlkabNm2Ii4ujS5cu3Hnnndx0002MGzfOqa+jNAq+p+eJyDpjTExh+3vUFcF5UU3rsOCBvtSv6cfYd9fw3daj7g5JKVVJ9OzZ86Ix+K+++iqdO3emd+/eHDx4kISEhMuOadmyJV26dAGge/fuJCYmVlS4TuHOmcVu1bx+IPMm9+GeaWuZPGMdf7u1E6N7tnB3WEp5tOK+uVeUmjVrXvh92bJlLF68mNWrVxMYGMjAgQMLnQHt7+9/4Xdvb+8q1zXkkVcE5wXX8mf2pN5cHdmQ3y/YwiuLE6hqXWVKqfIJCgoiNbXw+4VnzpyhXr16BAYGsmPHDn766acKjq5ieOwVwXmBfj68MyGGp+dv4T+Ld3EsNZPnb+6It5dOcFHKEwQHB9OvXz86duxIjRo1aNSo0YVtw4YN46233iI6Opq2bdvSu3dvN0bqOh55s7gwxhj+tXAnby7bw9AOjXh1dFcCfL2dfh6l1MUKu7GpykdvFpeRiPDUsHY8c2MHFm0/phPPlFIeQxPBJe7u15LXRuvEM6WU59BEUIgbonXimVLKc2giKELfNg346Fe9yc03jHhrNXGJV1p+WSmlqiZNBMXQiWdKKU+gieAKzk88a9ekNpNnrGP2zwfcHZJSSjmVJoISCK7lz+z7e+nEM6UUtWrVAuDw4cOMGDGi0H0GDhzIlYa5T5kyhYyMjAuP3VnWWhNBCZ2feHZ7t1D+s3gXf/w0nrx8TQZKeaqmTZsyb968Mh9/aSJwZ1lrTQSl4OvtxUt3RPPgwNbMWnOAB2as0xXPlKrinnrqqYvWI3j22Wd57rnnGDRoEN26daNTp0589tlnlx2XmJhIx44dATh37hyjRo0iOjqakSNHXlRr6IEHHiAmJoaoqCieeeYZwBayO3z4MNdccw3XXHMN8EtZa4CXX36Zjh070rFjR6ZMmXLhfK4qd+3xJSZKS0T43bB2hAT589yX2xj/3hrendCDOoG+7g5Nqarvm6fh6Bbnttm4Ewx/scjNo0aN4rHHHuPBBx8EYO7cuXz77bc8/vjj1K5dm5SUFHr37s1NN91U5HrAb775JoGBgWzevJnNmzfTrVu3C9v+9re/Ub9+ffLy8hg0aBCbN2/mkUce4eWXX2bp0qU0aNDgorbWrVvHBx98wJo1azDG0KtXLwYMGEC9evVISEhg9uzZvPPOO9x5553Mnz/fKeWu9YqgjCbqxDOlqoWuXbuSnJzM4cOH2bRpE/Xq1aNJkyb84Q9/IDo6msGDB3Po0CGOHTtWZBs//PDDhQ/k6OhooqOjL2ybO3cu3bp1o2vXrmzdupVt27YVG8/KlSu59dZbqVmzJrVq1eK2225jxYoVgOvKXesVQTncEN2U+jX9mDR9Hbe/uYrp9/QkolGQu8NSquoq5pu7K40YMYJ58+Zx9OhRRo0axcyZMzl+/Djr1q3D19eX8PDwQstPF1TY1cK+fft46aWXWLt2LfXq1WPixIlXbKe4gSiuKnetVwTl1Le1TjxTqqobNWoUc+bMYd68eYwYMYIzZ84QEhKCr68vS5cuZf/+/cUef/XVVzNz5kwA4uPj2bx5MwBnz56lZs2a1KlTh2PHjvHNN99cOKao8tdXX301n376KRkZGaSnp/PJJ5/Qv39/J77ay2kicILzE8+CdeKZUlVSVFQUqampNGvWjCZNmjB27Fji4uKIiYlh5syZtGvXrtjjH3jgAdLS0oiOjuZf//oXPXv2BKBz58507dqVqKgo7rnnHvr163fhmEmTJjF8+PALN4vP69atGxMnTqRnz5706tWL++67j65duzr/RRegZaid6ERaFvd8GMeWpNO64plSJaRlqJ1Py1C7kU48U0pVRZoInOz8xLMR3XXimVKqatBRQy7g6+3Fv0dEExLkzxvL9pCSmqUrnilVDGNMkWP0VemUpRdCrwhc5PzEs2d1xTOlihUQEMCJEye0G9UJjDGcOHGCgICAUh2nVwQuNrFfSxoE+fPER5u44+1VTLu7J03r1nB3WEpVGqGhoSQlJXH8+HF3h1ItBAQEEBoaWqpjNBFUgPMTz36lE8+Uuoyvry8tW7Z0dxgezWVdQyLyvogki0h8EdvHishmx59VItLZVbFUBnbiWR9y8w2j3/mJjOxcd4eklFKAa+8RTAOGFbN9HzDAGBMNPA9MdWEslUKHprV5fUw3UtKy+WzjYXeHo5RSgAsTgTHmB6DIegvGmFXGmFOOhz8BpevUqqJ6hNejXeMgpq/erzfHlFKVQmUZNXQv8E1RG0VkkojEiUhcVb+hJCJM6BPO9iNnidt/6soHKKWUi7k9EYjINdhE8FRR+xhjphpjYowxMQ0bNqy44Fzklq5NCQrwYfrq4gtZKaVURXBrIhCRaOBd4GZjzAl3xlKRAv18uKN7c76NP0JyavElaZVSytXclghEpAWwABhvjNnlrjjcZXyfMHLyDHN+PujuUJRSHs6Vw0dnA6uBtiKSJCL3ishkEZns2OUvQDDwhohsFJHKWVLURVo2qEn/iAbMXLOfnLx8d4ejlPJgLptQZowZfYXt9wH3uer8VcGEPuHcPz2ORduOcV2nJu4ORynlodx+s9iTXdsuhGZ1azB9daK7Q1FKeTBNBG7k7SWM6x3GT3tPsuvY5UvWKaVURdBE4GYjezTHz8eL/+lQUqWUm2gicLP6Nf24IboJC9YnkZqpZaqVUhVPE0ElMKFPOOnZeSxYf8jdoSilPJAmgkqgS/O6dA6tw/9+0vpDSqmKp4mgkhjfJ5zdyWms3uMxE6yVUpWEJoJK4oboJtQL9OXD1YnuDkUp5WE0EVQSAb7e3NmjOYu2HePw6XOuP+Hpg5CS4PrzKKUqPU0Elci4XmEYYNaaA64/2by7YeYdoPcklPJ4mggqkeb1AxnULoQ5aw+QlZvnuhOd3AdJa+HUPkjxuHp/SqlLaCKoZMb3CSclLZtv44+67iRbF/zy+66FrjuPUqpK0ERQyfRv04Dw4EA+XJXoupNsmQ+hPSEkChK+c915lFJVgiaCSsbLSxjfJ5z1B04Tf+iM80+QvB2St0KnERA5FA6shkwXnEcpVWVoIqiERnQPpYavt2vqD8XPB/GCDrdARCzk58Kepc4/j1KqytBEUAnVqeHLLV2b8tmmQ5zOyHZew8bYRBB+FQQ1gtAeEFBXu4eU8nCaCCqp8b3DyczJ5+O4JOc1emQjnNwLHUfYx94+0GYQJCyCfF0lTSlPpYmgkurQtDYxYfWYsWY/+flOGuu/ZR54+UD7G395LiIW0pNtklBKeSRNBJXYhL7h7D+RwfKE4+VvLD8ftn4CrQdBYP1fnm8zGBDtHlLKg2kiqMSGRTWmQS1/59w0PrgGzh6yo4UKqhkMoTE6n0ApD6aJoBLz8/FiTM/mLN2ZzIETGeVrLH4e+ARA2+GXb4uIhcPrIS25fOdQSlVJmggquTG9wvASYcaaclwV5OXC1k8hchj4B12+PXKo/bl7cdnPoZSqsjQRVHKN6wQQG9WIuXEHycwpY/2hxB8gIwU63l7ESaKhVmPtHlLKQ2kiqALG9w7ndEYOn286XLYGtswHvyCIGFL4dhG7bc9SyNN1k5XyNJoIqoDereoT2agW01cnln4py9ws2P4FtL8BfGsUvV9kLGSdsTeVlVIeRRNBFSAijO8dRvyhs2w4eLp0B+9ebD/gi+oWOq/VQPDy1e4hpTyQJoIq4tZuodTy9yn9UNL4+VCjvv2gL45/EIT11fkESnkgTQRVRC1/H27v1oyvNh8hJS2rZAdlp8POb6DDzeDte+X9I2Ph+A445YJid0qpSksTQRUyvk8Y2Xn5fLT2YMkO2PkN5GRcuVvovIhY+1OvCpTyKJoIqpA2IUH0bR3MrDUHyM0rQZG4+AV2WGhY35KdILg11GupiUApD6OJoIqZ0CeMQ6fPsWTHFWYBnzsNuxdBx9vAy7tkjYvY7qF9P0DOufIHq5SqElyWCETkfRFJFpH4IraLiLwqIrtFZLOIdHNVLNXJ4PaNaFIn4Mo3jXd8CXnZJe8WOi9iKORmwr4VZQ9SKVWluPKKYBowrJjtw4EIx59JwJsujKXa8PH2YmyvFqzcncKe42lF7xg/H+qGQbPupTtBWD/wDYQEHUaqlKdwWSIwxvwAnCxml5uB6cb6CagrIk1cFU91MrJHC3y9peirgrTjsHe5vRoQKV3jvgF2qOmu7+yKZkqpas+d9wiaAQWHvyQ5nruMiEwSkTgRiTt+3Am1+au4hkH+XNepCfPXJZGelXv5Dts+BZNX+m6h8yKGwpkDcHxn+QJVSlUJ7kwEhX1VLfQrqDFmqjEmxhgT07BhQxeHVTVM6BNOalYun2w4dPnG+AXQsB00iipb4xGOaqTaPaSUR3BnIkgCmhd4HAqUsaqa5+nWoi5RTWvzv9X7L64/dOYQHFhVtm6h8+o0g0YdbfeQUqrac2ci+ByY4Bg91Bs4Y4w54sZ4qhQRYUKfMHYeS+XnfQVuxWxdYH+WtVvovIihcGA1ZJ4pXztKqUrPlcNHZwOrgbYikiQi94rIZBGZ7Njla2AvsBt4B3jQVbFUVzd1bkadGr5M/6nATeP4+dCki50cVh6RsfY+w57vy9eOUqrS83FVw8aY0VfYboBfu+r8nqCGnzd3dA9l2qpEjp3NpFHOITi8AYY8X/7Gm8VAQF3bPRR1a/nbU0pVWjqzuIob1zuMPGOYteaAvUkMdjZxeXn7QJvBdnZyfgnKWSilqixNBFVceIOaDIhsyOyfD2Di50GLPlAn1DmNR8ZC+nE4ssE57SmlKiVNBNXAhD5h1EtLQI7vKP9N4oLaDAZERw8pVc1pIqgGBkSGMLbmWvLxgg63OK/hwPoQ2kPnEyhVzWkiqAa8BW7xWcPKvCh2pPk7t/HIofYGdNoVqp0qpaosTQTVwaH11M5M4lv6Mr20S1leyYXFahY5t12lVKXhOYkgLRmW/RPy89wdifPFzwcvX7yjbuLTDYc4m5njvLYbd4KgJto9pFQ15jmJIHElLPs7bJzl7kicKz/fziaOGMKdV3UiIzuP+euSnNe+CEQMgT1LIc+JCUYpVWl4TiKIuhVCe8L3z0NWMXX8q5oDqyD1CHS8nU6hdejSvC7/W72f/HwnlpCOiIWss3DgJ+e1qZSqNDwnEYhA7N8h7Rj8+Iq7o3Ge+Pl2IZm2wwG4q28Ye1PS+XFPivPO0WogePlq95BS1ZTnJAKA5j2g4whY9RqccWL3ibvk5cC2zyByGPjVBOC6Tk0Irunn3JvG/rUgvJ/OJ1CqmvKsRAAw+Bkw+bDECfV43G3vcsg4AZ1GXHjK38ebkT2as2T7MQ6dduIC9BGxkLITTiU6r02lVKXgeYmgbgvo82vYPAcOrXN3NOUTPx/86zhmAP9ibO8wAGb+5MSrgkgdRqpUdeV5iQCg/xNQsyEs/GPVXZc3JxN2fAntbwCfiyeRNatbg8HtGzFn7UEyc5w0XDa4NdRvBbv0PoFS1U2JEoGIPCoitR2LyLwnIutFZKirg3MZ/yC49k924ZXtn7s7mrLZvciO5CmittCEPuGcTM/m6y1OXOsnIhYSV0B2hvPaVEq5XUmvCO4xxpwFhgINgbuBF10WVUXoOh5ComDRXyA3y93RlF78fAgMhpYDCt3cr00wrRrWdO5N48ihkJtpk4FSqtooaSI4v/jtdcAHxphNFL74fNXh5Q2xL9ibnz9PdXc0pZOVBju/tQXmvAtfW0hEGN87jI0HT7M56bRzzhvWD3xraveQUtVMSRPBOhH5DpsIFopIEFD1Vytpfa1dm3f5vyH9hLujKbmd30DuuYtGCxXm9u6hBPp5O++qwMffzilI+K7q3ltRSl2mpIngXuBpoIcxJgPwxXYPVX1DnofsNFj2D3dHUnLx8yGoKTTvXexutQN8ubVrM77YdJhT6dnOOXfkUDhzEJK3O6c9pZTblTQR9AF2GmNOi8g44E/AGdeFVYFC2kHM3RD3Phzf6e5orizjJOxebJej9LryX9+EPuFk5eYzN+6gc84f4RgjkKCTy5SqLkqaCN4EMkSkM/A7YD8w3WVRVbSBvwe/WvDdn90dyZXt+BLyc0q8ElnbxkH0bFmfGWv2k+eM+kO1m0KjTpoIlKpGSpoIco0xBrgZeMUY8woQ5LqwKljNBnD1b20tnT3fuzua4sXPh3otoWnXEh8yoU8YB0+eY/kuJy0uEznUFqA7d8o57Sml3KqkiSBVRH4PjAe+EhFv7H2C6qPXr6BuGCz8U+VdsyD1GOz7wd4klpIP2oqNakxIkD8frnLSTeOIWDB5tjS1UqrKK2kiGAlkYecTHAWaAf92WVTu4OMPQ/4KyVthwwx3R1O4bZ/ZOkmlXKDe19uLMb1asHzXcRJT0ssfR2gM1Kin3UNKVRMlSgSOD/+ZQB0RuQHINMZUn3sE53W42Y7E+f4FyEp1dzSXi58HIR0gpH2pDx3TswU+XsIMZ9Qf8vK29Y0SFtmFcZRSVVpJS0zcCfwM3AHcCawRkeIHsVdFIjDs75CeDCv/4+5oLnb6ABxcU+qrgfNCagcQ27Exc+MOci7bCV1fEbGQkQKH15e/LaWUW5W0a+iP2DkEdxljJgA9gSowxKYMmnWH6JGw6r/2w7ey2PqJ/dnxtjI3cVefcM5m5vLZxkPlj6fNIBAv7R5SqhooaSLwMsYUHHJyohTHVj2D/mKvDpb81d2R/GLLPGjazVYALaMe4fVo1ziI6av3Y8o7MziwPoT20HITSlUDJf0w/1ZEForIRBGZCHwFfO26sNysTij0fRi2fAxJce6OBlIS4OjmK5aUuBIRYXyfMLYdOcv6A04Y+hkxFI5shNSj5W9LKeU2Jb1Z/CQwFYgGOgNTjTFPuTIwt+v3GNRqBAv/4P66OvELAIGoW8vd1C1dmhHk7+Oc+kPnF6vZvbj8bSml3KbE3TvGmPnGmCeMMY8bYz5xZVCVgn8tu2bBwTW/9M+7gzF2tFBYXzurt5xq+vtwe/dQvt5yhOOp5Sy/3aijrXmk3UNKVWnFJgIRSRWRs4X8SRWRs1dqXESGichOEdktIk8Xsr2OiHwhIptEZKuIVK5Cdl3G2nIKi5+xK4K5w7F4SNlV5tFChRnfJ4ycPMN7K/eVr+yECEQMsRPLcp1U1E4pVeGKTQTGmCBjTO1C/gQZY2oXd6xj9vHrwHCgAzBaRDpcstuvgW3GmM7AQOD/RMSvzK/G2c6vWXD6AKx5yz0xxM8H8bZzHJykdcNaDG4fwlvL99D9hUU8/tFGvtx8mLOZOaVvLDIWslPtam9KqSqp8FVNnKMnsNsYsxdAROZgaxVtK7CPAYJERIBawEkg14UxlV6rgRA5HFb8n71CqNWw4s5tjE0Era+x9ZCc6LXR3fh+RzJLth9j6c5kPtlwCB8voWfL+gxq34hB7UIIb1Dzyg21HADefnYYaavCV0tTSlVuUu5hhEU1bCecDTPG3Od4PB7oZYx5qJ+TXigAACAASURBVMA+QcDnQDtsEbuRxpivCmlrEjAJoEWLFt3373fi8oslkZIAb/SGbnfBDS9X3HkProX3BsMtb0KXMS47TV6+YcOBUyzebhNDQnIaAK0b1ryQFLqH1cPHu4gLyOm3wNlD8NBal8WolCofEVlnjIkpbJsrrwgKq4p2adaJBTYC1wKtgUUissKxPvIvBxkzFTtqiZiYmIofwtMgAmLuhbXvQM/7y1TioUzi54O3P7S73qWn8fYSYsLrExNen6eHt+PAiQyW7DjG9zuS+eDHfUz9YS91avgysG1DBrVvxIDIhtSpUaDmYGQsfPs0nNwH9Vu6NFallPO5MhEkAc0LPA4FDl+yz93Ai44S17tFZB/26uBnF8ZVNgOfhs1z4Ls/wbj5rj9ffh5sXWBvxgbUcf35CmgRHMjd/Vpyd7+WpGbmsCIhhSXbk1m6M5nPNh7G20voEV6Pwe0bcW27EFpFDLWJIGER9JpUobEqpcrPlYlgLRAhIi2BQ8Ao4NL+jQPAIGCFiDQC2gJ7XRhT2QXWh6t/B9/9ERIWQ8Rg155v/4+Qdsypo4XKIijAl+s6NeG6Tk3IyzdsPHiKJduTWbI9mRe+2s4LX22nVYOafOzfHK9NX1Ir5j58i+pCUkpVSi67RwAgItcBUwBv4H1jzN9EZDKAMeYtEWkKTAOaYLuSXjTGFFsDOiYmxsTFuWm2b242vNHLdtdMXgneLsyjnz9iy0o8uRv8Al13nnI4eDKD73cks3j7Ma5N/A9jvBZztbxHr7YtGNQ+hIGRIdQJrF7LVihVVRV3j8ClicAV3JoIALZ/AR+Ngxv+AzH3uOYcudnwf5G21PPt77rmHE52bsdiasy5nfdb/IPXD0VwIj3b3nsIq8eg9iEMat+IVg1qIqVYUEcp5TzuullcPbW7AcL6wfd/s902rui/37vMLgPp5m6h0qjRpj/41eKekAQmTnyAjUmn+X67vVr4+9c7+PvXOwgPDrSjkNqH0CO8vnYhKVVJ6BVBWRzeAFMH2npEQ55zfvsLJsGub+G3CXbltKpizlg4vBEej79oKc1Dp8/x/fZjLN6ezOo9J8jOyycowIcBkQ0Z1D6E2KjGBPrpdxKlXEmvCJytaVfoPBp+egNi7oZ64c5rO+cc7PjKFpirSkkAbDXSHV9C8jZoFHXh6WZ1azC+Tzjj+4STnpXLyt0pLNl+jO93HOfLzUeIDk1kzqTemgyUchO9Ni+ra/9sSz8sdvIVQcJ3kJ1WpbqFLogYan8Ws1hNTX8fYqMa868Rnfn5D4N4dXRX4g+d4ZHZG8tX90gpVWaaCMqqTjPo96gd63/QidMetsyDmiHQ8mrntVlRajeBxp1gV8lWLfPyEm7q3JRnboxi8fZjPP/ltisfpJRyOk0E5dHvEajVGL79vXPWLMg8a79NR91iC95VRRGxtnT3uZIvfHNX33Duvaol01Yl8v7KfS4MTilVGE0E5eFX0y5reSjOloMor53fQG5m1ewWOi8yFkwe7F5SqsP+cF17YqMa8fxX2/huq654plRF0kRQXp1HQ+NoWPysvdFbHvHzoE5zCO3plNDcoll3qFHflpsoBW8vYcrIrkSH1uWRORvYdPC0iwJUSl1KE0F5eXlB7N/gzEE7iqisMk7Cnu/taCGvKvzX4uVtJ8LtXmTrJZVCDT9v3p0QQ4Na/tz74VoOnsxwUZBKqYKq8CdOJdLyamh7Pax4GdKSy9bG9s8hP7dqdwudFxkLGSfg0PpSH9owyJ9pd/cgOzefu6et5UxGGRbLUUqViiYCZxnyV9u/v/RvZTt+yzwIbgNNOjs3LndofS2IV7HDSIvTJiSIt8fHsP9EOr+aEUd2br6TA1RKFaSJwFkatIGek2D9dDi2tXTHph6FxJX2aqA61OIJrG/vcySUfVH7Pq2D+deIaH7ae5Kn52+mqs2AV6oq0UTgTFc/Cf61YeEfSzecdOsngKke3ULnRQ6FI5tskiujW7uG8sSQSBZsOMQrSxKcGJxSqiBNBM4UWN8uYLN3aelGzcTPh0adoGFb18VW0SJi7c8ydg+d9/C1bRjRPZQpixOYty7JCYEpVUppyZCe4u4oXEoTgbPF3Av1W9sFbPJKcKPzVCIkrYWOt7k8tArVKApqNyt3IhAR/n5rJ/q2Dubp+ZtZtbt6/4dUlUzOOXhnELzWDbZ/6e5oXEYTgbP5+MHQ5yFlF6ybduX94xfYn9WpWwjsvY6IIbBnmV1foRz8fLx4c1x3Wjaoya9mrCPhWKpzYlTqSlb/F84csGVfPhoL3zxd7n/PlZEmAldoex2E94dl/4BzV5gYFb8AQntAvbCKia0iRcRCdiocWFXupurU8OWDu3sQ4OvNxA/Wkpya6YQAlSpG6lFY8R+7BskDP0KvybDmTXh/KJysXqVQNBG4goidZJZxEla8VPR+x3fCsS3V72rgvFYDwNuv1LOMixJaL5D37orhZHo2930YR0Z2rlPaVapQ3z8Pedl2aLiPPwz/J9z5PzixF94eANs+d3eETqOJwFWadIYuY2HN23Byb+H7xM8HxM4mro78akL4VbCr7MNILxUdWpfXtHS1crUjm2DDTOg9GYJb//J8h5tg8g/2ubnj4esnITfLfXE6iSYCV7r2T+DlY+sQXcoYmwjCr4KgxhUeWoWJiIUTCUUnwzIY3KHRhdLVL3ylpauVkxkD3/7BjgK8+snLt9cLh3sWQu8H4eep8N5Qp/77dgdNBK5Uu4ldznLbZ7B/9cXbjm6GE7urb7fQeRFD7M8SrlFQUudLV3/wo5auVk6240vYvxKu+UPRa5L7+MGwf8CoWXBqn+0q2vppxcbpRJoIXK3vQxDUFBb+AfILlErYMs9eLXS42X2xVYTg1rZ0RjmHkRZGS1crp8vNgu/+DA3bQ7eJV96/3fXwqxXQIBI+vgu++g3kVL2BDJoIXO38mgWH19sy02ATwtZPbE2ewPruja8iRMTaEhrZ6U5t9kLp6mZ1tHS1co6fp9pv+LEvgHcJ19CuFwZ3fwN9HoK178J7Q+DEHtfG6WSaCCpC9Eho0sXeK8jOgKSfbdnqjiPcHVnFiBwKeVmwd7nTm67h5827d/XQ0tWq/NJTYPm/oc0QW0q9NHz87EjB0XPg9AHbVeSMxaoqiCaCiuDlBbF/h7OHYPXr9h+ITwC0He7uyCpGi77gV8sl3UOgpauVkyz7B2Sn2Q/0smo7HCavhJB2MO8e+PLxKtFVpImgooT3g/Y3wsr/2EQQMRQCars7qorh4wetBtpE4KIqogVLV0+esU5LV6vSSd4OcR9Aj3vLX/OrbnPbVdT3EYh7H94dDCm7nROni2giqEiDn7MTVDJOQCcP6RY6LzLWXhGVtkR3KZwvXb167wmeXqClq1UpLPwj+NeCgb93TnvevrbUzJi5cDYJpg6wA0QqKU0EFSm4tR1FVDPEXhF4kjaOYaTlWKOgJC6Url6vpatVCSUsgj1LYMBTzh+8ERlru4oaRcH8e+GLR8u/trkLaCKoaIOegce2gG8Nd0dSsWo3gcbRTis3URwtXa1KLC/HXg3Ubw097nfNOeqEwsSv7JyiddMcXUWV60uKJoKKJgK+Ae6Owj0iY+HgGluDyYW0dLUqsXXTIGWn7cbx8XPdebx9YchzMOZjOHvYjiraPNd15yslTQSq4kTEgsmHPd+7/FRaulpd0blTsPTv0PJqWzG4IkQOtV1FTaJhwf3w+cN2SLmbuTQRiMgwEdkpIrtF5Oki9hkoIhtFZKuIOH+guao8mnWDwGCnFqErjpauVsX64SWbDGL/XrFrhddpBnd9CVc9Ydc4f3cQHN9VcecvhMsSgYh4A68Dw4EOwGgR6XDJPnWBN4CbjDFRwB2uikdVAl7edqLO7sWQn1chp9TS1apQJ/bYysDdxkPjThV/fm8fGPwMjJ0Pacdg6kDYNKfi43Bw5RVBT2C3MWavMSYbmANcWlhnDLDAGHMAwBiT7MJ4VGUQMRTOnYRD6yrslAVLVz86R0tXK2w9IR9/uOZP7o0jYrDtKmraBT75FXz6a7d0FbkyETQDDhZ4nOR4rqBIoJ6ILBORdSIyobCGRGSSiMSJSNzx48ddFK6qEG0GgXhVWPfQeedLVy/apqWrPd6+H2DnV9D/CQhq5O5ooHZTmPC5LXm9cSa8cy0k76jQEFyZCArrdLv0q5gP0B24HogF/iwikZcdZMxUY0yMMSamYcOGzo9UVZwa9aB5L5eVmyjOXX3Duaeflq72aPl5dq2BOi2g96/dHc0vvH3s+iXj5kP6cXjnGtg4q8JO78pEkAQ0L/A4FDhcyD7fGmPSjTEpwA9AZxfGpCqDiKF2PYazRyr81H+8vj1DO2jpao+1caZdHnbIc5VzGHebQbarqFl3+PQB+OQBp1ftLYwrE8FaIEJEWoqIHzAKuHSRz8+A/iLiIyKBQC9guwtjUpVBZKz96YarAm8v4ZVRWrraI2WlwpLnoXnvyr08bO0mMOEzO9N502yYeo2theRCLksExphc4CFgIfbDfa4xZquITBaRyY59tgPfApuBn4F3jTHxropJVRIhHaB2qFsSAVxaujpOS1dXMsYYViQc574P47jjrVUs3nbMOXWjVrwM6ckwrIKHi5aFl7ddIW38J3ZwxdRrYMMMlxVtlKpWmCsmJsbExcW5OwxVXl88Bls+ht/ttaM33GB3ciq3vbGKkNoBzH+gL3Vq+LolDmWlZ+WyYH0S01Ylsud4Og1q+VHDz5uDJ8/RpXldfju0Lf3aBCNl+RA/tR/+2wOiboHbpjo/eFdKPQrz74PEFXbuweBnytSMiKwzxsQUtq2ES/Ao5WSRsbDuA9i/ClpfU3HnPXfKfiic3k+bU/v5LnInO3bEc+LlNGoO/w0+3cZUXCwKgMSUdKav3s/HcQdJzcolOrQOL9/Zmeujm+Alwrx1Sby6JIFx762hd6v6PBnblu5hpSwOt/hZO1ptUNk+RN0qqLHtKvrhJTsz2QU0ESj3aHk1ePvb7iFnJoKcc3aFqFOJFz7wOZXo+HkAss5ctHvjgDoE1GvK0VNZ+Hz+APk5aXj1muS8eFSh8vMNK3an8OGqRJbuTMZbhOs6NWFiv3C6Nq970bf+0T1bcGvXZsz++QCvL93N7W+uZmDbhvx2aFs6NiticfmCDqyBrQtsn3udS0ewVxFe3jDwKZc1r11Dyn3+d5v9gH64FJPL8nJtffcLH/KX/Ew7dvH+PgFQN8yuK1vYzxp1AXhzyTZaL3uIod7rMIOfQ656zIkvVJ2XlpXL/HVJfLg6kb3H02lQy5+xvVowtlcLQmpfeRRPRnYu01Yl8vbyvZw5l8Pwjo15fEgkkY2CCj8gPx/eG2wLvT28zq4h7qG0a0hVTpGx8M3v7HT/4Nb2OWPsh/lFH/CJvzw+cwhMgfIU4m2/5dULt8NS64VB3fBfPuhrhZToxuDka9vzfNp/yIx7ipsWP2OH7F3zh8p/U7GK2JeSzoerEpm3Lom0rFw6N6/LlJFdGN6pMf4+3iVuJ9DPhwcHtmFc7zDeW7GP91bu49utR7mlSzMeHRRBeINLPujj59lZ7Le85dFJ4Er0ikC5z8l98GoXCO9vv7mf3m+7dXIvKQ5Xq1HR3+prh9rJOE6Qn294ZHYc/be/wEifZdDnIRj6gmclA2Ng+T9tieRb3oQWvcrcVH6+YXnCcT5clciyncfx9Rau79SEu/qG07VFPaeEezI9m7eX7+HD1Ynk5BnujAnl4WsjaFq3hi3V8N8Y+2Xgvu/t2uEerLgrAk0Eyr0+uA6OxV/yAR9e4HGLCl3EJys3j3s/WMOQA1O4y3shdL8brn/ZMz5Ecs7Bpw/a/nS/ILus6m1T7UibUkjNzGHeuiSmr97PvpR0GgbZ7p8xvVoQEuSaSVzJZzN5feluZv18AEEY06sFTwZ8Ss1V/7LrB4f1dcl5qxJNBKryMqbSfeNOzcxh9NTV3JTyLpO8PoPoUXDz60678qiUUo/CnDFwaL0dnth1PMweDUk/w5Dnoe/DV/x72nM8jemO7p/07Dy6tqjLxL7hDO/YBD+fikmkSacyeG3Jblas38xi3yc4ENyPxvd9RN1AFy46U0XoPQJVeVWyJAAQFODLB3f3YsRbeWRn1OChzXMgJwNuf8+1q1i5y5HNMHuUHVo7cga0v8E+f9fn8MlkWPRnO/Jq+L8uS4b5+Yblu47zwapEfthlu39ujG7KXX3D6dy8boW/lNB6gfxzRDSpua/itzOfSUdu5NQ/l3Jf/1bcc1U4QQE6V6QwekWgVBH2n0jn9jdXMc58xWN5H9ib0XdOr17rTe/4CubfDwF1YMwcaHJJqa/8fFjyLPz4in39Iz4A/1qczcxhXlwS01cnkngig5Agf8b1DmN0zxY0DHLPBMELDm+w9f37PcqOTr/lP4t2sXDrMeoF+jJ5QGsm9Amnhl/Jb1BXF9o1pFQZxR86w6ipP3Ff4HIePfcGEn4VjJ4D/rXcHVr5GGM/3Bc/a2vhj5pta9wUZe178PVvyQzuwKuNXmDaliwysvPoHlaPu/qGMyyqcYV1/xTLGHvf6UQCPLweAmoDsDnpNC99t4sfdh2nYZA/D13ThlE9m5dqxFJVp4lAqXJYtTuFiR+s5aEG63j47MtIs+4w9uMLcxCqnNxs+PJx2DjDFl+7+Q3wCyxy9/x8w9KdyWxYMpcHjj/PWWoyo/X/MezaQXQKLcGEroq09VP4+C64YQrE3H3Z5p/3neSl73by876TNKtbg0cHRXBbt2b4eFeCJOZimgiUKqcvNx/m4dkbeLL5Lh5I+RsS0h7Gfwo1g90dWumkn4C542H/j3am7YCnixwRdeZcDh/HHWT66v0cOJlBo9r+PN7xHHfs+i3e2ekwcjq0vraCX0AxcjLh9Z7gVwsmr7CzcQthi9ql8H/f7WRT0hlaNqjJY4MjuDG6KV5ele+elbNoIlDKCT5clcgzn2/lL+0OcXfSn5F64bYGTFBjd4dWMsd3wqw77ToQN78O0YUvEX7sbCavfZ/AgvWHyMjOIyasHhP7hRMb1Rhfby84kwQz74SUnfabd7fxFfxCirByCix+xiboEpQtMcawaNsxXl60ix1HU2nbKIgnhkYytEOjshW2q+Q0ESjlJC8t3Ml/l+7mn93PMHLXb+1kpbs+t/MdKrPdS+Dju+2op1GzoHnPQndbkXCcx+ZsJDUrl5s6N2Vi3/DC6/lknoW5E2DvUuj/W7u6ljs/PNOOw6tdIbwfjPmoVIfm5xu+3HKEKYt2sTclnejQOvxmaFuujmhQrRKCJgKlnMQYw9Pzt/BR3EHeGJDHdRsfAv8gmwzOl8mobH5+B755Chq2syODCklaefmGV5ck8Or3CbRpWIs3xnYjoqj6PRcOyoGvnoD106HTHfYqw00lxfniMdjwP3jwJ2gQUaYmcvPyWbDhEK8sTuDQ6XP0DK/Pb2Pb0rNlKSudVlKaCJRyoty8fCbPWM+SHceYPjyA/j/dD14+tpsopL27w/tFXi58+zSsfQcih8Ht79qkdYnjqVk89tEGftx9gtu6NuOFWzsS6FfCKUbGwMqXYclfoUVfGDUTAiv4g/PYVnjrKuj5Kxj+Yrmby8rNY+7ag7z2/W6SU7PoH9GAJ2PbEh1aRQcHOGgiUMrJMnPyGP/eGjYdPMNHt9al67KJtiTD+E/scEx3O3ca5t0Ne763NZOG/LXQm6dr9p7g4dkbOHMuh7/eHMWdMc3L1h2yZZ5dY7dumB1RVb+lE15ECRgD/7sFDm+ERzY4NQmdy85jxk/7eWPZbk5l5HBDdBOejG1LWHDVLF6niUApFziTkcMdb6/i8OlMFoxqTOS34yDzNIydV65ibeV2ci/MGml/3vAf6Dbhsl3y8w1v/bCHlxbuJCy4Jq+P6UaHprXLd979q2xZCi8fO9eieY/ytVcSO7+F2SNh2D+h92SXnCI1M4d3ftjLOyv2kZufz9heYTx8bRuCa7l54lwpaSJQykWOnDnH7W+sIjvP8Nm4FjT7fJSt2zN6NrQaUPEBJf4IH40DDNz5P2jZ/7JdTqVn88TcjSzdeZzro5vw4m2dnFd6ISUBZo6w78Ft70CHm5zTbmHycuCNPvb3B1eDt2vLRySfzeQ/ixOYG3eQGr7eTB7QinuvalVlZikXlwiq/ywKpVyoSZ0aTL+3J7n5+Yz5+BApd35qq6bOvAN2LazYYDbMgOk3Q2Aw3Lek0CSw/sAprn91BT/uPsHzN0fx39FdnVt/p0EE3LsYGnW0o4pWv+6yBddZ+56dQTz0BZcnAYCQ2gH847ZOLHysP31aB/PSd7sY8O+lzPn5ALl5+S4/vyvpFYFSTrD+wCnGvPMTbUJqMWdcW2rNvcPexLz93VKXcS61/DxbKmLVq9BqINwxDWpcXO/fGMN7K/fx4jc7aFwngDfGdnPtzc+cc7BgEmz/HHpOgmEvFjnBq0wyTtrhok272HkDbhjmuTbxJP/4ejvrD5ymTUgtnhrWjsHtQyrtkFO9IlDKxbq1qMcbY7ux/Ugqv5q/l6yxn0KzbvaG7cbZrjtxVhp8NN4mgZh77f2JS5LAmXM5TJ6xjhe+2s417UL46uH+rh8B41sD7vjQlq/+eaotcZ2V5rz2l/8Lss5C7N/dNn+hR3h95j/Ql7fGdSc/33D/9DhGvv0T6w+ccks85aFXBEo50bx1Sfz2403cEN2EV2+LxOujMbBvub1pG3OPc092JglmjYLkrfYbd89Jl30obkk6w4Oz1nHkdCZPD2/HvVe1rPhvrD+/Y5ckbdwJxswt/0zslAR4o7ddM+HGKc6JsZxy8vL5aO1BpixOICUti+EdG/NkbFtaNaw8xQn1ZrFSFeit5Xt48ZsdTOwbzjPDWyFz74KEhfbba59fO+ckSetgzmi7HOMdH0DEkIs2G2OYseYAz3+xjeBafvx3TFe6h7lxYtTOb+3VUWCwHV5anvkWs0baEUoPr4daDZ0XoxOkZ+Xyzoq9TP1hL1m5+Yzp2YJHBkW4vzQ32jWkVIX61dWtuPeqlkxblcibPx6yi710uAUW/sF2aZT3y1f8fJh2nV3n+b5FlyWBtKxcHpmzkT9/Gk+f1sF89Uh/9yYBgLbD4O6v7VyL94bC3mVla2fPUtj1LfT/TaVLAgA1/X14bHAky5+8hjE9WzDr5wMM/PdSpizeRXpWrrvDK5JeESjlAvn5hsfnbuSzjYf514ho7uzaBD5/GDbNgn6PweBnS9+3fX5h+WX/gBZ9bIKp2eCiXXYcPcuDM9aTeCKd3wxtywMDWleuipqnD9oRVScS4MZXoevYkh+bnwdv9YfsNHhorfvKWZTC3uNp/HvhTr6JP0qDWv48OjiCUT2a2+J9FUyXqlSqgnl5Cf8e0ZmT6dn8fsEW6gf6Mfjm1+1N1B+n2KUvh/2zyBLQl8k5B5/92l4NdB4NN75y2Qfh3LiD/PnTeGrX8GXmfb3p07oSlsiu2xzuXWhvcH/2IJzeDwN/X7KkuH66vR9yx4dVIgkAtGpYizfHdWfd/lO8+M12/vxpPB+s3MfvhrUlNqpxpRlhpFcESrlQWlYuY975iZ1HU5l1fy+6t6hn1wBe9Rp0GQc3vXrlYZWpxxwLy8fBoGfgqscv+uDMyM7lz59uZf76JPq2DuaVUV0rRZ90sQoujhM9yr4PxX24Z56F17pBcITtYqokH6ClYYxh8fZk/vntDnYnp9GtRV1+f117eoRXTLed3ixWyo1S0rK4463VnEzPZt7kPkSE1PqliyfqNrhtatEToo5usSODzp20+7W/8aLNu5NTeXDmehKS03jk2ggeGRSBd2XqCiqOMfDDS7D0BQjvDyP/d9nQ1wsWPWOX1py0FJp2rdg4nSw3L59565J4edEuklOzGNKhEU8Na0ubkCtUey0nTQRKudnBkxnc9uYqfLyE+Q/0pWndGvDjq/bqIHK4nQTmG3DxQTu+hvn3Fbmw/GcbD/H7BVuo4evNlFFd6B9R+W6elsimj2y3V/2WdkRRvfCLt5/cZ1ce6zgCbn3TLSG6QkZ2Lu+v3Mdby/eSkZ3LyB7NeWxwJI1qB1z54DLQRKBUJbDt8FlGvr2aRnUCmDe5D3UD/WDtu/DVb6DVNbaEs19N+0151Wuw6C+FLiyfmZPHX7/cxqw1B+gRXo/XRnejcR3XfHhUmMSVtvvL2w9GfwSh3X/ZNncCJCyyw0ULvA/VxYm0LF77fjcz1+zHx8uL+/q3ZNLVrZxb+gM3Dh8VkWEislNEdovI08Xs10NE8kRkhCvjUcqdOjStzdQJMRw4kcE909ZyLjsPetwHt7xpJ53NuN2uKfz5Q/ZKocPNMPHriz78ElPSue2NVcxac4DJA1oz+/7eVT8JAIRfBfcusjfTp10P27+0z+9fBds+syOtqmESAAiu5c+zN0Wx+IkBDGofwmvf72bAv5cx7cd9ZOdWTA0jl10RiIg3sAsYAiQBa4HRxphthey3CMgE3jfGzCuuXb0iUFXdN1uO8OCs9VzbNoS3x3fHx9sLtn5iu4G8fCH3HFz9OzuapsCoom+2HOF38zbj5SW8fGdnBrVv5MZX4SJpyTB7FBxabyfgbZlrn3soDvwC3R1dhdh08DT/+GY7P+09SVhwIE/GtuX6Tk3KPcLIXVcEPYHdxpi9xphsYA5wcyH7PQzMB5JdGItSlcbwTk14/uaOLNmRzO8XbMEYA1G3wsiZtvzCbe/AtX+8kASyc/N57outPDBzPa1CavHVI1dVzyQAjjWgv4R218PC38PhDXbOhYckAYDOzesy+/7efDCxBwE+3jw0awO3vP4jq/eccNk5XTmPoBlwsMDjJOCi1TpEpBlwK3AtUOQqFiIyCZgE0KJFJV8kXKkSGNc7jOOpWbyyJIGGQf78blg7O/u27bCL9ks6lcGvZ21g08HT3NOvJU8Pb4efTzUvCOAXCHdOh6V/sxPQOnpej7GIcE27JTvhDAAABn1JREFUEK6ObMj89Un8Z9EuRr/zEw8MbM1Tw9o5/XyuTASFXcdc2g81BXjKGJNX3GWPMWYqMBVs15DTIlTKjR4bHEFyahZvLNtDg1r+3HPVxcs7Ltl+jCfmbiI/3/Dm2G4M71Q9+8gL5eUNg/7i7ijczttLuDOmOTd1bsoHPybSI7yI4bXl5MpEkAQ0L/A4FDh8yT4xwBxHEmgAXCciucaYT10Yl1KVgojwwi0dOZmexV+/3EaDIH9u6tyUnLx8XvpuJ28v30tU09q8MbZblV0nVzlHgK83Dwxs7bL2XZkI1gIRItISOASMAsYU3MEYc+ErkIhMA77UJKA8ibeX8Mqorkx4/2d+M3cjefn5zFpzgLWJpxjbqwV/vqEDAb5VYylEVXW5LBEYY3JF5CFgIeCNHRG0VUQmO7a/5apzK1WVBPh6886EGEa+vZrHP9pEoJ83r4zqws1dmrk7NOUhdEKZUpXEsbOZvLlsD+N6h9EmpPIsaKKqB60+qlQV0Kh2AM/eFOXuMJQHqubj0JRSSl2JJgKllPJwmgiUUsrDaSJQSikPp4lAKaU8nCYCpZTycJoIlFLKw2kiUEopD1flZhaLyHFgfxkPbwCkODGcqk7fj4vp+/ELfS8uVh3ejzBjTKELW1e5RFAeIhJX1BRrT6Tvx8X0/fiFvhcXq+7vh3YNKaWUh9NEoJRSHs7TEsFUdwdQyej7cTF9P36h78XFqvX74VH3CJRSSl3O064IlFJKXUITgVJKeTiPSQQiMkxEdorIbhF52t3xuJOINBeRpSKyXUS2isij7o7J3UTEW0Q2iMiX7o7F3USkrojME5Edjn8jfdwdk7uIyOOO/yPxIjJbRALcHZMreEQiEBFv4HVgONABGC0iHdwblVvlAr8xxrQHegO/9vD3A+BRYLu7g6gkXgG+Nca0Azrjoe+LiDQDHgFijDEdsWuvj3JvVK7hEYkA6AnsNsbsNcZkA3OAm90ck9sYY44YY9Y7fk/F/kf32JXSRSQU+P/27ie0jiqO4vj3SFSa1L+gC1sxrYqIoE3diEER404RFxVBG8S1FrpSKorgWsWNaKEqlWYh1hRcCBYrBLqwLY3RYt2ptE8r7UIjFdRaj4u5wjONJYLJDbnns3pzmTecgTfvN3Pnvd/cD+yonaU2SZcCdwNvAtj+3fZPdVNVNQCskjQADALfV86zKFopBGuA433LPRr+4usnaRgYAQ7UTVLVq8DTwJ+1gywD64FTwNtlqmyHpKHaoWqw/R3wEnAMOAHM2t5bN9XiaKUQaJ6x5n83K2k18D6w1fbPtfPUIOkB4KTtw7WzLBMDwEbgddsjwC9Ak/fUJF1BN3OwDrgGGJK0uW6qxdFKIegB1/Ytr2WFXuItlKQL6YrAhO3J2nkqGgUelPQt3ZThvZJ21Y1UVQ/o2f77CnE3XWFo0X3AN7ZP2T4DTAJ3Vs60KFopBIeAGyWtk3QR3Q2fDypnqkaS6OaAv7L9Su08NdneZnut7WG6z8UntlfkWd9C2P4BOC7ppjI0BhytGKmmY8AdkgbLMTPGCr1xPlA7wFKw/Yekp4CP6O78v2X7y8qxahoFxoEjkmbK2LO2P6yYKZaPLcBEOWn6Gniicp4qbB+QtBuYpvul3Wes0FYTaTEREdG4VqaGIiLiX6QQREQ0LoUgIqJxKQQREY1LIYiIaFwKQcQSknRPOpzGcpNCEBHRuBSCiHlI2izpoKQZSdvL8wpOS3pZ0rSkfZKuKutukPSppC8k7Sk9apB0g6SPJX1e3nN92fzqvn7/E+VfqxHVpBBEzCHpZuARYNT2BuAs8BgwBEzb3ghMAS+Ut7wDPGP7VuBI3/gE8Jrt2+h61Jwo4yPAVrpnY6yn+6d3RDVNtJiI+I/GgNuBQ+VkfRVwkq5N9btlnV3ApKTLgMttT5XxncB7ki4B1tjeA2D7V4CyvYO2e2V5BhgG9i/+bkXML4Ug4lwCdtre9o9B6fk5652vP8v5pnt+63t9lhyHUVmmhiLOtQ/YJOlqAElXSrqO7njZVNZ5FNhvexb4UdJdZXwcmCrPd+hJeqhs42JJg0u6FxELlDORiDlsH5X0HLBX0gXAGeBJuoe03CLpMDBLdx8B4HHgjfJF39+tcxzYLunFso2Hl3A3IhYs3UcjFkjSadura+eI+L9laigionG5IoiIaFyuCCIiGpdCEBHRuBSCiIjGpRBERDQuhSAionF/AcKCtnl5BuIyAAAAAElFTkSuQmCC\n",
      "text/plain": [
       "<Figure size 432x288 with 1 Axes>"
      ]
     },
     "metadata": {
      "needs_background": "light"
     },
     "output_type": "display_data"
    }
   ],
   "source": [
    "from matplotlib import pyplot\n",
    "# plot train and validation loss\n",
    "pyplot.plot(history.history['loss'])\n",
    "pyplot.plot(history.history['val_loss'])\n",
    "pyplot.title('model train vs validation loss')\n",
    "pyplot.ylabel('loss')\n",
    "pyplot.xlabel('epoch')\n",
    "pyplot.legend(['train', 'validation'], loc='upper right')\n",
    "pyplot.show() "
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 6,
   "metadata": {},
   "outputs": [
    {
     "name": "stdout",
     "output_type": "stream",
     "text": [
      "INFO:tensorflow:Assets written to: saved_model/model3_10epochs_bit/assets\n"
     ]
    },
    {
     "name": "stderr",
     "output_type": "stream",
     "text": [
      "INFO:tensorflow:Assets written to: saved_model/model3_10epochs_bit/assets\n"
     ]
    }
   ],
   "source": [
    "import tensorflow as tf \n",
    "# Save fine-tuned model as SavedModel\n",
    "export_module_dir = 'saved_model/model3_10epochs_bit'\n",
    "tf.saved_model.save(model, export_module_dir)"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "### Experiment 2: "
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 4,
   "metadata": {},
   "outputs": [
    {
     "name": "stdout",
     "output_type": "stream",
     "text": [
      "WARNING:tensorflow:From /opt/conda/lib/python3.7/site-packages/tensorflow_core/python/ops/resource_variable_ops.py:1781: calling BaseResourceVariable.__init__ (from tensorflow.python.ops.resource_variable_ops) with constraint is deprecated and will be removed in a future version.\n",
      "Instructions for updating:\n",
      "If using Keras pass *_constraint arguments to layers.\n"
     ]
    },
    {
     "name": "stderr",
     "output_type": "stream",
     "text": [
      "WARNING:tensorflow:From /opt/conda/lib/python3.7/site-packages/tensorflow_core/python/ops/resource_variable_ops.py:1781: calling BaseResourceVariable.__init__ (from tensorflow.python.ops.resource_variable_ops) with constraint is deprecated and will be removed in a future version.\n",
      "Instructions for updating:\n",
      "If using Keras pass *_constraint arguments to layers.\n"
     ]
    },
    {
     "name": "stdout",
     "output_type": "stream",
     "text": [
      "Model sucessfuly built\n",
      "callbacks and data okay\n",
      "Train on 10 steps, validate on 4 steps\n",
      "WARNING:tensorflow:From /opt/conda/lib/python3.7/site-packages/keras/callbacks/tensorboard_v1.py:200: The name tf.summary.merge_all is deprecated. Please use tf.compat.v1.summary.merge_all instead.\n",
      "\n"
     ]
    },
    {
     "name": "stderr",
     "output_type": "stream",
     "text": [
      "WARNING:tensorflow:From /opt/conda/lib/python3.7/site-packages/keras/callbacks/tensorboard_v1.py:200: The name tf.summary.merge_all is deprecated. Please use tf.compat.v1.summary.merge_all instead.\n",
      "\n"
     ]
    },
    {
     "name": "stdout",
     "output_type": "stream",
     "text": [
      "WARNING:tensorflow:From /opt/conda/lib/python3.7/site-packages/keras/callbacks/tensorboard_v1.py:203: The name tf.summary.FileWriter is deprecated. Please use tf.compat.v1.summary.FileWriter instead.\n",
      "\n"
     ]
    },
    {
     "name": "stderr",
     "output_type": "stream",
     "text": [
      "WARNING:tensorflow:From /opt/conda/lib/python3.7/site-packages/keras/callbacks/tensorboard_v1.py:203: The name tf.summary.FileWriter is deprecated. Please use tf.compat.v1.summary.FileWriter instead.\n",
      "\n"
     ]
    },
    {
     "name": "stdout",
     "output_type": "stream",
     "text": [
      "WARNING:tensorflow:Issue encountered when serializing variables.\n",
      "Type is unsupported, or the types of the items don't match field type in CollectionDef. Note this is a warning and probably safe to ignore.\n",
      "'list' object has no attribute 'name'\n"
     ]
    },
    {
     "name": "stderr",
     "output_type": "stream",
     "text": [
      "WARNING:tensorflow:Issue encountered when serializing variables.\n",
      "Type is unsupported, or the types of the items don't match field type in CollectionDef. Note this is a warning and probably safe to ignore.\n",
      "'list' object has no attribute 'name'\n"
     ]
    },
    {
     "name": "stdout",
     "output_type": "stream",
     "text": [
      "Epoch 1/50\n",
      " 9/10 [==========================>...] - ETA: 29s - loss: 1.2054 - acc: 0.6838WARNING:tensorflow:From /opt/conda/lib/python3.7/site-packages/keras/callbacks/tensorboard_v1.py:343: The name tf.Summary is deprecated. Please use tf.compat.v1.Summary instead.\n",
      "\n"
     ]
    },
    {
     "name": "stderr",
     "output_type": "stream",
     "text": [
      "WARNING:tensorflow:From /opt/conda/lib/python3.7/site-packages/keras/callbacks/tensorboard_v1.py:343: The name tf.Summary is deprecated. Please use tf.compat.v1.Summary instead.\n",
      "\n"
     ]
    },
    {
     "name": "stdout",
     "output_type": "stream",
     "text": [
      "10/10 [==============================] - 444s 44s/step - loss: 1.1833 - acc: 0.6975 - val_loss: 0.6688 - val_acc: 0.8225\n",
      "Epoch 2/50\n",
      "10/10 [==============================] - 389s 39s/step - loss: 0.9255 - acc: 0.7939 - val_loss: 0.5701 - val_acc: 0.7659\n",
      "Epoch 3/50\n",
      "10/10 [==============================] - 389s 39s/step - loss: 0.6380 - acc: 0.7982 - val_loss: 0.4087 - val_acc: 0.7958\n",
      "Epoch 4/50\n",
      "10/10 [==============================] - 391s 39s/step - loss: 0.4879 - acc: 0.8201 - val_loss: 0.5079 - val_acc: 0.7932\n",
      "Epoch 5/50\n",
      "10/10 [==============================] - 392s 39s/step - loss: 0.4517 - acc: 0.8260 - val_loss: 0.5724 - val_acc: 0.7484\n",
      "Epoch 6/50\n",
      "10/10 [==============================] - 394s 39s/step - loss: 0.3993 - acc: 0.8439 - val_loss: 0.4144 - val_acc: 0.8010\n",
      "Epoch 7/50\n",
      "10/10 [==============================] - 387s 39s/step - loss: 0.2929 - acc: 0.8758 - val_loss: 0.4796 - val_acc: 0.7698\n",
      "Epoch 8/50\n",
      "10/10 [==============================] - 392s 39s/step - loss: 0.4633 - acc: 0.8152 - val_loss: 0.3785 - val_acc: 0.8212\n",
      "Epoch 9/50\n",
      "10/10 [==============================] - 390s 39s/step - loss: 0.3895 - acc: 0.8527 - val_loss: 0.6328 - val_acc: 0.7211\n",
      "Epoch 10/50\n",
      "10/10 [==============================] - 390s 39s/step - loss: 0.4255 - acc: 0.8363 - val_loss: 0.5637 - val_acc: 0.7646\n",
      "Epoch 11/50\n",
      "10/10 [==============================] - 393s 39s/step - loss: 0.3621 - acc: 0.8527 - val_loss: 0.4878 - val_acc: 0.7841\n",
      "Epoch 12/50\n",
      "10/10 [==============================] - 393s 39s/step - loss: 0.3415 - acc: 0.8592 - val_loss: 0.7952 - val_acc: 0.6827\n",
      "Epoch 13/50\n",
      "10/10 [==============================] - 394s 39s/step - loss: 0.3679 - acc: 0.8535 - val_loss: 0.4315 - val_acc: 0.8062\n",
      "Epoch 14/50\n",
      "10/10 [==============================] - 392s 39s/step - loss: 0.3530 - acc: 0.8576 - val_loss: 0.4554 - val_acc: 0.8056\n",
      "Epoch 15/50\n",
      "10/10 [==============================] - 395s 39s/step - loss: 0.2668 - acc: 0.8889 - val_loss: 0.4049 - val_acc: 0.8108\n",
      "Epoch 16/50\n",
      "10/10 [==============================] - 392s 39s/step - loss: 0.2797 - acc: 0.8799 - val_loss: 0.4435 - val_acc: 0.8088\n",
      "Epoch 17/50\n",
      "10/10 [==============================] - 395s 39s/step - loss: 0.2721 - acc: 0.8816 - val_loss: 0.4236 - val_acc: 0.8043\n",
      "Epoch 18/50\n",
      " 9/10 [==========================>...] - ETA: 26s - loss: 0.2356 - acc: 0.9036Restoring model weights from the end of the best epoch\n",
      "10/10 [==============================] - 398s 40s/step - loss: 0.2402 - acc: 0.9020 - val_loss: 0.4726 - val_acc: 0.7848\n",
      "Epoch 00018: early stopping\n",
      "Training model is finished\n",
      "5/5 [==============================] - 180s 36s/step - loss: 0.4060 - acc: 0.8259\n",
      "Test accuracy: 0.82585365\n",
      "CPU times: user 10h 14min 31s, sys: 2h 20min 14s, total: 12h 34min 46s\n",
      "Wall time: 2h 4min 1s\n"
     ]
    }
   ],
   "source": [
    "%%time\n",
    "history,model = BiT_L_ResNet.train_Bit_L(train,validation,test,model_name=\"r101x1\",epochs=50)"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 5,
   "metadata": {},
   "outputs": [
    {
     "data": {
      "image/png": "iVBORw0KGgoAAAANSUhEUgAAAYcAAAEWCAYAAACNJFuYAAAABHNCSVQICAgIfAhkiAAAAAlwSFlzAAALEgAACxIB0t1+/AAAADh0RVh0U29mdHdhcmUAbWF0cGxvdGxpYiB2ZXJzaW9uMy4yLjEsIGh0dHA6Ly9tYXRwbG90bGliLm9yZy+j8jraAAAgAElEQVR4nOzdd3hUZfrw8e+dRkISSkihBkKvoWNApKOIXVFB1MXGT13XssWyurZV19XV166rK3YRG1ZQEVFUpEtLQFqAhBJIKIEESHveP54JhpCESTJnZpK5P9eVKzNzzpxzz8lk7nm6GGNQSimlygrydQBKKaX8jyYHpZRSJ9DkoJRS6gSaHJRSSp1Ak4NSSqkTaHJQSil1Ak0OymNE5HURecjNfbeIyBgHY5ksIt84dXwnicj9IvK263aiiBwSkeCT7VvDc6WKyIiaPr+K434vItd6+rjKe0J8HYBS5YnI60CmMeaemh7DGPMO8I7HgvIRY8w2IMoTx6rouhpjenji2Kr+0ZKDqnNERL/UKOUwTQ4BxlWd8zcRWSUieSLyqogkiMhsETkoIt+KSNMy+5/rqnrY76oq6FZmW18RWe563gwgvNy5zhaRFa7nLhCRZDfimwpMBm53Vad8XibuO0RkFZAnIiEicqeIbHKdP01ELihznCki8lOZ+0ZErheRDSKyT0SeFxGp4PwtReSwiMSUe53ZIhIqIh1F5AcROeB6bEYlr+MrEbmp3GMrReRC1+2nRSRDRHJFZJmInFbJcdq5Yg9x3U9ynf+giMwBYsvt/4GI7HLFN19EerhxXce4bjcQkadEZIfr5ykRaeDaNkJEMkXkLyKyW0R2ishVFf8VT3gNQSJyj4hsdT33TRFp7NoWLiJvi0iO632yREQSXNumiMhm12tNF5HJ7pxPeYgxRn8C6AfYAiwEEoBWwG5gOdAXaAB8B9zn2rczkAeMBUKB24GNQJjrZytwm2vbBKAQeMj13H6uY58CBAN/cJ27QZk4xlQS4+ulxykX9wqgDRDheuxioCX2S86lrlhbuLZNAX4q83wDfAE0ARKBPcC4Ss7/HXBdmfuPAy+5bk8H7nadMxwYWskxrgR+LnO/O7C/zOu/HGiGrdr9C7ALCHdtux9423W7nSv2ENf9X4AnXX+rYcDB0n1d268Gol3bnwJWuHFdx7huP+h6b8QDccAC4J+ubSOAItc+ocB4IB9oWsnr/x64tkxMG4H22Cqyj4G3XNv+D/gcaOh6n/QHGgGRQC7QxbVfC6CHr/9/AulHSw6B6VljTJYxZjvwI7DIGPOrMeYoMBObKMB+4H5pjJljjCkE/gNEAEOAFOyHxFPGmEJjzIfAkjLnuA74rzFmkTGm2BjzBnDU9byaesYYk2GMOQxgjPnAGLPDGFNijJkBbAAGVfH8R40x+42tx58H9Klkv3eBSQCu0sVE12NgE2BboKUx5ogx5qeKD8FMoI+ItHXdnwx87LrGGGPeNsbkGGOKjDFPYD/Mu1T14kUkERgI/MMYc9QYMx/7wXqMMWaaMeag6zz3A71Lv6W7YTLwoDFmtzFmD/AAcEWZ7YWu7YXGmFnAoZPFXOa4TxpjNhtjDgF3ARNdpaFCbJLs6HqfLDPG5LqeVwL0FJEIY8xOY0yqm69DeYAmh8CUVeb24QrulzaAtsSWDgAwxpQAGdgSR0tguzGm7MyNW8vcbgv8xVVVsF9E9mO/9besRdwZZe+IyJVlqq32Az0pV81Szq4yt/OpvKH3Q2CwiLTEfjs32CQKtvQkwGJXddvVFR3AGHMQ+BKbWHD9PtZA7qqeWeuq/tkPND5J7GCv3T5jTF6Zx45dcxEJFpFHXVVtudhSAW4ct+zxy/4Nt3L83yvHGFNU5n5V1/Bkxw3Bll7fAr4G3nNVZT0mIqGu13gpcD2wU0S+FJGubr4O5QGaHFRVdmA/5IFj36LbANuBnUCrcvX2iWVuZwAPG2OalPlpaIyZ7sZ5K5sq+Njjrm/krwA3Ac2MMU2ANdgP7loxxuwHvgEuAS4DppcmQWPMLmPMdcaYltgqkRdEpGMlh5oOTBKRwdgS1zxX7KcBd7iO39QV+wE3Yt8JNBWRyDKPlb3mlwHnAWOwyaad6/HS455sCubj/t6uY+84yXPcUdFxi4AsVynkAWNMd2yJ9GxslRzGmK+NMWOxVUrrsH9v5SWaHFRV3gfOEpHRIhKKrRs/iq2L/gX7D36zq3H4Qo6v0nkFuF5EThErUkTOEpFoN86bha2frkok9sNuD4CrcbRndV7cSbyL/ZC6iN+rlBCRi0WktevuPlcMxZUcYxb2Q/FBYIar5AW2TaDIFXuIiNyLrWevkjFmK7AUeEBEwkRkKHBOmV2isX+fHGwd/iPlDnGy6zoduEdE4kQkFrgXqPEYinLHvc3VmB7limuGMaZIREaKSC+x4zhysdVMxWI7SZzrSoRHsVVYlV1n5QBNDqpSxpjfsA2nzwLZ2A+ic4wxBcaYAuBCbMPvPmwVwMdlnrsU2+7wnGv7Rte+7ngV6O6qLvqkktjSgCewSSoL6AX8XL1XWKXPgE7Yb7cryzw+EFgkIodc+9xijEmvJMaj2GsyhjIJBluNMhtYj61iOUK5KrMqXIZt5N8L3Ae8WWbbm67jbQfSsI3LZZ3suj6ETT6rgNXYjgpuDWo8iWnY6qP5QDr29f7Jta05thovF1gL/IBNSEHYLyM7sK91OHCjB2JRbpLjq4yVUkopLTkopZSqgCYHpZRSJ9DkoJRS6gSaHJRSSp2gzk1gFhsba9q1a+frMJRSqk5ZtmxZtjEmzt3961xyaNeuHUuXLvV1GEopVaeIyNaT7/U7rVZSSil1Ak0OSimlTuBYchCRaa6529dUsn2y2DUFVomd67+3U7EopZSqHifbHF7HTp3wZiXb04Hhxph9InIm8DJ2WgClVAApLCwkMzOTI0eO+DqUeiE8PJzWrVsTGhpaq+M4lhyMMfNFpF0V2xeUubsQaF3Zvkqp+iszM5Po6GjatWuHnLg4n6oGYww5OTlkZmaSlJRUq2P5S5vDNdiJyCokIlNFZKmILN2zZ48Xw1JKOe3IkSM0a9ZME4MHiAjNmjXzSCnM58lBREZik8Mdle1jjHnZGDPAGDMgLs7tbrpKqTpCE4PneOpa+jQ5iF1w/n/AecaYHCfPtSHrIA9+nsbRIp0SXimlTsZnycG1Hu7HwBXGmPVOny9jXz7Tfk7n543ZTp9KKVWH7N+/nxdeeKHazxs/fjz79+93ICL/4GRX1unYhVi6iEimiFwjIteLyPWuXe7FLiz+gmsdYEeHPQ/tGEd0eAhfrtp18p2VUgGjsuRQXFx1LcOsWbNo0qSJU2H5nJO9lSadZPu1wLVOnb+8sJAgxnZPYE7aLgqKehEW4vPmFqWUH7jzzjvZtGkTffr0ITQ0lKioKFq0aMGKFStIS0vj/PPPJyMjgyNHjnDLLbcwdepU4PepfA4dOsSZZ57J0KFDWbBgAa1ateLTTz8lIiLCx6+sdurc3Eq1cVavFny8fDs/b8pmZJd4X4ejlCrngc9TSduR69Fjdm/ZiPvO6VHp9kcffZQ1a9awYsUKvv/+e8466yzWrFlzrCvotGnTiImJ4fDhwwwcOJCLLrqIZs2aHXeMDRs2MH36dF555RUuueQSPvroIy6//HKPvg5vC6ivz0M7xRLdIIRZq3b6OhSllJ8aNGjQcWMEnnnmGXr37k1KSgoZGRls2LDhhOckJSXRp08fAPr378+WLVu8Fa5jAqrk0CAkmDHdE/gmLYtHiksIDQ6o3KiU36vqG763REZGHrv9/fff8+233/LLL7/QsGFDRowYUeEYggYNGhy7HRwczOHDh70Sq5MC7tNxfK8WHDhcqL2WlFIAREdHc/DgwQq3HThwgKZNm9KwYUPWrVvHwoULvRyd7wRUyQHgtE6xRDUIYdbqnYzQdgelAl6zZs049dRT6dmzJxERESQkJBzbNm7cOF566SWSk5Pp0qULKSkpPozUu8QY4+sYqmXAgAGmtov93Prer3y/fg9L7h6jVUtK+djatWvp1q2br8OoVyq6piKyzBgzwN1jBOQn4/heLdifX8gvmxwdlK2UUnVWQCaHYZ3jiAwLZtZq7bWklFIVCcjkEB4azOhuCXyduovC4hJfh6OUUn4nIJMD2KqlffmFLNysVUtKKVVewCaHEV20akkppSoTsMkhPDSYUd0S+Do1iyKtWlJKqeMEbHIAOKtXc/bmFbAofa+vQ1FK1RFRUVEA7NixgwkTJlS4z4gRIzhZl/unnnqK/Pz8Y/f9bQrwgE4OwzvHExEazJdataSUqqaWLVvy4Ycf1vj55ZODv00BHtDJISIsmFHd4vl6zS6tWlIqQN1xxx3Hredw//3388ADDzB69Gj69etHr169+PTTT0943pYtW+jZsycAhw8fZuLEiSQnJ3PppZceN7fSDTfcwIABA+jRowf33XcfYCfz27FjByNHjmTkyJGAnQI8O9tO6/Pkk0/Ss2dPevbsyVNPPXXsfN26deO6666jR48enH766Y7O4RRw02eUd1avFny5aieL0/cypGOsr8NRKrDNvhN2rfbsMZv3gjMfrXTzxIkTufXWW7nxxhsBeP/99/nqq6+47bbbaNSoEdnZ2aSkpHDuuedWuj7ziy++SMOGDVm1ahWrVq2iX79+x7Y9/PDDxMTEUFxczOjRo1m1ahU333wzTz75JPPmzSM29vjPnWXLlvHaa6+xaNEijDGccsopDB8+nKZNm3p1avCALjkAjOxiq5ZmrdGqJaUCUd++fdm9ezc7duxg5cqVNG3alBYtWvD3v/+d5ORkxowZw/bt28nKyqr0GPPnzz/2IZ2cnExycvKxbe+//z79+vWjb9++pKamkpaWVmU8P/30ExdccAGRkZFERUVx4YUX8uOPPwLenRo84EsOEWHBjOoaz1drsnjg3J4EB1X8zUAp5QVVfMN30oQJE/jwww/ZtWsXEydO5J133mHPnj0sW7aM0NBQ2rVrV+FU3WVVVKpIT0/nP//5D0uWLKFp06ZMmTLlpMepar47b04NHvAlB4AzezUn+9BRFmuvJaUC0sSJE3nvvff48MMPmTBhAgcOHCA+Pp7Q0FDmzZvH1q1bq3z+sGHDeOeddwBYs2YNq1atAiA3N5fIyEgaN25MVlYWs2fPPvacyqYKHzZsGJ988gn5+fnk5eUxc+ZMTjvtNA++WvcEfMkBYFTXeMJDg5i1eieDOzQ7+ROUUvVKjx49OHjwIK1ataJFixZMnjyZc845hwEDBtCnTx+6du1a5fNvuOEGrrrqKpKTk+nTpw+DBg0CoHfv3vTt25cePXrQvn17Tj311GPPmTp1KmeeeSYtWrRg3rx5xx7v168fU6ZMOXaMa6+9lr59+3p9dbmAnLK7Ije8vYwlW/ax6O+jtWpJKS/SKbs9T6fs9qDxvVqQfegoS7Zo1ZJSSmlycBnVNZ4GIUHM1gFxSimlyaFUZIMQRnSJY/aaXZSU1K2qNqXqurpWve3PPHUtNTmUMb5XC3YfPMrSrft8HYpSASM8PJycnBxNEB5gjCEnJ4fw8PBaH0t7K5UxulsCYSG219KgpBhfh6NUQGjdujWZmZns2bPH16HUC+Hh4bRu3brWx9HkUEZUgxBGdI5j9pqd3Ht2d4K015JSjgsNDSUpKcnXYahytFqpnLOSW5CVe5Tl27RqSSkVuDQ5lDOqazxhIUE6jbdSKqBpcignOjyUYZ3imL1aey0ppQKXJocKnJXcnF25R/g1Q6uWlFKBSZNDBUZ3SyAsOIgvV+3ydShKKeUTmhwq0Cg8lGGdY5m9ZqdWLSmlApJjyUFEponIbhFZU8l2EZFnRGSjiKwSkX4V7ecr43u1YOeBI6zI9J8Fv5VSylucLDm8DoyrYvuZQCfXz1TgRQdjqbbR3RIIDRZmrdJeS0qpwONYcjDGzAeqmuL0POBNYy0EmohIC6fiqa7GEaGc1snOtaTD+pVSgcaXbQ6tgIwy9zNdj51ARKaKyFIRWerNIfbje7Vg+/7DrMjQqiWlVGDxZXKoaG6KCr+iG2NeNsYMMMYMiIuLczis343t7qpa0gFxSqkA48vkkAm0KXO/NbDDR7FUqHFEKEM7xjJrtVYtKaUCiy+Tw2fAla5eSynAAWOM331FP9NVtbQq84CvQ1FKKa9xsivrdOAXoIuIZIrINSJyvYhc79plFrAZ2Ai8AtzoVCy1cXr3BEKCtGpJKRVYHJuy2xgz6STbDfBHp87vKU0ahnFqx1i+XL2TO8/siohO462Uqv90hLQbzurVgsx9h1m9XauWlFKBQZODG07vUVq1pHMtKaUCgyYHNzRpGMbgDs2YtXqn9lpSSgUETQ5uOqtXC7btzSd1R66vQ1FKKcdpcnDT6T2aExwkukKcUiogaHJwU0xkGEO0akkpFSA0OVTD+F4t2JqjVUtKqfpPk0M1nN49geAgYfYarVpSStVvmhyqoVlUA1Lax+hcS0qpek+TQzWd2bMF6dl5rM865OtQlFLKMZocqmlMtwQAvlu328eRKKWUczQ5VFPzxuH0aNmI79Zl+ToUpZRyjCaHGhjdNZ5lW/exL6/A16EopZQjNDnUwKhuCZQY+GG995YsVUopb9LkUAPJrRoTGxXGXG13UErVU5ocaiAoSBjZJZ4ffttNYXGJr8NRSimP0+RQQ6O7xZN7pIhlW/f5OhSllPI4TQ41NLRTHKHBol1alVL1kiaHGopqEEJK+2Z8u1a7tCql6h9NDrUwqms8m/fkkZ6d5+tQlFLKozQ51MLorjpaWilVP2lyqIXEZg3pFB+lo6WVUvWOJodaGtUtnkWb93LwSKGvQ1FKKY/R5FBLo7smUFRi+HFDtq9DUUopj9HkUEv9EpvQOCKUuWu13UEpVX9ocqilkOAgRnSJ4/vfdlNcogsAKaXqB00OHjCqazw5eQWszNzv61CUUsojNDl4wPDOcQQHCd9p1ZJSqp7Q5OABTRqG0b9tU52lVSlVb2hy8JDRXeNZuzOXHfsP+zoUpZSqNU0OHjK6Wzygo6WVUvWDJgcP6RAXRWJMQ+bqRHxKqXrA0eQgIuNE5DcR2Sgid1awvbGIfC4iK0UkVUSucjIeJ4kIo7vF8/OmHPILinwdjlJK1YpjyUFEgoHngTOB7sAkEelebrc/AmnGmN7ACOAJEQlzKianje6aQEFRCQs25vg6FKWUqhUnSw6DgI3GmM3GmALgPeC8cvsYIFpEBIgC9gJ19mv3oKQYIsOCtdeSUqrOczI5tAIyytzPdD1W1nNAN2AHsBq4xRhTZxdlDgsJYljnOL5bl4UxOlpa+TljIH+vr6NQfsrJ5CAVPFb+E/MMYAXQEugDPCcijU44kMhUEVkqIkv37Nnj+Ug9aFTXeLJyj5K6I9fXoShVtdUfwJPdIHenryNRfsjJ5JAJtClzvzW2hFDWVcDHxtoIpANdyx/IGPOyMWaAMWZAXFycYwF7wogu8Yhol1ZVB6z9DIqOwNaffR2J8kNOJoclQCcRSXI1Mk8EPiu3zzZgNICIJABdgM0OxuS4uOgG9G7dRNsdlH8rLoTNP9jb2xb6NhbllxxLDsaYIuAm4GtgLfC+MSZVRK4Xketdu/0TGCIiq4G5wB3GmDq/MMLorvGszNjPnoNHfR2KUhXLXApHcyEkQpODqpCj4xyMMbOMMZ2NMR2MMQ+7HnvJGPOS6/YOY8zpxphexpiexpi3nYzHW0a5RkvP+01LD8pPbZoLEgwDroKsNXDkgK8jUn5GR0g7oHuLRjRvFK6ztCr/tXEutB4Anc8ADGQs9nVEys9ocnCAiDCqWzw/btjD0aJiX4ej1PHycmDHr9BhNLQaYEsQ237xdVTKz2hycMjorvHkFRSzOF37kSs/s3keYKDjaGgQBS16a7uDOoEmB4ec2jGW8NAgXVta+Z9N30FEU2jZ195PHAzbl0GRdqBQv9Pk4JDw0GBO7RDLXB0trfyJMba9of0ICAq2jyWm2PEOO1f6MjLlZzQ5OGhUt3gy9h5m4+5Dvg5FKSsrFQ7tsu0NpRJT7G9td1BlaHJw0KiutkurDohTfmPTXPu7Y5nkEBUPMR203UEdx63kICK3iEgjsV4VkeUicrrTwdV1LRpH0L1FI+3SqvzHxrkQ3x0atTz+8cTBNjmU1Nl5L5WHuVtyuNoYkwucDsRh50R61LGo6pHR3eJZunUv+/MLfB2KCnQFebbqqMOoE7clpsDhvZCzwftxKb/kbnIonWF1PPCaMWYlFc+6qsoZ1TWeEgM/rPfv2WRVANjyMxQXHF+lVCpxsP29dYF3Y1J+y93ksExEvsEmh69FJBrQ8qcberduQrPIMO3Sqnxv01w7l1LikBO3NesAkXHa7qCOCXFzv2uw6y1sNsbki0gMtmpJnURQkDCyazzfpO6iqLiEkGDtA6B8ZOO30O5UCA0/cZuIrVrSHkvKxd1PqsHAb8aY/SJyOXAPoDN1uWl013hyjxSxbOs+X4eiAtW+rZCz8fgurOUlDob9WyG3/LIrKhC5mxxeBPJFpDdwO7AVeNOxqOqZoZ1iCQ0WXQBI+c6xLqxjKt/n2HgHrVpS7ieHImOH+Z4HPG2MeRqIdi6s+iU6PJSU9s10vIPynY1zoXEbiO1U+T7NkyG0oSYHBbifHA6KyF3AFcCXIhIMhDoXVv0zqms8G3cfYmtOnq9DUYGmuBDS59surFJFJ8PgUDuNt7Y7KNxPDpcCR7HjHXYBrYDHHYuqHjo2Wlp7LSlvK131raIurOUlDnYt/pPrfFzKr7mVHFwJ4R2gsYicDRwxxmibQzW0bRZJx/gobXdQ3le66lvS8JPvm5gCpgQylzgfl/Jr7k6fcQmwGLgYuARYJCITnAysPhrdNZ5F6TkcPFLo61BUICld9S2iycn3bT0QJEirlpTb1Up3AwONMX8wxlwJDAL+4VxY9dOorvEUFht+2pDt61BUoCi76ps7GkTbhmltlA547iaHIGNM2fqQnGo8V7n0b9uURuEh2mtJeU/ZVd/clTjYtlMU6XxggczdD/ivRORrEZkiIlOAL4FZzoXlgF1r4M3zIN93y3aGBAcxoks889btpqREFwBSXrBx7vGrvrkjMQWKDsOuVc7Fpfyeuw3SfwNeBpKB3sDLxpg7nAzM447m2knFZlzu0+UQR3eLJyevgJWZ+30WgwoQxtglQduP/H3VN3fo4j+KalQNGWM+Msb82RhzmzFmppNBOaLtEDjvBdj6M3x6k/3H8YHhneMIErTXknJe6apv1alSAohuDk2TtN0hwFWZHETkoIjkVvBzUETqXkfo5Ith1D2w+n2Y94hPQmjSMIwBbWN0vINyXumUGRWt33AyiYNtyUHXPw9YVSYHY0y0MaZRBT/RxphG3grSo077K/S9HOY/Br++45MQRnWLJ21nLjsPHPbJ+VWAqGzVN3ckpkB+jp2sTwWkwOtxJAJnPwXtR8DnN8Pm770ewphudrS0Vi0px1S16ps7Shf/0XaHgBV4yQHsHDKXvAnNOsGMK2H3Wq+evkNcFIkxDXVtaeWcLT9VvuqbO2I7QcNmsFWTQ6AKzOQAEN4YJn9gFz5552I4mOW1U4sIo7vF8+PGbHbs16qlOqGwjv2dNlax6ps7RH5vd1ABKXCTA0CTNnDZDFu3Ov1SWxT3kmuGJgHw2FfrvHZOVUMHtsNjHeCDKV59j9TKprnQbmjFq765KzEF9qXDwV2ei0vVGYGdHMAODpowDXauhI+uhZJir5y2ddOGXHdaEp+s2MGv23SFOL+W9gkU5kHqJ/DqGXZVNX9WuupbTauUSh1rd9AurYFIkwNAlzNh3L/ht1nw9d1eO+0NIzoSF92AB79Iw2iXQf+VOtPONzT5A9i/DV4eYddH8FfHurDWMjk0T7ZVU5ocApImh1KnTIWUG2HRi7DwJa+cMqpBCH87owu/btvPZyt13V6/tH+bnb66xwXQaSxc9x1ExsGb58Oi//rnOAB3Vn1zR0iYLv4TwBxNDiIyTkR+E5GNInJnJfuMEJEVIpIqIj84Gc9Jnf4QdD0bvroT1n3plVNO6NeaHi0b8e/Z6zhS6J0qLVUNaZ/a3z3Ot79jO8K130Kn02H27Xa0feER38VXXnEhbP7h5Ku+uSsxxc6xdPRg7Y+l6hTHkoNrKdHngTOB7sAkEelebp8mwAvAucaYHtj1InwnKBgufMW2Q3x0LWxf7vwpg4R/nN2dHQeO8Mr8zY6fT1VT6kxo0Qdi2v/+WHgjmPguDLsdVrwNr58FuTt9F2NZmUug4GDt2xtKHVv8Z6lnjqfqDCdLDoOAjcaYzcaYAuA94Lxy+1wGfGyM2QZQblpw3whraHswRcbCu5faagWHpbRvxpk9m/PC95vIyvWjb6GBbt9W2L7MVimVFxQEo+6GS96y42ReHgEZfrB62sZqrPrmjtaDXIv/aLtDoHEyObQCMsrcz3Q9VlZnoKmIfC8iy0TkyooOJCJTRWSpiCzds2ePQ+GWERUPl31gZ29952I47PwMqned2Y3iEsNjX/3m+LmUm9I+sb9Lq5Qq0v1cuHaO7TL6+nhY/pZ3YqvMprl2NTd3Vn1zR3gjSOgJ2xZ45niqznAyOVRU4Vm+9S4E6A+cBZwB/ENEOp/wJGNeNsYMMMYMiIuL83ykFYnvCpe+BTmb4P0rHF/4JLFZQ64a2o6PlmeyOvOAo+dSbkqdCS37QdN2Ve+X0AOum2dn/v3sJpj1N1v37215ObBjheeqlEqVLv7ji9ekfMbJ5JAJtClzvzVQvktOJvCVMSbPGJMNzMeuF+Ef2g+Hc5+x3Ra/uNXxnik3jexIbFQYD36Rql1bfW1vul1es6IqpYo0jIHJH8Hgm2Dxy/DWBZDn5eVgS1d9q20X1vISU6AwXxf/CTBOJoclQCcRSRKRMGAi8Fm5fT4FThOREBFpCJwCeHeio5PpcxkMvxNWvAPzH3f0VNHhofzl9C4s2bKPWat1VKpPuVOlVF5wCJzxMFzwX8hYDC+PhJ1e/EA9tupbH88e99jiP9ruEEgcSw7GmCLgJuBr7Af++8aYVBG5XkSud+2zFvgKWAUsBv5njFnjVEw1NuJOSJ4I8x6GlTMcPdUlA2bSKRMAACAASURBVNrQtXk0/5q9Vru2+lLqTGg1AJokVv+5vSfC1V+BKYZXT4c1H3k+vvKMse0N1V31zR2NWkKTtjreIcA4Os7BGDPLGNPZGNPBGPOw67GXjDEvldnncWNMd2NMT2PMU07GU2MicO6z0O40+PSPdsZLhwQHCfee3Z3MfYeZ9nO6Y+c5QcZi/+ht4w9yNtnpVNytUqpIq34w9Xto0Rs+vBq+vd/ZqVmy1sChLM+3N5RKHGxLDlrdGTB0hLS7QsJsA3VMErw32dH65CEdYxnbPYHnv9vI7oMOd20tKYbvHrbfcN+5CPL3Onu+uqC0Sql7+Z7X1RQVD3/4HPpfBT/9P9s12qmebxtrseqbOxJTIG8P7NWxOIFCk0N1RDS1/dqP5sKPTzp6qr+P70ZBcQlPfL3euZMczII3z7Or4nU9C47kwk/Ovq46IXWm7d/fpM3J9z2ZkDA45yk4+//ZBuNXRkG2A6urbarFqm/u0MV/Ao4mh+qK72obqZe8AvszTr5/DSXFRvKHwe14f1kGa7Y70LU1fT68NNR2UTzvBZj4jn1di1529HX5veyNsGt17aqUKjLgavjDF3BkP7w61rNVeAV5tsrHqSolgLguEBGjySGAaHKoieF3AgLfP+roaf40uhNNIkL5pydnbS0pgR8esyWGiCZ2Irm+k+22kX+3v+c94plz1UVpM+3v2lYpVaTtYLhmjl1o6o1z4LfZnjlu6apvnu7CWpaIrVrSleEChiaHmmjSBgZeCyvfhT3OjWhuHBHKn0/vwqL0vXyd6oGV6vKybbvCvIeh5wQ7cCuhzHRXjVvDKf8HK6dDVmrtz1cXpX4CbVKgcfnB/B7SrINNEPFd4b3LYOm02h/z2Kpvg2t/rKokpsDeTXDI97PcKOdpcqip0/4MoZHw3T8dPc2kgW3onBDFI7PWcrSoFr1dti6w1UhbfoZznoYLX4YGUSfuN/Q2O2XCt/fX/Fx11Z71ttePp6uUyouKs1VMHcfAF7fZDgG1KRlu/Lb2q765Qxf/CSiaHGoqMhaG3ARrP7eTszkkJDiIe87qzra9+byxYEv1D1BSYhvPXz8bQiPsdNP9p1Q+nXPDGDjtL7DhG0j/sTah1z1pnwBi50tyWoMomDgd+l5hOwR8elPNpqfYt8V+m3eyvaFUi94QEq7JIUBocqiNwX+Ehs3g2wccPc2wznGM6hrPs3M3kn3oqPtPzN9r18ae+4D9wJv6A7RIPvnzBk2FRq3g2/sCq1976kz77dipHj/lBYfY8TPD77RTf0+fCEcPVe8YGz206ps7QhpAq/7aKB0gNDnURoNoOO2vkP4DbJrn6Kn+Pr4bhwuLeXKOm11bty2y1Uibv4fx/4EJr9nqIneERsDIu22JqHSxm/pu9zrYneZ8lVJ5IjDyLjjnGfseev2s6tXpb/rOM6u+uSsxxQ4QLMjzzvmUz2hyqK0BV9t/zrkPOPotu2N8FJentOW9xdtYtyu38h2NgQXP2umjg0Lgmm9g0HXVXxWs90SI6wZzHwyM2Ti9WaVUkf5/gEnTIXs9/G+Me2MhPL3qmzsSB9tpQXTxn3pPk0NthYbbuZd2/Apry88r6Fm3julEdHgVXVsP77M9YL65BzqPg/+bb1e1q4mgYBhzv63PXv5GbcKuG1JnQttTIbq572LofIZtqC445N5YiGOrvo3xTnwAbQYBou0OAUCTgyckT4TYLvDdQ1Bc5NhpmjQM47Yxnfh5Yw5z15areti+DP47DDbMgXGPwqVv137Bl85n2A/M7/9d/brwumT3WtizrnozsDqldX/3x0KUrvrW3kOrvrkjvLFr8R9td6jvNDl4QnAIjP6HrRJYOd3RU01OaUuHuEgenrWWgqISW4208CV49Qy7lNLVX0PKDZ6pZhCBMQ9A3m745fnaH89fpc60S2F281GVUnnujoXY+K1d9S28sXfjS0yxpRYHvwgp39Pk4Cldz7Y9Ob5/FAqdmywv1NW1NT07jzcXpMOXf4av7rBVC//3g/3m6UltBtoPzQXPwCEvLNHqbcaUqVJK8HU0vzvZWIi8bNsw7I0urOUlptiqr6zV3j+38hpNDp4iAqPvg9xMWPqqo6ca0SWOYZ3jOPTdf+y3yiE328bMhjHOnHD0vVB42PbHr292p9kSnz9UKZVX1ViITQ6t+uYOHQwXEDQ5eFL74dB+BPz4hJ3h1CEiwqNdNnIr77Kq6VgY+6CzvVViO0G/K20iqm9TNvtblVJ5lY2F2OTQqm/uaNwKGidqu0M9p8nB00bfC/k5ztbRb1tEy+9uZWtkMpdmXc763V5oLB5xJwSH2Ub3+qK0SqndULv2gr+qaCzExm+dWfXNXYkpuvhPPafJwdNa9bffQn95zpkFgfZuhvcmQeNWNJryPiFhEfz5/RUs2JTtuZlbKxLd3I4IX/MRbF/u3Hm8KWsN5Gz0/sC3mio7FiJvj2/aG0olptiV5/Z5cbVC5VWaHJww6h9QmG+rlzwpfy+8cwmYEpj8IU3jWvCvC3uxfd9hLntlEWc8NZ+3F24lv8ChXiRDbnZNF+KFaTW88Y3U36uUKlI6FqL3ZbYThK+0HWJ/a7tDvaXJwQlxnV0LAv0P9m/zzDGLjsKMy2H/VttI2awDAGcnt+SXu0bz2IRk25PpkzWc8shc/vlFGluyPTzFQXgjGHa7XSho01zPHruszKXwTF/4/FbnkkRplVLSMDuJYl3Suj9c8GLtx7HURmwXCG+i7Q71mCYHpxxbEOjftT+WMfDZzbD1Zzj/RbtoTBnhocFcMqANX/xpKB/dMJgRXeJ5Y8EWRj7xPVe9tpjvf9tNSYmHPmQHXAVN2sKc++2Mr55UOmZj2jjbbrPsNbv2shN2rbJVdHWlSsnfBAX93u6gnFeQZ7szb/nZa6fU5OCUsgsC7V5Xu2P98G9Y9R6Mugd6Tah0NxGhf9sYnp3Ul5/vHMWfRnVi9fZcpry2hNFP/sC0n9LJPVLLeZJCGthG96zVsPqD2h2rrCO58MEUO2aj01i4ZaVdkGjuA85M/pc6044u7nqO548dKBJTXO0fDrStKaukBFbOgGcH2O7M6T947dSaHJxUuiDQvFr08Fn5Hnz/L+gz2c4A66aERuH8eWxnFtw5iqcn9qFJw1Ae/CKNwY/M5R+frGHj7oM1j6nHhXZu/+8estVdtbVrDbw8wq6NMfZBmPiuHbNx3vN2BPDH/1ejRnBjDNN+SufXbfvKb7DJof1wiGxW+/gDlY53cFbGEjvH1sypdoDm1d/8vpSvF2hycFJkLAz5k/3Qy6zBgkDpP9qBT0nD4OynajSWISwkiPP6tGLmjafy2U2nMq5nC2YsyWDMk/OZ/L+FfJO6i+LqVjkFBdlpNQ5sgyW1HPD36zvwv9G22DzlCzj1lt9fZ2i4TRSRcTB9EhzYXq1Dz9+QzYNfpHHpywv5fOWO3zfsXGEXydEqpdpp2ReCG2i7g6cd2A4fXQevjoEDmbYq+drvIPEUr4YhjnZ/dMCAAQPM0qV1aLrgowfh6T6Q0AP+UI1ZW/est2+OqOZ22m0PNj7mHDrKe0syeHvhVnYeOEKrJhFcMbgtEwe2oUnDMPcP9Ob5dgqHW1ZUf36fwsMw66/w69s2+V30auVjDbLS4NXTIaYdXPVVxcublmOM4cIXF5B14AitmkawZMs+bh/XhRuGd0C+vc+OQ/nrBudGlQeKaePsqO3rathBobgIti+14zY2zoWSIjvTQCcvzjTrLwry7TQ1Pz1leyQOuQmG/tmt97s7RGSZMWaAu/trycFpDaJhWDUXBMrLhncvtoPOJr/v8V4pzaIa8MeRHfnx9pG8OLkfrZtG8OjsdQx//HveXbTN/cbrMffD4b3w89PVCyBnk12z4Ne3be+nKz6pehBaQne4+DXISoWPr4OSk6+lPX9DNr9u288fR3XkrWtO4dzeLXnsq9+466NVmNSZdiS7JobaS0yxJbGCfPefcyATlr0BM66Ax9rDtDNst+/gUFuCfOciW1LcGyBjKIyB1R/CcwNtFXLnM+CmJbZtz0OJoSa05OANRUfh2f62mum6eVVXDxUehjfOtb1ppnwJrd1O9LWSuuMAD36exqL0vfRNbMJD5/ekR0s3SgMfXQtrv4Cbl7u3vGbap/DJH+0HwYWvVO8b4qL/wuzb7XiL0/9Z6W7GGC54YQF7Dh5l3l9HEBYSREmJ4ck565n//dd81uAfHB7/DBGD/uD+uVXF1n8N715ix14knVbxPoVHYNsCWzLYOBf2rLWPR7e0A/k6jrHtPxFN7f/Kwhfgh8dtKeLUW2DobRDW0HuvCewXkA1z7ASDbQbZBb2cmKJm+zL46i7IWATNk+10++1O9fx5qH7JIcSRKNTxQhrAiLvg0xvtgkDdz6t4v5ISmHm9nQ75kje8lhgAerRszHtTU/h4+XYembWWc579iSlDkvjz6Z2JalDF22TUPZD6iZ2N9txnKt+vqADm3AuLXrSNzBe/Do1bVy/IQVNt75gFz0CzjnbEcAV+WL+HFRn7eeSCXoSF2MJxUJDw1zO6cHbWCxRsCuaKn+J4pvNhWjaJqF4M6nhlF/8pTQ7G2NLhxm/tz5afoOiwLQm3HQJ9J9uEENf1xA/ckAY2GSRfCt/8w/bQWTkdznjYDlZ0esW7gnzbw/CX54+fR6xRK2hzim2ET0yx1cS1mbokd6ftibdyOkTGw7nP2bFRvpoOpQJacvCWkmJ4YbCtS7xxoZ1Qrbxv77f9+sf+E0692eshljqQX8i/v17H9MXbiI9uwL1n92B8r+ZIZf+Ys++Exf+FGxfZAYAnHDDTdlPNXAKn3GB7JIVUo22jrOIiW+WWPh+ucA1iK6OiUkOZjfBUL/ZGdmD49huJCAtm2pSB9Gzl5fUQ6psXhtgqupQbfm872L/VbovpYBNBxzH2G3FYZPWOveUnmHU77E6FpOFw5mN2nQtPO7QbFr9iB64e3gst+9n/wZj2dj32bb/YBHjQ1bEhLNpOZ98mxSaL1gPce22Fh+3UOj/+PygphJQb4bS/uL++ey1Ut+SgycGb1n5uRzmf+xz0u+L4bcteh89vsWtSn/Wk99YErsKv2/Zx98w1pO3MZVjnOB48twftYiv4B8jLto3u7YfDxHeO37bhW9tOUFwI5z3nmamxjxywDdQHd8G1cyG247FN837bzVWvLeGRC3px2SmJxz8vc6ntGXX+i/zW/Byuem0x+w8X8uykvozu5kdrOdQ1X/z592nqQyPt+6DjaDudeExS7Y9fXGRnBJ73kG2TOOV6GH6HZz5Q96y3H9Yr34PiAugy3jYEJw4+8X/QGDiQcXyy2J0GGDtmpkWyfV6bU2zCKLvkrDF2nfJv7rW9/LqebatGY9rX/jW4SZODPzPGfjgd3AV/Wm67aoL9pvXOxdBhJEyaUXGpwkeKikt4a+FWnvhmPQXFJfxxREeuH9GeBiHlir8/PG7/ea/+xna5Kym2VU3zH7dF8IvfOO5DvNb2pttrGd7YJoiGMRhjOP+FBWRXVGoA+Ppu227xt40Q0YTduUe4+o0lpO3I5f5ze3Dl4Haei6+c4hLDd+t28+OGPVwyoE39Kq3s3WwbVEs/GGtaKjyZvGxbFbP8Ldu9eeyDtvopqJr9aoyxsw0seBbWfwUh4dB7kp1YMrZT9Y51eL8tEW9baH+2L4Ui12JfTZNskmjVH9Z8bNtdEnrCuH+dUOL1Bk0O/m7zD/DmuXDGI/bNWNpNs2lbuGq2V4qXNZGVe4R/fpHGF6t2khQbyT/P68nQTmXmJCrIs/MhNU2CS9+yDdXpP0Dfy2H8fyDUgbr9bQvtOsttToHLP2bepv1c9doS/nVhLyYNKldqKCmBp3pB855w2YxjD+cdLeKW937l27W7uWZoEn8f343gIM+V2vblFTBjaQZv/bKV7fsPIwKhQUHcfVY3rhzctvKqOlW57ctsVdP2pdB6EIx/3L11LYqLYO2nNins+NVOIjloqp3JwFPzaxUV2M4kpSWLbQshPxsaxtr2uX5X+qxdQZNDXfDmebBzlR2/8Ob5YIrh2m+r30DrA/PX7+HeT9ewJSefc3q35J6zupHQyFUCWjrNLmnZoDEUH4WznrDJwUkrZ8DMqZg+l3N+xkSy8worLjVkLLajTS/4L/SeeNym4hLDP79I4/UFWzijRwJPXdqXiLDa/QOv2X6ANxZs4bOVOzhaVEJK+xj+MLgd/ds15c6PVvPdut2c0SOBxy7qTeOGobU6V0AqKbENx3Pus/Nw9Z9iu35W1D356EFb2lj4oq3Sielgq456T3LmS0tZxtgBl5FxPu2WCn6WHERkHPA0EAz8zxjzaCX7DQQWApcaYz6s6pj1IjlsXwavjILQhoDAVbN8s6JXDR0pLOalHzbxwvebCAsO4i+nd+bKwe0INsXw32G2WH3Jm/Zbujd89xDMf5xHCifR/ry/M7F8qQFsd8El/7NVSpUM2Jv2Uzr//DKN5NZN+N+VA4iLblCtMAqKSpi9ZidvLNjC8m37aRgWzAV9W3Hl4HZ0aR59bL+SEsOrP6Xz76/WkdAonOcu60vfxKbVOpdyObzfVl8uftmOKRr9D+h/lf12nrsDFr0ES1+HowcgcYidsaDzuOpXRdUDfpMcRCQYWA+MBTKBJcAkY0xaBfvNAY4A0wIiOQC8f6VtoJ44HbqM83U0NZKence9n67hxw3Z9GjZiIcv6EWf5g0gKNSr7SampJifHz2PIQU/UXzxW4T2KDeZXkkJ/L8eNgFPml7lsb5J3cXN7/1KbFQDXr9qIB3jo6vcH2DXgSO8u2gr7y7OIPvQUZJiI7lycFsu6t+aRuGVlwp+3baPm979lazcI9w+rgvXDm1PkAertAJKVpodA7PlR2jeC+K724WpTIntOj74T3aq8wDmT8lhMHC/MeYM1/27AIwx/yq3361AITAQ+CJgkkNBHuzbakf/1mHGGL5cvZMHP09jz6GjXDYokdvHdaVxhPeqSuat2831r//MgoQnaJa/ybbdlC2JbVtoR+Fe+AokX3LS463M2M81byyloKiYl67oz5AOJ9ZHG2NYlL6Xt37ZylepuygxhtFd47lycDuGdox1+0P+QH4hd3y0iq9SdzGySxxPXNKHmEiHGnTru9IJFb+5x5Yo+l1hu9c2befryPyCPyWHCcA4Y8y1rvtXAKcYY24qs08r4F1gFPAqlSQHEZkKTAVITEzsv3XrVkdiVjV38EghT85ZzxsLttCleSM+vmFIrevt3WGM4bznf2ZvXgHzru9G6KtjbBvOdd/9PmJ79h2w9DVXlZJ7Df4Ze/O5+vUlbMnJ418XJjOhv20Pyi8oYuav23lzwVZ+yzpI44hQJg5sw+UpbWkTU7NRvMYY3lq4lYe+WEtMZBhPT+zDKe11ttgaKy60veVKewMqwL/mVqroq1P5TPQUcIcxpsrJcowxLxtjBhhjBsTFxXksQOU50eGh3HdOD179w0DW7crl9o9WObumtcu833azKvMAfxrVkdDGLWxPpKMHYfpEWzorKbEjuDuNrVZPsDYxDfnwhiEMbBfDXz9YyaOz1/Hg52mc8shc7p65huAg4bGLkll412juGt+txokB7DocVw5ux8c3DiE8NIhJryzk2bkbqj9brrKCQzUxeICTFcOZQJsy91sDO8rtMwB4z9WdLxYYLyJFxphPHIxLOWhk13j+enoXHv/6N3q1asTUYR0cO5cxhqe+3UCbmAgu7Ofq6dW8p53h9b1J8PFUW61waFeNpuduHBHK61cN4u8zV/PSD5sICRLG92rBH4a0pV9iU493Q+3ZqjGf/2kof5+5hifmrGdR+l6evLQ38dH6Qae8z8lqpRBsg/RoYDu2QfoyY0xqJfu/TiC1OdRjxhj++O5yvlqzizeuHsRpnZwp7c1dm8U1byzlsYuSuWRgm+M3/vICfH0XRCXYEdV/22h7s9SAMYb5G7Lp1jya+EbOf1AbY5ixJIP7PkslOjyUpy7tc/yYEqVqwG+qlYwxRcBNwNfAWuB9Y0yqiFwvItc7dV7leyLC4xN60zkhmpve/ZVtOdWYztlNZUsNF/RrdeIOKTfYLo2HsmyVUg0TA9jXM7xznFcSQ+n5Jg5K5LObhtKkYShXTFvEE9/8RlGxh9fsVqoKOghOOWZrTh7nPvczLRqH89ENQ4isanbXaqqy1FCquBDm/8fO5xTfzWPn9qb8giLu+zSVD5ZlMqhdDE9P6kOLxjqTrKo+v+mt5BRNDnXL/PV7mPLaYsb1bM7zl/XzSD29MYZzn/uZA4cLmfuX4YQG1/8BTR8vz+SeT9bQICSIJy/pw8iuVSyO5EVZuUeYk5ZFenYeHeKi6NI8is4J0URXMb5D+Yau56D8yrDOcdwxriv/mr2OF3/YxI0jaj/53ty1u1m9/QCPTUgOiMQAcGG/1vRu04Q/vrOcq15fwjVDk5g0qA0d4qK8Oj+TMYb1WYeYk7aLOWlZrMw8AEBYcBAFZaq9WjWJoEvzaDonRNPV9btDfOSJEzYqv6UlB+U4Yww3v7eCL1btYNqUgYzsUvNvvcYYznnuJ3IPFwVMqaGsI4XFPPRlGm8v3AZATGQYA9s1ZVBSM05JiqFbi0YenTgQ7My8y7buY05aFnPWZrHV1YbUu00TTu+ewNjuCXSMi2L7/sOszzrIul0HWZ91kN92HWTTnkMUFtvPmOAgISk2ki7No+mSEH3sd2JMQx0Z7gVaraT8Un5BERe9+Avb9+Xz6U1DSapoXQg3zEnL4ro3l/L4hGQuHlBJW0MA2JqTx6LNe1mUvpfFW3LI2HsYgOgGIfRr25RBSTGckhRDr9aNa/RtPb+giPnrs5mTlsV367LYl19IWHAQQzo2Y2z3BMZ0S/h9wsUqFBaXkJ6dZxPGrt8Tx7a9v3dSiAgNplNCFF0SohneJY6zk91YblZVmyYH5bcy9uZzznM/ERfVgJl/PLXq5UcrYIzh7Gd/4tDRIub+eTghAVZqqMrOA4dZnL732M+G3YcAaBASRJ82TTglKYZBSc3o17YJDcMqvu57Dh5l7tos5qRl8dPGbI4WldAoPIRRXeMZ2705w7vEVftvVpm8o0Vs2H2I33bl8tuuQ/yWlcu6nQfJySvgzjO7cv1w58bHBCpNDsqv/bwxmyteXcTY7gm8OLl/taoTtNTgvr15BSzZ8nuySN1xgBIDIUFCj1aNbbJoF0PLJhHM37CHOWlZLN+2D2Nse8HY7gmc3j2BgUkxXqu6Kyou4dYZK/hi1U7uHt+N64Z5b5W0QKDJQfm9//24mYe+XMtfxnbmT6PdW3lLSw21c/BIIcu37Wdxeg6L0/eyMuPAcQ3IPVs1Ymy35oztnkC3FtE+W4SoqLiEW95bwZerd3LPWd249jRNEJ6ivZWU37tmaBJrth/gyW/X071lI7fWb56TlkXqjlz+c3FvTQw1EB0eyvDOcQzvbEerHyksZkXGfjL3HWZIh2a0bOIfYydCgoN4amIfDIaHvlxLkAhXD/XAOtSq2jQ5KK8TEf51YTIbdh/i1vdW8MlNp9IhrvJVskpHQ7dt1pDz+2hjpSeEhwaT4qczv4YGB/H0xL6UlPzKg1+kESQw5VRNEN6mX8GUT0SEBfPfK/oTGhLE1DeXcvBIYaX7zknLIm1nLn8a1UlLDQEiNDiIZy/ryxk9Erj/8zTe/GWLr0MKOPqfpnymddOGPH9ZP7bk5HPbjJWUVDBFdWmpoZ2WGgJOaHAQz07qx9juCdz7aSpvLdR1XLxJk4PyqcEdmnHPWd34dm0WT8/dcML2b7TUENDCQoJ4/rJ+jOkWzz8+WcM7izRBeIv+tymfmzKkHRf2a8XTczfwTequY48bY3j62w0kxUZynpYaAlZYSBDPT+7HqK7x3D1zDdMXb/N1SAFBk4PyORHhkQt6kdy6MbfNWMHG3QcB+Dq1tNTQUUsNAa5BSDAvXt6PkV3iuOvj1cxYognCafofp/xCeGgwL13en4iwYK57cxkH8gt5eq4tNZzbW0sNqjRB9Gd45zju/Hg17y/N8HVI9ZomB+U3WjaJ4PnL+pGxN5/zX/iZtVpqUOWEh9pebkM7xnLHR6v4cFmmr0Oqt/S/TvmVU9o3495zupOenUd7LTWoCoSHBvPKlQM4tUMsf/twJR8v1wThBB0Ep/zOFSltCRIhuXVjLTWoCpUmiGvfXMJfPlhJkAjn961guVhVY/qfp/yOiHB5SluSWzfxdSjKj0WEBfO/KweSktSMP7+/gk9XbPfo8Q8cLiT70FGKKxh/Ewi05KCUqrMiwoJ5dcoArn59CbfNWEGQCOdUoyqyoKiEbXvz2LQnj/TsPDbvOcRm1+2cvAIAggRiIhsQGxVGbFSZ39ENjrsfF92AmMiwerMAlSYHpVSd1jAshGlTBjLltSXcOmMFIhy3YJAxhqzco2zOth/89sP/EJuz88jYm0/ZgkFsVAPax0UytnsC7eMiCQsOIvtQAdmHjpJ96Ch7DhWwJSeP7ENHOVJYUkE00LRhqCtplCaQMPolNmVs9wTCQ+vOMqk6ZbdSql7IO1rEVa8tYdm2fVyR0pacvALSsw+RviePvILiY/uFhwaRFBtF+7hIOsRGkhQXSfvYKJLiImkUHurWuYwx5BUUk33w6HGJo+z90qSy5+BR8guKaRQewtm9WzKhf2v6tmni9WnRdT0HpVTAOnS0iOveWMrC9BxaN42wH/qxkXSIi6R9nL3dvFG4V9esLikx/LI5hw+XZTJ7zU6OFJbQPjaSi/q35sJ+rWjR2DvTpWtyUEoFNGMMhcWGsBD/q/s/eKSQ2at38eGyTBZv2YsIDO0Yy4T+rTm9e3MiwpyrdtLkoJRSdcDWnDw+Wr6dj5Zlsn3/YaIahHB2cgsm9G9N/7ZNPV7tpMlBKaXqkJISw6L0vceqnfILimnXrCEX9WvNhf1b08pDq/RpclBKqToq72gRs9fs4sNlGSzcJgrPSAAACBFJREFUbKudBrdvxoT+rRnXszkNw2rewVSTg1JK1QMZe/P5ePl2Plqeyba9+USGBXPb2M5ce1r7Gh2vuslBxzkopZQfahPTkFvGdOLm0R1ZsmUfHy7L8FrPJtDkoJRSfk1EGJQUw6CkGK+e1//6eimllPI5TQ5KKaVOoMlBKaXUCRxNDiIyTkR+E5GNInJnBdsni8gq188CEentZDxKKaXc41hyEJFg4HngTKA7MElEupfbLR0YboxJBv4JvOxUPEoppdznZMlhELDRGLPZGFMAvAecV3YHY8wCY8w+192FQGsH41FKKeUmJ5NDKyCjzP1M12OVuQaYXdEGEZkqIktFZOmePXs8GKJSSqmKOJkcKpo1qsLh2CIyEpsc7qhouzHmZWPMAGPMgLi4OA+GqJRSqiJODoLLBNqUud8a2FF+JxFJBv4HnGmMyTnZQZctW5YtIltrGFMskF3D5/qKxuwddS3muhYvaMzeUlnMbatzEMfmVhKREGA9MBrYDiwBLjPGpJbZJxH4DrjSGLPAkUCOj2lpdeYW8Qcas3fUtZjrWrygMXuLp2J2rORgjCkSkZuAr4FgYJoxJlVErndtfwm4F2gGvOCau7yorv0hlFKqPnJ0biVjzCxgVrnHXipz+1rgWidjUEopVX2BNkK6Lo6j0Ji9o67FXNfiBY3ZWzwSc51bz0EppZTzAq3koJRSyg2aHJRSSp2gXiYHNyb8ExF5xrV9lYj080WcZeJpIyLzRGStiKSKyC0V7DNCRA6IyArXz72+iLVcTFtEZLUrnhPWbvXD69ylzPVbISK5InJruX18ep1FZJqI7BaRNWUeixGROSKywfW7aSXPrfJ97+WYHxeRda6/+0wRaVLJc6t8D3k55vtFZHuZv/34Sp7rT9d5Rpl4t4jIikqeW/3rbIypVz/YbrObgPZAGLAS6F5un/HYqToESAEW+TjmFkA/1+1o7PiQ8jGPAL7w9fUtF9MWILaK7X51nSt4n+wC2vrTdQaGAf2ANWUeewy403X7TuDflbyeKt/3Xo75dCDEdfvfFcXsznvIyzHfD/zVjfeN31znctufAO711HWujyWHk07457r/prEWAk1EpIW3Ay1ljNlpjFnuun0QWEvV81DVFX51ncsZDWwyxtR0tL0jjDHzgb3lHj4PeMN1+w3g/Aqe6s773hEVxWyM+cYYU+S663eTalZynd3hV9e5lNiBYpcA0z11vvqYHNyZ8K+6kwJ6jYi0A/oCiyrYPFhEVorIbBHp4dXAKmaAb0RkmYhMrWC7315nYCKV/yP523VOMMbsBPtFAoivYB9/vtZXU8mkmpz8PeRtN7mqwqZVUn3nr9f5NCDLGLOhku3Vvs71MTm4M+Gf25MCepOIRAEfAbcaY3LLbV6OrQLpDTwLfOLt+CpwqjGmH3bNjj+KyLBy2/31OocB5wIfVLDZH6+zO/z1Wt8NFAHvVLLLyd5D3vQi0AHoA+zEVtOU55fXGZhE1aWGal/n+pgc3Jnwz61JAb1JREKxieEdY8zH5bcbY3KNMYdct2cBoSIS6+Uwy8e0w/V7NzATW+Quy++us8uZwHJjTFb5Df54nYGs0uo41+/dFezjd9daRP4AnA1MNq6K7/LceA95jTEmyxhTbIwpAV6pJBZ/vM4hwIXAjMr2qcl1ro/JYQnQSUSSXN8QJwKfldvnM+BKV2+aFODA/2/vbkLjqsIwjv8fLYhJSlSw+LFQrJsqSGili7Rd2YW4KAqRgm0IxU2gLtyJqBS6112hWQiNbVaKRSldNYtAFiFisNG2Um1XgS5LIIoi8XVx3qnj3Ewy7ZCZizw/GHI5OXM598yZ+96Pue9pnLb3Q14v/By4ERGftanzVNZD0n7KZ7dlFtvtImlQ0s7GMuUG5E8t1WrVz03aHmXVrZ/Tt8BELk8A32xQp5Nx3zOSXqek4D8SEb+3qdPJGOqZlvthb7VpS636OR0Gfo6IlY3++cD93Iu77L1+UX4lc5Pyq4KPsmwSmMxlUaYwvQX8CLza5/YepJyaLgM/5OuNlja/B1yj/DpiARjtc5tfyLZczXbVvp+zTQOUnf1wU1lt+pkStO4Af1GOUt+lJKecBX7Jv09k3WeAy03vrYz7Prb5V8q1+cZ4Ptva5nZjqI9tPp/jdJmyw3+67v2c5eca47epbtf97PQZZmZW8X+8rGRmZl1ycDAzswoHBzMzq3BwMDOzCgcHMzOrcHAw66HM+nqp3+0w24qDg5mZVTg4mG1A0nFJi5n/fkrSw5LWJH0qaUnSrKQns+6IpIWmuQsez/IXJV3JJH5Lknbn6ockfZXzHcw0nsg2qxMHB7MWkvYARynJykaAdeAYMEjJybQXmANO5Vu+AD6IiFcoT9g2ymeAM1GS+I1Snm6FknX3feAlytOrB7Z9o8zu045+N8Cshl4D9gHf5UH9o5Rkd3/zb3KzC8DXkoaBxyJiLsungS8zl82zEXERICL+AMj1LUbmwcmZu54H5rd/s8w65+BgViVgOiI+/E+h9ElLvc1yz2x2qejPpuV1/D20GvJlJbOqWWBM0i64N4fzc5Tvy1jWeQeYj4hV4K6kQ1k+DsxFmY9jRdKbuY5HJA30dCvMuuAjFrMWEXFd0seUmbMeomTBPAn8Brws6XtglXJfAkoa7bO5878NnMjycWBK0ulcx9s93Ayzrjgrq1mHJK1FxFC/22HWC76sZGZmFT5zMDOzCp85mJlZhYODmZlVODiYmVmFg4OZmVU4OJiZWcU/wsBs1NTTl60AAAAASUVORK5CYII=\n",
      "text/plain": [
       "<Figure size 432x288 with 1 Axes>"
      ]
     },
     "metadata": {
      "needs_background": "light"
     },
     "output_type": "display_data"
    }
   ],
   "source": [
    "from matplotlib import pyplot\n",
    "# plot train and validation loss\n",
    "pyplot.plot(history.history['loss'])\n",
    "pyplot.plot(history.history['val_loss'])\n",
    "pyplot.title('model train vs validation loss')\n",
    "pyplot.ylabel('loss')\n",
    "pyplot.xlabel('epoch')\n",
    "pyplot.legend(['train', 'validation'], loc='upper right')\n",
    "pyplot.show() "
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 6,
   "metadata": {},
   "outputs": [
    {
     "name": "stdout",
     "output_type": "stream",
     "text": [
      "INFO:tensorflow:Assets written to: saved_model/model3_50epochs_bit/assets\n"
     ]
    },
    {
     "name": "stderr",
     "output_type": "stream",
     "text": [
      "INFO:tensorflow:Assets written to: saved_model/model3_50epochs_bit/assets\n"
     ]
    }
   ],
   "source": [
    "import tensorflow as tf \n",
    "# Save fine-tuned model as SavedModel\n",
    "export_module_dir = 'saved_model/model3_50epochs_bit'\n",
    "tf.saved_model.save(model, export_module_dir)"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "### Experiment 3: "
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 4,
   "metadata": {},
   "outputs": [
    {
     "name": "stdout",
     "output_type": "stream",
     "text": [
      "WARNING:tensorflow:From /opt/conda/lib/python3.7/site-packages/tensorflow_core/python/ops/resource_variable_ops.py:1781: calling BaseResourceVariable.__init__ (from tensorflow.python.ops.resource_variable_ops) with constraint is deprecated and will be removed in a future version.\n",
      "Instructions for updating:\n",
      "If using Keras pass *_constraint arguments to layers.\n"
     ]
    },
    {
     "name": "stderr",
     "output_type": "stream",
     "text": [
      "WARNING:tensorflow:From /opt/conda/lib/python3.7/site-packages/tensorflow_core/python/ops/resource_variable_ops.py:1781: calling BaseResourceVariable.__init__ (from tensorflow.python.ops.resource_variable_ops) with constraint is deprecated and will be removed in a future version.\n",
      "Instructions for updating:\n",
      "If using Keras pass *_constraint arguments to layers.\n"
     ]
    },
    {
     "name": "stdout",
     "output_type": "stream",
     "text": [
      "Model sucessfuly built\n",
      "callbacks and data okay\n",
      "Train on 10 steps, validate on 4 steps\n",
      "WARNING:tensorflow:From /opt/conda/lib/python3.7/site-packages/keras/callbacks/tensorboard_v1.py:200: The name tf.summary.merge_all is deprecated. Please use tf.compat.v1.summary.merge_all instead.\n",
      "\n"
     ]
    },
    {
     "name": "stderr",
     "output_type": "stream",
     "text": [
      "WARNING:tensorflow:From /opt/conda/lib/python3.7/site-packages/keras/callbacks/tensorboard_v1.py:200: The name tf.summary.merge_all is deprecated. Please use tf.compat.v1.summary.merge_all instead.\n",
      "\n"
     ]
    },
    {
     "name": "stdout",
     "output_type": "stream",
     "text": [
      "WARNING:tensorflow:From /opt/conda/lib/python3.7/site-packages/keras/callbacks/tensorboard_v1.py:203: The name tf.summary.FileWriter is deprecated. Please use tf.compat.v1.summary.FileWriter instead.\n",
      "\n"
     ]
    },
    {
     "name": "stderr",
     "output_type": "stream",
     "text": [
      "WARNING:tensorflow:From /opt/conda/lib/python3.7/site-packages/keras/callbacks/tensorboard_v1.py:203: The name tf.summary.FileWriter is deprecated. Please use tf.compat.v1.summary.FileWriter instead.\n",
      "\n"
     ]
    },
    {
     "name": "stdout",
     "output_type": "stream",
     "text": [
      "WARNING:tensorflow:Issue encountered when serializing variables.\n",
      "Type is unsupported, or the types of the items don't match field type in CollectionDef. Note this is a warning and probably safe to ignore.\n",
      "'list' object has no attribute 'name'\n"
     ]
    },
    {
     "name": "stderr",
     "output_type": "stream",
     "text": [
      "WARNING:tensorflow:Issue encountered when serializing variables.\n",
      "Type is unsupported, or the types of the items don't match field type in CollectionDef. Note this is a warning and probably safe to ignore.\n",
      "'list' object has no attribute 'name'\n"
     ]
    },
    {
     "name": "stdout",
     "output_type": "stream",
     "text": [
      "Epoch 1/100\n",
      " 9/10 [==========================>...] - ETA: 28s - loss: 1.4719 - acc: 0.6526WARNING:tensorflow:From /opt/conda/lib/python3.7/site-packages/keras/callbacks/tensorboard_v1.py:343: The name tf.Summary is deprecated. Please use tf.compat.v1.Summary instead.\n",
      "\n"
     ]
    },
    {
     "name": "stderr",
     "output_type": "stream",
     "text": [
      "WARNING:tensorflow:From /opt/conda/lib/python3.7/site-packages/keras/callbacks/tensorboard_v1.py:343: The name tf.Summary is deprecated. Please use tf.compat.v1.Summary instead.\n",
      "\n"
     ]
    },
    {
     "name": "stdout",
     "output_type": "stream",
     "text": [
      "10/10 [==============================] - 424s 42s/step - loss: 1.5126 - acc: 0.6627 - val_loss: 0.7287 - val_acc: 0.8322\n",
      "Epoch 2/100\n",
      "10/10 [==============================] - 382s 38s/step - loss: 1.1978 - acc: 0.7949 - val_loss: 0.6690 - val_acc: 0.7776\n",
      "Epoch 3/100\n",
      "10/10 [==============================] - 379s 38s/step - loss: 0.7104 - acc: 0.8109 - val_loss: 0.6049 - val_acc: 0.7633\n",
      "Epoch 4/100\n",
      "10/10 [==============================] - 381s 38s/step - loss: 0.6281 - acc: 0.7891 - val_loss: 0.4030 - val_acc: 0.8270\n",
      "Epoch 5/100\n",
      "10/10 [==============================] - 379s 38s/step - loss: 0.4753 - acc: 0.8291 - val_loss: 0.3899 - val_acc: 0.8069\n",
      "Epoch 6/100\n",
      "10/10 [==============================] - 384s 38s/step - loss: 0.4342 - acc: 0.8279 - val_loss: 0.4368 - val_acc: 0.8075\n",
      "Epoch 7/100\n",
      "10/10 [==============================] - 390s 39s/step - loss: 0.5034 - acc: 0.8152 - val_loss: 0.4590 - val_acc: 0.7939\n",
      "Epoch 8/100\n",
      "10/10 [==============================] - 383s 38s/step - loss: 0.4319 - acc: 0.8420 - val_loss: 0.4107 - val_acc: 0.8075\n",
      "Epoch 9/100\n",
      "10/10 [==============================] - 387s 39s/step - loss: 0.3253 - acc: 0.8691 - val_loss: 0.4219 - val_acc: 0.8075\n",
      "Epoch 10/100\n",
      "10/10 [==============================] - 384s 38s/step - loss: 0.2804 - acc: 0.8814 - val_loss: 0.4093 - val_acc: 0.8017\n",
      "Epoch 11/100\n",
      "10/10 [==============================] - 381s 38s/step - loss: 0.3401 - acc: 0.8596 - val_loss: 0.4013 - val_acc: 0.8134\n",
      "Epoch 12/100\n",
      "10/10 [==============================] - 383s 38s/step - loss: 0.3267 - acc: 0.8609 - val_loss: 0.5038 - val_acc: 0.7867\n",
      "Epoch 13/100\n",
      "10/10 [==============================] - 384s 38s/step - loss: 0.3266 - acc: 0.8682 - val_loss: 0.4438 - val_acc: 0.7997\n",
      "Epoch 14/100\n",
      "10/10 [==============================] - 384s 38s/step - loss: 0.2915 - acc: 0.8768 - val_loss: 0.4581 - val_acc: 0.7952\n",
      "Epoch 15/100\n",
      "10/10 [==============================] - 388s 39s/step - loss: 0.2969 - acc: 0.8730 - val_loss: 0.4330 - val_acc: 0.7991\n",
      "Epoch 16/100\n",
      "10/10 [==============================] - 379s 38s/step - loss: 0.3203 - acc: 0.8656 - val_loss: 0.4518 - val_acc: 0.7984\n",
      "Epoch 17/100\n",
      "10/10 [==============================] - 385s 38s/step - loss: 0.2346 - acc: 0.9074 - val_loss: 0.4493 - val_acc: 0.7958\n",
      "Epoch 18/100\n",
      "10/10 [==============================] - 380s 38s/step - loss: 0.2373 - acc: 0.9025 - val_loss: 0.4246 - val_acc: 0.8114\n",
      "Epoch 19/100\n",
      "10/10 [==============================] - 386s 39s/step - loss: 0.2524 - acc: 0.8918 - val_loss: 0.4222 - val_acc: 0.8140\n",
      "Epoch 20/100\n",
      "10/10 [==============================] - 379s 38s/step - loss: 0.2403 - acc: 0.8994 - val_loss: 0.4180 - val_acc: 0.8062\n",
      "Epoch 21/100\n",
      "10/10 [==============================] - 385s 39s/step - loss: 0.2189 - acc: 0.9113 - val_loss: 0.4289 - val_acc: 0.7984\n",
      "Epoch 22/100\n",
      "10/10 [==============================] - 380s 38s/step - loss: 0.2196 - acc: 0.9100 - val_loss: 0.4346 - val_acc: 0.7939\n",
      "Epoch 23/100\n",
      "10/10 [==============================] - 383s 38s/step - loss: 0.2052 - acc: 0.9195 - val_loss: 0.4328 - val_acc: 0.7984\n",
      "Epoch 24/100\n",
      "10/10 [==============================] - 387s 39s/step - loss: 0.1939 - acc: 0.9252 - val_loss: 0.4341 - val_acc: 0.7984\n",
      "Epoch 25/100\n",
      " 9/10 [==========================>...] - ETA: 27s - loss: 0.1995 - acc: 0.9214Restoring model weights from the end of the best epoch\n",
      "10/10 [==============================] - 397s 40s/step - loss: 0.1999 - acc: 0.9203 - val_loss: 0.4361 - val_acc: 0.7984\n",
      "Epoch 00025: early stopping\n",
      "Training model is finished\n",
      "5/5 [==============================] - 172s 34s/step - loss: 0.4082 - acc: 0.8098\n",
      "Test accuracy: 0.8097561\n",
      "CPU times: user 13h 46min 40s, sys: 3h 14min 39s, total: 17h 1min 20s\n",
      "Wall time: 2h 45min 59s\n"
     ]
    }
   ],
   "source": [
    "%%time\n",
    "history,model = BiT_L_ResNet.train_Bit_L(train,validation,test,model_name=\"r101x1\",epochs=100,patience=20)"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 5,
   "metadata": {},
   "outputs": [
    {
     "data": {
      "image/png": "iVBORw0KGgoAAAANSUhEUgAAAYYAAAEWCAYAAABi5jCmAAAABHNCSVQICAgIfAhkiAAAAAlwSFlzAAALEgAACxIB0t1+/AAAADh0RVh0U29mdHdhcmUAbWF0cGxvdGxpYiB2ZXJzaW9uMy4yLjEsIGh0dHA6Ly9tYXRwbG90bGliLm9yZy+j8jraAAAgAElEQVR4nO3dd3gc5bX48e9R78WSLNlykZtsY+OGG9gGOzQDIUAuYDtAAsEYSLlJbsKFVJLcFO5N4AcpYExvMcXUhBq6HdyJe5WrZNmWLFu9S+f3x4zNSlZZldVK2vN5nn12p71zZleaM/O+M++IqmKMMcacFOTvAIwxxnQvlhiMMcY0YInBGGNMA5YYjDHGNGCJwRhjTAOWGIwxxjRgicF0GhF5UkR+4+W8+0XkAh/Gcp2IvOer8n1JRH4pIs+6nweJSKmIBLc2bzvXtVVEZrd3+RbK/VhEFnZ2uaZrhPg7AGMaE5EngRxV/Vl7y1DV54DnOi0oP1HVg0BMZ5TV1PeqqmM6o2zTu9gZg+lxRMQOaIzxIUsMAcatwrlDRDaJSJmIPCYiqSLytoiUiMj7IpLoMf9X3OqGQrd6YLTHtIki8rm73AtARKN1fVlENrjLfiYi47yIbxFwHfDfbhXK3z3ivlNENgFlIhIiIneJyB53/dtE5CqPcm4UkRUewyoit4nIbhE5ISJ/FRFpYv39RaRCRPo02s5jIhIqIsNF5BMRKXLHvdDMdrwjIt9pNG6jiHzV/fyAiGSLSLGIrBeRWc2Uk+HGHuIOD3HXXyIi/wSSG83/kogcceP7VETGePG9XuB+DheR+0Uk133dLyLh7rTZIpIjIj8UkTwROSwiNzX9K562DUEi8jMROeAu+7SIxLvTIkTkWREpcP9O1opIqjvtRhHZ627rPhG5zpv1mU6gqvYKoBewH1gFpALpQB7wOTARCAc+BO52580EyoALgVDgv4EsIMx9HQB+4E67GqgBfuMuO8ktexoQDHzDXXe4RxwXNBPjkyfLaRT3BmAgEOmOuwboj3OAM8+NtZ877UZghcfyCvwDSAAGAfnA3GbW/yFwi8fwH4DF7uelwE/ddUYAM5sp4+vAvzyGzwAKPbb/eiAJpzr3h8ARIMKd9kvgWfdzhht7iDu8ErjP/a3OBUpOzutO/yYQ606/H9jgxfd6gfv51+7fRl8gBfgM+B932myg1p0nFLgUKAcSm9n+j4GFHjFlAUNxqsVeAZ5xp90K/B2Icv9OzgLigGigGBjpztcPGOPv/59AedkZQ2D6s6oeVdVDwHJgtar+W1WrgFdxkgQ4O9s3VfWfqloD/BGIBM4BpuPsIO5X1RpVXQas9VjHLcDDqrpaVetU9Smgyl2uvf6kqtmqWgGgqi+paq6q1qvqC8BuYGoLy9+jqoXq1Nt/BExoZr6/AQsA3LOK+e44cJLfYKC/qlaq6oqmi+BVYIKIDHaHrwNecb9jVPVZVS1Q1VpVvRdnRz6ypY0XkUHAFODnqlqlqp/i7FRPUdXHVbXEXc8vgfEnj869cB3wa1XNU9V84FfADR7Ta9zpNar6FlDaWswe5d6nqntVtRT4MTDfPQuqwUmQw92/k/WqWuwuVw+MFZFIVT2sqlu93A7TQZYYAtNRj88VTQyfbOzsj3NWAICq1gPZOGca/YFDqurZC+MBj8+DgR+61QOFIlKIc7TfvwNxZ3sOiMjXPaqqCoGxNKpaaeSIx+dymm/UXQacLSL9cY7KFSeBgnPWJMAat4rtm00VoKolwJs4SQX3/VRjuFsls92t8ikE4luJHZzv7oSqlnmMO/Wdi0iwiNzjVq8V45wN4EW5nuV7/oYHaPh7FahqrcdwS99ha+WG4Jy1PgO8CzzvVl/9n4iEuts4D7gNOCwib4rIKC+3w3SQJQbTklycHTxw6uh5IHAIOAykN6qnH+TxORv4raomeLyiVHWpF+ttrsvfU+PdI/FHgO8ASaqaAGzB2Wl3iKoWAu8B1wJfA5aeTICqekRVb1HV/jjVIA+KyPBmiloKLBCRs3HOtD5yY58F3OmWn+jGXuRF7IeBRBGJ9hjn+Z1/DbgCuAAn0WS440+W21pXyg1+b7fs3FaW8UZT5dYCR92zj1+p6hk4Z6JfxqmGQ1XfVdULcaqRduD83qYLWGIwLXkRuExEzheRUJy68CqcuueVOP/c/+k2BH+VhtU4jwC3icg0cUSLyGUiEuvFeo/i1Ee3JBpnR5cP4DaEjm3LxrXibzg7qP/gi2okROQaERngDp5wY6hrpoy3cHaIvwZecM+4wGkDqHVjDxGRX+DUq7dIVQ8A64BfiUiYiMwELveYJRbn9ynAqbP/XaMiWvtelwI/E5EUEUkGfgG0+x6JRuX+wG04j3HjekFVa0VkjoicKc59GsU4VUt14lwQ8RU3CVbhVFs19z2bTmaJwTRLVXfiNJL+GTiGsxO6XFWrVbUa+CpOI+8JnNP+VzyWXYfTzvAXd3qWO683HgPOcKuIXmsmtm3AvTgJ6ihwJvCvtm1hi94ARuAc1W70GD8FWC0ipe4831PVfc3EWIXznVyAR3LBqTp5G9iFU61SSaNqshZ8DadB/zhwN/C0x7Sn3fIOAdtwGpI9tfa9/gYn8WwCNuNclODVDYuteBynyuhTYB/O9n7XnZaGU3VXDGwHPsFJRkE4ByK5ONt6HvCtTojFeEEaVhEbY4wJdHbGYIwxpgFLDMYYYxqwxGCMMaYBSwzGGGMa6HGdkSUnJ2tGRoa/wzDGmB5l/fr1x1Q1xZt5e1xiyMjIYN26df4OwxhjehQROdD6XA6rSjLGGNOAJQZjjDENWGIwxhjTQI9rYzDG9C41NTXk5ORQWVnp71B6hYiICAYMGEBoaGi7y7DEYIzxq5ycHGJjY8nIyEBOf6ieaQNVpaCggJycHIYMGdLucqwqyRjjV5WVlSQlJVlS6AQiQlJSUofPviwxGGP8zpJC5+mM7zJgEsOuoyX85h/bqKyxLt2NMaYlAZMYck6U8+iKfazbf8LfoRhjupHCwkIefPDBNi936aWXUlhY6IOI/C9gEsO0IUmEBgvLs/L9HYoxphtpLjHU1bVcu/DWW2+RkJDgq7D8KmCuSooOD2HioET+lXXM36EYY7qRu+66iz179jBhwgRCQ0OJiYmhX79+bNiwgW3btnHllVeSnZ1NZWUl3/ve91i0aBHwRfc8paWlXHLJJcycOZPPPvuM9PR0Xn/9dSIjI/28Ze0XMIkBYNbwZO57fxfHy6rpEx3m73CMMY386u9b2ZZb3KllntE/jrsvH9Ps9HvuuYctW7awYcMGPv74Yy677DK2bNly6nLPxx9/nD59+lBRUcGUKVP4j//4D5KSkhqUsXv3bpYuXcojjzzCtddey8svv8z111/fqdvRlQKmKglgxohkVLGzBmNMs6ZOndrgHoA//elPjB8/nunTp5Odnc3u3btPW2bIkCFMmDABgLPOOov9+/d3Vbg+EVBnDOPS44mNCGHF7mNcPr6/v8MxxjTS0pF9V4mOjj71+eOPP+b9999n5cqVREVFMXv27CbvEQgPDz/1OTg4mIqKii6J1VcC6owhJDiIc4YlsSLrGKrq73CMMd1AbGwsJSUlTU4rKioiMTGRqKgoduzYwapVq7o4Ov8IqDMGgJkjUnh361H2F5QzJDm69QWMMb1aUlISM2bMYOzYsURGRpKamnpq2ty5c1m8eDHjxo1j5MiRTJ8+3Y+Rdp2ASwyzhicDsGJ3viUGYwwAf/vb35ocHx4ezttvv93ktJPtCMnJyWzZsuXU+B/96EedHl9XC6iqJIDBSVGkJ0SyfLc1QBtjTFMCLjGICLNGJLNyTwG1dfX+DscYY7qdgEsMADNHJFNSVcvGnCJ/h2KMMd1OQCaGGcOSEbH7GYwxpikBmRgSo8MY2z+eFdbOYIwxp/FZYhCRx0UkT0S2tDLfFBGpE5GrfRVLU2YMT+bzgycorartytUaY0y358szhieBuS3NICLBwP8C7/owjibNGpFMbb2yem9BV6/aGNODxcTEAJCbm8vVVzd9PDt79mzWrVvXYjn3338/5eXlp4a7UzfePksMqvopcLyV2b4LvAzk+SqO5pw1OJHwkCC7bNUY0y79+/dn2bJl7V6+cWLoTt14+62NQUTSgauAxV7Mu0hE1onIuvz8znmeQkRoMFOH9GGFNUAbE9DuvPPOBs9j+OUvf8mvfvUrzj//fCZNmsSZZ57J66+/ftpy+/fvZ+zYsQBUVFQwf/58xo0bx7x58xr0lXT77bczefJkxowZw9133w04HfPl5uYyZ84c5syZAzjdeB875uyP7rvvPsaOHcvYsWO5//77T61v9OjR3HLLLYwZM4aLLrrIZ30y+fPO5/uBO1W1rrVnlKrqEmAJwOTJkzutk6NZI5L53Vs7OFJUSVp8RGcVa4xpr7fvgiObO7fMtDPhknuanTx//ny+//3v861vfQuAF198kXfeeYcf/OAHxMXFcezYMaZPn85XvvKVZp+n/NBDDxEVFcWmTZvYtGkTkyZNOjXtt7/9LX369KGuro7zzz+fTZs28Z//+Z/cd999fPTRRyQnJzcoa/369TzxxBOsXr0aVWXatGmcd955JCYmdln33v68Kmky8LyI7AeuBh4UkSu7MoCZw1MA7KzBmAA2ceJE8vLyyM3NZePGjSQmJtKvXz9+8pOfMG7cOC644AIOHTrE0aNHmy3j008/PbWDHjduHOPGjTs17cUXX2TSpElMnDiRrVu3sm3bthbjWbFiBVdddRXR0dHExMTw1a9+leXLlwNd1723384YVPVUh+ci8iTwD1V9rStjGJUWS1J0GCt253P1WQO6ctXGmKa0cGTvS1dffTXLli3jyJEjzJ8/n+eee478/HzWr19PaGgoGRkZTXa37amps4l9+/bxxz/+kbVr15KYmMiNN97Yajkt9fzcVd17+/Jy1aXASmCkiOSIyM0icpuI3OardbZVUJAwY3gyK7IKrBtuYwLY/Pnzef7551m2bBlXX301RUVF9O3bl9DQUD766CMOHDjQ4vLnnnsuzz33HABbtmxh06ZNABQXFxMdHU18fDxHjx5t0CFfc919n3vuubz22muUl5dTVlbGq6++yqxZszpxa1vnszMGVV3Qhnlv9FUcrZk5Ipk3Nuay40gJo/vF+SsMY4wfjRkzhpKSEtLT0+nXrx/XXXcdl19+OZMnT2bChAmMGjWqxeVvv/12brrpJsaNG8eECROYOnUqAOPHj2fixImMGTOGoUOHMmPGjFPLLFq0iEsuuYR+/frx0UcfnRo/adIkbrzxxlNlLFy4kIkTJ3bpU+Gkpx0pT548WVu7PrgtDhdVcPbvP+Rnl41m4ayhnVauMcY727dvZ/To0f4Oo1dp6jsVkfWqOtmb5QOySwxP/eIjGZYSbfczGGOMK+ATA8DM4cms3ldAVW2dv0Mxxhi/s8SA87jPypp61h844e9QjAlIPa1KuzvrjO/SEgMwfWgfgoPEels1xg8iIiIoKLArAzuDqlJQUEBERMdu2A24Zz43JTYilIkDE+z5DMb4wYABA8jJyaGzursJdBEREQwY0LH7siwxuGaOSOaBD3ZTWF5NQlSYv8MxJmCEhoYyZMiQ1mc0XcaqklwzhyejCp/tsW64jTGBzRKDa/zABGLCQ+yyVWNMwLPE4AoNDmL60CRWZFk9pzEmsFli8DBrRDLZxys4WFDe+szGGNNLWWLwMHOE0y/6cjtrMMYEMEsMHoYmR9MvPsLuZzDGBDRLDB5EhJnDk/lsTwF19XazjTEmMFliaGTmiGSKKmrYfKjI36EYY4xfWGJoZMZwp53B7oI2xgQqSwyNJMeEc0a/OJbvtgZoY0xgssTQhFkjkll/4ATl1bX+DsUYY7qcJYYmzBieTE2dsnrfcX+HYowxXc4SQxOmDulDWEiQXbZqjAlIlhiaEBEazJSMRGuANsYEJJ8lBhF5XETyRGRLM9OvE5FN7uszERnvq1jaY+bwFHYcKSGvpNLfoRhjTJfy5RnDk8DcFqbvA85T1XHA/wBLfBhLm80aYZetGmMCk88Sg6p+CjTbequqn6nqyYcsrwI69sihTnZGvzgSo0KtG25jTMDpLm0MNwNvNzdRRBaJyDoRWddVj/8LChLOHpbE6r12ZZIxJrD4PTGIyBycxHBnc/Oo6hJVnayqk1NSUrostjH94zlUWEFJZU2XrdMYY/zNr4lBRMYBjwJXqGq3e6ZmZmosALuOlvo5EmOM6Tp+SwwiMgh4BbhBVXf5K46WjEo7mRhK/ByJMcZ0nRBfFSwiS4HZQLKI5AB3A6EAqroY+AWQBDwoIgC1qjrZV/G0R3pCJFFhwew8YonBGBM4fJYYVHVBK9MXAgt9tf7OEBQkZKbGWmIwxgQUvzc+d3cjU2OtKskYE1AsMbQiMy2WgrJq8kuq/B2KMcZ0CUsMrbAGaGNMoLHE0IqTl6zusHYGY0yAsMTQipTYcJKiw9hlicEYEyAsMXghMzWWHVaVZIwJEJYYvDAyLZbdR0uor1d/h2KMMT5nicELI9NiKa+u41Bhhb9DMcYYn7PE4AVrgDbGBBJLDF7ITI0B7JJVY0xgsMTghdiIUNITIu2MwRgTECwxeGlUWqxdsmqMCQiWGLyUmRbLnvxSqmvr/R2KMcb4lCUGL41Ki6W2Xtl3rMzfoRhjjE9ZYvDSySuTdloDtDGml7PE4KWhKdEEBwk7jxT7OxRjjPEpSwxeCg8JZmhyNDuP2POfjTG9myWGNshMi2XnUTtjMMb0bpYY2mBUaizZxysoq6r1dyjGGOMzlhjaINMe2mOMCQCWGNrAnuZmjAkEPksMIvK4iOSJyJZmpouI/ElEskRkk4hM8lUsnWVgYhQRoUHWAG2M6dV8ecbwJDC3hemXACPc1yLgIR/G0imCgoTMVGuANsb0bj5LDKr6KXC8hVmuAJ5WxyogQUT6+SqezjIyNdbOGIwxvZo/2xjSgWyP4Rx33GlEZJGIrBORdfn5+V0SXHNGpsVyrLSKgtIqv8ZhjDG+4s/EIE2Ma/LZmaq6RFUnq+rklJQUH4fVspFp1jWGMaZ382diyAEGegwPAHL9FIvXRp7sM8m64DbG9FL+TAxvAF93r06aDhSp6mE/xuOVlNhwEqNC7ZJVY0yvFeKrgkVkKTAbSBaRHOBuIBRAVRcDbwGXAllAOXCTr2LpTCLulUl2xmCM6aV8lhhUdUEr0xX4tq/W70sj02J55fNDqCoiTTWVGGNMz2V3PrfDyLRYSqtqOVRY4e9QjDGm01liaAdrgDbG9GaWGNoh0y5ZNcb0YpYY2iEuIpT+8RF2xmCM6ZUsMbTTyDS7MskY0ztZYminzLRY9uaXUVNX7+9QjDGmU1liaKeRqbFU19Wz/1iZv0MxxphOZYmhnazPJGNMb2WJoZ2GpcQQHCTWzmCM6XUsMbRTRGgwGUlRlhiMMb2OV4lBRL4nInFuh3ePicjnInKRr4Pr7kamxVpVkjGm1/H2jOGbqloMXASk4HR4d4/PouohRqbGcfB4OeXVtf4OxRhjOo23ieFkT3GXAk+o6kaaftBOQBmZFoMqZOXZoz6NMb2Ht4lhvYi8h5MY3hWRWCDgL+DPdPtM2mHtDMaYXsTbbrdvBiYAe1W1XET60EOen+BLg5OiCQ8JYpclBmNML+LtGcPZwE5VLRSR64GfAUW+C6tnCA4SRqTGWAO0MaZX8TYxPASUi8h44L+BA8DTPouqBxmZGmeXrBpjehVvE0Ot+8S1K4AHVPUBINZ3YfUcI9NiyCup4kRZtb9DMcaYTuFtYigRkR8DNwBvikgw7vObA93ItDjAusYwxvQe3iaGeUAVzv0MR4B04A8+i6oHOfk0t12WGIwxvYRXicFNBs8B8SLyZaBSVa2NAUiNCycuIsQuWTXG9BredolxLbAGuAa4FlgtIld7sdxcEdkpIlkiclcT0+NF5O8islFEtopIj7sEVkQYlRZnl6waY3oNb+9j+CkwRVXzAEQkBXgfWNbcAm47xF+BC4EcYK2IvKGq2zxm+zawTVUvd8vcKSLPqWqPasnNTIvh9Q25qCoiAX9DuDGmh/O2jSHoZFJwFXix7FQgS1X3ujv653GuavKkQKw4e9MY4DjQ4zoeGpkWR0llLYeLKv0dijHGdJi3ZwzviMi7wFJ3eB7wVivLpAPZHsM5wLRG8/wFeAPIxbn8dZ6qntbVhogsAhYBDBo0yMuQu87JBuidR0ronxDp52iMMaZjvG18vgNYAowDxgNLVPXOVhZrqk5FGw1fDGwA+uN0ufEXEYlrYv1LVHWyqk5OSUnxJuQudSox2JVJxphewNszBlT1ZeDlNpSdAwz0GB6Ac2bg6SbgHvfmuSwR2QeMwmno7jHio0JJi4uwBmhjTK/QYmIQkRJOP8oH52xAVfW0o3sPa4ERIjIEOATMB77WaJ6DwPnAchFJBUYCe72MvVvJTIu1S1aNMb1Ci4lBVdvd7YWq1orId4B3gWDgcVXdKiK3udMXA/8DPCkim3GSzZ2qeqy96/SnUWmxPLm3gNq6ekKC7Ympxpiey+uqpPZQ1bdo1EjtJoSTn3NxngrX42WmxlJdW8/+gnKG943xdzjGGNNudmjbSUalWdcYxpjewRJDJxneN4Ygsae5GWN6PksMnSQiNJiMpGi7MskY0+MFVmIoPuzT4jNTY60qyRjT4wVOYtj6GvxpAqx5BLSpK3A7LjMtlv0FZVTW1PmkfGOM6QqBkxgGnQ0ZM+GtH8Fz10DJ0U5fxai0WOoVsvJKO71sY4zpKoGTGGJT4bplcMkfYP9yeOgc2Pl2p64i0+0awxqgjTE9WeAkBgARmLYIFn0Msf1g6Xz4+/ehuqxTis9IiiImPIQnP9tHYXmP6jncGGNOCazEcFLf0XDLB3DOd2H9k/DwuZD77w4XGxIcxAPzJ7DrSCkLHllNQWlVx2M1xpguFpiJASAkHC76DXz9daipgEcvgOX3Qn3HGo7PH53Ko9+YzN78UhY8soq8EntGgzGmZwncxHDS0PPg9n/B6Mvhg1/Dk1+GwoMdKvLczBSeuHEK2ccrmP/wKo7YA3yMMT2IJQaAyES4+gm4cjEc2QwPzYBNL3aoyHOGJ/P0zVPJK6ni2odXknOivJOCNcYY37LEcJIITFgAt6+AvmfAK7fAspuhorDdRU7J6MMzN0/lRHk18x5excECSw7GmO7PEkNjiRlw45sw52ew9VVYPAty1re7uImDEvnbwumUVddy7cMr2Ztv9zgYY7o3SwxNCQ6B8+6Am99zhh+/GFYtbvcd02cOiOdvC6dTU1fPvCWr2G3dZhhjujFLDC0ZMBlu/QSGXwDv3AkvfQMqi9pV1Bn943h+0XQA5i9ZxfbDxZ0ZqTHGdBpLDK2J6gMLlsKFv4bt/4Als+HwpnYVNSI1lhcWTSc0OIgFj6xic077kowxxviSJQZviMCM7zltDyfveVj/ZLuqloamxPDirWcTHRbC1x5dxecHT3R+vMYY0wGWGNpi8Nlw2wrImAF//x68eitUtb0xeVBSFC/cOp3EqDBueHQ1a/cf90GwxhjTPpYY2io62emMb85PYfNL8MiXIG9Hm4sZkBjFi7eeTWpcBF9/bI31yGqM6TYsMbRHUDCc999ww2tQcQIemQMbn29zMWnxESxdNJ3gIOGet7f7IFBjjGk7nyYGEZkrIjtFJEtE7mpmntkiskFEtorIJ76Mp9MNPQ9uWw79JznVSm9812mDaIPUuAi+NWcY72/PY+WeAh8Faowx3vNZYhCRYOCvwCXAGcACETmj0TwJwIPAV1R1DHCNr+Lxmdg0pyO+WT+Cz5+GRy9sc19L35wxhP7xEfzure3U1/vm6XLGGOMtX54xTAWyVHWvqlYDzwNXNJrna8ArqnoQQFXzfBiP7wSHwPk/d9oeTuyHd3/SpsUjQoP50cUj2XyoiDc25vomRmOM8ZIvE0M6kO0xnOOO85QJJIrIxyKyXkS+3lRBIrJIRNaJyLr8/HwfhdsJRlwIU74JO96CokNtWvTKCemM6R/HH97dac+MNsb4lS8TgzQxrnE9SQhwFnAZcDHwcxHJPG0h1SWqOllVJ6ekpHR+pJ1p8s2g9bD+iTYtFhQk/PTS0RwqrOCJf+33TWzGGOMFXyaGHGCgx/AAoHE9SQ7wjqqWqeox4FNgvA9j8r3EwZA517kBrrZtT3A7Z3gy54/qy4MfZdnT34wxfuPLxLAWGCEiQ0QkDJgPvNFonteBWSISIiJRwDSg51+3OXUhlOXDtsab27q7LhlFWXUtf/4wyweBGWNM63yWGFS1FvgO8C7Ozv5FVd0qIreJyG3uPNuBd4BNwBrgUVXd4quYuszQL0GfobD2kTYvOiI1lvlTB/HsqgPWRbcxxi98eh+Dqr6lqpmqOkxVf+uOW6yqiz3m+YOqnqGqY1X1fl/G02WCgmDKLZC9Gg5vbPPi379gBOEhQfzvO22/o9oYYzrK7nz2lQlfg9AoWNP2s4a+sRHcdt4w3t16lDX7rB8lY0zXssTgK5EJcOY1sHmZ021GGy2cNZTUuHB++9Z2tJ0PCDLGmPawxOBLU2+B2gr493NtXjQyLJgfXjSSjdmF/GPTYR8EZ4wxTbPE4EtpZ8Kgs2Hto1Bf3+bF/2PSAEalxfK/7+ygqtZuejPGdA1LDL42ZSGc2Ad7PmjzosFBwk8vG03OiQqe/uyAD4IzxpjTWWLwtdFfgZjUdjVCA8wakcJ5mSn8+cPdFJZXd3JwxhhzOksMvhYSBmfdCLvfg+P72lXETy4dTWmV3fRmjOkalhi6wlk3ggTBusfatfjItFiuOWsgT6/cz4GCsk4NrTmqyoGCssC9IurEASjtmZ39GtNRlhi6Qlx/GP1l+PezbX6Qz0n/dVEmIUFB/N87Ozs5uNNV1tTxo5c2cd4fPuaOZZuorm17w3mPlr8LFs+EP092Ljc2JsBYYugqUxc59zNsebldi6fGRbDo3KG8ufkw6w+0/b4Ib+UVVzJ/ySpe/jyHL43qy7L1OVz/2GqOlwVI+0b5cVg6D0LCISUTXr4ZXlkElUX+jsyYLmOJoasMngEpo2HNEmhn9cyic4eSEhvO73x009vG7EIu/6lnLYEAABopSURBVMsKdh0tYfH1k3j8xik8MH8CG7ILuerBf5GV18v7bqqrhWU3QWE2zHsWbnoHZv/YOWtYPBMOrvJ3hMZ0CUsMXUXE6XX18EbIWdeuIqLDQ/jhhZmsP3CCd7Yc6dTwXvv3Ia55eCWhwUG8fPs5zB3bD4ArJqSz9JbplFXV8tUH/8W/so516nq7lXd/Ans/hsvvh0HTnSfzzb4LvvkOIPDEJfDhb6Guxt+RGuNTlhi60rj5EB7Xrl5XT7pm8kAyU2P48aubefiTPZRV1XYopLp65fdvb+f7L2xg4sAE3vjOTEb3i2swz1mDE3n1WzPoFx/J1x9fw99Wt+2Z1j3C+idhzcMw/dsw8fqG0wZOhdtWOL/fp/8Hj8+F43v9EqYxXcESQ1cKj4HxC2Drq1DavkeUBgcJf/3aJMb2j+f3b+9g5v9+yF8+3E1xZduPYosqarj5qbU8/Mlebpg+mGcXTqNPdFiT8w7sE8Wy289m1ohkfvLqZv7nH9uoq+8lVyztXwFv/hCGnQ8X/rrpeSLi4KqH4OrHoWA3LJ7ldHUSqFdtmV5NetrliJMnT9Z169pXFdMt5O+Cv06BL/0czv1Rh4paf+AEf/0oiw935BEbEcJN52Rw04whJDazc/e0N7+UhU+v42BBOb+6YgzXTRv8xcSyY7DvE8jbAePmQfLwU5Nq6+r5zZvbefKz/Zw/qi8PLJhITHhIh7bDr07shyVzICoJFr7vdH7YmqIceOVWOLACzrgSvvz/IKqP9+usr3PuacnbBlUlMPQ8iB/Q7k0wxhsisl5VJ3s1ryUGP3j6CjiWBd/b6NRjd9CWQ0X85cMs3tl6hOiwYK4/ezC3zBpKckx4k/N/vDOP7y79N6HBQTx03SSmpYfBgc9g7ydOQjjq8aykoBCYeiucdwdEJp4a/czK/fzy79sY0TeGx26cQnpCZIe3o8tVlcBjF0HxIbjlI0ga5v2y9XXw2Z/gw99AdF/46sMw5NxG89RD4X4nweZvh7ztzudju6Cu0aNbU8+EzIudx8KmT4Kg4A5vHpXFcGgd5G4ArYPgcAiJcG66DImAYPc9JNx9eYyLTISYbv58ddMmlhi6u+3/gBeug3nPOfc3dJKdR0r460dZ/GNTLmEhQSyYOohbzx1GWnwE4Ny09ujyffzx7c1cnnSIn5+RT/zhz5ydR32ts+MYNA2GnAdD5zj3X3z8e/j8aWdHMecncNZNp5LZJ7vy+c5znxMeGsyj35jMhIFeHG13F/X18ML1sOsduH4ZDPtS+8rJ/Te8vBAK9sD0253uT/J3OEkgf6fTu+5JcQOg72joO8q5Qq3vKGcnnPU+7HrXuepJ6yAqGUZc5CSKYV9yqrFaowrHdkPOGsheAzlrnRjowP933zGQeRGMuBgGTOmUgxjjP5YYuru6WnhgvHOE+o22Pxe6NXvzS3nw4z28+u9DBItwzeQB3Dqmjo3vLyXu8GdMD9lFuFY6d2P3n+gmgvNg4DQIbeLI//Am54qd/cudHdrFv4Xh5wOw+2gJ33xqLXnFVdx77Xi+PK5/p2+PT3zwa1h+L8z9X5h+W8fKqi6Dd38K659whmP7QcooNwmMdr6zlJGt7+DLj8OeD50ksfs9qCx0ztgGn+OcSWTO/eKspqoEDq2H7LVOMshZ+8VzP8LjYcBkp9F8wBRIP8t5aFRdFdSefFVCXbXzXlvdaLjKqS7Leh8OrnQOGiISYPgFTrIafkHbqs5M29TXQ32N83vUnXx3P4fHtftMzhJDT7D8Xmfn9O21zo1UPpB9vJzn/rmSzK33c6UsJ0iU45FDSDzzQmTobOfeCm/q1ME5It3xJrz3M6e32My5cNFvIHkEBaVV3PrMetYdOMEdF4/k23OGt16eP216CV5ZCJO+Dpf/ybmUuDOcOODs/D2q3NqtrtbZ2e96x0kU+dud8UnDnbOMvG2g7h3pKaOcBDBwKgyYCsmZzuNlO0NlkZus3oOsf0JZvnNAMWDKF2c1qWM77zvsbHU1TswVhU6irTjh8bmFceBUqwWHQlDoF5+D3c9BIe64MOdMKijEqV6sr3Xfazw+1zqvupqGw/U1p+/466qdac2Z8X248Fft+iosMfQEZcfgvtFO1cyl/9f55VeVwIr7YeVfUa1ndd9rqZmyiFmTxnWs3NoqWP0wfPoHqCl37ug+77+pCo3jjpc28cbGXP7fvPFcNdHLxtSTf39dtWM5tB6euBT6T4Kvv+7Ut/cEJ/Y7O+fd7znVTQOmwsApkD7Z++TeUfX1TtXZ7nedZHV4gzM+Lh1GXAhp475oszjVfhHmtm2cbNcI9xgXDmHRzs62IzGVHnGScuGB09+LD32RQE8jEBHvfH8RCc57ZKIzDmm0465puONualyQmyCCgt3EEdpwOCjESTKewyFhHgkmtPXPKaOgX/v+hy0x9BSvLIKdb8N/bYPw2M4ps64W/v00fPQ750jpzGucK6ASB7e+bFuU5sNHv3HaHyLiYc5PqZn4Da5/fD0bcwpZdts5jE2Pd+atKnH+UU/sP/1VeNCpvkoa7vEa5rz3Gepc4ttZig/DktnOP9iijyA6ufPKDkQlR2D3P51EsedjqC5pXznBboIIj4Ew9xUe44wLi204TYKcaq5TO/+Dpzfkx/aDhMHO33zCYIjr5+7wExomgfD4zjuz6gG6TWIQkbnAA0Aw8Kiq3tPMfFOAVcA8VW2x17JelRiy18JjF8Bl9zoP9OkIVedo8r2fw7GdMOgcuPg3Tv2yLx3ZDO/82G1/GEXp2BtY9snnDJCjzO5bTkjRQShvdLd0eJzzT5uY4fzj1pRDQRYU7IXinIbzxvZrmCyShkOfYc4/e1uSaU2Fc6aQvxNufg/SxnZ4042H2mooL3DbMaobtmecHHeqHcMdV1Pp/C7VJU47TVUpVLuvqlJnnOdwvXuvTkTCFzv9xAz3s/sePxBCI/z5TXRb3SIxiEgwsAu4EMgB1gILVHVbE/P9E6gEHg+oxKDqHMHWVsEtH0JYVPvKObzRqfvf96mz07zw1zDqsq6rnlGFnW85DbAn9qESTHZ9EsUR6ZwxZjxBfTI8/okznKO35mKrLnfuKi7Icl97vvhccbzhvKHRENMXYtOc95iT76kNx0UlwWu3weaXOv1KMNOFaqud5BAW7e9IeqS2JAZfXn82FchS1b1uUM8DVwDbGs33XeBlYIoPY+meRGDarfDa7fC7fs7RTvIIp/Hw5HvSCGcn19SOtCjHuY5+4/POzvaSP8DkmzpWZ9ve7Rh1mdMYWXoUiUlj5ee53PnyZm4LHcZdM0d5X1ZYlHM039QRfflxJ1Ec3+NUY5TmOfXLpXlwdJtTnVHVVC+oAih86WeWFHqykDCgh7QJ9XC+TAzpQLbHcA4wzXMGEUkHrgK+RAuJQUQWAYsABg0a1OmB+tX4Bc4R7eFNzo1Px3bB589AjccDecJiPRLGcOf98EZY+VfnaH3G92DWf7mNZn4UHHrqDt55UwaxMaeIxZ/sYdyAeC49s1/Hy4/q47wGtnAMUVMBpUedZFFy5IvPMX07Xl1nTIDwZWJoqq6gcb3V/cCdqlonLVR7qOoSYAk4VUmdFmF3IOLe8XrxF+NUoTjXSRIFWV8kjP3LYdPzX8x35jVw/i8goXsmy7svP4Pth4v50UsbGd43hszUTmpgb0lo5BdVVsaYdvFlYsgBBnoMDwByG80zGXjeTQrJwKUiUquqr/kwru5PBOLTndewOQ2nVZU6ySI00rlpqhsLDwlm8fVncdmfVnDrM+t57dsziI/s4mouY0yb+fJarbXACBEZIiJhwHygwW2+qjpEVTNUNQNYBnwr4JNCa8JjoP+Ebp8UTkqNi+Ch6yeRfbyc/3phA/W9pUdWY3oxnyUGVa0FvgO8C2wHXlTVrSJym4h0sA8C05NMyejDLy4/gw925PHAB7v9HY4xphU+7RVLVd8C3mo0bnEz897oy1iMf90wfTAbs4t44IPdnJkezwVnpPo7JGNMMwLntj/jVyLCb68ay9j0OH7wwgb25vfy50cb04NZYjBdJiLUaYwODQni1mfWU9rBx5IaY3zDEoPpUgMSo/jLgonsyS/ljpc24uu+ulSV59ccZOFT6/j137fx3OoDrN5bwLHSKp+v25ieyp68YbrcOcOTueuSUfzurR089MkevjXbN910F5RWcefLm3l/+1HSEyL5V9YxKmrqTk2PjwxleN8YhqVEu+/Oa2CfKIKDumk30sZ0AUsMxi9umTWUTTlF/PHdnaQnRPKV8f1p6SbHtvpoZx53vLSJ4ooafnbZaL45YwgAuUUV7MkvY09eKXvyS8nKK+XDHfm8uO6LzvvCgoMYkhxNYvQX91yIx/2aJ8P0DFcQQoKF80f15cqJ6cRG2P0apueybreN35RX1zLv4VVsPlTEWYMTuXPuKKYO6diTwSpr6vj9W9t5auUBRqbG8sCCCYxKa/3RmEXlNWTlO8liT34pe/JKKa5020D05NsX/yuqDSY5ZVTUkJVXSnRYMFdNSuf66YO9Wrcv1NUrG3MK+XRXPltzi7npnAzOGW7djAeybtG7qq9YYuhdaurqeWldDg98sIujxVXMGZnCHReP4oz+bd+hbjlUxPdf2EBWXik3zxzCHRePJCI02AdRN29jdiHPrDrA3zfmUlVbz9SMPlx/9mDmjkkjLMS3TXq5hRV8uiufT3fns2L3MYoraxGBuIhQKmvqeOLGKZYcApglBtPjVFTX8dTK/Tz08R6KKmq4YkJ//uvCTAYntd7Fcl298sjyvdz73k4So8K499rxzBrRvufidpYTZdUsW5/Ds6sPcKCgnOSYMOZPGcSCaYNIT2jiudrtUFFdx6p9BU4y2JXPnnyn48XUuHDOHZHCrMwUZg5PRlX52iOrOXC8jCdunMrZw5I6Zf2mZ7HEYHqsoooalny6h8dW7KO2Tpk/dSD/+aUR9I1r+uErhwor+OGLG1i19zhzx6Tx+6+eSWJ09+maub5eWZ51jGdWHuDDHUcBOH90KjdMH8zM4ckEtdLIXV+vlFbXUlReQ3FlDUUVNWzOKWL57mOs2X+c6tp6wkOCmDqkD+dlpnBuZgoj+sac1l5zrLSKBUtWkXOigidvmsK0oZYcAo0lBtPj5RVX8ucPs1i65iAhwcI3Zwzh1vOGNeiE742Nufz01c3U1yt3f2UM15w1oFMbsDtbzolylq45yPNrsikoqyYjKYpLzuxHTW09RRXOjr+4ovZUAiiuqKGkqpam/kVHpsYya0Qy52amMHVIH6+qzPJLqpi/ZCWHiyp56ptTmZLRsfYc07NYYjC9xoGCMu775y5e35BLfGQot88exlcnpfO7N7fz2oZcJg5K4P55E7yqcuouqmrreGfLEZ5ZeYB1B04QFRZMfGQocRGhxEWGEBcR6gxHhhIXEeK8e0wfmhxDWnz7Hl+ZV1zJ/CWrOFpcydM3T+OswYmdvHWmu7LEYHqdbbnF/PG9nXy4Iw8RCBLhu18aznfmDCckuOfep1lXr11+z8RRNznkl1TxzM1TmTjIkkMgsMRgeq01+47zwtpsrps+iEm2Q2u3I0WVzFuykuOl1Ty7cBrjByb4OyTjY5YYjDGtyi2sYP6SVRSWV/PcwumcOcDPj4Y1PtWWxNBzz8GNMR3SPyGSpYumExcZyvWPrWbLoSJ/h2S6CTtjMCbAZR8vZ/6SVZRV1/K3hdPbdXOht4ora9h5pIQdh4vJyislPjKUjORoMpKjGZIUTUJUaLe+sqwns6okY0ybHCwoZ/6SlVTU1LF00fQOd+VRW1fP/oIyth8uYceRYnYcLmHHkRIOFVacmicmPITy6lo8n/YaFxHCEDdRZCRFMyQ5msFJUQxJjiYhqvvcn9ITWWIwxrTZgYIy5j28ipq6epYumk5maix19Up1bT1VtXVU1jjvVbX1VNWcPi63sILth0vYebSYXUdLqa6tByAkSBiWEsOofrGMSotjVFoso/rFkhYXQU2dkn2inP3Hyth3rIz9BWXsP1bOvmNl5BZVNLiHIyEqlGEpMUzJ6MPZw5KYPDiR6HDrB9RblhiMMe2y71gZ85es5FhpNUECNXVt2z/0jQ1nVL84RqfFMjLNSQTD+kYTHtL2Pqsqa+rIPl7O/gI3cRSUseNwMZtyiqitV0KChPEDE5g+tA9nD03mrMGJRIZ1bd9YPYklBmNMux0oKGPpmmyCBMJDggkPDSI8JIiI0GDCQ4KccSFBhIc2HJccE0ZSTLjP4yuvrmXd/hOs3FvAyj0FbD5URF29EhosTByYyPShfZg+LIlJgxK7vBPF7swSgzEmYJRW1bJ2/3FW7Slg5d4Cthwqol4hLCSISYMSGJYSQ706T/OrV6Wu3uOzQr0qqkpdvbrzQf+ECDJTnbOezL6xxEf1/OdrtCUx+LSCTkTmAg8AwcCjqnpPo+nXAXe6g6XA7aq60ZcxGWN6l5jwEOaM7MuckX0BpyPGdfuPs9JNFO9sOUJQkBDk3jEfJEJQkMfnBuMFVWXV3oIGzyRPjQt3EkVqLJmpsWSmxTKib0yvbePw2RmDiAQDu4ALgRxgLbBAVbd5zHMOsF1VT4jIJcAvVXVaS+XaGYMxxtdUldyiSnYdLWHXkRJ2Hi1h19ESdh8tpcptVAcY2CeSzL6xjEiNJSMpikF9ohjYJ4p+8RHdrquW7nLGMBXIUtW9blDPA1cApxKDqn7mMf8qYIAP4zHGGK+ICOkJkaQnRJ46EwGnb6vs4+VOonATxu6jpXy6O79BQ31IkDAgMZKBfZxkcerlJg/PR7/W1tVTXFnLifJqCstrKKpw3k+U11BUXk1hhfO5sLyay87sx/ypg3y+/b5MDOlAtsdwDtDS2cDNwNtNTRCRRcAigEGDfP+lGGNMU4KD5NQNeRePSTs1vraunsNFlWQfL+egxyv7eDlvbj5MYXlNg3ISo0KJiQihsLyGksraxqs5RQTiI0NJiAwlISqM2vquaRP2ZWJo6vbFJrdKRObgJIaZTU1X1SXAEnCqkjorQGOM6QwhwUEMdKuRzmlielFFDdluojh4vJwDx8spr6olISqMhChnx58YHeYkgagwZzgqjNiIkFYf5uST7fFh2TnAQI/hAUBu45lEZBzwKHCJqhb4MB5jjPGL+MhQ4tPjGZveMzoq9GXryFpghIgMEZEwYD7whucMIjIIeAW4QVV3+TAWY4wxXvLZGYOq1orId4B3cS5XfVxVt4rIbe70xcAvgCTgQbfjrFpvW82NMcb4ht3gZowxAcCex2CMMabdLDEYY4xpwBKDMcaYBiwxGGOMacASgzHGmAZ63FVJIpIPHGjn4snAsU4Mp6cJ5O0P5G2HwN5+23bHYFVN8WahHpcYOkJE1gXyfRKBvP2BvO0Q2Ntv2972bbeqJGOMMQ1YYjDGGNNAoCWGJf4OwM8CefsDedshsLfftr2NAqqNwRhjTOsC7YzBGGNMKywxGGOMaSBgEoOIzBWRnSKSJSJ3+TueriQi+0Vks4hsEJFe3zWtiDwuInkissVjXB8R+aeI7HbfE/0Zo680s+2/FJFD7u+/QUQu9WeMviIiA0XkIxHZLiJbReR77vhA+e2b2/42//4B0cYgIsHALuBCnCfLrQUWqOo2vwbWRURkPzBZVQPiJh8RORcoBZ5W1bHuuP8DjqvqPe6BQaKq3unPOH2hmW3/JVCqqn/0Z2y+JiL9gH6q+rmIxALrgSuBGwmM37657b+WNv7+gXLGMBXIUtW9qloNPA9c4eeYjI+o6qfA8UajrwCecj8/hfMP0+s0s+0BQVUPq+rn7ucSYDuQTuD89s1tf5sFSmJIB7I9hnNo5xfWQynwnoisF5FF/g7GT1JV9TA4/0BAXz/H09W+IyKb3KqmXlmV4klEMoCJwGoC8LdvtP3Qxt8/UBKDNDGu99ehfWGGqk4CLgG+7VY3mMDxEDAMmAAcBu71bzi+JSIxwMvA91W12N/xdLUmtr/Nv3+gJIYcYKDH8AAg10+xdDlVzXXf84BXcarWAs1Rtw72ZF1snp/j6TKqelRV61S1HniEXvz7i0gozk7xOVV9xR0dML99U9vfnt8/UBLDWmCEiAwRkTBgPvCGn2PqEiIS7TZEISLRwEXAlpaX6pXeAL7hfv4G8LofY+lSJ3eKrqvopb+/iAjwGLBdVe/zmBQQv31z29+e3z8grkoCcC/Ruh8IBh5X1d/6OaQuISJDcc4SAEKAv/X2bReRpcBsnC6HjwJ3A68BLwKDgIPANara6xppm9n22TjVCArsB249Wefem4jITGA5sBmod0f/BKeePRB+++a2fwFt/P0DJjEYY4zxTqBUJRljjPGSJQZjjDENWGIwxhjTgCUGY4wxDVhiMMYY04AlBmO6kIjMFpF/+DsOY1piicEYY0wDlhiMaYKIXC8ia9z+6x8WkWARKRWRe0XkcxH5QERS3HkniMgqt5OyV092UiYiw0XkfRHZ6C4zzC0+RkSWicgOEXnOvWPVmG7DEoMxjYjIaGAeTueDE4A64DogGvjc7ZDwE5y7igGeBu5U1XE4d52eHP8c8FdVHQ+cg9OBGTi9Xn4fOAMYCszw+UYZ0wYh/g7AmG7ofOAsYK17MB+J0/FaPfCCO8+zwCsiEg8kqOon7vingJfc/qnSVfVVAFWtBHDLW6OqOe7wBiADWOH7zTLGO5YYjDmdAE+p6o8bjBT5eaP5WupPpqXqoSqPz3XY/6HpZqwqyZjTfQBcLSJ94dQzgwfj/L9c7c7zNWCFqhYBJ0Rkljv+BuATtx/8HBG50i0jXESiunQrjGknO1IxphFV3SYiP8N56l0QUAN8GygDxojIeqAIpx0CnK6cF7s7/r3ATe74G4CHReTXbhnXdOFmGNNu1ruqMV4SkVJVjfF3HMb4mlUlGWOMacDOGIwxxjRgZwzGGGMasMRgjDGmAUsMxhhjGrDEYIwxpgFLDMYYYxr4/waPX6k9qbGCAAAAAElFTkSuQmCC\n",
      "text/plain": [
       "<Figure size 432x288 with 1 Axes>"
      ]
     },
     "metadata": {
      "needs_background": "light"
     },
     "output_type": "display_data"
    }
   ],
   "source": [
    "from matplotlib import pyplot\n",
    "# plot train and validation loss\n",
    "pyplot.plot(history.history['loss'])\n",
    "pyplot.plot(history.history['val_loss'])\n",
    "pyplot.title('model train vs validation loss')\n",
    "pyplot.ylabel('loss')\n",
    "pyplot.xlabel('epoch')\n",
    "pyplot.legend(['train', 'validation'], loc='upper right')\n",
    "pyplot.show() "
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 6,
   "metadata": {},
   "outputs": [
    {
     "name": "stdout",
     "output_type": "stream",
     "text": [
      "INFO:tensorflow:Assets written to: saved_model/model3_50epochs_P20_bit/assets\n"
     ]
    },
    {
     "name": "stderr",
     "output_type": "stream",
     "text": [
      "INFO:tensorflow:Assets written to: saved_model/model3_50epochs_P20_bit/assets\n"
     ]
    }
   ],
   "source": [
    "import tensorflow as tf  \n",
    "# Save fine-tuned model as SavedModel\n",
    "export_module_dir = 'saved_model/model3_50epochs_P20_bit'\n",
    "tf.saved_model.save(model, export_module_dir)"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "### R101x3"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": null,
   "metadata": {},
   "outputs": [
    {
     "name": "stdout",
     "output_type": "stream",
     "text": [
      "WARNING:tensorflow:From /opt/conda/lib/python3.7/site-packages/tensorflow_core/python/ops/resource_variable_ops.py:1781: calling BaseResourceVariable.__init__ (from tensorflow.python.ops.resource_variable_ops) with constraint is deprecated and will be removed in a future version.\n",
      "Instructions for updating:\n",
      "If using Keras pass *_constraint arguments to layers.\n"
     ]
    },
    {
     "name": "stderr",
     "output_type": "stream",
     "text": [
      "WARNING:tensorflow:From /opt/conda/lib/python3.7/site-packages/tensorflow_core/python/ops/resource_variable_ops.py:1781: calling BaseResourceVariable.__init__ (from tensorflow.python.ops.resource_variable_ops) with constraint is deprecated and will be removed in a future version.\n",
      "Instructions for updating:\n",
      "If using Keras pass *_constraint arguments to layers.\n"
     ]
    },
    {
     "name": "stdout",
     "output_type": "stream",
     "text": [
      "Model sucessfuly built\n",
      "callbacks and data okay\n",
      "Train on 10 steps, validate on 4 steps\n",
      "WARNING:tensorflow:From /opt/conda/lib/python3.7/site-packages/keras/callbacks/tensorboard_v1.py:200: The name tf.summary.merge_all is deprecated. Please use tf.compat.v1.summary.merge_all instead.\n",
      "\n"
     ]
    },
    {
     "name": "stderr",
     "output_type": "stream",
     "text": [
      "WARNING:tensorflow:From /opt/conda/lib/python3.7/site-packages/keras/callbacks/tensorboard_v1.py:200: The name tf.summary.merge_all is deprecated. Please use tf.compat.v1.summary.merge_all instead.\n",
      "\n"
     ]
    },
    {
     "name": "stdout",
     "output_type": "stream",
     "text": [
      "WARNING:tensorflow:From /opt/conda/lib/python3.7/site-packages/keras/callbacks/tensorboard_v1.py:203: The name tf.summary.FileWriter is deprecated. Please use tf.compat.v1.summary.FileWriter instead.\n",
      "\n"
     ]
    },
    {
     "name": "stderr",
     "output_type": "stream",
     "text": [
      "WARNING:tensorflow:From /opt/conda/lib/python3.7/site-packages/keras/callbacks/tensorboard_v1.py:203: The name tf.summary.FileWriter is deprecated. Please use tf.compat.v1.summary.FileWriter instead.\n",
      "\n"
     ]
    },
    {
     "name": "stdout",
     "output_type": "stream",
     "text": [
      "WARNING:tensorflow:Issue encountered when serializing variables.\n",
      "Type is unsupported, or the types of the items don't match field type in CollectionDef. Note this is a warning and probably safe to ignore.\n",
      "'list' object has no attribute 'name'\n"
     ]
    },
    {
     "name": "stderr",
     "output_type": "stream",
     "text": [
      "WARNING:tensorflow:Issue encountered when serializing variables.\n",
      "Type is unsupported, or the types of the items don't match field type in CollectionDef. Note this is a warning and probably safe to ignore.\n",
      "'list' object has no attribute 'name'\n"
     ]
    },
    {
     "name": "stdout",
     "output_type": "stream",
     "text": [
      "Epoch 1/100\n",
      " 9/10 [==========================>...] - ETA: 2:02 - loss: 3.3426 - acc: 0.6252WARNING:tensorflow:From /opt/conda/lib/python3.7/site-packages/keras/callbacks/tensorboard_v1.py:343: The name tf.Summary is deprecated. Please use tf.compat.v1.Summary instead.\n",
      "\n"
     ]
    },
    {
     "name": "stderr",
     "output_type": "stream",
     "text": [
      "WARNING:tensorflow:From /opt/conda/lib/python3.7/site-packages/keras/callbacks/tensorboard_v1.py:343: The name tf.Summary is deprecated. Please use tf.compat.v1.Summary instead.\n",
      "\n"
     ]
    },
    {
     "name": "stdout",
     "output_type": "stream",
     "text": [
      "10/10 [==============================] - 1792s 179s/step - loss: 3.3597 - acc: 0.6387 - val_loss: 1.8493 - val_acc: 0.8225\n",
      "Epoch 2/100\n",
      "10/10 [==============================] - 1731s 173s/step - loss: 3.3999 - acc: 0.7879 - val_loss: 1.4968 - val_acc: 0.8251\n",
      "Epoch 3/100\n",
      "10/10 [==============================] - 3881s 388s/step - loss: 2.1117 - acc: 0.7973 - val_loss: 3.3243 - val_acc: 0.6554\n",
      "Epoch 4/100\n",
      "10/10 [==============================] - 3064s 306s/step - loss: 1.7363 - acc: 0.8072 - val_loss: 1.2877 - val_acc: 0.7919\n",
      "Epoch 5/100\n",
      "10/10 [==============================] - 1732s 173s/step - loss: 1.0600 - acc: 0.8318 - val_loss: 1.4337 - val_acc: 0.7347\n",
      "Epoch 6/100\n",
      "10/10 [==============================] - 1717s 172s/step - loss: 0.7142 - acc: 0.8510 - val_loss: 0.7169 - val_acc: 0.7952\n",
      "Epoch 7/100\n",
      "10/10 [==============================] - 1717s 172s/step - loss: 0.5207 - acc: 0.8611 - val_loss: 0.6007 - val_acc: 0.8030\n",
      "Epoch 8/100\n",
      "10/10 [==============================] - 1715s 171s/step - loss: 0.3520 - acc: 0.8891 - val_loss: 0.5290 - val_acc: 0.8004\n",
      "Epoch 9/100\n",
      "10/10 [==============================] - 1716s 172s/step - loss: 0.2413 - acc: 0.9129 - val_loss: 0.4984 - val_acc: 0.8004\n",
      "Epoch 10/100\n",
      "10/10 [==============================] - 1711s 171s/step - loss: 0.2721 - acc: 0.8980 - val_loss: 0.7529 - val_acc: 0.7575\n",
      "Epoch 11/100\n",
      "10/10 [==============================] - 2558s 256s/step - loss: 0.2657 - acc: 0.8990 - val_loss: 0.4995 - val_acc: 0.8004\n",
      "Epoch 12/100\n",
      "10/10 [==============================] - 5298s 530s/step - loss: 0.2050 - acc: 0.9229 - val_loss: 0.5895 - val_acc: 0.7822\n",
      "Epoch 13/100\n",
      "10/10 [==============================] - 1713s 171s/step - loss: 0.1939 - acc: 0.9180 - val_loss: 0.4724 - val_acc: 0.8082\n",
      "Epoch 14/100\n",
      "10/10 [==============================] - 1715s 172s/step - loss: 0.1510 - acc: 0.9400 - val_loss: 0.4708 - val_acc: 0.8010\n",
      "Epoch 15/100\n",
      "10/10 [==============================] - 1716s 172s/step - loss: 0.1496 - acc: 0.9428 - val_loss: 0.5916 - val_acc: 0.7731\n",
      "Epoch 16/100\n",
      "10/10 [==============================] - 1715s 172s/step - loss: 0.1472 - acc: 0.9396 - val_loss: 0.5182 - val_acc: 0.7848\n",
      "Epoch 17/100\n",
      "10/10 [==============================] - 1715s 172s/step - loss: 0.1555 - acc: 0.9320 - val_loss: 0.5089 - val_acc: 0.7945\n",
      "Epoch 18/100\n",
      "10/10 [==============================] - 1726s 173s/step - loss: 0.2908 - acc: 0.8842 - val_loss: 0.6054 - val_acc: 0.7965\n",
      "Epoch 19/100\n",
      "10/10 [==============================] - 1719s 172s/step - loss: 0.1483 - acc: 0.9387 - val_loss: 0.6309 - val_acc: 0.7666\n",
      "Epoch 20/100\n",
      "10/10 [==============================] - 1712s 171s/step - loss: 0.1376 - acc: 0.9451 - val_loss: 0.5123 - val_acc: 0.7958\n",
      "Epoch 21/100\n",
      "10/10 [==============================] - 1848s 185s/step - loss: 0.1874 - acc: 0.9156 - val_loss: 0.5443 - val_acc: 0.7913\n",
      "Epoch 22/100\n",
      "10/10 [==============================] - 1723s 172s/step - loss: 0.1327 - acc: 0.9453 - val_loss: 0.5179 - val_acc: 0.7997\n",
      "Epoch 23/100\n",
      "10/10 [==============================] - 1719s 172s/step - loss: 0.1122 - acc: 0.9547 - val_loss: 0.4945 - val_acc: 0.8101\n",
      "Epoch 24/100\n",
      "10/10 [==============================] - 1716s 172s/step - loss: 0.0932 - acc: 0.9678 - val_loss: 0.4966 - val_acc: 0.8069\n",
      "Epoch 25/100\n",
      "10/10 [==============================] - 1720s 172s/step - loss: 0.0934 - acc: 0.9703 - val_loss: 0.5067 - val_acc: 0.8049\n",
      "Epoch 26/100\n",
      "10/10 [==============================] - 1725s 172s/step - loss: 0.0795 - acc: 0.9777 - val_loss: 0.5086 - val_acc: 0.8043\n",
      "Epoch 27/100\n",
      "10/10 [==============================] - 1723s 172s/step - loss: 0.0851 - acc: 0.9803 - val_loss: 0.5084 - val_acc: 0.8030\n",
      "Epoch 28/100\n",
      "10/10 [==============================] - 1727s 173s/step - loss: 0.0827 - acc: 0.9797 - val_loss: 0.5039 - val_acc: 0.8056\n",
      "Epoch 29/100\n",
      "10/10 [==============================] - 1725s 173s/step - loss: 0.0801 - acc: 0.9803 - val_loss: 0.4991 - val_acc: 0.8043\n",
      "Epoch 30/100\n",
      "10/10 [==============================] - 1727s 173s/step - loss: 0.0769 - acc: 0.9820 - val_loss: 0.5032 - val_acc: 0.8075\n",
      "Epoch 31/100\n",
      "10/10 [==============================] - 1728s 173s/step - loss: 0.0795 - acc: 0.9836 - val_loss: 0.4981 - val_acc: 0.8023\n",
      "Epoch 32/100\n",
      "10/10 [==============================] - 1729s 173s/step - loss: 0.0787 - acc: 0.9824 - val_loss: 0.4975 - val_acc: 0.8017\n",
      "Epoch 33/100\n",
      " 5/10 [==============>...............] - ETA: 9:52 - loss: 0.0808 - acc: 0.9828 "
     ]
    }
   ],
   "source": [
    "%%time\n",
    "history,model = BiT_L_ResNet.train_Bit_L(train,validation,test,model_name=\"r101x3\",epochs=100,patience=20)"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": null,
   "metadata": {},
   "outputs": [],
   "source": [
    "from matplotlib import pyplot\n",
    "# plot train and validation loss\n",
    "pyplot.plot(history.history['loss'])\n",
    "pyplot.plot(history.history['val_loss'])\n",
    "pyplot.title('model train vs validation loss')\n",
    "pyplot.ylabel('loss')\n",
    "pyplot.xlabel('epoch')\n",
    "pyplot.legend(['train', 'validation'], loc='upper right')\n",
    "pyplot.show() "
   ]
  },
  {
   "cell_type": "code",
   "execution_count": null,
   "metadata": {},
   "outputs": [],
   "source": [
    "import tensorflow as tf \n",
    "# Save fine-tuned model as SavedModel\n",
    "export_module_dir = 'saved_model/model4_50epochs_P20_bit'\n",
    "tf.saved_model.save(model, export_module_dir)"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": null,
   "metadata": {},
   "outputs": [],
   "source": []
  }
 ],
 "metadata": {
  "kernelspec": {
   "display_name": "Python 3",
   "language": "python",
   "name": "python3"
  },
  "language_info": {
   "codemirror_mode": {
    "name": "ipython",
    "version": 3
   },
   "file_extension": ".py",
   "mimetype": "text/x-python",
   "name": "python",
   "nbconvert_exporter": "python",
   "pygments_lexer": "ipython3",
   "version": "3.6.10"
  }
 },
 "nbformat": 4,
 "nbformat_minor": 4
}