Navigaton.ipynb

{
 "cells": [
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "# Navigation\n",
    "\n",
    "---\n",
    "\n",
    "In this notebook, you will learn how to use the Unity ML-Agents environment for the navigation task. This task has been taken from Udacity-Deep-Reinforcement-Learning Nanodegree. The algorithm you will be implementing is **REINFORCE** to achieve the task.\n",
    "\n",
    "(https://www.udacity.com/course/deep-reinforcement-learning-nanodegree--nd893).\n",
    "\n",
    "### 1. Start the Environment\n",
    "\n",
    "We begin by importing some necessary packages.  If the code cell below returns an error, please check that you have installed [Unity ML-Agents](https://github.com/Unity-Technologies/ml-agents/blob/master/docs/Installation.md) and [NumPy](http://www.numpy.org/)."
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 1,
   "metadata": {},
   "outputs": [],
   "source": [
    "from unityagents import UnityEnvironment\n",
    "import numpy as np"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "Next, we will start the environment!  **_Before running the code cell below_**, change the `file_name` parameter to match the location of the Unity environment that you downloaded.\n",
    "\n",
    "- **Mac**: `\"path/to/Banana.app\"`\n",
    "- **Windows** (x86): `\"path/to/Banana_Windows_x86/Banana.exe\"`\n",
    "- **Windows** (x86_64): `\"path/to/Banana_Windows_x86_64/Banana.exe\"`\n",
    "- **Linux** (x86): `\"path/to/Banana_Linux/Banana.x86\"`\n",
    "- **Linux** (x86_64): `\"path/to/Banana_Linux/Banana.x86_64\"`\n",
    "- **Linux** (x86, headless): `\"path/to/Banana_Linux_NoVis/Banana.x86\"`\n",
    "- **Linux** (x86_64, headless): `\"path/to/Banana_Linux_NoVis/Banana.x86_64\"`\n",
    "\n",
    "For instance, if you are using a Mac, then you downloaded `Banana.app`.  If this file is in the same folder as the notebook, then the line below should appear as follows:\n",
    "```\n",
    "env = UnityEnvironment(file_name=\"Banana.app\")\n",
    "```"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 2,
   "metadata": {},
   "outputs": [
    {
     "name": "stderr",
     "output_type": "stream",
     "text": [
      "INFO:unityagents:\n",
      "'Academy' started successfully!\n",
      "Unity Academy name: Academy\n",
      "        Number of Brains: 1\n",
      "        Number of External Brains : 1\n",
      "        Lesson number : 0\n",
      "        Reset Parameters :\n",
      "\t\t\n",
      "Unity brain name: BananaBrain\n",
      "        Number of Visual Observations (per agent): 0\n",
      "        Vector Observation space type: continuous\n",
      "        Vector Observation space size (per agent): 37\n",
      "        Number of stacked Vector Observation: 1\n",
      "        Vector Action space type: discrete\n",
      "        Vector Action space size (per agent): 4\n",
      "        Vector Action descriptions: , , , \n"
     ]
    }
   ],
   "source": [
    "env = UnityEnvironment(file_name='Banana_Linux/Banana.x86_64',no_graphics=True) #put path to environment file"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "Environments contain **_brains_** which are responsible for deciding the actions of their associated agents. Here we check for the first brain available, and set it as the default brain we will be controlling from Python."
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 3,
   "metadata": {},
   "outputs": [],
   "source": [
    "# get the default brain\n",
    "brain_name = env.brain_names[0]\n",
    "brain = env.brains[brain_name]"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "### 2. Examine the State and Action Spaces\n",
    "\n",
    "The simulation contains a single agent that navigates a large environment.  At each time step, it has four actions at its disposal:\n",
    "- `0` - walk forward \n",
    "- `1` - walk backward\n",
    "- `2` - turn left\n",
    "- `3` - turn right\n",
    "\n",
    "The state space has `37` dimensions and contains the agent's velocity, along with ray-based perception of objects around agent's forward direction.  A reward of `+1` is provided for collecting a yellow banana, and a reward of `-1` is provided for collecting a blue banana. \n",
    "\n",
    "Run the code cell below to print some information about the environment."
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 4,
   "metadata": {},
   "outputs": [
    {
     "name": "stdout",
     "output_type": "stream",
     "text": [
      "Number of agents: 1\n",
      "Number of actions: 4\n",
      "States look like: [1.         0.         0.         0.         0.84408134 0.\n",
      " 0.         1.         0.         0.0748472  0.         1.\n",
      " 0.         0.         0.25755    1.         0.         0.\n",
      " 0.         0.74177343 0.         1.         0.         0.\n",
      " 0.25854847 0.         0.         1.         0.         0.09355672\n",
      " 0.         1.         0.         0.         0.31969345 0.\n",
      " 0.        ]\n",
      "States have length: 37\n"
     ]
    }
   ],
   "source": [
    "# reset the environment\n",
    "env_info = env.reset(train_mode=True)[brain_name]\n",
    "\n",
    "# number of agents in the environment\n",
    "print('Number of agents:', len(env_info.agents))\n",
    "\n",
    "# number of actions\n",
    "action_size = brain.vector_action_space_size\n",
    "print('Number of actions:', action_size)\n",
    "\n",
    "# examine the state space \n",
    "state = env_info.vector_observations[0]\n",
    "print('States look like:', state)\n",
    "state_size = len(state)\n",
    "print('States have length:', state_size)"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "### 3. Take Random Actions in the Environment\n",
    "\n",
    "In the next code cell, you will learn how to use the Python API to control the agent and receive feedback from the environment.\n",
    "\n",
    "Once this cell is executed, you will watch the agent's performance, if it selects an action (uniformly) at random with each time step.  A window should pop up that allows you to observe the agent, as it moves through the environment.  \n",
    "\n",
    "Of course, as part of the project, you'll have to change the code so that the agent is able to use its experience to gradually choose better actions when interacting with the environment!"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 5,
   "metadata": {},
   "outputs": [],
   "source": [
    "import numpy as np\n",
    "from collections import deque\n",
    "import matplotlib.pyplot as plt\n",
    "%matplotlib inline\n",
    "import torch\n",
    "import torch.nn as nn\n",
    "import torch.nn.functional as F\n",
    "import torch.optim as optim\n",
    "from torch.distributions import Categorical\n",
    "\n",
    "from agent import Agent\n",
    "\n",
    "device = torch.device(\"cuda:0\" if torch.cuda.is_available() else \"cpu\")\n",
    "agent = Agent(state_size, action_size, 42)"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 6,
   "metadata": {},
   "outputs": [],
   "source": [
    "def update_policy(agent, rewards, log_probs, gamma):\n",
    "    discounted_rewards = []\n",
    "    for t in range(len(rewards)):\n",
    "        Gt = 0 \n",
    "        pw = 0\n",
    "        for r in rewards[t:]:\n",
    "            Gt = Gt + gamma**pw * r\n",
    "            pw = pw + 1\n",
    "        discounted_rewards.append(Gt)\n",
    "\n",
    "    discounted_rewards = torch.tensor(discounted_rewards)\n",
    "    discounted_rewards = (discounted_rewards - discounted_rewards.mean()) / (discounted_rewards.std() + 1e-9) # normalize discounted rewards\n",
    "\n",
    "    experiences = (log_probs, discounted_rewards)\n",
    "    agent.learn(experiences)\n",
    "    \n",
    "def play_episode(agent, env, T_max):\n",
    "    env_info = env.reset(train_mode=True)[brain_name]\n",
    "    state = env_info.vector_observations[0]\n",
    "    rewards = []\n",
    "    log_probs = []\n",
    "\n",
    "    for t in range(T_max):\n",
    "        action, log_prob = agent.act(state)\n",
    "        log_probs.append(log_prob)\n",
    "        env_info = env.step(action)[brain_name]\n",
    "        next_state = env_info.vector_observations[0]\n",
    "        reward = env_info.rewards[0] \n",
    "        rewards.append(reward)\n",
    "        done = env_info.local_done[0]\n",
    "        if done:\n",
    "            break\n",
    "        state = next_state\n",
    "    \n",
    "    return rewards, log_probs\n",
    "        "
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 7,
   "metadata": {},
   "outputs": [],
   "source": [
    "def REINFORCE(num_episodes=5000, T_max=1000, gamma=0.92):\n",
    "\n",
    "    scores = []\n",
    "    last100_scores = deque(maxlen=100)  \n",
    "    \n",
    "    for i in range(1, num_episodes+1):\n",
    "#         rewards, log_probs = play_episode(agent, env, T_max)\n",
    "        env_info = env.reset(train_mode=True)[brain_name]\n",
    "        state = env_info.vector_observations[0]\n",
    "        rewards = []\n",
    "        log_probs = []\n",
    "        \n",
    "        for t in range(T_max):\n",
    "            action, log_prob = agent.act(state)\n",
    "            log_probs.append(log_prob)\n",
    "            env_info = env.step(action)[brain_name]\n",
    "            next_state = env_info.vector_observations[0]\n",
    "            reward = env_info.rewards[0] \n",
    "            rewards.append(reward)\n",
    "            done = env_info.local_done[0]\n",
    "            if done:\n",
    "                break\n",
    "            state = next_state\n",
    "        \n",
    "        S = sum(rewards)\n",
    "        last100_scores.append(S)\n",
    "        scores.append(S)\n",
    "\n",
    "        update_policy(agent, rewards, log_probs, gamma)\n",
    "        \n",
    "        if i < 100:\n",
    "            continue\n",
    "        \n",
    "        if i % 50 == 0:\n",
    "            print('Episode {}: Average Score (last 100 episodes): {:.3f}'.format(i, np.mean(last100_scores)))\n",
    "        if np.mean(last100_scores)>=13.0:\n",
    "            print('Environment solved in {:d} episodes\\nAverage Score (last 100 episodes): {:.3f}'.format(i, np.mean(last100_scores)))\n",
    "            break\n",
    "        \n",
    "    return scores"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 8,
   "metadata": {},
   "outputs": [
    {
     "name": "stdout",
     "output_type": "stream",
     "text": [
      "Episode 100: Average Score (last 100 episodes): 0.220\n",
      "Episode 150: Average Score (last 100 episodes): 0.240\n",
      "Episode 200: Average Score (last 100 episodes): 0.930\n",
      "Episode 250: Average Score (last 100 episodes): 2.270\n",
      "Episode 300: Average Score (last 100 episodes): 4.730\n",
      "Episode 350: Average Score (last 100 episodes): 8.110\n",
      "Episode 400: Average Score (last 100 episodes): 10.750\n",
      "Episode 450: Average Score (last 100 episodes): 11.770\n",
      "Episode 500: Average Score (last 100 episodes): 11.940\n",
      "Episode 550: Average Score (last 100 episodes): 11.810\n",
      "Episode 600: Average Score (last 100 episodes): 11.390\n",
      "Episode 650: Average Score (last 100 episodes): 11.770\n",
      "Episode 700: Average Score (last 100 episodes): 11.950\n",
      "Episode 750: Average Score (last 100 episodes): 12.120\n",
      "Episode 800: Average Score (last 100 episodes): 12.890\n",
      "Episode 850: Average Score (last 100 episodes): 12.500\n",
      "Episode 900: Average Score (last 100 episodes): 12.100\n",
      "Episode 950: Average Score (last 100 episodes): 11.990\n",
      "Episode 1000: Average Score (last 100 episodes): 11.700\n",
      "Episode 1050: Average Score (last 100 episodes): 11.470\n",
      "Episode 1100: Average Score (last 100 episodes): 11.880\n",
      "Episode 1150: Average Score (last 100 episodes): 12.640\n",
      "Episode 1200: Average Score (last 100 episodes): 10.910\n",
      "Episode 1250: Average Score (last 100 episodes): 10.560\n",
      "Episode 1300: Average Score (last 100 episodes): 11.500\n",
      "Episode 1350: Average Score (last 100 episodes): 11.840\n",
      "Episode 1400: Average Score (last 100 episodes): 11.570\n",
      "Episode 1450: Average Score (last 100 episodes): 10.430\n",
      "Episode 1500: Average Score (last 100 episodes): 12.120\n",
      "Environment solved in 1523 episodes\n",
      "Average Score (last 100 episodes): 13.100\n"
     ]
    }
   ],
   "source": [
    "scores = REINFORCE()"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 23,
   "metadata": {},
   "outputs": [
    {
     "data": {
      "image/png": "iVBORw0KGgoAAAANSUhEUgAAAX4AAAEGCAYAAABiq/5QAAAABHNCSVQICAgIfAhkiAAAAAlwSFlzAAALEgAACxIB0t1+/AAAADh0RVh0U29mdHdhcmUAbWF0cGxvdGxpYiB2ZXJzaW9uMy4xLjEsIGh0dHA6Ly9tYXRwbG90bGliLm9yZy8QZhcZAAAgAElEQVR4nO3dd5wU9f3H8deHXlSKHAqCgojYBT0F1Ni7KJoYSzRGoyGWRI0tlkSNKT+jxkSNmhA11lhijy2xd8Gj96Kg0g8pCirtPr8/vrPe3nHH7d3t7Nzuvp+Pxz5mdnZ25nMD+9nvfudbzN0REZHi0SzpAEREJLeU+EVEiowSv4hIkVHiFxEpMkr8IiJFpkXSAWSiS5cu3qtXr6TDEBHJK6NGjVrs7iXVt+dF4u/VqxdlZWVJhyEiklfM7JOatquqR0SkyCjxi4gUGSV+EZEio8QvIlJklPhFRIpMbInfzO4xs0VmNrHa9p+b2VQzm2RmN8R1fhERqVmcJf57gcPTN5jZAcBQYFd33xG4Kcbzi4hIDWJL/O7+FrCk2uZzgOvdfVW0z6K4zi8ikteWz4HXfg+ff5T1Q+e6jn9b4DtmNsLM3jSzPWrb0cyGmVmZmZWVl5fnMEQRkSbgi3nw1g2wZFbWD53rxN8C6AwMAi4FHjMzq2lHdx/u7qXuXlpSsl6PYxGRwpaaJKvGDNk4uU78c4AnPRgJVABdchyDiEgeyX7mz3Xifxo4AMDMtgVaAYtzHIOISB6Ib1rc2AZpM7OHgf2BLmY2B7gGuAe4J2riuRr4kWvSXxGR9X1b1ZP9En9sid/dT67lpVPjOqeISOFIlYnzv6pHREQyEWOJX4lfRKRJUolfRKQ4qcQvIlIkYmz3osQvItIkqapHRKS46OauiEixUYlfRKQ4qcQvIlIkdHNXRKTYqKpHRKS46OauiEixUYlfRKS4fJv3lfhFRIqMEr+ISJFQqx4RkeKSjzd3zeweM1sUzbZV/bWLzczNTPPtiojUKD9v7t4LHF59o5n1BA4FPo3x3CIi+e3bEn/2Dx1b4nf3t4AlNbz0Z+Ay4qzAEhHJe/lZ4l+PmQ0F5rr7uAz2HWZmZWZWVl5enoPoRESaoHyq46/OzNoBVwJXZ7K/uw9391J3Ly0pKYk3OBGRpqZAxurpA/QGxpnZbKAHMNrMNs9hDCIieSK+qp4WWT9iLdx9AtA19TxK/qXuvjhXMYiI5I08bc75MPA+0M/M5pjZmXGdS0Sk8ORhid/dT67j9V5xnVtEpGDkU4lfREQaIcabuzmr4xcRKQrv3goVa6HbrjDxCRj7EOw5DA7/IzSrT1k7D6t6RESKzqov4eVfr7995HDosi3s+ZPMj5WPN3dFRIrOsmoj0RxxI/Q5MKy/cAm8/ad6HKxAeu6KiBS0kcPD8rRn4ReTYOAw+OFTMPhnYfur18HUFzI7lkr8IiJN3OqvYNS9YX3znaFDj8rX9jircv2Rk2Hys/U4sBK/SLxWrYC1q5KOQvLNoqlw83Zh/aCroV3nqq937g1XLYQtdg/PH/shfDG/joMWxpANIk3XikVwbQf4vy3glv5QsS7piCSf3H8MfLM8rO94XM37tGwDP3kNBvwwPL99IDzwXZjxSs37q6pHJAYV6+CuQ+Cve8BNfSu3fzkPpr2YXFySX5Z+AisWwrZHwFULoPPWG95/h6FhuWo5fPQqPPQ9eOEymPV2LW9Q4hfJnqnPw5yRsHh65bbTnoUWbUPb69VfhW1fLoCvappaogisXQXffJF0FE3bmAfCcv9fQsu2de/f9xDY86dVt438O9w3pOq2GEv8ascvxamiItSzQmhy980y6LgV9N43/FQf9y944iw47k74U7+q771wAnTcMvcx58qCifD+7XD0X+C+o+GzEWF76Y9hyJ+Tja2pWfMNvHVjWN9s58zfd+QNsMMx8L9fwbwxldvdQ6JfPheeTN0QVuIXyY4lH1WuDxxW9bVjbgu/AqY9D9fXkODvGAwHXwu9vgNdoxt65dPDB7ZL3/X3zzfPnAvzx4W/J5X0Acrugb0vhE5bJRdbU7NgfFhuvgs0r2c67bUPdN2xauL/Zhm07QRzPqzcpjp+kSx59Tdhefw/13+teQsYenvVbd36w4G/CuurV4TOOHcMhLmj4OM34PY94K+l8PXSWMOO3eqV4UsMQnUXwHHDoc9BYX3e6GTiaqo+uCMsT/pXw95/+P/BkTdVPl/5eVi227RxcdVBiV+Kz8LJMOU/0HNQ7S0wum4Hpz8P+/0SLv8Ufvom7Htp+IC371q53z8OhPuHVj6/sS/8vhu8e0u8f0Nc5o+DtV/D/leEm5SDzoNdT4STH4ZmLauWTgWWfAwdtoSOPRv2/jabwK5pAxl/FU1PUrG28bFtgBK/FI+vl4Ymm3cODs+PvHHDP6N77QMHXAltOlRu2+4ouHQGXLMMDvs/2Gizqu+pWANrvoKXr4Y/bQ9rV4d62/dug3GPZv9vyrbpL4XlDkPh/DFw+B/C8xatYbMdYcITycXW1Kz5OnxR9t/gCPR1a70RHHJdWF9ZQ+JXVY9IA816G/7Yq+q2brs0/HhmMPhcuGQ6XDYr/CrYo9oAXF/OC1UjH78ebuI9NQyWzm74OeNUsS60Q0/9UulYQz1+yXbwxZww4qSEJpyQnRv9O30vLF+4JHQiXPVl2ot5lPjN7B4zW2RmE9O23WhmU81svJk9ZWYd4zq/FIgpz4WbqS9entn45LPegun/q7rvfUdXNpXr3Ae23h+umJO9GNt1Dr8KUiMv7nsZXBDd9LvnMHggrTppbAPrguP28tWVN7KPGw6t2q2/T6pUOv2/uYurKUvdC9mke+OP1a5LWH45H0bfD4+fUflanpX47wUOr7btZWAnd98FmA5cEeP5pRA8+RNYNBlG3Fk1aa76EiY8HqpSvt22IiT5f30fftMRRvwdpr0UvgwgJOTzR8Npz0DrjbMfa0k/uHY5HHhVaPmy1T6Vrw34Yagzn/R09s+bDe//tXJ9+yE177PxZtDvKJhTlpuYGmL2O3DPEaEaJm4THoM2HWGrvRt/rJZtKte9ovHHq0Nsid/d3wKWVNv2P3dPVV59APRY742SjLEPh/rvN/4I4x6BJbOSjggeOy3Ul6csnFi5/sHf4Ikz4ZVrK7e9eX3V9794GTx8IrTaGPa/Eva7LNZw13PY7yrXD7ku3AhcPA0+fhP+vFO43v84EJ79eaiGKp+W2/hS5qa11NluCLRqX/u+PfcITWFTrU+akop1cO9R8Ol78NnI0FcjTrPfhb6Hhvsf2XBe1IRz0lPVXiisdvw/Bmq922Vmw4BhAFtuWcCdZZK0bg2sWx16Zj59dtj2xh9q3rdkOxj2RtWeiZ+NhLsPgXPeCzf+smntapj8TFi/ZCbctE1oOrfjceGG6utRUv3gdjjkN+EXwHu3hW3D3gijH75zc3h+6G+h9IzqZ4hf9wHhJnDFutBE9NDfw/+uCuO6pMwdFR4Ab98M3/17bmJbuxp+VwIt2sDab8K2Y++EXU7c8Pt6DgzLOSOh3xHxxlhfSz6uXL//mHCD+oT74znXN8vDPZxUP45sSPUBmVvtF1Wh9Nw1s6uAtcBDte3j7sOB4QClpaXxDVNXbNzhrZtg063h8R9n/r7yqXD7nnDuB6FEOPWFMLwswJ17hY49+14aWihkYtWXMH98+EJpX0Ob5XeiHqKnPA4blUDpmVB2d/iiqX4z7Q/dwxcYhI5V3QeER5e+oVS260mZ/53ZZlbZsaf/D0Lir0nXHWDaC2GYiJrq17NhwYRwXRfPqOx4lEr6qfjq0n0ANGsRvvTjSPxLPob2JfWripv9TrhHkfoCTZn8DHz+Uahiq1gXmqlmq4pv6vNh2TWLBZ5aE3wBJH4zOx0YAhzkHuNswhKsXR0+ED33DHXls96uLC2na9YSrl4cxqRp1gKmPgdfzAtfFAsnwqw3w+xCf+gOh/y2cnq5Fm3DB+rdv4SmaMfevv6xq3v31sr3N28NV84LyXHlYpj8dDh/6pdHquPQkTeGxA+Vsxyd+QrcfXBl0gfY5xeV6/1/kFkyy5V2nWHg2TBvbGgmuWpFqN7ZtE/40D9wHPyhW2hRM+TPsM1B2Tv3ujXwt31qfm3r/WG30zI7Tsu2oanhOzfDdy7O/Is+EysWwa0DwvpZr0GP3TN7371HVX1+zG1hXPy5o+C23aq+duX8xn2xTnwCXvwlrCwPz7tu3/BjZSrfS/xmdjhwGbCfu39V1/6SBW/fBG/+sebXNu4Wqk2WzoKLpoZtqXHEqyfMNV/DrbuFn7eppL3Xz+HQ38En78M/D4exD4aElt7uvbrUL46Udavgt5uG+UjTB0sDOOiaysmpmzUPXxAvXx1+LexzUfiZPfQOeOkK2PMs2Ov8zK5Jko6o9m+x9X5hmT4M9LJP4MHvhp7C/U+F5q3Cr6IP74YXLoXj/ga7nFD5PmtWc3IY82CYEeqsV6sOAbDlYNj9dNj5++G61teWg+HT90NBouee9X9/bSY+Wbl+14GhH0FdI10CtN4EVn0RvogGnhN+Ie5wLFxfQ6eq+WNhq70aFt/aVev/Su7QwI5btdn9DBhVvTd59hO/xVXoNrOHgf2BLsBC4BpCK57WQOrO0AfufnZdxyotLfWysibckqAp+9P2IVmn++HToZS5SY/KxJqpO/eBhRNC1c4hv6ncPuOVMLwshFYzW+9f8/unPAePnhKqhXY6PpTsU3X56focCKc8Uf/48ln5tHCfYsdj4cHvbXjfs14LN4qfPie0HtrnwlAFt+TjUB3TqVf4dVbdOe+FaqXGlCKXzIJb+4cb5vv/suHHqe7eITC72tDEe50f7tGkzHwl3NBfOBkunQkLJ4WmuodfH35Npf9dz5wXvvzSDTwHjqjWCCATFRVwXaeq2466GfY4s/7H2pCP36jaExzgF5OhwxYNOpyZjXL30urbYyvxu3tN3dnujut8UoPUDaidT4Atdgv16b33bVgpL+XoW2DM/SFxp+t7cGja9s2y8B/33BGhZP7l/NBSpFmzUN3w3IWhFLff5aF6J3XzbcTwUBrb+wIY+Y8wZEAxJX0IzUGHRs0qm7cOv4Zqc9eBleufvBMe6bY/hhpt2rfxVQede4f7LIuz2App1YqQ9LtsCwf+OrTowuG9W8O5Un0kHv5B5XW5oXe42dxu03BTuvrfNfT2MMZSq/bhF+y1HUKz4IOurl91z9dLqzYlHvYmdO/fqD+3Vqn2/OnyvapHcmxRVH3T7wjY6bvZOWaP3Wuve/3l7FDX/8q1YQCzlM13hu/dHUqKK8vDjdrqIxmmj5B51E0UvZ++CXcMCl/UqX4IAN/9R/iFNPW58Dx1j6W6KdGcrpfMgE/eha+XwYBToXnL7MS36Tbhxmm2pHo07/fLMFzxvpdUDnf8wiUh9lH3rf9l+NmIUGVVfarDlD3TelPvdX74IimfGgpCmUrv8X3qE/ElfYinf0kNiqxIVUQqKuCtG8L61vvn5pxm4eZq9ZYOCyaEFkEPnxhu3A78ac3vl0pdt4eLp8GP/gOXfgS/XhyGhtjlBDgxrfrirFcqS4n7XQ4/K4ODoyq4Q66DjbqGJrClZ2Qv6QN06ReqptZlaTCxpVG/kVSd/gFXwa8WwY5RgeXV38KMqMfwRVNCQSIl09ZFqSkP36nHnALpVeEdt4StD8j8vQ1RU+KPYRpQJf5CNfutUB/adcfaS0NxOfud8MG8/LPQkzV9iOM9zsp9PPlq483Dsn2XkLRT180s3NCE8AVxyHVhEu9dTwpNWAeeDac+CYN/Fl9sW+wefmksmpyd46U6DHbuHZZmoWPUcX8Lz0ffB5+OCCNZbtIddj4+9JG4eFrlODd1SR17yrNhcLVMrFgUlvteCuePbVw1aSZS/65QeeM4hp68SvyFavIzYM3hzP/l/tzNmoUPZpvoP/GAU+FX5WHs+4OvzX08hejsd+CMl0IiGnBKmMQ7ldhatglNQeNMUqnqvuqdjRpq6exwj6httRuoLVqHwsvqFbBmZWitk2JW+eWYieYtw68igL/vm9l7Uq2heg6KP+lDZUe/4/8ZWmsBuEr8kom1q0O37x2Py24768Zo0SrcZ8hkTlKpW6etYKvBCZ6/N7TtvH6nKYA3bwzt6Otj6azQEqkm+14SltYMtjm4fsetLr2fx9JPNrzv2tWhBRrkdtaxvX4WPiupL5oYWl4q8ReainXwxI9DS4Qdj617f5GGMAutxJbMDs+/WhJ6BFesCx0E/3NBqGrM1OKZlb9Yqut7aJjv9/yx9Z/esLqWbUJzZoCJj29432fOq1zPpD9Btn1b4ldVj9Rl3CNhdikI84CKxGXjzUIz0rWrQrv+v5bCb9OaIz74PXi9lrGf0n3+ESz/tOY5ACD8ah3y5+yVuvscEJp5vnpdGKeqNnNGhuUF43NTzVNdatTPGFr6KPEXmlQpptuumhRb4rVl1AP2d11DnxEIpdNWG4cOZhB6jaeaas4pg/f+un7VxYyXw7LvobGH/K3UYHS13Zxe9ll47HJicp+jI28K/WHqcx8jQ0r8heLzj0IHlY9eC+Pb/PStut8j0hh7/qTyZmlKh55wxWfh5u8p0Uxd9w+F0Q/AXQeFQeqev6jqe2a+HCbIaehQCg2x8/HQsn2YGa2mOvTZb4ebqoPPW/+1XGnRKrujf6YfOpajSu49f3Hl+hE3JBeHFA8zOOAK6LV3GAZi9cpw8z7V07RvdCN26Wx4Nq1padk9YYiFFq1D0p03BrY9IpYeqrXaqGsYcuSFS0InsC0HVX39s5Hhi2GznXIXUw6pxF8IFk0Jg2Y1bw2nvwBdtkk6IikmvfcNfQ06bRUSarpjbqtc73tYGIMJQv06hPb0X32+fuLNhVQLoeo9kNeuhgn/hu2OSqZuPwdU4i8EU54L46pfPD3ccBNpKnY7LdxIfefPYaKXr5eE2cfK7gmjuL7++7BfEpO6pObK/c/5oXf7mq9DoWnRpNBvoKlNNJNFSvyFYPpLoZOLkr40Rd12ge9HQw237B56wb55fWXSh/CLIddatA4jxE58HP68Q9jW58DKXwDdB+Q+phxRVU++WzQ19J4ccGrSkYhkZvfTqz7f56Iad8uJQedWff7Ra2E+BKi9Q1kBUIk/3z0T/cfd8bhk4xDJ1Cbd4Iq58OkHlTeAk9Kx2kQqPfYITVJ3PyO3N5tzTIk/382P5k7dpFuycYjUR+uNkk/6EOb3bdMhdOT69eLG9wzOE7H9lWZ2D2Fu3UXuvlO0rTPwKNALmA2c4O5L44qh4K3+CirWhCFsRaT+zMK0o98sK5qkD/HW8d8LHF5t2+XAq+7eF3g1ei4NVX0McxGpv1btKlv4FInYEr+7vwUsqbZ5KHBftH4foFHEGmPxjLDcVO32RSRzuW7Vs5m7z4/WFwC1tj80s2FmVmZmZeXl5bmJLt9MejKMX17SL+lIRCSP1Jn4zaydmf3azP4RPe9rZkMae2J3d6DWgabdfbi7l7p7aUlJSWNPV3jcYfY7oXehxrgXkXrIpMT/T2AVkJr1YS7wuwaeb6GZdQOIlosaeBxZPid0de8W48TPIlKQMkn8fdz9BmANgLt/BTS0geuzwI+i9R8BzzTwODL6/jBRw9b7Jx2JiOSZTBL/ajNrS1QtY2Z9CL8ANsjMHgbeB/qZ2RwzOxO4HjjEzGYAB0fPpSE+fT+Mud+lb9KRiEieyaTh6jXAS0BPM3sI2Bs4va43ufvJtbx0UMbRSc3WrgpDye5xVtKRiEge2mDiNzMDpgLfBQYRqngucPfFOYhNarNwEqxbDT0HJh2JiOShDSZ+d3cze8Hddwaez1FMUpfp/w3LLXZLNg4RyUuZ1PGPNrM9Yo9EMuMebuz2PRQ6bpl0NCKShzJJ/AOB983sIzMbb2YTzGx83IFJLRZNhi/nwfZHJx2JiOSpTG7uHhZ7FJK5ma+E5TZNYGRDEclLdZb43f0ToCNwdPToGG2TJMx4Ocy2VWSDSolI9mQyZMMFwENA1+jxoJn9PO7ApAbr1oTJK/ockHQkIpLHMqnqORMY6O4rAczsj4SOWbfFGZjUYMmsMP7+5jsnHYmI5LFMbu4asC7t+ToaPmSDNMbi6WGp3roi0giZlPj/CYwws6ei58cCd8cXktQqlfg3VeIXkYarM/G7+81m9gawT7TpDHcfE2tUUrN5o6FDT2izSdKRiEgeqzPxm9kgYJK7j46eb2JmA919ROzRSVVzx8BWeyUdhYjkuUzq+O8EVqQ9XxFtk1xa9SV8MUezbYlIo2V0czeaLQsAd68gs3sDkk3jHw3Lku2SjUNE8l4mif9jMzvfzFpGjwuAj+MOTNIsnwPPXwLNW2lEThFptEwS/9nAXoQpF+cSxu4ZFmdQUs3CSYDDyY/ARpp/WEQaJ5NWPYuAk7J5UjP7BXAWYVavCYSWQt9k8xwF5fOZYbn5LsnGISIFodYSv5n9xMz6RutmZveY2fJohM4GDwRvZlsA5wOl7r4T0Jwsf7EUnHljYePuKu2LSFZsqKrnAmB2tH4ysCuwNXARcEsjz9sCaGtmLYB2wLxGHq+wfTYCuvdPOgoRKRAbSvxr3X1NtD4EuN/dP3f3V4D2DT2hu88FbgI+BeYDy939f9X3M7NhZlZmZmXl5eUNPV3+mz8eln0CPTQXjohkx4YSf4WZdTOzNoQJ0l9Je61tQ09oZp2AoUBvoDvQ3sxOrb6fuw9391J3Ly0pKeIqjnlRJ+kdhiYbh4gUjA0l/quBMkJ1z7PuPgnAzPajcc05DwZmuXt59IviSUKrIanJvNHQsj106p10JCJSIGpt1ePuz5nZVsDG7r407aUy4MRGnPNTYJCZtQO+JvyaKGvE8QpXRQVMegr6HQHNMml5KyJStw0253T3tcDSattWNuaE7j7CzB4HRgNrgTHA8MYcs2At+wS+WQ699006EhEpIIkMveDu1wDXJHHuvPLZyLBUix4RySLVHzRlcz6EVhvBZjslHYmIFJBMhmWuqbPWcuCTqCpI4jJ3FHQfAM2aJx2JiBSQTKp67gB2A8YTplzcCZgEdDCzc2pqgy9ZsHYVLJgAg89NOhIRKTCZVPXMAwZEbep3BwYQmnMeAtwQZ3BFbcHEMLH6FrsnHYmIFJhMEv+2qTb8AO4+GdjO3TU0c5zmjgrLLUqTjUNECk4mVT2TzOxO4JHo+YnAZDNrDayp/W3SKHNHwUabwybdk45ERApMJiX+04GZwIXR4+No2xrggLgCK3pzR4VqHrOkIxGRApNJif8I4K/u/qcaXltRwzZprK+XwuczYFeNVi0i2ZdJif9oYLqZPWBmQ6KhlCVOqYHZdGNXRGJQZ+J39zOAbYB/E8bl/8jM7oo7sKI2f1xYqseuiMQgo9K7u68xsxcJUyW2BY4lTJ0ocVg4GTbpAW07JR2JiBSgOkv8ZnaEmd0LzAC+B9wFbB5zXMVt4STYbMekoxCRApVJif804FHgp+6+KuZ4ZO1qWDwNtj006UhEpEDVmfjd/eT052a2D3Cyu58XW1TF7PMZULFWA7OJSGwyquM3swHAD4DvA7MIs2ZJHBZMDMuuOyQbh4gUrFoTv5ltS2jFczKwmFDdY+6uTltxmjcGWrSFLtsmHYmIFKgN3dydChwIDHH3fdz9NmBdNk5qZh3N7HEzm2pmU8xscDaOm/fcYcSd0HV7aK7uEiISjw0l/u8C84HXzewfZnYQYVjmbLgFeMndtwN2BaZk6bj5be7osFQzThGJUa2J392fdveTgO2A1wnj9HQ1szvNrMFNTsysA7AvcHd0ntXuvqyhxyso86MeuwdrVkoRiU8mPXdXuvu/3P1ooAdhcvRfNuKcvYFy4J9mNsbM7jKz9tV3MrNhZlZmZmXl5eWNOF0eWTAR2nSAzXdJOhIRKWD1mnPX3Ze6+3B3P6gR52xBmNHrTncfAKwELq/hXMOjyV9KS0pKGnG6PLJgAmy2s0bkFJFYJTHZ+hxgjruPiJ4/TvgiKG4V62DRZNh856QjEZECl/PE7+4LgM/MrF+06SBgcq7jaHKWzII1X8Hm6rglIvFKqs3gz4GHzKwVYWKXMxKKo+mYF7XoUY9dEYlZIonf3ccCmkw23ch/QIctlfhFJHZJ1PFLdQsmwJyRUHqGOm6JSOyU+JuC5y8Oyz4aDUNE4qfEnzR3+HI+tNoIug9IOhoRKQJK/ElbPgeWfQr7X5F0JCJSJJT4k/bJu2G51V7JxiEiRUOJP2nv3gqbbKFhGkQkZ5T4k/TVElg0CfY4S615RCRnlPiTVD4tLNV2X0RySIk/SeVTw7Kk34b3ExHJIiX+JC2aDC3bQ4eeSUciIkVEiT9Jn34APUqhmf4ZRCR3lHGSsm5tqOPvptY8IpJbSvxJWTwN1q2CrjskHYmIFBkl/qQ8ckpYlmyXbBwiUnSU+JOydHZYasYtEckx9RpKwrq10Kw57HU+NG+ZdDQiUmQSK/GbWXMzG2NmzyUVQ2KWfQIVa2HTbZKORESKUJJVPRcAUxI8f3IWzwjLLn2TjUNEilIiid/MegBHAXclcf7ELZoUlkr8IpKApEr8fwEuAypq28HMhplZmZmVlZeX5y6yXJg7Gjr3gbadko5ERIpQzhO/mQ0BFrn7qA3t5+7D3b3U3UtLSkpyFF2OzB0FW+yedBQiUqSSKPHvDRxjZrOBR4ADzezBBOJIxhfzwlSLSvwikpCcJ353v8Lde7h7L+Ak4DV3PzXXcSRmzodhqcQvIglRB65cG/8YtOsC3XZNOhIRKVKJJn53f8PdhyQZQ059uRCmvQgDToEWrZKORkSKlEr8uTTlWfB1sOsPko5ERIqYEn+uuMPYf0GXbaGrBmYTkeQo8efKoskwbzSU/jjpSESkyCnx58qU/wAGO30v6UhEpMgp8efCurUw8QnYcjBs1DXpaESkyCnx58KEx2DxdBhQPN0VRKTpUuLPhTll0HoT6K/WPCKSPCX+XJg/Nsy0ZZZ0JCIiSvyxWzwzDMq25aCkIxERAZT44zfzlbDsf0qycYiIRJT447RyMUx8HDpuBZv2SToaERFAk63HZ/VKuDFK9jscm2wsIiJpVOKPy/hHK9f3vzy5OEREqlGJPy6z3oaNu8NFk9WaR/RYA9MAAAusSURBVESaFJX447B0dhiiYZsDlfRFpMlR4o/Dfy6AijWw57CkIxERWU8Sk633NLPXzWyymU0yswtyHUOslsyCj9+AQedpli0RaZKSqONfC1zs7qPNbGNglJm97O6TE4gl+0bfH5Z7nJlsHCIitUhisvX57j46Wv8SmAJskes4YrFwMrxzM2xzsNrti0iTlWgdv5n1AgYAI2p4bZiZlZlZWXl5ea5Da5gxD4TloHOTjUNEZAMSS/xmthHwBHChu39R/XV3H+7upe5eWlJSkvsA62vUffDBHdDrO7DNQUlHIyJSq0QSv5m1JCT9h9z9ySRiyKpln8GLl4X1g65ONhYRkTrk/OaumRlwNzDF3W/O9fmzbtmn8MzPoGIdXDgBOm6ZdEQiIhuURKuevYEfAhPMbGy07Up3fyGBWBpnwuPwRNR6Z9/LlPRFJC/kPPG7+ztA/ndnXbWiMun/4DHY9rBk4xERyZDG6mmocQ+H5enPQ699ko1FRKQeNGRDQ3wxH/57FWyxO2y1d9LRiIjUixJ/Q0x6CtatggOu1CBsIpJ3lPgbYurz0K4L9FF7fRHJP0r89bWiHD55F3Y5QaV9EclLSvz19d8rAIddTkw6EhGRBlHir4/Jz8KEf8O2R0D3/klHIyLSIEr8mapYB69eB537wAn3JR2NiEiDqR1/XT7/CF66HL75Aj6fAYdfDy1aJx2ViEiDKfFvyIi/Vw6+BtCmIww8O7l4RESyQIm/Jis/D2Prv3JNeD7wHNhsR+i8tVryiEjeU+Kv7q2b4LXfhvVOveAH/4aSbRMNSUQkm5T4080dXZn0+x0J378PWrRKNiYRkSxT4k/5dATcc2hYv2BcKO2LiBSg4k78FRWw6gsouzs01QQ46mYlfREpaIWd+N+8EV7/XVjvsQcM/llYtmwL//s1jH2w6v5D/gKlZ+Q+ThGRHEok8ZvZ4cAtQHPgLne/PpYTtWpXuT7nQ/j3j2reb7OdQvv83t+JJQwRkaYkiTl3mwO3A4cAc4APzexZd5+c9ZMNPg/6nwIt24XE/9kI+HI+NG8F/Y6ALQdDs+ZZP62ISFOWRIl/T2Cmu38MYGaPAEOB7Cd+gLYdw7LX3uEhIlLkkhirZwvgs7Tnc6JtVZjZMDMrM7Oy8vLynAUnIlLomuwgbe4+3N1L3b20pKQk6XBERApGEol/LtAz7XmPaJuIiORAEon/Q6CvmfU2s1bAScCzCcQhIlKUcn5z193XmtnPgP8SmnPe4+6Tch2HiEixSqQdv7u/ALyQxLlFRIpdk725KyIi8VDiFxEpMubuScdQJzMrBz5p4Nu7AIuzGE62Kb7GUXyNo/gap6nHt5W7r9cePi8Sf2OYWZm7lyYdR20UX+MovsZRfI3T1OOrjap6RESKjBK/iEiRKYbEPzzpAOqg+BpH8TWO4mucph5fjQq+jl9ERKoqhhK/iIikUeIXESkyBZ34zexwM5tmZjPN7PIEzt/TzF43s8lmNsnMLoi2dzazl81sRrTsFG03M7s1ine8me2Wozibm9kYM3suet7bzEZEcTwaDaaHmbWOns+MXu+Vg9g6mtnjZjbVzKaY2eCmdP3M7BfRv+1EM3vYzNokff3M7B4zW2RmE9O21fuamdmPov1nmFkt85ZmLb4bo3/j8Wb2lJl1THvtiii+aWZ2WNr2WD7fNcWX9trFZuZm1iV6nvPrlxXuXpAPwgBwHwFbA62AccAOOY6hG7BbtL4xMB3YAbgBuDzafjnwx2j9SOBFwIBBwIgcxXkR8C/guej5Y8BJ0frfgHOi9XOBv0XrJwGP5iC2+4CzovVWQMemcv0IEwjNAtqmXbfTk75+wL7AbsDEtG31umZAZ+DjaNkpWu8UY3yHAi2i9T+mxbdD9NltDfSOPtPN4/x81xRftL0nYXDJT4AuSV2/rPyNSQcQ2x8Gg4H/pj2/Argi4ZieIcw1PA3oFm3rBkyL1v8OnJy2/7f7xRhTD+BV4EDgueg/8OK0D+G31zH6Tz84Wm8R7WcxxtYhSqxWbXuTuH5UzibXOboezwGHNYXrB/Sqlljrdc2Ak4G/p22vsl+246v22nHAQ9F6lc9t6hrG/fmuKT7gcWBXYDaViT+R69fYRyFX9WQ0xWOuRD/rBwAjgM3cfX700gJgs2g9iZj/AlwGVETPNwWWufvaGmL4Nr7o9eXR/nHpDZQD/4yqou4ys/Y0kevn7nOBm4BPgfmE6zGKpnP90tX3miX5+fkxoRTNBuLIaXxmNhSY6+7jqr3UJOKrr0JO/E2GmW0EPAFc6O5fpL/moTiQSJtaMxsCLHL3UUmcPwMtCD+573T3AcBKQjXFtxK+fp2AoYQvqO5Ae+DwJGKpjySvWV3M7CpgLfBQ0rGkmFk74Erg6qRjyZZCTvxNYopHM2tJSPoPufuT0eaFZtYter0bsCjanuuY9waOMbPZwCOE6p5bgI5mlpqrIT2Gb+OLXu8AfB5jfHOAOe4+Inr+OOGLoKlcv4OBWe5e7u5rgCcJ17SpXL909b1mOf/8mNnpwBDglOjLqanE14fw5T4u+qz0AEab2eZNJL56K+TEn/gUj2ZmwN3AFHe/Oe2lZ4HUXf4fEer+U9tPi1oKDAKWp/08zzp3v8Lde7h7L8L1ec3dTwFeB46vJb5U3MdH+8dWcnT3BcBnZtYv2nQQMJkmcv0IVTyDzKxd9G+diq9JXL9q6nvN/gscamadol82h0bbYmFmhxOqHI9x96+qxX1S1CKqN9AXGEkOP9/uPsHdu7p7r+izMofQaGMBTeT61VvSNxnifBDuuE8n3P2/KoHz70P4ST0eGBs9jiTU674KzABeATpH+xtwexTvBKA0h7HuT2Wrnq0JH66ZwL+B1tH2NtHzmdHrW+cgrv5AWXQNnya0kGgy1w/4DTAVmAg8QGh9kuj1Ax4m3HNYQ0hSZzbkmhHq2mdGjzNijm8moU489Tn5W9r+V0XxTQOOSNsey+e7pviqvT6bypu7Ob9+2XhoyAYRkSJTyFU9IiJSAyV+EZEio8QvIlJklPhFRIqMEr+ISJFR4peiYGb/Z2YHmNmxZnZFtO1eM5tlZmOjx3t1HKO7mT2ehViuNbNLGnsckYZS4pdiMRD4ANgPeCtt+6Xu3j967LWhA7j7PHc/fkP7iOQDJX4paNE47+OBPYD3gbOAO82s1nFXohL5A2b2fjSW+k+i7b1SY7Sb2Y5mNjL6pTDezPpG2y+yMDb/RDO7MO2YV5nZdDN7B+iXtr2Pmb1kZqPM7G0z2y7a/v3oGOPM7C1EsqhF3buI5C93v9TMHgNOI8w78Ia77w2hqge40cx+Fe0+ycOQFQC7EMZXbw+MMbPnqx36bOAWd38oGjKguZntDpxB+HVhwAgze5NQwDqJ0Au5BTCaMIonhMm6z3b3GWY2ELiDMGbS1cBh7j7X0iYlEckGJX4pBrsRJurYDphS7bVL3b2mevtn3P1r4Gszex3YkzCUQMr7wFVm1gN4Mkrc+wBPuftKADN7EvgOIfE/5dEYNGb2bLTcCNgL+HcY6gcIQz4AvAvcG31ppQb3E8kKJX4pWGbWH7iXMDLiYqBd2GxjCRN5bEj1sUyqPHf3f5nZCOAo4AUz+2kDQmxGGLu//3ondz87+gVwFDDKzHZ391yN5CkFTnX8UrDcfWyUVFNTXr5GqD7pH5XmN2SohflzNyUMYPdh+otmtjXwsbvfShjpchfgbeDYaLTO9oSZpN4m3Ew+1szamtnGwNFRfF8As8zs+9Exzcx2jdb7uPsId7+aMBlN+hC/Io2iEr8UNDMrAZa6e4WZbefuk6vtkl7HD6FKB8JooK8DXYDfuvs8qzo5+gnAD81sDWFGqz+4+5LovsHIaJ+73H1MFMejhOqmRVT9EjmFcLP5V0BLwrwI46K4+hLuFbwabRPJCo3OKVKNmV0LrHD3m5KORSQOquoRESkyKvGLiBQZlfhFRIqMEr+ISJFR4hcRKTJK/CIiRUaJX0SkyPw/h+2VX1SZ2goAAAAASUVORK5CYII=\n",
      "text/plain": [
       "<Figure size 432x288 with 1 Axes>"
      ]
     },
     "metadata": {
      "needs_background": "light"
     },
     "output_type": "display_data"
    }
   ],
   "source": [
    "average_scores = [np.mean(scores[i:i+100]) for i in range(0, len(scores)+1, 1)]\n",
    "# average_scores.append(np.mean(scores[-100:]))\n",
    "plt.plot(15)\n",
    "plt.plot(average_scores)\n",
    "plt.xlabel('#Episodes')\n",
    "plt.ylabel('Avg Score')\n",
    "plt.show()"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 19,
   "metadata": {},
   "outputs": [
    {
     "data": {
      "image/png": "iVBORw0KGgoAAAANSUhEUgAAAYAAAAEGCAYAAABsLkJ6AAAABHNCSVQICAgIfAhkiAAAAAlwSFlzAAALEgAACxIB0t1+/AAAADh0RVh0U29mdHdhcmUAbWF0cGxvdGxpYiB2ZXJzaW9uMy4xLjEsIGh0dHA6Ly9tYXRwbG90bGliLm9yZy8QZhcZAAAgAElEQVR4nO2debgUxdX/v+dedlF2EAUFEVFURETABV9XAhhxSXzVoKIStwSNRo1GTSRGk+irRuMWcUWD+4p7FP0pbuhFEUFEUEBBhcsiyM69t35/VPednp7q7uq9Z/p8nmee7qmurjpT012n6tSpKhJCgGEYhskfVWkLwDAMw6QDKwCGYZicwgqAYRgmp7ACYBiGySmsABiGYXJKk7QF8EPHjh1Fjx490haDYRimrJg+ffpyIUQne3hZKYAePXqgpqYmbTEYhmHKCiJapApnExDDMExOYQXAMAyTU1gBMAzD5BRWAAzDMDmFFQDDMExOYQXAMAyTU1gBMAzD5BRWAAzDMEnyzQfA0tnAhlXArKdTFaWsJoIxDMOUPff9TB57HQZ8NQXYfgDQrkcqonAPgGEYJg1+/EYe6zalJgIrAIZhmJzCCoBhGCansAJgGIZJFUot59gVABF1J6I3iehzIppNRL8zwtsT0WtENM84totbFoZhmOwg0hYgkR5AHYCLhBB9AQwB8Fsi6gvgMgBThBC9AUwxvjMMwzAJEbsCEEJ8L4T42Dj/CcAcANsDOBrARCPaRADHxC0Lw2SCtcuA9+8AhEYL8OMHgZULgPo6YOqNwOb18ctXaXw3A5j9bNpS6PHTUmDaXYlll+g8ACLqAWBvANMAdBFCfG9c+gFAF4d7zgJwFgDssMMO8QvJMHHz5BnAwqnATv8DdNndOV5DPTD5PGCrTsARVwNTrpaTh4Zdk5yslcCE/5HH3VenK4cTZBkDePwU4NtpwM6HAx16xZ51YoPARNQawFMALhBCrLFeE0IIOBjEhBAThBADhRADO3Uq2dGMYcqPjT/KY0OdXvx1y4HN6+S5eWQqk/Ur5FE0JJJdIgqAiJpCVv6ThBDm3OelRNTVuN4VwLIkZGGYzOBlAlJeT89jhEmAhnp5pGTa5kl4ARGAewHMEULcZLk0GcAY43wMgOfiloVhsoFmJW62AokKyoBYAVQMKgXf+J8nowCSGAM4AMApAD4johlG2OUA/gHgcSIaC2ARgP9NQBaGKSOE4pwVQOVh+U8TVvSxKwAhxDtwfmoPizt/hskuAUxA3AOocEwFUCEmIIZhbGhX4hYFoOMyypQ/lTYGwDCRUbcZePky6Qpp5cdvgdeuAhoi8JxY9B7w4d3h04mCchkE/vRR4Mv/OoS/Whq+8F3go3v85bFqEfD6X9RlsvAd4KN7/aXnRJRpeWL7LVs2Aj99Z1wyrr11PbDsi9gkYAXAlA+fPQ5Mu1NWBFaeGgu8ezPw/Qz1fX64fwTw0sXh09HBs1WvGgPIIM+cDTx8vEO4YmjvgZHAixf5y+PxU4B3bgKWzVGkdyTw4u/9pedElGnpYvYIpz9QCBMNctLfm9cC9w2LLWtWAEz5YHaPG7YUh9dtTF6WUOh6ASlMQHkdA6g3//MMK8Kw1G+2fLFMjarfooodCawAmPLBrPzsdUDFVo5lYgJiosE6+Uul/GOAFQBTRpiVX4kGsF2vEESZmICYYJRU7NZKv4JmAjNMJDT2AGwvjrBdrxiMH2b9vRX3G5lGEqr0rbACYMqISusB+JgHwG6glU+JCSj+55kVAFM+OPYAkp08E5og8wDKVskx2gjHL7FRJm8Mkwlq5wJPnFbqlbBpLfDIScCa72IWwKEHYF0zJw42/WT8vu8LYUIAk8+X8wbc4i6bY5SZw8qf794CfPyQ+ppV0b17izzOmBToJwRCCLkc9aL39eJ/8h9gzvNy2Wo7S2fLZbCdyiEo7/wz2vR0+XwycNu+6t/qep9lybO1y4rD37Qs881jAEzmePZcYPYzwPefFofPfhqY+xLwxrXx5u/UA4i7dTzzcfn73rrOkqUAPp4o5w1Y+ewJI+4/5Penz5RltnRWaboCwGt/BiaPU+dr9W5aVyvPzaWkk6B+i9yQZuJRevGf+y3w2Mly4xo7T/0amPUUsHxutDK+Pj7a9HR5/BRg+Zfq3+p636mF8y3r1OGA8d/H3wtgBcBEQEJmiUYTj5MJKCY5lD0MXaWj6rUEMQGlQNw9qyhmblcsyXiAsQJgIiTuCiulHoA9f6UMTreEkCn1gd8Iy1X1W3Q3xKl0VM+IaEjk/2cFwOjj9EA6mmZiE8T2Ne4egCp9n3kqy8bPUhApEEu5WpVofYTpVhhFJqD4GjasAJgA2B9IJ/fMqLNNqwegSF9b2SnKJpTSSJDGgci4TECsAJxhExBTLiQ+OcnBCyiuF0XpZurTBBRItIwogEj+X9XuV6wAHBGCTUBM1giyh22EmBWwUz5x5a/apk/o9jpceke6ewKn1RMQMa9Nzz0AidvWkDHDCoDx5sO7gUmW5X5r7gUmWZf5dajk7jlcuv75zevhE9TXzJbo588W+1+bL9Adg+USun65+zBg1tPFYWtrgVv6A7VfAh/eVchfCBn/82cLcadNAB4+sfh+c2lfu9lKCGDJdGdZpt0l5xHIyLajwfJ5Mp49T0CuZ3/bvsCDRwPj28hzL54bB7xxTWm4l5Jb+A5w2yDv9IUAahVr2nspgA/ulL/hHzsCyz6XYXOel8dvPgDuPMA7byvj2wCfPQkseBu4fbBcf3/B2zL9/+sNLJ9fiPvPPYEVXxXfv2Bq4T6/NDQAN+6mvvbW9aVhE/4HWPlVaXjEJLEnMFPumOvjb7e3PNonIzmZCBZ/BDz5EbDHL/znpcSSz9QbgcP+bHyxVJDL5gDd9tHPDwCW1ABPng7scVwh7IvngVULgA9uB1YtLOQvhIz/TE0h7suX6Mlccklx7eU/FM6dWv4f3AHU3Ke+9vJl0j99+Zfyu3l04xNjItqhVxaHe5mAXr5Uz6+/aJljRfpOvHKZPFrnPvy/vwMHXwZMn6ieW+HFU2OBjn2k3KsWAC9dUkj/wwmFeKu/kd9HWOZ+vHypVGQr5sM39ZsKm73YmfmoOnya0fDg1UCZTKBrstCJGwSniqgoX59dZz9yEqGkNe53WQdfZRTA1BWlXd1zEFjzt1tb+sq5FGlBKP4NDt5lJtVGe9m+H0VcOCnOCGEFwESAwgSUqN06oWV0ifynH8ZF1nGsw0WGKH9/Yw8gZDpO/v5peznZ8ZKnylAAgZazCFCIdZsC5OMPVgBMeJwmskSej8PjGmcPoOh6AAVQohz9uPcFURoxKIAglZd1lq+jAsjaTGAvBdBUHpOawBbjTmAmrACY6Ih9A5MYTEC+sq9SKAyPylE1CKyLY1w3E1CEv7/BwwvI7acLDQWQugkIxY0Xr7KrCmMCCvBb2QTElAcJmYAc7e1hFIDPMQB7+p5jAGGWgghQmUfpWhlmHoB1LMKqAJy2PUwUh3WZvOQxxwCCtMyD/NYE9kFmBcCER2nnrsAeABSDwNooTECxDAJnxQRkUQDWCrNIvjDPSATPF5G/QelQJiDuATBZ44uXpG/0xtXF4T9+K8Mf+Dlw857B0rYu0zu+jXs613QBPvi3d3qPnuRw0fJyPTgKeHJs8eW/di5N//4jgWu7Ale3L5bTZOE78mitIJSDwJbr49sAL1xou2ybvDZncuHafcOUv6Yxrak3qa81uqUCePosefzvlfKeHxeVxhcCuLqjnK/gxtUd5dLP49vIfQ3M37p+OXDDLvJ85QJ5fcl09zpt9jOFc2uF+fTZFrlclNV9w91ldcpzfBtgyceFMNV+AU6usebcDRNz/oeJaQJaPq84/LtP1Ok9OlrKc+cBwKtXOIrtyOIP5bF+M/DXTsD81/2n4QErgDwz9QZ5tD/QX78pjwunAj9+Y7ngo8X6wR3Fl4rSsd4igLqNwCuXurds3Tb+sN8368nia/WbZPpWFr0DbHGZNNZsK3ls2b44PKgXkFk2qkk/TnzisFHMwqmF85mPyeN7tzqn01Av7damX71jvC1yYhQArFpUXK5rl8qjWQl9Msnd5fTNvxXnb7JsduHc7f/+xmsTGkWv5InT5PHTRwphn/zHIx0fVFXL49yXisNnPFIaFwC+eEEel84Cpt8fLu/6zbF0qlkB5BnHF9Bnlz8qV8egNmFXt8igaRr3WbvhqiV6/S7slobnSxBbPlWpZbUrNCesrX7HQdMETECeZhQ/ZWI+57ZySeo/jWHJLVYAucZc5Mz2ZPke9IvoyQz8IsVhEzcVgM1+HdYNNI2BzyCTw6qqvRWr228pUgApzgOIdAtKBwWQBW+mgLACyDPaC5rZ4zt8D/QiOAzgRlU5BFUAjT0Ay2ScKFp6afQAGk0wPhW1UnFo9gCcvICK4iRQcbq6bJK/Imkcz0mpBxBDF4AVQK7xawKyKwCbaSH0bNeA3jyu+YY1AW0pDnMbBFalUVI2afQAApiAGupLf2tDve33uPUArF5AccwDUP0WxXPoZgIi+1IQXlkace2utomZgMpQARDRfUS0jIhmWcLGE9ESIpphfEbGLQejwO+OTyU9ALubYJQ9AD8vlY4JyO/LY6RpnY6vVABuSTSU2sxTNQH5UQB1pb+1bhOK/mtXE1DcPQDFvarnOMrZtI09ALsCYBOQGw8AUPl0/VMI0d/4vKS4ziSG5hiAveXTOFNUs2JRLdXs5MPvt6L1uua39WTe5zUI7Eb9FmDLBuPeNAeBfSp6QJpO7LLWb/bRA9AZA0igLNw8vQD9MmmoLzzvJZPtklIA0fcAYl8OWgjxNhH1iDsfRoOb+gKtuwBnGW6e5oM7Y5JcfxwATn8FePZc9f21c4q/219gc612J/7WVR4v+RrYqgPw8UPA5HGW9GzK4OMHgcnnyfgqxrcBxq9WV8q1c4HbBwFNWhbCbugDrP3BXUYT073SqgCatVK0ijc4p3Ftl8L5Q8cAQy+GY2VhnYMQNU+Mkce6jTKfU56V8gDArj9X39NQX9o8rN8M/R6ApdLftMYhUsCK84nTiucZNCZn+2+u38k7LS8ltGAq0HOo3F/BdL+1KzQ2AQViHBHNNExE7ZwiEdFZRFRDRDW1tbVJyld5rFkCfPdxafiHdxfOP3XwaVYR1Lzy40J5tE+8sfcAzDXvrROf1IKUBi16Tx4bK2jSr/ztMrXvJc/b7xTuZZ96QzrmAtO338S6SY/pq26noU496F+0xaWb6c3SSt6wyiFOwLJQVf4q1q/wjlPn4SZqNmqscy9KxgD0xMkiaSmAOwH0AtAfwPcAbnSKKISYIIQYKIQY2KlTp6Tkywei5MRfKyPwWjFG/OqmTgL5NAGpwiJqlQkBNG3lkZnP9NJGRwbVGAAEtHsAVoJuCBM7pDFPQPEb0+oBVIoXkBBiqRCiXgjRAOBuABr7yjHRoxpI8/FIhH3wq2wWyKIVJP34ris1QPHXoN1n0VBIy+8gsDrBkPdHgM68gPotipausJWjrgKIYyKYC77GOjzmCaiupzUGUCkmICLqavl6LIAAe7sx8RCgB2C9p8FHBWnvAZQMCAf0TnIKC4TNTTVsunG0Fv3KpLOYmcoN1NoD8JoIZsWxBxBTxelroN6jB6BqiKTWA4ie2AeBiegRAAcD6EhEiwFcBeBgIuoP+UQtBHC2YwJMfKhelLAmoPrNQFULvftLegBOy0J4TTpSvICRTSSzyRT2ZY+j0vO7BLSWAlCZgGBbDiHscgxp94aE965bqt5SBZmAkvACUi3heG/c+TJBCdIDsFC/GWjqoQDMSqTK3gOwDQL73W/XMywI1pZuRk1A9T63DtRRGCo3UBF0DMDBBJSFlrPXPAFVjzateQCVYgJisoLiwfWzBvmGVdKLom5j8f2b1rrft3GN9I/fsLI43OouuGEVsGUjtFC9gPaWXdBt/NYuA1Z+ZeTTAGz2+G1ebPop3P0q/K4br6MA6jYV5jCYrF1a8KoSwtvH3mTzOuc8vLxw4mTzWmCLg2wmor50JnPJsyRkHPN53bIR2PBjZGLGSew9ACbDqCrOjyfq33+7Yux+6o2lS0HbeXCUOvzWAZa09y2cB9k45bU/edyjybLPC+fv3qLhkuqBbqXpB7+V6NwXveM8fWZp2N2HFM5FA7BO0y37/dvU4ff9DGi7A3DBZ3rpRM2Eg73jNG0FTLTNlVAtBfHASODbaXJeinX+R6RwD4CJlBi6rp89EX2aXiSwcxKA8JV/XCT1+61EtTG60z4RWWHrbUv3JigZAxCy8o8bNgExkZKFAckoyIItOU3KWQFkHaUXUEqLwcUAK4BcE4MCCLL2vHeiMaRZQaShAMq40vNFhXsBsQJgoiXpma5ZmFmbNl6ujHGQRk8vDZQ9nZSeOTYBMZESR+WZdMswLy1RN6Jc8liXvJS7jqIr40YIKwAmWpKe6ZqXisgNv/MAoiAWU19URFgh6zxfZWwCYjfQSmTzeunjXNVE7u3aUC+PVpbNAVbMiz7vONwc1y1zvrZqUfT5lRtp+Jx/9aZ3HF1WLwbadIsuPQBYv9I7jg46g90bLeXvNQcmY7ACqETuHQYsNX2rCcWrOAL4Zhpw37AUBAvIYyc7X7ttn+TkyCqPjU4h0whb2bfuA1y5NLr0QMD1PaNJSscEtPijwvldQ6PJVwWPATBaLLVOrFHsRbv8yySlYRh36jRnfKeBjqmrRdvC+UqHzYsigRUAEwUxtCQYpuxRLYWuNQhcvuNQrAAYhmEAoLpZaViWFACbgJho4B4Aw5RgX54c0DMBJeYGygqAiQI2ATEVTcAKWaUAstQDiAFWALmEFQCTYdKaWKU0AWm4gSZmAoo+SVYAlcTSz/X8n1d/G78sDOOXdSuAFV9lSwForfJZviYgngdQSdy5n168N6+NV440aL5N8YYyTPlx6wA5qerPEU3i8ot9siQA/PS9931lvC4S9wCYyuDU5+JL+1cp7HGQR8wZteVmU9eRt80O4fNhLyDGkTJekCoSmjSPL22vPY6ZaElLAQSuYDXevbYRKAD2AmIcybsCIEX3PbK0+TVJlLSe5aD56iisjHre8ZNdMeRcAajst1HBCiBZKtEEFMUzxCYgxpHc9wBifJRZASRM5Wy40kgkDRRWAIwT5dZqihrVJJ6oYAWQLJX4LGf0GcqmVEwAct4DYBNQefPFi4XzsArg44fC3R8HbAJiYiVpE1Ccg65B4EHg8ubRXxXOwz7LfndI63eCPG69Xbh83Yjk+WQFwDiRZLe5ujkw7Jrk8tOhqho48ZFo02xiuH9m1IMjcxx1i/v1fifqpeP0LJ/8lDq8WWu9dJ0YOBYYvxpo0SZcOm6Yz9C2exbChl4cX36asAKoGBLsAWTRRlvVJIaWuvHScg9Aj6h6YU49gDjHeYB4Fb3yGfL5zrIJiHEkSRNQFhUAVcVXUWfN3JVVvMZhtCswh2fZ8X8IWTE2ypWEArDkkYElJFgBVAxJjgGI7JlFqqpjVAD8mmjh1ULXbaQ4NTBiG+hP4FlWPUM6ew0UJxKJKFZif7KJ6D4iWkZEsyxh7YnoNSKaZxzbxS1HxZNkqzyTPYDq+JQSKwA9oqqgnZ6vuHtiSc8D8NtrL1MT0AMAhtvCLgMwRQjRG8AU4zsThrxPBItlDMCAFYAe5ToGkERvVlU2OnsNxEzsT7YQ4m0A9vVdjwYw0TifCOCYuOUoC1Z8Bcx7TT/+oveBr98CZjwMngfAPYDUiWoMwGkJ5qqY/4ekB4F9jwGUZw9ARRchhPkv/wCgi1NEIjqLiGqIqKa2tjYZ6dLi1gHApF/qx79/OPDgKODZc4HvP41PLjvDr0Pqu4qNuL74exyDwGaF4Ldi6D+6cN5jqDx2H+I//467+L8nTaJqod99iDq8fa9o0i/BYRC4ZYSW6T4j5PGQywthbbo5x1c9L2magIjoQCI63TjvREQ9oxBACCHg0nwVQkwQQgwUQgzs1KlTFFlWJls2JJfXkHP04o28Abjoy8L3sa9Jf+vLNTbZ8GLw2cXfifwpAKs/thNVTY20fSqW7QfI48AzgNNekL9537H+0gCAcR/5vycJhvxG/iaT7oPlUVcBHH0HcMFnenGbG775e/0KaLEN8IcF+nIOPhe45GvveKp6deAZQNNW+nl50X4nWWZ9Rsjj+NWF9Pc9U36/cHYh/thXo8vbBa0nm4iuAnApgD8aQU0B/CdEvkuJqKuRdlcAy0KkxWQV4eAtVC62+mqjQgva8rLasrM4cB4Uu43e/O5Z/kHK0Uw7wL1U5e8+e9wox9W05gF4yZpeD+BYAKMArAMAIcR3ALYOke9kAGOM8zEAYtzOKSfEVsFE+dDFPLHKV7oav6txj1i/ZaCIX1GD9HYFYDx7vkxAmmXaWG4uz45TJa9d+ScxCOySR4ou1bpvzGarqYaIttLNgIgeAfA+gD5EtJiIxgL4B4AjiGgegMON70wYMtvCtDzcQW3q2llFrFhME1AUA+yZ/X8CYP8tjQrAhxeQ7jNgpt1opo+zsiSH86jTzg66KvtxIroLQFsiOhPAGQDu1rlRCHGSw6XDNPNmdMhkCzPDJiCdisQ0AdUHdderUBOQHd89AAH9HoBZbgF6j7omIMeGSdILLHrIGoPy0/rHhBA3ENERANYA6APgz0IIH/6KTPzE9LASRahc4l5bx8cLovObzB5AwxafYqhMQBWkAErGAMxWug830MDLQvi16YeoNCMdA4jALBhDI89TARBRNYDXhRCHAOBKP6u88kfvOFESxL4adzc+6nTNMYB6nwpARSUpgJIxAOO7Lz99n2MA5NZ4cErL7zOaVg8ggbWIHPD8x4QQ9QAaiCjGtVKZ0KxZkrYE3nTazTvOPqd5x9m6qzyOvKE4vF0PYKvOpfHt8QBgxHWlYa062O67Xsrcsbe7PAPGFPuM73oU0HZHYL9xlrCfu6cRNy3bF3/vtCuwz+nBlkAefG7x9xHXyXLq3BfYfh+gi8XFtrq5PB53DzDgVOl+u8N++sp60JnGicv40cjrS8PMuGFMQLGbVctnKYi1AD4jonuJ6F/mJ3JpmHSx+nbr4lap291Am1n8qq3zA6x4rSkPACcZ6/43Vg4GrdoD57wjz6uaFPytB51ZODfZcf/SdE+dDAy7Vp4P+Y2M89sPgKYtgT4jneUZ9S/g0oWF7607ARfMLFYcrTvplW/X/t5xnHD7Lw69ovj7Uf8CjroZOH+Ge5pH31E477Kn/A0ddy6O0+OAQjmd+QbQd1ThWs+D5D39jgd2GCL/nw69oNXaPeB30n8ecO8B9DhQfT9V6eWjvJeQWA9Ae0A8BROQwdPGh2GKCdoqyaQZyGWV08TWbQnxkru6GjrY573Kq8izJ4BsYVw0i+aIuplJXExAWs+DKu0ox740ScEdVHcQeCIRNQNgzk2fK4SIwDDKlC86D2tML5Dri0kacQKkG8U4QOy4/Cd2F03dyibQAm/WcR8nI4OOAmhQjAGEmNiVKtmcG6KlAIjoYMhF2xZC/pLuRDTGWOiNyTUuL5lORR01Ub30dtkzsHKjJ27eVY4umn56ALpKQyN97R6ALR1fE8E0TUCpu4GmNwisawK6EcAwIcRcACCiXQA8AmCfuARjyoSgvsthTTXOGYZM18kElP7uTZ64/fSSlrxmizrQGv867p5Be5B+/t8wJiBkbxA4BnQHgZualT8ACCG+hFwPiMk9WepmI5xicXvh/c4FSAUfJiDtJAPcV+Tv71DFaPUAdE1AMTQyoh4EzuhSELo9gBoiugeFBeBGA6iJRyTGlVevALrsDrybkBNWi7bABvt2DhY8n92wPtpJ4qYAysEE5KOS0d0HN7ZtGDVQDQL7qiw1J4KZaVbb2rQ8CNzIuQB+C+B84/tUAHc4R2dioaEBeP+2cGmc9hLw1j+AJZ8A3QYCX7+pjjfsWrn8bqsOwKO/ck7P7R1xqzStD/tpLwEPuLhZ2tm2n0u6ZovTQbBh10ilpqLLnnKTHRXH3gXcPqg0Lav/v/17GPa1ubjucxow/QH3e9zGAKhanYanCc8lzRMmAd99orrJ+36dyu6gi4EmLYAFbwMHXeIio8sYgJ9KdfA5wIxJ+vF101y/Euiwc+m1vU8GFr4LHHhhtHn6QFcBNAFwixDiJgDm7ODmsUnFqAlqhjjmTrlpDCB9tns8X7g23mEi0P7j1OF72ZSBm0wNW/RewB4HeMexYm+pWTHzc2q97X+eOnzQWYW1f2QCxdc79ZG+88+fXwizp+WUth9MufceXRx+xF+9FYBba5eq5ByLBW8DK7+2xLXcM+o2YPK40vuc2O3n8lOSVwReQO17Aa2NSX0nBqyUSSMfqyxd+8mNhl7+Q7D8VHTZXU6AU9GiDXDSw6VyOBJ9j0R3DGAKgJaW7y0BvB65NIzEqfKq25SsHCrsttj6zc5x67cgeVNPyPwy5Tpowc+MVtdrfscJgpSHziCwVxJ+TT1+wnVJf5A2bnQVQAshxFrzi3Ee4XY5TBFO68Zkwhfd9lK5yeR2LZMTwSyk5aMdauDS7XW2jwEo8lMN+JLDuascGjd5/s4I/ke/q4ECtk18wosQbQMo+ndGVwGsI6IBjWIQDQSQ4B6EOcOp8qnPQA/AjqsC2JxCizpsfin3AEIpHg8TkNc9Xj2AIKIFNQFF8dxorwbq5FmUtUHg9JaCuADAE0T0nfG9K4ATIpeGMXBSAC7mlrRwk8nVcybjPYDMEZUJyJaelr3eLxomoCh7AKHmGrgQRS8w48+j6z9ORPsS0bZCiI8A7ArgMQBbALwCwMfuzIwvnExAdVlUAB49gDh8tF0x041jrfWs24RdZ4J5367qARQpCF0xdOxGCfUAQo2dhJ2IFjXJm4DuAmDWOvsBuBzA7QBWAZgQuTR5ZPazwIsXFYepKqONa4D7RwTIIOKHptreacxYpZjxFpc+Tn77bre4uYFqzAPwtaevqyDeMnn9ntovguVXFBygR2MqQaeySGxLShXJewFVCyHMWUAnAJgghHhKCPEnAArHVsY3T4wBPrrHFqj4o+dMBtYv95/+OVP9xT/jVfX6+YBc2vfw8cVhux8L9D0a6H9yIezMN+VyyodeGax77iTDARcAJz1WHHbcPbawqBLT7wIAABxPSURBVF5Ql5et4y7AkTcGS7b7EHnsPcx/viqGWhoPrpWTRk+syx7697kRhRuoLkfe5CaIdBk+9Er99PY+Re53cPBl6oaYb6WS7QaJpwIgIlMVHgbgDcu1qJoLjB3Vg6djouinGJbZds/SMDd2GFK6zr7JmOdLJzpVNwX+90HgmNsLYdsPAIb/3X1SlFtl5STDEX8B+gwvDut3fHFY2Baazv07DAH2/XWw9PsdL49tugW7307RvgYBxgAav1YXV249DwormUO+HuF+qGoC7DvWOw+3SWR2mrYARvxDToJM2gSUwZnAjwB4i4iWQ3r9TAUAItoZQIDdQxgtAm8fmMXWRrl4ATlscxg1nun67THp+twrBn117g9UKenIlMBzEXZQW9kD8Cl3xk2SrgpACHEtEU2B9Pr5rxCNJVIFIIJpj4waVSWhUSE5PWwZWHe8lKx6Abncn0g5htgmMMhaQI3hESrASOYB+MrQZ7gNx9+atUHg6PE04wghPlCEOeznx0RCUBNQkg+n9oYiaXkBBSXmco58ZqyOx02Y+8u5BxCDG2jOxgCYNAhqAkq8ss0gkfmyq8hiT8qCn5nAqoopKt/8qJaD9s7IPS3dZ8HXLmOVVWVW1q+pGAKagJxe0FRNQAm3AINWLI1lFMZvPEKC9LCCrDnvZK6xr8MvA/Vk0nEDzRJ+3o84//sUGmpl8O/khLW1hfNXryi9/vzvvNPIYkM/ql5Jy/b+8tON74iiUmhqLH/VrHXwZL3SMD2nquwrnuqUlw83UHv5t+rofZ9uPdnMskxY86315ImD0BVqxsYAIpunUYBdObPCfMviqjMmAccE2W7B8nCe8B9gw6riy01alN5yzrvAB3eWLj+sS1S9i9FPFVccds6cop/WqNv8LzFt4lZp7Hk88NP3wKCzg6UNSFfdtUtlGu8Zm/occiXw5jXy/Bf3AbOeAjrvppeetfyDrAZaVQ2MuhXoMdS5tX/olcAb1wBCc1vMfifKcqrfIuduqKhuAgy/DnjlUr00AxHWJdhoHw+/Dpj3KvDVG6VlPOhs4MO7wuXjxdCLgdZdgA69Ik+aewBZwW2Ne91K1vpw7nZU6Trkux9Xes+2e0gf/iJ/8ijRNAH1PtxdhvY76Wc54BR/8VWoyryqWm7e4aaovFClMdhUKAS07gQMOcfZa8debtZKOegYwIBTgfY9FXGMfHc11vzXXY22uon0vT/kcveyclonPyuYvbA+w4FuxmZA9vGNIeeU3tfnyIAZOrwrh/0JGHxWwDTdYQWQFVwVgO6gsFeLJ0ODmHkamPYiTFk0WJ8Ntx6Az1fdqgCrmxl5RbwceVTPQFzOD+ayEFSFkq0pzXNVuVaVT7VaPpJWOiU2XwvaCiDJCj6kGygTDdZnI8g8AK8wUEEBRL4fRcafDbNRRlUOg+JQK4ByGPg2KB9JKx3zJVPRoGl79SSOFy6o0snay58FTyk3GZw8vKwKwI8JyCuOogcQ9XLkkTUOYnqWrI2yxnK2eV0pFYBqXwUNMrgURKwQ0UIAPwGoB1AnhBiYpjypUrLKpoXITEBpkEWZMkaYF1/omoACzgMgKrSEI9+PIu5nI2T65jvZUIdGhVjSA1BU9sqNdbJJFryADhFCBFjmssJw6wFk0gQUksybhtIoywB+/EWDwBpuoLrlrhoDqHfb4CcAcY8BhMV0u6yvc54nEqkJiOcB5IPxbYCptmVs7Wv9L3wXuHeYjPv37fXSXfS+w4UYKrOwS0FktWeQimLyMxPVFlfXPKgzBuC0FEST5vK0bXe9vLTJ6DNgYnqSVTdBoQdgqzJ5DCAUAsB/iWg6ESn9nIjoLCKqIaKa2tpaVZTyZMpf3K9/PBH4dpq/NFfMCy6PG2NfB855J7p4uox+Krq0vEhjtvR5HwOnTvZ3j13OKMcAnOYBVDcFTnwEOPU5HQn1cTN7+sL229r19L7l9Je94/ziHuCX9wPtehTK2T5zOsoxgBRI2wR0oBBiCRF1BvAaEX0hhHjbGkEIMQHG7mMDBw4sIxtHSCL1uAjZ0uq+b4h4IVz0uqU4JJSEQujQS362bNCIrLHMh6sXkL2i8tmD23WkXny/dOgdfcOldWdg1QL38ujc1zudlu2APYy5M44mIF1vKg3ythSEEGKJcVwG4BkAg9KUJxF0K5ZIfa5jrMwCV5QZWXOn3NEeBPabbkJtraj2/o39HoUJyNELKG3Dij6pSUpEWxHR1uY5gGEAZqUlT2LoDuhGPeAWOUnMA8iLAgihDH3PBNYdaPaxOF4oQqSf5P4XfuYBBF6zJ19uoF0APEOyQJsAeFgI8UqK8iSDdg8g6wogwPIUOuF+40RGUhVeSIiKn6EGTS8grbKM0JyhS5TLQjei8186jHc4IRwGgVUun+wG6o0Q4msAe6WVf2ro9gCinnYPZMukkiVZ0iaJeQB+FVtig+IxPAexyO6wFIRSabIJiHEiTRNQpC9GEhV4mkqiTPwNgi4Fofp92q6hERJLQyDE9qmOSZo9AI10ymgmMCsAXaZPlD75Py31jntdT+CO/eT5+DbFny8t7me3D3FOI2gPIMxa9XHSZU9/4UD8L4R1eey2O8rjNtsVf08EH+aZrrZOc+sulii217n5NupzLTmSGgQOUQWZZWF/Tsw9F6J8flRLQXTdS13Zuy3sqEVyiiBtN9Dy4ZP/yOOqhcDWXVyjYsNK+VG1uGc/UzivneOcRtDW+nnTgZ9+UF9Ly+wy9jWgw8764Ulx/gy5Nv+65UCvQ2TYLsPl/APze5K4/T9VVcAZ/wU67SIbIXcMluF9RgB9RgJzX0JJxdFnJLDrkUCLbYzlniOWKRICpP+bacAPn8klxO0cfbsxIXKqR9pBvYBMRdwfGP0E0KQZ0OtQuVcAIOdLLA+6ZXq+BoHLC+VEEA9UA7lRmmG26gysW1YctvW28pMYGr+nu4N3r1N4IzG/ENt0lZ+iLEldscSJ+Ux5PRs7GJW+uWuYee/Oh0kFoPJQ6TtKnZebHDqyREaAfDrvKj+NWOTe+2SXGfHWW4KOiRj37TAEaNlWnncfUlAAu44E3g05ryHBhhqbgLQJ4CWiVACaYwBaD0GKduos7Ytb9kS1Jo7GMgW+5SjH/yDGQWCl3385lpGEFYAuTm5gboRRADotMN1WWqqbwoehfF+sdLD3APzensIYQBTPZiBXY789AD8WAJ4JXHk0PgA+7lEt56C9sqcGUabFpEdkq2J6BhTQ3mY0qDAVhtNSELHAJqAMEsQEpFipMY3WeJwtizh/Txl3rQMR+vdqzFLVvT/LYwAlpLQURJj83eAxgJjYvB7YslE+3BtW+bt34xp59PNSbVytCPtR795NP2lEYhMQY0FryWcf98vAwOKkRixLQfgwAQWuwNkEFC9/6wrcuAvw3r+A63oAqxfr37tqgTz6+XNv26c0THeJZzcXURPzQTdXNtymm17aTMYwnqmdDg52u7lu/bb9jOSM17rLHiFkSqjR0H1w+DTs72RHw7W4VcdC2Fad7DcVTlu08c7DbQywDHWkSf7cQDeuBr54UZ6vXgy08VtpZunfNh7KI64GtqwHeh2WYN7sBRQZVVXAuJrCJDQdLvy8sFNXr0OBs9+WCmCng2U6K+YXFII2ChNQ3P/BiOuBQWfJiY93HVQIv3h+8DSH/BboNgjYcf9C2LiP5Lt/i2L1Ga1ydzMBl0wP1pOzJBmeB5BNdNdcT4vqpkDvo9OWIgYyWNZx0bG3v/htbLvEmbNizXTsM4Yb8btgXNyzsZsBXfoC61cWh7e2t9jdsM8EbgH0HFoc1rJd6fwJoHjjdzca63+NBfOiHsuJkXyZgIKivetSwjQ+lBmSKUqyqGwrmhTcQBuzzvgzrFoKIi54EDhjZNbdskyWMGbKl6Qqo8wvoeywGFykWSTvrMEKQAerAsiSR02QyWnRCxFj2qzY4sNjNdCkH/Mwz3ASSsp1HkBEYwCR3a8PKwAdihRAhnoDja5pFfo3sgkoYVJ0A01rI3XtZywLja3oqaxfo8vKr0vDfvymeHPuTWuBNd/JyVwL37FEFIX4m9YCK76SK4QunS3D13wfl9TOaD/EXKEyGlAVkh8DiODZjFOJuM0DiKyhkrx1IZ9eQOtqS8Nu3hPY+XDg5Kfk93uPAJZ9Dgy9CJh6YyGe2RW8WbGO/fmfABNHlYbHRe/Dgc+f8+E+GOED1mV3edxhf/d4YTBfrL7HxJdH1tl2T7n0cVS4rgZqtAd3Pw5ouwOwdBbQunN0ebsReB9dFOTe4xc+7jGUhe6z5ae33dWv+62BWQa7H1sIa9E2WFqa5FMB2DGXbJj/eiFs2efyuOi94rhuJqCflgKrv41WNpPj7gGe/nVx2EGXSD/qRJd/Nug2EPj9HGDrrt5xw3DxfL2JOpXK2NeKe6ZxUlUly7tlW1nRDT47uWerqlrmfUOA/SHMe1v6qCyrm/i7x1zXS8dttMeBQM+DgAVv68sDSHduq0yXLopgcxl3WAEAQN0m52slGl84DwQLxdo/UaFq5VO1zxc0YhOQn4lLQfHlD16BNG0pP0lhLe+kGxZh/usg9/q5x1QAygpZ8V5tHfDdsMrkR6EFJJ9jAHbqNztfsysAAedeQJwDxJl3k2PKiix5s5UDZh3RpHm6ckQMKwDAnwKASEcBqAa42EuGYZLBrCPM5TeslPF7yAoA8NkDaEipBxBmGVpu7TFMKFxNQOVLvhWA2Q32MwYghHqdfyD5HgDD+KZ8W6upYjYSlYPA5Vum+VAAdZuBbz8sDRcNQO2XwFvXF8JWLwa+/n+F719NKb7npYtLPYNM3r89tKiOhBoDKN8HlGEyQWMPgE1A5cerl0u/fjuiAbh9X2Dmo4Wwf+4OPOiysuYPM4FJDv7GX70RTk43zJ7IPqdbAnVNO2wCYsqIjrukLUEp/Y6XR3OugdfS6+ZS1O17xidTBOTDDfT7T9XhaS7r0HsYMO+/+vGpCvjTcmkKWvgOsGIee3IwlcefVmSzRb3/+cDgc6QX0JXLbN5AhryDzykE7TMG2O0ooFX7RMX0Sz56AE6kqQA6aE54aWL4gFOVHICqqrLMmmQFwFQY1U2y6fJMVKj0nVxB7eOFGa/8AVYA6eXdpIVmPNPmaGkVBW0hZbBhxaQFNx4iI4s9Fk3yoQCc/qA0TSjaCsCMZ5XV+D1+5ed3nmEYC6kqACIaTkRziWg+EV2WuABxLt3gRbXm8Eu10d1sqCuEmQotS0tTM0zeKcMxudQUABFVA7gdwAgAfQGcRER9ExUizQpUd13xJgoFEHjT6WC3MRVEGZsrskv5lmmaPYBBAOYLIb4WQmwG8CiAZHc2f+TERLMrQndil6kA6usUF8uvxcEwlUv5vY9pKoDtAVjXTl5shBVBRGcRUQ0R1dTWKtbx1yFrXTOq0u8B7H+ePFr9icnnGMCOB8jjnsfrxWcqlyHnymOb7unKkRZtugO9Do02zV6HyGMZ7luR+XkAQogJACYAwMCBA4PV5EfeCNw1VJ73GAqc+hxwtcVFq2krYMv6sKLqc9Uq4P07isPGr7Z9N9bA73cCsJetp9LYjdcsjg69StNn8sneJ8tPXrlwVvRpdt6tbN+vNHsASwBYmyHdjLDosfrtVjct9TNOo4eg2wNws9lmrWfDMExZkaYC+AhAbyLqSUTNAJwIYHIsOVm3m1PZ3tPwBgqzuXTjvawAGIYJTmomICFEHRGNA/AqgGoA9wkhZseSmbWyVe096rS6Z5yE8sYwxwAikYRhmJyS6hiAEOIlAC/FnpHV5KOaZp5GDyCS6e6sARiGCU4+ZgJbW/1ZWWcklAko4ExghmEYC/lQAFa7f1Y2VgmjAODTC4hhGEZBPhSAtdU/5DfyeOifCmFH3lg4P+SK6PPvP7qwkURfY67bzocXrh/+l9J7DrkC6OwwMfrgy+TORJ36RCsnwzC5gkQZmREGDhwoampq/N+4fiVwvTGR6qof1QOwpt/9VT8CT54OzH5Gfh96EXDYn0vjAcAJk4DHRhenW3M/8MIFhTinvQj0ONC/zAzDMBFBRNOFEAPt4TnpAVjdQD28b4iK191pUC3BYKZrNS2RXvoMwzAZIScKwKfd3+oW6uYiqjWewAqBYZhskg8F4Hfg19wA2n5up0pVfLYKn3sEDMNklHwoANXkLzd0TUAqxVLi3cMKgGGYbJITBeDXBGRVAG49AJUC4B4AwzDlQT4UgN9KuEgBuIwBKHsW9rxYATAMk00yvxx0pFQ1db529B3A/Nfk+c/+Bjw1Vpp4Dvx9cbz9xgErvwbWLgW69jfue71wvc+Iwvl2ewNd+0UnP8MwTITkYx5A0phzBcp0jXCGYSqLfM8DYBiGYUpgBcAwDJNTWAEwDMPkFFYADMMwOYUVAMMwTE5hBcAwDJNTWAEwDMPkFFYADMMwOYUVAMMwTE5hBcAwDJNTWAEwDMPkFFYADMMwOYUVAMMwTE5hBcAwDJNT8rUfQFL8egrww2dpS8EwDOMKK4A46DZQfhiGYTIMm4AYhmFyCisAhmGYnMIKgGEYJqekogCIaDwRLSGiGcZnZBpyMAzD5Jk0B4H/KYS4IcX8GYZhcg2bgBiGYXJKmgpgHBHNJKL7iKidUyQiOouIaoiopra2Nkn5GIZhKhoSQsSTMNHrALZVXLoCwAcAlgMQAP4KoKsQ4gyvNAcOHChqamoilZNhGKbSIaLpQoiSyUmxKQBdiKgHgBeEEHtoxK0FsChgVh0hlU5WYfnCwfKFg+ULR9bl21EI0ckemMogMBF1FUJ8b3w9FsAsnftUP8BHnjUqDZgVWL5wsHzhYPnCkXX5nEjLC+h6IuoPaQJaCODslORgGIbJLakoACHEKWnkyzAMwxTIkxvohLQF8IDlCwfLFw6WLxxZl09J6oPADMMwTDrkqQfAMAzDWGAFwDAMk1NyoQCIaDgRzSWi+UR0WQr5dyeiN4nocyKaTUS/M8LbE9FrRDTPOLYzwomI/mXIO5OIBiQkZzURfUJELxjfexLRNEOOx4iomRHe3Pg+37jeIwHZ2hLRk0T0BRHNIaL9slR+RHSh8d/OIqJHiKhF2uVnzLJfRkSzLGG+y4yIxhjx5xHRmJjl+z/jP55JRM8QUVvLtT8a8s0lop9ZwmN5v1XyWa5dRESCiDoa3xMvv0gQQlT0B0A1gK8A7ASgGYBPAfRNWIauAAYY51sD+BJAXwDXA7jMCL8MwHXG+UgALwMgAEMATEtIzt8DeBhyYh4APA7gROP83wDONc5/A+DfxvmJAB5LQLaJAH5tnDcD0DYr5QdgewALALS0lNtpaZcfgIMADAAwyxLmq8wAtAfwtXFsZ5y3i1G+YQCaGOfXWeTra7y7zQH0NN7p6jjfb5V8Rnh3AK9CTkrtmFb5RfIb0xYg9h8I7AfgVcv3PwL4Y8oyPQfgCABzIZfBAKSSmGuc3wXgJEv8xngxytQNwBQAhwJ4wXiQl1texsZyNB7+/YzzJkY8ilG2NkYFS7bwTJQfpAL41njJmxjl97MslB+AHrYK1leZATgJwF2W8KJ4Uctnu3YsgEnGedF7a5Zh3O+3Sj4ATwLYC3IOk6kAUim/sJ88mIDMl9NksRGWCkZ3f28A0wB0EYUZ0T8A6GKcpyHzzQD+AKDB+N4BwI9CiDqFDI3yGddXG/HjoieAWgD3Gyaqe4hoK2Sk/IQQSwDcAOAbAN9Dlsd0ZKf8rPgtszTfnzMgW9VwkSNR+YjoaABLhBCf2i5lQj6/5EEBZAYiag3gKQAXCCHWWK8J2TxIxSeXiH4OYJkQYnoa+WvQBLIrfqcQYm8A6yDNF42kXH7tABwNqai2A7AVgOFpyOKHNMvMCyK6AkAdgElpy2JCRK0AXA7gz2nLEhV5UABLIG12Jt2MsEQhoqaQlf8kIcTTRvBSIupqXO8KYJkRnrTMBwAYRUQLATwKaQa6BUBbIjJni1tlaJTPuN4GwIoY5VsMYLEQYprx/UlIhZCV8jscwAIhRK0QYguApyHLNCvlZ8VvmSX+/hDRaQB+DmC0oaSyIl8vSCX/qfGudAPwMRFtmxH5fJMHBfARgN6GR0YzyEG3yUkKQEQE4F4Ac4QQN1kuTQZgegWMgRwbMMNPNTwLhgBYbem2R44Q4o9CiG5CiB6Q5fOGEGI0gDcB/NJBPlPuXxrxY2tJCiF+APAtEfUxgg4D8DkyUn6Qpp8hRNTK+K9N+TJRfjb8ltmrAIYRUTujpzPMCIsFIhoOaYocJYRYb5P7RMODqieA3gA+RILvtxDiMyFEZyFED+NdWQzp3PEDMlJ+vkl7ECKJD+QI/ZeQ3gJXpJD/gZBd7ZkAZhifkZB23ykA5gF4HUB7Iz4BuN2Q9zMAAxOU9WAUvIB2gnzJ5gN4AkBzI7yF8X2+cX2nBOTqD6DGKMNnIT0qMlN+AP4C4AvIlW0fgvRWSbX8ADwCOSaxBbKyGhukzCBt8fONz+kxyzcf0mZuvif/tsS/wpBvLoARlvBY3m+VfLbrC1EYBE68/KL48FIQDMMwOSUPJiCGYRhGASsAhmGYnMIKgGEYJqewAmAYhskprAAYhmFyCisApiwxVmK80fL9YiIaH0G6zYnodSKaQUQn2K49QEQLjGsziOg9j7S2I6InI5BpPBFdHDYdhrGT1qbwDBOWTQCOI6K/CyGWR5ju3gAghOjvcP0SIYRWpS6E+A6FiWAMkzm4B8CUK3WQ+7BeaL9ARD2I6A1jXfYpRLSDIk57InrWiPMBEfUjos4A/gNgX6OF30tHEKOF/hARvW+s+X6mRY5ZxvnuRPShke5MIupthP+e5B4Cs4joAkuaVxDRl0T0DoA+lvBeRPQKEU0noqlEtKsRfryRxqdE9LaPcmTyTNoz0fjDnyAfAGsBbAM5G7MNgIsBjDeuPQ9gjHF+BoBnFfffCuAq4/xQADOM84NhzIRW3PMA5LLU5ixVc6ni8ZDr0LcE0BFyJut2sCwlbOQ32jhvZsTdB3LW6FYAWgOYDdkDMcNbGb9xPoCLjXunAOhtnA+GXEYCRvztjfO2af8//CmPD5uAmLJFCLGGiB4EcD6ADZZL+wE4zjh/CHITFDsHAviFkc4bRNSBiLbRyNbJBPScEGIDgA1E9CaAQZBKwuR9AFcQUTcATwsh5hHRgQCeEUKsAwAiehrAUMie+TPCWAuHiCYbx9YA9gfwhFxyCIBccgIA3gXwABE9DrkYHcN4wiYgpty5GXINma1SlsO+pkrRdyHEwwBGQSqql4jo0AB5VEHuMdDf8tnNSP8cAFdCrjw5nYiS2l+AKWNYATBljRBiJeTWi2Mtwe9BrgoJAKMBTFXcOtW4BiI6GMByYdujwSdHk9wHuAOkGekj60Ui2gnA10KIf0GuwNnPkOEYYxXRrSB3wJoK4G0jvCURbQ3gKOO3rgGwgIiON9IkItrLOO8lhJgmhPgz5OY51iWIGUYJm4CYSuBGAOMs38+D3D3sEsjK8HTFPeMB3EdEMwGsR2GJZC/+j4iutHwfZBxnQi7/3BHAX4UQ31HxZu//C+AUItoCuRPX34QQK4noAcgVQQHgHiHEJwBARI9BjissQ7EyGQ3gTkOGppD7N3xqyNUbclXKKUYYw7jCq4EyTEiM+QdrhRA3pC0Lw/iBTUAMwzA5hXsADMMwOYV7AAzDMDmFFQDDMExOYQXAMAyTU1gBMAzD5BRWAAzDMDnl/wM2ORpRoZPfZAAAAABJRU5ErkJggg==\n",
      "text/plain": [
       "<Figure size 432x288 with 1 Axes>"
      ]
     },
     "metadata": {
      "needs_background": "light"
     },
     "output_type": "display_data"
    }
   ],
   "source": [
    "plt.plot(15)\n",
    "plt.plot(scores)\n",
    "plt.xlabel('No of Episodes')\n",
    "plt.ylabel('Score')\n",
    "plt.show()"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 20,
   "metadata": {},
   "outputs": [],
   "source": [
    "env.close()"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": null,
   "metadata": {},
   "outputs": [],
   "source": []
  }
 ],
 "metadata": {
  "@webio": {
   "lastCommId": null,
   "lastKernelId": null
  },
  "kernelspec": {
   "display_name": "Python 3",
   "language": "python",
   "name": "python3"
  },
  "language_info": {
   "codemirror_mode": {
    "name": "ipython",
    "version": 3
   },
   "file_extension": ".py",
   "mimetype": "text/x-python",
   "name": "python",
   "nbconvert_exporter": "python",
   "pygments_lexer": "ipython3",
   "version": "3.6.8"
  }
 },
 "nbformat": 4,
 "nbformat_minor": 4
}