Assignment_2.ipynb

{
  "cells": [
    {
      "cell_type": "markdown",
      "metadata": {
        "id": "ssLI7a7GKzgB"
      },
      "source": [
        "# CS4240 Deep Learning - Assignment 2\n",
        "\n",
        "*These lab assignments are new in the CS4240 Deep Learning course. We'd like to hear what you think!*\n",
        "\n",
        "*Please post any feedback you have on Brightspace. Thanks!*"
      ]
    },
    {
      "cell_type": "markdown",
      "metadata": {
        "id": "OL148V_8rxq5"
      },
      "source": [
        "To start working on the assignment in Colab, save a copy on your Google Drive (`File` $\\rightarrow$ `Save a copy in Drive`).\n",
        "\n",
        "To work on the assignments locally, configure your conda environment (see instructions on Brightspace) and download this assignment as an IPython Notebook (`File` $\\rightarrow$ `Download .ipynb`)."
      ]
    },
    {
      "cell_type": "markdown",
      "metadata": {
        "id": "bdx7o5N2S2Tn"
      },
      "source": [
        "## Introduction\n",
        "\n",
        "In this assignment you will be introduced to the backpropagation algorithm and you will train your first neural network. We will continue from the layers implemented in the previous assignment and define the backward passes to compute gradients. Finally, you will use your neural network to solve the XOR problem from the previous assignment, as well as to classify handwritten digits from the MNIST dataset.\n",
        "\n",
        "**Prerequisites:**\n",
        "* Completion of previous assignment.\n",
        "* Basic knowledge of Python and Numpy. Recommended tutorial for Python and Numpy [here](https://cs231n.github.io/python-numpy-tutorial/).\n",
        "* We recommend you to have a look at [this excellent tutorial on PyTorch Tensors](https://pytorch.org/tutorials/beginner/blitz/tensor_tutorial.html#sphx-glr-beginner-blitz-tensor-tutorial-py).\n",
        "\n",
        "**Learning objectives:**\n",
        "* Understanding, deriving and implementing the backward pass for a fully connected layer and the ReLU and Sigmoid non-linear activation functions in PyTorch;\n",
        "* Understanding and implementing the backpropagation algorithm and the training loop;\n",
        "* Understanding Softmax and the cross-entropy loss.\n",
        "\n",
        "We will share the solutions one week after the assignments are published. Throughout the assignment you will validate your code by comparing the outputs of your own implementations to the equivalent PyTorch implementations.\n",
        "\n",
        "**For your own implementations you may only use basic tensor operations from the `torch` module: no `torch.nn`, `torch.nn.F` or others.**\n",
        "\n",
        "---\n",
        "\n",
        "When answering coding questions make sure to write your own code within the designated part of the code block as illustrated here:\n",
        "```python \n",
        "#############################################################################\n",
        "#                       TODO: Implement function x                          #\n",
        "#############################################################################\n",
        "# Replace \"pass\" statement with your code\n",
        "pass\n",
        "#############################################################################\n",
        "#                            END OF YOUR CODE                               #\n",
        "#############################################################################\n",
        "```\n",
        "\n",
        "Please pay attention to the question boxes and try to think about them. The boxes are indicated as follows:\n",
        "\n",
        "****\n",
        "**Questions?**\n",
        "****"
      ]
    },
    {
      "cell_type": "code",
      "execution_count": 1,
      "metadata": {
        "id": "gIvAOzFjhtw6"
      },
      "outputs": [],
      "source": [
        "# Setup\n",
        "import torch\n",
        "import torch.nn as nn\n",
        "from torch.utils.data import DataLoader\n",
        "from torchvision import datasets, transforms\n",
        "import matplotlib.pyplot as plt"
      ]
    },
    {
      "cell_type": "markdown",
      "metadata": {
        "id": "IuIHuF2dMzSV"
      },
      "source": [
        "## A2.1 Linear layer - backward pass\n",
        "\n",
        "In A2.1 we will build upon the linear layer from A1.1 and implement the backward pass. We then perform a forward pass and backward pass on some dummy input and compare the gradients to the Pytorch implementation of a Linear layer.\n",
        "\n",
        "**Backpropagation**\n",
        "\n",
        "Backpropagation is an efficient algorithm to modularly calculate the gradients of the loss $\\mathcal{L}$ with respect to the trainable parameters in our network. The gradient is used in (Stochastic) Gradient Descent to update the trainable parameters to minimize the loss. For now we only consider how to calculate the gradients in the `backward` function of a layer, later on we will also implement the update step.\n",
        "\n",
        "The forward pass of a linear layer is defined as $y=xw+b$, where $w$ and $b$ are the trainable weight and bias parameters, respectively. The gradients w.r.t. $w$ and $b$ are then calculated using the chain rule as\n",
        "$$\\frac{\\partial \\mathcal{L}}{\\partial w} = \\frac{\\partial \\mathcal{L}}{\\partial y} \\frac{\\partial y}{\\partial w}, \\qquad \\frac{\\partial \\mathcal{L}}{\\partial b} = \\frac{\\partial \\mathcal{L}}{\\partial y} \\frac{\\partial y}{\\partial b}.$$\n",
        "\n",
        "Here, $\\frac{\\partial \\mathcal{L}}{\\partial y}$ is the *upstream gradient*, i.e. the gradient flowing from the deeper layer into the current layer. The upstream gradient will be the input variable in our `backward` function.\n",
        "The other two terms are the *local gradients* and will be computed inside the `backward` function as\n",
        "$$\\frac{\\partial y}{\\partial w}=x, \\qquad \\frac{\\partial y}{\\partial b}=1.$$\n",
        "\n",
        "As you can see, the input of the layer $x$ is used for computing the gradient in the backward pass. We therefore need to store this value during the forward pass as a cache variable so that we don't have to compute it again. \n",
        "\n",
        "Furthermore, since the shallower layer also requires an upstream gradient, our `backward` future needs to return a *downstream gradient*. The downstream gradient of a layer is simply its upstream gradient times the local gradient of its ouput with respect to its input: $\\frac{\\partial \\mathcal{L}}{\\partial x} = \\frac{\\partial \\mathcal{L}}{\\partial y} \\frac{\\partial y}{\\partial x}$. For a linear layer $\\frac{\\partial y}{\\partial x}=w$ and so $\\frac{\\partial \\mathcal{L}}{\\partial x} = \\frac{\\partial \\mathcal{L}}{\\partial y} w$.\n",
        "\n",
        "Let us further clarify using an example with a two-layer neural network. The network output is given by\n",
        "$$y_2 = y_1 w_2 + b_2 = (x_1 w_1 + b_1) w_2 + b_2.$$\n",
        "The gradients for the first layer are calculated using the chain rule:\n",
        "$$ \\frac{\\partial \\mathcal{L}}{\\partial w_1} = \\frac{\\partial \\mathcal{L}}{\\partial y_2} \\frac{\\partial y_2}{\\partial y_1} \\frac{\\partial y_1}{\\partial w_1} = \\frac{\\partial \\mathcal{L}}{\\partial y_1} \\frac{\\partial y_1}{\\partial w_1}.$$\n",
        "\n",
        "From the perspective of layer 2, $\\frac{\\partial \\mathcal{L}}{\\partial y_2}$ is the upstream gradient, $\\frac{\\partial y_2}{\\partial y_1} = \\frac{\\partial y_2}{\\partial x_2} = w_2$ is the local gradient, and $\\frac{\\partial \\mathcal{L}}{\\partial y_2} \\frac{\\partial y_2}{\\partial y_1} = \\frac{\\partial \\mathcal{L}}{\\partial y_1}$ is the downstream gradient. From the perspective of layer 1, $\\frac{\\partial \\mathcal{L}}{\\partial y_1}$ is the upstream gradient.\n",
        "\n",
        "You will now implement the `backward` function in the code block below."
      ]
    },
    {
      "cell_type": "code",
      "execution_count": 3,
      "metadata": {},
      "outputs": [
        {
          "data": {
            "text/plain": [
              "tensor([[0.4550, 0.3080],\n",
              "        [0.1982, 0.7886],\n",
              "        [0.2442, 0.3228]])"
            ]
          },
          "execution_count": 3,
          "metadata": {},
          "output_type": "execute_result"
        }
      ],
      "source": [
        "a = torch.Tensor(3, 2)\n",
        "torch.rand_like(a)\n"
      ]
    },
    {
      "cell_type": "code",
      "execution_count": 14,
      "metadata": {
        "id": "oWcqUhVgfR5A"
      },
      "outputs": [],
      "source": [
        "class Linear(object):\n",
        "    \"\"\"\n",
        "    Fully connected layer.\n",
        "    \n",
        "    Args:\n",
        "        in_features: number of input features\n",
        "        out_features: number of output features\n",
        "    \"\"\"\n",
        "\n",
        "    def __init__(self, in_features, out_features):\n",
        "        super(Linear, self).__init__()\n",
        "\n",
        "        # Define placeholder tensors for layer weight and bias. The placeholder\n",
        "        # tensors should have the correct dimension according to the in_features\n",
        "        # and out_features variables.\n",
        "        self.weight = torch.Tensor(in_features, out_features)\n",
        "        self.bias = torch.Tensor(out_features)\n",
        "\n",
        "        # Initialize parameters\n",
        "        self.init_params()\n",
        "\n",
        "        # NEW: Define a cache varible to save computation, because some of the\n",
        "        # forward pass values would be used during backward pass.\n",
        "        self.cache = None\n",
        "\n",
        "        # NEW: Define variables to store the gradients of the weight and bias\n",
        "        # calculated during the backward pass\n",
        "        self.weight_grad = None\n",
        "        self.bias_grad = None\n",
        "\n",
        "    def init_params(self, std=1.):\n",
        "        \"\"\"\n",
        "        Initialize layer parameters. Sample weight from Gaussian distribution\n",
        "        and bias uniform distribution.\n",
        "        \n",
        "        Args:\n",
        "            std: Standard deviation of Gaussian distribution (default: 1.0)\n",
        "        \"\"\"\n",
        "\n",
        "        self.weight = std*torch.randn_like(self.weight)\n",
        "        self.bias = torch.rand_like(self.bias)\n",
        "\n",
        "    def forward(self, x):\n",
        "        \"\"\"\n",
        "        Forward pass of linear layer: multiply input tensor by weights and add\n",
        "        bias. Store input tensor as cache variable.\n",
        "        \n",
        "        Args:\n",
        "            x: input tensor\n",
        "\n",
        "        Returns:\n",
        "            y: output tensor\n",
        "        \"\"\"\n",
        "\n",
        "        y = torch.mm(x, self.weight) + self.bias  # forward pass\n",
        "        \n",
        "        ########################################################################\n",
        "        #                TODO: Store input as cache variable                   #\n",
        "        ########################################################################\n",
        "\n",
        "        self.cache = x\n",
        "\n",
        "        ########################################################################\n",
        "        #                         END OF YOUR CODE                             #\n",
        "        ########################################################################\n",
        "\n",
        "        return y\n",
        "\n",
        "    def backward(self, dupstream):\n",
        "        \"\"\"\n",
        "        Backward pass of linear layer: calculate gradients of loss with respect\n",
        "        to weight and bias and return downstream gradient dx.\n",
        "        \n",
        "        Args:\n",
        "            dupstream: Gradient of loss with respect to output of this layer.\n",
        "\n",
        "        Returns:\n",
        "            dx: Gradient of loss with respect to input of this layer.\n",
        "        \"\"\"\n",
        "        ########################################################################\n",
        "        #                  TODO: Implement this function                       #\n",
        "        ########################################################################\n",
        "\n",
        "        x = self.cache\n",
        "        dx = torch.mm(dupstream, self.weight.T)\n",
        "        self.weight_grad = torch.mm(x.T, dupstream)\n",
        "        self.bias_grad = torch.sum(dupstream, dim=0)\n",
        "\n",
        "        ########################################################################\n",
        "        #                         END OF YOUR CODE                             #\n",
        "        ##############w##########################################################\n",
        "\n",
        "        return dx\n"
      ]
    },
    {
      "cell_type": "markdown",
      "metadata": {
        "id": "TtE7ytKSFBst"
      },
      "source": [
        "Now test the forward and backward pass of the layer on some dummy input.\n",
        "\n",
        "****\n",
        "**What will be the shape of gradient of x w.r.t. y?**\n",
        "****"
      ]
    },
    {
      "cell_type": "code",
      "execution_count": 15,
      "metadata": {
        "id": "bSALY1wPFGXZ"
      },
      "outputs": [
        {
          "name": "stdout",
          "output_type": "stream",
          "text": [
            "Shape of gradient x is: torch.Size([2, 3])\n",
            "Shape correct:  True\n"
          ]
        }
      ],
      "source": [
        "# Define layer dimensions\n",
        "n_samples, in_features, out_features = 2, 3, 4\n",
        "# Make random input tensor of dimensions [n_samples, in_features]\n",
        "x = torch.randn((n_samples, in_features))\n",
        "# Define upstream gradient dL/dy as randn\n",
        "dy = torch.randn((n_samples, out_features))\n",
        "\n",
        "# Create a layer from the Linear object class above\n",
        "layer = Linear(in_features, out_features)\n",
        "# Forward pass\n",
        "y = layer.forward(x)\n",
        "# Backward pass\n",
        "dx = layer.backward(dy)\n",
        "\n",
        "# What will be the shape of gradient of x w.r.t. y?\n",
        "print('Shape of gradient x is:', dx.shape)\n",
        "print('Shape correct: ', dx.shape == x.shape)"
      ]
    },
    {
      "cell_type": "markdown",
      "metadata": {
        "id": "8L5vbHyFIE9m"
      },
      "source": [
        "If the above code block has not returned any errors we can compare our implementation to the PyTorch linear layer [[docs](https://pytorch.org/docs/stable/generated/torch.nn.Linear.html#torch.nn.Linear)] from `torch.nn`. We do so by initializing a `nn.Linear` layer and setting the `weight` and `bias` to the same values as in our own linear layer.\n",
        "\n",
        "PyTorch layers store their parameters as a `Parameter`, which is a `Tensor` subclass with some special properties [[docs](https://pytorch.org/docs/stable/generated/torch.nn.parameter.Parameter.html)]. We therefore need to wrap our `layer.weight` and `layer.bias` in a `nn.Parameter` when using in `nn.Linear`. Moreover, the `weight` tensor is transposed in `nn.Linear`.\n",
        "\n",
        "PyTorch does not require the gradients to be specified for each layer individually but instead uses a mechanism called Autograd for automatic differentiation. To enable this for our dummy input we need to set the `requires_grad` parameter of the input tensor to `True`. More information about Autograd can be found in the official [docs](https://pytorch.org/docs/stable/notes/autograd.html) and in this excellent [tutorial](https://blog.paperspace.com/pytorch-101-understanding-graphs-and-automatic-differentiation/)."
      ]
    },
    {
      "cell_type": "code",
      "execution_count": 16,
      "metadata": {
        "id": "xa-_W-MgFJzN"
      },
      "outputs": [
        {
          "name": "stdout",
          "output_type": "stream",
          "text": [
            "tensor([[ 1.5787, -1.3974,  1.0676,  0.7066],\n",
            "        [ 1.7520,  3.3848,  1.6630, -0.5684]], grad_fn=<AddmmBackward0>)\n",
            "tensor([[ 1.5787, -1.3974,  1.0676,  0.7066],\n",
            "        [ 1.7520,  3.3848,  1.6630, -0.5684]], grad_fn=<AddmmBackward0>)\n",
            "Shape of gradient x is: torch.Size([2, 3])\n",
            "Shape correct: True\n"
          ]
        }
      ],
      "source": [
        "# Create Linear layer from torch.nn module\n",
        "torch_layer = nn.Linear(in_features, out_features)\n",
        "\n",
        "# Load the parameters from our layer into the Pytorch layer\n",
        "torch_layer.weight = nn.Parameter(layer.weight.T) # transpose weight by .t()\n",
        "torch_layer.bias = nn.Parameter(layer.bias)\n",
        "\n",
        "# We create a copy of the input tensor so we can use x for all other layers\n",
        "# without overwriting its gradients\n",
        "x_lin = x.clone()\n",
        "# Enable requires_grad for x_lin\n",
        "x_lin.requires_grad = True\n",
        "\n",
        "# Perform forward pass\n",
        "torch_y = torch_layer(x_lin)\n",
        "print(torch_y)\n",
        "# Perform bacward pass\n",
        "torch_y.backward(dy)\n",
        "print(torch_y)\n",
        "\n",
        "# What will be the shape of gradient of x w.r.t. y?\n",
        "print('Shape of gradient x is:', dx.shape)\n",
        "print('Shape correct:', x_lin.grad.shape == x_lin.shape)\n"
      ]
    },
    {
      "cell_type": "markdown",
      "metadata": {
        "id": "NjE7DIp9Ltut"
      },
      "source": [
        "We now compare the gradients of both implementations using `torch.allclose` [[docs](https://pytorch.org/docs/stable/generated/torch.allclose.html)], which returns `True` if all elements in both tensors are sufficiently \"close\" to each other.\n",
        "\n",
        "Your backward implementation of the linear layer is therefore correct if `True` is returned for all gradients."
      ]
    },
    {
      "cell_type": "code",
      "execution_count": 17,
      "metadata": {
        "id": "3DMC0nrYNkIf"
      },
      "outputs": [
        {
          "name": "stdout",
          "output_type": "stream",
          "text": [
            "dx identical:  True\n",
            "dw identical:  True\n",
            "db identical:  True\n"
          ]
        }
      ],
      "source": [
        "# Compare gradients of x, weight, bias w.r.t. y using torch.allclose\n",
        "dx_same = torch.allclose(dx, x_lin.grad)\n",
        "print('dx identical: ', dx_same)\n",
        "dw_same = torch.allclose(layer.weight_grad, torch_layer.weight.grad.T)\n",
        "print('dw identical: ', dw_same)\n",
        "db_same = torch.allclose(layer.bias_grad, torch_layer.bias.grad)\n",
        "print('db identical: ', db_same)"
      ]
    },
    {
      "cell_type": "markdown",
      "metadata": {
        "id": "N41y5ntsMw40"
      },
      "source": [
        "## A2.2 Non-linear activation functions - backward pass\n",
        "\n",
        "Similarly we implement the backward passes for both non-linearities. The non-linearities are defined as follows:\n",
        "\n",
        "$$\\text{ReLU}(x)=\\max(0,x)\\\\\n",
        "\\text{Sigmoid}(x) = \\sigma(x) = \\frac{1}{1+\\exp(-x)}$$\n",
        "\n",
        "The derivations of the derivatives are left for you as an exercise."
      ]
    },
    {
      "cell_type": "code",
      "execution_count": 22,
      "metadata": {
        "id": "rOphZSwhMqn6"
      },
      "outputs": [],
      "source": [
        "class ReLU(object):\n",
        "    \"\"\"\n",
        "    ReLU non-linear activation function.\n",
        "    \"\"\"\n",
        "\n",
        "    def __init__(self):\n",
        "        super(ReLU, self).__init__()\n",
        "\n",
        "        # NEW: Define a cache variable because some of the forward pass values\n",
        "        # would be used during backward pass.\n",
        "        self.cache = None\n",
        "\n",
        "    def forward(self, x):\n",
        "        \"\"\"\n",
        "        Forward pass of ReLU non-linear activation function: y=max(0,x). Store\n",
        "        input tensor as cache variable.\n",
        "        \n",
        "        Args:\n",
        "            x: input tensor\n",
        "\n",
        "        Returns:\n",
        "            y: output tensor\n",
        "        \"\"\"\n",
        "\n",
        "        y = torch.clamp(x, min=0)  # forward pass\n",
        "\n",
        "        ########################################################################\n",
        "        #                   TODO: Update cache variable.                       #\n",
        "        ########################################################################\n",
        "\n",
        "        self.cache = y\n",
        "\n",
        "        ########################################################################\n",
        "        #                         END OF YOUR CODE                             #\n",
        "        ########################################################################\n",
        "\n",
        "        return y\n",
        "\n",
        "    def backward(self, dupstream):\n",
        "        \"\"\"\n",
        "        Backward pass of ReLU non-linear activation function: return downstream\n",
        "        gradient dx.\n",
        "        \n",
        "        Args:\n",
        "            dupstream: Gradient of loss with respect to output of this layer.\n",
        "\n",
        "        Returns:\n",
        "            dx: Gradient of loss with respect to input of this layer.\n",
        "        \"\"\"\n",
        "\n",
        "        # Making sure that we don't modify the incoming upstream gradient\n",
        "        dupstream = dupstream.clone()\n",
        "\n",
        "        ########################################################################\n",
        "        #                  TODO: Implement this function                       #\n",
        "        ########################################################################\n",
        "        y = self.cache\n",
        "        dx = dupstream\n",
        "        dx[y==0] = 0\n",
        "\n",
        "        ########################################################################\n",
        "        #                         END OF YOUR CODE                             #\n",
        "        ########################################################################\n",
        "\n",
        "        return dx\n",
        "\n",
        "class Sigmoid(object):\n",
        "    \"\"\"\n",
        "    Sigmoid non-linear activation function.\n",
        "    \"\"\"\n",
        "\n",
        "    def __init__(self):\n",
        "        super(Sigmoid, self).__init__()\n",
        "\n",
        "        # NEW: Define a cache variable because some of the forward pass value\n",
        "        # would be used during backward pass.\n",
        "        self.cache = None\n",
        "    \n",
        "    def forward(self, x):\n",
        "        \"\"\"\n",
        "        Forward pass of Sigmoid non-linear activation function: y=1/(1+exp(-x)).\n",
        "        Store input tensor as cache variable.\n",
        "\n",
        "        Args:\n",
        "            x: input tensor\n",
        "\n",
        "        Returns:\n",
        "            y: output tensor\n",
        "        \"\"\"\n",
        "\n",
        "        y = 1.0 / (1.0 + torch.exp(-x))\n",
        "        \n",
        "        ########################################################################\n",
        "        #                  TODO: Implement this function                       #\n",
        "        ########################################################################\n",
        "\n",
        "        self.cache = y\n",
        "\n",
        "        ########################################################################\n",
        "        #                         END OF YOUR CODE                             #\n",
        "        ########################################################################\n",
        "\n",
        "        return y\n",
        "\n",
        "    def backward(self, dupstream):\n",
        "        \"\"\"\n",
        "        Backward pass of Sigmoid non-linear activation function: return\n",
        "        downstream gradient dx.\n",
        "        \n",
        "        Args:\n",
        "            dupstream: Gradient of loss with respect to output of this layer.\n",
        "\n",
        "        Returns:\n",
        "            dx: Gradient of loss with respect to input of this layer.\n",
        "        \"\"\"\n",
        "\n",
        "        ########################################################################\n",
        "        #                  TODO: Implement this function                       #\n",
        "        ########################################################################\n",
        "        y = self.cache\n",
        "        dx = dupstream * y * (1 - y)\n",
        "\n",
        "        ########################################################################\n",
        "        #                         END OF YOUR CODE                             #\n",
        "        ########################################################################\n",
        "\n",
        "        return dx"
      ]
    },
    {
      "cell_type": "markdown",
      "metadata": {
        "id": "FtOu5BCKQvky"
      },
      "source": [
        "Test the backward pass of both non-linearities using the dummy input and gradients from before.\n",
        "\n",
        "****\n",
        "**What will be the shapes of the output tensors?**\n",
        "****"
      ]
    },
    {
      "cell_type": "code",
      "execution_count": 23,
      "metadata": {
        "id": "i-sCWIhNOetw"
      },
      "outputs": [
        {
          "name": "stdout",
          "output_type": "stream",
          "text": [
            "dy: tensor([[ 0.6360,  0.2131, -0.7608],\n",
            "        [ 0.2550,  0.6353, -0.7345]])\n",
            "Output shapes from ReLU and Sigmoid:  torch.Size([2, 3]) torch.Size([2, 3])\n",
            "Shape of gradient x from ReLU correct: True\n",
            "Shape of gradient x from Sigmoid correct: True\n"
          ]
        }
      ],
      "source": [
        "# Define layer dimensions and dummy input\n",
        "n_samples, in_features = 2, 3\n",
        "# Make random input tensor of dimensions [n_samples, in_features]\n",
        "x = torch.randn((n_samples, in_features))\n",
        "\n",
        "# Define ReLU and Sigmoid layers\n",
        "relu = ReLU()\n",
        "sigmoid = Sigmoid()\n",
        "\n",
        "# Define upstream gradient dL/dy as ones\n",
        "dy = torch.randn((n_samples, in_features))\n",
        "print('dy:', dy)\n",
        "\n",
        "########################################################################\n",
        "#    TODO: Perform a forward and backward pass with the ReLU and       #\n",
        "#                          Sigmoid functions                           #\n",
        "########################################################################\n",
        "\n",
        "y_relu = relu.forward(x)\n",
        "dx_relu = relu.backward(dy)\n",
        "\n",
        "y_sigmoid = sigmoid.forward(x)\n",
        "dx_sigmoid = sigmoid.backward(dy)\n",
        "\n",
        "########################################################################\n",
        "#                         END OF YOUR CODE                             #\n",
        "########################################################################\n",
        "\n",
        "# What will be the shapes of gradient tensors dx_relu and dx_sigmoid?\n",
        "print('Output shapes from ReLU and Sigmoid: ', dx_relu.shape, dx_sigmoid.shape)\n",
        "print('Shape of gradient x from ReLU correct:', dx_relu.shape == x.shape)\n",
        "print('Shape of gradient x from Sigmoid correct:', dx_sigmoid.shape == x.shape)\n"
      ]
    },
    {
      "cell_type": "markdown",
      "metadata": {
        "id": "7hiL-1zwRh_G"
      },
      "source": [
        "Now perform a forward and backward pass with the ReLU and Sigmoid activation functions from `torch.nn` and compare the outputs to your implementation.\n",
        "\n",
        "A list of all available non-linearities in PyTorch can be found [[here](https://pytorch.org/docs/stable/nn.html#non-linear-activations-weighted-sum-nonlinearity)]."
      ]
    },
    {
      "cell_type": "code",
      "execution_count": 24,
      "metadata": {
        "id": "rCpv6e_bR6Dg"
      },
      "outputs": [
        {
          "name": "stdout",
          "output_type": "stream",
          "text": [
            "dy:  tensor([[ 0.6360,  0.2131, -0.7608],\n",
            "        [ 0.2550,  0.6353, -0.7345]])\n",
            "---\n",
            "dx_relu identical:  True\n",
            "dx_relu:\n",
            " tensor([[0., 0., 0.],\n",
            "        [0., 0., 0.]])\n",
            "pytorch x_relu.grad:\n",
            " tensor([[0., 0., 0.],\n",
            "        [0., 0., 0.]])\n",
            "---\n",
            "dx_sigmoid identical:  True\n",
            "dx_sigmoid:\n",
            " tensor([[ 0.1287,  0.0332, -0.1404],\n",
            "        [ 0.0402,  0.1568, -0.1495]])\n",
            "pytorch x_sigmoid.grad:\n",
            " tensor([[ 0.1287,  0.0332, -0.1404],\n",
            "        [ 0.0402,  0.1568, -0.1495]])\n"
          ]
        }
      ],
      "source": [
        "# We create a copy of the input tensor so we can use x for all other layers\n",
        "# without overwriting its gradients\n",
        "x_relu = x.clone()\n",
        "x_sigmoid = x.clone()\n",
        "# Enable requires_grad\n",
        "x_relu.requires_grad = True\n",
        "x_sigmoid.requires_grad = True\n",
        "\n",
        "print('dy: ', dy)\n",
        "print('---')\n",
        "\n",
        "# ReLU forward pass\n",
        "torch_relu = nn.ReLU()\n",
        "torch_y_relu = torch_relu(x_relu)\n",
        "torch_y_relu.backward(dy)\n",
        "\n",
        "# Sigmoid forward pass\n",
        "torch_sigmoid = nn.Sigmoid()\n",
        "torch_y_sigmoid = torch_sigmoid(x_sigmoid)\n",
        "torch_y_sigmoid.backward(dy)\n",
        "\n",
        "# Compare outputs using torch.allclose\n",
        "dx_relu_same = torch.allclose(dx_relu, x_relu.grad)\n",
        "print('dx_relu identical: ', dx_relu_same)\n",
        "\n",
        "print(f\"dx_relu:\\n {dx_relu}\")\n",
        "print(f\"pytorch x_relu.grad:\\n {x_relu.grad}\")\n",
        "\n",
        "print('---')\n",
        "\n",
        "dx_sigmoid_same = torch.allclose(dx_sigmoid, x_sigmoid.grad)\n",
        "print('dx_sigmoid identical: ', dx_sigmoid_same)\n",
        "\n",
        "print(f\"dx_sigmoid:\\n {dx_sigmoid}\")\n",
        "print(f\"pytorch x_sigmoid.grad:\\n {x_sigmoid.grad}\")"
      ]
    },
    {
      "cell_type": "markdown",
      "metadata": {
        "id": "X7bn6PyoNF8r"
      },
      "source": [
        "## A2.3 Network class - backward pass and update step\n",
        "\n",
        "We will now extend our neural network class with a `backward` function that calculates the gradients for all parameters in the network and a `optimizer_step` function that performs a parameter update step using gradient descent. After calculating the gradients the weights and biases are updated as\n",
        "$$w' = w - \\eta\\frac{\\partial L}{\\partial w}, \\qquad b' = b - \\eta\\frac{\\partial L}{\\partial b},$$\n",
        "where $w'$ and $b'$ are the new weights and biases and $\\eta$ is the gradient descent step size, also called the learning rate."
      ]
    },
    {
      "cell_type": "code",
      "execution_count": 25,
      "metadata": {
        "id": "vi8luwCMgobr"
      },
      "outputs": [],
      "source": [
        "class Net(object):\n",
        "    \"\"\"\n",
        "    Neural network object containing layers.\n",
        "    \n",
        "    Args:\n",
        "        layers: list of layers in neural network\n",
        "    \"\"\"\n",
        "    def __init__(self, layers):\n",
        "        self.layers = layers\n",
        "\n",
        "        # Initialize params\n",
        "        self.reset_params()\n",
        "\n",
        "    def reset_params(self, std=1.):\n",
        "        \"\"\"\n",
        "        Reset network parameters. Applies `init_params` to all layers with\n",
        "        learnable parameters.\n",
        "        \n",
        "        Args:\n",
        "            std: Standard deviation of Gaussian distribution (default: 0.1)\n",
        "        \"\"\"\n",
        "\n",
        "        for layer in self.layers:\n",
        "            if hasattr(layer, 'init_params'):\n",
        "                layer.init_params(std=std)\n",
        "    \n",
        "    def forward(self, x):\n",
        "        \"\"\"\n",
        "        Performs forward pass through all layers of the network.\n",
        "        \n",
        "        Args:\n",
        "            x: input tensor\n",
        "\n",
        "        Returns:\n",
        "            x: output tensor\n",
        "        \"\"\"\n",
        "\n",
        "        for layer in self.layers:\n",
        "            x = layer.forward(x)\n",
        "        \n",
        "        return x\n",
        "\n",
        "    def backward(self, dupstream):\n",
        "        \"\"\"\n",
        "        Performs backward pass through all layers of the network.\n",
        "        \n",
        "        Args:\n",
        "            dupstream: Gradient of loss with respect to output.\n",
        "        \"\"\"\n",
        "\n",
        "        ########################################################################\n",
        "        #                  TODO: Implement this function                       #\n",
        "        ########################################################################\n",
        "\n",
        "        dx = dupstream\n",
        "        for layer in reversed(self.layers):\n",
        "            dx = layer.backward(dx)\n",
        "\n",
        "        ########################################################################\n",
        "        #                         END OF YOUR CODE                             #\n",
        "        ########################################################################\n",
        "\n",
        "        return dx\n",
        "    \n",
        "    def optimizer_step(self, lr):\n",
        "        \"\"\"\n",
        "        Updates network weights by performing a step in the negative gradient\n",
        "        direction in each layer. The step size is determined by the learning\n",
        "        rate.\n",
        "        \n",
        "        Args:\n",
        "            lr: Learning rate to use for update step.\n",
        "        \"\"\"\n",
        "\n",
        "        ########################################################################\n",
        "        #                  TODO: Implement this function                       #\n",
        "        #    Hint: use `if hasattr(layer, 'weight')` to check if a layer has   #\n",
        "        #                       trainable parameters.                          #\n",
        "        ########################################################################\n",
        "        for layer in self.layers:\n",
        "            if hasattr(layer, 'weight'):\n",
        "                layer.weight -= lr * layer.weight_grad\n",
        "                layer.bias -= lr * layer.bias_grad\n",
        "\n",
        "        ########################################################################\n",
        "        #                         END OF YOUR CODE                             #\n",
        "        ########################################################################\n"
      ]
    },
    {
      "cell_type": "markdown",
      "metadata": {
        "id": "7xLhaGmqNdAw"
      },
      "source": [
        "We will create a simple 2-layer network with ReLU non-linearity and test the forward and backward pass using the same dummy input as before."
      ]
    },
    {
      "cell_type": "code",
      "execution_count": 32,
      "metadata": {
        "id": "5k4uPheAjLsY"
      },
      "outputs": [
        {
          "name": "stdout",
          "output_type": "stream",
          "text": [
            "Shape of gradient x: torch.Size([2, 3])\n",
            "tensor([[ 0.9535,  0.8796, -1.6943],\n",
            "        [-0.4227, -1.3689,  0.4933]])\n"
          ]
        }
      ],
      "source": [
        "# Define layer dimensions\n",
        "n_samples, in_features, hidden_dim, out_features = 2, 3, 5, 4\n",
        "# Make random input tensor of dimensions [n_samples, in_features]\n",
        "x = torch.randn((n_samples, in_features))\n",
        "\n",
        "\n",
        "# Define and initialize layers\n",
        "layers = [Linear(in_features, hidden_dim),\n",
        "          ReLU(),\n",
        "          Linear(hidden_dim, out_features)]\n",
        "\n",
        "# Initialize network\n",
        "net = Net(layers)\n",
        "\n",
        "# Do forward pass\n",
        "y = net.forward(x)\n",
        "\n",
        "# Gradient of y w.r.t. y is 1\n",
        "dy = torch.randn((n_samples, out_features))\n",
        "\n",
        "# Do backward pass\n",
        "dx = net.backward(dy)\n",
        "\n",
        "# What will be the shape of gradient x?\n",
        "print('Shape of gradient x:', dx.shape)\n",
        "print(dx)"
      ]
    },
    {
      "cell_type": "markdown",
      "metadata": {
        "id": "_LMryIa6V7Zd"
      },
      "source": [
        "We will now again create the same neural network in PyTorch. PyTorch uses the Autograd mechanism to calculate all gradients in the network, so there is no need to define a backward function."
      ]
    },
    {
      "cell_type": "code",
      "execution_count": 33,
      "metadata": {
        "id": "Mln6iWyrP1NF"
      },
      "outputs": [
        {
          "name": "stdout",
          "output_type": "stream",
          "text": [
            "TorchNet(\n",
            "  (layer1): Linear(in_features=3, out_features=5, bias=True)\n",
            "  (relu): ReLU()\n",
            "  (layer2): Linear(in_features=5, out_features=4, bias=True)\n",
            ")\n"
          ]
        }
      ],
      "source": [
        "class TorchNet(nn.Module):\n",
        "    \"\"\"\n",
        "    PyTorch neural network. Network layers are defined in __init__ and forward\n",
        "    pass implemented in forward.\n",
        "    \n",
        "    Args:\n",
        "        in_features: number of features in input layer\n",
        "        hidden_dim: number of features in hidden dimension\n",
        "        out_features: number of features in output layer\n",
        "    \"\"\"\n",
        "    \n",
        "    def __init__(self, in_features, hidden_dim, out_features):\n",
        "        super(TorchNet, self).__init__()\n",
        "\n",
        "        self.layer1 = nn.Linear(in_features, hidden_dim)\n",
        "        self.relu = nn.ReLU()\n",
        "        self.layer2 = nn.Linear(hidden_dim, out_features)\n",
        "\n",
        "    def forward(self, x):\n",
        "        x = self.layer1(x)\n",
        "        x = self.relu(x)\n",
        "        x = self.layer2(x)\n",
        "        return x\n",
        "\n",
        "# Initialize Pytorch network\n",
        "torch_net = TorchNet(in_features, hidden_dim, out_features)\n",
        "print(torch_net)"
      ]
    },
    {
      "cell_type": "markdown",
      "metadata": {
        "id": "jQAFYgOtXTFX"
      },
      "source": [
        "We will now again compare the gradients of the loss w.r.t. to the input tensor for both networks. Therefore we again need to load the weights from our network into the PyTorch network."
      ]
    },
    {
      "cell_type": "code",
      "execution_count": 34,
      "metadata": {
        "id": "dBYiJnRwQXqB"
      },
      "outputs": [
        {
          "name": "stdout",
          "output_type": "stream",
          "text": [
            "Shape of gradient x is correct: True\n",
            "Gradients identical:  True\n"
          ]
        }
      ],
      "source": [
        "# Load the parameters from our model into the Pytorch model\n",
        "torch_net.layer1.weight = nn.Parameter(net.layers[0].weight.T) # transpose weight by .T\n",
        "torch_net.layer1.bias = nn.Parameter(net.layers[0].bias)\n",
        "torch_net.layer2.weight = nn.Parameter(net.layers[2].weight.T) # transpose weight by .T\n",
        "torch_net.layer2.bias = nn.Parameter(net.layers[2].bias)\n",
        "\n",
        "# Make copy of x\n",
        "torch_x = x.clone()\n",
        "torch_x.requires_grad = True\n",
        "\n",
        "# Perform forward pass\n",
        "torch_y = torch_net(torch_x)\n",
        "\n",
        "# Perform backward pass\n",
        "torch_y.backward(dy)\n",
        "\n",
        "# What will be the shape of gradient x?\n",
        "print('Shape of gradient x is correct:', dx.shape == torch_x.grad.shape)\n",
        "\n",
        "# Compare gradients using torch.allclose\n",
        "dx_same = torch.allclose(dx, torch_x.grad)\n",
        "print('Gradients identical: ', dx_same)"
      ]
    },
    {
      "cell_type": "markdown",
      "metadata": {
        "id": "eKrDW462NvNQ"
      },
      "source": [
        "##A2.4 Revisiting the XOR problem\n",
        "\n",
        "We revisit the XOR problem for the previous assignment and solve it by training our network with the backpropagation algorithm.\n",
        "\n",
        "The XOR problem consists of 4 data points belonging to 2 classes which cannot be separated by a linear decision boundary. \n",
        "\n",
        "| x0   | x1   | y    |\n",
        "| ---- | ---- | ---- |\n",
        "| 0    | 0    | 0    |\n",
        "| 0    | 1    | 1    |\n",
        "| 1    | 0    | 1    |\n",
        "| 1    | 1    | 0    |\n",
        "\n",
        "The class labels `y` are [one-hot encoded](https://en.wikipedia.org/wiki/One-hot#Machine_learning_and_statistics), i.e. for each class a binary value indicates whether the sample belongs to it or not. For instance, given classes `1,2,3,4,5` the one-hot encoding of class `4` is given by `[0,0,0,1,0]`. One-hot encoding is a natural way to represent class labels in a classification task since a neural network outputs a class probability vector. `[0,0,0,1,0]` then simply corresponds to a 0% chance of the sample belonging the classes `1,2,3,5` and a 100% chance of it belonging to class `4`."
      ]
    },
    {
      "cell_type": "code",
      "execution_count": 35,
      "metadata": {
        "id": "lR9D37H6JytZ"
      },
      "outputs": [],
      "source": [
        "x_xor = torch.tensor([[0, 0], [0, 1], [1, 0], [1, 1]], dtype=torch.float32)\n",
        "y_xor = torch.tensor([[1, 0], [0, 1], [0, 1], [1, 0]]) # one-hot encoded"
      ]
    },
    {
      "cell_type": "code",
      "execution_count": 36,
      "metadata": {
        "colab": {
          "base_uri": "https://localhost:8080/",
          "height": 295
        },
        "id": "Ll4xMtgPKpSc",
        "outputId": "b1a09d63-afac-4894-e54d-2fde039bcab3"
      },
      "outputs": [
        {
          "data": {
            "image/png": "iVBORw0KGgoAAAANSUhEUgAAAYIAAAEWCAYAAABrDZDcAAAAOXRFWHRTb2Z0d2FyZQBNYXRwbG90bGliIHZlcnNpb24zLjUuMSwgaHR0cHM6Ly9tYXRwbG90bGliLm9yZy/YYfK9AAAACXBIWXMAAAsTAAALEwEAmpwYAAAXCElEQVR4nO3de5RdZZ3m8e+TcAkB5JYgyEVQQMU1olgGl4piM7QBVHQ1YwMK4qhINzg92qM4jpdptWfUtXq8AXKXYVAYR2nFFkWxEVRAU1EEgg2dBiURkATwwp0kv/njHFyHopJU4tlVVN7vZ61anL3ft/b+vaTWec6+nP2mqpAktWvGVBcgSZpaBoEkNc4gkKTGGQSS1DiDQJIaZxBIUuMMAmmSJTkgydI1tJ+b5GOTWZPaZhBoWkmyRZJfJjlqYN2WSW5Lcnh/eeckX0xyd5L7k/wkyavHbKf6bfcl+XWS/5Vk5mSPR3oyMAg0rVTVfcBxwGeSzO2v/iQwWlVfSbIt8EPgEeC5wBzgU8CXHguKAftU1RbAK4C/BP7jMGpMstEwtiNNFoNA005VfQf4JvDZJAcAbwBO6De/C7gPeGtV3VlVD1bVBcDfA/+QJONsbzHwI+D5q9tn/yjkvya5Mcm9Sb6QZFa/7YAkS5OclORO4AtJNk3y6SS3938+nWTTMdt8f5Ll/W2/cQ37fnWSa5P8NslVSZ43pq73JLmuf4RzdpKnJvlWkj8kuSzJNhP436qGGQSart4FHAB8BfgvVXVHf/1BwFeratWY/l8GdgX2GruhJM8G9gcWr2WfbwReBTyzv50PDLTtAGwLPJ3eEct/A15ML1z2AeaN038OsBPwZuCMJM8ap7Z9gXOAdwDbAacDF48Jlb/oj3sv4DXAt4D397c/A/hPaxmXGmcQaFqqqnuBRcBs4KKBpjnAHeP8yh0D7Y/5aZL7gV8A3wdOXctuT66qJVV1D70jjCMH2lYBH66qh6vqQXqh8ZGququqlgF/Bxw9Znsf7Pe/gt4RzhvG2efbgdOr6sdVtbKq/jfwML2Qecznquo3VfVr4AfAj6vqZ1X1MPCPwAvWMi41ziDQtJTkTcBuwGXAJwaalgM7jvMrOw60P2ZfYAt61wf2AzZfy26XDLz+FfC0geVlVfXQwPLT+n1W1//eqrp/De2PeTrwt/3TQr9N8ltglzF9fzPw+sFxlrcYfzhSj0GgaSfJ9vQuAL+d3imTNyR5eb/5MuAvkoz9234DvTfymwdXVs+XgauBD61l17sMvN4VuH1wU2P63k7vTXx1/bdJsvka2h+zBPj7qtp64Gd2/7qHNBQGgaajk4GvVdXl/WsD7wXO7J83/xTwFODsJDskmZXkSHrn7N9Tq3/u+seB45LssIb9ntC/NXVbeufg/+8a+l4AfCDJ3CRz6IXM+WP6/F2STZLsD7wa+H/jbOdM4Pgk+6Vn8ySHJtlyDfuW1olBoGklyeuAlwHveWxdVZ0FLAU+VFV399tnATcCdwPvBo6uqtW+cVfV9cAVg9sdx5eA7wC39H/W9KWvjwGjwHXA9cBPx/S/E7iX3lHAF4Hjq+pfxqlrlN6Rz8n9/ouBY9ewX2mdxYlppLVL8kvgbVV12VTXIg2bRwSS1DiDQJIa56khSWqcRwSS1Lhp93CsOXPm1G677TbVZUjStLJw4cLlVTV3vLZpFwS77bYbo6OjU12GJE0rSX61ujZPDUlS4wwCSWqcQSBJjTMIJKlxzQTBbbfdxpVXXsmyZcumuhRJWif3338/P/zhD7nxxhs72X5nQZDknCR3JblhNe1J8tkki/vT7O3bRR0PPPAAh7/6NezzrOfwzsMO55m77Mq7T3wnq1aNncBKkp58Tj/jTHbYcWcO/8u/4iUvPZAX7LsfS5YsWfsvroMujwjOBeavof1gYM/+z3HA57so4l0nnMjd/3wNZz/6NP7Hfdtw+oqd+M65X+SUk0/uYneSNDQ/+tGPeO97P8ieL/wMe+x7Ms996Rf53UPP49BXv55hPhWisyCoqiuBe9bQ5TDgvP7EINcAWycZb2ap9fbwww/zxQsu4G2PPIVN+/OUPCUzOfqhzfn8pz4zzF1J0tCdfMrpzNn5cGZv0ZvjKJnJjs84iiVLf8N11103tP1M5TWCnXj81H9L++ueIMlxSUaTjK7LOf4HH3yQWrWKpzDzceu3Z2Puvvfe9ShZkibPb+5cxiaznvq4dckMZm++PcuXL1/Nb627qQyCjLNu3GOdqjqjqkaqamTu3HG/IT2urbbaimfs+nQWPG5qWLiC+3jlKw+YeKWSNAUOOeTf8/vllz/uNNBDD9zBb+9ZzMjIyND2M5WPmFjK4+eA3Znx52xdb0n49Omf5w2vfR2/emQFz1y1MT/f+BF+MOtRfviJjw9zV5I0dO94x3GceeYX+OUNH2Or7Q/k4YeWcffSL/ORj/wdW2211dD2M5VHBBcDx/TvHnox8Lv+/LNDdeCBB/L9a65i4yPnc/kLd2G3dxzJwuuvY6+99hr2riRpqLbccksWLLiKdx5/ENtsehn/7plL+cqXz+Vv3/2fh7qfzuYjSHIBcAAwB/gN8GFgY4CqOi1J6M3DOh94AHhLf37WNRoZGSkfOidJ6ybJwqoa93xSZ6eGqurItbQXcEJX+5ckTUwz3yyWJI3PIJCkxhkEktQ4g0CSGmcQSFLjDAJJapxBIEmNMwgkqXEGgSQ1ziCQpMYZBJLUOINAkhpnEEhS4wwCSWqcQSBJjTMIJKlxBoEkNc4gkKTGGQSS1DiDQJIaZxBIUuMMAklqnEEgSY0zCCSpcQaBJDXOIJCkxhkEktQ4g0CSGmcQSFLjDAJJapxBIEmN6zQIksxPclOSxUneN077Vkm+keTnSRYleUuX9UiSnqizIEgyEzgFOBjYGzgyyd5jup0A3FhV+wAHAP+QZJOuapIkPVGXRwTzgMVVdUtVPQJcCBw2pk8BWyYJsAVwD7Ciw5okSWN0GQQ7AUsGlpf21w06GXgOcDtwPfA3VbVq7IaSHJdkNMnosmXLuqpXkprUZRBknHU1ZvlVwLXA04DnAycnecoTfqnqjKoaqaqRuXPnDrtOSWpal0GwFNhlYHlnep/8B70FuKh6FgO3As/usCZJ0hhdBsECYM8ku/cvAB8BXDymz23AgQBJngo8C7ilw5okSWNs1NWGq2pFkhOBS4GZwDlVtSjJ8f3204CPAucmuZ7eqaSTqmp5VzVJkp6osyAAqKpLgEvGrDtt4PXtwJ93WYMkac38ZrEkNc4gkKTGGQSS1DiDQJIaZxBIUuMMAklqnEEgSY0zCCSpcQaBJDXOIJCkxhkEktQ4g0CSGmcQSFLjDAJJapxBIEmNMwgkqXEGgSQ1ziCQpMYZBJLUOINAkhpnEEhS4wwCSWqcQSBJjTMIJKlxBoEkNc4gkKTGGQSS1DiDQJIaZxBIUuMMAklqXKdBkGR+kpuSLE7yvtX0OSDJtUkWJbmiy3okSU+0UVcbTjITOAU4CFgKLEhycVXdONBna+BUYH5V3ZZk+67qkSSNr8sjgnnA4qq6paoeAS4EDhvT5yjgoqq6DaCq7uqwHknSOLoMgp2AJQPLS/vrBu0FbJPk+0kWJjlmvA0lOS7JaJLRZcuWdVSuJLWpyyDIOOtqzPJGwAuBQ4FXAR9MstcTfqnqjKoaqaqRuXPnDr9SSWpYZ9cI6B0B7DKwvDNw+zh9llfV/cD9Sa4E9gFu7rAuSdKALo8IFgB7Jtk9ySbAEcDFY/p8Hdg/yUZJZgP7Ab/osCZJ0hidHRFU1YokJwKXAjOBc6pqUZLj++2nVdUvknwbuA5YBZxVVTd0VZMk6YlSNfa0/ZPbyMhIjY6OTnUZkjStJFlYVSPjtfnNYklqnEEgSY0zCCSpcQaBJDXOIJCkxhkEktQ4g0CSGmcQSFLjDAJJapxBIEmNMwgkqXHrFQRJDhp2IZKkqbG+RwRnD7UKSdKUWe1jqJOMnTvgj03Adt2UI0mabGuaj2B/4E3AfWPWh97E9JKkDcCaguAa4IGqumJsQ5KbuitJkjSZVhsEVXUwQJK9q+rGMc0f6rQqSdKkmcjF4i8nOSk9myX5HPA/uy5MkjQ5JhIE+wG7AFfRm5D+duClXRYlSZo8EwmCR4EHgc2AWcCtVbWq06okSZNmIkGwgF4QvAh4GXBkkq90WpUkadKs6a6hx7y1qkb7r+8EDktydIc1SZIm0VqPCAZCYHDd/+mmHEnSZPOhc5LUOINAkhpnEEhS4wwCSWqcQSBJjTMIJKlxBoEkNc4gkKTGdRoESeYnuSnJ4iTvW0O/FyVZmeTwLuuRJD1RZ0GQZCZwCnAwsDe9ZxTtvZp+nwAu7aoWSdLqdXlEMA9YXFW3VNUjwIXAYeP0eyfwVeCuDmuRJK1Gl0GwE7BkYHlpf90fJdkJeD1w2po2lOS4JKNJRpctWzb0QiWpZV0GQcZZV2OWPw2cVFUr17ShqjqjqkaqamTu3LnDqk+SxMQeQ72+ltKb2ewxO9Ob3WzQCHBhEoA5wCFJVlTV1zqsS5I0oMsgWADsmWR34NfAEcBRgx2qavfHXic5F/gnQ0CSJldnQVBVK5KcSO9uoJnAOVW1KMnx/fY1XheQJE2OLo8IqKpLgEvGrBs3AKrq2C5rkSSNz28WS1LjDAJJapxBIEmNMwgkqXEGgSQ1ziCQpMYZBJLUOINAkhpnEEhS4wwCSWqcQSBJjTMIJKlxBoEkNc4gkKTGGQSS1DiDQJIaZxBIUuMMAklqnEEgSY0zCCSpcQaBJDXOIJCkxhkEktQ4g0CSGmcQSFLjDAJJapxBIEmNMwgkqXEGgSQ1ziCQpMZ1GgRJ5ie5KcniJO8bp/2NSa7r/1yVZJ8u65EkPVFnQZBkJnAKcDCwN3Bkkr3HdLsVeEVVPQ/4KHBGV/VIksbX5RHBPGBxVd1SVY8AFwKHDXaoqquq6t7+4jXAzh3WI0kaR5dBsBOwZGB5aX/d6rwV+NZ4DUmOSzKaZHTZsmVDLFGS1GUQZJx1NW7H5JX0guCk8dqr6oyqGqmqkblz5w6xREnSRh1ueymwy8DyzsDtYzsleR5wFnBwVd3dYT2SpHF0eUSwANgzye5JNgGOAC4e7JBkV+Ai4OiqurnDWiRJq9HZEUFVrUhyInApMBM4p6oWJTm+334a8CFgO+DUJAArqmqkq5okSU+UqnFP2z9pjYyM1Ojo6FSXIUnTSpKFq/ug7TeLJalxBoEkNc4gkKTGGQSS1DiDQJIaZxBIUuMMAklqnEEgSY0zCCSpcQaBJDXOIJCkxhkEktQ4g0CSGmcQSFLjDAJJapxBIEmNMwgkqXEGgSQ1ziCQpMYZBJLUOINAkhpnEEhS4wwCSWqcQSBJjTMIJKlxBoEkNc4gkKTGGQSS1DiDQJIaZxBIUuM2muoCJsPdd9/Neeedx003/xvzXrQvRxxxBLNnz57qsiRprVauXMk3v/lNLvv2pWw3dw7HHHssu++++1D3kaoa6gYft/FkPvAZYCZwVlV9fEx7+u2HAA8Ax1bVT9e0zZGRkRodHZ1wDTfccAMvf8WBbL71C9h41h48fN/P2HSj5Vxz9Q/Yfvvt13VIkjRpHn30UV77qvncuuBaXvLADO7dOHx/xgN84Uvn87rXvW6dtpVkYVWNjNfW2RFBkpnAKcBBwFJgQZKLq+rGgW4HA3v2f/YDPt//79C87e0nsO1Ob2KHp7+2v+Y/cNu/fI4PfOC/c8YZpw5zV5I0VOeffz53/ORaPvnQdsycEVgJ+6+YxduOeTMHL7uLTTfddCj76fIawTxgcVXdUlWPABcCh43pcxhwXvVcA2ydZMdhFfCHP/yBn/10AdvvcvDj1s/d+fX849e+PqzdSFInLjr/S/z5g5swM/njumdlM3aYsQlXX3310PbTZRDsBCwZWF7aX7eufUhyXJLRJKPLli2bcAEzZ86EwKpVjz5u/cqVDw4tSSWpK5tuthkP8/jT91XFQ7WKWbNmDW0/XQZBxlk39oLERPpQVWdU1UhVjcydO3fCBcyePZuDDprPnbecz2PXQmrVCu761fm8+Zg3Tng7kjQVjjnubXxj9sPcVyv/uO5HdR8rZs9i3rx5Q9tPl3cNLQV2GVjeGbh9Pfr8Sc4681QOOOAg/nXhX7PZFnvw+3t+xr7Pfy4f+MD7h7kbSRq617zmNVxx7NH81dln86IZW3LPzFUsmbmSb/7Td5gxY3if4zu7ayjJRsDNwIHAr4EFwFFVtWigz6HAifTuGtoP+GxVrTHm1vWuIYBVq1bxve99j1tvvZV99tmHefPmkYx3MCJJTz6LFy/m8ssvZ9ttt+XQQw9dr9NCU3LXUFWtSHIicCm920fPqapFSY7vt58GXEIvBBbTu330LV3UMmPGDA466KAuNi1Jndtjjz3YY489Ott+p18oq6pL6L3ZD647beB1ASd0WYMkac18xIQkNc4gkKTGGQSS1DiDQJIa1+lD57qQZBnwq/X89TnA8iGWMx045jY45jb8KWN+elWN+43caRcEf4oko6u7j3ZD5Zjb4Jjb0NWYPTUkSY0zCCSpca0FwRlTXcAUcMxtcMxt6GTMTV0jkCQ9UWtHBJKkMQwCSWrcBhkESeYnuSnJ4iTvG6c9ST7bb78uyb5TUecwTWDMb+yP9bokVyXZZyrqHKa1jXmg34uSrExy+GTW14WJjDnJAUmuTbIoyRWTXeOwTeBve6sk30jy8/6YO3mK8WRJck6Su5LcsJr24b9/VdUG9UPvkdf/BjwD2AT4ObD3mD6HAN+iN0Pai4EfT3XdkzDmlwDb9F8f3MKYB/r9M72n4B4+1XVPwr/z1sCNwK795e2nuu5JGPP7gU/0X88F7gE2mera/4QxvxzYF7hhNe1Df//aEI8I5gGLq+qWqnoEuBA4bEyfw4DzqucaYOskO052oUO01jFX1VVVdW9/8Rp6s8FNZxP5dwZ4J/BV4K7JLK4jExnzUcBFVXUbQFVN93FPZMwFbJnebFNb0AuCFZNb5vBU1ZX0xrA6Q3//2hCDYCdgycDy0v66de0znazreN5K7xPFdLbWMSfZCXg9cBobhon8O+8FbJPk+0kWJjlm0qrrxkTGfDLwHHrT3F4P/E1VrZqc8qbE0N+/Op2YZoqMNwfl2HtkJ9JnOpnweJK8kl4QvKzTiro3kTF/GjipqlZuIFOTTmTMGwEvpDdF7GbA1Umuqaqbuy6uIxMZ86uAa4E/A54JfDfJD6rq9x3XNlWG/v61IQbBUmCXgeWd6X1SWNc+08mExpPkecBZwMFVdfck1daViYx5BLiwHwJzgEOSrKiqr01KhcM30b/t5VV1P3B/kiuBfejNHz4dTWTMbwE+Xr0T6IuT3Ao8G/jJ5JQ46Yb+/rUhnhpaAOyZZPckmwBHABeP6XMxcEz/6vuLgd9V1R2TXegQrXXMSXYFLgKOnsafDgetdcxVtXtV7VZVuwFfAf56GocATOxv++vA/kk2SjIb2A/4xSTXOUwTGfNt9I6ASPJU4FnALZNa5eQa+vvXBndEUFUrkpwIXErvjoNzqmpRkuP77afRu4PkEGAx8AC9TxTT1gTH/CFgO+DU/ifkFTWNn9w4wTFvUCYy5qr6RZJvA9cBq4Czqmrc2xCngwn+O38UODfJ9fROm5xUVdP28dRJLgAOAOYkWQp8GNgYunv/8hETktS4DfHUkCRpHRgEktQ4g0CSGmcQSFLjDAJJapxBIA1Rkjcn+df+z5unuh5pIrx9VBqSJNsCo/S+0VzAQuCFAw/7k56UPCKQ1kN/joPrksxKsnmSRcAJwHer6p7+m/93gflTW6m0dhvcN4ulyVBVC5JcDHyM3sPdzgceZcN6qq0a4RGBtP4+AhxE71TQJ9nwnmqrRhgE0vrblt5EKFsCs9jwnmqrRnixWFpP/VNDFwK7AzvSe7DfQnrTDAL8lN7F4jXNNiVNOa8RSOuhP/PXiqr6UpKZwFXA8+k9CXNBv9tHDAFNBx4RSFLjvEYgSY0zCCSpcQaBJDXOIJCkxhkEktQ4g0CSGmcQSFLj/j9LhMY02TQ2/QAAAABJRU5ErkJggg==",
            "text/plain": [
              "<Figure size 432x288 with 1 Axes>"
            ]
          },
          "metadata": {
            "needs_background": "light"
          },
          "output_type": "display_data"
        }
      ],
      "source": [
        "def plot(x, y, net=None):\n",
        "    \"\"\"\n",
        "    Plotter function for XOR dataset and classifier boundaries (optional).\n",
        "\n",
        "    Args:\n",
        "        x: Nx2 dimensional data\n",
        "        y: N dimensional labels\n",
        "        net: Model which has a forward function\n",
        "    \"\"\"\n",
        "    # Convert one-hot to class id\n",
        "    y = torch.argmax(y, dim=1)\n",
        "\n",
        "    # Plot decision boundary if net is given\n",
        "    if net:\n",
        "        h = 0.005\n",
        "        x_min, x_max = x[:, 0].min() - 1, x[:, 0].max() + 1\n",
        "        y_min, y_max = x[:, 1].min() - 1, x[:, 1].max() + 1\n",
        "\n",
        "        xx, yy = torch.meshgrid(torch.arange(x_min, x_max, h),\n",
        "                                torch.arange(y_min, y_max, h))\n",
        "        \n",
        "        in_tensor = torch.cat((xx.reshape((-1,1)), yy.reshape((-1,1))), dim=1)\n",
        "\n",
        "        z = net.forward(in_tensor)\n",
        "        z = torch.argmax(z, dim=1)\n",
        "        z = z.reshape(xx.shape)\n",
        "        plt.contourf(xx, yy, z, cmap=plt.cm.coolwarm)\n",
        "\n",
        "    # Plot data points\n",
        "    plt.scatter(x[:, 0], x[:, 1], c=y, cmap=plt.cm.coolwarm, edgecolors='k')\n",
        "    plt.title('XOR problem')\n",
        "    plt.xlabel('x0')\n",
        "    plt.ylabel('x1')\n",
        "    \n",
        "    plt.show()\n",
        "\n",
        "# Visualize\n",
        "plot(x_xor, y_xor)"
      ]
    },
    {
      "cell_type": "markdown",
      "metadata": {
        "id": "g1PAklq65pew"
      },
      "source": [
        "## A2.5 Revisiting the training loop\n",
        "\n",
        "We will now revisit the training loop and implement the backward pass and optimizer step to train the network from the training data. We therefore need to implement the gradient of the MSE loss with respect to the prediction $\\hat{y}$.\n",
        "\n",
        "Given prediction $\\hat{y}$ and groud-truth $y$ the MSE loss is defined as:\n",
        "$$\\text{MSE}(\\hat{y},y)= \\frac{1}{n} \\sum_{i=1}^n (\\hat{y}-y)^2 $$\n",
        "\n",
        "Again keep in mind that $\\hat{y}$ is a class probability vector and $y$ is a one-hot encoded representation."
      ]
    },
    {
      "cell_type": "code",
      "execution_count": 37,
      "metadata": {
        "id": "gLZE2-Pe0zkq"
      },
      "outputs": [],
      "source": [
        "def MSELoss(y_true, y_pred):\n",
        "    \"\"\"\n",
        "    Computes mean squared error loss.\n",
        "\n",
        "    Args:\n",
        "      y_true: Tensor containing true labels.\n",
        "      y_pred: Tensor containing predictions.\n",
        "\n",
        "    Return:\n",
        "      loss: Mean squared error loss.\n",
        "      grad: Gradient of loss w.r.t. y_pred.\n",
        "    \"\"\"\n",
        "    # Calculate mean squared error between y_true and y_pred\n",
        "    loss = torch.mean((y_pred - y_true) ** 2)\n",
        "\n",
        "    ########################################################################\n",
        "    #             TODO: Implement the gradient dL/dy_hat                   #\n",
        "    ########################################################################\n",
        "\n",
        "    grad = 2 *(y_pred - y_true)\n",
        "\n",
        "    ########################################################################\n",
        "    #                          END OF YOUR CODE                            #\n",
        "    ########################################################################\n",
        "\n",
        "    return loss, grad"
      ]
    },
    {
      "cell_type": "markdown",
      "metadata": {
        "id": "FZ8jint2Kbqk"
      },
      "source": [
        "You will now implement the training loop, which roughly speaking consists of the following steps:\n",
        "```\n",
        "while not converged:\n",
        " 1. feed training sample to network to predict output (forward step)\n",
        " 2. compare prediction to label (compute loss)\n",
        " 3. use comparison to update network parameters (backward and optimizer step)\n",
        "```"
      ]
    },
    {
      "cell_type": "code",
      "execution_count": 40,
      "metadata": {
        "id": "--1sxOkIQzA4"
      },
      "outputs": [
        {
          "name": "stdout",
          "output_type": "stream",
          "text": [
            "tensor([[ 0.8271, -0.0297],\n",
            "        [ 0.1786,  1.0272],\n",
            "        [ 0.1453,  1.0257],\n",
            "        [ 0.8407, -0.0310]])\n"
          ]
        },
        {
          "data": {
            "image/png": "iVBORw0KGgoAAAANSUhEUgAAA0AAAAEWCAYAAABPK/eBAAAAOXRFWHRTb2Z0d2FyZQBNYXRwbG90bGliIHZlcnNpb24zLjUuMSwgaHR0cHM6Ly9tYXRwbG90bGliLm9yZy/YYfK9AAAACXBIWXMAAAsTAAALEwEAmpwYAABG4ElEQVR4nO3dfZxcdX33/9d7ZndzT0IIBgiQBAwQpBIwclPv1lpboCJaUUGr3LTNRS+oFbVobdFfpT4uqkULgmJUEmkFL4t3eDUCagngDUoIAYGAJkAghHtyQ253d+bz++OcXYZlNzvJzsw5M/N+Ph77yM6cM+e8Z5M9mc/53ikiMDMzMzMzaweFrAOYmZmZmZk1igsgMzMzMzNrGy6AzMzMzMysbbgAMjMzMzOztuECyMzMzMzM2oYLIDMzMzMzaxsugMzMzMzMdoGkbklrd7J9saR/aWQmq54LIDMzMzNrGEkTJT0i6X0Vz02S9KikU9PH+0v6lqTnJG2R9BtJbxt0nEi3bZb0uKQvSCo2+v1Y83EBZGZmZmYNExGbgQXApZL2Tp/+HLAsIq6TNBX4OdADvAqYBnwRuKa/QKpwZERMBN4EvBc4uxYZJXXU4jiWTy6AzMzMzKyhIuIm4L+ByyR1A+8Bzk03nw9sBv4yIp6MiG0RcS3wWeASSRrieKuAXwDzhjtn2ur0D5Lul7Re0iJJY9Nt3ZLWSvq4pCeBRZLGSPp3SevSr3+XNGbQMT8p6dn02O/fybnfJmmFpA2Sfinp1YNy/b2ke9IWrW9Imi7px5JekPRTSXtW8WO1KrkAMjMzM7MsnA90A9cBH4uIJ9Ln3wp8NyLKg/b/DnAgcMjgA0k6DHgDsGqEc74f+FPg4PQ4/1SxbR9gKjCTpIXqH4HjSIqqI4Fjhth/GjADOANYKOnQIbIdDVwF/C9gL+CrwPWDiql3pe/7EOBk4MfAJ9PjF4APjfC+bBe4ADIzMzOzhouI9cB9wHjgexWbpgFPDPGSJyq291suaQuwElgKfHmE014eEY9FxPMkLUqnV2wrA5+OiB0RsY2kWPpMRDwdEc8A/wx8YNDxLkz3v4WkRes9Q5zzr4GvRsSvI6IUEd8EdpAUV/2+FBFPRcTjwG3AryPirojYAXwfOGqE92W7wAWQmZmZmTWcpL8AZgE/Bf61YtOzwL5DvGTfiu39jgYmkoz/ORaYMMJpH6v4fg2wX8XjZyJie8Xj/dJ9htt/fURs2cn2fjOBj6bd3zZI2gAcMGjfpyq+3zbE44lDvx3bHS6AzMzMzKyhJL2CZGKDvybpGvYeSW9MN/8UeJekwZ9T30NSwPyu8slIfAf4FfCpEU59QMX3BwLrKg81aN91JMXLcPvvKWnCTrb3ewz4bERMqfgan45rsgy4ADIzMzOzRrsc+EFE3JyO/bkA+Fo6LuaLwB7ANyTtI2mspNNJxuT8fUQMLlT6XQwskLTPTs57bjrF9lSSMTb/dyf7Xgv8k6S9JU0jKa7+c9A+/yypS9IbgLcB/zXEcb4GnCPpWCUmSPozSZN2cm6rIxdAZmZmZtYwkt4BvB74+/7nIuLrwFrgUxHxXLp9LHA/8BzwEeADETFswRIRvwVuqTzuEK4BbgIeSr92tljpvwDLgHuA3wLLB+3/JLCepNXnW8A5EfHAELmWkbR0XZ7uvwo4cyfntTrT8EW0mZmZmVlrkPQI8FcR8dOss1i23AJkZmZmZmZtwwWQmZmZmZm1DXeBMzMzMzOztuEWIDMzMzMzaxsdWQfYVdOmTYtZs2ZVte+WLVuYMGGk9bCy55y15Zy1N1LWO++889mI2LuBkWqi2utJK/1d5YVz1lYr5WzW68mUrs7Yb+yYrGPYKPTufyjPPtuTdQyrkQ3P3DvstaTpCqBZs2axbNmyqvZdunQp3d3d9Q1UA85ZW85ZeyNllbRm2I05Vu31pJX+rvLCOWurlXI26/Vkv7Fj+M/Xzss6ho3CRXMXZR3BaugHVxwy7LXEXeDMzMzMzKxtuAAyMzMzs7Z28AlHZR3BGsgFkJmZmZm1tTPXfCjrCNZALoDMzMzMzKxtuAAyMzMzs7Y198Jzs45gDeYCyMxyT9JVkp6WdO8w2yXpMkmrJN0j6ehGZzQzs+b0vuvmZx3BGswFkJk1g8XACTvZfiIwJ/1aAHylAZnMzMysCTXdOkDV2LS9l6/f+hBTt5WyjmJmNRARt0qatZNdTgGujogAbpc0RdK+EfFEYxLumm09JRb/8hG29fTV/NiPrOlhec+DNT9urTlnbeU+p8RH3npI1inMzIAWLYC27ihx2f+s4sxXdWUdxcwaYwbwWMXjtelzLyuAJC0gaSVi+vTpLF26dMSDb968uar9qnXPM3184c4dSZ6aHbVfwOpVNT9q7TlnbeU7pwRHd66r+e+S2Wj96Kyb4fZHso5hDVa3AkjSAcDVwD5AGVgYEZcO2qcb+CHwcPrU9yLiM6M9d0cx+UjRVx7tkcysSQxVR8RQO0bEQmAhwPz582OkVemhutXrd0Xv/U/Bncv4f3/7eo6YMblmx4XaZ60X56wt5zTbPStc/LSlerYA9QEfjYjlkiYBd0r6SUTcP2i/2yLibbU8cWchGdpUGvLjj5m1oLXAARWP9wfWZZRlRKVycnFS7Zt/zMysSgefcBSsyTqFZaFukyBExBMRsTz9/gVgJUmXlLrrbwEqhSsgszZxPfDBdDa444CNeR3/A1BOr03FgisgMzOzRmvIGKB08PJRwK+H2Hy8pLtJ7tZ+LCLuG+L1u9Rnvze9u7pte09T9DVulj7RzllbzZITss8q6VqgG5gmaS3waaATICKuBJYAJwGrgK3AWdkkrU5/C1DRTUBmZpk5c82Hso5gGal7ASRpIvBd4MMRsWnQ5uXAzIjYLOkk4Ack09i+xK722S+VA25aQkdnV1P0NW6WPtHOWVvNkhOyzxoRp4+wPYCmWcmuvwVILoDMzMwarq7rAEnqJCl+vhUR3xu8PSI2RcTm9PslQKekaaM9b7EgJOhzDzgzyyF3gTMzy9aUhd/MOoJlqG4FkJJbm98AVkbEF4bZZ590PyQdk+Z5rhbn7ywUKHkWODPLof5rk7vAmZll46Nf9IfEdlbPLnCvAz4A/FbSivS5TwIHwkC//VOBv5HUB2wDTku7soxaR1GeBMHMcqm/BahQ1zZ4MzOz9nThyrP4wU62160AioifM8IafxFxOXB5Pc7fUZBbgMwsl8rpJAgFtwCZmZnV1IUrR54HqWXvP3YWC14HyMxyqeQxQGZmmblo7qKsI1idVFP8QAsXQEkXuKxTmJm9nFuAzMzMaqva4gdauQDyJAhmllNp/eMWIDMzsxrYleIHWrgA6vQkCGaWU6WBFqCMg5iZtZnHP7c06whWY7ta/EALF0AdxQJ9bgEysxx6cRY4V0BmZo101aKHs45gNbQ7xQ+0cgFU8BggM8un/hYgrwNkZma2e3a3+IFWLoCKGuhnb2aWJ/3XJk+CYGbWOO7+1jpGU/xAKxdAngTBzHLKC6GamTWeu7+1hmtOXTbqY9RtIdSsdRbFDk+CYGY55C5wZmZmu27xzMtYedFdoz5OyxZAHQUvhGpm+VT2QqhmZg3lxU+b34Urz2L1ytocq2U7YHQU5VngzCyX+hdClVuAzMzMRjTaMT+DtWwB1Fl0C5CZ5VMpwq0/ZmYNMmXhN7OOYKNQ6+IHWrgA6ihooJ+9mVmelMoe/2Nm1igf/aK7BDWrs8+aXZfjtmwB5BYgM8uriPAMcGYZkXSVpKcl3TvM9m5JGyWtSL8+1eiMZgaXnF9gxgXddTl2y/4X3FH0Qqhmlk+lcngNILPsLAZOGGGf2yJiXvr1mQZksjrx2j/Nad5xs9iw4Iy6Hb9lC6BiQV4HyMxyqRThLnBmGYmIW4Hns85hjeG1f5rTyYveXNfjt2wB1OlpsM0sp8rloOBJEMzy7HhJd0v6saRXDbeTpAWSlklatr6nr5H5rApzLzw36wi2G+ox6cFgLVsAJV3gXAGZWf6Uw2sAmeXYcmBmRBwJfAn4wXA7RsTCiJgfEfP37GrZpRWb1vuum591BNtFjSh+oIULoM5iwV3gzCyXShG4/jHLp4jYFBGb0++XAJ2SpmUcy6zlLZ55WcPO1bIFUEfBkyCYWT6VPQmCWW5J2kfpKsWSjiH5rPRctqlsVx18wlFZR7BdtPqGuxp2rpZtr+1wC5CZ5VSp7IVQzbIi6VqgG5gmaS3waaATICKuBE4F/kZSH7ANOC3CfeqbzZlrPpR1BKvSvONm1X3Sg8FatgDq9DTYZpZT5cAtQGYZiYjTR9h+OXB5g+KYtb1GFz/Q0l3gCgTJnVYzszwpeyFUMzOzhk16MFjL/hfcUUzurva6H5yZ5Uyp7HWAzMzq5aK5i7KOYFXIqviBFi6AOtMCqM8tQGaWM0kLkAsgMzNrT2efNTvT87dsAVRM+5f0uQXIzHKmHJ4FzsysHqYs/GbWEWwEl5xfYMYF3ZlmaNkCqHOgC5xbgMwsX9wFzsysPj76Rd/4zrsNC87IOkLrFkAdaQuQJ0Ewaw2STpD0oKRVkj4xxPY9JX1f0j2SfiPpiCxyVqNUxl3gzMys7WQ57qdS6xZAngTBrGVIKgJXACcChwOnSzp80G6fBFZExKuBDwKXNjZl9SKCYstefc3MsvGjs27OOoLtRF6KH2jhAsiTIJi1lGOAVRHxUET0AN8GThm0z+HAzwAi4gFglqTpjY1ZnZLHAJmZ1dyK2x/JOoIN45pTl2Ud4SXqthCqpAOAq4F9gDKwMCIuHbSPSO7SngRsBc6MiOW1OH+HJ0EwayUzgMcqHq8Fjh20z93AnwM/l3QMMBPYH3iqcidJC4AFANOnT2fp0qUjnnzz5s1V7VetZ5/dzta+qOkx+9U6a704Z205p7W7g084CtZkncKGcvZZs1l5QX5af6COBRDQB3w0IpZLmgTcKeknEXF/xT4nAnPSr2OBr/DyDzW7xZMgmLWUoZpLBv9yXwxcKmkF8FvgLpLr0EtfFLEQWAgwf/786O7uHvHkS5cupZr9qvW1VbczprdMd/cf1uyY/WqdtV6cs7ac09rdmWs+lHUEG0bWM74NpW4FUEQ8ATyRfv+CpJUkd3ErC6BTgKsjIoDbJU2RtG/62lEZaAEquwXIrAWsBQ6oeLw/sK5yh4jYBJwFA63LD6dfuVMu41ngzMys5eVp3E+lerYADZA0CzgK+PWgTUN1a5lBWjhVvH6Xu6zc/0xy4/c3d9zJ86uKuxe8QZqlS4Bz1laz5IRcZL0DmCNpNvA4cBrwvsodJE0BtqZjhP4KuDUtinKnFIHrHzOz2nj8c0thUS7vd7W1vBY/0IACSNJE4LvAh4f4MFJNt5bd6rLSuepZuPPX/MGR8zj2oL12OXcjNUuXAOesrWbJCdlnjYg+SecBNwJF4KqIuE/SOen2K4G5wNWSSiQtzX+ZWeARlMtBV0fLzkFjZtZQV7n4yZ08Fz9Q5wJIUidJ8fOtiPjeELuM2K1ld3UUPAucWSuJiCXAkkHPXVnx/a9IxhPmXimCotcBMjOzFpT34gfqOA122gf/G8DKiPjCMLtdD3xQieOAjbUY/wNeB8jM8qscIPeBMzMbtYvmLso6glU4+6zZWUeoSj1bgF4HfAD4bTorEyQLFR4IA3dul5BMgb2KZBrsmpWM/ZMglNwCZGY5Uy4HRdc/ZmbWYvI449tQ6jkL3M8ZeoxP5T4BnFuP83d4Gmwzy6lS2V3gzMxGy2v/5EszdH3r17KjcDuLngbbzPKpHEHBXeDMzEbFa//kRzMVP9DCBdDAJAhuATKznHEBZGZmreKS85uvnGi+xFXqbwHyJAhmljfuAmdmZq3gmlOXsWHBGVnH2GUtWwD1jwHyNNhmljflgIILIDOz3fajs27OOoIBKy+6IusIu6V1C6B0Frg+twCZWc6Uw7PAmZmNxorbH8k6QttrtnE/lVq2AOr0LHBmllOlsscAmZntroNPOCrrCG2vmYsfaOECqMOzwJlZTpXL4S5wZma7ybO/ZWvxzMuyjjBq9VwINVP9s8C5BcjM8qYUQdEtQGZm1mQuXHkWq1dmnWL0WrcFKC2ASp4EwcxyxpMgmJlZs5l33KysI9RMyxZAxYF1gNwFzszypVwOXP+Yme26xz+3NOsIbevkRW/OOkLNtGwXOEkUBb1uATKznCmF1wEyy4qkq4C3AU9HxBFDbBdwKXASsBU4MyKWNzZl67rxqWf41iOP88yOHg7fYxILDj6AQydNrPr1Vy16uI7p8qfUt53fLf8aax/8EeVyL/se9Mcc9tpz6Ro7paE5mn3Sg8FatgUIoFhwC5CZ5Y9ngTPL1GLghJ1sPxGYk34tAL7SgExt4dpH1/HlBx7h1G2T+T+xP4duKPI3y+/j95u3ZB0tlyKC25ecy7OPreCVr/4HDnvNv7B1w/Pc9v2/oNS3o2E5Wq34gVYvgORJEMwsfyJwAWSWkYi4FXh+J7ucAlwdiduBKZL2bUy61tVTLvONhx/jH9mP1xYmMl2dnFzYk3fFFBY/tLaqY7Rb97fnn7yLzesf4dCjP83EKYcyftJsDvqDj1AsTmLd6hsakqEVix9o8QKoQ54G28zyp1QOii199TVrajOAxyoer02fs1F4avsOxiAOUNdLnj9aE3jghc1VHaPdur9tfPZ+Ju91NCq8OGJFEpOnvZYNz9xX9/Nfcn7r/kfVuu+MZCKEPrcAmVnOlMLrAJnl2FC/nEN+mJC0QNIyScvW9/TVOVZz26uri81RZkO89Of0cOxgv3FjM0qVb+MnzWDLC6te9vzWF1Yzfo8D6n7+DQvOqPs5stLaBZC7wJlZDoXXATLLs7VA5afL/YF1Q+0YEQsjYn5EzN+zq2XnlaqJ8R1FTt7nFXyRp3g2eokI7o9tXM1zvH/WyA1scy88twEp8+UVB76BKG/n0QcXUSptp1zu48k117PpuRUccMjb63ruVu361q+lf1uLBXeBM7P88SQIZrl2PXCepG8DxwIbI+KJjDO1hL87ZBaXFx7hvHWPUgAmdnTw4VfO5ripU0Z87fuum1/3fHlTKHTwulMWseLmT3PHT94JiMnTDud1pyyia+zkup231YsfaPECqCDcBc7MciUivBCqWYYkXQt0A9MkrQU+DXQCRMSVwBKSKbBXkUyD3fqfBhuks1Dg/EMO4txXzmJLX4nJnR2+GTSCcRP34fiTv0pvz2aiXKpr4QPtUfxAixdAHYJeT4NtZjnSvzSZu8CZZSMiTh9hewDt19+qgboKBbq6WnoURs11dlW/VtLuWjzzMlavrPtpcqGl//UVC6LkhVDNckXSnpJeJekgSS19DRpKOZJrkmeBMzOrzo/OujnrCG1h9Q13ZR2hYVq6Bago6HUBZJY5SZNJ7qieDnQBzwBjgemSbge+HBFt8T9c/00ZuQXIzKwqK25/JOsILW3ecbM4edGbs47RUC1fAPW5C5xZHlwHXA28ISI2VG6Q9BrgA5IOiohvZBGukV5sAXIBZGY2koNPOArWZJ2itbVb8QOtXgAVPAmCWR5ExFt3su1O4M4GxslUfwuQxwCZmY3szDUfyjpCS2uXSQ8Ga+0CSNDrabDNckfS3sDfAeOAr0TEy1d6a1H9vXJd/5iZWZbatfiBNpgEwS1AZrl0CXArcANwbcZZGqpcdhc4MzPL1tlnzc46QqZauwDyNNhmuSDpBklvqHiqC3gk/RqTRaaslDwGyMysKo9/bmnWEVrS2WfNZsYF3VnHyFTLF0B9ngXOLA/eC5wi6RpJBwMXAp8CLgb+d6bJGqx/EgQv/mdmtnNXLXo46wgtqd2LH2iDMUB9fW4BMstaRGwEPibpIOCzwOPAuenzVZF0AnApUAS+HhEXD9o+GfhP4ECSa9u/RcSiGr2FmukflugCyMzMGq2dx/1Uau0WoILo9Rggs8yli55+Hvgr4KPAD4HvSPpbScUqXl8ErgBOBA4HTpd0+KDdzgXuj4gjgW7gEkldNXwbNVHyQqhmZiNy97fac/Hzopb+L7ioF6ecNbNMXUsy4cHtwH9ExG0R8afAJuCmKl5/DLAqIh6KiB7g28Apg/YJYJKSFUYnAs8DfbV6A7XSPwmCW4DMzIbn7m+1dc2py7KOkCt16wIn6SrgbcDTEXHEENu7Se4C9/8L/15EfKaWGYoF6PM02GZ5MJbkd30CML7/yYj4pqTvVPH6GcBjFY/XAscO2udy4HpgHTAJeG9EvOwCIGkBsABg+vTpLF26dMSTb968uar9qvH01iTS7x58gKWbV9fkmJVqmbWenLO2nNPMdmblRVdkHSFX6jkGaDHJB5Krd7LPbRHxtnoFSGaBcwuQWQ78b+DzQA9wTuWGiNhWxeuHai4Z/Mv9p8AK4I+Ag4GfSLotIjYNOt9CYCHA/Pnzo7u7e8STL126lGr2q8ZDz2yGW2/hVYcfTvdRM2pyzEq1zFpPzllbzmmtZO6F58J1WadoHe769nJ1K4Ai4lZJs+p1/Gp0CPo8DbZZ5iLiF8AvRnGItcABFY/3J2npqXQWcHFEBLBK0sPAYcBvRnHemhuYBc7TYJuZDel9183POkLLcPEztKxngTte0t0kH2Q+FhH3DbXT7nRZASj19bKjT7lvbm+WLgHOWVvNkhNGn1XSj4CvAjdGRO+gbQcBZwKPRMRVwxziDmCOpNkkM8idBrxv0D6PAm8BbpM0HTgUeGi3Q9dJ/z2ZoscAmZlZHbn4GV6WBdByYGZEbJZ0EvADYM5QO+5OlxWA7/7uJsrRm/vm9mbpEuCctdUsOaEmWf8a+AhwqaTngWdIxgXNAlYDl0fED4d7cUT0SToPuJFkGuyrIuI+Seek268ELgIWS/otSZe5j0fEs6MJXQ9lzwJnZmZ15uJn5zIrgCr75UfEEklfljStlh9YigUoRzLrkrubmGUnIp4ELgAuSLvG7gtsA34XEVurPMYSYMmg566s+H4d8Ce1ylwv/TNTyi1AZmYv86OzbobbH8k6RlO75PwCGxZknSLfMiuAJO0DPBURIekYkim5n6vlOYrp54vecpkxhRGXGjGzBoiIR4BHMo6RmYEWIBdAZmYvs8LFz6icfdZsNizozjpG7tVzGuxrSRYjnCZpLfBpoBMG7tqeCvyNpD6SO8GnpYOXa6a/i0lfKRiT9WgnMzNebAEqulXazOwlDj7hKFiTdYrmNuOC7qwjNIV6zgJ3+gjbLyeZJrtu+u+w9nkqbDPLif61md0AZGb2Umeu+VDWEZqax/1Ur6WH4VZ2gTOz7El6m6SWvu6M5MVJEFwBmZlZbbj42TUt/UGkvwDq73JiZpk7Dfi9pM9Jmpt1mCwMdIFzE5CZmdXAJee39Mf5umjpn1j/GKBeL4ZqlgsR8RfAUSRTXy+S9CtJCyRNyjhaw3ghVDOzl5uy8JtZR2hK15y6jA0Lzsg6RtOpqgCSNKG/24qkQyS9XVJnfaONXn8LkMcAmeVHOgX+d4Fvk0yH/U5guaS/zTRYg/T3yC24BcjMbMBHv+ib1btj5UVXZB2hKVXbAnQrMFbSDOBnwFnA4nqFqpX+PvZ9HgNklguSTpb0feB/SGaFPCYiTgSOBD6WabgGKXkhVDMzqwGP+9l91f4XrHSxwj8HvhQR7wQOr1+s2hiYBMEtQGZ58W7gixHx6oj4fEQ8DZBeX87ONlpjlNMxQG4BMjNLPP65pVlHaDoufkan6gJI0vHA+4H/Tp/L/co6HRXrAJlZLnwa+E3/A0njJM0CiIifZRWqkTwLnJnZS1216OGsIzQVFz+jV20B9GHgH4DvR8R9kg4Cbq5bqhrxNNhmufNfQOUvZCl9rm2U3AJkZma7ycVPbVRVAEXELRHx9oj413QyhGcjIverVXkhVLPc6YiInv4H6fddGeZpuIFZ4FwAme0ySW+twTFOkPSgpFWSPjHE9m5JGyWtSL8+Ndpz2vDmXnhu1hGaxrzjZmUdoWVUOwvcNZL2kDQBuB94UNLf1zfa6BUGZoFzC5BZTjwj6e39DySdAjybYZ6G678cuQuc2W75xmheLKkIXAGcSDKW+XRJQ41pvi0i5qVfnxnNOW3n3nfd/KwjNI2TF7056wgto9pxPIdHxCZJ7weWAB8H7gQ+X7dkNdA/BqjXC6Ga5cU5wLckXQ4IeAz4YLaRGqvsWeDMRvJKSdcP8byAvUZ57GOAVRHxEICkbwOnkNzcNcstd32rrWoLoM503Z93AJdHRK+k3FcV/WOASh4DZJYLEbEaOE7SRJLZJV/IOlOj9RdAchc4s+FMBL4KbB70vEgKmNGYQXLjpd9a4Ngh9jte0t3AOuBjEXHfUAeTtABYALDPmDGjjGY2NBc/tVdtAfRV4BHgbuBWSTOBTfUKVSv9d1g9DbZZfkj6M+BVJGuLAdBOXUz6J0EougAyG84WYGtE3DJ4g6QHR3nsoX7xBn9IWA7MjIjNkk4CfgDMGepgEbEQWAhw+B4T/WFjFz3+uaXgGeB26ppTl7HyoqxTtJ5qJ0G4LCJmRMRJkVgD5L4joidBMMsXSVcC7wX+luSDyLuBmZmGarCBAshjgMyG8/uIuHmYsTmjnZBgLXBAxeP9SVp5BkTEpojYnH6/hKQXzLRRnteG4Omvd+6S8wusvOiKrGO0pGonQZgs6QuSlqVflwAT6pxt1Pq7wPW5C5xZXvxhRHwQWB8R/wwcz0s/jLS8tAccbgAyG9F3JH1ciXGSvgT8n1Ee8w5gjqTZkrqA04CXjDeStI/S5mlJx5B8VnpulOc122UbFpyRdYSWVe0w3KuAF4D3pF+bgEX1ClUr7gJnljvb0z+3StoP6AVmZ5in4UpeCNWsWseS3CD5JUnhsg543WgOGBF9wHnAjcBK4Dvp+obnSDon3e1U4N50DNBlwGkR4Q8SNXbR3Nx/jMyUx/3UV7VjgA6OiHdVPP5nSSvqkKemip4G2yxvfiRpCskMkstJ+t5/LdNEDeYxQGZV6wW2AeOAscDDETHq/9DTbm1LBj13ZcX3lwOXj/Y8ZrvLxU/9VdsCtE3S6/sfSHodyUUp14qeBtssN9JFlH8WERsi4rskY38Oi4i2WmSw/0ZywS1AZiO5g+SzxmuB15Os2XNdtpGsFrz46fBc/DRGtS1A5wBXS5qcPl4P5L5jYsfAJAhuATLLWkSU0/GDx6ePdwA7sk3VeP0tQAW3AJmN5C8jYln6/ZPAKZI+kGUgqw0vfjq0xTMvY/XKrFO0h2pngbs7Io4EXg28OiKOAv6orslqoDDQBc4tQGY5cZOkd/UPMG5H/Zcjd4Ez27mK4qfyuf/IIotZI6y+4a6sI7SNaluAgGRqyIqHHwH+vaZpauzFLnBuATLLiY+QzCDZJ2k7yVTYERF7ZBurccr9LUDVdkA2M7OWNu+4WZy8KPery7SUXSqABsn97cuiW4DMciUiJmWdIWtlzwJnZm3Ms7+9nIufxhtNAZT7qqKoZK2Nnj63AJnlgaQ3DvV8RNza6CxZ6Z8G22OAzMzMkx5kY6cFkKQXGLrQEcm0lLkmicnjOtm0vTfrKGaW+PuK78cCxwB30gRjCmul7EkQzKxNzb3wXPA8fgNc/GRnpwVQK3RXmTKukw1bXQCZ5UFEnFz5WNIBwOcyipOJ/kkp3QXOzNqNZ3970dlnzYYLsk7Rvlp+GO7k8V1s2OYCyCyn1gJHZB2ikcoDXeAyDmJmZpmZcUF31hHa2mjGADWFyeM62bi1J+sYZgZI+hIvdqstAPOAuzMLlIFyBFLSRdfMrF24+9uL3PUtey1fAE0Z18ma57ZkHcPMEpXrevQB10bEL7IKk4VSObwGkJm1HXd/S7j4yYfWL4DGd7LRXeDM8uI6YHtElAAkFSWNj4itI71Q0gnApUAR+HpEXDxo+98D708fdgBzgb0j4vlavoHRKgcU3P/NzKztuPjJj5YfAzRlXFIA9c+8ZGaZ+hkvnUFyHPDTkV4kqQhcAZwIHA6cLunwyn0i4vMRMS8i5gH/ANySt+IHki5wrn/MzNrLNacuG3kna5i6FUCSrpL0tKR7h9kuSZdJWiXpHklH1yPH5PFdRMAL2/vqcXgz2zVjI2Jz/4P0+/FVvO4YYFVEPBQRPcC3gVN2sv/pwLWjSlon7gJnZu3Gi5/CyouuyDqCVahnF7jFwOXA1cNsPxGYk34dC3wl/bOmpozrBGDDth4mj++s9eHNbNdskXR0RCwHkPQaYFsVr5sBPFbxeC3DXC8kjQdOAM4bZvsCYAHA9OnTWbp06Ygn37x5c1X7VePRx3ZQLpdqdrzBapm1npyztpzTLL/c9S1/6lYARcStkmbtZJdTgKsjIoDbJU2RtG9EPFHLHFPSomfD1l5m7lXLI5vZbvgw8F+S1qWP9wXeW8XrhmoyGa5f68nAL4br/hYRC4GFAPPnz4/u7u4RT7506VKq2a8aN2+8lzHPrKvZ8QarZdZ6cs7ack7LqykLvwlfLGcdIzMufvIpy0kQhrqjOwN4WQG0O3dsIbnT9MT6pAferbcvY/3qfM750Cx3xJyztpolJ9Qua0TcIekw4FCSouaBiKhmlpK1wAEVj/cH1g2z72nktPsbQCmCgrvAmVmb+KiLH8uhLCuCqu/o7s4dW0juNL358Pl89te3cOCcuXTPm7G7WeuqWe6IOWdtNUtOqF1WSecC34qIe9PHe0o6PSK+PMJL7wDmSJoNPE5S5LxviONPBt4E/MWow9ZJqYwLIDOzFufiJ9+ynAVuV+7o7rbJ6RggT4Vtlgt/HREb+h9ExHrgr0d6UUT0kYzpuRFYCXwnIu6TdI6kcyp2fSdwU0TkdvGvcjkotvz8m2Zm8KOzbs46QibmHTcr6wg2gixbgK4HzpP0bZLBzBtrPf4HXiyANmx1AWSWAwVJSsf+9U9v3VXNCyNiCbBk0HNXDnq8mGQCltwqh2eBM7P2sOL2R7KO0HBnnzWbGRd0Zx3DRlC3AkjStUA3ME3SWuDTQCcMfGhZApwErAK2AnVpK+zqKDChq+gWILN8uBH4jqQrSbq8ngPckG2kxipFIBdAZtbiDj7hKFiTdYrGc/HTHOo5C9zpI2wP4Nx6nb/SlPFdbgEyy4ePk0xo8jck4wBvAr6WaaIGS7rAuQAys9Z25poPZR2h4Tzup3m0RU/0yeM62bitJ+sYZm0vIsoRcWVEnBoR7wLuA76Uda5GKgUugMzMWoyLn+bSFgXQlPGdbgEyywlJ8yT9q6RHgIuABzKO1FDlCNwDzsysdVxyflt8nG4pbfE3NmV8Jxs8BsgsM5IOkfQpSSuBy0lmgVREvDki2qoFqFz2JAhm1toe/9zSrCM01IYFZ2QdwXZRPlcGrbHJ4zwGyCxjDwC3ASdHxCoASednGykbJY8BMrMWd9Wih7OO0DDu+tac2qYFaOO2HtKZd82s8d4FPAncLOlrkt7C0Isht7xyeCFUM2tvG597kKfW3ML2Lc9kHWVUGlX89JWD5es38uvnN7C9VGrIOVtdm7QAddJbCrb2lJgwpi3eslmuRMT3ge9LmgC8AzgfmC7pK8D3I+KmLPM1UjmCQlvcejLLJ0knAJcCReDrEXHxoO1Kt59EskzHmRGxvOFBm9RFcxcNu61n+3p+veQ8Nm98jPETD+SF9SuZ+ar3cMQfXtB0ywMsnnkZq1fW/zz3bNzEx+95kMlRoIsCj0cP/zT3YN78imn1P3kLa4tqYEq6GOrGbb0ugMwyFBFbgG8B35I0FXg38AmS6bDbQsljgMwyky6+fAXwVpKxiHdIuj4i7q/Y7URgTvp1LPCV9E8bpeU/+yfGjJ3JoUdfjFSkt2cj9//mAibteTCzDj8163hVu3DlWQ0pfraVSnxkxUrOi1dwbGEiAL9jO//f/as4dNJE9hs3tv4hWlRb3IecMj4pgDwOyCw/IuL5iPhqRPxR1lkaKWkBcgFklpFjgFUR8VBE9ADfBk4ZtM8pwNWRuB2YImnfRgdtRlMWfnPYbT3b1/Ps47/iwMP+iqQOhc6uyRww5yzW3P9fjYrYVG555nleqTEDxQ/AIRrLG5nEkiebu/tg1tqiAJo8rguADV4LyMwyVg63AJllaAbwWMXjtelzu7oPAJIWSFomadn6nr6aBm1GH/1iedhtvT2bKXaMo1AY85Lnu8buRe+OjfWOVjONnPTghb4+9oyX91yaSpEXev3vbTTaogDqbwHa6BYgM8tYqRyeBMEsO0P98g2eIamafZInIxZGxPyImL9nl7vY78z4STModoxj43MvHU71zNobecWBb8go1a5p9Ixvr91zMnfEFjbHixMf9EZwqzZz3F5TGpql1bTFb+tAFzivBWRmGSuX8SQIZtlZCxxQ8Xh/YN1u7GOD/Oism+H2R4bdLhU48k0XsuwnF7DPzHcwftIs1j/9S17YcB9vetP/bVzQ3ZTFdNezJoznz/bbmwueWMvJ5cl0UeAGbeSgyRM4duqUhudpJW3x3/CU/i5wbgEys4yVwusAmWXoDmCOpNmSuoDTgOsH7XM98EEljgM2RsQTjQ7abFbspPjpN33mm3jDO/+Tjq5eNjx7K3vNOII3v+d7jJ2wd/0DjsLimZdldu4Pz5nNR151EKunlrlnSg/vP/QA/s+rD3NPglFqixagsZ0FujoKHgNkZpkrh7vAmWUlIvoknQfcSDIN9lURcZ+kc9LtVwJLSKbAXkUyDbZXuqyhPfaaw5Fv+lTWMap29lmzWX3BXZmdXxJvmDaVN0ybmlmGVtQWBZAkpozrZP0WF0Bmlq2yxwCZZSoilpAUOZXPXVnxfQDnNjpXM5uy8JuwkwkQmtmMC7qzjmB10BZd4ABm7DmOteu3ZR3DzNqcu8CZWavZ2exvzSyLcT/WGG1TAM3aawJrntuadQwza3OlMm4BMjPLORc/ra1tCqCZe41n3cZt7OgrjbyzmVmdRARuADKzVvH455ZmHaHmrjl1WdYRrM7aqgCKgMeedzc4M8tOqewucGbWOq5a9HDWEWpq8czLWHnRFVnHsDprowJoAgBrntuScRIza2elCAougMzMcmn1DdnN+GaN0zYF0Ky0AHrE44DMLEMRUPQYIDNrARfNXZR1hJryuJ/20TYF0J7jO5k0poNH3QJkZhkqlT0GyMwsb1z8tJe2KYAkMXPaeLcAmVmmSmV3gTOz5jf3wtZZKsnFT/tpmwIIknFAjz7vAsjMslOOcBc4M2t677tuftYRzHZbexVAU8fz2PNb6Su15oJdZpZ/ZS+EamaWG279aU9tVQDN2msCfeVg3YbtWUcxszZVKiddcs3MLFsuftpXWxVAB+41HoA1z3siBDPLRtIClHUKM7P25uKnvbXVf8OeCtvMslYqewyQmVmWXPxYWxVAr5g0hrGdBdY86xYgM8tGOcJd4MzMMnL2WbOzjmA50FYFUKEgZk6dwCNeC8is6Ug6QdKDklZJ+sQw+3RLWiHpPkm3NDpjNcplT4JgZpaVGRd0Zx3BcqAj6wCNNnffSfxi9XOE78KaNQ1JReAK4K3AWuAOSddHxP0V+0wBvgycEBGPSnpFJmFHUPIscGZmmXDXN+tX1xagke7YpndrN6Z3bFdI+lQ98wAcPXNPnnlhB2vXb6v3qcysdo4BVkXEQxHRA3wbOGXQPu8DvhcRjwJExNMNzliVckDBN1/MrMldc+qyrCPsEhc/VqluLUDV3LFN3RYRb6tXjsGOPnBPAJY/up4Dpo5v1GnNbHRmAI9VPF4LHDton0OATklLgUnApRFx9eADSVoALACYPn06S5cuHfHkmzdvrmq/apRKZdY+9ihLlz5Zk+MNVsus9eScteWcZsO75tRlrLwo6xSWJ/XsAjdwxxZAUv8d28EFUEMdts8kxncVuevRDZwyb0aWUcysekM1mcSgxx3Aa4C3AOOAX0m6PSJ+95IXRSwEFgLMnz8/uru7Rzz50qVLqWa/apRv/G9mz5pJd/ehNTneYLXMWk/OWVvOaY228qIrYO6irGOMaPHMy1h50V1Zx7CcqWcBVM0dW4DjJd0NrAM+FhH3Dd5hd+7YwvB3mg6cGNxy76Ms3eOZqo5Tb81yR8w5a6tZckIusq4FDqh4vD/JNWPwPs9GxBZgi6RbgSOB35ETEUG4C5yZWcOsvsHFj71cPQugau7YLgdmRsRmSScBPwDmvOxFu3HHFoa/0/Sb7Q+w8NaHOO51b2BsZ7GqY9VTs9wRc87aapackIusdwBzJM0GHgdOIxnzU+mHwOWSOoAukhsuX2xoyhGU0yugJ0EwM6s/j/ux4dRzEoQR79hGxKaI2Jx+v4Sk//60OmYCknFAfeXgnrUb630qM6uBiOgDzgNuBFYC34mI+ySdI+mcdJ+VwA3APcBvgK9HxL1ZZR5KKa2AXP+YmdWXix/bmXoWQAN3bCV1kdyxvb5yB0n7KJ2LWtIxaZ7n6pgJgKMOnALAnWvW1/tUZlYjEbEkIg6JiIMj4rPpc1dGxJUV+3w+Ig6PiCMi4t8zCzuMcqQFkCsgM2sBeV1U1MWPjaRuBVA1d2yBU4F70zFAlwGnRcTgbnI1t9fEMczaazzLH3UBZGaN098CVPQYIDNrAV5U1JpVXRdCTbu1LRn0XOXd2suBy+uZYTjHzJ7KDfc+SU9fma6Oui6HZGYGvNgC5DFAZma1N++4WZy86M1Zx7Am0Laf/P/k8H3YtL2PXz1U9x53ZmYAlMvJn3ILkJlZzbn4sWq1bQH0+jnTmNBV5IZ7n8g6ipm1iVJ/C5DrHzOzmvK4H9sVbVsAje0s8kdzp3PTfU8N9Ms3M6ungTFA7gJn1nCSpkr6iaTfp3/uOcx+j0j6raQVkpY1OmezycNECC5+bFe1bQEEcMKr9uG5LT385uHns45iZm2gf44Xd4Ezy8QngJ9FxBzgZ+nj4bw5IuZFxPzGRLPd5eLHdkdbF0Ddh+7NmI4CN973ZNZRzKwNlDwJglmWTgG+mX7/TeAd2UVpHVnOBHfNqW6gs93T1gXQhDEdvOmQvfnxvU/QVypnHcfMWpynwTbL1PSIeAIg/fMVw+wXwE2S7pS0YGcHlLRA0jJJy9b39NU4ru3M4pmXsfKiK7KOYU2qrQsggHfPP4CnNu1gyb1uBTKz+upf5cwLoZrVh6SfSrp3iK9TduEwr4uIo4ETgXMlvXG4HSNiYUTMj4j5e3bVdWURG2T1DXdlHcGaWNv/tr7lsFcwe9oEvn7bQ5z86n3dN9/M6qa/Bcj1j1l9RMQfD7dN0lOS9o2IJyTtCzw9zDHWpX8+Len7wDHArXUJbLvF435stNq+BahQEGe/fjb3rN3IHY+szzqOmbUwjwEyy9T1wBnp92cAPxy8g6QJkib1fw/8CXBvwxLaiFz8WC20fQEEcOrR+7Pn+E6+dttDWUcxsxZWHmgBcgFkloGLgbdK+j3w1vQxkvaTtCTdZzrwc0l3A78B/jsibsgkrb2Mix+rlbbvAgcwrqvIB46byZduXsU9azfw6v2nZB3JzFpQ/5JjbgEya7yIeA54yxDPrwNOSr9/CDiywdGsCi5+rJbcApT6qzcexLSJY7jwB/cO3KU1M6sljwEyM9t1eVhs1VqLC6DUHmM7+ceT5nL32o18+47Hso5jZi2oHO4CZ2a2K84+a3amaw1Za3IBVOGUeftxzOypfO7GB3h60/as45hZixlYB8hNQGZmVXHxY/XgAqiCJD77jiPY0Vvmf39rOT19XhzVzGpnoAXIBZCZ2Yg87sfqxQXQIHOmT+JfT301y9as51/++/6s45hZC3EXODNrRYtnXlbzY7r4sXpyATSEtx+5H3/9htlc/as1XHnL6qzjmFmLKKWNykUXQGZmw7rm1GVZR7AW52mwh/HxEw7jyU07uPjHD9BXKnPeH83JOpKZNbmBWeB868nMbFgrL7oi6wjW4lwADaOjWOCL7zmSzoL4t5t+x1ObdvBPb5vLmI5i1tHMrEmFu8CZme2Uu75ZI/g+5E50FAt8/t1HsuCNB/Eft6/hvV+9ncee35p1LDNrUqXwLHBmZsNx1zdrFBdAIygWxCdPmstX3n80q57ezFu/eAtX3rKa3pJniDOzXfPiQqgugMysday+4a6aHMdd36xRXABV6cQ/2Jcbz38jb5izNxf/+AHe+oVb+P5dawc+0JiZjSRtAHILkJnZIO76Zo3kAmgXzJgyjq99cD5XnTmfcV0dnP9/7+bN/7aUr936EBu29mQdz8xy7sUWoIyDmJnlyCXn++OoNZYnQdgNf3TYdLoPeQU33f8kV/38ET67ZCWfv/FBug/dm7fP2483HbI3k8Z2Zh3TzHKm5EkQzMxeZsOCM7KOYG3GBdBuKhTECUfsywlH7Mv96zZx3Z1r+dE967jp/qfoLIpjZk/ltbOm8pqZe3L4vnuw18QxWUc2s4yVy54Ewcys0uKZl7F6ZdYprN24AKqBw/fbg0/tdzifPOkwlj+6gZ+ufIpbf/cMl/7s9wN9/qdNHMPBe0/goL0nMmuv8Rw4dTz77zme/aaMHZga18xaW9ljgMzMBsw7bharF9VmAgWzXeECqIY6igWOmT2VY2ZP5ZMnzWXT9l5+u3YjK5/YxINPvsBDz27hhnufYP3W3pe8rrMA+y27memTxrL3HmN4xaQxTJs4hr0njmHqhC6mTuxirwld7Dmhi0ljOpC7z5g1pRe7wGUcxMysxuYdN4sVtz+yS685edGb6xPGbAQugOpoj7GdvO6V03jdK6e95PlN23t59LmtPL5hG4+v38Yd9/2ejj2m8NSm7dy/bhO3vLCDzTv6hjxmR0FMGd/JlPFdTB7X+ZKvPcZ1ssfYjoE/J47pZNLYjvSrk4ljOhjbWXABZZaRsqfBNjMDkjV/Vl6UdQprVy6AMrDH2E6OmDGZI2ZMBuCgvjV0dx/1kn2295Z45oUdPL+lh+e27OD5Lb2s39LD+q09rN/ay6Ztvazf2sNTm7bzu6deYOO2Xl7YPnTRVKlYEOO7ikzo6qCzQ3QWC0wamxRM4zqLjO8qMqajSFdHgfFjikwa08H4rg7GdBZ4+PFeXrh7HeM6i0wc28H4rmS/zmKBcZ3F5PVjkteb2cuVPAbIzAzwmj+WLRdAOTW2s8gBU8dzwNTxVb+mVA427+hjU1oMvbA9+XPzjuT7zTtKbN7Ry5YdJbbs6KOvHPT0lXkhfc0zL+xga0+JHX0levrKbOlJ/nyJ347cV7ejkBRWHQUxprPIuK4C4zs7GJcWTEWJMZ0FJqSF1ZiOImM6CnQWxZiOIuO6kmKqsyiKhQJjOgqM6SwwNt3WURDFgpIiLS3YOosFOorJeXtKQUS4pavFSDoBuBQoAl+PiIsHbe8Gfgg8nD71vYj4TCMzjqTsWeDMzLzmj2WurgVQFR9YlG4/CdgKnBkRy+uZqZUVCxroDlcr23tL7Ogts6OvxNKf/5KjXvNatvYkBdSWnhJ9pTI7+pLt23pKbOl5sbjqTbdt6+nf1kdvqUxfucyWLX2sXb+NbT0ldvSV6ekr0VcOdvSVa7O47E+W0NVReLFYKvYXUkmxVax4fmxFsdWZFlGdxaTo6koLuY5iId1HFCUK6Ws7ism2okSxkHyw7T+2pPR5vfja9PWSePD5EpPWPP+S/YoF0ZG+VmLg+Y6iKFQ8V5AoFpN9+58vqP/YtFzxJ6kIXAG8FVgL3CHp+oi4f9Cut0XE2xoesEoDBZBbgMysxRw9V6y4feT93PXN8qBuBVCVH1hOBOakX8cCX0n/tJwY21lkbGcR6OQV4wvMmT6prueLCHpKSdHUVw76Skkr1fa+Ett7SwPPl9LWq629JXb0lgZas3pLZR74/Sr2O2AWO3pLlMrxkmKsJy3WSuWgt5Q8v7Wnv2AL+krlgWNVFmZ9paCvXKYWtdlL/OZXNT7gizoKSaHWX0RJIJIP3wOFWsW2gkQh/bO/iPrXd/0Br5k5tW4Zd8ExwKqIeAhA0reBU4DBBVDN/fi3T3DRz7cyYfktoz7Wxm3JBCjFFitQzcxmXNANcxeNuJ+7vlke1LMFqJoPLKcAV0cyD/TtkqZI2jcinqhjLssxSWmXuN0fR7S09Cjd3YfUMNWLyuWgFPFiYdVXHijIShHJ9vT7iKBUhr5yOS2ggnL62nIEd911N3/w6le/5HXlSAqx/paCgcfloBwMHLcv3aevHEQk+5UHzpsUkqV0e3LsF1sf+reV+rOWIXjxHOUIIt1vfFduesnOAB6reLyWoW+WHC/pbmAd8LGIuG/wDpIWAAsApk+fztKlS3d64oefKzF9TJmitu1m9BdNHg9HTu3g/uW/4oE6FUGbN28e8T3lgXPWlnNaM3DXN8uLen66qeYDy1D7zABeUgDt6geWfs1yoXXO2mqWnDPHbqO87j5E0ke0v+QbuzsH6z9IjT314HKeejAXP9OhqoXB7XHLgZkRsVnSScAPSFqXX/qiiIXAQoD58+dHd3f3Tk/cDRy+dCkj7ZcXS5skq3PWlnNa3nnBU8uTehZA1XxgqWafXf7A0q9ZLrTOWVvOWXs5yLoWOKDi8f4krTwDImJTxfdLJH1Z0rSIeLZBGc3MbBirb/CCp5YfhToee8QPLFXuY2Z2BzBH0mxJXcBpwPWVO0jaJ51YBUnHkFzfnmt4UjMzewl3fbO8qWcBNOIHlvTxB5U4Dtjo8T9mNlhE9AHnATcCK4HvRMR9ks6RdE6626nAvekYoMuA09LxhWZmlpHFMy/LOoLZy9StC1xE9Enq/8BSBK7q/8CSbr8SWEIyBfYqkmmwfYvAzIYUEUtIrhmVz11Z8f3lwOWNzmVmZsNz1zfLo7pO8VTFB5YAzq1nBjMzMzNrPHd9s7yqZxc4MzMzs1yQ9G5J90kqS5q/k/1OkPSgpFWSPtHIjK3EXd8sz1wAmZmZWTu4F/hz4NbhdqhYxP1E4HDgdEmHNyZea3HXN8szF0BmZmbW8iJiZUQ8OMJuA4u4R0QP0L+Iu1XhkvOTj5Xu+mZ5l5tl3qt15513PitpTZW7TwOaYQ0Q56wt56y9kbLObFSQWtqF60kr/V3lhXPWVivlzPJ6Us0i7gMqF2oHdrzmf35xbx2z7a7G/dt45SuBZBXqKuT532xesznXrhn2WtJ0BVBE7F3tvpKWRcSw/Xzzwjlryzlrr5my7opqryfN9P6bJatz1pZzDhz/p8A+Q2z6x4j4YTWHGOK5YafTr1yoPa9/B8616/Kazblqp+kKIDMzM7OhRMQfj/IQXqDdrA14DJCZmZlZoppF3M2sybV6AbQw6wBVcs7acs7aa6as9dBM779ZsjpnbTnnCCS9U9Ja4HjgvyXdmD6/n6QlkCziDvQv4r4S+E5E3FflKfL6d+Bcuy6v2ZyrRpSsRWpmZmZmZtb6Wr0FyMzMzMzMbIALIDMzMzMzaxstWQBJOkHSg5JWSfpE1nn6STpA0s2SVkq6T9Lfpc9PlfQTSb9P/9wz66yQrIgt6S5J/y99nNecUyRdJ+mB9Gd7fB6zSjo//Xu/V9K1ksbmIaekqyQ9LeneiueGzSXpH9LfrQcl/Wmj8zaarye10QzXE19LapKtba4nkt6d/j2UJQ07BXCjryHV/luQ9Iik30paIWlZHfPs9P0rcVm6/R5JR9cryy7m6pa0Mf35rJD0qQbletnv0KDtmfy8qsyWyc9sd7RcASSpCFwBnAgcDpwu6fBsUw3oAz4aEXOB44Bz02yfAH4WEXOAn6WP8+DvSAaB9strzkuBGyLiMOBIksy5yippBvAhYH5EHAEUSWYXykPOxcAJg54bMlf67/U04FXpa76c/s61JF9PaqoZrie+lozeYtrnenIv8OfArcPtkNE1ZFf+Lbw5IubVaw2XKt//icCc9GsB8JV6ZNmNXAC3pT+feRHxmXrnSi3m5b9DlRr+86qwmJ1ng2x+Zrus5Qog4BhgVUQ8FBE9wLeBUzLOBEBEPBERy9PvXyD5z3UGSb5vprt9E3hHJgErSNof+DPg6xVP5zHnHsAbgW8ARERPRGwgh1lJ1t0aJ6kDGE+ytkTmOSPiVuD5QU8Pl+sU4NsRsSMiHgZWkfzOtSpfT2qgGa4nvpbURjtdTyJiZUQ8OMJuWVxDcvFvIVXN+z8FuDoStwNTJO2bg1yZGOZ3qFIWP69qszWNViyAZgCPVTxemz6XK5JmAUcBvwamR8QTkHyoAV6RYbR+/w5cAJQrnstjzoOAZ4BFafear0uaQM6yRsTjwL8BjwJPABsj4iZylrPCcLma4verhpri/fp6UhO+ltRPO19PsniP1f5bCOAmSXdKWlCnLNW8/yx+RtWe83hJd0v6saRX1TlTtfL+e5PHn9nLtGIBpCGey9Vc35ImAt8FPhwRm7LOM5iktwFPR8SdWWepQgdwNPCViDgK2EI+utK8RNoH+xRgNrAfMEHSX2Sbarfk/verxnL/fn09qRlfSxqvGX6/fqpkrNXgr2pbC+ryHmuQC+B1EXE0SZeqcyW9cbS5hoo6xHOD338W/w6qOedyYGZEHAl8CfhBnTNVK8+/N3n9mb1MR9YB6mAtcEDF4/1JugfkgqROkg8r34qI76VPPyVp34h4Im3GfDq7hAC8Dni7pJOAscAekv6T/OWE5O97bUT8On18HcmHlrxl/WPg4Yh4BkDS94A/JH85+w2XK9e/X3WQ6/fr60lN+VpSP017PYmIPx7lIeryHneWS1JV/xYiYl3659OSvk/SLWzY8Uy7qZr3n8W/gxHPWXlDKSKWSPqypGkR8Wyds40kt783Of6ZvUwrtgDdAcyRNFtSF8kAy+szzgQkM3eQ9C9fGRFfqNh0PXBG+v0ZwA8bna1SRPxDROwfEbNIfn7/ExF/Qc5yAkTEk8Bjkg5Nn3oLcD/5y/oocJyk8em/g7eQjNnIW85+w+W6HjhN0hhJs0kGYf4mg3yN4uvJKDXL9cTXkrpq5+tJFteQEf8tSJogaVL/98CfkEzqUGvVvP/rgQ8qcRxJt84n6pBll3JJ2if9HUPSMSSfmZ+rc65qZPHzqkqOf2YvFxEt9wWcBPwOWA38Y9Z5KnK9nqSZ8h5gRfp1ErAXyUwtv0//nJp11orM3cD/S7/PZU5gHrAs/bn+ANgzj1mBfwYeIPlP5j+AMXnICVxLMpagl+TO0l/uLBfwj+nv1oPAiVn/XBvw8/H1pHaZc3098bWkJtna5noCvDN9jzuAp4Ab0+f3A5ZU7NfQa8hwP+/KXCRj3u5Ov+6rZ66h3j9wDnBO+r1IZmRbDfyWZIbDRvz9jZTrvPRnczdwO/CHDco11O9Q5j+vKrNl8jPbnS+lgc3MzMzMzFpeK3aBMzMzMzMzG5ILIDMzMzMzaxsugMzMzMzMrG24ADIzMzMzs7bhAsjMzMzMzNqGCyDbKUmb0z9nSXpfjY/9yUGPf1nL45tZvvh6YmbNQNIZkn6ffp0x8ius2XgabNspSZsjYqKkbuBjEfG2XXhtMSJKIx27BjHNrAn4emJmeSdpKsl6YPNJ1lq7E3hNRKzPNJjVlFuArFoXA2+QtELS+ZKKkj4v6Q5J90j6XwCSuiXdLOkakgW6kPQDSXdKuk/SgvS5i4Fx6fG+lT7Xf3dY6bHvlfRbSe+tOPZSSddJekDStypWHL5Y0v1pln9r+E/HzHaFrydmljlJr01/z8dKmiDpPuBc4CcR8Xxa9PwEOCHbpFZrHVkHsKbxCSru2KYfPDZGxGsljQF+IemmdN9jgCMi4uH08dkR8bykccAdkr4bEZ+QdF5EzBviXH9OsiL7kcC09DW3ptuOAl4FrAN+AbxO0v0kK3IfFhEhaUpt37qZ1ZivJ2aWuYi4Q9L1wL8A44D/BHqBxyp2WwvMyCCe1ZFbgGx3/QnwQUkrgF8DewFz0m2/qfiwAvAhSXcDtwMHVOw3nNcD10ZEKSKeAm4BXltx7LURUQZWALOATcB24OuS/hzYOsr3ZmaN5euJmWXlM8BbSbq8fQ7QEPt4vEiLcQFku0vA30bEvPRrdkT037HdMrBT0tf/j4HjI+JI4C5gbBXHHs6Oiu9LQEdE9JHcJf4u8A7ghl14H2aWPV9PzCwrU4GJwCSS68lakpsr/fYnaSW2FuICyKr1AsnFod+NwN9I6gSQdIikCUO8bjKwPiK2SjoMOK5iW2//6we5FXhvOi5gb+CNwG+GCyZpIjA5IpYAHybp7mJm+eXriZnlxULgQuBbwL+SXI/+RNKekvYkaaG+McN8VgceA2TVugfoS7ueLAYuJekusjwdOPwMyd3SwW4AzpF0D/AgSbeVfguBeyQtj4j3Vzz/feB44G6SZucLIuLJ9APPUCYBP5Q0luRu7/m79Q7NrFF8PTGzzEn6INAXEddIKgK/JLnpcRFwR7rbZyLi+YwiWp14GmwzMzMzM2sb7gJnZmZmZmZtwwWQmZmZmZm1DRdAZmZmZmbWNlwAmZmZmZlZ23ABZGZmZmZmbcMFkJmZmZmZtQ0XQGZmZmZm1jb+fyce1ZefhLVDAAAAAElFTkSuQmCC",
            "text/plain": [
              "<Figure size 1008x288 with 3 Axes>"
            ]
          },
          "metadata": {
            "needs_background": "light"
          },
          "output_type": "display_data"
        }
      ],
      "source": [
        "# Define network dimensions\n",
        "in_features, hidden_dim, out_features = 2, 10, 2\n",
        "\n",
        "# Training parameters\n",
        "learning_rate = 1e-2  # step size for gradient descent\n",
        "optim_steps = 100  # network should generally converge within 100 steps\n",
        "\n",
        "# Define and initialize layers\n",
        "layers = [Linear(in_features, hidden_dim),\n",
        "          ReLU(),\n",
        "          Linear(hidden_dim, out_features)]\n",
        "\n",
        "# Initialize network\n",
        "net = Net(layers)\n",
        "\n",
        "# Define list to store loss and accuracy of each iteration\n",
        "losses = []\n",
        "accs = []\n",
        "\n",
        "for i in range(optim_steps):\n",
        "    # Perform forward pass with x_xor as input and y_pred as output variables\n",
        "    y_pred = net.forward(x_xor)\n",
        "\n",
        "    ########################################################################\n",
        "    #    TODO: Calculate MSE loss between prediction and labels (y_xor)    #\n",
        "    #                         and append to list.                          #\n",
        "    ########################################################################\n",
        "\n",
        "    loss, grad = MSELoss(y_xor, y_pred)\n",
        "    losses.append(loss)\n",
        "\n",
        "    ########################################################################\n",
        "    #                          END OF YOUR CODE                            #\n",
        "    ########################################################################\n",
        "\n",
        "    ########################################################################\n",
        "    #            TODO: Perform backward pass and optimizer step.           #\n",
        "    ########################################################################\n",
        "    \n",
        "    net.backward(grad)\n",
        "    net.optimizer_step(learning_rate)\n",
        "\n",
        "    ########################################################################\n",
        "    #                          END OF YOUR CODE                            #\n",
        "    ########################################################################\n",
        "\n",
        "    # Calculate accuracy of prediction\n",
        "    correct = torch.argmax(y_pred, axis=1) == torch.argmax(y_xor, axis=1)\n",
        "    accs.append(torch.sum(correct)/len(y_pred))\n",
        "\n",
        "# Print prediction\n",
        "print(y_pred)\n",
        "\n",
        "# Plot loss and accuracy\n",
        "plt.figure(figsize=(14,4))\n",
        "plt.subplot(1,3,1)\n",
        "plt.xlabel('Iterations')\n",
        "plt.ylabel('Loss')\n",
        "plt.plot(losses)\n",
        "plt.grid()\n",
        "\n",
        "plt.subplot(1,3,2)\n",
        "plt.xlabel('Iterations')\n",
        "plt.ylabel('Accuracy (%)')\n",
        "plt.plot(accs)\n",
        "plt.grid()\n",
        "\n",
        "# Show decision boundary\n",
        "plt.subplot(1,3,3)\n",
        "plot(x_xor, y_xor, net)\n"
      ]
    },
    {
      "cell_type": "markdown",
      "metadata": {
        "id": "zyitaiDVfc4Q"
      },
      "source": [
        "The network should now successfully converge within a reasonable number of iterations and solve the XOR problem with 100% accuracy. You trained your first neural network, great job! This is still a relatively primitive optimization algorithm and we will dive more in depth into optimizing neural networks in later assignments.\n",
        "\n",
        "Replace the ReLU non-linearity by the Sigmoid function and train the network again. You will see that the network has much more difficulty converging - in fact, you might need to increase the number of `optim_steps` to `500`. If you look at the loss graph you'll see that the network initially learns quickly, i.e. the loss rapidly decreases, but learning slows down far before reaching 100% accuracy. Since the network parameters are updated by gradient descent, slow learning simply means that the gradients have become small. To understand why this happens, let's again look at the Sigmoid function $\\sigma(x)$ and its derivative $\\sigma'(x)$:\n"
      ]
    },
    {
      "cell_type": "code",
      "execution_count": 41,
      "metadata": {
        "colab": {
          "base_uri": "https://localhost:8080/",
          "height": 281
        },
        "id": "SSCQHbk0Sy7L",
        "outputId": "a4ff8560-1837-4a93-fae4-ecf71b5f3528"
      },
      "outputs": [
        {
          "data": {
            "image/png": "iVBORw0KGgoAAAANSUhEUgAAAXQAAAEICAYAAABPgw/pAAAAOXRFWHRTb2Z0d2FyZQBNYXRwbG90bGliIHZlcnNpb24zLjUuMSwgaHR0cHM6Ly9tYXRwbG90bGliLm9yZy/YYfK9AAAACXBIWXMAAAsTAAALEwEAmpwYAAA18klEQVR4nO3dd3hUVfrA8e+bTjokJJQAofdeLWBQQLCLusq62Faxoevq/tay9nUtuKurqyuyyioqsqxrQUWxEUAB6b13AgFSIH1Sz++PO+AQEjIhk9yZyft5nnkyM/fMve+ZTN6cOffcc8QYg1JKKd8XYHcASimlPEMTulJK+QlN6Eop5Sc0oSullJ/QhK6UUn5CE7pSSvkJTeg+SESuF5FvvO24IpIqIrdWs01E5N8iclREltVflFUe+ysRubEhj1lbIpIiImm1KF/te+3m6/NFpMOZvv40+x0uIls9vV/lHk3oXkpEzhWRxSKSIyLZIvKTiAwGMMZ8YIwZ09Ax1fG45wKjgSRjzBAPhnUSEXlSRN53fc4YM84Y8259HdMXGWMijTG76rofETEi0sllv4uMMV3rul91ZoLsDkCdSkSigS+AO4HZQAgwHCi2M646agfsMcYU2B1IYyYiQcaYMrvjUPVDW+jeqQuAMeZDY0y5MabIGPONMWYdgIjcJCI/Hi8sImNEZKuzNf9PEVlw/Ou4s+xPIvKyiBwTkV0icrbz+f0icsS1O0JEYkRkhohkiMheEXlURAKqOe5oEdniPO5rgFRVGRH5LfAWcJbzq/5TlfflLHeitSci74jI6yLypYjkicjPItLRpWxPEfnW+e3lsIg8IiJjgUeAa53HWesse6J7QkQCnHXa66z7DBGJcW5LdsZwo4jsE5FMEflTdb8kEblYRFaLSK7zvXzSZdtp9yUiTZx1PCoim4DB1R3HnfdaRG4Rkc3O/c0TkXaV3te7RWQ7sN31vRaRYSJySEQCXcpfKSLHP2tDRGSJ87OTLiKviUiIc9tC50vWOt/va8Wl60hEHhKRjyrF+YqIvOq8HyMibzv3e0BEnnGNQ50BY4zevOwGRANZwLvAOKBppe03AT8678cDucB4rG9cvwNKgVtdypYBNwOBwDPAPuB1IBQYA+QBkc7yM4DPgCggGdgG/PY0x70aCAZ+7zzOrdXU6cRrq3rsfM4AnZz33wGygSHOen0AzHJuiwLSgQeAMOfjoc5tTwLvV9pvqsv7cQuwA+gARAIfA+85tyU7Y/gX0AToi/WtqHs1dUoBemM1jPoAh4Er3NkX8DywCGgGtAE2AGnVHOe07zVwhbNO3Z3v1aPA4krv67fOYzWp4r3eCYx2Kf9f4CHn/YHAMOd+k4HNwH1V/c5c3pM05/12QCEQ7Xwc6Py9DXM+/hR4E4gAEoBlwO12//358s32APRWzS/G+uN8B0hz/vHOARKd227il8R6A7DE5XUC7OfkhL7dZXtv5x9hostzWUA/5x9cMdDDZdvtQGo1x11a6bhpeDahv+Wy7SJgi/P+BGB1Ncd5ktMn9O+Bu1y2dcX6B3g8YRmsfv7j25cB17n5O/s78LLz/mn3BewCxrpsm0T1Cf207zXwFc5/us7HAViJtJ3L+3r+ad7rZ4DpzvtRQMHx11YRy33AJ1Xtx/k4xbUewI/ADc77o4GdzvuJzs9aE5eyE4D5dv/t+fJNu1y8lDFmszHmJmNMEtALaIWVMCprhZXAj7/OYP2xuzrscr/IWa7yc5FYLcEQYK/Ltr1AazePu7+KcnVxyOV+oTNGsFq0O89wn604tX5BWAmmpuOeRESGish8Z/dUDnAH1nvoqrp9nfT+VYqpqphP9163A15xdoscw/pmI5z8ezvd72YmMF5EQrG+6a0yxux11rGLiHzh7JbJBZ6too6nMxMrUQP82vn4eMzBQLpL3G9itdTVGdKE7gOMMVuwWqy9qticDiQdfyAi4vq4ljKxWqvtXJ5rCxyo5rhtKh23TRXlqlMAhLu8vkUtXrsf6FjNtpqmDz3IqfUr4+R/eu6aifXNqY0xJgaYSjXnEapw0vvnjMOtslW81/uxuipiXW5NjDGLXcpU+74YYzZh/UMZx8lJF+ANYAvQ2RgTjXWOwt06gtV9kyIiScCVLvvej9VCj3eJOdoY07MW+1aVaEL3QiLSTUQecP4RICJtsFo5S6so/iXQW0SuEJEg4G6gNsnxBGNMOdaomr+ISJTzxNr9wPtVFP8S6Cki453HvbeWx13rfH0/EQnD6ipx1xdACxG5T0RCnbEOdW47DCSL80RuFT4Efi8i7UUkEqvF+R9zZiM/ooBsY4xDRIZgJUN3zQYeFpGmzt/zPacpW9N7PdW5r55w4mTjNbWqiZVo7wVGYCXh46Kw+u/zRaQb1sgrV4exzkdUyRiTgdXl9W9gtzFms/P5dOAb4G8iEu08Wd1RRM6rZdzKhSZ075QHDAV+FpECrES+Aesk4EmMMZnANcAUrL7wHsAKznyI4z1YreddWP2fM4Hppznu887jdgZ+cvcgxphtwNPAd1gjL348/StOem0eVn/spVhdGtuBkc7Nx5NRloisquLl04H3gIXAbsDB6ZPp6dwFPC0iecDjWEnaXU9htYp3YyW296orWNN7bYz5BHgBmOXsFtmA1dqujQ+x+r9/cB7vuD9g/aPKwzrB+59Kr3sSeNfZbfKravY9ExjFyS1/sM4NhACbgKPAR0DLWsatXIjzZITyE86WaRpwvTFmvt3xKKUajrbQ/YCIXCgisc6TWsf7OKvqnlFK+TFN6P7hLKxRH5lY3RBXGGOK7A1JKdXQtMtFKaX8hLbQlVLKT9g2OVd8fLxJTk626/BnrKCggIiICLvDaFBaZ//X2OoLvlvnlStXZhpjmle1zbaEnpyczIoVK+w6/BlLTU0lJSXF7jAalNbZ/zW2+oLv1llEqr2qWLtclFLKT2hCV0opP6EJXSml/IRXrVhUWlpKWloaDofD7lCqFRMTw+bNm+u8n7CwMJKSkggODvZAVEop5UZCF5HpwCXAEWPMKbP9OWd+ewVrvupC4CZjTFVzaNQoLS2NqKgokpOTsXbrffLy8oiKiqrTPowxZGVlkZaWRvv27T0UmVKqsXOny+UdYOxpto/DmiyoM9Yk/W+caTAOh4O4uDivTeaeIiLExcV59TcRpZTvqTGhG2MWYk2YX53LgRnGshSIFZEznjHN35P5cY2lnkqphuOJPvTWnLwaSprzufTKBUVkElYrnsTERFJTU0/aHhMTQ15engdCqj/l5eUei9HhcJzyHnij/Px8n4jTkxpbnRtbfaF+6lxaYSgqhcIyQ1GZwVEGjnLrZ3G5ocj5s1NsAL3iPX8K0xN7rKqpWeUEMcaYacA0gEGDBpnKg/o3b95c5/7p+uaJPvTjwsLC6N+/v0f2VZ989QKMumhsdW5s9YXT17miwpDrKCUzv5jM/BKy8kvIKijmaEEpx4pKOFZYyrHCEo4VlZJTWEquo5RcRxklZRVuHfvOlI6kpHTzYG0snkjoaZy8HFYS1jJfSinldYrLyjl4zMHGzHIyVuzncK6DQ7kODuUUczjXweFcB9kFJZRVVD1xYVRoEDHhwTQNDyE2PJjWsU2IbhJMVFgQ0WHWz6iwIKJCg4kMCyIiJIjw0EAiQ4MIDwkkPCSIwID66XL1REKfA0wWkVlYq+zkOJeX8jtFRUWMGzeOBQsWEBgYWGWZkpISRo0axQ8//EBQkFeNClWq0cgvLmNXRj47M/LZnVlIWnYh+48Wsj+7iMN5Dk5MMrtiHQCx4cG0iA4jMTqM7i2jaB4VSlxEKHGRIcRHWj/jIkKJDQ8mONB7L99xZ9ji8aWp4kUkDXgCa7VujDFTgblYQxZ3YA1bvLm+grXb9OnTufTSS6tN5gAhISFccMEF/Oc//+H6669vwOiUanwKisvYciiPzem5bD2Ux86MfHZlFHAo95cRZCLQMjqMpGbhnNMpnjbNmtCmaTgZe7cy7rxhJEaHERZc/d+0L6kxoRtjJtSw3WAtTOxRT32+kU0Hcz26zx6tonni0poXFV+7di333HMPmZmZbNmyBWMMjz/+ON9++y3Tpk07UW7kyJE88sgjjB49mkcffZTc3FxeffVVrrjiCh5++GFN6Ep5UJ6jlDX7j7EuLYdN6blsPpjL7qyCE63tqLAgOjaP5JxO8XRMiKBDfCSdEiJo2yyCkKBTW9WpeTtoF+d7sy2ejvYJVOJwOLj22muZMWMGQ4YM4bHHHsPhcPDII4/w5ptv0q5duxNln3rqKR5//HGOHDnC6tWrmTNnDgC9evVi+fLldlVBKZ9njGFfdiEr9x49cdt6OO9E8m7TrAk9WkZzeb/W9GgVTY9W0bSKCWv0w4G9NqG705KuD9999x0DBgxgyJAhAPTp04evv/6arKwsYmNjTyo7YsQIjDG89NJLpKamnuiKCQwMJCQkxKMjYpTydxl5xfy0I5NF2zP5cUcGh3OLAeskZL+2sVzYswUD2zWlb1IsMeE6ZUZVvDah22XDhg307t37xONVq1YxYMAAmjRpcsqVnevXryc9PZ34+PhTEndxcTFhYWENErNSvqisvIJle7JZsDWDRdsz2ZRudbHGhgdzTqd4zuoQx6DkpnROiKq3USH+RhN6JXFxcfzwww8AbNu2jY8//pjFixfTtGlTysvLcTgcREVFkZ6ezvXXX89nn33Gvffey7x587jwwgsByMrKonnz5jrxllKVlJRVsGRXFl+tT+ebTYfJLighOFAY0LYp/3dhV4Z3jqdnqxhN4GdIE3olEyZMYM6cOfTq1Yv4+Hg+/PBD4uLiABgzZgxLlixh1KhRjB8/nr/97W90796dxx57jAcffPBEQp8/fz4XXXSRndVQymuUlVewaHsmX6xL59tNh8h1lBEREsj53RMZ16sF53VpTkSopiJP0HexksjISD7//PMqt02ePJkXXniByy67jCVLlpx4fsSIESc9njlzJs8991y9x6qUN9ubVcDsFfv5aGUah3OLiQ4LYlSPRMb1asnwzvF+M1TQm2hCr4X+/fszYsQIysvLT3th0RVXXEHXrl0bODql7OcoLWfexkPMWrafJbuyCBBI6ZrAU5e14fxuCVUOH1Seowm9liZOnFjjhUU33HBDA0aklP2y8ot5d8leZizZw7HCUto0a8IfxnThqoFJtIxpYnd4jYYmdKXUGdubVcBbi3Yze8V+issqGN0jkZvOTuasDnEE6InNBqcJXSlVaxsO5PDGgp18tT6dwABhfP8kbhvRgU4JkXaH1qhpQldKuW13ZgEvztvC3PWHiAoN4rYRHbjlnPYkRus1F95AE7pSqkaZ+cX84/vtfPDzPkKCArj3gs7cOrw90WF6rYU30YSulKpWYUkZby/azdQFO3GUVXDd4Db8blRnEqK0Re6NNKErpU5hjOGrDYd46vONHM4t5sKeifxxbDc6Ntc+cm+mCb0WPvnkE0aNGmV3GErVq0M5Dh77bAPfbjpMr9bRvP7rAQxKbmZ3WMoNOsrfTTt37uS777478bioqIjzzjuP8vLyKsuXlJQwYsQIysrKGipEpeqkosLw/tK9jH5pAYu2Z/DIRd349K5zNJn7EE3obpo3bx733HPPicfTp09n/Pjx1V5k5LpykVLe7mB+BddOW8Kjn26gT5sY5t03gkkjOhLkxcutqVN5b5fLVw/BofWe3WeL3jDu+RqLVbViUYsWLbjrrrvIy8sD4IMPPmDmzJmArlykfJcxhncW7+EvPxUREVbGi1f34eqBSY1+oQhf5b0J3SbVrVg0ZcqUE2VKSkrYtWsXycnJgK5cpHxTTmEpf/zfWuZtPEzf5oG8Nek8mkeF2h2WqgPvTehutKTrQ3UrFrm2WDIzM09avUhXLlK+ZvW+o0yeuZrDuQ4evbg7Hcv2ajL3A9pBVkl1Kxa5qrx60fGVi0JDQ3XlIuXVjDH8a+Eurpm6BBH46M6zuXV4B+1i8ROa0CuJi4tj3bp1wC8rFl133XUnlXFdvch15aKIiAjmzZt3opyuXKS8SU5hKbe+u4K/zN3MBd0T+PLe4fRrE2t3WMqDNKFXMmHCBPLz8+nVqxeTJk06acUiV2PGjOGbb745ZeWiJ5988kQZXblIeYu9WQVc+cZPLNyewZOX9mDqbwYS00QbGv7Ge/vQbXK6FYtcTZ48mZdeeklXLlJeb+XebG6bsZIKY3j/t0MZ2uHUBoryD5rQz1D//v0ZOXJktasX6cpFyht8vvYgD/x3La1iwph+02A66KX7fk0Teh3ccsst1W7TlYuUnYwx/DN1Jy/O28rg5Ka8OXEQzSJC7A5L1TNN6Er5mdLyCh79ZAP/WbGfy/q2YsrVfXRB5kbC6xK6MaZRDKEyxtgdgvJDxWXl3P3BKr7bfIR7z+/E70d3aRR/T8riVaNcwsLCyMrK8vtkZ4whKytLx6crj3KUlnPHeyv5bvMR/nx5T+4f01WTeSPjVS30pKQk0tLSyMjIsDuUajkcDo8k4rCwMJKSkjwQkVJWMr9txgoWbc/kufG9mTCkrd0hKRt4VUIPDg6mffv2dodxWqmpqfTv39/uMJQ6oaiknFtnLGfxziymXN2HXw1qY3dIyiZudbmIyFgR2SoiO0TkoSq2x4jI5yKyVkQ2isjNng9VKVVZQXEZN/17GUt2ZvG3a/pqMm/kakzoIhIIvA6MA3oAE0SkR6VidwObjDF9gRTgbyKiY6SUqkf5zmS+fE82L1/bj/EDtAuvsXOnhT4E2GGM2WWMKQFmAZdXKmOAKLHOwEQC2YAu1aNUPXGUlnPru8tZte8Yr07oz+X9WtsdkvICUtOIEhG5GhhrjLnV+XgiMNQYM9mlTBQwB+gGRAHXGmO+rGJfk4BJAImJiQNnzZrlqXo0mPz8fCIjG9fVdlpn71JhDP9cU8yKw+VM6hPK2a3qfirMm+tbX3y1ziNHjlxpjBlU1TZ3PglVjXuq/F/gQmANcD7QEfhWRBYZY3JPepEx04BpAIMGDTIpKSluHN67pKam4otx14XW2XsYY3hizkZWHN7Lny7qzm0jOnhkv95a3/rkj3V2p8slDXA905IEHKxU5mbgY2PZAezGaq0rpTzo9fk7mLFkL7cNb++xZK78hzsJfTnQWUTaO090XofVveJqH3ABgIgkAl2BXZ4MVKnG7j/L9/HXb7ZxRb9WPDyuu93hKC9UY5eLMaZMRCYD84BAYLoxZqOI3OHcPhX4M/COiKzH6qJ50BiTWY9xK9WofLfpMA9/vJ7hneOZcnVfAgL0ClB1KrfOphhj5gJzKz031eX+QWCMZ0NTSgGs3HuUu2euonfrGKb+ZiAhQV41Y4fyIvrJUMqLHThWxKQZK2jpnM88ItSrLu5WXkYTulJeqqiknNvfW0FJWQVv3TiYuMhQu0NSXk7/3SvlhYwxPPi/dWw8mMtbNwyiU4LvjZdWDU9b6Ep5oWkLdzFn7UH+MKYrF3RPtDsc5SM0oSvlZVK3HuH5r7dwce+W3JXS0e5wlA/RhK6UF9mVkc89H66mW4toXrymjy5QoWpFE7pSXiLPUcptM1YQHBjAtIkDCQ/RU1yqdvQTo5QXMMbwwOy17Mkq5P3fDqVNs3C7Q1I+SFvoSnmBdxbv4ZtNh3l4XDfO6hhndzjKR2lCV8pm69NyeHbuZkZ1T+C353r3EozKu2lCV8pGeY5SJn+4ivjIUF68uq+eBFV1on3oStnEGMOfPtlA2tEiZk0aRtMIXbVR1Y220JWyyewV+5mz9iC/H9WZwcnN7A5H+QFN6ErZYNvhPJ6Ys5FzO8VzZ0onu8NRfkITulINrKiknLs/WEVkaBAvXduXQJ3bXHmI9qEr1cCe/mIjOzLymXHLEBKiwuwOR/kRbaEr1YC+23SYD5ft5/YRHRneubnd4Sg/owldqQaSXVDCQx+vp3vLaO4f3cXucJQf0i4XpRqAMYZHP11PTlEJ7/12iC4jp+qFfqqUagBz1h5k7vpD3D+6K91bRtsdjvJTmtCVqmeHchw89ukGBrZryqQRHewOR/kxTehK1SNjDP/30VpKyw1/u0aHKKr6pQldqXr0/s/7WLQ9k0cu7k5yfITd4Sg/pwldqXqyJ7OAZ7/czIguzfnN0LZ2h6MaAU3oStWD8grDA/9dS3CgMOUqXUpONQwdtqhUPXhvyR5W7j3KS7/qS4sYvRpUNQxtoSvlYWlHC5kybyspXZtzZf/WdoejGhFN6Ep5kDGGRz7ZgAB/ubK3drWoBqUJXSkP+njVARZuy+DBcd1oHdvE7nBUI6MJXSkPycgr5ukvNjGoXVN+M7Sd3eGoRkgTulIe8uTnGykqKef5q/oQoBcQKRu4ldBFZKyIbBWRHSLyUDVlUkRkjYhsFJEFng1TKe/2zcZDfLkunXsv6ESnhEi7w1GNVI3DFkUkEHgdGA2kActFZI4xZpNLmVjgn8BYY8w+EUmop3iV8jo5RaU89tkGurWI4vbzOtodjmrE3GmhDwF2GGN2GWNKgFnA5ZXK/Br42BizD8AYc8SzYSrlvZ7/ajMZecVMuboPwYHai6ns486FRa2B/S6P04Chlcp0AYJFJBWIAl4xxsyovCMRmQRMAkhMTCQ1NfUMQrZXfn6+T8ZdF1rn6m3NLufDZQ7GJgeTvWMNqTvqP7b6oL9j/+BOQq/q7I6pYj8DgQuAJsASEVlqjNl20ouMmQZMAxg0aJBJSUmpdcB2S01NxRfjrgutc9VKyip45tVFtI5twku3jCA8xHcvvNbfsX9w5xOYBrRxeZwEHKyiTKYxpgAoEJGFQF9gG0r5qWkLd7LjSD7/vmmwTydz5T/c6fBbDnQWkfYiEgJcB8ypVOYzYLiIBIlIOFaXzGbPhqqU99ibVcA/ftjBRb1bMLKbjgFQ3qHGZoUxpkxEJgPzgEBgujFmo4jc4dw+1RizWUS+BtYBFcBbxpgN9Rm4Unax1gfdQHBgAE9c2tPucJQ6wa3vicaYucDcSs9NrfT4ReBFz4WmlHf6fF06i7Zn8uSlPUiM1pkUlffQMVZK1UJOUSlPf76JPkkxTDwr2e5wlDqJnslRqhamfL2F7IJi3rl5sK4PqryOttCVctOqfUeZuWwfN53dnl6tY+wOR6lTaEJXyg1l5RU88vF6WkSHcf+YLnaHo1SVNKEr5YZ3Fu9hy6E8nri0J5Gh2lOpvJMmdKVqkJ5TxMvfbuP8bglc2DPR7nCUqpYmdKVq8MwXmymrMDx5aU9dUk55NU3oSp3Ggm0ZfLk+nckjO9E2LtzucJQ6LU3oSlXDUVrOE59toH18BJPO62B3OErVSM/uKFWNNxfsYk9WIe/9dgihQYF2h6NUjTShK1WFI4UVvL54B5f0acnwzs3tDkcpt2iXi1KVGGN4b1MJIYEBPHZJD7vDUcptmtCVqmTexkOszyzn96O76ORbyqdoQlfKRUFxGU99vok2UQHceFY7u8NRqlY0oSvl4tXvt5Oe4+CGHiEE6YLPysfoJ1Ypp62H8nj7x91cO6gNnZvqqBblezShK4V1IvSxTzcQGRbEg+O62R2OUmdEE7pSwMerDrBsTzYPje1Gs4gQu8NR6oxoQleNXk5hKc/O3Uz/trH8alAbu8NR6ozphUWq0Xvxmy0cLSxhxm+HEKCrECkfpi101ait3X+MD37ex41nJ9Ozla5CpHybJnTVaJVXGB79dAPNI0O5f7SuQqR8nyZ01WjN/Hkv6w/k8NglPYgKC7Y7HKXqTBO6apQy8oqZMm8r53aK55I+Le0ORymP0ISuGqVnvtxEcWkFT12uqxAp/6EJXTU6C7dl8Nmag9yZ0pGOzSPtDkcpj9GErhoVR2k5jzlXIbozpaPd4SjlUToOXTUqr8/fwd6sQmbeOpSwYJ2vRfkXbaGrRmPHkTymLtjJ+P6tObtTvN3hKOVxmtBVo2CM4ZFPNhAeEsQjF3e3Oxyl6oUmdNUo/HdlGst2Z/PwuG7ER4baHY5S9cKthC4iY0Vkq4jsEJGHTlNusIiUi8jVngtRqbrJLijhubmbGdSuqU6+pfxajQldRAKB14FxQA9ggoicsnKus9wLwDxPB6lUXTw7dzN5jjKeHd9bJ99Sfs2dFvoQYIcxZpcxpgSYBVxeRbl7gP8BRzwYn1J1smRnFh+tTGPSiA50SYyyOxyl6pU7wxZbA/tdHqcBQ10LiEhr4ErgfGBwdTsSkUnAJIDExERSU1NrGa798vPzfTLuuvDVOheXGx7/qYiEcKFvcDqpqYfcfq2v1vlMNbb6gn/W2Z2EXtV3VFPp8d+BB40x5ae7jNoYMw2YBjBo0CCTkpLiXpReJDU1FV+Muy58tc7Pzd3M4cJdzLxtKGd3rN0wRV+t85lqbPUF/6yzOwk9DXA9k5QEHKxUZhAwy5nM44GLRKTMGPOpJ4JUqrbWpR3jX4t2MWFIm1onc6V8lTsJfTnQWUTaAweA64BfuxYwxrQ/fl9E3gG+0GSu7FJaXsEfP1pH86hQHhqnY85V41FjQjfGlInIZKzRK4HAdGPMRhG5w7l9aj3HqFStvLlgJ1sO5TFt4kBimug856rxcGsuF2PMXGBupeeqTOTGmJvqHpZSZ2bHkTxe/X4HF/dpyZieLewOR6kGpVeKKr9RUWF48H/rCQ8N5MlLe9odjlINThO68hvvLd3Lyr1HefySHjSP0sv7VeOjCV35hbSjhUz5egsjujTnyv6t7Q5HKVtoQlc+r6LC8MeP1gHw7JW9dEk51WhpQlc+b8aSPSzemcWjl/QgqWm43eEoZRtN6Mqn7czI57mvtjCya3OuG6wzKarGTRO68lll5RXcP3stTUICeeGqPtrVoho9XVNU+aw3Uneydv8xXvt1fxKiw+wORynbaQtd+aQNB3J45fvtXNq3FZf0aWV3OEp5BU3oyuc4Ssu5f/YamkWE8OfL9QIipY7TLhflc17+dhvbDufz75sHExseYnc4SnkNbaErn7JsdzbTFu1iwpC2jOyaYHc4SnkVTejKZxwtKOG+Watp2yycRy/WaXGVqky7XJRPMMbwfx+tIyO/mP/deTYRofrRVaoybaErn/Dvn/bw3ebDPDSuO32SYu0ORymvpAldeb11acd47qvNjOqeyC3nJNsdjlJeSxO68mp5jlLu+XA18ZGhvHi1Xg2q1OloR6TyWsYYHv54PWlHi5g1aRhNI3SIolKnoy105bVmLd/PF+vS+f2ozgxObmZ3OEp5PU3oyittPZTHk3M2cm6neO5M6WR3OEr5BE3oyuvkFJVyx/sriQoL5qVr+xIYoP3mSrlDE7ryKuUVhns/XE3a0ULe+M0AEqJ0FkWl3KUnRZVXmTJvCwu2ZfDslb2131ypWtKErrzGZ2sO8OaCXfxmWFt+PbRt/R+wIBPyDkFhpnW/MNu6X3SM9oeyIHQjhMdDRJzzZ3OIbgU6dFJ5KU3oyitsOJDDHz9ax5DkZjx+ST1MiWsMZG6DvYth3xLYuwRy9lVRUCA0mrbFebDvo1M3h8dB27OsW7uzoEVfCNQ/I+Ud9JOobJeZX8ykGSuIiwjhn78ZQEiQh07tlDpgyxew6VMrgRdmWs9HNLcS8tDbIaa1s/Udb/1s0hQCg1gw/wdShvaDwixn693Zmj+wCvYttvYLEBIJSYOh28XQ6yoI124iZR9N6MpWJWUV3PX+KrILS/jojrOJjwyt2w6NsZLumvdhw//AkQNRraDzaGer+hyI61hzt4kEWMk5vBnEdz51e+7BX1r6uxfC3D/AvEeg60XQ/zfQ8XwICKxbXZSqJU3oyjbGGJ6Ys4Fle7J5dUJ/erWOOfOdFWbD6vdhzQeQsQWCwqD7ZdD/ekgeAQEeHtAV3cpqkfe6ynqcvhbWzIR1s61vBFEtoe91MPAmaJrs2WMrVQ1N6Mo2f/9uOx8u288953fisr5nuC5oQSYseQ2W/QtK8iFpCFz6CvS8EsLq8A+itlr2tW6jn4ZtX1vJ/adXrVvf62D4A9Y3A6XqkSZ0ZYv3l+7lle+386tBSdw/ukvtd5B3GBa/CiumQ2mRlcCHPwAtenk+2NoICoUel1u33IOw+B9WjGs/tFrzw/8ACd3sjVH5LU3oqsF9vSGdxz/bwAXdEnj2yt61m0Ex7xD8+DKsfAfKS6D3r6xE3vwM/inUt+hWMPY5OPf3VmJf/jas/8hK9ikPQYKuuqQ8y62ORREZKyJbRWSHiDxUxfbrRWSd87ZYRPp6PlTlD5buyuLeWWvo1yaW1349gKBAN/u2S4tg4V/h1QGw/C3ofTVMXgHj3/TOZO4qMgHG/BnuW2/989nxPbxxNnzxe6vLSCkPqbGFLiKBwOvAaCANWC4ic4wxm1yK7QbOM8YcFZFxwDRgaH0ErHzXlkO53DZjBW2bhfP2jYNpEuLGKBBjYOPH8O0TkLMful1i9VP7Yn90RBxc8BicdTcseMHq91//EYz4P2sIZVAdR/ioRs+d5tEQYIcxZpcxpgSYBVzuWsAYs9gYc9T5cCmQ5Nkwla9LO1rIjdOXERESxLu3DHFvbvO0lTD9QvjoFgiLhRs/h+s+8M1k7iq8GYx7Ae5aAm2HwbePwetDYfMX1j8wpc6QmBo+QCJyNTDWGHOr8/FEYKgxZnI15f8AdDtevtK2ScAkgMTExIGzZs2qY/gNLz8/n8jISLvDaFB1rXNWUQVTljvIKzE8MrQJSVGnb0cEl+TSYdc7tDz0PSXBMezqMJFDLc4Habhx3Q35e26WtYqOO98mojCNo7F92NbldorCG7ZNpJ9r3zFy5MiVxphBVW40xpz2BlwDvOXyeCLwj2rKjgQ2A3E17XfgwIHGF82fP9/uEBpcXeq8P7vAnPvC96bX41+blXuzT1+4vNyYle8a83w7Y55qZsy8R40pyjnjY9dFg/+ey0qNWfqmMc+2MebpeGN++IsxJUUNdnj9XPsOYIWpJq+6M8olDWjj8jgJOFi5kIj0Ad4Cxhljstz9b6P81/7sQq6btpQ8Rynv3zqUvm1iqy98eCN8cT/sX2pd0XnJy41rFEhgEAydZI2A+eZRq4993Wy4+K/QaZTd0Skf4U4f+nKgs4i0F5EQ4DpgjmsBEWkLfAxMNMZs83yYytfsySzg2jeXkF9cxszbhlWfzEsK4NvH4c0R1uRZl78ON81tXMncVVQiXPUvuOEza+qA96+C/94Muel2R6Z8QI0tdGNMmYhMBuYBgcB0Y8xGEbnDuX0q8DgQB/zTOaa4zFTXx6P83q6MfCb8ayklZRV8eNswerSKrrrgtm/gywesWQ/7T7RGr+jkVpYOKXDnYvjpFWu45o7vYNQTMPAWz09joPyGWxcWGWPmAnMrPTfV5f6twCknQVXjs+OIlcwrKgwfThpGtxZVJPP8I/D1Q9bkWfFd4eavoN3ZDR+stwsKhfP+aF1h+sXvrX9+62ZbUxs01m8w6rT0X73ymJV7s/nVm0swBmZVlcwrKmDlu/DaINj8OYz8E9yxSJN5TeI6Wl0wV0yFzO0wdTj88Iw1PbBSLjShK4/4fO1BJvzrZ6LDgvjvHWfROTHq5AIZ2+DdS+DzeyGxt9WdcN4f9WIad4lAvwnW1bG9r4aFL1pXm+5eaHdkyotoQld1Yozh9fk7uOfD1fRNiuGTu86hfXzELwVKHTD/WZh6jjWS5bLX4KYvqp5jXNUsIg6unAoTPwVTAe9eCp/cCQU6sEzp5FyqDkrLK/jTJ+uZvSKNy/u1YsrVfQgNcrn4Z1eqNRQxe6c1idaFf7HmNVF113GkdaXpwr9aJ063fQWj/2wtrqFrnjZa2kJXZySnqJSb/r2M2SvSuPeCzvz92n6/JPP8DPh4Esy4HDAw8RNrKJ4mc88KbmLNDXPHj9C8G8yZDO9cDBlb7Y5M2URb6KrWth7K464PVrIvu5C/XtOXqwc6L1OvqIDVM6yJtEoKrEmnhj9gJR5VfxK6WWP317wP3zwGb5wD5/zOeu9Dwu2OTjUgTejKbcYYZq/Yz+OfbSQqLJgZtwzlrI5x1sa0lda6mgdXWet2XvIyNO9qb8CNSUAADLgBuoyzrjRd9FdY9x+48Fnofql2wzQSmtCVW/KLy3j0k/V8uuYg53SK4+Vr+5EQFWbN5/3dk7D6PYhsAeP/Bb2v0QRil8jm1hzxA26Auf8HsydCh5Ewbor3zxuv6kwTuqrRvtxynv7Hj+zJKuD+0V24e2QnAk05/DwN5j9jda+cfQ+c9yCERtW8Q1X/ks+B2xfCirfhh7/AG2fBsLusoaL6O/JbmtBVtYwxfPDzPp5e6qBZRCgzbxvGsA5x1oo73z4OhzdYl6iPm6LdK94oMMhaOKPnePj+SWsN1nWz4fxHod+vrblilF/RUS6qSnsyC7j+rZ959NMNdGsWyNzfDWdY+EF470p4fzwU58E171rjoTWZe7fI5takZ7d+DzFJ1miYqcNh+3e6oIaf0Ra6OklZeQVv/bibl7/dRkhgAM9e2Zt2R38m/tv7rJXrw2KsE22Db9WrPH1N0iC49TvY9Kl13uODq6xvWKP/bHNgylM0oasTNhzI4cH/rWPjwVzG9Ejkzxe2JnH9NMqXvQYCnD3ZGgrXpKndoaozJQI9r4SuF1v96wtegDdH0C0xBfq0g2bt7Y5Q1YEmdEWuo5R/fL+dt3/cTVxkKG//qgMXHPsfvD0VSvLITBhB4oTXoGk7u0NVnhIUAsPuhL4T4MeXSFj8T2vStL4TrH/amth9kib0RqykrIKZP+/l1R92kF1Qws39Y3gw9gfCvp4GJXnQ4wo470E2bz5CoiZz/9QkFkY/zdKKvpxdsQxW/NvqWtPE7pM0oTdCxhi+XJ/Oi/O2sjerkLHtAni69xISNr1zUiInsYf1gs1H7AxXNYCS0DhIeQHOuQ9++ruV2NfMtGZ4PPt3OobdR2hCb2SW7sriubmbWZuWw4XNs/mwRyot985BDpda61m6JnLV+ES3hHHHE/srsPLfsPp96DLWGsfefoReNObFNKE3AhUVhvlbj/Dmgl0s25PFFVFbeLPN97TIWAxFTayrCofeCfGd7A5VeYvoljDueavbZcXbsOxfMOMyaNEbzppsjW0PCrE7SlWJJnQ/VlJWwZy1B5m2cCeHDx/ipshlvBk3n6YFO6EoEc5/DAbdout4qupFNoeUh6wW+/rZsOR1+OR2awK2gTda0/XGtrU7SuWkCd0P5RSWMnvFfqYv2klywWoeiviR88KXEFhWAs37wug3rHUqdRy5cldwmPVNrv9E60rhpf+EBVOsW8eRMOBG6HqRttptpgndT1RUGJbuyuI/K/azZsMGLjI/8knYAlqEpGMCopF+N8KAidCyr92hKl8mAp1HWbdj+2D1B1Yf+39vhPA4a3RM3+sgsZf2tdtAE7qPO5Tj4KOV+/l+2Tr65c3nluCl9A3abm1MOgcGPIX0uEznJFeeF9sWRj5sTfi18wdY9S78PBWWvAbxXax+9l7jdWqIBqQJ3QcdznUwb+Mhfl6znri077k4cAl3BWwlINhQkdALej9uXQ3YrIPdoarGICAQOo+2bgWZsOkz2PiJdRXqguchoaf1eex+qZXcteVebzSh+4gDx4r4eu1+dq1ZQKuMhZwfsIYbAvZBMJQ060JAn4eh13gCdPFlZaeIeBj8W+uWd8hK7hs+tqZZnv+M1arvPAY6Xwjth+s3Rw/ThO6likrKWbY7iw0b1lK8YyEd81dwVcBaYqWAiqBAHK0GQ89bofOFhCR0sztcpU4V1cKavnfo7ZBzALZ/Y93WzITlb0FQmDWuveP5kHyu1ZIP0Alg60ITupcoKatg08EcNm1cQ+G2BTTPWs5g2cR5kg1AYVgzKjpeDL0vIqDDSMKbxNobsFK1EdMaBt1s3UodsPcnK7lvm2f9BAiLtZYvTD7XWqAjsZfO2V5LmtBtkplfzPptOzmydQkcWEVC7kZ6y076SS4A+SFNKWw5jJIe5xPScQTh2veo/EVwGHS6wLqNewGO7bcS/J5FsOdH2PqlVS4kClr1g9YDoPVA6xbdWv8OTkMTej0zxnDgaAG7t2/k2O41VBzeSHTONjqU7WRkgDVHSgVCVngyjsQLyO10FtHdUoiM70KkfnBVYxDbBmKvs4Y7AuSkwZ6fIG0ZHFgFS/4JFaXWtogEK8kn9rS6aBJ7QFxnHf/upAndQ8rKKziQkU367k3kHdhCecZ2QnN3E1+0i44mjSQpBqzknRncivy43qS1HURC93MISepP87Bom2uglJeISYK+11o3gLJiOLQBDq6CtBVwaD3snP9Lkg8IhvjOkNAd4jo5bx2tn2Ex9tXDBprQ3WSM4WheIZlHDrL0h08pOrIbc2w/wXn7iXSkk1h2kHaShesks9kBcWRHJrM3bhjhbfqQ2GkAYa16khASQYJtNVHKxwSFQtJA6zbkNuu5shLI2g6HN8GRjdbPtBXWcElT8ctrI5pbw3dj21q3mDYn7geUF9tTn3rU6BN6WVk5R49lkZORTkF2Oo5jhynJO4LJTSew4DBhjiNElWTStCKLOHK5Wn5Zg7ECIVuakRPaguzYwWTFdyaiRReaJ/cgqlU3moVGorOkKFUPgkKsbpfEnsA1vzxfVgxH90DWjl9u2bth/zIr2VeUnSg6AmB5LES3skbkRLW0fka2sOawiXDewuOtVbp8YASOWwldRMYCrwCBwFvGmOcrbRfn9ouAQuAmY8wqD8daJVNRQZGjkIK8HBwFeTjyjlKcf5TigmOUFR6jvCgX48hFHEcJcBwjuCSH0LJcmpTnElWRS6zJpbmU07zSfiuMcDQghpzAOArDE0gL78P+yESOFAXSdcC5xCd1JiohmfigEOIboqJKqZoFhVoXL1V1dWpFOeSlWydhj+1j15qFdIhvYj2Xlw4ZW62x86b81NdKoDWJXZNmVnI/6RZrde2ERkFoNIRF//IzJApCIqzx9g1wTqzGhC4igcDrwGggDVguInOMMZtcio0DOjtvQ4E3nD89bu38/xK78EnCTBFhxkE4DsKlnPAaXldkQsgLiKIgIApHUDT5oW05FhLNnibxBEQ2Jzg6gdCYRCKatSAmriVRcS2JCwohrtJ+UlNTaT8kpT6qppSqTwGBVv98TBK0O4t9RxPpkJJycpmKcijMsq54LciwboVZv9wvOgZFRyH3ABzeaN0vyXPj4AIhkVZyD4mwZjk9e7LHq+hOC30IsMMYswtARGYBlwOuCf1yYIYxxgBLRSRWRFoaY9I9HXBYVFOyIjtRERROeXAkBIdDaCQSGkVASARB4TGERMYSGhlLeGRTwqObEhHdjCah4eg1aUqp0woIhMgE6+au8lJw5EJxDhTnOe/nWj9L8qGk4NSftdl/LYiVg09TQORqYKwx5lbn44nAUGPMZJcyXwDPG2N+dD7+HnjQGLOi0r4mAZMAEhMTB86aNcuTdWkQ+fn5REZG2h1Gg9I6+7/GVl/w3TqPHDlypTFmUFXb3GmhV9XxU/m/gDtlMMZMA6YBDBo0yKRU/rrjA1JTU/HFuOtC6+z/Glt9wT/r7M5p2zSgjcvjJODgGZRRSilVj9xJ6MuBziLSXkRCgOuAOZXKzAFuEMswIKc++s+VUkpVr8YuF2NMmYhMBuZhDVucbozZKCJ3OLdPBeZiDVncgTVs8eb6C1kppVRV3BqHboyZi5W0XZ+b6nLfAHd7NjSllFK14f2XPimllHKLJnSllPITmtCVUspP1HhhUb0dWCQD2GvLwesmHsi0O4gGpnX2f42tvuC7dW5njKk8/RRgY0L3VSKyorqrtPyV1tn/Nbb6gn/WWbtclFLKT2hCV0opP6EJvfam2R2ADbTO/q+x1Rf8sM7ah66UUn5CW+hKKeUnNKErpZSf0IReByLyBxExIuLXy4qKyIsiskVE1onIJyISa3dM9UVExorIVhHZISIP2R1PfRORNiIyX0Q2i8hGEfmd3TE1FBEJFJHVzgV6/IIm9DMkIm2w1lndZ3csDeBboJcxpg+wDXjY5njqhcv6ueOAHsAEEelhb1T1rgx4wBjTHRgG3N0I6nzc74DNdgfhSZrQz9zLwB+pYmUmf2OM+cYYU+Z8uBRrARN/dGL9XGNMCXB8/Vy/ZYxJN8asct7Pw0pwre2Nqv6JSBJwMfCW3bF4kib0MyAilwEHjDFr7Y7FBrcAX9kdRD1pDex3eZxGI0hux4lIMtAf+NnmUBrC37EaZBU2x+FRbs2H3hiJyHdAiyo2/Ql4BBjTsBHVr9PV1xjzmbPMn7C+on/QkLE1ILfWxvVHIhIJ/A+4zxiTa3c89UlELgGOGGNWikiKzeF4lCb0ahhjRlX1vIj0BtoDa0UErO6HVSIyxBhzqAFD9Kjq6nuciNwIXAJcYPz34oVGuTauiARjJfMPjDEf2x1PAzgHuExELgLCgGgRed8Y8xub46ozvbCojkRkDzDIGOOLs7a5RUTGAi8B5xljMuyOp76ISBDWSd8LgANY6+n+2hiz0dbA6pFYrZJ3gWxjzH02h9PgnC30PxhjLrE5FI/QPnTljteAKOBbEVkjIlNreoEvcp74Pb5+7mZgtj8nc6dzgInA+c7f7Rpny1X5IG2hK6WUn9AWulJK+QlN6Eop5Sc0oSullJ/QhK6UUn5CE7pSSvkJTehKKeUnNKErpZSf+H9srrdf0x30OQAAAABJRU5ErkJggg==",
            "text/plain": [
              "<Figure size 432x288 with 1 Axes>"
            ]
          },
          "metadata": {
            "needs_background": "light"
          },
          "output_type": "display_data"
        }
      ],
      "source": [
        "# Plot the sigmoid function and its derivative\n",
        "x = torch.linspace(-5, 5, 51)\n",
        "y = 1 / (1+torch.exp(-x))\n",
        "dydx = torch.exp(x)/(1+torch.exp(x))**2\n",
        "\n",
        "plt.title('Sigmoid function and derivative')\n",
        "plt.plot(x,y)\n",
        "plt.plot(x,dydx)\n",
        "plt.legend([r\"$\\sigma(x)$\", r\"$\\sigma'(x)$\"])\n",
        "plt.grid()"
      ]
    },
    {
      "cell_type": "markdown",
      "metadata": {
        "id": "hx2pRNyoXY-U"
      },
      "source": [
        "As you can see, the Sigmoid function saturates for very small and very large input values and as a result its derivative becomes very small, causing learning to slow down. In the next assignment we will learn how to circumvent this."
      ]
    },
    {
      "cell_type": "markdown",
      "metadata": {
        "id": "DDE8wsMIcHCN"
      },
      "source": [
        "## A2.6 Softmax and Cross Entropy Loss\n",
        "\n",
        "So far the classifier scores predicted by our network (also called 'logits') were unbounded and we simply trained the network to minimize the mean squared error between the scores and the one-hot encoded labels. However, in a classification task we would like to interpret the output of our network as class probabilities, i.e. for all inputs $x$ and all classes $k$ we should have $$0 \\leq P(Y=k|X=x) \\leq 1, \\qquad \\sum_kP(Y=k|X=x) = 1.$$\n",
        "This can be achieved by normalizing the logits $z$ using the Softmax layer:\n",
        "$$P(Y=k|X=x) = \\text{softmax}(\\mathbf{z})_k = \\frac{\\exp{z_k}}{\\sum_i^K \\exp{z_i}},$$\n",
        "where $K$ is the number of classes.\n",
        "\n",
        "Implement Softmax below and run the example. Verify that indeed the smallest and largest logits get assigned the smallest and largest probabilities and that the outputs are equal to the PyTorch implementation."
      ]
    },
    {
      "cell_type": "code",
      "execution_count": 48,
      "metadata": {},
      "outputs": [
        {
          "name": "stdout",
          "output_type": "stream",
          "text": [
            "tensor([[1, 2],\n",
            "        [3, 4]])\n",
            "tensor([[4, 6]])\n",
            "tensor([[4, 6]])\n"
          ]
        }
      ],
      "source": [
        "x = torch.tensor([[1, 2], [3, 4]])\n",
        "print(x)\n",
        "# Output: tensor([[1, 2],\n",
        "#                 [3, 4]])\n",
        "\n",
        "sum_x = torch.sum(x, dim=0, keepdim=True)\n",
        "sum_x_dimf = torch.sum(x, axis = 1,  keepdim=True)\n",
        "print(sum_x)\n",
        "print(sum_x_dimf)"
      ]
    },
    {
      "cell_type": "code",
      "execution_count": 49,
      "metadata": {
        "id": "vjwd9ZGNgtEv"
      },
      "outputs": [
        {
          "name": "stdout",
          "output_type": "stream",
          "text": [
            "Softmax implementation correct:  True\n"
          ]
        },
        {
          "data": {
            "image/png": "iVBORw0KGgoAAAANSUhEUgAAAXwAAAEKCAYAAAARnO4WAAAAOXRFWHRTb2Z0d2FyZQBNYXRwbG90bGliIHZlcnNpb24zLjUuMSwgaHR0cHM6Ly9tYXRwbG90bGliLm9yZy/YYfK9AAAACXBIWXMAAAsTAAALEwEAmpwYAAAYsklEQVR4nO3dfbQcdX3H8fenCVHLgwi5PCQkhGIKpgqaXh56sOUZCajBqhVUQhFOSoUKp3IkWg+nLbYN9VgpRyAnTVGociJWolHCk1TkKKK5URpIAA0hkJBAbngQ8QkC3/4xc9PJZpedvTt7d+/9fV7n7Lm7M7/fzG/J7IfZ2ZnvKCIwM7Ox7/e6PQAzMxsZDnwzs0Q48M3MEuHANzNLhAPfzCwRDnwzs0Q48M3MEuHANzNLhAN/jJC0StIx3R6HpUvSOkkndKJvcfsutm223ftzsT0Hfhe088FoJCL+KCLu6tTybWzKt5XfSHpB0lOSvihpl26Pq1Zx+3616bXbfqN+qXLgm9m7ImIXYCZwGPDp2gaSxo/4qKxyDvweIulNku6S9Fz+VfTdNfNnSvqppF9K+pqkr0r6TD5vnaQTJP0XMBX4Vr7X9ol8/iWSnsj7Pizp+AZjuDHvN/QISRd0+r1b90XEE8AtwJth2zZ1iaSVwK8kjW+2jQKHSVot6dn828Jrh2ZImifpkXwbXC3pPS30rfuttebwzg7bfm0/SZMkfV3SoKRHJX2sMK/sZ+RDku7JP39PSlovaVbJ/8zdFRF+jPADWAecUDNtJ2AN8ClgAnAc8EvgoHz+BOAx4MK87Z8DLwKfqV1m7fKBg4D1wKT89TTgwBLj/Fvgp8Ae3f5v5kfnt0VgCrAKuKww7758+utKbKPrgAfy9nsAPxjaPvP57wcmke1ofgD4FbBvyb51t+8623rD1/l6VwCX5uP/A2At8I5WPiPAfOA3wF/k/00uBh7r9r9lmYf38HvHkcAuwPyIeDEi/gf4NnBGYf544MqIeCkibgJ+XHLZLwOvAWZI2iki1kXEI6/WQdKFwByyD8szw3g/Nnp8Q9JzwPeB7wH/XJh3ZUSsj4jf0HwbBfhC3v4Z4J+K8yLiaxGxMSJeiYivAj8HDi/TtyKHAX0R8Y/5+NcC/wGcTmufkbcAn4+IGyPiJeB6YGrxG0mvcuD3jknA+oh4pTDtMWByYf4Tke9i5NaXWXBErAEuAv4e2CxpsaRJjdrnh3DOIQv7p0u/AxutTouI3SNi/4j4aB7uQ4rbWLNttLb9Y3kfACTNkXRffjjoObJDRxPL9K3I/sCkofXnY/gUsHeLn5G3AP9deL0X8EJE/Lbi8VbOgd87NgJTJBX/TaYCT+TPNwGTJakwf0qDZe1wk4OIuCEi3k620Qdweb2Okv4a+Cvg+IjY0tpbsDGouC0120Zh+21yat4HSfuT7U1fAOwZEbuTHcJptD1v69vGeGutBx7N/+c29Ng1Ik6Bcp8RSbvn4xwsTH4f2W8fPc+B3z07SXrt0AP4EdkxzU9I2ik/d/hdwOK8/Q/JvnZekP94Npvtvw4XPUV2fBIASQdJOk7Sa4Dfkh1/fLm2k6S5ZB/IEyJisHa+Ja/ZNgpwvqT9JO1Btvf81Xz6zmQhOggg6WzyH4dL9G3Fdtt+jR8Dz+c/zr5O0jhJb5Z0WNnPCNne/cvAB/PP4anAR8m+GfQ8B373LCPbqIYelwLvBmYBW4CrgTkR8RBARLxI9kPtOcBzwIfJjp/+rs6y/wX4dP619WKyY5Pz8+U+SfYV9FN1+v0rcCDwSOEsnTMrebc26uXbYMNtNHcDcDvZj6Frgc/kfVcDnyPbcXmKLDh/ULOKun1bVLvtF8f/Mtn/oN4KPJq/h0XA6yn/GXkL8BXgT4BngX8gOyS2ehhjHXHa/pCwjSaSfgQsiIgvdnssZimQdA3ws4j4fLfHMhzewx9FJB0taZ/8q+RZwCHArd0el1lC3gI82O1BDJevnhtdDgJuJDs17hHgfRGxqbtDMkvKm4GHmrbqUT6kY2aWCB/SMTNLRE8f0pk4cWJMmzat28OwMWrFihVbIqJP0snAvwPjgEURMb/YLj8F9jLgFWArcFFEfD+ft46svMDLwNaI6G+2Xm/X1klD23W9eT0d+NOmTWNgYKDbw7AxStJjksYBVwEnAhuA5ZKW1pxmdyewNCJC0iFkv6McXJh/bCsXqXm7tk6S9FijeT6kY6k7HFgTEWvz88wXA7OLDSLihUJJi6ELiMxGHQe+pW4y29dw2cD2tWEAkPQeSQ8BNwMfKcwK4HZJK/Irlc16lgPfUqc60+rVIloSEQcDp5Edzx9yVETMJLv69HxJf1Z3JdJcSQOSBgYHXbXCusOBb6nbwPZFu/bjVYp2RcTdwIGSJuavN+Z/NwNLaFDfKCIWRkR/RPT39dX9Pc2s4xz4lrrlwHRJB0iaQFYbfWmxgaQ3DlUplTST7OYZT0vaWdKu+fSdgZPIKkCa9aSePkvHrNMiYmte//82stMyr42IVZLOy+cvAN4LzJH0Elmhuw/kZ+zsDSzJ/18wHrghIlzqwnqWA9+SFxHLyKqXFqctKDy/nDq10fM7Jh3a8QGaVcSHdMzMEuE9/JKmzbu5Y8teN//Uji3bzGyIA98scd6ZSYcP6ZiZJcKBb2aWCAe+mVkiHPhmZomoJPAlnSzpYUlrJM2rM//1kr4l6X8lrZJ0dhXrNTOz8toO/EI98VnADOAMSTNqmp0PrI6IQ4FjgM/ll7GbmdkIqWIPv2k9cbLqg7vm9Uh2AZ4hu3OQmZmNkCoCv0w98S8AbyKrQng/cGFEvFJvYS4ja2bWGVUEfpl64u8A7gMmAW8FviBpt3oLcxlZM7POqCLwy9QTPxu4KTJrgEfZ/p6gZmbWYVUEftN64sDjwPEAeUnZg4C1FazbzMxKaruWTsl64pcBX5J0P9khoEsiYku76zYzs/IqKZ5Wop74RrK7AZmZWZf4Slszs0Q48M3MEuHANzNLhAPfzCwRDnwzs0Q48M3MEuHAt+SVKO89W9JKSffldZ7eXravWS9x4FvSSpb3vhM4NCLeCnwEWNRCX7Oe4cC31DUt7x0RL0TEUEHAnfn/4oBlSoOb9QwHvqWuTHlvJL1H0kPAzWR7+aX75v1d9tu6zoFvqStT3puIWBIRBwOnkdWGKt037++y39Z1DnxLXZny3ttExN3AgZImttrXrNsc+Ja6puW9Jb0xvz0nkmYCE4Cny/Q16yWVVMs0G61Klvd+LzBH0kvAb4AP5D/i1u3blTdiVoID37pu2rybO7bsdfNPbdqmRHnvy4HLy/Y161U+pGNmlggHvplZIhz4ZmaJcOCbmSXCP9qamXVAt09GqMd7+GZmiXDgm5klwoFvZpaISgK/zE0gJB2T30BilaTvVbFeMzMrr+0fbQs3gTiRrJjUcklLI2J1oc3uwNXAyRHxuKS92l2vmZm1poo9/DI3gfggcFNEPA4QEZsrWK+ZmbWgisAvcxOIPwTeIOkuSSskzalgvWZm1oIqzsMvcxOI8cAfA8cDrwN+KOneiPjZDguT5gJzAaZOnVrB8MzMDKrZwy9zE4gNwK0R8auI2ALcDRxab2G+M5CZWWdUEfhlbgLxTeBPJY2X9PvAEcCDFazbzMxKavuQTpkbSETEg5JuBVYCrwCLIuKBdtdtZmblVVJLp9kNJPLXnwU+W8X6zMysdb7S1swsEQ58M7NEOPDNzBLhwDczS4QD38wsEQ58S16zaq+SPiRpZf64R9KhhXnrJN2fV4IdGNmRm7XGtzi0pJWp9go8ChwdEc9KmgUsJLt4cMix+RXkZj3Ne/iWuqbVXiPinoh4Nn95L1n5ELNRx4FvqStT7bXoHOCWwusAbs+rwM5t1EnSXEkDkgYGBwfbGrDZcPmQjqWuTLXXrKF0LFngv70w+aiI2Jjf1OcOSQ9FxN07LDBiIdmhIPr7++su36zTvIdvqStT7RVJhwCLgNkR8fTQ9IjYmP/dDCwhO0Rk1pMc+Ja6ptVeJU0FbgLOLN7DQdLOknYdeg6cBLgooPUsH9KxpJWp9gpcCuwJXC0JYGtE9AN7A0vyaeOBGyLi1i68DbNSHPiWvGbVXiPiXODcOv3W0uBGPma9yId0zMwS4cA3M0uEA9/MLBEOfDOzRDjwzcwS4cA3M0uEA9/MLBEOfDOzRDjwzcwS4cA3M0tEJYHf7BZxhXaHSXpZ0vuqWK+ZmZXXduAXbhE3C5gBnCFpRoN2l5MVqTIzsxFWxR5+01vE5f4G+DqwuYJ1mplZi6oI/Ka3iJM0GXgPsIAmfCs4M7POqCLwy9wi7grgkoh4udnCImJhRPRHRH9fX18FwzMzM6imHn6ZW8T1A4vzG0VMBE6RtDUivlHB+s3MrIQqAn/bLeKAJ8huEffBYoOIOGDouaQvAd922JuZjay2A7/kLeLMzKzLKrnFYbNbxNVM/8sq1mlmZq3xlbZmZolw4JuZJcKBb8lrVhpE0ockrcwf90g6tGxfs17iwLeklSwN8ihwdEQcAlwGLGyhr1nPcOBb6pqWBomIeyLi2fzlvWTXmpTqa9ZLHPiWuqalQWqcA9zSal+XDLFe4MC31JUpDZI1lI4lC/xLWu3rkiHWCyo5D99sFCtTGgRJhwCLgFkR8XQrfc16hffwLXXbSoNImkBWGmRpsYGkqcBNwJkR8bNW+pr1Eu/hW9JKlga5FNgTuDovALg1PzxTt29X3ohZCQ58S16z0iARcS5wbtm+Zr3Kh3TMzBLhwDczS4QD38wsEQ58M7NEOPDNzBLhwDczS4QD38wsEQ58M7NEOPDNzBLhwDczS4QD38wsEQ58M7NEOPDNzBJRSeBLOlnSw5LWSJpXZ/6HJK3MH/dIOrSK9ZqZWXltB76kccBVwCxgBnCGpBk1zR4Fjo6IQ4DLgIXtrtfMzFpTxR7+4cCaiFgbES8Ci4HZxQYRcU9EPJu/vJfsVnBmZjaCqgj8ycD6wusN+bRGzgFuaTRT0lxJA5IGBgcHKxiemZlBNYGvOtOibkPpWLLAv6TRwiJiYX77uP6+vr4KhmdmZlDNLQ43AFMKr/cDNtY2knQIsAiYFRFPV7BeMzNrQRV7+MuB6ZIOkDQBOB1YWmwgaSpwE3BmRPysgnWamVmL2t7Dj4itki4AbgPGAddGxCpJ5+XzFwCXAnsCV0sC2BoR/e2u28zMyqvikA4RsQxYVjNtQeH5ucC5VazLzMyGx1faWvJKXDh4sKQfSvqdpItr5q2TdL+k+yQNjNyozVpXyR6+2WhVuHDwRLITEJZLWhoRqwvNngE+BpzWYDHHRsSWjg7UrAIOfEvdtgsHASQNXTi4LfAjYjOwWdKp3RmiVWHavJs7tux180fHpuFDOpa6Vi8crBXA7ZJWSJrbqJEvKLRe4MC31JW+cLCBoyJiJlktqfMl/Vm9Rr6g0HqBA99SV+rCwUYiYmP+dzOwhOwQkVlPcuBb6ppeONiIpJ0l7Tr0HDgJeKBjIzVrk3+0taSVuXBQ0j7AALAb8Iqki8hKgU8EluQXE44HboiIW7vwNsxKceBb8kpcOPgk9Ut6Pw/4Zj42aviQjplZIhz4ZmaJcOCbmSXCgW9mlggHvplZIhz4ZmaJcOCbmSXCgW9mlggHvplZInylrW3HNcPNxi4HvlkP8f9wrZN8SMfMLBEOfDOzRDjwzcwS4cA3M0tEJYEv6WRJD0taI2lenfmSdGU+f6WkmVWs18zMyms78CWNA64iu4nzDOAMSTNqms0CpuePucA17a7XzMxaU8Ue/uHAmohYGxEvAouB2TVtZgPXR+ZeYHdJ+1awbjMzK6mK8/AnA+sLrzcAR5RoMxnYVLswSXPJvgUwderUuivs5LnKUP985W6cw9yNc7K78T59frjZyKhiD191psUw2mQTIxZGRH9E9Pf19bU9ODMzy1QR+BuAKYXX+wEbh9HGzMw6qIrAXw5Ml3SApAnA6cDSmjZLgTn52TpHAr+IiB0O55h1Q4mzzA6W9ENJv5N0cSt9zXpJ28fwI2KrpAuA24BxwLURsUrSefn8BcAy4BRgDfBr4Ox212tWhcJZZieSfRNdLmlpRKwuNHsG+Bhw2jD6mvWMSoqnRcQyslAvTltQeB7A+VWsy6xi284yA5A0dJbZttCOiM3AZkm1vy437WvWS1wt01JX5iyzTvRNmquCdodLK1jqSp9B1k5fSXMlDUgaGBwcLD04syo58C117ZxBVrqvTze2XuDAt9SVOcusE33NRpyP4VvSypxlJmkfYADYDXhF0kXAjIh4vl7frrwRsxIc+Ja8EmeZPUl2uKZUX7Ne5UM6ZmaJcOCbmSXCgW9mlggHvplZIhz4ZmaJcOCbmSXCgW9mlggHvplZIhz4ZmaJcOCbmSXCgW9mlggHvplZIhz4ZmaJGJXVMn0LMzOz1nkP38wsEQ58M7NEOPDNzBLhwDczS0RbgS9pD0l3SPp5/vcNddpMkfRdSQ9KWiXpwnbWaWZmw9PuHv484M6ImA7cmb+utRX4eES8CTgSOF/SjDbXa2ZmLWo38GcD1+XPrwNOq20QEZsi4if5818CDwKT21yvmZm1qN3A3zsiNkEW7MBer9ZY0jTgbcCPXqXNXEkDkgYGBwfbHJ6ZmQ1pGviSviPpgTqP2a2sSNIuwNeBiyLi+UbtImJhRPRHRH9fX18rqzAbFkknS3pY0hpJOxyWVObKfP5KSTML89ZJul/SfZIGRnbkZq1peqVtRJzQaJ6kpyTtGxGbJO0LbG7QbieysP9KRNw07NGaVUzSOOAq4ERgA7Bc0tKIWF1oNguYnj+OAK7J/w45NiK2jNCQzYat3UM6S4Gz8udnAd+sbSBJwH8CD0bEv7W5PrOqHQ6siYi1EfEisJjst6mi2cD1kbkX2D3fwTEbVdoN/PnAiZJ+TraHNB9A0iRJy/I2RwFnAsflX3vvk3RKm+s1q8pkYH3h9QZ2PKng1doEcLukFZLmNlqJf5uyXtBW8bSIeBo4vs70jcAp+fPvA2pnPWYdVG/bjBbaHBURGyXtBdwh6aGIuHuHxhELgYUA/f39tcs3GxG+0tZStwGYUni9H7CxbJt854aI2AwsITtEZNaTHPiWuuXAdEkHSJoAnE7221TRUmBOfrbOkcAv8hMVdpa0K4CknYGTgAdGcvBmrRiV9fDNqhIRWyVdANwGjAOujYhVks7L5y8AlpEdolwD/Bo4O+++N7AkOy+B8cANEXHrCL8Fs9Ic+Ja8iFhGFurFaQsKzwM4v06/tcChHR+gWUV8SMfMLBEOfDOzRDjwzcwS4cA3M0uEA9/MLBEOfDOzRDjwzcwS4cA3M0uEA9/MLBEOfDOzRDjwzcwS4cA3M0uEA9/MLBEOfDOzRDjwzcwS4Xr4PWzd/FO7PQQzG0O8h29mlggHvplZIhz4ZmaJaCvwJe0h6Q5JP8//vuFV2o6T9FNJ325nnWZmNjzt7uHPA+6MiOnAnfnrRi4EHmxzfWZmNkztBv5s4Lr8+XXAafUaSdoPOBVY1Ob6zMxsmNoN/L0jYhNA/nevBu2uAD4BvNLm+swqJ+lkSQ9LWiNph2+pylyZz18paWbZvma9pOl5+JK+A+xTZ9bflVmBpHcCmyNihaRjSrSfC8wFmDp1aplVmA2bpHHAVcCJwAZguaSlEbG60GwWMD1/HAFcAxxRsq9Zz2ga+BFxQqN5kp6StG9EbJK0L7C5TrOjgHdLOgV4LbCbpC9HxIcbrG8hsBCgv78/yrwJszYcDqyJiLUAkhaTHaoshvZs4PqICOBeSbvn2/u0En3Neka7V9ouBc4C5ud/v1nbICI+CXwSIN/Dv7hR2NdasWLFFkmPtTnGbpkIbOn2IDpstL/H/YHJwPrCtA1ke/FF9dpMLtkX2P6bK/CCpIeHP+zh0eWVLKalf/OK1tmSFN5nk/Xt32hGu4E/H7hR0jnA48D7ASRNAhZFxCntLDwi+tocX9dIGoiI/m6Po5PGwnuU9P46k2u/WapBm0bTd5xY+OY6mo2Ff/Myxur7bCvwI+Jp4Pg60zcCO4R9RNwF3NXOOs0qtgGYUni9H7CxZJsJJfqa9QxfaWupWw5Ml3SApAnA6WSHKouWAnPys3WOBH6Rn5VWpq9Zz3C1zM4Z9V/fSxj17zEitkq6ALgNGAdcGxGrJJ2Xz18ALCP7xroG+DVw9qv17cLbGEmj/t+8pDH5PpWdeGBmZmOdD+mYmSXCgW9mlggHfoUkTZH0XUkPSlol6cJuj6mTXAE1HSlt22N5u/aPttXaCnw8In4iaVdghaQ7xvCl9kMVUHfr9kCs41Latsfsdu09/ApFxKaI+En+/JdkG83k7o6qM1wBNS2pbNtjfbt24HeIpGnA24AfdXkonXIFroCapDG+bV/BGN6uHfgdIGkX4OvARRHxfLfHU7ViBdRuj8VG1ljetlPYrh34FZO0E9kH4isRcVO3x9MhQxVQ1wGLgeMkfbm7Q7JOS2DbHvPbtS+8qpAkkd3565mIuKjLwxkRhQqo7+zyUKyDUtu2x+p27T38ah0FnEm2Z3Bf/mirYqhZj/C2PQZ4D9/MLBHewzczS4QD38wsEQ58M7NEOPDNzBLhwDczS4QD38wsEQ58M7NE/B/qBd0+SFWi8AAAAABJRU5ErkJggg==",
            "text/plain": [
              "<Figure size 432x288 with 2 Axes>"
            ]
          },
          "metadata": {
            "needs_background": "light"
          },
          "output_type": "display_data"
        }
      ],
      "source": [
        "def Softmax(z):\n",
        "    \"\"\"\n",
        "    Computes softmax output for each sample in batch. \n",
        "\n",
        "    Args:\n",
        "      z: Tensor of logits, dimension [batch, classes].\n",
        "\n",
        "    Return:\n",
        "      p: Softmax probability distribution.\n",
        "    \"\"\"\n",
        "\n",
        "    ########################################################################\n",
        "    #                  TODO: Implement Softmax function.                   #\n",
        "    ########################################################################\n",
        "\n",
        "    p = torch.exp(z) / torch.sum(torch.exp(z), dim=1, keepdim=True)\n",
        "\n",
        "    ########################################################################\n",
        "    #                          END OF YOUR CODE                            #\n",
        "    ########################################################################\n",
        "\n",
        "    return p\n",
        "\n",
        "# Generate random logits, dimension [batch, classes]\n",
        "z = torch.randn(1,5)\n",
        "# Calculate softmax\n",
        "p = Softmax(z)\n",
        "\n",
        "# Verify against PyTorch implementation\n",
        "p_torch = torch.nn.functional.softmax(z, 1)\n",
        "print('Softmax implementation correct: ', torch.allclose(p, p_torch))\n",
        "\n",
        "# Plot z and p\n",
        "plt.subplot(1,2,1)\n",
        "plt.title(r'Logits $z$')\n",
        "_=plt.bar(torch.arange(1,6), z[0,:])\n",
        "plt.subplot(1,2,2)\n",
        "plt.title(r'Probabilities $p$')\n",
        "_=plt.bar(torch.arange(1,6), p[0,:])"
      ]
    },
    {
      "cell_type": "markdown",
      "metadata": {
        "id": "4cthwWyegrsl"
      },
      "source": [
        "The Softmax layer is a vector extension of the Sigmoid function and so unfortunately comes with the same saturation problem causing learning to slow down. Luckily we can overcome this by replacing the MSE loss by the Cross Entropy (CE) loss:\n",
        "$$H(y,p) = -\\sum_i^K y_i \\log(p_i),$$\n",
        "where $y$ is the one-hot encoded label and $p$ is the class probability vector from the Softmax layer.\n",
        "Using the CE loss with the Softmax layer is effective as the $\\log$ in the CE loss can undo the $\\exp$ in the Softmax layer:\n",
        "$$\\log \\text{softmax}(\\mathbf{z})_k = z_k - \\log \\sum_i^K \\exp z_i.$$\n",
        "Because now $z_k$ has a direct contribution to the loss it can never saturate and the gradients will never become too small.\n",
        "\n",
        "The Softmax layer and Cross Entropy loss are used together so often that they are implemented as a single function in Pytorch as `torch.nn.CrossEntropyLoss`. Below we have implemented the CE loss with Softmax so we can use it to train our network. Feel free to go over the code, but you do not need to understand it exactly as the implementation is not very straightforward."
      ]
    },
    {
      "cell_type": "code",
      "execution_count": 50,
      "metadata": {
        "id": "FEZF9yQ8cT01"
      },
      "outputs": [],
      "source": [
        "def CrossEntropyLoss(y_true, y_pred):\n",
        "    \"\"\"\n",
        "    Computes softmax output and cross-entropy loss.\n",
        "\n",
        "    Args:\n",
        "      y_true: Tensor containing true labels.\n",
        "      y_pred: Tensor containing predictions.\n",
        "\n",
        "    Return:\n",
        "      loss: Cross-entropy loss.\n",
        "      dy_pred: Gradient of loss w.r.t. y_pred.\n",
        "    \"\"\"\n",
        "    \n",
        "    # Calculate softmax using previously defined function\n",
        "    softmax = Softmax(y_pred)\n",
        "\n",
        "    # Convert one-hot vector to class id\n",
        "    y_true = torch.argmax(y_true, axis=1)\n",
        "    # Get number of samples in batch\n",
        "    n = y_true.shape[0]\n",
        "    # Calculate cross entropy loss between y_true and y_pred\n",
        "    log_likelihood = -torch.log(softmax[torch.arange(n),y_true])\n",
        "    # Average over all samples\n",
        "    loss = torch.mean(log_likelihood)\n",
        "\n",
        "    # Caculate the gradient \n",
        "    grad = softmax\n",
        "    softmax[torch.arange(n), y_true] -= 1\n",
        "    grad /= n\n",
        "\n",
        "    return loss, grad"
      ]
    },
    {
      "cell_type": "markdown",
      "metadata": {
        "id": "oUlGT8Z7A4PW"
      },
      "source": [
        "We will now solve the XOR problem by training the network with the Cross Entropy loss and Sigmoid activation function. The network should be able to converge much faster than using the MSE loss."
      ]
    },
    {
      "cell_type": "code",
      "execution_count": 51,
      "metadata": {
        "id": "7K3xJzgrcQc6"
      },
      "outputs": [
        {
          "name": "stdout",
          "output_type": "stream",
          "text": [
            "tensor([[ 0.5100, -0.4166],\n",
            "        [-0.6127,  0.2500],\n",
            "        [-0.4432,  0.4599],\n",
            "        [-0.0715, -0.9488]])\n"
          ]
        },
        {
          "data": {
            "image/png": "iVBORw0KGgoAAAANSUhEUgAAA0AAAAEWCAYAAABPK/eBAAAAOXRFWHRTb2Z0d2FyZQBNYXRwbG90bGliIHZlcnNpb24zLjUuMSwgaHR0cHM6Ly9tYXRwbG90bGliLm9yZy/YYfK9AAAACXBIWXMAAAsTAAALEwEAmpwYAABR1UlEQVR4nO3dd5xU1f3/8ddnG0svIisCAgIKWCiugAVdNSZoNNgSQWPBgijGqIklRfP7xhRLEqNRg2jEDjEaFRNijStGRSkCgmsBRWkKSF3ats/vj5nFYZ3dnS0zd8r7+XjsgylnZt57l7l3PnPOPcfcHRERERERkUyQFXQAERERERGRRFEBJCIiIiIiGUMFkIiIiIiIZAwVQCIiIiIikjFUAImIiIiISMZQASQiIiIiIhlDBZCIiIiISAOYWZGZrajj/gfN7DeJzCSxUwEkIiIiIgljZm3MbJmZnRVxW1sz+9zMzghf725mj5nZV2a21czeMbOTajyPh+8rNbOVZvYnM8tO9O8jqUcFkIiIiIgkjLuXAuOBO8xsz/DNtwJz3P1JM+sE/A8oAw4AOgO3A49XF0gRBrl7G+Bo4EzggubIaGY5zfE8kpxUAImIiIhIQrn7i8C/gTvNrAj4ATAxfPdVQClwobt/4e7b3X0q8Fvgj2ZmUZ5vCfAGMLi21wz3Ov3MzN43sw1mNsXM8sP3FZnZCjO7zsy+AKaYWQsz+7OZrQr//NnMWtR4zp+b2brwc59dx2ufZGbzzWyjmb1pZgfXyHWNmS0M92j9zcwKzOw/ZrbFzF42s44xbFaJkQogEREREQnCVUAR8CTwU3dfHb79eOApd6+q0f4JYB9gv5pPZGb9gZHAknpe82zgO0Cf8PP8MuK+vYBOQE9CPVS/AEYQKqoGAcOitO8MdAPOAyab2f5Rsg0FHgAuAfYA7gWm1yimTg//3vsBJwP/AX4efv4s4Ip6fi9pABVAIiIiIpJw7r4BWAy0Av4ZcVdnYHWUh6yOuL/aPDPbCpQAxcA99bzsXe6+3N3XE+pRGhtxXxXwK3ff6e7bCRVLv3b3Ne6+Fvg/4Jwaz3dDuP1rhHq0fhDlNS8G7nX3t9290t0fAnYSKq6q/cXdv3T3lcDrwNvu/q677wSeBobU83tJA6gAEhEREZGEM7MfAr2Al4FbIu5aB3SN8pCuEfdXGwq0IXT+z3CgdT0vuzzi8mfA3hHX17r7jojre4fb1NZ+g7tvreP+aj2Bn4SHv200s41Ajxptv4y4vD3K9TbRfx1pDBVAIiIiIpJQZtaF0MQGFxMaGvYDMzsqfPfLwOlmVvNz6g8IFTAfRd7oIU8AbwE31vPSPSIu7wOsinyqGm1XESpeamvf0cxa13F/teXAb929Q8RPq/B5TRIAFUAiIiIikmh3Ac+4+6vhc3+uBe4LnxdzO9AO+JuZ7WVm+WY2ltA5Ode4e81CpdrNwHgz26uO150YnmK7E6FzbP5eR9upwC/NbE8z60youHq0Rpv/M7M8MxsJnAT8I8rz3AdMMLPhFtLazL5rZm3reG2JIxVAIiIiIpIwZnYKcCRwTfVt7n4/sAK40d2/Ct+fD7wPfAVcDZzj7rUWLO7+HvBa5PNG8TjwIvBJ+KeuxUp/A8wBFgLvAfNqtP8C2ECo1+cxYIK7fxAl1xxCPV13hdsvAc6v43Ulzqz2IlpEREREJD2Y2TLgInd/OegsEiz1AImIiIiISMZQASQiIiIiIhlDQ+BERERERCRjqAdIREREREQyRk7QARqqc+fO3qtXr5jabt26ldat61sPKzGUJTpliS7VssydO3edu++ZoEjNJtb9Sar9PRIlmbJAcuVRlujSeX/SIS/X985vEXQMkd1s7Lw/20vLgo4RiI1rF9W6L0m5AqhXr17MmTMnprbFxcUUFRXFN1CMlCU6ZYku1bKY2Wd1NkhSse5PUu3vkSjJlAWSK4+yRJfO+5O981vw6KGDg44hspubBkwJOkJgnrl7v1r3JRoCJyIiIiKSZjK5+KmPCiARERERkTTy3LhXg46Q1FQAiYiIiIikkfmzlgUdIanFtQAys1Fm9qGZLTGz66Pc397MnjOzBWa22MzGxTOPiIiIiEg609C3+sWtADKzbOBu4ARgIDDWzAbWaDYReN/dBwFFwB/NLC9emURERERE0pWKn9jEswdoGLDE3T9x9zJgGjC6RhsH2pqZAW2A9UBFHDOJSAoyswfMbI2ZLarlfjOzO8O9zQvNbGiiM4qIiARp5a3FQUdIGfEsgLoByyOurwjfFukuYACwCngP+LG7V8Uxk4ikpgeBUXXcfwLQL/wzHvhrAjKJiIgkhT6jhvDAlE+DjpEy4rkOkEW5zWtc/w4wHzgW6AO8ZGavu/vm3Z7IbDyhDzUUFBRQXFxc5wtvLXdeXFZO/7ZlUE/bRCktLa03d6IoS3TKEl0yZHH3mWbWq44mo4GH3d2BWWbWwcy6uvvqxCTMDO7Oo29/ztrNOwBY9lkZ88o+DDjV15Ipj7LUYMbVx+8XbAaRNHb+Z1cEHSGlxLMAWgH0iLjenVBPT6RxwM3hDy1LzOxToD/wTmQjd58MTAYoLCz0+hZRW7NlBxNfeYV2A1twaQot/pYoyhKdskSXTFnqUFuP8zcKoIZ+oQLJUQRWCzLLxh1V3FC8Haj+hsth6ZJAskSXTHmUJZIZDM1dlVTvJZF0ofN+Gi6eBdBsoJ+Z9QZWAmOAs2q0+Rw4DnjdzAqA/YFPmvrCHVqG5lHYWl6zw0lE0lQsPc6hGxv4hQokVxEYZJbl67dB8avcdsbBfL+wR1JtF9DfqTbKIpK+VPw0TtzOAXL3CuBy4AWgBHjC3Reb2QQzmxBudhNwuJm9B7wCXOfu65r62nk5WbTOy6a0TAWQSIaIpcdZmqi8MnSKZm62lpATEQnagBsmBh0hZcWzBwh3nwHMqHHbpIjLq4Bvx+O1O7TKY2t5eTyeWkSSz3TgcjObBgwHNun8n+ZXXhn6UkkFkIhI8M56sjDoCCkrrgVQkDq0yqW0vCzoGCLSDMxsKqG1wjqb2QrgV0Au7PpSZQZwIrAE2Ebo/EJpZtU9QDnZ0UYciohIomjoW9OkbQHUsVUeq9eWBh1DRJqBu4+t534ntLCyxFF1AZSnHiARkcAMuGEiPBl0itSWtkexDq1yNQmCiEgzqqgK7VPVAyQiEowOkx/S0LdmkNYFUKkKIBGRZqNJEEREgvWT26uCjpAW0vYo1rFVHlvLoapKRZCISHP4ehIE9QCJiCSazvtpPmlbAHVolYcDW3ZUBB1FRCQtVKgHSEQkECtvLQ46QlpJ26NYh5a5AGzYppngRESaw65Z4LLS9tAhIpKUHpjyadAR0kraHsU6tlYBJCLSnDQETkQk8TT0rfmlbQHUoVUeABu3azFUEZHmoEkQREQSS8VPfKTtUax6CNxG9QCJiDSLikpNgy0ikijPjXs16AhpK20LoI7hHqANW9UDJCLSHMq0EKqISMLMn7Us6AhpK22PYu1a5mJoCJyISHOpngUuRwWQiEhcaehbfKXtUSw7y2iVqyFwIiLNpaJKkyCIiMSbip/4S9sCCKBNrrFxm3qARESaQ5kmQRARiSsVP4mR1kex1rmmabBFRJpJxa5psNP60CEiEog+o4YEHSFjpPVRrH0L4+MvS9lZURl0FBGRlFdeWYVZaIixiIg0r/M/uyLoCBkjrQug4/bJ5YvNO3h01udBRxERSXnlla7eHxGRONDQt8RK6yPZgZ2zGdmvM3f992O+Kt0ZdBwRkZRWXllFrnp/RESaldb7Sby0LoAAfnbCALaVVXLelHfYskMTIoiINFZFZZWmwBYRaUYDbpio9X4CkPZHsoF7t2PSDw/hg9VbuPChOewo1/lAIqnIzEaZ2YdmtsTMro9yf0cze9rMFprZO2Z2YBA501mZhsBJGjCzB8xsjZktquX+IjPbZGbzwz83JjqjZI6zniwMOkJGyogj2TH9u/CnMwcze9l6Ln10roogkRRjZtnA3cAJwEBgrJkNrNHs58B8dz8YOBe4I7Ep019FZZXWAJJ08CAwqp42r7v74PDPrxOQSTKQzvsJTkYUQADfG7Q3vz3lIF79cC3nPfAOmzUcTiSVDAOWuPsn7l4GTANG12gzEHgFwN0/AHqZWUFiY6a3iir1AEnqc/eZwPqgc0hmU/ETrIw6kp01fB/uGDOYuZ9tYMy9s1i7RRMjiKSIbsDyiOsrwrdFWgCcBmBmw4CeQPeEpMsQZZVV5KgHSDLDYWa2wMz+Y2YH1NbIzMab2Rwzm7OhrCKR+SSFdZj8UNARMl5O0AESbfTgbnRolceER+Zy6j1vMPmcQgbu3S7oWCJSt2ifur3G9ZuBO8xsPvAe8C7wjU8kZjYeGA9QUFBAcXFxvS9eWloaU7tECDLLF1/uoGx71a7XT6btAsmVR1miS6YsdZgH9HT3UjM7EXgG6BetobtPBiYDDGzXpuY+SSSqn9xeFXSEjJdxBRDA0fvtyd8vGcH4h+dy+l/f5LbvH8xJB+8ddCwRqd0KoEfE9e7AqsgG7r4ZGAdgZgZ8Gv6hRrtdH1gKCwu9qKio3hcvLi4mlnaJEGSWh5fNZmfODoqKRgaeJZpkyqMs0SVTltqE9yXVl2eY2T1m1tnd1wWZS9KDhr4lh4waAhfp4O4dmP6jIxi4dzsuf/xdfjejhLIKVeQiSWo20M/MeptZHjAGmB7ZwMw6hO8DuAiYGflBRpquvLJK5wBJ2jOzvcJfolQPp80Cvgo2laQDFT/JI6OPZF3a5jP14hH8cMQ+TJ75CWdMepNl67YGHUtEanD3CuBy4AWgBHjC3Reb2QQzmxBuNgBYbGYfEJot7sfBpE1foYVQM/qwIWnAzKYCbwH7m9kKM7uwxr7kDGCRmS0A7gTGuLuGt0mTaLHT5JKRQ+Ai5eVk8ZtTDuLIvp257qn3+O6dr/PLkwYy5tAehL8AEpEk4O4zgBk1bpsUcfktahmnL82jotLJy1EBJKnN3cfWc/9dwF0JiiMZQoudJhcdycJGHdiV//x4JAd378DP/vkeZ933tnqDREQiaAiciEjDaehb8tGRLMLeHVry2EXD+f1pB7Fo5SZG3TGTyTOXUlGpc4NERMorXQuhiog0gIqf5KQCqIasLGPssH146eqjObLvnvxuxgecfNcbvLVU5z+KSGZTD5CISOxU/CQvHclqsVf7fO479xDuOXsom7eXM/a+WVz22FyWr98WdDQRkUBUVDk5KoBERCTFZfwkCHUxM048qCvH9u/C5Jmf8NfipbxcsoaLjuzNJUf3oX3L3KAjiogkTKgHSEPgRETqo96f5Kav8mKQn5vNFcf1478/PZoTD9yLe4qXMvKW/3JP8RK2lX1joXkRkbSkabBFROqn4if56UjWAF3bt+TPY4Yw44qRHNqrE7c+/yFH3VrMQ28uY2dFZdDxRETiqqLSyc1RD5CISG1U/KSGuBZAZjbKzD40syVmdn2U+68xs/nhn0VmVmlmneKZqTkM3Lsdfzv/UJ669DD67NmaX01fzLF/eI2p73xOWYVmjBOR9FRWWUWOeoBERKIacMPEoCNIjOJ2JDOzbOBuQiuyDwTGmtnAyDbufpu7D3b3wcDPgNfcfX28MjW3Q3p2Ytr4ETx8wTA6t8njZ/98j6Nve5WH3lzGjnL1CIlIeqnQNNgiIrU668nCoCNIjOL5Vd4wYIm7f+LuZcA0YHQd7ccCU+OYJy7MjKP225NnJh7BwxcMo3vHlvxq+mKOvOVVJs9cytadOkdIRNKDpsEWEYlOQ99SSzxngesGLI+4vgIYHq2hmbUCRgGX13L/eGA8QEFBAcXFxTEFKC0tjbltc5nYHz7oks9zS8v43YwPuPOlD/h2r1xG7FGW8Cy1CWK71EZZolMWSTburmmwRUSiUPGTeuJZAEUbJ+G1tD0ZeKO24W/uPhmYDFBYWOhFRUUxBSguLibWts2pCJgAzPt8A3f/dwn//GAN//nUOP/IvbjgiN7s2bZFwjNFCmq7RKMs0SmLJJvyytDuO09D4EREdlHxk5ri+VXeCqBHxPXuwKpa2o4hBYe/1WfoPh352/mH8u8rjuSgztlMem0pR97yX258dhErNmhBVRFJHRVVoQle1AMkIhKy8tbioCNII8XzSDYb6Gdmvc0sj1CRM71mIzNrDxwNPBvHLIE6YO/2XDY4n1euPppTBndj6jufU3RbMT95YgFL1mwJOp6ISL2qe4B0DpCISMgDUz4NOoI0UtyOZO5eQeicnheAEuAJd19sZhPMbEJE01OBF919a7yyJIt992zDLWcczMxrj+Hcw3ox473VHH/7TCY8MpeFKzYGHU9EpFbllaEeIM0CJyKioW+pLp7nAOHuM4AZNW6bVOP6g8CD8cyRbLq2b8mNJw/k8mP78uAbn/Lgm8t4fvEXjOzXmcuK+jJi306Y6UOGiCSPCvUAiYgAKn7SgY5kAerUOo+rv70/b1x/LD87oT8lq7cw9r5ZnPbXN3n5/S+pqqptzggRkcSq7gHKydKXMyKSuVT8pAcVQEmgbX4ulxzdh/9ddww3nXIga7fs5KKH53DCHa/z7PyVVIQ/eIiIBKW6AMrL0WFDRDKTip/0oSNZEsnPzeacET0p/mkRt585CMf58bT5HPvH13js7c/YUV4ZdESRwJjZKDP70MyWmNn1Ue5vb2bPmdkCM1tsZuOCyJmuqidByMnSYUNEMk+fUUOCjiDNSEeyJJSTncWpQ7rz/I+PYvI5h9CxdR6/eHoRR936KpNnLqV0Z0XQEUUSysyygbuBE4CBwFgzG1ij2UTgfXcfRGg5rj+GZ6CUZrBrCJwmQRCRDNNn1BDO/+yKoGNIM1IBlMSysoxvH7AXz1x2OI9fNJx+BW343YwPOOLm//Knlz5i47ayoCOKJMowYIm7f+LuZcA0YHSNNg60tdAMIm2A9YC+LWgmu4bAaRIEEckwKn7ST1xngZPmYWYc3rczh/ftzPzlG7nn1SXc+crHTPnfp1w4sjcXHNmbdvm5QccUiaduwPKI6yuA4TXa3EVorbFVQFvgTHf/xgl0ZjYeGA9QUFBAcXFxvS9eWloaU7tECCrLRxtCQ3AXL3oP+yI70Cy1SaY8yhJdMmURiYXO+0lPKoBSzOAeHZh8biEffLGZP7/0MX9++WOmvLGMS47el/MP70WrPP1JJS1FG3dVc5rE7wDzgWOBPsBLZva6u2/e7UHuk4HJAIWFhV5UVFTvixcXFxNLu0QIKkve0nXw9tsUDh3MiH33CDRLbZIpj7JEl0xZROqj4id9aSxDiuq/VzsmnXMIz11+JEP36cCtz3/IyFte5f7XP9FkCZKOVgA9Iq53J9TTE2kc8E8PWQJ8CvRPUL60V75rHSCdAyQi6U/FT3pTAZTiDureninjhvHUpYfTv2tbfvPvEo6+7VUef/tzTZ8t6WQ20M/MeocnNhhDaLhbpM+B4wDMrADYH/gkoSnTWPX+RAuhiki6W3lrcdARJM50JEsTh/TsyGMXjWDqxSPo3rEVP3/6PUbd8Tovv/8l7lpQVVKbu1cAlwMvACXAE+6+2MwmmNmEcLObgMPN7D3gFeA6d18XTOL08/VCqDpsiEj6em7cqzww5dOgY0ic6YSRNHNYnz14csJhvLD4S259/gMuengOw3p34hcnDmBQjw5BxxNpNHefAcyocdukiMurgG8nOlemqB4Cl5ejIXAikr7mz1oWdARJABVAacjMGHXgXhw3oAvTZi/njpc/YvTdb3DSwV259js6JUJEGk49QJIuzOwB4CRgjbsfGOV+A+4ATgS2Aee7+7zEpkxfL3y5lseWrWTtzjIGtmvL+D492L9tm6BjAcl53k9lxQ4+mncfKz58jqqqcrru+y36HzqRvPwOQUdLaTqSpbHc7CzOGdGT4muO4Ypj+/JKyRqO+1Mx0z4oY8uO8qDjiUgKqQj3AGkhVEkDDwKj6rj/BKBf+Gc88NcEZMoIUz9fxT0fLOOM7e35vXdn/43ZXDpvMR+Xbg00V59RQ5Ky+HF3Zs2YyLrl8+l78M/of8hv2LZxPa8//UMqK3YGHS+lqQDKAG1a5HD1t/en+JoiTh3SjReWlXPMH17jybkrqKrS+UEiUr8yLYQqacLdZxJaKLk2o4GHwzNKzgI6mFnXxKRLX2VVVfzt0+X8gr05NKsNBZbLyVkdOd078OAnKwLNlqwLna7/4l1KNyxj/6G/ok2H/WnVtjf7HnQ12dltWbX0+aDjpTQdyTJIQbt8bj1jEDcclk/3ji356T8WcPqkN1m4YmPQ0UQkyVXPApejAkjSX7SFl7sFlCVtfLljJy0weljebrcPtdZ8sKU0oFTJOeyt2qZ179N+j6FY1tdnrJgZ7Tsfysa1iwNMlvxuKBlX5/06kmWgfdtn889LD+e2Mw5m+fptjL77Da57ciHrStWdKiLRaR0gySCxLLwcamg23szmmNmcDWUVcY6V2vbIy6PUq9jou2+nT30ne7fMDyRTMhc/AK3admPrliXfuH3blqW0atcjyiPkwZ531lv8gAqgjJWVZXy/sAf//WkRFx7Rm6fmreCYPxTz8FvLqNSwOBGpobxK6wBJxohl4WUA3H2yuxe6e2HHPM0rVZdWOdmcvFcXbudL1nk57s77vp2H+YqzeyW+gy3Zix+ALvuMxKt28PmHU6is3EFVVQVffDadzV/Np8d+3ws6XtK5oWQcS59/N6a2erdmuHb5ufzypIGMGdaDX01fzI3PLubpd1fy+9MOov9e7YKOJyIJVlXl/OHFD1m7Zfce4fdXbwZUAElGmA5cbmbTgOHAJndfHXCmtPDj/XpxV9YyLl/1OVlAm5wcruzbmxGdOiQ0RyoUPwBZWTkcMXoK81/9FbNfOhUw2nceyBGjp5CX3z7oeEkjlh6fmlQACQB9u7Tl0QuH8/S7K7npX+9z0p3/Y/xR+3LFcf3Iz80OOp6IJMjqzTu4p3gp7Vvm0jpv9/f+yH6dyc7SEDhJbWY2FSgCOpvZCuBXQC7sWltsBqEpsJcQmga74Z+uJKrcrCyu2m9fJvbtxdaKStrn5pBlid2npErxU61lm7047OR7KS8rxasqVfhEaEzhU00FkOxiZpw2tDtF+3fht/8u4Z7ipfz7vdX89pSDOLJf56DjiUgClFeEhrr9v+8N5NQh3QNOI9L83H1sPfc7MDFBcTJSXlYWeXmJ701OteInUm5ecqyVlAwuGNebbtcWNek5NJZBvqFT6zz++INBPHbRcAz44d/e5uq/z2f91rKgo0kaMLOOZnaAme1rZtoHJRkteCoi6SiVix/52g0l45pc/IB6gKQOR/TtzPNXHsVd/13CpNeWMvPjtfz21IP4zgF7BR1NUoyZtSf0jepYIA9YC+QDBWY2C7jH3V8NMKKEabY3EUk3Kn5SX1OGu0Wjr/ikTvm52fz0O/vz3I+OpEvbfC55ZC5X/X0+m7aVBx1NUsuThNbVGOnu+7v7keGZk3oANwOjzezCYCMKfN0DpMkORCQdqPhJbY+fMafZix9QD5DEaEDXdjwz8QjufnUJd7+6hDeWrOOW0w/mmP5dgo4mKcDdj6/jvrnA3ATGkTpUVGnBUxFJDyp+UtsNJeMouSk+z60CSGKWl5PFVcfvx/EDC/jJEwsY9+BsflDYnV+eNJB2+blBx5MUYmZ7Aj8GWgJ/dfdvrvQmgSir0BA4EUl9Kn5SVzx6fGrSV3zSYAd2a8/0Hx3BZUV9eHLuCkbdPpM3l6wLOpaklj8CM4HngakBZ5EIFVrwVERSWJ9RQ1T8pKgHe96ZkOIHVABJI7XIyebaUf156tLDyc/L5uy/vc3N//mAsvAUuiKRzOx5MxsZcVMesCz80yKITBJdxa5JEHR4EJHUMuCGiZz/2RVBx5AGumBcb24oGcfS599N2GvqCCdNMmSfjvzrR0cy5tB9mPTaUs6Y9CafrtsadCxJPmcSmujgcTPrA9wA3EhoAoTLAk0muynbNQ22hsCJSOroMPkhznqyMOgY0kDNNa11Q+kcIGmyVnk5/P60gzh6v85c99R7fPfO1/n16AM5fWg3LMErPEtycvdNwE/NbF/gt8BKYGL49piY2SjgDiAbuN/db65x/zXA2eGrOcAAYE93X98Mv0LGqO4BysvR92MikhpW3lrMTbd/GnQMaYBEDXWrjY5w0mxGHdiV568cycHd2/PTfyzgimnz2bRd02ULhBc9vQ24CPgJ8CzwhJn9yMyyY3h8NnA3cAIwEBhrZgMj27j7be4+2N0HAz8DXlPx03Dl6gESkRRy04ApPDBFxU+qiNe01g2lAkiaVdf2LXnsohFc8539+c97qznxjteZ+5k+gwpTCU14MAt4xN1fd/fvAJuBF2N4/DBgibt/4u5lwDRgdB3tx6LJFRpF6wCJSKrQZAepY/CIXuFpre8OOgqgIXASB9lZxsRj+nJE385cMfVdzrx3Ftef0J8Lj+ytIXGZKx/4FGgNtKq+0d0fMrMnYnh8N0ILqVZbAQyP1tDMWgGjgMtruX88MB6goKCA4uLiel+8tLQ0pnaJEO8si5eHem3nvDOLpfl1F0HJtF0gufIoS3TJlEVSm4qf1HFDyTgoCTrF7lQASdwM7tGB5350JNf8YwG/+XcJc5Zt4NbvH6w1gzLTZcBtQBkwIfIOd98ew+OjVc5eS9uTgTdqG/7m7pOByQCFhYVeVFRU74sXFxcTS7tEiHeWz99aBosXM/LII+jcpu4J+pJpu0By5VGW6JIpi6SmDpMf4ie3a8bZVJAMQ91qozEOElftW+Zy7zmH8MvvDuClki/53l/+x+JVMZ/3LmnC3d9w99Pdfay7L2jEU6wAekRc7w6sqqXtGDT8rdGqp7LXEDgRSTYrby1W8ZMCqqe1TmY6wkncmRkXjdyXaeNHsL28ktPueZMnZi+v/4GSNszsOTM7ycy+0f0XniDh12Z2QR1PMRvoZ2a9zSyPUJEzPcpztQeOJjTJgjRCRVX1OkAarioiyUOTHSS/6vN8gpjWuqHiOgSuvmlrw22KgD8DucA6dz86npkkOIf26sS/rxjJldPmc+1TC3ln2XqO71TbKCZJMxcDVwN3mNl6YC2h84J6AUuBu9y91qLF3SvM7HLgBUL7kwfcfbGZTQjfPync9FTgRXfXYlSNVKFJEEQkiQy4YaLW90kByXieT13iVgBFTFt7PKHhK7PNbLq7vx/RpgNwDzDK3T83sy7xyiPJoXObFjx0wTDueOVj/vLfj3mnTRYHDN1G946t6n+wpCx3/wK4FrjWzHoBXYHtwEfuvi3G55gBzKhx26Qa1x8EHmx64sxVFl4HSNNgi0jQOkx+iLM05C2pJftQt9rEswdo17S1AGZWPW3t+xFtzgL+6e6fA7j7mjjmkSSRnWVcffx+DOnRgcsenc337nqDe84eyoh99wg6miSAuy8DlgUcQ2pRUVlFbrZpxkYRCdRNA6aAip+klaqFT7V4FkCxTFu7H5BrZsVAW+AOd3+45hM1ZtpaSK7pNpXlmwz46SDn/g/KOfu+WYztn8dx++QE9sErWbYLKIsEp7yyipwsDX8TkWD0GTWE8z+7IugYUovBI3px8pRjgo7RZPEsgGKZtjYHOAQ4DmgJvGVms9z9o90e1IhpayG5pttUlloUF/PCyUdw1bT5PFqyhvLWBfz6lANokZMdQJTk2S7KIkEpr3RNgCAigbhpwBT4LOgUUptUO8+nLvEsgGKZtnYFoYkPtgJbzWwmMAj4CMkY7fJzue/cQv700kfc9eoSPl6zhUk/PIQu7fKDjibNzMxOAma4u8Y1JKnyyipNgCAiCaeFTZNXqg93iyaeR7lYpq19FhhpZjnh1duHkza1pTREVpbx0+/szz1nD6Vk9RZOvut/LFi+MehY0vzGAB+b2a1mNiDoMPJNFZVOjnqARCRBbhowRcVPkrqhZFxaFj8QxwLI3SuA6mlrS4AnqqetjZi6tgR4HlgIvENoquxF8cokye/Eg7ryz8sOJzc7izMnv8Xzi1YHHUmakbv/EBhCaOrrKWb2lpmNN7O2AUeTMPUAiUgidJj8kAqfJFW9nk86i2kInJm1Bra7e5WZ7Qf0B/7j7uV1PS7GaWtvA25rUGpJawO6tuOZiUdw8cNzuPSxefzshP5cPHJfzUqVJtx9s5k9Rei8vysJrd1zjZnd6e5/CTScUF7lKoBEJK40w1vySqfzfOoS6zlAMwkNVesIvALMAc4Ezo5XMMlsndu0YOrFI/jJPxbwuxkf8Om6bfx69AH6YJbizOxk4AKgD/AIMMzd14SHwJYAKoACVl5RpUkQRCQutKhp8kr3Hp+aYi2AzN23mdmFwF/c/VYzezeewUTyc7P5y5gh9OzUinuKl7JiwzbuPnso7fJzg44mjfd94HZ3nxl5Y3j/ckFAmSRCRZWmwRaR5nfTgCnwZNAppKY/XpXFxvHnBR0j4WIugMzsMEI9Phc28LEijZaVZVw7qj+99mjNz59+jzP++iYPnH8o3Tu2CjqaNM6vgF0ndplZS6DA3Ze5+yvBxZJq5ZVObo4KIBFpHjrPJzlVr+ezcXzQSYIR61HuSuBnwNPhiQz2BV6NWyqRGn5waA8eumAYqzft4JS732Thio1BR5LG+QcQOfC7MnybJInyyipyszQETkSaRpMcJK8bSsalxWKmTRFTAeTur7n799z9FjPLIrR2j5bplYQ6om9n/nnp4eTnZjFm8ixmfrQ26EjScDnuXlZ9JXw5L8A8UkNFpSZBkORlZsc3w3OMMrMPzWyJmV0f5f4iM9tkZvPDPzc29TUzzU0DpvATTXKQdB7seWfGnetTm5iOcmb2uJm1C88G9z7woZldE99oIt/Ur6At/7z0cHru0ZoLHpzNs/NXBh1JGmatmX2v+oqZjQbWBZhHaiirrNI6QJLM/taUB5tZNnA3cAIwEBhrZgOjNH3d3QeHf37dlNfMJFrTJ3ndUDKOpc/r9P1qsZ7HMzA8de3ZhKa1vg6Yi6avlgB0aZfP3y8ZwfiH5/DjafNZu2UnF43cN+hYEpsJwGNmdhdgwHLg3GAjSaSKKq0DJIHra2Y1F06H0D5jjyY+9zBgibt/AmBm04DRhL7clUboM2oI53+mQUHJ6vEz5lBy091Bx0g6sRZAuWaWC5wC3OXu5Wbm8YslUrd2+bk8OG4YV/19Pr/5dwlrS3dy/aj+Wisoybn7UmCEmbUhNLvklqAzye7KK1zTYEvQ2gD3AqU1bjdCBUxTdCP0xUu1FcDwKO0OM7MFwCrgp+6+ONqTmdl4YDzAXi1aNDFa6rlpwBT4LOgUUpsbSsZRclPQKZJTrAXQvcAyYAEw08x6ApvjFUokFvm52dx11lB+NX0R9772Ceu2lHHz6Qfp2+skZ2bfBQ4A8qsLVg0xSR7lVVXk6D0kwdoKbHP312reYWYfNvG5o1X3Nb/QnQf0dPdSMzsReAboF+3J3H0yMBlgYLs2GfPFsIa5JTed51O/mAogd78TuDPips/MLLOnj5CkkJ1l3DT6QLq0zedPL33E+q07ufvsobTK0yztycjMJgGtgGOA+4EzgHcCDSW7Ka+sIk8FkATrY3d/1cwGunvNoWlNnZBgBdAj4np3Qr08u7j75ojLM8zsHjPr7O4Zf76iCp/k9mDPO3WeT4xinQShvZn9yczmhH/+CLSOczaRmJgZVxzXj9+dehCvfbSWc/72Dpt3lAcdS6I73N3PBTa4+/8Bh7H7hxEJWEWlk6NpsCU5PGFm11lISzP7C/D7Jj7nbKCfmfU2szxgDLDb+UZmtpeFu6fNbBihz0pfNfF1U9aAGyZqcoMkN3hEL01y0ECxfs33ALAF+EH4ZzOgd4IklbOG78NdZw1lwfKNnH3f22zYWlb/gyTRdoT/3WZmewPlQO9YHljf1LXhNkXhaWsXm9k3hs9I/bQQqiSR4YS+IHmTUOGyCjiiKU/o7hXA5cALQAnwRHh9wwlmNiHc7AxgUfgcoDuBMe6eMcPbqlUXPmc9WRh0FKmD1vRpnFjHCfVx99Mjrv+fmc2PQx6RJjnxoK7k52Yx4dF5jJk8i0cuGkaXtvlBx5KvPWdmHQjNIDmP0Nj7++p7UMTUtccTGsIy28ymRw6PCT/vPcAod//czLo0f/z0p4VQJYmUA9uBlkA+8Km7N3lxGXefQWhG28jbJkVcvgu4q6mvk6oG3DAxVPQ8GXQSqY/O9Wm8WL/m225mR1ZfMbMjCO2URJLOsf0LmHL+oXy+fhtj7p3Fqo36r5oMwosov+LuG939KaAn0N/dYxnTv2vq2vDiqdVT10Y6C/inu38O4O5rmjF+xqio1DTYkjRmE/qscShwJKE1e/SxPE6eG/eqenxSxA0l41T8NFGsPUATgIfNrH34+gbgvPhEEmm6I/p25pELhzFuymy+P+ktpl48gn32aBV0rIzm7lXh8wcPC1/fCeyM8eGxTF27H6Ep+4uBtsAd7v5wzSeKnLa2oKCA4uLiel+8tLQ0pnaJEO8sO8srWbVyBcXF9dePybRdILnyKEt0DcxyobvPCV/+AhhtZufEJVgG23Vuz6xlgeaQ+l0wrjfdri0KOkZaiHUWuAXAIDNrF76+2cyuBBbGMZtIkxT26sRjFw/n3Afe4fv3vsljF42gb5c2QcfKdC+a2emEemoaMqY+lqlrc4BDgOMIDZl5y8xmuftHuz0oYtrawsJCLyoqqvfFi4uLiaVdIsQzi7tT+cIM+vbuSVHR/oFmaYxkyqMs0TUkS0TxE3nbI82dKRPtGuYmKeOGknFwbdAp0keDxjm4++aI6SGvjkMekWZ1cPcOTBs/gsoqOPPet3h/lZavCtjVwD+AnWa22cy2mFksf5R6p64Nt3ne3beGp6udCQxqjtCZorLKcUfrAImkqerZ3FT8pI4LxvXWcLc4aMpRTmfJSkrov1c7nrhkBHk5WZx9/yxKVqsICoq7t3X3LHfPc/d24evtYnhovVPXAs8CI80sx8xaERoiV9K8v0F6q6gKdarlZGv3LpJONI11avrjVVka8hYnTVktMuOmhJTUte+ebZh68QjGTJ7F2fe/zdSLR7D/Xm2DjpVxzOyoaLe7+8y6HufuFWZWPXVtNvBA9dS14fsnuXuJmT1PaGhuFXC/uy9q3t8gvZVVhibY0kKoIqlv5a3FPDDl06BjSCPdUDKOjeODTpG+6iyAzGwL0QsdIzTGXiRl9OrcmqnjRzBm8lucdd8spo4fwX4FKoIS7JqIy/mEZnebCxxb3wPrm7o2fP02QlNsSyNUVIZ7gDQNtkhK2u3cHhU/KUnD3RKjzgLI3fXpUNJK786td/UEnXXfLKZePIJ+KoISxt1PjrxuZj2AWwOKIzVUhHuAtBCqSGrZNbxNk4SnNBU/idOUIXAiKWnfPduEe4JmMfa+t5k2fjh9u6gICsgK4MCgQ0hI9RC43CwVQCLJTuf0pBcVP4mlAkgyUp+Ic4JCRdCIoCNlBDP7C18Pq80CBgMLAgsku6keApeboyFwIslI5/WkHxU+wVABJBmrb5c2TL14OGPvm8XYybO4erC+9U6AyHU9KoCp7v5GUGFkd+XhHqAc9QCJJI3dih4VP2lFxU9wVABJRutX0JbHLx7B2MmzuHX2Do48fBvdO7YKOlY6exLY4e6VAGaWbWat3H1bwLkEKK/uAdIscCKBUtGT/lT8BEsFkGS8/Qra8siFwznjntc5+/63+cclh9GlXX7QsdLVK8C3gNLw9ZbAi8DhgSWSXap7gHK1DpBIwu12To+KnrSlwic56Gs+EWDg3u24+pB81m7ZyTl/e4cNW8uCjpSu8t29uvghfFldbkmioio8BE49QCIJsfLWYi1SmkFU/CQP9QCJhPXtmM195w5i3IOzOX/KOzx60XDa5ucGHSvdbDWzoe4+D8DMDgG2B5xJwsoqqofAqQdIJB52W6cH1NOTQVT8JBcVQCIRjujbmXvOGsolj87lwofm8PAFw8jPzQ46Vjq5EviHma0KX+8KnBlcHIlU3QOkc4BEmk+fUUM4/7MrQle0Tk9GUvGTfFQAidTwrYEF/OkHg7jy7/O59NG53HtOIXlaGLJZuPtsM+sP7A8Y8IG7lwccS8K+PgdI/99FmmK3IW2fBZdDgqXCJ3npKCcSxejB3fjdqQfx6odruerv86ms8vofJPUys4lAa3df5O7vAW3M7LKgc0lI9SxwOVkaAifSUKvze+t8HtlFxU9yUwEkUouxw/bhFycO4N/vreYXT7+Hu4qgZnCxu2+svuLuG4CLg4sjkaoXQlWPp4hI46n4SX4aAidSh4uP2pdN28u569Ul7Nm2BT/59v5BR0p1WWZmHq4mzSwbyAs4k4R9vRCqeoBERBpDxU9qUAEkUo+ffHs/1pXu5C//XUKXti0457BeQUdKZS8AT5jZJMCBCcDzwUaSajoHSESk8VT8pI64HuXMbJSZfWhmS8zs+ij3F5nZJjObH/65MZ55RBrDzPjNKQfyrQEF3Dh9MTPeWx10pFR2HaHFUC8FJoYvXxNoItml+hwgFUAiIg2j4ie1xO0oFx7acjdwAjAQGGtmA6M0fd3dB4d/fh2vPCJNkZOdxV/GDmHoPh25ctp83lr6VdCRUpK7V7n7JHc/w91PBxYDfwk6l4R8vRCqhsCJiMRi8IheKn5SUDy/5hsGLHH3T9y9DJgGjI7j64nEVcu8bP52XiE992jF+Ifn8P6qzUFHSklmNtjMbjGzZcBNwAcBR5KwsgoNgRMRidUF43pz8pRjgo4hjRDPo1w3YHnE9RXh22o6zMwWmNl/zOyAOOYRabIOrfJ46IJhtMnP4bwp77B8/bagI6UEM9vPzG40sxLgLkL7A3P3Y9w9ph4gDamNv4qq6iFw6gESEanLH6/Kotu1RUHHkEaK5yQI0Y6gNecRngf0dPdSMzsReAbo940nMhsPjAcoKCiguLg4pgClpaUxt403ZYkuVbNcfqDx27d38P27ivn5iJa0y2veD4ypul3q8AHwOnCyuy8BMLOrYn1wxJDa4wkVT7PNbLq7v1+j6evuflJTw2aqcvUAiYjU6/Ez5lAy/u6gY0gTxLMAWgH0iLjeHVgV2cDdN0dcnmFm95hZZ3dfV6PdZGAyQGFhoRcVFcUUoLi4mFjbxpuyRJfKWfY/aD1n3/82D3ycx7TxI8jPzQ4sSzw1U5bTgTHAq2b2PKEhsQ2pGncNqQUws+ohtTULoGZX/OEa/jB7B39b+nazPef3Bu3N9wt71N+whhUbtnHHvObNEumzr0I9mpoGW0TiadNXH7Kj9Avadx5Ifus9g47TIA/2vJOSm95N6GtWVDkLN22m3J1B7duSn918nzcyVTwLoNlAPzPrDawk9OHnrMgGZrYX8KW7u5kNIzQkT2eXS0oo7NWJO8YM4dLH5nLltPncffZQsvXBMSp3fxp42sxaA6cAVwEFZvZX4Gl3f7Gep4g2pHZ4lHaHmdkCQl+2/NTdF9ds0NAe5XfXVFBaVsH2NevriRiblaVVrFm3nj1Llzb4sW+uquDdNZXss309uXHopMkFjuqew2uvvRZT+2TqqYTkyqMs0SVDFjMbBdwBZAP3u/vNNe638P0nAtuA8919XsKDpqGyHRt4e8bllG5aTqs2+7BlQwk9D/gBBx5+LaHNntxuKBnH0pLEvubCTZu5buGHtPcs8shipZfxywF9OKZL58QGSTNxK4DcvcLMLie07kc28IC7LzazCeH7JwFnAJeaWQWwHRhTvUCiSCoYdeBe/PK7A7npX+/z+xkl/PKkaBMdSjV33wo8BjxmZp2A7wPXA/UVQM02pLahPcpFwJBm7JE7+/5Z7Civoqjo8AY/ds2c5bBwIY9dejQ9OrVqljxNkUw9lZBceZQluqCzxDic9gRC+45+hL5o+SvRv3CRBpr3yi9pkd+T/YfejFk25WWbeP+da2nbsQ+9Bp4RdLw6BTHT2/bKSq6eX8Ll3oXhWW0A+Igd/L/3l7B/2zbs3TI/4ZnSRVwHerv7DHffz937uPtvw7dNChc/uPtd7n6Auw9y9xHu/mY884jEwwVH9OL8w3tx//8+5aE3lwUdJ2W4+3p3v9fdj42heUxDat29NHx5BpBrZkn3FVludhYV4QVHG6oivE5PXo7O0RFppFhmqB0NPOwhs4AOZtY10UHTTdmODaxb+Rb79L+IUB0KuXnt6dFvHJ+9/4+A09Vt8Ihegbzua2vX09da7Cp+APazfI6iLTO+WBtIpnSho6hIE5kZN5w0kG8NKOD/nlvMy+9/GXSkdLRrSK2Z5REaUjs9soGZ7RUeukIyD6nNycqirLJxHd3l4cJJ5+iINFosM9TGOostZjbezOaY2Zyd25tnmGy6Ki8rJTunJVlZLXa7PS9/D8p3bgooVf2CnOp6S0UFHf2bg7U6kc2W8ooAEqUPFUAizSA7y7hz7GAO7NaeH019l4UrNgYdKa24ewVQPaS2BHiiekht9bBaQkNqF4XPAbqTJB1Sm5djje4B2lUAaZY2kcaKZThtLG1CN7pPdvdCdy9s0bJTk8Ols1Ztu5Gd05JNX+1+OtXaFS/QZZ+RAaWqW9BTXR/asT2zfSulXrnrtnJ3ZlopI/boEFiudBDPSRBEMkqrvBz+dt6hnHrPG1zw4ByevuzwpDhPI12Eh7XNqHHbpIjLdxFaYyip5WRl7SpkGqq8egicCiCRxqp3OG2MbaSBzLIYdPQNzHnpWvbqeQqt2vZiw5o32bJxMUcf/feg40W1cfx5gb5+r9at+O7ee3Lt6hWcXNWePLJ43jaxb/vWDO/UIdBsqU5HUZFmtGfbFjw47lDKKioZ9+BsNm0vDzqSJJmcbNtVyDRUxa4eIA2BE2mkeofThq+fayEjgE3uvjrRQdNRQc+jGXnqo+TklbNx3Uz26HYgx/zgn0k5FXYQkx5Ec2W/3lx9wL4s7VTFwg5lnL1/D35/cH+yUmDWvGSmHiCRZta3S1vuPaeQcx94m0sfnctDFwzTwpKyS152E3qAqkKFk84BEmmcGGeonUFoCuwlhKbBTo5Pwmmi3R79GHT0jUHHqFOyFD8QOs94ZOdOjOysIZbNSZ/KROLgsD57cPNpB/Pm0q/41fTFJOGpKBKQnGyjoqrxkyBkGymxXoZIsophhlp394nh+w9y9znBJpZESqbiR+JHPUAicXL6Id1ZsraUvxYvpV+XNow7onfQkSQJ5DahB6iisgrNgC0iEh8qfjKHCiCROLrm2/uzdE0pN/3rfXp1bs0x+3cJOpIErCkFUHmlo9N/RESan4qfzKLvEkXiKCvLuP3MwfTfqx0/evxdPvpyS9CRJGC52bZrQdOGKq+sQqeTiYg0LxU/mUeHUpE4a90ih/vPK6RlXjYXPjSbr0p3Bh1JApSTlUVFlTfqvLDyyipydP6PiEizuWCchqdnIhVAIgmwd4eW3HduIWs272TCo3PZWVFZ/4MkLeWFT+JpzFTYFZWuHiARkWYS9EKnEhwdSkUSZHCPDvzh+4OYvWwDP//nIs0Ml6Gqp7BuzHlAZZVV5KgDSESkWQS90KkER5MgiCTQyYP2ZunaUv788sf07dKGS4v6BB1JEiwn3IXTmPOA1AMkItI8dN5PZlMBJJJgPz6uH0vWlHLrCx+w756t+c4BewUdSRIoLzyNW1kjeoAqqqq0CKqISBOp+BF9lyiSYGbGH74/iIO7d+DKafMpWb056EiSQLt6gKoaMwRO02CLiDTF42doXVtRASQSiPzcbO475xDatczh4ofnsH5rWdCRJEFywwVQeUVjhsBpIVQRkaYouenuoCNIEtChVCQgXdrlM/mcQtZs2cllj81t9OKYklpyw1045Y3oASqvrFIPkIhII2nom1RTASQSoEE9OnDzaQcx65P13PSv94OOIwmQ24RJEMornWytAyQi0mAqfiSSCiCRgJ02tDsXj+zNw299xtR3Pg86jsRZU6bBLq+s0ixwIiIN9GDPO4OOIElGs8CJJIHrTxjAB19s4cZnF9G3S5ug40gc5e5aCLURs8BVOvkqgEREGmTp8+8GHUGSjA6lIkkgO8u4a+xQundsxaWPzuWr7TofqCYzG2VmH5rZEjO7vo52h5pZpZmdkch8scrNqi6AGjEErkrnAImINISGvkk0KoBEkkT7Vrncd+4h7Civ4s53d7K9rDLoSEnDzLKBu4ETgIHAWDMbWEu7W4AXEpswdjnhCqZCQ+BEROJKxY/URodSkSTSt0tb7hgzmM83V3HdUwtxb3gvQZoaBixx90/cvQyYBoyO0u5HwFPAmkSGa4jqSRAatRBqpZOjSRBEROo1eESvoCNIEtM5QCJJ5rgBBZzeL5cnF6xiQNd2XFrUJ+hIyaAbsDzi+gpgeGQDM+sGnAocCxxa2xOZ2XhgPEBBQQHFxcX1vnhpaWlM7WLx6aZQz967CxbC6obtgrdu30FVS2+2LE3VnNulOSRTHmWJLpmySHo7ecoxQUeQJKYCSCQJfXffXHa07MytL3xA/73ackz/LkFHClq0bo+a3WN/Bq5z90qro5fE3ScDkwEKCwu9qKio3hcvLi4mlnaxKFi9Gd56nf4DDqDooK4Neqy99iL5ed5sWZqqObdLc0imPMoSXTJlkfSloW9SHw2BE0lCZsZtZwxiYNd2XDH1XZasKQ06UtBWAD0irncHVtVoUwhMM7NlwBnAPWZ2SkLSNcDXC6E2Zh2gKnK01xYRqZWKH4mFDqUiSaplXjaTzy0kLyeL8Q/PYdP28qAjBWk20M/MeptZHjAGmB7ZwN17u3svd+8FPAlc5u7PJDxpPb5eCLVx5wBpIVQRkeh03o/ESgWQSBLr1qElf/3hISzfsI0rpr5LZSN6DdKBu1cAlxOa3a0EeMLdF5vZBDObEGy6hsnJbtw6QO5OeZV6gEREaqPzfiRWOpSKJLlhvTvxf987kNc+WsutL3wQdJzAuPsMd9/P3fu4+2/Dt01y90lR2p7v7k8mPmX9dg2Ba+A6QJVVjjtaB0hEJAoNfZOGUAEkkgLOGr4PZw/fh3tf+4Rn568MOo40wdcLoTasB6gi3PunHiARkd1dMK530BEkxehQKpIifnXyAQzr1YnrnlrIopWbgo4jjZSbU30OUMN6gKrXDdI5QCIiu+t2bVHQESTFqAASSRF5OVnc88OhdGqVx/iH57CudGfQkaQRcrJCBUxDF0KtLpiytdcWEdlFQ9+kMXQoFUkhndu04N5zCvlqaxmXPTavwcOoJHhfzwLXsB6g6r91jjqAREQADX2TxlMBJJJiDurenlvPOJh3Pl3Pr597P+g40kDZWUaWNfwcoOr26gESaRwz62RmL5nZx+F/O9bSbpmZvWdm881sTqJzSuw09E0aS4dSkRQ0enA3LjlqXx6Z9RlT3/k86DjSQDnZWZRXNW4IXPUQOhFpsOuBV9y9H/BK+HptjnH3we5emJho0lAa+iZNEdcCyMxGmdmHZrbEzGrd0ZjZoWZWaWZnxDOPSDq5dlR/jtpvT258dhFzlq0POo40QF52VqOHwGkabJFGGw08FL78EHBKcFFEJEhxK4DMLBu4GzgBGAiMNbOBtbS7hdAChyISo+ws4y9jhtCtQ0smPDqP1Zu2Bx1JYpSTbY0YAqdpsEWaqMDdVwOE/+1SSzsHXjSzuWY2vq4nNLPxZjbHzObs3K4vohJFvT/SVPE8lA4Dlrj7J+5eBkwj9O1LTT8CngLWxDGLSFpq3yqX+84tZHtZBZc8Mpcd5ZVBR5IY5GZnNXghVPUAidTPzF42s0VRfqJ9/qjNEe4+lNAXuBPN7KjaGrr7ZHcvdPfCFi07NTm/1E8TH0hzyInjc3cDlkdcXwEMj2xgZt2AU4FjgUNre6LwNzDjAQoKCiguLo4pQGlpacxt401ZolOW6Bqa5cIDcrjz3U1c8NeXufigPKwZ14pJpu2SLnKzGt4DVFGlAkikPu7+rdruM7Mvzayru682s67U8sWru68K/7vGzJ4m9IXuzLgElgbTxAfSHOJZAEU7TNf8yvPPwHXuXlnXBzZ3nwxMBigsLPSioqKYAhQXFxNr23hTluiUJbqGZikCsvf4mNtf/ohjh/TjopH7BpZF6pebk0VFAwugsgpNgiDSRNOB84Cbw/8+W7OBmbUGstx9S/jyt4FfJzSl1EpD36S5xHMI3AqgR8T17sCqGm0KgWlmtgw4A7jHzE6JYyaRtPWjY/sy6oC9+N2MEv738bqg40gdcrKswUPgdvUA6Rwgkca6GTjezD4Gjg9fx8z2NrMZ4TYFwP/MbAHwDvBvd38+kLSyGw19k+YUzx6g2UA/M+sNrATGAGdFNnD3Xf+bzexB4F/u/kwcM4mkraws448/GMRp92xl4uPzmH75EfTco3XQsSSK0DlAjZwGWx1AIo3i7l8Bx0W5fRVwYvjyJ8CgBEeTGGjomzSnuH2X6O4VwOWEZncrAZ5w98VmNsHMJsTrdUUyWesWOUw+9xAAxj88l607KwJOJNE0pgAq00KoIpKhNPRNmltcD6XuPsPd93P3Pu7+2/Btk9x9UpS257v7k/HMI5IJeu7RmrvPGsrHa7Zw9RPzqapq2FArib+cbKOigX+Xr3uA1AUkIiLSFPouUSQNHdmvMz8/cQAvLP6Sv/x3SdBxpIbc7CzKKhq6DpB6gEQk86j3R+JBh1KRNHXhkb05bUg3bn/5I15c/EXQcSRCbiN6gLQOkIhkGk18IPGiAkgkTZkZvzvtIA7u3p6r/j6fj77cEnQkCcvNbvg02NWzxuVory0iGUITH0i86FAqksbyc7O595xDaJmXw/iH57BpW3nQkRrNzEaZ2YdmtsTMro9y/2gzW2hm881sjpkdGUTOWORkZVHW2GmwdQ6QiGQADX2TeFIBJJLmurZvyb3nDGXlxu1cPnUelSk4KYKZZQN3AycAA4GxZjawRrNXgEHuPhi4ALg/oSEbIC/HGrEQqs4BEhERaQ46lIpkgEN6duLXow/k9Y/XcevzHwQdpzGGAUvc/RN3LwOmAaMjG7h7qbtXV3etgaSt9HKyGrEOUJXWARKRzKDeH4m3eC6EKiJJZOywfVi8ahP3zvyEAV3bccqQbkFHaohuwPKI6yuA4TUbmdmpwO+BLsB3oz2RmY0HxgMUFBRQXFxc74uXlpbG1C5WX63dyZatlQ16zo+WlgGwY9vWZs3SFM29XZoqmfIoS3TJlEWS0wXjesO1QaeQdKcCSCSD3HjSAXz0ZSnXPbWQPnu24aDu7YOOFKto/R7f6OFx96eBp83sKOAm4FtR2kwGJgMUFhZ6UVFRvS9eXFxMLO1i9fxXC/l4y5oGPee88o/g449p17Z1s2ZpiubeLk2VTHmUJbpkyiLJSRMfSCJoCJxIBsnLyeKes4eyR+s8LnlkDutKdwYdKVYrgB4R17sDq2pr7O4zgT5m1jnewRqjcQuhVpGbbZgmQRCRNKWhb5IoKoBEMkznNi2YfG4h67eVcdmj8xq8IGdAZgP9zKy3meUBY4DpkQ3MrK+FqwMzGwrkAV8lPGkMcrOzKG/EQqg5Wdpli4iINJWOpiIZ6MBu7bnl9IN5Z9l6/u+5xUHHqZe7VwCXAy8AJcAT7r7YzCaY2YRws9OBRWY2n9CMcWdGTIqQVHKzsyivavg6QDlaBVVE0pR6fySRdA6QSIYaPbgb76/ezL2vfcIBe7fnrOH7BB2pTu4+A5hR47ZJEZdvAW5JdK7GyM02Khq4DlB5ZRV5mgNbRNLQ4BG9Ql9tiSSIjqYiGeza7/Tn6P325FfTFzF72fqg42SMnKwsKqqchnRQVagHSETS1MlTjgk6gmQYFUAiGSw7y7hzzBC6dWjJpY/OZdXG7UFHygh5OaFdb3kDeoHKq6rIVQ+QiKSZB3veGXQEyUA6mopkuPatcrnv3EJ2lFdxySNz2VFeGXSktJeTFerJachiqOWVrgJIRNLO0uffDTqCZCAdTUWEfgVtuf3Mwby3chPXPrmwQUOzpOGqC5mGnAdUPQ22iEi60MQHEhQVQCICwPEDC7jmO/szfcEq7n51SdBx0lp1IVPWoB4gTYMtIulj8IheQUeQDKZZ4ERkl8uK+vDxl1v4w4sf0WfPNpxwUNegI6WlnOoeoAZMhR0aAqceIBFJD5r4QIKkAkhEdjEzbj79YD5bv42rn1hAj06tgo6UlqqHwI2dPCvm83pWbNjOwL3bxTOWiIhIRlABJCK7yc/NZvI5hVz9xHxa5WUHHSctHd5nD04d0o2dFbFPONGvoA2jDuwKGz6KYzIRkfjTuT8SNBVAIvINe7ZtwSMXDgfg84CzpKO9O7Tk9jMHN+qxxcUqgEQkdan4kWSgM2pFRERERCRjqAASERERkbjToqeSLFQAiYiIiEjcadFTSRYqgEREREQkrnTujyQTFUAiIiIiIpIxVACJiIhI2jOz75vZYjOrMrPCOtqNMrMPzWyJmV2fyIzpSr0/kmxUAImIiEgmWAScBsysrYGZZQN3AycAA4GxZjYwMfHSkyY+kGSkdYBEREQk7bl7CYCZ1dVsGLDE3T8Jt50GjAbej3vANKWJDyQZpVwBNHfu3HVm9lmMzTsD6+KZpwGUJTpliS7VsvRMRJDm1oD9Sar9PRIlmbJAcuVRluiSfX/SDVgecX0FMLy2xmY2Hhgfvrrzmbv3WxTHbI0V6N//mdrvSqb/lzUlazblapha9yUpVwC5+56xtjWzOe5e6zjfRFKW6JQlOmVJjFj3J8m0DZSldsmUR1mii3cWM3sZ2CvKXb9w92djeYoot3ltjd19MjA5/NpJs50jKVfDJWs25Wo+KVcAiYiIiETj7t9q4lOsAHpEXO8OrGric4pIktEkCCIiIiIhs4F+ZtbbzPKAMcD0gDOJSDNL9wJoctABIihLdMoSnbIkl2TaBspSu2TKoyzRBZbFzE41sxXAYcC/zeyF8O17m9kMAHevAC4HXgBKgCfcfXGML5FM2zmScjVcsmZTrmZi7rUObRUREREREUkr6d4DJCIiIiIisosKIBERERERyRhpWQCZ2Sgz+9DMlpjZ9Ql+7R5m9qqZlZjZYjP7cfj2/2dmK81sfvjnxATlWWZm74Vfc074tk5m9pKZfRz+t2MCcuwf8bvPN7PNZnZloraLmT1gZmvMbFHEbbVuBzP7Wfj/z4dm9p0EZLnNzD4ws4Vm9rSZdQjf3svMtkdsn0kJyFLr3ySe2yVZaX+yWx7tT9D+pIFZ0nZ/YmbfD78vq8ys1imAE70PifU9Ge39HKc8df7+FnJn+P6FZjY0XlkamKvIzDZF/N+9MUG5vvE+qnF/INsrxmyBbLNGcfe0+gGygaXAvkAesAAYmMDX7woMDV9uC3wEDAT+H/DTALbHMqBzjdtuBa4PX74euCWAv9EXhBaoSsh2AY4ChgKL6tsO4b/XAqAF0Dv8/yk7zlm+DeSEL98SkaVXZLsEbZeof5N4b5dk/NH+5Bt5tD9x7U8amCVt9yfAAGB/oBgorOP/Z0L3IbG+J6O9n+OQpd7fHzgR+A+hNZhGAG8n4G8XS64i4F8B/L/6xvso6O3VgGyBbLPG/KRjD9AwYIm7f+LuZcA0YHSiXtzdV7v7vPDlLYRmkemWqNeP0WjgofDlh4BTEvz6xwFL3f2zRL2gu88E1te4ubbtMBqY5u473f1TYAmh/1dxy+LuL3po9iGAWYTWnoi7WrZLbeK6XZKU9if10/4kRPuTDNufuHuJu39YT7Mg9iFBvycjxfL7jwYe9pBZQAcz65oEuQIRw/soiO0Va7aUkY4FUDdgecT1FQT0gcHMegFDgLfDN10e7q58IBHDRMIceNHM5prZ+PBtBe6+GkIfsIAuCcpSbQwwNeJ6ENsFat8OQf8fuoDQtzvVepvZu2b2mpmNTFCGaH+ToLdLEJLmd9b+pFban9RN+5NgBfF7xvqejPZ+bm6x/P5BbKNYX/MwM1tgZv8xswPinClWyf7eScZt9g3pWABZlNsSPte3mbUBngKudPfNwF+BPsBgYDXwxwRFOcLdhwInABPN7KgEvW5UFlpY7nvAP8I3BbVd6hLY/yEz+wVQATwWvmk1sI+7DwGuBh43s3ZxjlHb3yQp3lsJlhS/s/Yn0Wl/Us8La3/SZGb2spktivITa29BXH7PZsgFiXk/x/L7B/F/IZbXnAf0dPdBwF+AZ+KcKVbJ/N5J1m32DTlBB4iDFUCPiOvdgVWJDGBmuYQ+rDzm7v8EcPcvI+6/D/hXIrK4+6rwv2vM7GlC3b5fmllXd18d7jZdk4gsYScA86q3R1DbJay27RDI/yEzOw84CTjOw4Np3X0nsDN8ea6ZLQX2A+J2wmodf5PA31sBCPx31v6kTtqf1EL7k+bh7t9q4lPE5fesK5eZxfSerOX9PLOp2WqI5fcP4v9Cva8Z/rKp+vIMM7vHzDq7+7o4Z6tP0r53knibfUM69gDNBvqZWe/wt4NjgOmJenEzM+BvQIm7/yni9sjxmacCUWfQaOYsrc2sbfVlQifGLiK0Pc4LNzsPeDbeWSKMJWK4ShDbJUJt22E6MMbMWphZb6Af8E48g5jZKOA64Hvuvi3i9j3NLDt8ed9wlk/inKW2v0nCt0sS0P7k69fU/qRu2p9Ez5Lp+5Mg9iH1vifreD83t1h+/+nAuRYyAthUPYQvjurNZWZ7hffBmNkwQp+Zv4pzrlgEsb1iksTb7JsaO3tCMv8QmiHjI0IzfPwiwa99JKGuyIXA/PDPicAjwHvh26cDXROQZV9CM5ssABZXbwtgD+AV4OPwv50StG1aEXojtI+4LSHbhdCHpNVAOaFvTy6sazsAvwj///kQOCEBWZYQGtNb/X9mUrjt6eG/3QJCXcsnJyBLrX+TeG6XZP3R/mRXFu1Pvn4d7U9iz5K2+xNCBd0KQr1qXwIvhG/fG5gR0S6h+5Da/i9G5qrt/RynPN/4/YEJwITwZQPuDt//HrXMqBdArssj3i+zgMMTlCva+yjw7RVjtkC2WWN+LBxYREREREQk7aXjEDgREREREZGoVACJiIiIiEjGUAEkIiIiIiIZQwWQiIiIiIhkDBVAIiIiIiKSMVQASZ3MrDT8by8zO6uZn/vnNa6/2ZzPLyLJRfsTEUkFZnaemX0c/jmv/kdIqtE02FInMyt19zZmVgT81N1PasBjs929sr7nboaYIpICtD8RkWRnZp2AOUAhoXXY5gKHuPuGQINJs1IPkMTqZmCkmc03s6vMLNvMbjOz2Wa20MwuATCzIjN71cweJ7RAF2b2jJnNNbPFZjY+fNvNQMvw8z0Wvq3622ELP/ciM3vPzM6MeO5iM3vSzD4ws8ciVhy+2czeD2f5Q8K3jog0hPYnIhI4Mzs0/D7PN7PWZrYYmAi85O7rw0XPS8CoYJNKc8sJOoCkjOuJ+MY2/MFjk7sfamYtgDfM7MVw22HAge7+afj6Be6+3sxaArPN7Cl3v97MLnf3wVFe6zRgMDAI6Bx+zMzwfUOAA4BVwBvAEWb2PqEVufu7u5tZh+b91UWkmWl/IiKBc/fZZjYd+A3QEngUKAeWRzRbAXQLIJ7EkXqApLG+DZxrZvOBt4E9gH7h+96J+LACcIWZLQBmAT0i2tXmSGCqu1e6+5fAa8ChEc+9wt2rgPlAL2AzsAO438xOA7Y18XcTkcTS/kREgvJr4HhCQ95uBSxKG50vkmZUAEljGfAjdx8c/unt7tXf2G7d1Sg01v9bwGHuPgh4F8iP4blrszPiciWQ4+4VhL4lfgo4BXi+Ab+HiARP+xMRCUonoA3QltD+ZAWhL1eqdSfUSyxpRAWQxGoLoZ1DtReAS80sF8DM9jOz1lEe1x7Y4O7bzKw/MCLivvLqx9cwEzgzfF7AnsBRwDu1BTOzNkB7d58BXElouIuIJC/tT0QkWUwGbgAeA24htD/6tpl1NLOOhHqoXwgwn8SBzgGSWC0EKsJDTx4E7iA0XGRe+MThtYS+La3peWCCmS0EPiQ0bKXaZGChmc1z97Mjbn8aOAxYQKjb+Vp3/yL8gSeatsCzZpZP6Nveqxr1G4pIomh/IiKBM7NzgQp3f9zMsoE3CX3pcRMwO9zs1+6+PqCIEieaBltERERERDKGhsCJiIiIiEjGUAEkIiIiIiIZQwWQiIiIiIhkDBVAIiIiIiKSMVQAiYiIiIhIxlABJCIiIiIiGUMFkIiIiIiIZIz/DynWfw43zdM6AAAAAElFTkSuQmCC",
            "text/plain": [
              "<Figure size 1008x288 with 3 Axes>"
            ]
          },
          "metadata": {
            "needs_background": "light"
          },
          "output_type": "display_data"
        }
      ],
      "source": [
        "# Define network dimensions\n",
        "in_features, hidden_dim, out_features = 2, 10, 2\n",
        "\n",
        "# Training parameters\n",
        "learning_rate = 5e-1  # step size for gradient descent\n",
        "optim_steps = 150  # network should generally converge within 100 steps\n",
        "\n",
        "# Define and initialize layers\n",
        "layers = [Linear(in_features, hidden_dim),\n",
        "          Sigmoid(),\n",
        "          Linear(hidden_dim, out_features)]\n",
        "\n",
        "# Initialize network\n",
        "net = Net(layers)\n",
        "\n",
        "# Define list to store loss and accuracy of each iteration\n",
        "losses = []\n",
        "accs = []\n",
        "\n",
        "for i in range(optim_steps):\n",
        "    # Perform forward pass with x_xor as input and y_pred as output variables\n",
        "    y_pred = net.forward(x_xor)\n",
        "\n",
        "    ########################################################################\n",
        "    #      TODO: Calculate Cross Entropy loss between prediction and       #\n",
        "    #                  labels (y_xor) and append to list.                  #\n",
        "    ########################################################################\n",
        "\n",
        "    loss, grad = CrossEntropyLoss(y_xor, y_pred)\n",
        "    losses.append(loss)\n",
        "\n",
        "    ########################################################################\n",
        "    #                          END OF YOUR CODE                            #\n",
        "    ########################################################################\n",
        "\n",
        "    ########################################################################\n",
        "    #            TODO: Perform backward pass and optimizer step.           #\n",
        "    ########################################################################\n",
        "    \n",
        "    net.backward(grad)\n",
        "    net.optimizer_step(learning_rate)\n",
        "\n",
        "    ########################################################################\n",
        "    #                          END OF YOUR CODE                            #\n",
        "    ########################################################################\n",
        "\n",
        "    # Calculate accuracy of prediction\n",
        "    correct = torch.argmax(y_pred, axis=1) == torch.argmax(y_xor, axis=1)\n",
        "    accs.append(torch.sum(correct)/len(y_pred))\n",
        "\n",
        "# Print prediction\n",
        "print(y_pred)\n",
        "\n",
        "# Plot loss and accuracy\n",
        "plt.figure(figsize=(14,4))\n",
        "plt.subplot(1,3,1)\n",
        "plt.xlabel('Iterations')\n",
        "plt.ylabel('Loss')\n",
        "plt.plot(losses)\n",
        "plt.grid()\n",
        "\n",
        "plt.subplot(1,3,2)\n",
        "plt.xlabel('Iterations')\n",
        "plt.ylabel('Accuracy (%)')\n",
        "plt.plot(accs)\n",
        "plt.grid()\n",
        "\n",
        "# Show decision boundary\n",
        "plt.subplot(1,3,3)\n",
        "plot(x_xor, y_xor, net)\n"
      ]
    },
    {
      "cell_type": "markdown",
      "metadata": {
        "id": "X7fATXbHjJe0"
      },
      "source": [
        "## [EXTRA] A2.7 MNIST handwritten digit classification\n",
        "In this extra exercise you will use your neural network to solve a real-world classification problem: classifying handwritten digits.\n",
        "\n",
        "The MNIST dataset consists of a training set of 60,000 and a test set of 10,000 samples of 28x28 pixels, which we will flatten to a 1D vector representation to feed into the neural network. In every training iteration we will sample a random batch of 64 data samples (without replacement) to calculate the loss, gradients and update the weights. This random sampling is where the 'Stochastic' in Stochastic Gradient Descent comes from. After every epoch, i.e. after having processed all sampels in the training set, we will evaluate the model on the test set.\n",
        "\n",
        "The following code block downloads the dataset and visualizes some samples.\n",
        "\n"
      ]
    },
    {
      "cell_type": "code",
      "execution_count": 52,
      "metadata": {
        "id": "X4mMxSNwjHKw"
      },
      "outputs": [
        {
          "name": "stdout",
          "output_type": "stream",
          "text": [
            "Downloading http://yann.lecun.com/exdb/mnist/train-images-idx3-ubyte.gz\n",
            "Downloading http://yann.lecun.com/exdb/mnist/train-images-idx3-ubyte.gz to ../data\\MNIST\\raw\\train-images-idx3-ubyte.gz\n"
          ]
        },
        {
          "name": "stderr",
          "output_type": "stream",
          "text": [
            "9913344it [00:00, 29357744.04it/s]                             \n"
          ]
        },
        {
          "name": "stdout",
          "output_type": "stream",
          "text": [
            "Extracting ../data\\MNIST\\raw\\train-images-idx3-ubyte.gz to ../data\\MNIST\\raw\n",
            "\n",
            "Downloading http://yann.lecun.com/exdb/mnist/train-labels-idx1-ubyte.gz\n",
            "Downloading http://yann.lecun.com/exdb/mnist/train-labels-idx1-ubyte.gz to ../data\\MNIST\\raw\\train-labels-idx1-ubyte.gz\n"
          ]
        },
        {
          "name": "stderr",
          "output_type": "stream",
          "text": [
            "29696it [00:00, 7435175.00it/s]          \n"
          ]
        },
        {
          "name": "stdout",
          "output_type": "stream",
          "text": [
            "Extracting ../data\\MNIST\\raw\\train-labels-idx1-ubyte.gz to ../data\\MNIST\\raw\n",
            "\n",
            "Downloading http://yann.lecun.com/exdb/mnist/t10k-images-idx3-ubyte.gz\n",
            "Downloading http://yann.lecun.com/exdb/mnist/t10k-images-idx3-ubyte.gz to ../data\\MNIST\\raw\\t10k-images-idx3-ubyte.gz\n"
          ]
        },
        {
          "name": "stderr",
          "output_type": "stream",
          "text": [
            "1649664it [00:00, 21340185.34it/s]         \n"
          ]
        },
        {
          "name": "stdout",
          "output_type": "stream",
          "text": [
            "Extracting ../data\\MNIST\\raw\\t10k-images-idx3-ubyte.gz to ../data\\MNIST\\raw\n",
            "\n",
            "Downloading http://yann.lecun.com/exdb/mnist/t10k-labels-idx1-ubyte.gz\n",
            "Downloading http://yann.lecun.com/exdb/mnist/t10k-labels-idx1-ubyte.gz to ../data\\MNIST\\raw\\t10k-labels-idx1-ubyte.gz\n"
          ]
        },
        {
          "name": "stderr",
          "output_type": "stream",
          "text": [
            "5120it [00:00, 5106976.57it/s]          "
          ]
        },
        {
          "name": "stdout",
          "output_type": "stream",
          "text": [
            "Extracting ../data\\MNIST\\raw\\t10k-labels-idx1-ubyte.gz to ../data\\MNIST\\raw\n",
            "\n"
          ]
        },
        {
          "name": "stderr",
          "output_type": "stream",
          "text": [
            "\n"
          ]
        },
        {
          "data": {
            "image/png": "iVBORw0KGgoAAAANSUhEUgAAAV8AAAFgCAYAAAAcmXr5AAAAOXRFWHRTb2Z0d2FyZQBNYXRwbG90bGliIHZlcnNpb24zLjUuMSwgaHR0cHM6Ly9tYXRwbG90bGliLm9yZy/YYfK9AAAACXBIWXMAAAsTAAALEwEAmpwYAAC+JklEQVR4nOy953NcaX7v9+mcE7ob3ehuZBCBJJiGYdJO2p3VZu1dybr3SvZVleVb5TfXfqOy/wS/sF3lFw63VNdySWtL2ruSdrWzs7uzYWbD5BlyGEEih0ajc87ZL1jPMwDTckiAaGDOp4o1HHaj+5yDc77P7/lFVbfbRUFBQUHhyaLe7wNQUFBQ+DyiiK+CgoLCPqCIr4KCgsI+oIivgoKCwj6giK+CgoLCPqB90IsqlepzlQrR7XZVD/M+5brcjXJN7o1yXe6Ncl0Uy1dBQUFhX1DEV0FBQWEfUMRXQUFBYR94oM9X4cmiUqnu+f/b/11UJG6vTFSqFBUUDh6K+PYIExMTDA4Oolar0Wg0mEwmnE4nNpuN4eFhtNrbv6pGo8GtW7coFAqkUilKpRLhcJhEIrHPZ6CgoPBZUMS3B1CpVAwODvLUU0+h1WrR6XQ4nU5CoRB+v58LFy5gMBgAqFQq/OIXvyAcDrO8vEwqlaJcLiviq6BwwFDEdx/RaDScPHmSYDDI008/zezsLCqVCq1Wi8lkwuFwYLPZUKs/dc1rtVrGx8dxu9309fWRyWRIp9NEIhEajQb1en0fz2j/sVgsGAwGzp8/z7Fjx1hfX2dpaYlsNsvGxsahdNGoVCo0Gg0ul4uZmRlsNhsjIyPodDry+TyVSoUrV64QiUSo1+s0Go39PmQFFPHdV7RaLV/4whd49tlnOXHiBEeOHNnx+r18vVqtlqNHj9LpdBgbG6NQKLC8vMz169cpFoufa/FVqVTY7XYcDgd/8id/wr/7d/+ON998k9dee41bt24RiURotVr7fZi7jlqtRq/XMzAwwNe+9jWGhoZ49dVXsVgsrK6ukkql+L/+r/+LSqVCNptVxLdHUMR3H9DpdExNTeH1epmenmZoaAi73b5DbFutFtVqlXq9Ti6XA8BoNKLVaunr60On02EymQDw+/2Mjo6ysbEh3/t5RKVSYbFYcDqdGI1G1Go17XabWq1Gs9nc78PbM8ROyeVyMTY2RjAYxGAwoNVqsdvtwO17JBAI0Gw2KRaL+3zEe4PJZMJoNBIMBhkdHSUSiXDlyhXa7fZn/iyj0YjD4aDdblOtVmm329Tr9V3dOSniuw9YLBb+8A//kGPHjnH+/HmGhoZ2uBYAqtUq8XicVCrF9evX6Xa79Pf3Y7PZOHXqFH19fTgcDqxWKzMzM2QyGdRqNevr64dya/0wqFQqPB4Pw8PD2Gw2ut0utVqNXC5HqVQ6tNdFp9PhcDgIBAI888wzeL1eNBoNKpUKv99PX18fR48epVqtUiqViEaj+33Ie0JfXx9er5dvfOMb/Omf/imvv/46c3NzjyS+DoeDmZkZarUa0WiUWq1GOp3e1Z2TIr5PELVajclkwm634/f7CQaDWK1WNBoNjUaDVqtFMpkkFotRLBZJJBLkcjmWlpbodrvE43EcDgehUAiDwYDBYECj0UgrWGREfF5RqVTy+up0OtrtNqVSiUQiQbFYPHTiKzJj3G43U1NTjIyMYDKZ0Gg0O96z/c/21w4bFosFj8eD0+mUvv870zcfFqPRSH9/P+12G5PJRLFYpFgsKuJ7UNHr9YRCIQKBAKdOneLUqVMyiyGXy5HNZvnJT37CP/zDP1Cr1SgWizQaDcrlsvx5j8eD1+ul0+kQCASw2WwYDAZ5s32eUalUeL1ehoaGMJlM1Go1Njc3+fDDD2k2m3Q6nf0+xF1Fr9djNps5ceIEf/EXf4Hf75euqM8bKpWKQCDA7OwsoVAInU73WAuN2+3m9OnTmM1mTCYT0WiUzc1NKpXKrh3zroqvyE/V6XRYLBbUajWdTkfe9N1ul2q1SqPRkP/ebrcfaVtwEFGpVBiNRsxmM0ajEYPBQKfTodlsEo/HCYfDrK+vE4vFqNVqVCoV6WvSarXSj6fRaFCr1XJVFwL9eQ6kCOu/r68Pn8+HSqUil8tRKBSo1+uHSnjF/WO323ekJPb19d3lvhKIBdrhcNDX10er1ZJ/DsN9I/z9Xq8Xs9n82J/X6XTodrsyxlKr1dDpdKhUql3bQe2q+DocDk6cOIHX6+WZZ57BYrFQKBTklrrdbjM3N8fKygqVSoVKpUK5XCaTyRy6LeG90Gg02Gw2uS0GSKfTFAoF/vmf/5mf/exnJBIJ4vG4/OWL62Kz2XjllVcYGhpienqaYDCI0Wik2+0SDoe5dOkS4XD4c3Ed70Sj0eDz+ejr6+O5557jxRdfZGVlhd/97nesra0dqmuiUqk4evQo09PTjI6OcuTIEQYHB5menkan093T9aRWqwmFQjSbTXQ6HYODg2QyGZLJJKlUio2NjQO9OAl3yuDgIKdPn6a/v/+R3Q2CUqnEysoKQ0NDnDhxgm63K3eXwnh8XHZVfA0GAwMDA/j9fsbHx7HZbORyORqNBs1mk3a7TblcptlsUi6XKZVKFItFaSE/KkKkRGRbWNS99tB1u106nQ6tVotsNks8Hicej5PP59nY2GBtbY1KpXLPyLxOp8Pn8zEwMIDVasVgMKBWq+l2u9TrdUql0qGO6D+I7TsKp9OJy+Wi2WwSi8UolUr7fXi7jsvlYmRkhJGREUZHR/F6vZhMpvtavSqVCqvVitvtplwuo1arsVgs0gCIRqPSODqIaLVa9Ho9NptNZgKJHfajakCn05EiK7IotFrtfa/xIx33rn0S4PP5+M53vkMgEGB0dBSDwUCr1dphxT3zzDNUKhXq9Tq1Wo18Pk80Gn0s8W00GhSLRTKZDL/73e/IZrNsbW1RrVZ38ewen1qtxo0bN1hdXSWbzeJwOCiVStTrdVZXV8nlcvd9AEwmE6dOnWJiYgKXyyWj2b22wOwHGo0Gj8fDwMCA9IGHw2Hee+89VldXD9U1UqlUnD17lj//8z/HaDRiMpnQ6/UPFAW1Ws3ExAShUIgTJ07QaDTI5XKk02kuXrwoM0Ki0eiBE2C1Ws3g4CAej4fp6WkmJydZXFzk448/5ubNm4+sK8J16nQ6CQQCwG13j16v713L1+/34/f7sdls6HQ6KboqlUqmAqlUKlqtFs1mk0KhcJf4/r4tw/aHSaVSUa/XyWQyJBIJFhYW6HQ6PVlu2263yefzFItF2u02BoOBWq1Gq9WiUqnc0/cmqpdMJpMMtun1eim8233nh0lkPgvb83vFtSkWi4fO8tXpdLKYYnx8fIdb6n6IZ89qtWKxWORzWCgU8Hq9pNNpvF4vAPF4/ECJr3g2nE4nfr8fl8uF1WqlXq+zublJOp1+5GdC7KbEjkpkEwmjZzfYVfGNxWJ873vfo6+vj8HBQbRaLel0mmazicvlwmw243K5cDgcuFwuKSSjo6O0Wi2KxSIqlQqbzXaX70rcaGKbXa/XZbRXbBFSqRTRaJSNjQ2ZrtWLdLtd8vk8Go1Giub9UljcbjdnzpxhbGyM4eFhec1E6lmhUGBlZYWVlZWePd+9xmg08vTTT3Ps2DGsViupVIpIJMLa2tqhuSY6nY6vf/3rzMzMcPr06YcSABHgbrfb0u1nsVgwm82YzWa0Wi3PPfccfr+fS5cu8T/9T//TgSnSUavVeDwe7HY73/rWtzhz5gyDg4MkEgmuXr3K66+/TjKZfOTFxO12c+LECQYHB6V7ZrfZVfEtFApcvnwZm81GMplEo9EQiUSo1WoEAgHsdjvBYBCv10ur1cJms0mLpdFo0Gg0UKvVOJ3OewYOhJVXqVRQqVTSxyduRKPRyODgIO12G6PRuJuntquIBeRhsFgsTE5OMjIyQl9fn8wL7na7FItFkskk6XR61xPADxJarZbh4WEmJyfR6/VUKhUKhQKZTObQ+MG1Wi0zMzO8+OKLDA4OAjzQ7SR2RSLeUqvVaLfbcuHW6XRyaz08PCwtvYPiylKr1dhsNtxuN8eOHeOZZ56hXC5TKBTY3Nxkbm7usdwDJpOJQCCAx+PZVT/vdnZVfCuVCisrK+j1eiKRCGq1mlKpRKvVYmNjA4PBgNVqlSlBYuXy+/1Uq1U2Nzdl1PJeOasiLa1QKJDL5Thx4gTf/va3ZdpWu92W+bIH/aFzOBwMDAwwMzPDq6++is/nw263o1arZdnxhx9+yM2bN1laWjqUeay/D41GIwNJAwMDBINBuStqNpu75pvbT0Qmh8vlIhAI4PV6ZZbL/dwOlUqF+fl5CoUC6+vrFItF6d4KhUL09/czMjKyo5eI3W7n7NmzbG5uMj8/33PxkjsxGAw89dRTspxap9Nx+fJlPv74Yy5evEi9Xn8kq1cE79xuN2NjYzgcDrRa7a65GnZ8125+WL1eZ2tr66HeazAYpAgPDQ3J1A6dTsf4+Pg9k8XF1klYet/4xjf40pe+JBuLiIqmQqFw4K1Aq9XK6OgoU1NTXLhwAafTiU6nk9ZMqVRibm6Od999l3A4fODP91EQUXuHw4HX66W/v59YLEalUpE5rAcdjUaD1+vF5/Ph9XpxuVwYDIYHNtOv1WrcunWLeDzO1atXyWQyVKtVWq2W7ButUqkYHx+XomK1Wjl69CgWi4W1tbWeF1+dTsfMzAyzs7P09/ej1WqZn5/npz/9KbFY7JFzl0U+vcPhIBgMYjKZ9mwnsG8VbqJ4IJfLyaBZrVaj0WiwtbX1QLeDaJ4hUmxUKhWZTIZ4PM7i4iKrq6s9f/P8PkR5Y19fn3T0A3IXkUgkWF1dZXNz81AFlT4LOp0Ov9/PwMCAFKRGo3Hf4OVBQq1WYzabsdvtfOELX2B8fFymb+r1+h3vFU2YyuUysViMWCzG22+/TSKRIBwOy+shso7y+TyhUIhKpSKDeE6nk2eeeUbmAMfjcdbW1iiVSj21exABR5/Px/j4OMPDw8DtCtFUKkU8Hn8sP7/f72dmZoaJiQlUKhW1Wk3Gksrl8q4W7Oyb+ArLpFqtkslkdrz2+8RkcnKS0dFRBgYGMJvNsidCOBzm2rVrrK+vH3hBEj6n/v7+HalErVaLpaUlVldXmZ+fZ2VlZZ+PdP8QBQPDw8Nyp1Sr1SiXywfe6tVoNNjtdtkm8vTp01itVumX3Y5IHYvH43z00UeEw2F+9rOfkUwm79p+53I57HY7U1NTlMtlmarmdrv50pe+JIO4a2trshfww2RVPCmMRiNTU1MMDg4yMzPDkSNHyGQyUiDD4fBjff7IyAivvPIK09PT0m26trbGxsaGdN8cePF9FIQFGAwGOXfuHOPj42g0GvL5PDdv3mRlZUXmzfbSav1ZCIVCjIyMMD09zfHjxwkEAmi1WprNJolEgkwmw40bN1heXj4wkem9QgiU8Mt1Oh0ikQhLS0t3LegHEVG5ZTAYZJK/SBUDZKFSOBzm8uXLpNNp5ufn5Xipe8UBREHO1tYWN27cYGBggMnJSdnE32azcfToUex2O5cuXSKbzVKtVnsmhqLT6RgYGCAUCmGxWNBoNMTjcSKRCNls9pE/V6/Xo9Pp8Hq9jIyM0N/fD9w2BOfn51leXpbC25PlxXuNwWDAbDZz5swZ/uzP/gyr1YpOpyOVSvHzn/9cbsd3s/nFk+b06dP86Z/+KaFQiOPHj6PX69Hr9RQKBT755BM2Nzf58Y9/zNLSkmy483lle9Wf8PlfvnyZd955h/X19f0+vMdC5LBqtVrMZjNWq/Wu92SzWVZWVnj33Xf5v//v/5tSqUQ+n5eB6XuJRLlcplKpcO3aNZxOJ2fOnGF8fBy9Xi87pH3lK18hkUjw29/+lnQ6TTKZ7BnxNZlMHD9+nPHxcVlsdPPmTflsPCoWiwWbzcbExARPP/20dGcmk0l+8YtfsLm5ST6f31Wj7kCJr9PpxOv14vV6sVqt8oFrNBoUCgVZvHAQ0Wg0Mnrf39+P0+mUJcStVotyuczGxgbr6+vkcrld3f4cNDQajUwzHBgYkK3/isUi6XT6wC/AD0KkKTabTSKRCHNzc6ytrVEoFB66pLbb7cq+vslkkkKhIHN/VSqVTEPbqxSrR0Wj0Uj/tAhAd7tdkskk6+vrFAqFR/5s0Q/Z7XbLtpyVSkW2dt2LVM4DI74qlYrp6WnOnj3LsWPHsNvtsldEPp8nFouRTCYPrK/PZDJhNpsJhUJMTk5isVjQ6/VSeGOxGG+88QYrKyvEYrFd76p/kDCbzYyNjTE2NsZzzz1HMBiUD8nNmze5du0atVptvw9zz0in06RSKX71q1/x/e9/n2w2Szab/Uxb4ng8zscff4zJZGJlZUU2oe/VntCiytNut8ueFiaTiVarxdWrV/nFL37xyNa5SqVicnKSCxcucPToUWw2G+VymXg8zsbGBrdu3SKVSu269d9bS9t9EJVsoleraJ1XqVTY2NggEolQLBapVqsH1hrs6+tjZGREZnCI9nXNZlPmLmezWXK5HM1m83MrvHDb9+9wOHA6ndhsNkwmk2zWVK1WZU7rYUH4eYWvN5FIMD8/TzgcJpVKUSwWP7MvUuwYRcfBO6+XGMjZ39/fEwVLKpUKtVqNVquVZb/VapVCoSB/74/zOxdjg0SDona7LTsvip3GbtOby9w21Gq13A68+OKL/OEf/qGsUb9x4wZ//dd/TTgcZmFh4bF/AfuFWq3m5Zdf5hvf+AYTExOymAJuR6fff/99lpeX2djYONDW/W4hRieNjY3hcrkwmUxkMhm5CB+2xWm78HY6HV5//XX+n//n/6FYLFIoFB45CLRd1EV/aNFh0GQy8corrzA6Osrf//3fk06nd/u0PvOx6nQ6DAYDTqcTs9nM1atXiUajpFKpx/58u92Oz+fDZrOhUqmoVCqEw2ESicSeuTJ7XnxFUxCXyyWr4kQ+Zy6XY3V1lVgs1lMR2c+C8K+JqiO3241Go5FN1svlMltbW8Tj8QN7jruNyHIQzZvEwyImfxwG4RXlvvdqFZnJZFhbW3uszxcWpMFgkH2AhcALQdbr9RiNxp4ZPXRnYUm9Xpe73UctixaBTdGcXlTWNptNSqWSTLXbC3pefNVqNSMjI8zMzOD3+9FoNCQSCSKRCDdv3mRhYUGWMB801Go1Y2NjeL1eZmZmZBtOgHw+Tzgc5vr16/zgBz8gkUg8VkDhMGE0GuU4Jp1OR6PRYG5ujrm5uUORYga3rftnnnmGkZERnE6nFIDdyrnt7+/n9OnTHD9+nNHRUcxms+wP3el0qFarvPXWW1y+fPmxc2d3g06nQ71ep1gsyjYEDocDtVote57U6/XPXFwjpoJMTk7uyHIoFossLCywsbHx+bZ8bTbbjmq2SqUiI5D5fJ5arXYgrR3RRGhgYIC+vj5sNpu0chqNBtlslkQiwfr6+qFqEvM4bM9HtVgswG0rJZ1Oy/FLhwGRzxoIBGQzHLi7lPizIu4v0VPF4/HsqJoT4i6a0a+vr/dESqPo/CdGZlUqFdlA3Wq1YrPZAB4olNvdLOKP6PLmcrnweDzyvSKDqlKp7FkcqafFV9RZj4yMcPz4cTl7an5+nn/5l39hYWHhQDZP2T554fz58zz11FOyYGR7An04HJbuhsOynX4cdDqdDLxOTEzg9XopFApks1muX7/OlStXHivRvpcwGo1MT08zMTFxzxzfz4oI4jocDiwWCxcuXOAb3/gG/f39O9wKnU5HFm9sL0neb4QbToxwdzgcHDlyBJPJxB//8R9z+vRprl27xtLS0n0/Q6vV4vV6Ze8Gg8GAy+XCYrFw8uTJu96/1+fds+IrfDGi9FFMZu10OiSTSa5fvy6bP/fCzfFZEP40k8nE8PAwR48exe127ygbrdVqZLNZ8vk8jUbjwOYv7ybb0408Ho+cBJLL5YjFYg/d1OkgoNVq6e/vx+/339XL4bMiglUmkwmXy4Xb7ZauvDt9yp1Oh1qtJltQ9pJhI0ZwlUolyuWyDL7Nzs4SCASkxX4/DAYDg4ODWCwWfD4fJpMJt9uN1WplYGDgLp+ycMHsFT0rvnq9ntOnTzMwMMDU1BTBYJBSqcTi4iLLy8usra1RLpcPnPDC7XM7deoUgUCAmZkZQqGQ3DalUilisRiXLl3ijTfeIJFIHPgmMbuF2+3m1KlTHD16VCbZx2Ix2TNa4d5oNBpmZ2cZGRnh2LFjjI+PMzo6itVqlXm9zWaTSqVCJpPhl7/8JZubmywtLckZjL1CqVTi7bffZnV1FY1GQygUwmq1EgwGefHFFzl27NgDf14ULpnNZrkTEAv6dtLpNB9//PGeVvf1rPjqdDqmpqaYmJiQ+a/pdJpwOEwkEmFra+tACi/cPrfJyUnZJN3n88nXcrkcy8vL3Lhxgw8++OC+AzU/jzgcDqanpxkbG8NqtdJqtWQnq14SiF5DzHA7c+YMzz333D232GKSTDQa5a233mJ9fb0nO+ZVKhWuXLlCOBxmeHiYZrMpJ6aLgPz9EFWQ24Pzwt99Z7Mi0S9mL4P5PSe+ol7f4/Hw1FNPMT09jcPhoF6vs7y8zEcffXTg6/bFwnL69Gn6+vp2vLa+vs6bb77J4uKiLBY4qIvMbqPT6bDb7bIMVqQcHuRGSnuJ0Wjk/Pnz+P1+nnrqKUZHR3E6nfd8r5ionUql2NraIhqN9uSCJgYmNJtNfvOb33Djxg3m5+fp7+/HYrE8sCCkVquxuroqd0larZannnqKwcFBfD7fjmex3W7v+UTnnhNfg8HA2NgYg4ODfOELX+DYsWMyn+/mzZv88pe/JBKJHGhB0uv1nDx5kueee+6ulXppaYkf/vCHsmrnIJ/nbqPT6WRVm0iLEv5JRXzvxmQy8fWvf53Z2VkGBgaw2+24XK57vlfMQBQZDr2QXnYv2u02mUyGTCbD5uYmarWa4eFh6ct2OBz3/dlCocD7778v+/0aDAb+4i/+ggsXLnDmzJm7DKHPjfiKoIDdbmd2dpbh4WEcDgcqlYpcLif7lSYSiZ5IfXlc7iwZFYgSStHk+n50u10ZELlzmrNoRbg9aV6r1aLVagkEApjN5vt+ZjabJRwO91ywBW53nhJztTQajWyz+TiTCw4K23+XYsSUwO12Mzo6Kkfg3DmdIhgMSt+mKEq5k1wux7vvvitjKQcBERAT5dViOMP9EKXC4r5ut9tUq1VZnLMd4Q8Wqa17YQT1jPiKTlUDAwN885vfZHx8XPpCw+Ew6+vr3Lp1i5WVlUNtDep0OqxWqyzzfNCARFHxJiZ8qNVq1Gq1jGyLh0xcW7vdzle+8hUCgcA9P6/T6XDlyhV++MMfyllovYTL5WJ2dhaXy4VWq6VWq3Hz5k2Wl5d7zje5W2wv/YXbi/PAwACzs7PyPefOneNP//RPsVqtOJ3OHdkLBoNBDlwVn3cvIpEI/+k//Sfi8fiBijF0u11SqZQsf37QrLU7sxe63S6FQoFUKnXX5Buz2UwgEMBgMEhjZLfpGfEVTZzHxsbweDxYrVbZtUw0yBadmw4zAwMDnDt3jlqt9kC3g0i5EyNiWq2WtG5FW8rtD6zRaJSTkLcnkwtEak0+n2doaIhcLkc0Gu2JFDfRTFw00dlu3YkdwGGj1WqRSCRkA6HtvsxQKMTp06flvTExMSF7XIhKNYH4+4OGbeZyORKJhMwnP2g8TtXf9p/dLtzCKt7LLJqeEF+1Ws3o6Cj//X//3xMMBjly5AhGo1FWsL355pu88847hyqP8368+OKLnD17FuCBC0273ebSpUtsbW1RLBapVCqYzWYsFguhUIgzZ87s8CcLC8pgMDzQOvD5fNRqNZaXl/npT3/aE31xxZBV4dvbbtUfVkqlEu+++y5bW1v09/fvSIV6+eWXee6556Rw3LnT2S5Eou/B/e6laDTKe++9x7Vr1w6k8D4u97uPKpUKm5ublMvlw1terNFoZOWSz+ejv79f9jdIpVIkEglSqRSZTObQ5HIKl0GpVMJisaDT6eRrol3e76PdbhMMBtFoNDI4J6xbMfvtzoYs2x9K0QGu2WzuSKXpxSCfmHJts9nkFloEYSuVCtVq9dBZvyKNzmw2y9+VcCuJxudwb6vvTktObLdFlVin05FTvtfW1mRzqsN2DR8HUdBxqANuRqMRj8dDKBSSTWZ0Oh2FQoGf/vSnzM3N8cknn8hqtsNAs9lkYWEBs9nM0aNH5byoz4JGo+HIkSOMjY3d5fPVaDRSeMU12/6QttttFhYWiMfjxGKxHe0CxWyvVCrVM82K/H4/58+fZ2JiAq1WS71eZ2tri+XlZebm5nakDx0WqtUqV65cIZlM8vWvfx2/3y8b7D8KItgkdpPvvPMOv/jFL+Qggmq1euAnfj8K+2lo7Jv4im2wKPUT4zu0Wi2VSoV8Ps/W1pacGnqQggC/j3a7TSKRYHNzk/7+fjl5915bIFFmLUqSt79HWD/NZlP2sBVWTqlUot1uyzLR7WXY7XabcDgsS3KTyaT8zHQ6TSaT6amR4WazWTaAUalUtNtt8vk8uVyOUql0oJvo3w9REJDL5eTkina7jclkkm0gH5Zut0ulUqFUKhGPx0mlUnL6tWhI3ovZLU+C7c/TneXFe82+ia/JZMJms3HhwgX+/M//nEAggNVqJZ/P88Ybb7CxsSHLCA9bJLtcLvO9730Ph8PBK6+8wuTkpLRa78RoNOL1enE4HBw/fvyeaWKLi4syRUjkMMLtAYs3b96kXC7vKJMUM7xEC77tvr5msym3ufu90xDTqgOBAKdOncLn86FWq8nn81y6dImVlZVDKbxwW3yz2Sz1ep2/+7u/49e//rVsq3rixAmOHDny0J/VarX48MMPuXXrFpcuXeLWrVty3LrYXveaq+lJsL1/jHj2trsbDm1jHb1eLxtazM7OyvHf9Xqd9fV1VlZWiMfjh6Y/63ZarRZra2vodDqZ9nU/8bVYLNRqNcrlMkNDQ/f07yUSCdbW1qSlJN6TSqW4ePEihUKBra2tAxdQEbsj4fM1mUwyCh2LxUgkEodqR7QdUb3X7XZZXl4mm82i0+moVqsEg0EGBwfvmkABSNEQLiO1Wk2j0SAajbKyssLc3BzXr1/fz1PrKURutLh+rVZrz8YG3fXde/4N9yEUCnHu3DlOnjyJx+OR3ZUajQarq6ssLS31RKR9L2m1Wly5coWVlZX7Rl3F6HC9Xo/H47nndjOTyZDP52k2mztycxuNhuwD3Cv+289Cp9ORs8ZEUDEWi8lKx2QyeWAKAh4V4SISxSRms5mbN29y5MgRhoaGOHLkCH19fQwPD1Mul1lZWSGXy3Ht2jVKpZKcR/bJJ5+wtbW1KyN3DgsajYapqSmee+45vF4vlUqF+fl5rl69yocffrjnO799E9++vj4mJiYIBoM7uiu1Wi1ZPnjQLLXPSrfb/Vykzz0qwn8t+rg2Gg2KxSLr6+s92XFrL+h0OuTzeeD24Ey43aMgGo1y4sQJjEYj3W6XYDAoR07FYjHee+89crmcTDPb2tqSwyYVbqNSqeT4LpVKRa1WY2tri6tXrxIOh/fcnfXExVfUX8/OzvL8889LP56Cwp0I8f34449lT+NGo0E6nT6UgzIfFtFgP51Oc/PmTaxWK16vl1qtJsvvNzY25C6o2+1SLpdpNBqHLivkceh2u6TTaTY2NlhZWSESiXD9+nU++eQTstns4bN8HQ4HgUCAI0eOcOrUqR3+FgWF7YjsjVu3bnHr1q39PpyeIZvNks1mD3x3v/1GVHTGYjE+/vhjLl68yMbGBsvLy0/k+5+o+KpUKux2O6FQCJfLhU6nk1VYrVaLSqVCoVCQDu/Po1WjoKDwZBBZIIlEQlq+D2rMs9vsi9tBNEffXurabDblii78e4cxhUhBQaE3aLVa/PrXv5b//6SNvScuvne2URTjS2KxGO+++y7hcJhoNEqxWNz3PFMFBYXDzeeywk0gupZ98skn/K//6/9KKpUim81+bhO/FRQUPh88cfHNZDKsra3J0c2lUolkMsnCwgK5XE52EVKEV0FB4TCjepDIqVSqXVdAvV6PTqdDp9NhMBh2dFsql8sPbCC+13S73YfqU7gX16WXeZjrolyTe6Ncl3ujXJd9sHzv7CWgoKCg8HnkgZavgoKCgsLeoFQ3KCgoKOwDivgqKCgo7AOK+CooKCjsA4r4KigoKOwDivgqKCgo7AOK+CooKCjsA4r4KigoKOwDivgqKCgo7AOK+CooKCjsA4r4KigoKOwDD+ztoDS/uDfKdbkb5ZrcG+W63BvluiiWr4KCgsK+oIivgoKCwj6giK+CgoLCPqCIr4KCgsI+oIivgoKCwj6giK+CgoLCPrDv04sVFBSeHFqtFo1Gg16vx2g03vV6u92mWq2i0WgYGRnBarVSKpVoNBpUKhXq9TqVSoVyubwPR3+4UMRXQeFzhBhca7fbcTqdd73eaDRIpVLo9XqefvppAoEAkUiEfD5PMpkkm82STCYV8d0Fel58VSoVTqcTs9nM+Pg4wWCQWCxGJBKhUCiQSCTodDr7fZiPjE6nk1ZGMBikXC5TrVYpFoskEglarRb1ev2xJzoLi8dut6PVasnn81Sr1X2bFK2w96jValwuF0ajEZfLhdlspq+vD4fDgd1up6+v766faTQaJBIJNBoNp0+fxuPxEAwGqVQqFItFyuUyH3zwAbFYTLl3HpOeF1+1Ws3w8DChUIh/9+/+HV//+td58803ef3111lcXCSTydBqtQ6sAJtMJkwmE1/96lf5gz/4A6LRKBsbGywvL/P2229TKpVIpVKPdaOr1WqMRiNGo5HJyUmsVis3btwgFovR6XQO7LVTeDA6nY7x8XG8Xi8nTpwgEAhIA8blcuF2u+/6mUajwdbWFp1Oh/7+fkwmExqNBrVaTbfbpdvt8r/9b/8bv/3tbxXxfUx6WnyFb2poaIipqSn8fj9ms5lAIMDRo0epVqtotVra7fZ+H+pnRqPRoNVqmZycZGBggImJCXw+H41Gg3w+j8FgoNFo0Gw2H/t7LBYLR48exel0Mjo6itFoJB6Pk8/nqdVqPS++KpUKtVqN1WrF6/Wi1WoxGo3UajXW1tZ2ZWdwkNFoNNKdICxdp9OJxWJhcnKSvr4+JiYm8Hg8+Hw++vr6sFqt9/T5arVaXC4X7XYbs9mMTqdDq9WiUql2vEfh8enZq6hSqTCbzdhsNl599VW+9KUv4fP5ADh+/DhHjhzBZrPxox/9iFardaAEWKVSYTQasdls/Pmf/zmvvvoqTqcTm82GWq2mXC6ztLREOp2mXq8/sjhqNBpMJhNDQ0P85V/+JSMjIzidTjqdDoVCgVwuRzabJZ/P7/IZ7i5arRaDwcDMzAxf+9rXcDqdBAIBotEo/8v/8r8QiURot9ufWwE2Go243W4CgQAvvfQSAwMDvPDCC7hcLkwmE1qtFp1Oh0qlQqfToVarUavvneik1Wrxer10u13UavUO0VXYXXpafE0mE1arFafTSV9f346VWmyBxN8PCmq1Gp1Oh9/vlw9Mf38/7XabUqlEJpNha2uLdDr92O4UvV6Px+PB7/fj8/nwer00m02q1SrNZrPnXQ5GoxGDwYDT6cTlcjE+Ps7IyAh2ux2/30+n08FgMKBWqw/U4rvbaDQajEYjDoeDoaEhBgYGGBgYwOFwoNVq7yu02+l2u/Ke+H33Xb1e383D33NUKpW03sXCIxYlg8GAVquV51uv16lWq7RaLRqNhrwee0HPiq9Op2NkZISBgQGCwSB9fX1oNBoANjc3uXXrFjdv3qRWq+3ZxdltxNbZ5XLxZ3/2Z0xPT3Pq1CmsViuXL19mfn6e3/72t/z0pz+lXC4/9nl5vV6+8Y1vMDo6ysjICCaTiX/6p39iaWmJS5cuEY1Ge1q0xsfHGR0d5dy5c3zhC1/A5XIRCoXQarXSmrPb7ZhMpj19SHodvV5PX18fo6OjfPnLX6avrw+LxYJGo3loy7Ver5NIJKhWq0SjURqNxn3fu76+fqAMHmGE6HQ66W6ZnZ3F6/UyOjpKX1+fNOZu3brFlStXyGQybGxsUK1WSSaTe2Kk9Kz4ikhtf38/Fotlh5+pXC4Tj8fJ5XJ0Op0DcyOo1WoMBgMmk4lQKMTIyIh0NeTzeTY3N9nc3CQSiTzWOalUKjQaDVarlcHBQQYGBtBoNDQaDWKxGBsbGxQKhcf2J+8V4vhdLheDg4OMjIwwMTGBxWLB4XBIQbFYLDidThwOB41G49C6HlQqFSqVCoPBgEajod1uy8Wm3W6jVqulFScyGbbvDIEdO51Op0O73ZbXq91uU6lUiMViVCoVIpHIA63bXC73BM760RHWrVarRa/XY7FY8Pv90tVnMpkYGxvD4/FIX7i4Xs1mk1wuh9VqpdPpUCqVUKlUNJtNuSMQlvHj0pPiq9VqsdlsvPDCC8zOzhIIBHa8vrm5ye9+9zuWl5d72nK7E51ORzAYJBgMMj09zczMDCaTiW63y5UrV/jHf/xH4vH4YwuI2WzG4/EwNTXFV77yFaxWK2traySTSd577z1u3bpFOp3epbPafcQD8oUvfIE//MM/pL+/H6/Xe5cP0uVy8Z3vfIf19XVee+01wuHwrj0YvYKIfRiNRs6ePUsoFJJuqWg0SjgclkIsCiH0er20XHU6HQCrq6uk02kymQz5fJ50Ok08HpcpjY1Gg1KpRKvVkv+9H5lMpqcXOZvNhsvlYnh4mNOnT8tsD4vFgs1mQ6fTYbPZ0Ov1mEwm9Hq9PB+/38/58+dpNBqUy2UqlQpbW1uUy2XW19fJZDL87Gc/25U4Q8+Jr7B69Ho9AwMDDA8PY7Vad7ynVCoRjUbJZDI97bO8E61WK5PbnU4ndrudVqtFrVYjmUyyvr5OpVJ5rO8QwTyXy4XP5yMYDKLVallYWCAej5NIJEgmkz3rtxPHb7VaCQQCTExMYDKZMBgM8j3ihtfr9YyOjsptZT6fp9vtUq/XpXV40FGr1ZjNZiwWC0NDQ0xMTMi0LxEo7Xa7UnxLpRJqtZparUa325Vxkng8TiwWIx6Pk06nicVihMNhstkskUiEZrNJs9mk3W4fuOwRsTMQFq/NZsPj8RAIBJiZmcHv93Pq1CksFgtWqxWNRiN3zOI8RTqd1WqVgf1ut0ulUiEajVIoFLBYLMTjcd555x30er28zx6VnhNfEYwSwjs0NITFYgE+DbLF43GuXbtGuVw+EDeJRqPBbDbjdruZnZ2VC0qr1eKtt95icXGRS5cuUSwWH8tqs9lsOBwOzpw5wx//8R8TCoWwWCzk83nm5uZYXV0lk8k89k2zl2g0Gk6ePMnk5CSTk5N3uZy2YzabOXHiBOPj4/h8PtLpNB988AEbGxvcunWLtbU1ut3ugRRhkZlgt9v5t//23zIxMcGxY8fo7+/n9ddfJxqNyl1AsVhkZWWFUqlEs9lEo9GQzWbpdrs4HA40Go0UkEqlQq1Wo1arUa1WpbW8XYwOwjMlUKlUuN1uLBYLU1NTDA8PMzY2xtTUFC6Xi2AwiMlkoq+vD5VKJa9RJBKhXC5TKBRoNBocO3aMsbEx8vk8mUxmR1qj2WxGo9EwMzPDwMAAzz77LD6fT+bKPyo9J74ajUZuG/r6+naUQIoHqVwuE41GD8xDJaLRwpobGBjAYDDQbrdZXFzko48+IhwOP7Y1Knx+IyMjvPDCC9hsNrRarfT1RiIRKpVKT2/L1Wq1dMt4vV70er0UmTtFQavV4vf76Xa7+P1+qtUqnU4Hs9lMMpkkHA737CLz+xAReqvVypkzZzh9+jShUAibzcZ7770nLT24XRiRyWSoVCqUSiU6nQ7JZJJ2u43H40Gv15PNZqlUKgdKWH8f2/O/nU4nk5OTnDhxgunpaU6cOIFOp8NoNKJSqaQ/t1qtSr92LpcjmUxSrVbx+XyMjo5SqVRkUZPX65W7cI1Gg8FgwGw2Mzo6SrfbZX19/bGOv+fE12AwMDU1xdDQEGazecdriUSCVCpFMpncp6N7NDweD6+88gqhUIjnn3+evr4+6vU6m5ubzM/Pc+XKlV3xwTocDkZHRwkEAtjtdprNJleuXGF9fZ2PP/6YcDhMsVjchTPafVQqFVarFavVyszMDGfPnpXbv1KpRLFYpNlsUqvVMJlMMogoflaI9OzsLB6Ph0gkwsLCArVa7cAJsAg2fulLX2JwcJCjR4/S39/P0tIS2WyWixcvsrCwQCaT2fFzIlgkgmjdble6IRqNxqESXqPRyDPPPIPf72d6ehqfz8fIyAiBQEAWmpTLZSKRCLVaTea0v/POO7I/hbD+2+02drsdj8fDJ598wrvvvsvU1BRf+tKXMJlMWCwWGWgzGAy88MILHD9+nI2NDZkZ8ijB654TX1HRNjo6islk2vFaNptlY2Oj5x3+d+J0Orlw4QIjIyOcOnUKk8nE6uoqiUSCtbU1FhcXH5ja87BYrVaCwSBerxez2Uwmk2FxcZGlpSVu3bpFNBrdhbPZG4RvUywgx44dQ6/XA1CtVkmlUlSrVQqFgsyCEeILyEqsiYkJAoEAv/nNbzAajbTbbWq12n6d1mdGWHMOh4MXXnhBptvZbDbeffdd5ubmuHXrFuFw+K4HXuSKb+dxYwi9isFg4KmnnmJmZoYzZ85IF9v2WoBsNsvm5iaFQoGNjQ0ikQh///d/LwOMQkPUajUnT57k+PHjXL58mTfeeIN8Ps/Ro0dxu90yw6ZaraLT6Th9+jTtdpsf/vCHMs3xUIiv8PkGAgEZZBEr+fLyMu+++y5ra2v7e5CPiPCniVW0XC7LRO5HWUxEgGF4eFgGFZ5++mlGRkZQq9Wk02neeustNjY27nooew2RWujxeLBareh0OkqlEpVKhYsXL/L222/LoJKwbF0ul0xBE345UXRx5swZyuUy169f5+LFi7RarZ5NrYNP3QwOh4PJyUmGhoaYnJwkGAyiUqkol8ssLCzw0UcfyQDZQbPoHxdRsSlK8U+ePMmRI0fweDxyoa1WqyQSCaLRKGtra3zwwQeUy2VSqRT5fP6+cRWdTiczHzQaDalUivfff5/R0VEGBwel2xDYtZYGPSm+oVCIoaEhuYqJVJq5uTl++tOfkkgkDpTlu53tfutisSiDX49yPqKm/+jRo5w7d45Tp07xzDPPyHzQeDzOj370o11JX9tr1Gq1rPiz2WwYDAYikQhbW1v8/Oc/56/+6q9kEMrn87GyssLg4CDf/va38fl86PV6+QCZTCaee+45xsfH+cd//EdZjNPL4it8iz6fjxdffJHh4WFmZ2fp6+ujWCxSKBS4evUqv/nNb3r+XPYCkbfrcDh45plnGBkZ4dlnn2VkZETGBMrlMuVymcXFRd5//33m5uZ44403qNVqMoPjfs+BXq/HbDZjMBhQqVTEYjHefPNNTp06xauvvoper5c7MdidKr+eEV+Re+fxePB6vbjdbpmj2G63Zf5hNpulWq3u89E+HhqNBrfbjUql4uzZsxgMBpnqk8vlSKVS9/1ZtVqN3W7HYDDISqZz585x/PhxgsGgbMiTzWZlS8peF174NF1IuBK63S6JRIJbt27taBsqItarq6u0221yuRwWi0UmyovgiljkRMpZr18Ds9mM3+9nZGSEo0ePykBiLpfjww8/ZGtri3A4LItJPm/odDo8Hg8DAwNMT0/vSEEVPleR4XLr1i2uXr3K5uYm9Xr9vuXSIlag1+spFAqsrq5isVh48cUXMRqN2O12RkdHZV8MuK1FiUSCbDYrC5Ue9ffRM+JrMpkYGRnhyJEjTExMMDw8vCOaW6/XSaVSbGxs7PORPj5arZbx8XGGh4cxGo2yrLNer3PlyhU++uijB67Q09PTuFwupqamZIlkIBCQbohMJsPly5dZWFg4UBaS8Nuq1Wo6nQ7z8/P8/Oc/Z3FxUV6PZrMpU8pisRhf/OIXMRgMBAIBLBaLfJ9wUYgqsF4XLJfLxcmTJ5mdneUP/uAPsFgsss/Hf/pP/4krV67I++TziMViYWJigrGxMV555RUGBwdlCqqYrPHLX/5SpuFtbGzIxfdeiGfFbrdjNpuJx+O89957zM7O8pWvfAWDwYDFYkGn0+0I/DebTebm5giHw2xtbVGpVA6++IothWj2LYS32+2SzWbJZrOUSqWet2DuRa1WY2trC71eT6VSkU09RPs+lUpFvV6XqTAPslZ1Oh3Dw8PY7XYCgQBOpxOr1Sr9ULVaTQbaNjc3e150BGq1GqfTSX9/v0wPEs1itm/34NMYQKfTwWg0Yjab72oeU6vVZMP4g9DvWbSDFOcrAjyFQkH+OUgL6V4gXA9iAIGwRkXJtEjpdLlcwKeunO2I3ZHRaJQd3CwWi+wj4/F4cDgcMk1NfI8oZKlWq4TDYVZWVigWi4+1q+oZ8RWWbygU2nHBWq0WV65cYX5+/rHz6vaLdDrNm2++ydjYGE899ZRcaES58cDAAHD7xjh58iTf+ta3Hvh5YhskOjWJrXq5XCaXy3H16lW+//3vk06nD0ykX6/XMzs7y+zsrGzyHQwGOXXqFOl0mosXL951k4vMmOHh4R2ZMd1ul2QyyeLiIrFY7ED0LLbZbBw5coRQKIRGo5GFABsbG8Tj8QNXzbnbiPtc7IqEa0mlUkkL1+12y4pIq9WKzWajv79/hyHXaDRQq9UMDAxgsVgYGBjAZrNJt5/dbpfiLVxh8GkmSSKR4K233uLKlSuPnUfeM+IrEsqtVqu8WOKiZjIZIpFIz0fs70er1ZI+2OXlZarVKh6PB4PBIFdgi8WCyWRCp9PR6XRka7s7Eb142+22bDokaDQaFItFWaVTKBQOzAMrkuC3C6XJZJKWvQiECKtH5PpaLBaZFSOazMBtS9JqtWI2m9Hr9bRarZ4uLjEYDLjdbux2uywfFr0HREtV0cHv85jpIJr/iKBapVJBp9PJ+0Gn09HX1ycrYoX4ejyeHRZyq9VCrVbT39+P2WzG6/VitVqpVqvUajWMRuOOXZTIoBAWryjRz2azjx106xnxFZbv4OCgbHRRq9UolUpcunSJX/3qVz2dp/ogqtUq6+vrxONx4vE4FouFwcFBbDYbk5OT9Pf3c/78eY4dOybLPcPhMJcvX77rIatWq1y8eJFyucx/+A//geeee06+ls1mmZ+fZ3V19cBYfIJGo8G1a9coFAqyTNTj8XDkyBHm5+cJBoOy7t7tdnP8+HHZbtRkMsm8TeHSmZycxOfzkcvluHTpEoVCgWQy2bNuK4/Hw7lz5+jr65P3vyhn/da3vsXx48e5efOmfPjvLLA47JTLZW7evCnTB8vlMlNTUzgcDiwWCxaLhZdeeolnnnlG+nOFmwLYEbAVu0bxuujvK9xdAhHwvH79Opubm3z/+99na2tLuhwedwHcd/FVq9XSVyPaA4qVp91u02g0KBQKB2oLfSfCkm02m6ytrWEwGKjX6zKftVKpEAgE6Ovrk+WPm5ubrK2t3fULrtVqrKysyFJSlUol/Z+FQoFYLCZX5V629O6k0+nIzAXRFMZgMGCz2XC73QSDQWw2m/TLjY+P43a7ZavMTCZDs9mUPjzhrxNl6p1O57Fn4e0ler1eWvlCIMxmM91ul1AoJNMTxc5IBBI7nc5d22PhE9+OEKPtrSR79VrcC2H55vN5otGobFJVq9XkzkgEyba7JcR1EB3xxCSPO3sd3xkzEINrRatXMVdRdILbDf/7vouv3W4nGAxy4sQJnn32WVmdJbpTVatVcrkc6XT6wG+1RHFFvV5neXkZjUbD6uoqOp2O1157DavVKm+YWq12X7eBmMsmHp5sNksul+Odd97he9/7nqzrP0i0Wi1WVlbI5XIkEgkqlYrcOv6rf/WveP7553fMKrNarbTbbTY3N8nlcrz22mskEgn+9b/+15w6dQqbzYbFYmF6epqvfOUrXLt2TbZf7EVEqqXZbJaiIFxxL730ErVajWKxSK1W49KlS7ItaDwel2lR9XqdbDZLrVYjHo9LgVCpVLJ4JRaLkU6n5Rb+oNDpdGg0GsTjcf76r/8ao9GIx+PBYrHIfF+xMNfrddk8qFAo3NUpUfTTvrOCdjuxWIzLly+zvLzMv/zLv8hMq90c3rDv4qvT6XA4HLhcLrxer+w+JKxecSF3o/y2FxCWh3gwPmtjao1GQzAY3NHbtlKpkMlkiEaj0qfcqyJzPzqdjuw7IUaUC5+txWJheHhYLjYiFiDyvre2trh+/TqRSIQvfOELjI+Py+GPTqeToaEhotGovGa9aPHd65jE9nh7i0MhQiqVimg0KpvzC5+wWLiE8SI+R+wchO9brVZL6/mg7JC63S7VapWlpSW5MxA+cXFPiBatlUqFSqVCNpsFPq1gMxgMspXrvRCfkc/nWV9fZ3V1lcXFRfL5vOz+tlvsu/iK1d1kMu0QlFarxdraGpFIpGebwewHRqNRVm8FAgE6nQ6XL1/md7/73a60pdwvxIPV6XT4yU9+wtraGi+//DLnzp2TgRUhnNlslitXrrC1tcWPf/xjYrGY7I9x8+ZNLBYLTz31FH19fXg8Ho4fP046ncbn88mAZK8tTtFolF/84hcEg0FOnz59V4oUfNr3QfizhYtKWHWdTkdaZtuFQsxDFK0m8/k88XhcjuP65S9/eeDuGbG4tFot3nnnHa5duybFeLtbplarSV3xer380R/9EaFQiOHh4R2fJxalGzducPHiRZaWlnj77bfJ5XLSpbXb98y+i6+ox9fr9XdFGUXT589rYvm9ELPtZmZm5LiYra0trl69SjgcPrB+cUDmN9+8eZNcLsfExAQnTpyQFiB82uB6eXmZ1dVV3nvvPdkoRafTEY/HCYfDHDlyBECOkPF4PNjtdtrtNoVCYT9P857k83lu3bpFq9Xi2LFj9x18KVwIHo/nkb6nXC7vyFXtdDr8+te/PpAjmIRgPkyvF2H9v/zyy7hcrrvccqIoZ3Nzk48++ojl5WUuXbq0pzni+y6+TqeT48ePMzY2hkajkbl4xWKR69evc/PmzZ4eefOkMBqNjIyM4Pf7OXv2LEePHsVms8m2eOVy+VAk4Xc6HaLRKMVikZ///Oek02kCgQBDQ0PSXbO2tsbPfvYzEokE+Xxeinar1eLq1avE43HpshCTQ8bHx/niF7/I6uoqv/jFL3puQd/a2uKNN97g8uXLrK6u0t/fz4svvig7aomub9tLXR8FUcAhskdE/CEWi3Ht2rVD497bjsvl4tixYwwPD/PUU0/JDmgCMcZLWL0ffPABuVxuz6eh7Lv42u12JiYm5FRaIb6iQcaNGzd60lJ50uj1eoaHhxkeHmZ6epqpqSnZMET4xQ/a1vFedLtd0um0LCGORqPMzMxw6tQpWQEoutsVi0Wq1eoOX/Di4iLr6+scOXKEwcFBJiYmGBwcZHBwkPPnz2M0Gvn1r3/dc+KbTCZJpVLY7XaWl5cZHR1lYmIC+DQjSJRfP474CheOyHGtVqucP3+excVFbt26dSjF12azcfLkSUZHR5mZmbmrHWm322VxcZFf/epX3Lx5k+vXrz+R49o38RVR6/7+funD0mq1MqWmUCiQSqWIx+M996DsB6Lbm2hv1+l0ZOWTSP4+bL7xbDYr/ZjxeFxaviJafy9LX/RWjcVizM/P43A46Ha7mM1mQqEQiUQCs9ksO4P1ylZbHIcImrVaLb73ve9Jy1ej0TA4OChTMdVqNYFAQFb3ORyOz/ydosy2v7+fVCp1TzfHQcbpdBIMBhkfH+fChQuyKGf7eZZKJZmHv7i4+MCmVrvNvomvGHkjSkhFfX69XqdYLJLL5dja2iISiezXIfYUBoNBjlA3m810Oh2ZCywCk4eNZDJJMplkeXn5rtfuJ5piq7i5ucmVK1cIBoPA7UbzY2NjJJNJrFarHKfUK+ILyMKira0ttra2uHHjhrRyxQwxn88nxffZZ5/l1Vdfxe12Y7PZHkk8LRYLwWCQRCJx6MTX7XZz9uxZZmZmePnll3E6nZhMph1jqcSEi5WVFa5du/ZEUzT3TXzFtsdut8uabbjdNSgej7O1tdWzE3b3A9HHVuwQms2mjFYf9mqnzyqQ3W6XTCbD+vo6GxsbrK+vo9frZdMVsYCJMUO9yvZCCJHl0W63pSUciUQIh8OoVCoZvRduO9EBLRaL0Wg08Pv9sm+BzWa767u2F2ocdESzpaGhIU6fPs3w8LDsULa9GKXZbLK4uMjq6irRaPSJT7zeN/EVQRC/379jxa3Vaty8eZPV1dUD28thLzAYDHKir16vp1qtcvnyZTmpQmEn4XCYSCRCf3+/bN955swZhoaGeOGFF1hdXZXzvQ4C3W6Xzc3NHQJpsVhwuVx0Oh1Onjwpm86USiU++ugjYrEYv/71r0mn03zxi19kcnKSkydP3lN8DxMOh4NQKMTZs2f5oz/6I2w2246eMaKPSLlc5te//jW/+93vWFpaeuIxk30TX7VajV6vlwEEUdlVrVaJRCKyEfLnHb1eT19fH36/H6vVitFolJH9QqFAJpM5MALyJBFWTDabZXV1VeZ/Go1GOX3WZrPtGOXU69xZElwqlUilUhSLRdnhS6TliZLqgYEB9Hq9LGDaPuMMPi3h7yX/9+NiMpno7+/H5XLJxkvbm3U1m02WlpZk1ZqYYPyk2Vefr91ulzeDqFgSq/XS0tKh304/DH19fXKQos/nw2azySh/LBZjbW3tQAjHftDtdllaWuK1116jUqnwwgsv0NfXx9e//nWWl5d588036Xa7xOPxA7mAJZNJrl27xsDAwI5hkHa7nWeeeYZarSYLS8R0DDEdRiCmnhykDni/D7/fz5kzZ6R7SRSsCIs3k8nwd3/3d9y4cYMrV65Il8OTZl+zHWw2m3SAi7JJ0cmsVCoduP4Ee4HBYMDv98s5ZYCc/ybSy0QeqPjvgzr4f96oVquk02mZOeNwOGTBhd/vp1Qqkcvl5Iyvg4RoOlUoFMjn87JSVKVSSWvP7XbLfOftVq8o3Rdl6el0+sCLr7D8Rb+Yvr6+uwq3xG4xFovJNrX7lR+/b+Irpo+KcUHiIclkMjL9QxGQ2wniL730EqFQCLPZTKPRYHFxkXA4vKNuXavVyv7A1WqVfD5/4MRkL8jn85TLZT766CO+//3vMz4+zje/+U2cTiff+MY32NjY4G/+5m9k6tlBEqBSqUSr1WJ+fp733nuPQCDAyZMn5SItxlW12+0dk8C73S7RaJSVlRU++OAD/r//7/8jn88fSOt/O+I5mJmZ4Zvf/Kac8CKoVCp8/PHHbGxscPnyZebn5/c1r3nfxFev1+NwOGRnIbElEI2iD9JDsJeI6ySyQtrttmwoLZqKi2ounU4ny7Tz+fx+H3pPIBoZ5XI5NjY2sFqtNJtNOc2g1WrJa3fQ7jvR9rBQKBCJRNBoNNJ3KWIpQnTFbki0Nk2lUoTDYTY3N2Xv54Ns7KhUKmw2Gw6Hg/7+frxe746SdBFPikajRCIRcrncvtcPPHHxFdtju92Oz+fD6XTKgJsQX9EUQ7Hc7o9KpWJychKLxcLp06eZmpqS+dFXr17lRz/60aEoN94tkskk77zzDpVKhS9+8Yu4XC6ZNzs6Oko2myUejx+oDBvhqltcXOT//X//X6ampmSrxUAgIPvbajQaCoUClUqFhYUF1tfX+eijj/jtb38rdwYHWXj1ej06nY4//uM/5qtf/Srj4+M70spE4crq6irf//73WV1dJZFI7PNR75PlK3yTBoPhrmwHkfiuCO/9EaWmokzyyJEjHD9+nGQySTQalbmfCp8iquTEFAiDwSBHzDidTmw224EL8IrnJJ/Ps7Kygk6nIxKJ0Gq1duyGtFotpVJJWsiilHhubu5QPGeiq9vIyAjnz5/HYrHsuP9brRa5XI5kMsnq6ipra2s9YZjsi/h2u10KhQKrq6u0220CgQCNRoNUKkUmkzkUPQr2CoPBwMmTJ5mcnOTMmTNySu/6+jrXrl3jo48+YmtrS7mGd9BsNikWi9y6dYv/4//4PxgfH+c//If/gM1m48KFC/T19fHaa69JP/pBotlsUigUmJ+f56/+6q9wuVycP38el8slx3K9++67spgglUr19FSPz4JarSYYDNLf308oFMLpdO7w8wLE43F++MMfsra2JvWlF879iYuvOOlKpUI6nZZJ4q1Wi1KpRLlcPlB+t71GjEQR/9VoNPj9fvlap9Phww8/lNVcYr5VL9xcvYTYoicSCd59910ymQzlchmn08nw8DAajYbf/OY3+32Yj4QYUyXKsR0OB1qtFo/HQ6VSwWg08t5773Hjxg0ZzD4siJ2Lz+fbEUOCT3cGhUKBq1evEolEdr0h+uOwb+K7srLCT3/6U44ePYpGo6FUKrG2tkYsFuuJLUGvkEqleP311wmFQrzwwgtYrVbi8bjs+pZMJmVjHRFIOEwJ87uNmFKQSqVkP1tRejs0NMTq6irFYpFKpbLfh/rIVKtVbty4gclkYmFhAa1Wy/Ly8r6mVe0VGo2GU6dOce7cOUZGRna8Fo/HuXXrFtevX2dhYYF0Ot1TXdv2ze2wubkpgwDBYJB6vS63RAfZ+b/bZLNZ3n33XdnRzG63c/PmTVKpFG+99RZLS0tyLpyI7CvcH1HMk8vl2NzcxGg0Mjk5id/vx+/343K5aDabB1p8G40GKysr+30YTwSNRsPU1BTPPPPMjmITgEwmwyeffMLNmzdZX1/vuWDqvqWaidpq0dy61WqRTqcplUoHPt9wNxGLUrlc5rXXXsNoNErLd2trS7ZWPGhpUvtNsVjkrbfeYnl5WY6kFxO0lTS9g4VOp8NoNMrgvciaisVifPLJJ4TD4Z60+PdNfBuNhqzQmZ+fl/+ubJd3UqvVZOOcGzdu7HhNuVaPTi6X40c/+hE+n4+zZ8/Kakuv19sTaUgKD4dKpUKv12MymWSgTRh24XCYd955h3w+r4jv/VBE5OFQrtPuIVovFgoFfve737G2tiaDMr22PVW4P51Oh62tLebn55mcnMRqtZJKpVheXmZtbY1yudyzpeM9Ib4KCk8aMf02mUzyf/6f/ydqtVq6bhT3zcGh3W5z9epVms0mBoOBwcFBlpeXee2112Sv6160ekERX4XPOcICVjiYdDodEokEBoOBixcv0mq1uHTpEmtraySTyZ5eSFUPMsdVKlXv2ep7SLfbfaiyMOW63I1yTe6Ncl3uzW5eF9EX3GAwoNfrZcc2MQ6+F7jXdVHEdxvKA3VvFPG9G+VeuTfKdbk3n1l8FRQUFBT2hsM1rlRBQUHhgKCIr4KCgsI+oIivgoKCwj6giK+CgoLCPqCIr4KCgsI+oIivgoKCwj6giK+CgoLCPqCIr4KCgsI+oIivgoKCwj6giK+CgoLCPvDArmZK/fW9Ua7L3SjX5N4o1+XeKNdFsXwVFBQU9gVFfBUUFBT2AUV8FRQUFPYBZZJFj6FSqVCpVPLvouVnt9vtyTlUCgoKj4Yivj2CWq1GpVLx1FNP8dxzz2GxWHC5XFQqFTY3N0kkErz55pvKWHMFhUOCIr49gkqlQqvVcuzYMf7tv/23eDwehoaGSKfTXLp0icXFRT766CNFfBUUDgn7Jr5WqxWHw0G1WiWXy6HRaLBarRgMBnw+HyaTCZ/Ph16vJ5lM7hjnXSqVKBaLVCoVcrncgd6Oq9Vq1Go1ExMTDA8PMzs7i8/nw2KxANBoNEgmk6TTaVqt1j4frYLCwcPhcHDkyBEcDgeTk5MYDAbS6TSlUomPP/6YcDi8L8e1b+LrcDgYHR0llUpRLBYxGo14vV6cTidPPfUUbreb06dP43A4uHr1KtFoVP7s1tYW6+vrJJNJ8vn8gRdfrVbLiRMnePHFFzlx4gSBQED6fev1OtFolHg8roivgsIj4HK5ePHFFxkZGeFP/uRPsNvtzM3NEYvFyGazh198NRoNarWa4eFhAoEAg4ODjI2NUSgUiEaj6HQ6+vr6sFqtTExMYLPZGBoawmg0Mj09jd/vl581NDTE+Pg4uVyO2dlZcrkcKysrVCoVUqnUgREptVpNIBDA5XIxOTnJkSNHsFgsJBIJcrkckUiEcDjM1atXicVi1Ov1/T5khR5ErVaj0WjQ6XRYLBb0ej0ulwuj0UgwGMRsNksDJZfLUSwWaTabNBoN+fy1Wi2azeY+n8neoNfr8Xq9eL1eOenY7XYD4PP56O/vp1gsUq1Wn+hxPTHx1el0GAwGXnnlFb785S9LAW2329TrddRqNTqdDo1Gg9lslttxlUrF1NTUDuu20WjQbDZptVrUajUWFhb427/9WyKRCB988MEOF0Uvo9FoOHbsGNPT07z88ss899xzRCIRbt26xbVr13j99ddJJpMsLCzQaDQOzKKi8GQRz4zNZmN4eBin08mJEyfw+Xx87WtfIxgM0ul06Ha7XL9+naWlJYrFIrlcjlu3bvGrX/2KSqVCoVCg0+ns9+nsOmazmfHxcQYHB6X4Dg0N4Xa7mZqaIhKJsLq6SiQSeaLH9UTEV61W4/f76evrk5av2+3GbDbT6XQwGAx0Oh3a7TbdbleuQOKG2Y5KpUKn02E2m2m32xgMBvr7+5mensZisbCwsECn06FWqx2IG8loNGK1WuVNUa/XSafTpFIpaQHX63Xa7fZ+H6pCD6FSqbDZbDidTux2OwMDA1itVkKhEFarlSNHjtDX14fT6ZSWb6fTwePxUKvVKJfL9PX10Ww2WV1dJZPJUC6XD8Qz81nRarXyWmg0GuBTHbHZbLjd7h1uzSd2XE/kS7Rann/+eU6dOiX/K3ydQlyFy6DRaFAqlWi1WlSr1R2iI3Jgh4aGmJiYQKfTATAxMcGf//mfs7KywsrKCsvLy2xtbVGpVJ7E6T0yKpUKi8WCw+HAaDQCkEqluHbtGjdu3JAWryK8CtsRu8LJyUmefvppjhw5wvPPPy/TE7VaLVqtFrVajdFo3JEjPjQ0tMMSnpmZob+/n7m5OcLh8KF0PVitViYnJ/H7/Wi1n0qeSqUiFApx9OjRJ271whMQX51Oh9FoxOPxEAgEcDgc6PV6Op0OrVaLSqVCNpulVCoRjUap1+sUCgWazSbVanXHVlu4IVQqFQ6HA5PJhNVqRafT4XQ6cbvd+P1+yuUymUzmQFi/wlcnAmzVapVsNiv9corwfopwTRkMBux2O1qtFqPRKO8LQIpMrVaj0WhQqVQolUp0Oh06nQ5qtRqDwUC326Ver99zd9XrmM1mzGYzAwMDjI6OMjg4iN/vx2QyYbPZ5LUA5Dnq9Xo0Go0UZoHL5SIYDJJMJlGrD2fBq7Byt5+3QGRMNRqNJ35ceyq+Go2G/v5+XC4XJ0+e5Omnn6avrw+4LTKlUolPPvmEf/7nfyafzxONRmk0GpTLZenPvZfle/78eV588UUmJyd56aWX5EPY39/P1772NTY2NiiVSvcU8F5Dr9dLAWm1WsTjca5evUo8Hu/5heNJY7VaGRgYYGhoiK985St4vV5mZmYwm83y3hAiu7CwwPr6OhcvXuTtt9+W95vNZmNiYoJms8nCwgKVSoVms3lgBFjEQKanp3nppZf4+te/jsFgwGKx7FiEAJrNJmtra1QqFUZGRnC5XHd9Xn9/P1/4whfodrvo9foneSr7TqvV4t133+Vf/uVfKJfLT/z791R8xYpjNBqlf8pgMAC3V+RcLkc8HmdpaYlcLifdDkIwxaq9/fPg9g0zODiIzWajWCzKm89gMOD3+2m321itVoxGY88GqlQqFRqNRvp81Wq1tNTy+TyVSuXACMJeo9Fo0Gg00rcZCoUYHx/f4evfbvmKe0ar1RKLxXC5XJhMJgwGAw6Hg1AoJFP4xPt78R65E5HV0NfXx+DgIIFAAK/Xu0Nw4XaspFqtUq1WiUajFItFuePU6/XSXQe3d6YiS+KwWr73o9PpUCwWSaVS+/L9ey6+RqMRs9mMxWKRIgOwtLTEb37zG65fv87169ep1+s0Gg263a4MvN1p+QkxWlhYIJFIsLy8TLvdZnBwkGeeeQa9Xs/x48cZGBjg2LFj0gLqtRQtESxxOBycPn2al156iXK5zOrqqvxTr9c/9+IrLLn+/n4CgQCnTp3i29/+Nm63m5GREYxGo3QhCLb7Nr1er1z4dTodJpOJvr4+ZmZmyGazfPe73yUcDnP58mWy2ex+neZDoVKpcLvd2Gw2nnvuOb7+9a/j8/nuEl6AfD7PL37xCyKRCG+++SaZTIYvf/nLTE9Pc+rUKaampvbhDBTuZM99vttzELVaLe12m2azSSaTYWVlhc3NTXK53GeyPAqFAoVCAbvdzsrKCmq1mmazicFgkFurvr4+XC5XT26l1Gq1XJC8Xi9+v5+1tTV5XsVi8ZGEV2y97/VAiqDLQRJ0ce/Y7Xb8fj8jIyMcP34cq9WK3W5HpVLdlZ+63eq12+14PB6CwaC81i6Xi5mZGRKJBD6fj2KxuMMS7EXELkkE1AYGBhgeHsZkMsn3CGNF5O4uLy+zvr7OjRs3SKVSjIyMoNPpCIVCDA0NodVqd5y3CM4ZDAZpBB0Wut2uTE290+8rtKndbj9xN9+ei69w8AtBWF1dZXNzk9/+9rf86le/olgsPnJQaXNzkx/+8IecOXOGCxcuyAo5nU7H1NQUarWatbU1tra2dvOUHhuDwcAzzzzD2NgYAwMDu/a5IuhoNBp3bMWbzSbJZJJarbZvwYVHweVy4XK5+NKXvsR3vvMdPB6PTI4vlUpkMhnefPNNuXhvf3gsFgsmk4lsNksymWRqaoqTJ09isVjodrtSpIrFYk+7HNRqNW63G4vFwssvv8yRI0c4fvw4NptN7iLr9TqlUom1tTVee+014vE4n3zyCblcjnQ6Tb1e54MPPmB+fp6bN28yMjIi4yZiRzA2Nsb/+D/+j6ysrPC3f/u3JBKJfT7z3SOVSvGzn/2MoaEhnnnmGWw2G3Db5XLu3DmazSaXL19maWnpiR7XnrsdREqZaI+YyWRYXV1leXmZ5eXlx1phc7kcuVwOm81GoVCQFpFGo8Hr9VKtVndYB72CVqtlZGSEmZkZHA7HI3/O9taTcFtw+vv7pYUk/l0ELkulEuVy+UAEmEQaXl9fH1NTU7zwwgs7Sq7z+TzpdJqPP/5YVv9tF1+Xy4XdbqfVatFqtRgaGpIpfcLHW6vVqFarPR3YVKvVWCwWnE4nExMTzM7O4vf7pbul2+3SarUolUrSzRCPx4lEIjtcV+FwmEgkQi6XY3FxEbvdzoULF+Q1dbvdvPzyywQCAX7wgx+QTCZ7/h55WEqlEouLi7RaLc6cOSPFV61WMzg4yNGjR9nc3Dxc4mswGLhw4QJTU1PSwstms2xsbJDL5Xbte6rVKsvLyzQaDVwuFzqdDr/fT6fTkQ1qegkhvlNTU9jtdrrdLuvr61y6dOmh68w1Gg3nzp1jcHAQn8+H2+3G6XTS19eHwWCQGQBw2/JNp9MUi0U++ugjEokE169fJx6P7+VpPhLb0whffvllnn32WWZnZ1GpVBSLReLxOLFYjPfff59YLMZHH31EoVCQcQKB0+nEarVy/Phxnn32WUZHR7FYLDLDZm1tjZs3b7K5ufnEy0o/CyJjyO/3c+TIEY4ePYrb7ZbGTLfbJZFI8OGHH3Lr1i22trbIZrN3LbAipS6TyVCtVrl58yYfffQRwWCQ2dlZ9Ho9/f39NJtNvv71r7O2tsZ7771HMpncx7PfHdxuNy+++KIsQOkV9lR8RQDszJkzcrtYKBSIxWKP7Ne8F9VqlXA4jEaj4fjx4xgMBvr6+uh2uz1r+QYCAUZGRjCbzQBEo1GuX79OLBZ7qOsizvXs2bPMzMwwOjqK0WjEZDLd5c8T0e9CoYDFYmFlZUU26+lFrFYrTqeTc+fO8a1vfUv+DkVv44WFBX7yk5+QSCRYW1ujVqvd9Rm5XA6r1crp06c5ffq0dMekUimuX7/OysoKq6urPS8uarUap9MpM3xGR0d3+PS73S7ZbJZr166xvLxMIpG4Z3GREGoRV1hbW2Nubg6AY8eOodfr6evrQ6VS8fzzzzM4OMjc3FzPX5+HwW6389RTT+Hz+fb7UHawJ+IrAkoOhwObzSatT1FAkUqldrX/Qr1eJxKJYDAYer5CRyS6C3+dSIXb2NiQ4nsnotevw+HAarVy9uxZfD4fzz33HCMjI/h8PmlBt1otyuUyxWJR/oz4TlF2ajabZSs9kWXSK2g0Go4cOcLExASDg4OYTCYqlYpMSfztb3/L1tYWkUjkgf5ai8WCx+PB6/VKV4xKpaJWq7GxsUEkEun5ewVuXw8RYBP+arh9TxQKBTKZDIuLi1y8eJF4PP7Q5yTiLq1Wi9OnT8sWr1qtFo/HQ6VSkWmhCnvDnoivRqORW2AhwKJnQzablbmHu0WlUmFtbQ2tVttTQnInIu9ZNBBSq9VUKhXa7TbLy8t8/PHH9/wZUZXl9/sJBAL8+3//7zl69Kis2xcZDtVqVVb3LS0tYbFYpHXt8XgwGo0cP36cUCjEm2++yeLiItlstqeumUajYXZ2lmeffZaJiQksFgvxeJxbt25x6dIlXnvtNXK5HNFo9IGBWrvdjs/nY2BggEAgIBe7SqXC8vIy4XC4p877fogmMJOTk9JXub1D2fLyMteuXeN3v/vdPXcA90NY/mq1mi9/+ct4PB6sVisajQafz0er1VLEd4/ZE/HV6/UEg0HZLlH42mq1GolEgkQisesVJQchOKDX6wmFQoRCIRwOBwaDgWg0KqPu29neItDv9+NyueTWaWBgQFpBojyyUCjIQEsmkyEcDqPT6fD5fDidTi5cuIDdbkev12MymaSLYjcXwcdBp9MxPj6Ox+NhamqK4eFhrFYr7XabWCwmo9G5XI5yuXzf37dYrESut8iFrdfrFItFEokE8XicTCbT01kOYrdjNBpxuVy43e4dYtjtdsnlcqytrT1yNWS326VYLLK+vk6r1WJgYOC+qYoKu8+eiK/JZOLUqVOMjIwwODhIX18f6+vrhMNh5ufnWVlZORBiuduYzWbOnTvH8PCwFNBoNMry8vJdVTaiX0UgEODFF19kaGhIpluJXhD5fJ5qtconn3zC/Pw8V65c4f3336dSqZDJZOh2u7IR0V/+5V8yNjbGiRMncDqdOJ1OHA5Hz4wlMplMfPWrX2V6epoXX3yRsbExWeU4NzfHP/3TP8ld04P6MYjsmpmZGb72ta8xPDyMWq2mWCyyuLjI/Pw88/PzZLPZns9yMJvN2O12gsEgIyMjdwWPt7a2eO+992Sx0aOQSqX44IMPmJqa4vjx4z2f83yY2BPx1el09Pf34/P50Ol0tFotEokE6+vr5PP5nr7p9xJRMOBwOORNXqvVKJVKd1XhWa1WhoeHGRwc5MiRIwwMDGCz2dDr9ZTLZer1OisrK6RSKW7cuMHy8jKbm5vk83lqtdqOLahoLnNny8BesnDUajV2ux23243JZEKj0cjEeDFqShz/g4S3v79flhB7vV4Z0CyVSqyurrK1tUWz2ez5e1CkmNlsNqxWq+xxLdx3IoMlFos91jQXkXa3/XqIGIEoR77zdYXdYU8t3yNHjqDX6ykWi1y8eJF33nmHtbW1vfjKA4Fwx4RCIdnqL5fLEYvF7kp3CoVCfOtb32JkZIQvfelLmM1mjEYjrVZLRun/+Z//mWvXrhGJREilUjSbzXvm8LZaLZLJJDabjUaj0ZPpdxqNhkAgIFPCRPCwVquRzWbZ2tqi1WrdV2Q0Gg0Gg4HnnnuOo0eP8vzzzzM9PS19vRsbG/zgBz8gGo1+Jt/ofqHX6xkcHCQUCuH3+/F4PPJckskksViM69evc/nyZSqVyq6Jo7C4RQDO4XBQKBR6rkT/MLBn2Q5i1RRtEUWlUS/nVO41KpUKg8GAyWSSD5JoIi+2jWLbLJrI+Hw+2TazWCxSLpdZW1sjGo2yubkp51A9KHtE9MsQ23hhGddqtZ7xewof5/ZqSJEedS/rTKDVatFoNDgcDsxmM6FQiOHhYZnvLUpLRSDyoAxcFSIo+lKI+0Us2Jubm6TTaSqVyq4GDkVQWNynZrN5Xzp+PQlE6l0ymdyX3t97mufbbrdJp9OUy2WWlpaYn58/EOk9e4UQCafTec/eonDb3SBmuj377LOyG1WpVOK9994jEonwve99j+XlZXK5nMyWeBharRZbW1uyynBlZeVAWzRiQoHNZuP5558nEAhIv7HIDRZTG5LJpHR7HYQeyXq9nqGhoXv2cLh8+TKvv/46i4uLslfxbqHVanG5XHQ6Hdnnt1wu9/xggkfh0LaUFCt1s9mU1tbn2eqFnalm92vfJ6LbIiBmMpmo1WoUi0U2NzfZ2NhgfX2dSCTywG34nd+7Pa8Ybhem3Nkveb8R1vn2XFZhAXo8nrtERqfT4fV6sdvtDA0Nyabiomc0IK1eISAHpVuc6Agodknb/fP5fF6WCu/270/sQHQ6ndy9itE7h41D1VJy+6QBsR3spaDOfqNSqXY0T9+e1iPEcXR0lJdeeomTJ09itVqpVCrMzc2xvr7Od7/7XTY3N0kkEg8tvOJ7xTYSkG6Oz/IZe0273SaZTLK1tSUbJIn76Mtf/jI+n++uY9VqtTidToxGI0NDQ7IybjupVIqbN2+ytrYm0x175ZwfhLhX9Hr9XQt1LBbj5s2bu75rUdLMniy7Kr7ihhFFBE9KfEXLvTtX6F58yO51Xbb/XQxB9Hg8MlgpotorKyufuUObsB7Fgwyfim8vXR9hhWSzWWmRi51CIBCQ/tvtaLVa6RO1WCyycrDdbsvFrVKpkEwmyefzBypqv90Hfqf41mq1PUkRVIT3ybKr4mu325mdnWV8fJxgMIjb7X4irgabzcbx48cZHR2V1p0YJ9NLAiPY3lt3e1AJbrsdhLsBPu1Lu72F4MOi1WplkcaFCxcIhULS59trvvdqtcpPfvITPv74YxqNBqdPn5aTGkRxyJ0Ica5Wq/zwhz8kFosxOTkp+yAMDAwQDof59a9/zcrKSk+5WH4fIjNmcHDwiVWaHcSezweZXRVfo9FIMBgkGAxit9uxWCxPxF8kSm/7+/tlIGv7KPqDhEiZEuehVqtlRZqwmB/2nDQajUzUF2lLYqpzr2Q5CBqNBrdu3WJjY4OzZ8/S398vm6Fvt9oF269BuVzm+vXrLCwsALf9vMLvm81mWVpaOnAtEu8Mzopj38tz2C68B+laHVSeyOj4vcJsNuNyuRgeHubo0aOyz2m9Xuf999+XI+QPEvPz8/zjP/4jTz/9NOPj46jVakKhEAAvvPACGxsbzM/Py9zLB1lzAwMDfPvb32Z0dFSmDIlquP0KMjyIdrtNtVrll7/8JTdv3mRycpKhoSE5/0/QaDTI5XJUq1U2NzdpNBqyqm12dpaxsTHZwSyVSpHJZCiVSoqgPAQiNa9er1OtVntyoT4sHGjxNRqNeL1e2Z5RlN5WKhWuXbvGlStXSKfT+32Yn4n19XWSySQmk4nvfOc72Gw2fD4fGo1GtkZMp9OyG9qDxNftdvPlL3+ZQCAgxXd+fp5Lly6RyWSe4Fk9HMIX/fHHH/Pxxx8zMTEhiwyCwaD0SZbLZdnD4pNPPkGj0fAXf/EXjI6OMj4+zvj4OMlkkmw2KxvuV6tVRXwfAlHc0mw2ZS74QXLXHCQOpPiKCpzBwUFOnTrF9PS0HM6ZTCZJJpMyJasXEsSFK8FqtWKxWDCbzfd1x4iqrvn5ef7u7/6OUCjEs88+i0ql4sSJEwwPD6PX64lGo7z33nusr6/L+VMOhwO3243H42FsbEyKl8FgYG5ujlQqxerqKvF4/EBUeWWzWVlUEI1G5b+L1qRi7LvRaGRsbIzZ2VmcTqdMnhcFKNVq9UB0MNsPnE4nMzMzDA8Po9VqqVarXL58mUgkIg2Bg5wLDrfvl62tLbrdLh6PR7r0tFotk5OTPP3006yvr++4x54EB1J8bTYbwWCQ6elpnn/+eUKhEDabjXa7zdbWFpubmywuLrK0tNQTq7ZGo5HTiu12u2zddy/E2JurV6+ysbHByZMnCQaD+Hw+Lly4gFqtZnR0lHg8TiqV2jFCx+12MzU1xYkTJ/ja175GX18fIyMj5HI5PvnkE1ZXV1lYWHjoaRn7TTqdljuXOyPx261YnU7H8ePHOXfunHxNVIElk0nF5fAAXC4Xp0+flqKUz+d59913WVlZYWFh4cC57e6FaDnbbrex2+1SfDUaDSdOnEClUvHzn//8YItvu92mUqnIOWF7kdojRmhPTU0xOTnJ6OgoLpeLZrNJoVDgxo0bOxr49MJD1+l0ZKWVaHAjJis7HA76+/ulIItjbrVaVCoVtra2+NWvfoXX62VqagqdTie7oBUKBQwGA4FAALvdzvT0NLOzszJVDWBpaYlEIsG1a9fY3Nzc1Sb2T5J7/R5FIYJIM1OpVNJ1EYvFuHXrVs9O6/h91Go1bt26Ra1WIxQKYTab5TkODw9z4cIFotEoGxsbj/U9om2l6IVdLBZl9eNhK4q63z20X/nNuyq+otNSX1/fnlQTiYs0NjbGV77yFUZHRzl37hydTodCoUAkEuH1119ndXWVRCLRMzmdrVaLfD5PLpcjkUjIRt8iO6Tb7XLx4kXZQUo0yMnn81y/fp35+XlsNhtPPfUURqORzc1NyuUytVoNs9nMM888w9mzZzlx4gRnz54Fbgt+OBzmV7/6Fevr6/z4xz8mmUz2xE5gtxBFFi6XS1ozzWaTRqPB3Nwcv/jFL4hGoz2xAH9WCoUCb7zxBsFgkKNHj8opJmLirtls5o033iAcDj/W+Yl0RL1eT6VSIZFI8P7777O0tHToxLfX2HXxzeVy0s9Wr9d3JIvrdDrpn/x9iKosUWtuMpkwmUxyLHwwGJTNU/L5PPPz86yvr/d0o+xarcbS0hKdTodAIIBGo8FiseD1epmYmODkyZNy3LeoQBMYDAZZpRYKhWi327IE9NixY4yMjOB0OuVCFIvF2NjYYHFxka2tLcrlck9ek8dBp9PJXFjRJa5YLFIsFslkMnJY5EGk3W7Lnc36+joWi4WxsTH6+vpwOp0Eg0H6+/tlp7rP6sPv6+vD6/XKmADcLlsuFArUajUajcaBXLS2o9PpZEaU0+mUk817hV0VXxFNF/17LRaLzFO1Wq3YbDYqlcpD3ShinInD4eC5555jbGwMv9+P1+uVgwTFKJ7NzU3++q//mnA4zOXLlymVSj1p4WWzWf7zf/7PcjLFsWPH6O/vZ3R0FLvdzvPPP8/ly5dlQ/TtUyasVivj4+P09fUxOTmJ0+lkaGgIl8slF6lyuUw0GuXGjRv85Cc/YXNzkw8//PChr/lBw2638+qrr8oJGJ1Oh9XVVTY2NmRJca/sfj4rjUZD9m947bXXuHr1Kv/lf/lf4na7GRkZIRAIsLGxwfvvv08mk2Fzc/MzieWJEyf4xje+wdTUFB6PR46UX1pakm7Dg47D4WBiYoJjx47Jqc/3a2i1H+zqkYgcQRE06na7Mjl+YGCA0dFRSqXSQ1kjOp2OoaEhnE4no6OjDA8P4/P55Mqv0+lk05R0Ok08HieZTPZUm8Q7abVa5HI5NBoNa2tr0poXvVODwSDZbJZ0Ok21Wt3RscpisTA4OIjD4cDv98uJtg6HQ0bzk8kk4XCYlZUVwuEwiUSCYrF46CL9orJNVO/5fD70er20fNPpNKVSqWfvg4el3W7TaDTIZDIYjUYymQyFQgGtVrvjvqnVag9dfCPK/91ut6xC7Xa71Go1otEosVjsUAgv3M6KGhgYwOv1Sr92L7GnR6PRaPB6veh0Ov7sz/6ML37xi7In7e9Dr9czPj6Ow+HA4/FgsVhkz4hWq0WxWJRBlRs3bsjRML0sNO12W7aB/Ou//ms8Hg//7X/73/LMM89gsVhwu90MDg7y4osvyh68sHOIpkhbE/8Pt4Nqy8vLvP/++7z11lvk83kSiYT0HR82jEYjPp+P0dFRzp49K3cOnU6HtbU1Ll++fChGnsPtBVsEkD/66CPq9TozMzNyavXRo0fRaDQsLy8/lJU/MDCA3+/n1KlTXLhwAZ1OJ6d8/OhHPyISiVAoFJ7Ame09YvTW0NAQRqNxvw/nLvZEfDudjhxd4/F40Gg0+P1+zGbzQ1u+Wq2W0dFROXp+e2OVer1OPp8nlUqxsbEhfZoHoWNVu92mXq8TjUYplUpsbm6ytbWFz+eTvnFRUqrX6+l0OtL/tv3cOp0O+XyeZrPJ1tYW6+vrrK2tsbq6Sr1e74n85r1C+MqtVqvse6HVamVhwINGyh80xNggrVYrB6SGQiE6nQ4mkwmfz0c6ncZms8miiO33iWgwJBr0uN1uAoGA3DWJ3aOYqJJKpXrSZfdZEAMJRErq9ikg8Ong0EqlQj6f37fd4Z6Ibz6f5yc/+QmhUIhXX32VQCCA0+nE7XbvmNrwIISvWPh1xdZIRLIvX77MrVu3+NWvfkWhUJC9TXtdfOHTDl7VapW//du/5ac//SnT09OMjY1hs9mw2+2EQiFOnDghc3Tv9NmKXGAhvIlEQk4xPqh+zofFZDLJ/r1WqxWj0Si33cLldVjEV0yBKRaLvPnmm1y7dk1OpQ6FQvzRH/0Rx48fx+PxsLGxwZtvvimNGyG2JpOJgYEBHA4Hr776KufPnycQCGCxWFhfX+fDDz+UbUvFgn6Q8Xg8BAIB6esVqYiCYrHI3/zN3zA/P88nn3zC5uYm2Wz2iR/nnohvo9FgY2ODRqNBLBbDaDTidDqxWCyfKZ9OiKkYeCis5ng8zurqKktLSywsLBzIB00IxMrKCpubm7TbbWq1Gk6nk76+PlQqFSMjI6TTaVZXV++yaJrNJleuXJFDNHO53P6dzBNGo9FgtVqxWq3SohNd7Fqt1qGI1G+n0WjQbDaJxWKUSiVZuWexWAgGg9TrdSYnJ4HbgVnhshLNeaxWK4FAALfbzcTEBEePHsVgMKBSqahWq2xtbRGLxeTu8aBjMBjk/Dmn0ylTOMXinM/nuXnzJleuXGFhYeFwNFMXVKtV5ubmWF1dJZvN4vV6+S/+i/+CU6dO4XQ6sVqtv/czGo0G169fJ5PJyCqlmzdvsrq6SjqdlkUGB32LVK/XaTab3Lx5k83NTdnBy2az8Q//8A/U63USicRd59lut0mlUofmgfksCPEVZdpiXFUul+P69etcvHjx0C1GYqtcrVZ57bXXmJub4/Tp0zz77LNoNBq+8IUvcPz4cSYnJ8lms1y5coVut8vLL78shddisUiLV1RHXrlyhd/+9rfEYrGejpc8DqlUig8++IBEIsG7774rJ37ncrkdGUVPmj0R31arRTweR61WUywWcTgcnDt3jqGhIfR6/Y6ZVPej0WiwublJJBJhY2ODTCbDe++9x/Xr1/fikPcN4YYRPSkUfj9qtRqj0SitN7ErEn7Lzc3N/T7EPaFer1Ov17lx4wZra2uo1WqZgjk+Pk4gEMDn88lJ1q1Wi2effZaRkRGsVqvMPOp2u7I50fr6OsvLywdmtt3DImIkIl9aGG7/8i//0jMd/fY020H8klutFj/4wQ/48MMPsVgsDxV5bLVahMNhmTRfq9WIxWJ7ebgKBwS73c6JEydkFLvZbHLr1i3C4XBPdmvbbUQvDxELcLvdMi1zYmICnU7HF7/4RTqdDqlUikQiIccRbW1tkUql2NzcZHV1lVgsRiKRkJ95GMhms9IFlUgkKJfLLC4uUigUeioQvefiKyKwb7311l5+lcLnCJPJxJEjRwgGg+j1ehljWFhY2Ndt5JOi0WjQaDRYWFhgY2MDu92O3++XFXB+v182Gfrxj38sdwLdbpcrV66wvLwsMyeEr/wwUSqVKJVKRCIR3n333f0+nPvSW1nHCgoPQavV2hFw+rwiijDEgtNsNvnxj3+MzWbjd7/7HQBzc3Myb7fT6RCNRmURSq80nvq8ooivwoFDVPMZjcZD5af8rIh4gRioubm5ydWrV4FPW3DeKa6K2PYOivgqHDgajQbxeBy9Xk8qlUKtVstGOge98ffjosxgOzgo4qtw4CiVSty6dYtCocDg4CBarZaVlRXW19epVCr7fXgKCg+FIr4KBw5RvNNqtbh8+TJqtZpoNKpYvgoHCtWDticqlepztXfpdrsPVX6nXJe7eZLXRK1WyyZLer1eVmqJHshPYsut3Cv3Rrku9+Ze10UR320oN8696TXx7QWUe+XeKNfl3nxm8VVQUFBQ2BvUv/8tCgoKCgq7jSK+CgoKCvuAIr4KCgoK+4AivgoKCgr7gCK+CgoKCvuAIr4KCgoK+4AivgoKCgr7gCK+CgoKCvuAIr4KCgoK+4AivgoKCgr7wAO7min11/dGuS53o1yTe6Ncl3ujXBfF8lVQUFDYFxTxVVBQUNgHFPFVUFBQ2AcU8VVQUFDYBxTxVVBQUNgHFPFVUFBQ2Af2fYCm0WjEbrdjNpvx+/2YTCb8fj9GoxGbzYZGo7nrZ6rVKqVSiXq9TqlUolAosLS0RK1Wo1gs0ul09uFMFBQeH5VKhUajQaPRYDKZMBgMDA8PY7FY5Hui0SjpdJpKpUK5XN7Ho1V4HPZdfE0mE4FAAJ/Px9mzZ+nv7+f8+fO4XC4CgQAmkwmV6naKnBh5FIvFiEaj5PN5otEoq6ur1Go10uk01WqVRqOxn6ekoPDIqNVq9Ho9er0ej8eDw+Hg5ZdfJhAIANDpdPjwww+Zm5sjkUgo4nuA2VPx1Wg00po1Go3o9XrMZvOOVbyvr4+hoSGcTidjY2PY7XY8Hg8WiwWdTodKpSKRSFAqlcjlchSLRba2ttjY2KBSqZDJZEilUmSzWSqVimL1KhxIhMVrtVoZGRnB4XBw9OhRnE4nJ06cwOVyAbcNEK1Wy+DgIOFwmNXVVbLZLOvr67RaLVqt1j6fycEmEAgwMjJCpVIhl8tRrVZJJpN7oit7Kr56vZ4TJ04QDAbx+XzY7XZGRkbkKg6fiq9Go0GlUsmbUFi7rVaLubk5VlZWuHHjBisrK6yurrK0tES325V/2u22/LuCwkFDo9FgMBjw+/28/PLLjIyM8K/+1b/C7XbveB4AXnjhBdrtNvPz81y9epVLly7x/e9/X7ohlGfg0Tl58iT/+l//a6LRKFevXiUajZLL5ajX67v+XXsqvlqtlpGREY4cOUJfXx8WiwW/34/b7abZbNJoNOh0OqTTaTqdDvV6XQppu92mUChQrVa5cuUKW1tbrK2tyYtRq9X28tD3DYPBIH19NpsNvV6P3W5Ho9Gg1d7+ddVqNVqtFvl8nkajQTabpVgs7vORKzwOFouFgYEBRkdHOXLkCIFAAJvNhtFovOu9Go2GbrcrDZdSqcT58+dJpVKsrKxQq9UolUrKLvARsFqt+Hw+ms0mdrudXC6HWr03eQl7Kr5ms5lvfvObPPfcc9KqFdZpNpsllUqxtbXF+vo65XKZWCxGo9GgWCxSLpf55JNPyGQyNBoN2u02rVZLCvNhxe12MzQ0RCgU4sSJE/T393PmzBlMJhNOpxOAcDhMPp/n8uXLRCIR3n77bS5fvryvx63weIRCIb72ta8xMTHBt771LaxWKyaT6Z7vFc/S4OAgfr+fEydO8MorrzA/P8/f/M3fEI1GmZubo1KpPOGzOLiInYXH4+HYsWMYjUY2NzcplUoHU3y3YzAY0Gq1VCoVWq0WmUyGlZUV0um0FN9kMkm9Xpfbp1gsRi6Xe1KHuC+o1WrUajVOpxOr1crQ0BCjo6MMDAwwPDyMx+PB6/ViMpmw2+10u13q9Tomk4mhoSH0ej2Li4s4HA7q9XrP7gh0Oh0GgwGLxYLL5ZI39PbtNHwaVO10OjQajR3B006nQ7vdptlsSkvfYDCgUqmk20ks1AcNlUqFVquV18hsNgO3r0epVKLZbNJsNuW5qdVqzGYzVqsVq9WKSqWiVCoxPj6O2Wwmk8mQz+cpFos0m03FFfF70Gq16HQ6jEYjZrNZ6tVeCS/ssfg2m03m5ubQ6/UcPXqU/v5+CoUC6XSa1157jb/5m7/Z4X4QN0mn06HT6fSskOwmFosFk8nEt7/9bV5++WUGBgYYGhpCp9NhMpmkO6ZerxOPx4HbGSLChdPpdFCr1Wi1WpaXl7l169Y+n9G98fl8jIyMcO7cOf7sz/4Mg8GAWq2+p/iKwNHS0hLhcFi+Vq1WyeVybG1t8fbbb9PtdhkZGUGn01Eul2k0Gqyvr5PNZp/06T02Ilsnl8vtcBdUq1Xef/994vE48XhcWmJqtZrTp0/z/PPPo9VqcTgcHDt2jP/uv/vvSCQSvP7662xubvLWW2/JHaXihrg/brcbt9vNwMAANpvtvruO3WRPxbfdbpNMJolEIgwNDeF2u+WDlcvl2NjYkBbL5w2VSoVarcbhcOBwOBgeHubIkSN4vV78fj+tVotqtUq9XieVStFqtaQVaLPZMBgMeDweTCYTHo+H/v5+4vG4/FydTrcjELnf1qDBYMDpdDIwMMDU1BRGo/GeOdxwe9FutVqo1WoMBoP890qlQjabRa/Xs7q6SrfbZXBwEL1eL/O+q9UqarVaWoliIRd/xOLeawhBFZaWsOIrlQpbW1uEw2FisRjFYlG+z+fzkUqlsFqtuN1umbZpNBoZHR1Fq9Xi8Xio1WqHOk4iFnGRUSWMFfH7fhi0Wq38eRFb2evskT0V30qlwo9//GM+/PBDLBYLVqsVtVqN1+vF4/Hg8XioVCoUCoW9PIyeQ6VSYTabMZvN/PEf/zFnzpzh2LFjjI+PS5fM3NwcP//5z8lms6ytrUlhUalUuFwubDYbf/Inf8KJEydwu908/fTTFItFLl26hNfrZWpqinq9TiwWk1ZVs9nct3MWi4Tdbker1d5XeAH5+vj4+I7MGLE7qtVqfPOb3wRu7xzUarV8UJaXl0kmk6ysrLC5uUmhUJBpiCKAKwK8vURfXx9PPfWU3PVUKhWuX79ONBrlP//n/8zCwgKNRoNWqyV9vktLS1y6dInTp0/zne98R6Z0ulwuXn31VQqFAh6Ph3A4zI9//GNu3rz5mQTpICCeB4vFwrPPPsvJkye5ePEib7/9NtVqlXw+/0ifW6lU2NjYIJlM7pnhsqfi22q15AOQSqUol8sYjUaMRqO8UcTNdJhuiN+HWKUtFgtjY2OcOHGCgYEBrFYr2WyWbDbL5uYmV69eJZlMyiKSarUK3N4i2e12nn76aYLBIN1uF4fDgclkQqvVYrVaCYVC0r8ucqRbrda+XmfhYmg0GlJAgPsek/C/3Um322V4eFj6ScW/tdttHA4H6XRaWjGZTAaTyUSpVEKr1VIqlaQPtZdyYk0mE/39/dIfXq/XSSQSRCIRVldXWVlZuefPtVotHA4HhUKBbrcrLbdAIEBfXx9TU1NYLBbef/99TCYT9Xp9XxfhvcBgMMj86JMnT5JIJDCbzbTb7YfWFnE/inuy2WxSqVSoVqt79szsqfh2u11Z+HDlyhV0Oh1nzpxhenpaBgp66QF4UhgMBs6fP8/w8DDHjx9ncHAQgFwux89+9jP++Z//mUQiwcrKiiyhFttmgEKhQK1W47XXXuOTTz6hVqvJ983OznL27Fn+zb/5N6jVasrlMuvr6/zP//P/TDgc3jcBjkQiNBoNaYFbLBYcDgfdbpdisSjvg+3ltaFQCJ/Pd9dnCReGTqeTlq/4OZ/Ph8vlwuPxcPbsWbkFFVv4WCzGO++8Qzwe59133+2ZFD2n08nx48dxOBxotVqKxSK//vWvWVpaIp1O3/NnstksjUaDbrdLs9lkaGiIF154AYfDQTAYRKfTcfToUYaGhigWi8zOzvLOO+9w+fLlQ5MTr1ar6e/vZ3BwkJmZGU6ePCkNF1GG/TDnabPZ8Pl8WK1W4LaBIIq69mqXtOfZDmKVjcVirK6uMjk5iV6vx2AwYDQa0Wq1nzvLV6fTMTIywtTUFAMDAzidTnK5HOVymYWFBX7605/uiGzfiRCTmzdvSr95u93G5/MRCAQ4cuQIFy5ckP7Subk5/uN//I9oNJp987GLyDvcFhpxs3c6HenTFuj1ejQazX3LxC0WCwaDgW63i9Fo3BG0s9lswO3dwXaEy2J9fZ1SqcTq6iqXLl3qGfE1Go309/fL6s96vc7S0hILCwv3TRmrVqtyNwSQTCaZmJiQ94Jer8fn8+F2uzl16hROp5OVlRWuXbt2aGItKpUKh8OBx+PB5/MxMDAgdxD5fP6ugO79PkOv18u8ergdryqXy7L2YC/Yl94Owufp8/nQarU7rDpBp9ORRQT7vV3eLYxGIyMjI/j9fp599lmmp6ex2+2Uy2XeeOMNfve733H16lWazebvXW2FxVir1WRq0uzsLF/+8pcZGRlBq9VSLpcJh8MsLi5SLpf39TqKQFcikeDDDz+UKVUiq2X7+YqA0vXr13E4HHd9lsViYXBwUG41RSGCWq1meHgYl8slU/cEwkXh8Xh4+umn8Xg8/OxnP6NQKDzU9d5rSqUSy8vLuN1ufD6ftGYfJktB7G7q9ToOh4OhoSG8Xi/9/f1yIRsZGcHpdDI3N0c0GiUejxOJRJ7Q2e0dKpUKv9/PxMSEzIOv1+vk8/nP5DIYHR3l6aefJhQKyQDl1tYW+Xz+4Fq+2xGReEAGBkSw5M6LJKL7Qpj3O1q/G+j1ekZHRxkeHubUqVNMT09Lv9L777/Pd7/73c+UpyosIuHfHB8f5w/+4A/kA1etVllbWyMcDsuquP1CbHNzudxj524L8XU6nZw8eVL6hXU6Hc1mk+HhYfR6/V3iq9FocDgczM7OotPpcDgcxOPxnnB9lUol6RbyeDw7Uu5+n4AIC7hSqaBWq0kkEnz5y1+WLgzhA/b7/UxOTrK8vEyr1ToU4qtWq+nr6yMUCsldT6PRkFbrwyAEfHZ2Fo/HI114yWSScrl8cMVXlMuOjY1x/PhxuR0cGxvjq1/9KtVqlWKxeNcN1mg0mJ+fJ5fLEQ6HyeVypFKpA110odVq6evrw+12y+1NoVCgUChQLBYfuUCgXq/Ln99+HcVW+7Al2TebTTKZjNxyi2up0WhIJpP09fVJ36/f72doaAiXy0UwGESlUqHT6bDb7Rw/fhyTycT169f3/b5qNptks1msVus9f1dqtZpgMIjD4cDv99PX10cikSAajcr3GI1GHA4Her2e9fV1ms0mk5OTUpTEjtPlcj2RPNa9RhSl+Hw+hoaGqNVqLCwssLa2xtbWFqVS6aHve7VajUajQa1WS0NhrzND9lR81Wq17Nc7NTUlW0UCTE1NMTk5CXBPn2+tVuOjjz4iHo/z/vvvs76+zrVr1/b9IXkctFqt3A4Kf2yhUCAej0vxfBTEtn27/w8+zRU9bAn2jUaDRCIBwPr6+g6/nk6nk0UHFouFCxcu8NJLLzExMcHAwIBsYONyuTh//jxer1cu7vtJrVYjmUxitVrvuQCr1WomJiYYHx/n9OnTTE1NcfXqVT744IO7nh29Xs/CwgKZTEYWDQjMZjMejwez2XygYy1iETUajQQCAcbHx1lbW5N+8s3Nzc/k1xauLnFNtueF7xV7Ir7iwphMJs6dO0cwGJQNoYXVIkphhdUmTlKv1+P3+1GpVFgsFrxeLydPnmRoaIiRkRFisRhra2sy71EIz0EQF7VaLfOdNRoNnU6HSCTC4uLifSPaD4PNZsPlcsltpsihrdVqbG5uEo1Ge2JrvVdsf0DEA1epVGi320QiEebm5jAajbIasN1uU61W2dzclC6Z/aZWq5HJZHC73XS7XTQaDXa7HafTKRfP6elpTp48ydjYGD6fj+npabRa7V0CYTAYZGOeO1P1vF4v4+PjxGIxlpeXZf7zQeuZolKpsFqt2O12rFYrZrOZRqNBKpWS2UEPI5wajQadTofZbMbhcNBqtWSv8L1emPZEfEXdeX9/P//Vf/VfcerUKUKhEA6Hg62tLZLJJJlMhmQySTKZZHl5WYqn0+nklVdekUETr9fLU089JR+eTqfD3//93/Mf/+N/JJ/PE4vFaLVae9Lybbe50+3Qbre5fPkyb7/9Nuvr64/0mSqVioGBAVmQIPy9cNuqvnz5MuFw+C6r+LAiREREuq9du0YsFkOj0fBHf/RHMoc2m81y8eJFlpaWeqLIp1KpsL6+jt1up91uo9Pp8Pl8VKtV+ft89dVXefXVV6Ufd3JykpdffvmenyesuO29CVQqFWNjY7jdbjqdDplMhmg0yvz8vCziOSiWsEajwe1209/fj8fjwel0yiyWTCbz0Ochsq5EaXE4HJYFOntt0O2p+FqtVpxOJ06nU5bJzs/Ps7y8LCuPcrkckUhEnqjNZsPpdGK322UZrQiumM1mmYx+7tw5Ockin8+zuroqczp78QYSEwpEDbko/xWjjx4l8V0khXu9XiYnJ/F6vfKha7fbskPc53W0kig6cDgc0gIUgTfx706nk2Qyuc9H+qnbYXNzk/n5eYrFIul0mmKxSLvdlk1exM5meynyZ0EUJASDQU6ePEkwGMTtdstnqFarSUu4l+8ZYfk6nU70ej0qlYpKpUI6nX7onsYiTc3hcMg0s2azSTqdplQq7fk57In4arVa+vv7CQQCBAIB+vv7uX79OltbW/zDP/wDb7zxxo56++3bHZVKxc9+9jPZ1d9sNnP+/HmGhoaYnp6W+bHPPfecFN+5uTn+6q/+Srao7LUKHiG8drudEydOcOTIEYxGo+y7ms1mH2nrKx7Gp556in/zb/4NHo8HnU4nM0WKxSKRSIR4PN5z1+RJIcZTjY2NSdESRRqnTp3CbrfLngn7STqd5uLFi2xublKpVKjX63z00UcUi0WsVis2m03+Xg0GwwPLsx+ESEt84YUXuHDhAs1mk2q1yuLiIt/97neJRqN88sknlEqlno4VaLVahoeHGR0dld3+4vE4N2/elPGA34dGo2FycpKxsTFGR0ex2WyUSiVu3brF1tbWwbR8RaBH5B/qdDoWFhaIRqNEIhEymcwDf75SqaBSqajVahiNRsLhMJ1OB61WS6PRYHh4WPaJcDqd9Pf3Mz4+jsVi2VE+2ks3zvaGN0ajUVotj+rYFyXKRqMRp9Mpm9UDssF6LpejUqlQq9V6cjewlwihdblchEIh+vr6dpQzi/ziO3OM9wvhh87n84TDYZrNJqVSiVqthk6nQ6fTkUgk2NzcxG63Y7FY5D0lmsI8DGK3JO4dkRFTLBYZGxvDbDZTKBSkS0+UqPfCNdqOcOH19/ej0+l2NFEym80EAgG5ywF2pLiKv2u1WiYmJhgeHsbhcMhgm+gbvufnsBcfWq/XWVlZYWNjg7/8y7/EYDDsmL7wMIigSbVa5YMPPpBVcTqdjmPHjnH27FkGBwc5deoUg4OD/A//w//A5uYm//v//r+zsbEhLYheQtwMj9sjVJTfDg8P4/V6GRkZwefzyRstHo/z3nvvycYs+Xz+UAfc7oXNZsNqtXL27Fn+5E/+RE7Cbrfb1Go1UqkUb7/9NktLS/ue6QBIgWs0GjLYI3ywomjgBz/4AZcvX2ZmZobR0VHpr+zv72dycvKR7iuxKxsfH+ff//t/T6VSYWVlhUQiwXe/+12WlpbkDMVeQavVykyWM2fOSH+vVqvF5XJx7NgxxsbG5DSY7UG18fFxmWkkAuCiug0+DcCJniF7eh578aHClwk81i9NrLZ3fobVaqWvrw+AYDCI0+lkaGgIlUpFMBik2WySy+Vk4+1eWLW3z5oTUXe4nRq1PUj2IIQ/V4wZ8vl88vx1Op18X7VaJRqNkkqlqNfrnzvhhU99my6XS1pHwhdeqVQoFotyd9ArLhlhud15PGJnFI1G6Xa7mEwmmcZpMpnkyJvt99CdQnxnLwdhzAj/t5gfJ5oe2Ww2BgYGyOVy0gLvhayi7da+0+nE7XbLUnORHTU4OCjFV/T+0Gq1mM1mRkZGZI6zqHrcPiNPFN+ITJFarbZn09D3fXT8o7C+vk4mk+Hjjz/mt7/9LTMzM/w3/81/g8lk4r/+r/9rstks//RP/8TKygoLCwv7HlAR21zhDkin0zidTum3OnnyJLVaTTZLvxfCB26xWDh58iQ+n4+vfvWrzMzM4Pf7d7w3Ho/z9ttvs7W1tWc3Ti+jUqkIhULMzMwwPDy8w0eazWZ55513WF5eJp1OH5iJ1+12WxYPLCwsYDKZ5C5KTAcRAiLuFZHOCLddedvvBdGAKRQKyYo/8UcMu/3Wt77FmTNn+PnPf878/HxPFDmJNFaDwSDdDsIo+YM/+AMuXLggu+EJN069XpeulJs3b8qFSa1WMzIygsvlkgHNmZkZPB4PH3/8MbVaTQ7S3Ivn6ECKb7lcplwuy3Q1rVZLPp/HYDAwNjZGpVKRFS+RSERawfv5kAlfUj6fJ5/PY7FYZDFAf38/DodjRzrd9vZ2YoUWGSCDg4OEQiGmp6c5duyYfI+wbqrVKrFYjFQqdSCEZTcRlpHdbmdgYEBahOJainsiGo1Sq9UOVG6r2AFms9m7CkvEOCW4vS232Ww72m0Wi8Ud6ZidTkdORh4dHf3/2zvT3riu847/Z7kzc2ffZzgLh8OdFClRsk1ZsqQ4sR0ZWQy4rRMULQIURdoXRfMN+rYfoQVaBEbRJUBgNG0TS3Ht2vJCSTQpixLFnZzh7Pu+730hnGNSoiRKETmL7g8wYImL7j1z73Oe8yz/h8ZCSaUScQxYlsXdu3dpI1AnsDd/QrobSYuwwWCg5YbkXarX67Tue+89CAQCGAwGqq4HgDbnxGIx2Gw21Ov1Z05uPomuNL4EolXrdrvx61//GlarFW+88QYUCgXeeOMNnD17FhaLBZubm1hbW8Pu7m5brpMY3kgkgn/+539Gf38/fv7zn2NwcBCzs7MYHx/H+Pg4tra2EAqF4PV6IZVKqXdMGlZIRtbhcECpVMJgMKBYLNJ5U+VymW5KyWTyhZtgS0qHpFIpXn31Vbzzzjvo6+sDj8ejL2E8HsfNmzfh8/k6LifwJCQSCW2NlslkdAjtg6E1kqwmrbIAHvoeYlD7+vpw8+ZN9Pf3480334RSqYRer6eTx41GI3K5HEZGRvC73/3usaez44C8S0SeNBqN0lPAvXv36Ogpl8uFQqGAVCqFer1+YDWRRCLB3/zN39D8AMuy9PcbDAZcvHgRKpUKc3NzR1In39XGlxzlE4kElpaWkEwmcfr0aYjFYgwPD0MgECAcDoPH4yESibTN+ALfeh/z8/Pwer149913MTAwAJvNRuNRVqsVm5ubkEgkUCgUsFgstDpCJpNRHWSVSgWGYWiZHnmpiAB0oVCg5UovWpUDGTTqdDoxNTVFPaO9WX2Px9ORJYmPg8fjQSwW0zI5jUZDO7oOisU+6ZgcCoUQCoXg9/uRTCYxMTGBkydPAgA9hms0GiiVSqoPMT8/f2T39zSQjTSXy1HVMZFIBK/Xizt37uDevXv45ptvaM3yo5BKpfijP/qjfZoq5J1iWRYOhwOxWIzzfB9HuVyGx+NBOp3Gv//7v8NsNuPdd9+FzWbDzMwMBgcH4fP58M0337T1OpvNJp3BdfXqVWxvb+PkyZN07tbw8DD0ej1GR0dpdpZk55vNJkKhEKrVKkKhEGq1Gi5evAin00ljWKlUCtvb2wgEAigUCi9MiRkxTBKJBGfPnsXIyAhGRkbokE7gvrG5desWVlZWEAgEqEfU6QiFQtodevbsWSqZqVAo8PXXX+Ozzz6jjUrP8lmTclAiOWq32/Hnf/7nMJlMtFJAr9dDJBJheHgY4+PjSCaTh66lfd7sDd998MEHmJubo58z6UwjTRKP+nyJwBA5QbAsi2KxiEwmg5WVFdy6dYv++Sjbz3vC+JJZZbFYDJFIBCaTCbOzszAajRgZGYFUKsVvf/vbdl8mms0mjT3duHEDbrebJoPsdjtV5Ae+HYtTKpUQjUaRy+WwsbGBRCKBubk55HI5jI6OYnR0lMZ7s9ksfD4fotEoSqVSV3l2fygks33ixAnMzs7CbrfvKxeKxWKYn5+nM966JeQgEAhgsVjQ19eHy5cv4/Tp09RoiMVi+Hw++Hw+hEKhZ4pfEzlKMq5qeHgY3/3ud2k3KamnJTKeAwMDaLVabTO+wP33KJ/P4+OPP6aJMtLhdhiZAVIxRAyvWCxGLpdDOp3GrVu38Otf/5pqKZdKJa7a4TCQMeuFQgHRaBTBYJAG0DsJIviSy+Xw0Ucf4c6dO7Db7TAYDPR7iPHN5/MIBoMol8uIRqMoFov0KNRoNPaNX0+lUtja2kI4HH6hYr0ikYhWfUxOTmJwcHCfsDZJst2+fRvhcLirNiWSbBWJRNRgkA17dHQU77zzDpaWlqigTCqVeqbPnoRlUqkUPvvsM3g8Hly+fBkWi4UqwTmdTszOzqJer2Nzc7OtpyoiNk/eAQCH3nwEAgHMZjPMZjMN4aVSKbjdbvj9fjo086hLVXvK+JJMfy6Xo51ADoej3Zf1EPV6Hbu7u+DxeFhdXT1wXhn58HO5HNxuNzUYQqEQZrMZOp0OjUZjXzwqHo9jeXkZfr//hQg3EEQiEZVZPHPmDMbGxujXiFqYy+WiiZNu25hEIhEkEglV8SLe3tTUFE6cOAGz2Yy7d+8iGo0im80+0/2RrtRYLIYPP/wQVqsVp0+fpnkHhmHoMM5IJIL/+7//O4I7fTqedRMVCoXUi9doNGAYBvF4HOvr63C73bSe+qjpKeNLymSI4DQZO57NZjuy3nWvYHM2m91XPkQ8X9IZSB6GVqtF45sPJgKKxSLi8fiB4vS9CMMw0Ov10Ol0GB0dxdjYGJRKJYBvGxO8Xi9u376N9fX1jmyTfRKNRgPhcBiNRgNra2uQSCQ0RLV34u5hZpUdBj6fT438g89XuVxGJpPpCAnOPwQys21vTiCXyx27xkdPGV8yo8tisWBqagrT09MoFosIhUIoFArtvrwDIcYgHo8fqOn7YGcSeTk0Gg3N5BMymQxcLldXenfPAsuyOHnyJOx2O773ve9RfVsAVNVtYWEB//RP/4RoNNpV4QZCrVbD6uoqrXLwer14++23YTQa6feQutfnYYBJk0VfX98+vQgyAsrv9x9aIqBTIdoWJKZNYtjr6+uIRCLH5rgc+SQL0v5HSqJyudxzT3aQgmsyUJGo9wuFQiom3u4utydx2FHeRCzGYDDQHvW9v6MTWkCPGlKaRypD+vv7oVQq97VYh8NhRKNReDyep5IZ7ERIPDYajdJjfzqdpuI4KpUKo6OjtMuNJNAOM8WEjBUi0z3UajUmJydhNpvBsuxDp7Gjnu5wXDy4UR12Xt7z5EiNr0Qiwblz52A2m+mk3eXlZWxvbz/Xf4dhGGi1WvT19eFHP/oR7HY7LBYL+Hw+fv/73+PatWvPLFbeaTAMg8nJSYyPjz80Hv1FQSqVUinAP/uzP0N/fz/V+gDuG4m5uTl8/PHHWF1dhdfr7fpR6Y1GA/fu3YPL5YLVaoXRaITVasXAwABGR0fx13/910gmk1hbW0M4HMbVq1dp9c/jwgR9fX24dOkSrFYrLly4AJVKBavVCpZl940f6nX2JvCOiyMTU2dZFiqVirbCEmWkw0rfPfj7yFFh71GbNCCwLAuTyQSj0QiDwQClUknbeMPhMCKRSNeUFj0OMjBQq9Xu83zJkMxKpdL1RuZJkLCL0+mE0+mEXq+n45MAUBEYMho9lUp1ZbjhQYhYFan33t7eBsMwtBqBtAoXCgVIJBKMjIxAp9PRZoxH4XQ6MTg4CIvFAqvVSkNae5W9yPOVTqcRi8U6SuGsmzkS48uyLFXJf++99zA0NIT5+Xns7u5iY2PjmX6fWCymR0yC2WzGxMQEVCoVHA4HGIaBQCBAsVjEb3/7W/h8PszPzyMYDHZFQf3jIB1HJpMJL730Ek6fPk29PbLBBAIBlEqlrr/XR0F0a8fGxvCLX/yC1r8SXYN6vY47d+7A7Xbjiy++wMLCQtcnh/ZSrVZRq9Xw0Ucf4euvv8Z7770Hi8UCsVgMlUoFpVJJ9Qhef/112ob7uM2YyC6Sph4iV0qSeM1mE+FwGKlUCteuXcOVK1eeakwPx6M5Ms9XJpNBqVTum7OUy+VgMBhgMplojIWwd97U3mGa5HeRLO/e0jGz2YyBgQHI5XKYzWY0m03EYjHkcjn4fD643W6k0+mOrHR4WkgXF+nMIeNTgPtVDslkEoVCoe0CQkcJiesTCVHi/ROpyEqlgkgkAo/HQ5+DXloLkhdIpVK0nDIQCEChUNDRVFKplE4MBx6ObT7Ig3PegG+1hWu1Gur1OqLRKGKxGI2jd/uGRmqn91Y7ELW0o2olPogjr3YgybCTJ09iaGgIExMT+NnPfga/3w+3241Wq0VDCjqdDjweD4VCAUKhEOPj41CpVHQ3Jl03BDI4s1AoYGFhAbFYDJ9++ikSiQQ8Hg9yuVzPHJFIJYfRaIRCoaCarq1WCz6fD4uLizS22UsGh8Dj8ehIoJMnT0Iul9PZXY1GA8FgEKlUClevXsVXX31FC+V70UMjRvGjjz7C8vIyHSqg1+sxPT0NuVwOk8lENY33JiKfRKVSgdfrRT6fx87ODpLJJObn5+HxeOByuXpiQyMdpUNDQ1AoFODxeFCr1TCbzUilUsd2HUdmfIlnS4ZaSqVSOjbd4XBAq9XuU5RnWZbq0mazWQiFQpw5c2ZfIoU0HpDdKpvNIhQKoVQqIRgMIhgMYnl5mQ4e7KXj917x7L07dLPZRDabpTWKvWhsiNShXC6H3W6HyWQCwzD7RjFls1kkEgn4fD46DbsX1wL41gMmz3w8HodIJEJfXx+0Wi3t2iJC4iR+S9rQiWQp2bzJf81mk1ZKpNNpOtFidXUVbrf7sXoJ3cTefgCyLuTU8GD55lFyJMa3VCphdXUVgUAAcrkcNpsNTqcTWq2WzkeSy+U4ffo0lUskOgbkyNNoNLC8vEwTDbVaDV6vl6qUAdg3Zyoej6NYLCISidDEUy9Bwi9EJBsAbX9cX1/HJ5980rOxOJVKBYPBgFdeeQU/+clPoNfr9yVuK5UK5ufnsbW1hUAg0LMe76OIxWK4ceMGWJbFwsICJBIJDAYDRCIRDdlNTEzAaDQiEolQMX+dTkcFx4vFIm1fDwaDdBIwmarcK4YX2C/EROp8q9XqQ4LzR82RGF+iXZtKpTA/Pw+Xy4VsNgubzUa/Z2RkBGazmconFotFqswkFAqp/kE+n6cKXUSvk0C6UoiH3csc1JXTaDToWm9tbfWswZFKpXRW3czMDI3z7tWq3d3dxfr6Op1/9iJBhgsAwMbGBhXpJ8+LRCJBsViE0+nEzs4O/H4/TCYTbDYbcrkcIpEIHR1fKpWoRnCtVuvZtSShTODbOmrSTXpcHGnMt9FowOfz0e6tvQI3arWaijaLRCK685AMbb1eR6FQoA8BEcLeq89ZqVQ6erz184Rk+Z1OJ2QyGU28ZLNZ5PP5QzdpdBNkvtbp06fx7rvvYnh4mB4TgftGNxKJ0JHhKysrXd999TxoNBrUYREIBBAKhZibm8PKygqdyeb1erG2toZKpYJSqURH7ZD3r5dPD/V6HV6vl+aPdDod1tbW6Kik4+LIjS9Rvm+nkHkvIBKJ4HA44HQ6qeJ+LpdDLBajU257DaKmNT4+jnfeeechPQuSiSfJW7fb/UJsxE9i7wBbQjKZbNPVdB5EL0MsFkMkElGvf3d391hrwntK26GXEYvFGBgYwNDQEDW+mUymq7RpnxatVgudTkdjvCRrX6lU6Py+Tz75hJ6uetH753j+kDxJJBKB2+2GVCrFxsZGb3S4cTx/SJPJxMQEAFDtVRIX70V0Oh3NDUilUhpuKJVK8Hq9cLvd+M1vfgOv14tkMsl5vRyHolqt4s6dOw/pVhw3nPHtErLZLK5cuYKlpSUA949Oq6urtLutFyEzuMhMOiKWHwwGMT8/D5/PR8e/91p1C8fR0+5TEmd8u4RAIIC/+7u/29eNRBoqetXwZLNZBAIB2q1GJpSsrKzg/fffRyKRQCQS6YkORo4XD874dgkHJVF6nWq1ikKhAJfLha+++grlcpmOeyFZ+17deDh6H97jXG8ej/dCZS9ardah1Ki5dXmYo1iTvWp2JMlIxoYXi0X653bAPSsHw63LwRy0Lpzx3QP34BxMu4xvJ8M9KwfDrcvBPLXx5eDg4OA4GvhP/hYODg4OjucNZ3w5ODg42gBnfDk4ODjaAGd8OTg4ONoAZ3w5ODg42gBnfDk4ODjaAGd8OTg4ONoAZ3w5ODg42gBnfDk4ODjaAGd8OTg4ONrAY1XNuP7rg+HW5WG4NTkYbl0OhlsXzvPl4ODgaAuc8eXg4OBoA5zx5eDg4GgD3CSLDofH4+0b9Ef+n0zq7VVJUHLfPB6vraLpHBxHBWd8OxSWZaHRaKDX6zEzMwOlUgmr1QqxWIx6vY5KpYJPP/0Um5ubyGQyPTPBmM/nY3x8HCaTCSdOnMDQ0BBu3bqFr776Cvl8HolEomc3HI4XC874digSiQQGgwHDw8P4wQ9+gL6+Ppw+fRoymYzONkun00ilUqjVaj1jfAUCAZxOJyYmJvDjH/8Yly5dwn/8x3/A7XYjEokgmUxyxpejJ+Bivh2K0WjE+fPn8corr2BoaAhWqxUMw6DVaqFWq6HRaGBgYAAvv/wyjEZjuy/3ucHn8zE4OIgzZ87AZDK1+3I4OI4MzvPtUCwWC77//e/DYrHgxIkTEIlEAECNb71ex9jYGJRKJXw+H+7du9fmK34+CAQCjI+P48KFC1Cr1e2+HA6OI4Mzvh2GVCqFXC6HwWCATqeDUqkEn89HtVqF3+9HoVCAx+NBLpdDOBxGNptFOp1u92X/wfB4PIhEIkilUshkMkilUkQiEWxvb2N7exuxWAz5fJ4LOTwDPB4PKpUKEokEarUacrkcCoUCarUaQqEQAoFg3/fuhax3q9XCxsYGvF4vSqUSSqXSsd7D0+BwODAyMoJ0Og2fz4dyuYxsNttxzw5nfDsMnU4Hp9OJkZERDAwMQC6XQyAQIJvNYm5uDn6/H1988QUikQhYloVQKITP52v3Zf/B8Hg8KJVKaDQaaLVaaDQa3LhxA3Nzc5ifn8fOzg6azWbHvUDdAJ/Ph81mg8lkwtTUFIaGhjAyMoKpqSmIxWKwLPvQ9wPYV2HSaDTwD//wD/jggw8QiUQQCASO9R6ehldffRU///nPsby8jP/6r/9CPB7H5uYmarVauy9tHx1tfBmGgVAohEajAcuykEgkYBgGqVQK8XicZv17AT6fDz6fD5PJhJMnT2JgYAASiQStVgupVIp6gX6/H5FIBKlUCsViEUKhEOVyud2X/wfD5/Oh0WhgNBohkUjA4/FQqVSQy+VQqVQ4w3sIGIaBSqUCwzBQKBRgGAZyuRxisRhDQ0PQ6XQ0f6DX68EwDOr1OpLJJOr1OorFIprNJiqVCng8HnQ6HSQSCfWatVotTCYTCoVCu2/1QMRiMRiGgVqthl6vh1wuR6vVQqPRaPelHUjHGl8+nw+1Wg2ZTIZLly5hcHAQFosFRqMRn3/+Oa5evYpMJoNwONwTL6VYLIZEIsH58+fxt3/7t5DL5VCr1UilUrh37x5cLhd+85vfwO/3o1wuo9Fo0DrYTn24ngaRSITJyUk4nU7odDq0Wi0Ui0UkEgkUi8We+IyPGpVKhZdffhk6nQ6nTp2CVqvF1NQUNBoNFAoFxGIxhEIhhEIhCoUCstksQqEQVlZWkE6n4Xa7USgUEAgEIBQK8dZbb8Fut+PcuXOw2+1wOByYnZ1FvV7H5uZmx30mWq0WOp0ODocD/f392NzcRKFQQLlc7rhrBTrI+PJ4PPpgkN3aZrNBqVRicHAQAwMDMJvN0Ov1GBgYwPDwMEKhEDKZDFqtFo1b1et1tFotVKvVjlzwRyEWiyGTyaBSqaDVaiESiWisNxwO03stFAo9YWwfhHi+BoMBEokEAFCpVJDP53vCsz8KpFIpFAoFRCIRZDIZ9Ho9hoeHodVq4XA4oFarYTAYoFAoIBAIwOfz6Xomk0nE43GEQiHs7Owgm83C6/WiWCwiFAqBYRjE43HIZDJks1kUCgXkcjlks9mOjPeSuLbdbodGo4FQeN+01et1ahMO+hmlUgmxWEwriCqVyrGdpjvG+IrFYhgMBmi1Wly8eBEmkwlnz56F0WiEVquFVCqlYQi9Xo/Z2VncvHkT77//PgBArVaj2WwimUyiXC4jFAp15EPyKPR6PRwOB/r6+iCTyehmEo/HceXKFfh8PmQymZ40vMB9z/fEiROYmZmBTqcDAESjUWxubiKZTLb56jqTyclJfOc738HAwABeffVVSCQSKJVKCIVCGsfN5XJIp9MIh8PIZDK4desWVldXEY1GEQqFUKlUUCwW0Wg0UKvV0Gw2UavVwLIsXC4XarUaDAYDCoUCPvnkE/zP//wPCoVCRzk2PB4PfD4fr7zyCt5++20MDg6iXq+jXC4jk8k88uQkFApx4cIFjIyMIBKJIJFIYHd3F5ubm8dy3W03vjweDwKBACzLwmQywWAwwOl0wmw2Y2hoiHpCZCcD7ielxGIxgsEgrFYr/btGowGGYVAsFhGPx7vG+PJ4PEilUiiVSkilUggEAuq953I5RCIRGuPuRfh8PoRCIfX6hUIhGo0GyuUy8vk8qtVquy+xI1GpVHA4HBgeHsb4+DiEQiGazSaN29ZqNcTjcRQKBfj9fqRSKezs7GBzcxPxeBzhcJh+/4Pw+fx9hpvH4yEUCiEajXaU4QXux7pFIhG0Wi36+vogEAgQjUaRTqdpWebjPF+j0YhmswmBQIB4PH5s191W48vj8SCTyaDT6TA+Po6//Mu/hMFgQH9/P1iWhVqtBsMwNPtKYFkWDMPg1VdfhdlsBp/Ph0wmQ6VSgcvlQjgcxj/+4z8il8t13IPyKEg7rclkAo/HQyKRwNbWFpaWlrC5uYlEItGTRkggEEClUtFNd2hoCLVaDYlEAvF4HIlEomc3nT8Uu92OS5cuQaPRQCKRIJPJYG1tDfF4HHNzc0gkEvD5fMjn8ygWi6hWq7QVvVqtotFoPPL9qNfrcLvdiMfjWF5eBgD4fL6O09gQCAQYHR1FX18fhoaGoNVq8eWXX+Ljjz9GIBBAIpFArVY78D5JMq7RaMDhcGB8fByFQgG3b98+FrvRNuNL6guJ8bXZbJienoZer4dard5Xe/ggfD4fIpEIBoOB1sFKJBKUy2UwDAOpVAqpVEpFWTodsgmR8AoAlEolRCIRxGIxGuvtRfh8Po1dKpVKKBQK6q2Vy+Wnjr/tFeR58O/5fP6+7HenGZKnhWVZGAwGelqq1+uIxWLw+/24c+cOIpEIvF4vCoXCI72/R9FsNpHL5VCtVpFOp1EqlTpyEySxXqPRCJlMBj6fj2g0isXFRRSLRVQqlcfeN3ke5HI5rZA4Lo7d+IrFYohEIgwODmJsbAxDQ0M4f/489Ho9LBYLxGIx9XRJiRHDMBAIBA+9UAKBAGKxmL5sQqEQBoMB1WoVIpFonypWp8KyLMRiMRwOB06ePAmz2QwAiEQi+OKLL+B2u3vS4yXI5XK89tprNEHUbDaxs7MDj8eDcDj8VL+Lx+PBbDZDoVBAKpXSxB0AerzM5XLY3t5GPp9HKBTq6rVNpVLY2tqi95zL5bCysgKfz4ednR0kk8lnMrzAfeObSqUgEAie6CW3Ex6PB41GA7PZjFgshuvXr2NjY4OGHJ5keNPpNCKRCGw2G8xmM5RK5bFd+7EbX4ZhwLIs+vr6MDU1henpaVy+fHlfTJfsRuRDJzWwwP4OHGJwCXw+HyzL0uaDTod0dbEsS+NV5MPP5XJwuVwIhUI9m2QD7ifaBgYGMDg4CJZl0Ww2EY/H4fP5kMvlnup38fl8KJVKenpSKBT0a6R5JZFIUHW0WCzW1ca3XC4jkUhAJpOh1WqhXC4jEokgFAohHo8jm80+8+8mv6/TIfkSlUqFfD6PdDqNWCyGUql0qM2iVCohn8/TZ4c4bT0XduDxeBgfH8fU1BROnTqFV199FXq9fl9HTTabxfz8PDKZDFKpFOr1OmZmZmC1WqHVag/cmZrNJkqlEpLJJH7/+9/D6/UiGAx2fGE+n8+H0+mE3W7H8PAwjEYjGIZBtVpFuVzu6BrF54VAIIBcLodSqaTCQfF4HF6v99BKbQKBAAaDAXK5HG+//TYmJiYeMr4sy0KpVKJUKuGll15CIBDAv/7rvyIcDiMWi3VVs45cLgfLsrDb7RgdHYVMJqOJsZWVFVrF0MvweDzI5XLI5XJMT0/j3LlzWFtbg8fjeaZwklQqhVarpd2VpVIJxWLxCK78W47dPRwcHMT58+dx8uRJzM7O0r8ngtm5XA43b95EMBhENBpFs9mk1Q6klOZBSH1eLBbDJ598ApfLhVgs1vFGi8/nw263Y2JiAjabDVqtFrVaDZVKBdVqFaVS6Ykxq26Hz+fTl4hUeRBDcljjy+fzodPpYDAYcP78eZw/f55qGDwKt9uNGzduoNVqIZPJdJWxIlrPZrMZAwMDNJEWj8fhdrsRjUY7rpX2eUOS7CqVCsPDwzh58iSi0Sh2dnae6aRINmeVSkU37cN6z8/KsRhfoVBIvdcLFy5gamoKZrMZrVaLFtLH43EsLS0hm82iXC5DLpfDbrdDKpVibGyM1r8eRLVaRTwep0euSCTSFcdJsntrtVoanyS1iclkkibbuj0xdNQwDIPR0VEMDAzAYrHQ1lrgfobe5XKhVCohnU5Dp9Ph9OnTAICXX34ZBoMB0Wj0qUMc7YLH48FisWB0dJTKjKbTaXg8Hlrb/qRYZy8gEAhgs9nQ19cHlmVRqVQQCoWwtrZ26HI4kiMYHh6mCnoikQhyufxYNq8jN748Hg8Mw+DixYt47bXXMDU1heHhYQCgLaShUAirq6v45S9/iUajQaseLl68iL6+vicGwsnCBwIBeL1e+P3+o76t58LeTC2pciDdR9FoFIFAoCvibu1GJBLh1KlTmJ6ehsPhgEqlol/b2dnBlStXqFc0OTlJN/JLly4hFArhiy++6CpxItJU4XQ6aV371tYWPB4PisViz3u9wP0Nd2hoCE6nkw4Y8Hg8uH379qErg8jJc3JyElqtllZNqVQqlMvlI4/9HqnxFQgEsFqt0Gg0GBwcRH9/P1QqFXg8HqrVKiqVCiKRCNbW1pDNZnHu3DkIBAIMDAxAqVTCZDLRIPhBkC6WSCSCb775Bm63+8jjNM8Tkqm1WCzUq0+lUtje3qYF8ByPRiAQQKFQQK/Xw2azwW6303V0u90IBAK0oyuTySAajcJgMCAej6PZbEKpVKJWq1EvuVsgTSkkV0KacV4EDQwejwexWAylUomRkRGMjIygXC7TUONhSuJ4PB5YloVcLofJZILNZkOlUsHW1ha8Xi/i8fixyJceqfEVi8V46aWX6E49MzMDPp8PHo9Hu9DW1tbw0Ucfwel04q/+6q+gUqlo+RgpL3uwxIxQLpcRjUaxtraGX/3qV/SY3i3w+Xz09/fjxIkT1Fvz+Xz4/PPPsb6+zhnfJyASidDf3w+r1YqZmRn6fLVaLczNzeF///d/sba2hjt37tDkq0QioW2zs7OzkMlk+0rSugGBQACRSETfj0qlgmg0ilQq1fPPjEAggFqthslkwqVLlzA9PY1PP/0Ui4uLcLvdhwofkRyBTqfDxMQETp06hRs3buDu3btYXFzE9vb2sSTrj8z4knCD1WrF4OAglErlvsYJ0n+u0+kwPDwMq9UKuVxO5QQbjQbi8Tiq1SpkMhlEIhEkEsk+L5iELEKhELLZLPL5fNc9fMSLIZsMOfaYTCYMDg7uSzjK5fIDS+iIKEg6nUY6naZx4173ghiGgcViobkBPp+PQCCAdDoNl8sFn8+HZDK57xhO2mlbrRb4fD4YhoFWq4Ver6fylZ3M3q5QlmXRaDRQKpVoO223Pf9PC8MwMBqNNNYrFAqRSCTg8XieqrSuXq/T96bRaNA/E32Lri41I51LFy5cwOzsLAwGw76vy+VySKVSGAwGnD59GgKBAFKpFM1mE8ViEdlsFteuXUMsFsPY2BgMBgPsdvu+uV7RaBRffPEFNjc3aXa8Gx8+0n1FEgAvvfQSnE4nhoeHIZFI0NfXB5VKhenpachksofGx6fTaeTzeSwuLuLLL7+Ex+PB4uIi6vV6V67HYVEoFLh06RJtK61Wq7hy5QoWFhawuLiItbW1J2a+iaBPrVbDnTt3EAwGj+nqnx2bzYYzZ85AJpOhXC4jHA7jm2++eWij6TX4fD4UCgVmZ2f3NeXcvXsXH3/88aFPvc1mk9b2EsW2SqVy7PX0Rxp2IE0QB+kzEIMjEonAMAxqtRpVJItGo8hms1QMZGBgYF+Mi+xS6XQawWAQ8Xic7ljdjkQigU6no119YrGYxr5NJhOkUintWCIeM8uyyOfzsNvtGBoaQrPZxO7uLt3EOnldWq0W/TyJtyEQCB4ab7MXEpKSSCQwGo0wGo00eUtaalOp1KG8WNLoIpFIHtvS3gnsfWdYlqUhh3K5TJ9/kUh04OdNNuJuPQ2RexeLxbBYLLBYLACAQqGAfD5PW6EPC3nmisXiU//s8+LIjC95qUjmXiqV7htXQlz9ZrOJer2OSCSCubk5hEIhfPrpp0in0xAKhZDJZLhw4QKcTif9+VQqhXA4jNu3b+Pjjz9GOp3u+OPi4yA1zq1WC0ajESqVikr7kbAEMTb1eh3RaBSVSoU2FshkMrAsiwsXLmBmZgbLy8vQaDTweDy4du1aR6u7NRoNJJNJWuPM4/GgUCjosfogyLSCvr4+vPTSS3A4HDTmeevWLdy4caOrn4dHQdS7SBdnLpejZXIikQgKheLANWu1WlSUvltPQ6QSwWg04gc/+AEcDge8Xi+8Xi8ikchTJchI9x6Px4Pb7cbKygoSicQR38HDHKnnS/R1I5EIlErlvsQZqXYgDRKhUAi7u7sIBoO0995ms9F4L8uytAif9OVHo9GemnRAvFmBQIBms0njUdVqlVZ21Gq1fVrF1WqVesksy0KhUCCZTMLhcKBWq0EqlVKt1k5cI3IEzGQyqNfrB9Y+PwjDMHTWm1KphEwmQ7FYRDKZRCaTeWLShYR4yMZPvMdOb+NmGIaO0iInwVarBbFYDKPR+EjvrdVqgWEY5HI52jVJRMa7CT6fT8W4SANNvV6nYvLEgdkroP6oZ55IGJCWZJJbIYnMB3+OOEjPkyMzvqRV+N/+7d+g0+n2NVYQ0Y5oNIpCoYBEIoFyuYxUKoVqtYp8Pg+WZfGd73wHw8PDcDgcdN5UtVrFwsICfvWrX8Hv9yOXyz2TcEinQoxtOp2mPfp37tyh7Y7lcpmORyFzuaampjA4OAibzQan04mBgQH85Cc/werqKmKxGILBIFZXVzuyDC+Xy1EBoe9+97tUJvHUqVPw+Xy4efPmQz9jNpvxJ3/yJ3A6nVCpVKhWq1hZWcHOzs4T9VjJTEAiRJPNZrG0tITFxcWObrTg8/no6+ujYRYiq0o0kGdmZh5pHJrNJg3h3bp1C2tra/D7/fB6vXSD73RarRYNE5BhqhqNBiqVCm+//TYGBgao+p/P56OCVI879TWbTUSjUbjdboyNjWFqagrBYBBff/31Q2tZr9eRzWafq605Us+3VqvB7XbTJEYikaDZ5lgsRqsUIpEIvSHSNigSiWC1WukEX1IBQcpq1tbWkMlkOtajOyx7JwiQP5MNiFRyrK6uIp/PI5/Po1QqYWNjg8a5iNg8OXY2m01IpVI4HA7k83mYTCZUq9WOjWfWajUq1k3aOY1GI3Q6HbRaLRiGQaPR2PcysCyLgYEB2O12GuNMp9O0OuYg9sYMieEqlUrIZrOIx+PHKqL9LJBKGLlcTit+yNgtchx/8Pv3qvoZjUakUinkcjk6SogIC3VyWGovzWaTdrOSDliJRAKz2YxKpYJUKoVsNotKpYJEIkEHgT7KPjAMQ4XnJRIJTCYT9Ho9tFotVXHbK/KVz+efa+PFkRpf0j5cq9WwtraG3d1dekOVSuXAVkiWZXHixAlYrVacOXMGY2Nj0Gq1AACPxwOv14u1tTWEw+Gum9P2II1GAxsbG9RzGRkZQSgUwubmJlZWVvDhhx8il8vRKRb1eh2NRoPuwCTeFY/HcfPmTfzwhz+kuqZEqEar1SKXy3Ws8W00GshkMhCLxfB4PDAYDDCbzbSt/Pvf/z52dnawvr7+yN/RarWQy+XoyekgdDodBgcH6Trz+XzcuHEDfr8f6XT6iO7u+UHCbUS/YWVlhZZGkU1kb8KSDMw0GAxgWZY+E2+//TZmZ2exubmJu3fvYn19HdeuXaMKgp0KyYGEw2G8//77+4aDkveiv78fSqUSr7zyCqrVKtVJeRSkzl6r1dJKqh//+MeYnJxEoVBALBZDOp3G9vY2EokErl+/Ttf5edidI28vJnGlaDR6qO9nGAZ9fX2w2+2w2Wx0TBAZob67u0uTDN1seIH79xSNRrG7uwun0wng/jHc7/djbW0Nc3Nzj83CEqORTqfBMAzGx8eRy+XoeBQSB5ZIJA9Vm3QKJPlRKBSQTCaRSCSg0Wggk8lgMpkwMTGBQqGAra2tfdn6vfkDspmThNKDkNrY/v5+2Gw26PV6lEqlfboPnQ5ZJ2KAg8EgjVXncjnEYjF6OmAYBnq9nlZEKBQKWK1WKJVKqNVq2llJJDyvX79OcwydDBHeWlhYAMuyKJfL0Ov1MJvNUKlUUCqV6O/vpzFhYrD38qDGNzllEyGm8fFxjI2NIZlMwuv1IhwOo16v02Qnn89/buvUMaK35Eio0+lw9uxZGs8DQOvwvv76a1y9ehVut7vrDS/w7Qu112goFArY7XZsbW09srPvQUgGO5FIwOVywWaz7auH7gaq1Sru3LmDcrkMlUpFY9lqtZrGsSORCDY2NjA8PIzp6WmYTCZIJBJks1mEw2E6fXcvpCB/cnISly9fhtlspkfu1dVVuFyurpgS0mq16JH6008/xdbWFg1ZEe93b+iONCBotVqwLAuLxQKVSkUFiPh8PmZmZtBoNLC7u4tQKISlpaWOF6QiCbVyuYyNjQ34fD5IpVJqQBUKBSQSyb5kLakUItNviHSpQCDA5OQkTCYTGIYBwzBYWFjAtWvXaDg0l8shFArRMF/XxHyfBmJ8VSoVTpw4geHhYcjl8n07/urqKj777LOO36GfBvIgEeNLPD6igXEYSNIkk8kgFApRce1uolarYWtrC+VyGefOnQNwX350cHAQJpMJRqMRa2tryOVysNlsdEQ6cP/+E4kEwuHwQ16sRqPB6OgoTp06hYsXL1K95FQqBZfL1TWeL3C/o7NYLGJxcRGLi4uH+hmRSAShUAiLxQK1Wo233noLzWYTw8PDGB0dRaVSoUM1V1ZWOt74AqBxWq/Xe+DX9/YWkFZslUoFmUyGoaEhapjJ30ulUshkMkilUiwtLeFf/uVfaBjrKBOSHWN8DQYDXn/9dQwMDNA4DEmmbG5uwuPxIBAIPJR86Xb2NhiQIxF5aAQCAfh8/hPvl3QL2mw2DA0N0SGc3QSJYedyOczPz6NarcJms9EY9ujoKNRqNXQ6HSwWy756VoFAAJ1OR0WcyLhztVqN8fFxnDx5EjabDSqVCvF4HNevX6djivL5fNeVXD0NxHCQExaZEi0WiwHcX/de040mJYQkSU+SZ8ViEc1mk7boi8ViOrRWrVbTxFq5XD6W0UkdY3z7+vrwp3/6p7BarRgeHqYvV6VSwfLyMhYWFuDxeHrK6yWZVFLtQAwvMb57a34fB5n+Ozg4iMnJSVoT3U0GuF6vY2dnByKRCH19fQgGg7h48SLEYjHkcjmN/b/xxhsA9sd8BQIBzGYzHA4H/dr09DRGR0cxNjaG6elpurHt7u7iP//zP+H3+xEIBLrG631WiLNSLBZpx6lGo6ESpvV6vecmppCKqr2Q0FIoFKJ/J5FIcPbsWdjtduj1epqgI4M3j9rWtN34CoVCKhFH6vaIwSGlI7u7u9jZ2emKrPTT0Gq1EAwGwePxsLOzQ2NxKpWKVnvEYjFsb2+jWq0+9ECR1myn04mJiQn09/fTInzg2+NZN3k1jUaDjigXCoWIxWKwWq1Up9dsNj+0qYhEIkxPT9MwBI/HQ39/P8xmMyQSCZLJJOLxOLa3t7GxsQG/349kMtnTHi/w7Uw7lmWpjvHIyAjUajVarRYt9yRC7L3k2BwG8k60Wi0kk0naCUiamo6athtfsVgMvV4Po9EIq9UKg8EAoVCIWq2GnZ0dBINBzM/P4/r16z33cDQaDayurmJzcxNGoxESiQTj4+M4deoUZmZm8NOf/hTr6+uIRCLIZrP7SutIFpu0X1++fBk2m43WRJOi9EKhQI9b3UCj0cDy8jJWVlawsLAApVKJM2fO4LXXXsP4+Dh9PvYilUrxzjvv7NtgSBdbJBKBy+XC9evX8ctf/pJOq+2lxpxHQaY9mEwm/PSnP8Xp06dhtVphMploJ+nKygrm5+eRz+d7WpTnSXg8HmxsbGBnZ+fY3pe2GV8S0zQYDDh16hRGR0fpcZl0srjdbuzu7tJBmt1iQJ6GvfWL6+vrUKvVmJychEQiQX9/P8rlMkZGRpBMJqm3Ro6PIyMjNNyg1+shkUho5juTycDn81HhoW7y8kicjrSNB4NBrK2toVgsgs/nQ6vVYnR0lHr+RL6UeDDk2Fgul+H3+7G9vY319XX6tV4yvA8W/RPhHYPBAIVCgVOnTsFoNMJut9NpDblcDuFwGJubm/D7/S/M6KEHIfXhiUSCtm4LhcJjW4e2GV+JRAKFQoFz587hF7/4Be1oAkBbj//7v/8bS0tLdBJxr9JqtbCwsICNjQ00m03MzMxAoVDg4sWLGBoagkgkQjQaxeLiIvL5PO3S+uM//mPMzMzQUenlchnpdBq7u7tYWFjA5uYmrl271jbVpj8U4rUvLCxgaWkJQqEQIpEIs7Oz+Pu//3uYzWZoNBrqCTcaDdy+fRvb29vw+/2IRqPY3NzE8vIy7eTqZmWvByG5AVKjSwyvXq/Hj370I/T39+PNN9+ExWKBUqmEWCxGMBikgv0ffvghFaXptVPlYSDqf6Tszul00llux0HbjC/DMHT6qNFopE0BRCoykUggkUg8tmuplyCGIRwOY3d3F3q9Hg6Hg1YxsCyLdDqNYrFIJRBJNYBQKES1WqUiRru7u3SMDqmR7kb2dkPuvYdYLIZUKgWWZffNawNA9T9IpnpvKV+3GxiSNBSLxbQulbQat1otiEQiKJVK6PV6DA4O0jCeUqlEtVqlugdE14Eb0AqaZBMIBJDL5WBZFiKR6Fiel7YZX5VKBafTCbvdTlsgBQIB0uk05ufnsbu7C4/Hg0Qi0fUvzWEg7ZBffvkl4vE4Zmdn8bOf/QwSiQTnz59Hs9nE9773PVqSRpIpYrEYPp8P4XAYCwsL+PzzzxGLxbCzs4NyudyTAzgzmQwWFxfhcDig1+v3TTeRy+X0JFCv1yEWi1EoFJBKpeD1erva6yU1u+S9IR1rRPNBq9VicnKSdnqR5oNWq4WbN29iZ2cH169fx927d5HJZOi71c1r8jwgHX9kJJXJZEI+n0cqleqtUjNyVFIoFPTYSB4q4L7nEo/HDz0Mr1fYO5XC7XbDYrEgGo3SeVWkRIjULpJYMZmFFwgE4PF44Ha7adimV1+qcrmMYDAIkUhEtWxJTbhAIKAF9iRMIRaLqfB8t60J8XSFQiE1tFarlTZNEN1jpVIJrVaLkZERSKVSqFQqOqmhWCzC7/fTHIrH40GtVnshTpSPg5yMiOdLdJIlEsmxnBaP3fgaDAZoNBq89dZbeO+992j2mtTmkU62nZ0d5PP54768tpPJZOgxOZPJwOFw4M0334ROp8PIyAgYhqHzqlwuFxXVWV5epspevZ48CYfD+OCDD9DX1wfg/lidEydOQKFQUIMSi8WQTCbh9/uRSCSOZRrt84ZMKTl//jwsFgtef/11DA0N0TAMmT23NwRB9JsDgQBSqRR+97vfwePxYHV1FdFolD5fL3KogUBivul0GpcvX6YnCaJqFovFesfzJQ+TXq+H3W7H2NgYFX3Z23CQTqeRSqVeyNIX0qsfDodx7949FItFTExMoF6vw2w2g2EYOrXB5XIhFAphbW0NKysrXSGO8jwol8vweDwolUpwuVyo1+vQ6/X7GgbIsTqXy9GOpW6CjDciMf/BwUGcOXMGk5OTdGPRarXQ6XS0MYechsrlMhKJBCKRCO7du4etrS34fL6umux9HBClOCJVSTQglErlsTh+x2Z8ifbo6dOn8dprr2FmZgYymYxKHZIprEQwvFKpvNC7M0mOkBCCRCKBRqOhR0kiuE5EYl7E2F02m8XVq1chlUpx5coViEQi5PN52qVEFL/IlIxuWh+lUolLly7BarXihz/8IRwOB2w2GwQCAVV9I+3nZCBBJBLBzZs3EY/Hce/ePWQyGbhcLuRyuY4U0u8E9p6U/H4/+vv78Rd/8Rf46quvsLm5eaSb9rEaXyKQPjU1BYvFsi9R0mw2aYKoVqt13cvyvKnVashkMshkMggEAu2+nI6kXC4/Vue3m5FIJBgdHcXg4CDVpiCdfWKxmMa4m80mSqUSHZ9+/fp1hMNhLC0tdYVaW7shOaVCoYB0Og2VSoWXX34ZwWDwyGVYj8X4CgQCDA8Pw2Kx4NSpUxgbG6MlQqQUyOPx4Pe//z18Ph92dnaQSCReyLADBwdw3yMjm+69e/eQSCQgk8nAMAxcLhf8fj8dEUXqu9PpNLa2tlAsFrsuzNIuSLjT7Xbjxo0bNI5O2o2PkmMxvnw+Hw6HA+Pj4xgZGaECKMC3wh5+vx8ffvghwuEw/H5/19amcnA8D8hE71arRZPPZKgoaTrZ2trCysoKnQTO8fSQmZLBYBDLy8uw2Wyo1+tIp9O9YXyBb5WGSBF8OBxGIBCgwjEejwd+vx+ZTOaFSBpxcDyOarVKY/6VSgVyuRwymQxCoRAul4u2jfeaxGo7aLVa8Pv9aLVacLlc0Gq12N3d7a0mCyL2Ui6Xsb29ja+++grb29v48ssv6RgZ7kHi4Lgvpbq9vQ0AuHPnzkNff5HzIUcBqX/eS094vsStb7VaEIvFCAQCVD2fjOjoJtlDDo7jhHsvjofjXmfe4/5BHo/33K5mr0g4qUnc+18nPGCtVutQ6uPPc126gcOsC7cmB8Oty8Fw69KmmC8HBwfHi05nzhPn4ODg6HE448vBwcHRBh4b8+Xg4ODgOBo4z5eDg4OjDXDGl4ODg6MNcMaXg4ODow1wxpeDg4OjDXDGl4ODg6MNcMaXg4ODow38P/xWogeVFvG2AAAAAElFTkSuQmCC",
            "text/plain": [
              "<Figure size 360x360 with 25 Axes>"
            ]
          },
          "metadata": {
            "needs_background": "light"
          },
          "output_type": "display_data"
        }
      ],
      "source": [
        "# Preprocessing data: convert to tensors and normalize by subtracting dataset\n",
        "# mean and dividing by std.\n",
        "transform = transforms.Compose([transforms.ToTensor(),\n",
        "                                transforms.Normalize((0.1307,), (0.3081,))])\n",
        "\n",
        "# Get data from torchvision.datasets\n",
        "train_data = datasets.MNIST('../data', train=True, download=True, transform=transform)\n",
        "test_data = datasets.MNIST('../data', train=False, download=True, transform=transform)\n",
        "\n",
        "# Define data loaders used to iterate through dataset\n",
        "train_loader = DataLoader(train_data, batch_size=64, shuffle=True)\n",
        "test_loader = DataLoader(test_data, batch_size=1000)\n",
        "\n",
        "# Show some example images\n",
        "fig, axs = plt.subplots(5, 5, figsize=(5, 5))\n",
        "for i in range(25):\n",
        "    x, _ = test_data[i]\n",
        "    ax = axs[i // 5][i % 5]\n",
        "    ax.imshow(x.view(28, 28), cmap='gray')\n",
        "    ax.axis('off')\n",
        "    ax.axis('off')\n",
        "plt.tight_layout()\n",
        "plt.show()"
      ]
    },
    {
      "cell_type": "code",
      "execution_count": 56,
      "metadata": {},
      "outputs": [
        {
          "name": "stdout",
          "output_type": "stream",
          "text": [
            "torch.Size([64, 784])\n",
            "torch.Size([64, 10])\n"
          ]
        }
      ],
      "source": [
        "for i, (x_batch, y_batch) in enumerate(train_loader):\n",
        "    # Flatten input to 1D tensor\n",
        "    x_batch = x_batch.flatten(start_dim=1)\n",
        "    # Convert labels to one-hot encoding\n",
        "    y_batch = nn.functional.one_hot(y_batch, num_classes=10)\n",
        "\n",
        "    print(x_batch.shape)\n",
        "    print(y_batch.shape)\n",
        "    break"
      ]
    },
    {
      "cell_type": "code",
      "execution_count": 57,
      "metadata": {
        "id": "bbTGMd7kkcy-"
      },
      "outputs": [
        {
          "name": "stdout",
          "output_type": "stream",
          "text": [
            "Train Epoch: 1 [0/60000 (0%)]\tLoss: 8.301888\n",
            "Train Epoch: 1 [12800/60000 (21%)]\tLoss: 1.048659\n",
            "Train Epoch: 1 [25600/60000 (43%)]\tLoss: 0.656033\n",
            "Train Epoch: 1 [38400/60000 (64%)]\tLoss: 0.656552\n",
            "Train Epoch: 1 [51200/60000 (85%)]\tLoss: 0.604041\n",
            "\n",
            "Test set: Average loss: 0.0006, Accuracy: 8258/10000 (83%)\n",
            "\n",
            "Train Epoch: 2 [0/60000 (0%)]\tLoss: 0.527008\n",
            "Train Epoch: 2 [12800/60000 (21%)]\tLoss: 0.611426\n",
            "Train Epoch: 2 [25600/60000 (43%)]\tLoss: 0.408790\n",
            "Train Epoch: 2 [38400/60000 (64%)]\tLoss: 0.334990\n",
            "Train Epoch: 2 [51200/60000 (85%)]\tLoss: 0.461749\n",
            "\n",
            "Test set: Average loss: 0.0004, Accuracy: 8618/10000 (86%)\n",
            "\n",
            "Train Epoch: 3 [0/60000 (0%)]\tLoss: 0.407931\n",
            "Train Epoch: 3 [12800/60000 (21%)]\tLoss: 0.554228\n",
            "Train Epoch: 3 [25600/60000 (43%)]\tLoss: 0.492707\n",
            "Train Epoch: 3 [38400/60000 (64%)]\tLoss: 0.525322\n",
            "Train Epoch: 3 [51200/60000 (85%)]\tLoss: 0.470328\n",
            "\n",
            "Test set: Average loss: 0.0004, Accuracy: 8814/10000 (88%)\n",
            "\n",
            "Train Epoch: 4 [0/60000 (0%)]\tLoss: 0.484231\n",
            "Train Epoch: 4 [12800/60000 (21%)]\tLoss: 0.546624\n",
            "Train Epoch: 4 [25600/60000 (43%)]\tLoss: 0.361683\n",
            "Train Epoch: 4 [38400/60000 (64%)]\tLoss: 0.433487\n",
            "Train Epoch: 4 [51200/60000 (85%)]\tLoss: 0.259598\n",
            "\n",
            "Test set: Average loss: 0.0004, Accuracy: 8928/10000 (89%)\n",
            "\n",
            "Train Epoch: 5 [0/60000 (0%)]\tLoss: 0.352790\n",
            "Train Epoch: 5 [12800/60000 (21%)]\tLoss: 0.365158\n",
            "Train Epoch: 5 [25600/60000 (43%)]\tLoss: 0.192692\n",
            "Train Epoch: 5 [38400/60000 (64%)]\tLoss: 0.411015\n",
            "Train Epoch: 5 [51200/60000 (85%)]\tLoss: 0.348522\n",
            "\n",
            "Test set: Average loss: 0.0003, Accuracy: 8977/10000 (90%)\n",
            "\n",
            "Train Epoch: 6 [0/60000 (0%)]\tLoss: 0.377849\n",
            "Train Epoch: 6 [12800/60000 (21%)]\tLoss: 0.582686\n",
            "Train Epoch: 6 [25600/60000 (43%)]\tLoss: 0.251515\n",
            "Train Epoch: 6 [38400/60000 (64%)]\tLoss: 0.190964\n",
            "Train Epoch: 6 [51200/60000 (85%)]\tLoss: 0.278326\n",
            "\n",
            "Test set: Average loss: 0.0003, Accuracy: 9004/10000 (90%)\n",
            "\n",
            "Train Epoch: 7 [0/60000 (0%)]\tLoss: 0.351267\n",
            "Train Epoch: 7 [12800/60000 (21%)]\tLoss: 0.290932\n",
            "Train Epoch: 7 [25600/60000 (43%)]\tLoss: 0.336378\n",
            "Train Epoch: 7 [38400/60000 (64%)]\tLoss: 0.240247\n",
            "Train Epoch: 7 [51200/60000 (85%)]\tLoss: 0.552777\n",
            "\n",
            "Test set: Average loss: 0.0003, Accuracy: 9015/10000 (90%)\n",
            "\n",
            "Train Epoch: 8 [0/60000 (0%)]\tLoss: 0.417852\n",
            "Train Epoch: 8 [12800/60000 (21%)]\tLoss: 0.187432\n",
            "Train Epoch: 8 [25600/60000 (43%)]\tLoss: 0.110748\n",
            "Train Epoch: 8 [38400/60000 (64%)]\tLoss: 0.297640\n",
            "Train Epoch: 8 [51200/60000 (85%)]\tLoss: 0.261560\n",
            "\n",
            "Test set: Average loss: 0.0003, Accuracy: 9107/10000 (91%)\n",
            "\n",
            "Train Epoch: 9 [0/60000 (0%)]\tLoss: 0.187832\n",
            "Train Epoch: 9 [12800/60000 (21%)]\tLoss: 0.294297\n",
            "Train Epoch: 9 [25600/60000 (43%)]\tLoss: 0.191512\n",
            "Train Epoch: 9 [38400/60000 (64%)]\tLoss: 0.223854\n",
            "Train Epoch: 9 [51200/60000 (85%)]\tLoss: 0.201405\n",
            "\n",
            "Test set: Average loss: 0.0003, Accuracy: 9137/10000 (91%)\n",
            "\n",
            "Train Epoch: 10 [0/60000 (0%)]\tLoss: 0.107723\n",
            "Train Epoch: 10 [12800/60000 (21%)]\tLoss: 0.253108\n",
            "Train Epoch: 10 [25600/60000 (43%)]\tLoss: 0.136093\n",
            "Train Epoch: 10 [38400/60000 (64%)]\tLoss: 0.359570\n",
            "Train Epoch: 10 [51200/60000 (85%)]\tLoss: 0.395563\n",
            "\n",
            "Test set: Average loss: 0.0003, Accuracy: 9141/10000 (91%)\n",
            "\n"
          ]
        },
        {
          "data": {
            "image/png": "iVBORw0KGgoAAAANSUhEUgAAAh8AAAEGCAYAAADFbPcfAAAAOXRFWHRTb2Z0d2FyZQBNYXRwbG90bGliIHZlcnNpb24zLjUuMSwgaHR0cHM6Ly9tYXRwbG90bGliLm9yZy/YYfK9AAAACXBIWXMAAAsTAAALEwEAmpwYAAA93UlEQVR4nO3dd3xV9f3H8dcnm4S9IhtERlFRBFFwxT3QOjocba2tFke1df0ctWptq6W1w2rVFjuse68qAopEVEQRZAooICPsEUL2/P7+uDfJTXKT3CR3neT9fDzug3vO+Z5zPvcC537O9/s936855xARERGJloRYByAiIiIdi5IPERERiSolHyIiIhJVSj5EREQkqpR8iIiISFQlxTqAQL1793ZDhw5ttlxhYSEZGRmRDyhMFG/keS1mL8S7aNGi3c65PrGOo6V0HYkPXosXvBezF+Jt9DrinIub1/jx410o5s6dG1K5eKF4I89rMXshXuAzFwfXhZa+dB2JD16L1znvxeyFeBu7jqjZRURERKJKyYeIiIhElZIPERERiSolHyIiIhJVSj5EREQkqpR8iEhMmNm/zWynma1oZLuZ2YNmttbMlpnZEdGOUUQiQ8mHiMTK48AZTWw/Exjhf00FHo1CTCISBZ5LPp5asJH5WytiHYaItJFzbh6wt4ki5wJP+IcLWAB0N7N+0YlOwmXmim3sKShttty6XQXMX7c7ChHVmvflLjbvLYrqOQOVlFfy8qIcfMNh1JVfUs7rS7Y0uu/anfms3ltZZ92mPUXM+3JXnXUrtuRx52srKCmv5MXPNlNaUXefQP9bupVtecW8+nlOnfVvLdtGbmEZALsLSpm5Yluzn605cTXCaSieW7iJxDIlHyIdwABgc8Byjn9dgyufmU3FVztCZmYm2dnZzR68oKAgpHLxwovxvjl7Lte+V8SB3RK4a1KnJstfNrMQgMfPiN6InZfNLMSA//jPGe3v+KkvSnl3UwVb16/m0D51f44fXlLCwu2V5G1azeCuiQ32rf6+RvfMbrAu8DusXjdr2SZ2FjneW7SKC0elNDjejsIqbv2guGZ594Y1jOiRSG5JFTdkFzO6ZwK3TezEr+YXs2F/FQ+fnE5GsrX6s3su+RCRDiPYla3hLSLgnJsOTAeYMGGCy8rKavbg2dnZhFIuXngx3oPHT4L33iWvIqn52Ge+BRDdzzjzLVzAOaP9Hb+wZRGwnWGjxpA1tn+dbQ9+8RGwj4MPG8f4IT0b7hzs+2pi3c4i33+dTj36kpV1eIPDrd6+Hz74oGZ55MFjOWFkH1/NUPZcClwqWVlZ7Js3G6hi8uRj6JHRMIkJlZIPEYlXOcCggOWBwNYYxdJquwtK6ZmeQkKCL5faV1RGp5REUpMa3s1Wq6isYn9JBT0DLu6llY79JeV0TUtuVRxlFVUUlVXQPb35H4z9JeUkJyTQKcUXY2WVI7eojN6dUxuUrapy7PVvyy8pp6C0Nm4LSB8LSiswICPV97NTXFZJeVUVXdOSySsqrym3cmsew3pnUFBawe78Mnp1TmFvYRlDe2VQXlVFRkoS+4rKSEpMIDUpgbTkht9jcVklRWUV5JdU0LOzL5aS8kpyC8sZ1LMTu/JLCWzp2Ly3iIE9OrG1oIqd+0vokZFCcmIC+0vK2Z3vazIa1DOd5MTangp5xeXsLy4nMcHolJxI9/RkFm7IpXt6Mgd0S6NrWjKFpRW+v/+MFHILyxncK52S8kpKyiupqHLsLvA1ZWzcU8Tanfn0ykilR0YKuwtK2bHfd94F6/eSkZpEUkICmV1TWb09n/7da2uRduWXkpacQHF5bXPKzvwSqqp8f4/17S0sY09BKb06p1JaUcmSTfvo1TmVxRv31Sm3I6+EPQWlNc1SObnF5BaWsa+o4TFbw3PJhwW9GRKRdugN4Fozew44CshzzrW9sTmKduwv4aj75vCzk0dw46kjATj81+8weXgvnvnJ0Y3u98vXVvDcws2s/s0ZNT+ut84rZt87s9kwbUqrYrnyyc+Yu2ZXSPuP/dVs+nZJ5dM7TgHg9zNXM33eej6/89QGd7sPvbeWv7z7JfNvO4nJ094DYPLwXkwdUfeYh9w9iwSD9b/znf/4++eyK7+UDdOmcNivZ9eUm/Lgh03G9sNJQ/jvxxt9xxzQlTevO65BmZP+lM22vJJmP2e14/4wl9PGZDL7i2J+8eEczh7bj79dcgRjf1Ub1wVHDODP3z28Zvmwe2bXOcY1WcN5JHtdzfKGaVM4+O5ZdcpsmDaFsx/6kLU7C+qsv3/WGu6ftQaAT+84mYn3zgm6LZgj7323wbrA/evLXrOL8b99l6V3n9bgMwS65eVlDdaN+807Ne+tjT/FEe1wamY3mNlKM1thZs+aWVokzyci3mFmzwIfA6PMLMfMLjezq8zsKn+RGcB6YC3wGHBNjEJttZ3+u9f3Vu+os37+uj1N7vfWMl+OVVZZVbNuX2nQFqeQzV2zq/lCAXbm13YSnb1yuy+G4oZ3ve+t2Qn4Eq1q9T9fdeRVAR9hV37znVCDmbFie837FVv2By3TksSj2uwvav+O3lzWMMf939KmK91eXpzT5PZq9ROP+nbnl4V0nLYqKI1t38mI1XyY2QDgZ8AY51yxmb0AXITv8bo2adt/QRGJB865i5vZ7oCfRimciAryMIMnBXsqo6kPZzVFwvcFhPNYLTtv09sT21oVEGWxjjbSzS5JQCczKwfSCUN7rcf+fkWkHdqZX0LW/dm8cOUkDhnQrdFy1derYD9cZzwwj5nXH9/keYLt9+THG5j+wXoKSip48OJxHDeiT822N5dt5b63VjHvlhNJSgxesT30treYdf3xjDqgC9/9+8ccP7I31540otGygU760/s8cOHhXP/8EgD+etHhLM3JA+D8R+bXKXvZzEKY6WsSyA3oJzDu17PrLNc/R3Oq+0kE7j9lbD/WbM9vtlahJerHVVHlmox1a73almBlQ/msZz34QbNlwqG6iay1Nu4pCqn/UGMilnw457aY2R+BTUAxMNs516CBqaWPyOXvLyY9sdJzj5wp3sjyWsxei1fqyl6zi6KySh6fv4E/fuewVh1j9fb8xjc2cZN15+sra97/cdaaOsnHL15Zzv6SCgpLK+mW3nir+jOfbOSecw/h0w17+XTD3kaTj2Bueam2L8AvXlke8n7VcsPUYTHQW0GaSSSyqtpYAxXJZpce+AYJGgbsA140s+87554KLNfSR+T+vOJDKC3w3CNnijeyvBaz1+KVtmknrS4NtNfPJc2zNjZDRLLZ5RTga+fcLgAzewWYDDzV5F6h0L94EfGAtjYT/+g/n5KTW8x7N2cF3b40J49T/vw+a3cW0Csjhf0lvk6EVc7XRJCUYKy97ywWb8qts99/P95I57Tay//k383h0slDKS2vanJUTajbCbaorPHRMqV9a2sPiEgmH5uAo80sHV+zy8nAZ209qKHcQ0TaOf9FbvGmfYCveaUx1f0c9hTW9oWYs9r3BEpFlaOorIIL6vXHAHh4bu1joVvzSpj29uq2Ri0dSNw+auuc+wR4CVgMLPefa3qkziciEq/a+oRGYQsfiyyrqK2dqNLdmkRAQhw3u+Ccuxu4O+zHDfcBRUTaqLLKsWrbfg4Z0I09BaUUlVXy2ue+JoxteSU8kr2WwwZ2r7PP8ws3MXl4bwb1TOe1z7fQp0sqizbmkl8v2XhxUWhjSFT714fra94/Mndt6z6QSAR5boRTPWsrIjEX5A7oofe+4oF3v+KNa4/hm3/7qM62vOJy/jCzYdPJrS/7nhb54JYTax5fDYd1uwpr3geOuikSLyI6wqmISHsWeCtUPdrm9laMrhmu+TJEoiW5kXFkQuXJ5EPNLiLSnqhCV7ymjbmH95IP/R8VkXjx4qIcht72Fk8t2FizbuqTi1p8nLMfanoyNZF409YOp55LPkRE4s30eeubLyTSjnTM5EPtLiISS/Wuu04XJelgEhPi+FHbSFDbqIjEQkVlFQfd8XbQbZv3FrN5b3GUIxKJnYQ2Jh/erPkQEYmyj9fviXUIInGjX9e0Nu3vyeRDVZwiIiKx09aaD+81u8Q6ABHpEJxzzFq5g6xRfbjnf19QHjChmoi0jeeSDxGRaHhtyRZueH5prMMQaZc82ewiIhJpO/eXxjoEkXZLyYeIiIhEleeSD9OztiISBerWLhI5nks+REREJHTDemfEOoQGPJl86I5EREQkNNefMiLWITTguadd1OgiIpF0+yvLObB3BtPeXh3rUETaLU/WfIiIRMqzn27i3hmrYh2GdFA/OW4YYwd2C+sxnYMnfjyRrFF96qxPTmzd7fz3jx7c5pg8mXw4tbuIiEg7dMeUMbxx7bFtOsaGaVPYMG1KzXKVcxw/sg8njKxNPgb17MRX957VYN8DAoZNP7BPw74ic2/O4rfnHdqm+MCDyYcedhEREQld9Q17WUXzo/QGTl8S7Oc2XD/Bnks+REREvOaoYT1JS276J/f2M0e3+vhXHDsMgIsnNmwSqfJnH2ce0q9m3bQLxjYod+7h/essTxjSs0GZA7q1bUK5ako+RESkQ7n5tJERP8eIvp3rLJ/yjUxW/+bMOs0hgTZMm8KVJwyvsxys7Jh+XYOu/+XZY9gwbQq/u6Bhk0h1XcbgXuk1xz3moN4Nzv/Xi8bV6dbQPSOZDdOmMG5wdwBevnoSacmJQeNvKc8lH6bnXUQkQvKKymMdgrQT9bsIxLTLQAv6SQYrWh16OPtbei75EBGJlMN+PTvWIUgURGqk7EMGdK15f+LovnW2jenftX7xVqn/xEoohterhQk0YUiPOsunjcmseX/4wO4AnDDS91kyu4anyQU8OM4HaJAxEZFou3TSEJ74eGOr9z/6wJ4sWL+3RfssuetUtu4r4awHPwi6/ePbT2LS794DYMrYfry1bBsAN506khGZnbnqqcV1yp8/bgCvfr4FgA9vPZFjfz+3zvZ7Jqdx0VlZ5JdUsK+onD6dU+skpEvuOpUqB7sLSumRnsKR975bZ/+XrppMUVkl5ZVV9O6cyk9PPIgVW/IY2D2dwb3Sa8qlJCWE1PkTYOU9p7M5t4gzHvB9BzedNqpm/Suz53HnR8XNHmN8vQQj0FNXHEVhaUXtd/DNg/n5ySOoqHL0794JgOtOOogLjxwUtv4e4MXkQ60uIiJRN+qALi0q3ykJimt/05g4rFeLk4/u6Sl0T0+pWe7fLY2teSU1y/26daotHHBXOqBHJ84I6FxZrfrO3QwG9kivs214nwyGdIXUpERSOyfSu3Nq0HgAemakNNgGkJacWKdPRNe0ZCYP792gXLdOyezKD23W5IzUJEYfUFtrkphgNesHdWl740X9mJMSE+hbr4YjIcHCmniAml1EJEbM7AwzW2Nma83stiDbu5nZ/8xsqZmtNLMfxSJO8enWKblF5Xt3Cv/PS1pK450dAx8R7Zwa/L66e7rvMwT7LNGctHRQj07NF2rnvFfzISKeZ2aJwMPAqUAOsNDM3nDOfRFQ7KfAF865c8ysD7DGzJ52zpXFIOQOb8qh/biWzxvdfu/5h5CckMDIA7pw3sMf8Yuj0rj63aKa7cF+2l+9ZjKlFVVcNH0BAL//1qGkJiXy3MJNXDppaE25d288gd/NWEX39BTW7yqkV0YKD1x0OABv/exYCkoqeHz+BgCOG9GbU/39FubcdAI5ucX88N+fAnD5scPokpbERUf6Hkd98apJbN5bxI0vLG30c7130wmc9Kf3g2574scTWbJ5HwN7dOLQAaGPSvrs1KMZ9cuZAJw0um9Ic688N/Vouqa1LAGMZ55MPjTCqYjnTQTWOufWA5jZc8C5QGDy4YAu5rsl7QzsBSrqH0h8UhITKKsMrR9Ba5gZ4wZ35/NN+4Juv2Ti4Jragw3TppCdnV1ne7DL9pj+XUlNqq3NyBrVl8yuaZw3bkCdcgf17cy/LjuSm1/0JQm3njma40b4Ol4e3N/3o1+dfFwcEMfwPp0Z3qe2s2VyYgLfO2pIzfKRQ3vW/KA3Vu9xYJ/GO2seP7IPx49seQfQ1KRErs4azqPZ6xg/pAdj/R07m3L0gb1afJ545rnkQ10+RNqFAcDmgOUc4Kh6Zf4GvAFsBboAFzrngv66mtlUYCpAZmZmgx++YAoKCsjOzqbKOX48q4gBnb19dakK/tWETXZ2Nnl5jXdufP/9urUDBQUFBF6xN27Y0GCfD+bNIymhtsz8+fPpkdZ4c8327b5+EmtWrya7YF2dbTt3+fqCrFy5kvQ9axr9DPVtzvd9b0VFhRQUVDX5byeUf1eh2rTRV4H39fr1ZFtOq44R+B1HK+5w8VzyISLtQrBf+vo3x6cDS4CTgOHAO2b2gXNuf4MdnZsOTAeYMGGCy8rKajaA7OxssrKymL92N/AJWwq8U6V6+5mj+V3ArLvjh/TgzEMO4LdvtWxCvO9OGMgLnwX/4euVkcKewtoWrqysLB5Y+RHk7Qtavv53np2dzQMXjuD655cAMGToUB4/vjvPL9zM2yu2A3D88SeQkpQAM98CYNKkyU12bHxz11LYksOo0aPJmjCozrbBBxdw2yvLueb8I8mo3+fDf/xg/y5Wb98PH31A54zOdO5cFbTM4/128vzCzWRljW80tpYac0QJO55ezG0Xjg/auTUU2dnZ3HrGICoqq8jKath0c/He5Rw1rCdZ9WqS4oE6nIpILOQAgb8eA/HVcAT6EfCK81kLfA20fvzpRlTGWTtuYyNbVjt0QLc6I2ECvHDlJIb2ajgJWHP+8O3DGj3fojtPbbA+8JsaldmlyTgBzhs3oE5/hqxRfXn0+7U/4OHs43lgn868cOWkholHM0L5668fdzj07ZrGS1dPbnXiUe3qrOFcd3LwPiO/u+DQBk1Y8cJzyYcmlhNpFxYCI8xsmJmlABfha2IJtAk4GcDMMoFRwPpwB+K1UZOr4ixZ8rrqr1O/LdHlueRDRLzPOVcBXAvMAlYBLzjnVprZVWZ2lb/Yb4DJZrYcmAPc6pzbHZuI48cFRwxssM6AkZktG4ejKd+qd47DB3X3nTvgLvrb431lzh7bcDyNQNVziBwzvGGHyfq/966ZISSrn2I5LIQOmqHq52/mqf48Eh2e7POhvF/E+5xzM4AZ9db9PeD9VuC0SMcRT3e8gT+A6+47iyrnGHHH2wD8+/R0jj/+hJpBpqp9de+ZJCRYnRE0A48x/Bd1vmLW3nsmB/mPGcy6+84i8BTr7zur5v2lk4bw/aN9T4tUl3no4nH89aJxjR7vyKE9WXvvmSQlNn+v21ylzukHHxDysULVIyOl5jO///6msB1Xmua55MNrVaQiIqEKvLolJhiJAWsSzIL+6CY38UNcP1EBmv3hrr9PQsCymZHYYMK0hutaes6WCOexqgX7niSy1OwiIiIxUX9U0dQk/SR1FJ6r+RARaS8+v/NUxv3mnSbLzLnpBBLN2LBiYZ31M68/ji4tGPHyzeuObXIQsvm3nURecXnIx2uLt39+HIs25tbUOLxw5SS25RXTq41Pfoh3eC75iKf2WRHxvlheUnrUm6AsWJeH6hE6N9RbHzjZWCgOaWb47/7dO9XMYhpp3+jXlW/0q41/4rCeUTmvxI+I1nGZWXcze8nMVpvZKjObFMnziYi0xO6CUi755yexDqNGou6upIOIdM3HX4GZzrlv+5/lb9gduxX0mLuIhMO/Pvw64ufo1ik5aHPGq9dMrrP8w0lDuP6UkWE//18vOrxBX4r7vz2WXp2DTwvfHj148TiS1ak0rkQs+TCzrsDxwGUA/pko2zwbpW4MRCRcIn056dsllccuncC5D3/UYNu4wT3qLN9z7iERieHcwxuOcPmdekOTt3ffPKx/rEOQeiJZ83EgsAv4j5kdBiwCfu6cKwws1NIJoXJzi6msqozLiXIaUz2BlVd4LV7wXsxei7e9ivTNjCppRYKLZPKRBBwBXOec+8TM/grcBtwZWKilE0I9tnYBO3fnBp38J15VT2DlFV6LF7wXs9filda54ZSRDO3d9Jwrx43oTUl5ZVjOl5GSyPcnDeEf74d9FHqRsIpk8pED5DjnqntzvYQv+Wgz3U2ISDiEa9DCA7qmsX1/CQf370pBaQUb9xQBcMlRg5vd98nLjwpLDAArf30GgJIPiXsRe9rFObcd2Gxmo/yrTga+aOtxNcKpiIRLuJpdAo+jid9Emhfpp12uA572P+myHt8U2SIicSFctzKHD+rOtrztgJ7GEwlFRJMP59wSYEIkzyEiEm0vXz2Jbz36MV1Sk3jlmskUl1fy9orGk49P7ziZpIQEdheUUlhaQe8Ij+S5+M5TqahqfDRTkVjTCKci0nG18oKSnuK7dPbpmsqIzC6s2JIHNF7r0beLb9r2nhnRGVsjWucRaS3N4iMiHVZb72WC7d+RBu8SaS1PJh9qUxWRcGhtTWpyI3PIm8ETP57IgX0y+OQXJ7chMpH2zXPNLiIi8co56J6ewns3ZcU6FJG45smaDxGRcNCj+yKx4cnkQ60uIhJLg3qmc2CfDO75pm8+loP6dmZY7wx+efY3YhyZiDd4rtnF9LiLiIRJay8nqUmJdZpW0pITmXtzVqPlRaQuzyUfIhJ/zKwH0B8oBjY45zwxyMSijbmxDkGkQ1LyISKtYmbdgJ8CFwMp+GaxTgMyzWwB8Ihzbm4MQ2zSqj2VvP/lrliHIdIheS75UKOLSNx4CXgCOM45ty9wg5mNB35gZgc65/4Vi+Cas7PYE5UzIu2S55IPEYkPzrlTm9i2CFgUxXBaTj3XRWJGyYeIhIWZ9QF+DnQCHnXOrY1xSCISp/SorYiEy5+AecBM4NkYx9IsXUdEYsdzyYeetBWJD2Y208yOC1iVAmzwvyI7bWsUNDaEuoi0neeSDxGJGxcC55rZM2Y2HLgTuAuYBlwT08hEJK55M/lQfalIzDnn8pxzNwO/BH4LXAn81Dn3Lefch7GNru0eungcAH+58LCadTeeOpJ/XjohViGJtBue63CqilCR+GBmBwJXA+XATcBw4AUzexPfGB+VsYyvOc3Njn1gn85smDYFgBueXwrAz04eEemwRDoEb9Z8iEg8eBZf59IFwJPOuQ+cc6cD+4HZMY1MROKa52o+QK0uInEiDfgayADSq1c65/5rZi/ELKowyeyaVvM+JSmB4w7qHcNoRNoXzyUfmlhOJG5cA9wPlAFXBW5wzhXHJKIw6tYpueb9l789M4aRiLQ/anYRkVZxzn3k71x6sXNuaUv3N7MzzGyNma01s9saKZNlZkvMbKWZvd/2qEUkHngy+VCzi0jsmdn/zOxsM0sOsu1AM/u1mf24kX0TgYeBM4ExwMVmNqZeme7AI8A3nXMHA98JZ/y6jojEjveaXWIdgIhU+wlwI/BXM9tL7ay2Q4F1wN+cc683su9EYK1zbj2AmT0HnAt8EVDmEuAV59wmAOfczkh8iPq+d9RgLps8NBqnEumwPJd8iEh8cM5tB24BbjGzoUA/oBj40jlX1MzuA4DNAcs5wFH1yowEks0sG+gC/NU590QYQm/SvecfGulTiHR4Sj5EpM2ccxvwDaseqmCVmPVbQpKA8cDJ+Car+9jMFjjnvmxwMLOpwFSAzMxMsrOzmw2gtLQ0aBih7BsLBQUFcRtbMF6LF7wXs9fiDeS55EMPu4i0CznAoIDlgcDWIGV2O+cKgUIzmwccBjRIPpxz04HpABMmTHBZWVnNBvDuxnfwPahT677zDyXrqMEhf4hoys7OJpTPFS+8Fi94L2avxRvIkx1ORcTzFgIjzGyYmaUAFwFv1CvzOnCcmSWZWTq+ZplV4QqgMkiP00viNPEQaW+UfIhIm/ifeGnRtcQ5VwFcC8zCl1C84JxbaWZXmdlV/jKr8I2gugz4FPinc25FuOLeUlAVrkOJSAt5rtkFmp+TQUSi6iJ8T7y8DPzHnzQ0yzk3A5hRb93f6y3fj28gs7Cbl1MRicOKSAg8WPOhTh8i8cQ5931gHL7Ha/9jZh+b2VQz6xLj0EQkToWUfJhZRnW1qpmNNLNvBhtYSEQ6JufcfuBl4Dl8j9yeDyw2s+tiGpiIxKVQaz7mAWlmNgCYA/wIeDxSQTVHrS4i8cPMzjGzV4H3gGRgonPuTHxPptwc0+BEJC6F2ufDnHNFZnY58JBz7g9m9nkkA2s0ELW6iMSb7wB/cc7NC1zpv2YEHV49HmlUU5HoCbXmw8xsEvA94C3/Ok92VhWRsLsb39MoAJhZJ/+Ipzjn5sQqqJZKStCdjUi0hJp8XA/cDrzqfxzuQGBuxKISES95EQh8brXSv85TuqerG5tItIRUe+Gcex94H8Df8XS3c+5nkQysMbo3EYk7Sc65mqFCnXNl/oHDPGNA905MPX54rMMQ6TBCfdrlGTPramYZ+GadXGNm/xfZ0ETEI3aZ2TerF8zsXGB3DONpsUuOGkxKkgdHHhDxqFD/t43xP0p3Hr5BgQYDP4hUUM1xGmVMJJ5cBfzCzDaZ2WbgVuDKGMckInEs1E6jyf5xPc4D/uacKzezmGQAetpFJL4459YBR5tZZ3xPxuXHOqaWmjy8V6xDEOlQQk0+/oFvuuylwDwzGwLsD2VHM0sEPgO2OOfObk2QIhLfzGwKcDC+8YAAcM79OqZBtUBGqh7eE4mmkJpdnHMPOucGOOfOcj4bgRNDPMfPCeNMlCISX8zs78CFwHX4+oR/BxgS06BEJK6F2uG0m5n92cw+87/+BGSEsN9AYArwzzbGWWPB+r3kFKjPh0gcmeycuxTIdc7dA0wCBsU4phZRNzKR6Aq1rvHfwArgu/7lHwD/AS5oZr8HgFuARieYMrOpwFSAzMxMsrOzmzxgXnE5QLPl4klBQYHijTCvxey1eJtR4v+zyMz6A3uAYTGMR0TiXKjJx3Dn3LcClu8xsyVN7WBmZwM7nXOLzCyrsXLOuenAdIAJEya4rKxGi/rM9A2w2my5OJKdna14I8xrMXst3mb8z8y6A/cDi/FNv/RYTCNqIXVkF4muUJOPYjM71jn3IYCZHQMUN7PPMcA3zewsIA3oamZP+affFpF2wD/o4Bzn3D7gZTN7E0hzzuXFNrKWSU9JjHUIIh1KqMnHVcATZtbNv5wL/LCpHZxzt+Mbkh1/zcfNSjxE2hfnXJW/D9gk/3IpUBrbqFpuYI/0WIcg0qGE+rTLUufcYcBYYKxzbhxwUkQjExGvmG1m3zJT44WIhKZFD7f7RzmtdiO+DqWh7JcNZLfkXCLiGTfie/qtwsxK8D1u65xzXWMblojEq7ZMZhCTu5wLxg2gZ5pusETihXOui3MuwTmX4pzr6l9W4iEijWrLsH4xGl7dNLOtSBwxs+ODrXfOzYt2LCLiDU0mH2aWT/Akw4BOEYmoGS8vzonFaUWkcYEzXKcBE4FFqF+YiDSiyeTDOdfo4GAiIgDOuXMCl81sEPCHGIUjIh7Qlj4fIiLB5ACHxDoIEYlfmspRRNrEzB6itnk2ATgc3wzYIiJBKfkQkbb6LOB9BfCsc+6jWAUTivnrdsc6BJEOTcmHiLTVS0CJc64SwMwSzSzdOVcU47gatXSzp0Z/F2l31OdDRNpqDnWffusEvBujWETEAzybfBSVVcQ6BBHxSXPOFVQv+N/H9WQpVS4mwxSJiJ9nk4/9xUo+ROJEoZkdUb1gZuNpftbrmNpfUh7rEEQ6NPX5EJG2uh540cy2+pf7ARfGLpwQqOJDJKY8m3xo/kyR+OCcW2hmo4FR+EY/Xu2cU9WCiDTKs80uyj1E4oOZ/RTIcM6tcM4tBzqb2TWxjqspqvgQiS3PJh8iEjd+4pzbV73gnMsFfhK7cJrn1OFUJKa8m3yo6kMkXiSY1TaEmlkikNLcTmZ2hpmtMbO1ZnZbE+WONLNKM/t2mOKlSrmHSEx5NvkwZR8i8WIW8IKZnWxmJwHPAjOb2sGfoDwMnAmMAS42szGNlPu9/xwi0k54tsOpiMSNW4GpwNX46iRnA481s89EYK1zbj2AmT0HnAt8Ua/cdcDLwJHhDFitLiKx5dnkw6nLmEhccM5VAX/3vzCzY4GHgJ82sdsAYHPAcg5wVGABMxsAnA+cRDPJh5lNxZcAkZmZSXZ2dpMxb84prbPcXPl4UFBQ4Ik4q3ktXvBezF6LN5Bnkw8RiR9mdjhwMb7xPb4GXmlulyDr6t9RPADc6pyrtGaerXfOTQemA0yYMMFlZWU1WT57/0rYuKFmubny8SA7O9sTcVbzWrzgvZi9Fm8gzyUfYwd2Y1lOnp6VE4kxMxsJXIQv6dgDPA+Yc+7EEHbPAQYFLA8EttYrMwF4zp949AbOMrMK59xrbQy9jt9dcGg4DyciIfBch9MLjxzUfCERiYbVwMnAOc65Y51zDwGVIe67EBhhZsPMLAVfEvNGYAHn3DDn3FDn3FB8M+deE+7EA+DIoT3CfUgRaYbnko9qqvgQiblvAduBuWb2mJmdTIgPwTvnKoBr8T3Fsgp4wTm30syuMrOrIhZx7fkD3kf6bCJSn+eaXfSIrUh8cM69CrxqZhnAecANQKaZPQq86pyb3cz+M4AZ9db9vZGyl4Uj5mqB43wo9xCJPu/WfOiKIRIXnHOFzrmnnXNn4+u7sQRodNCweBD4tJyuJSLR57nkQxPKicQv59xe59w/nHMnxTqWpijhEIktzyUf1TTOh4iEwwFd02IdgkiH47nkIye3CIB73qg/EKKISGgCb126pSfHLA6RjspzycfDc9cBMHPl9hhHIiJepWYXkdjyXPIhItJ2yj5EYknJh4h0OFVVsY5ApGNT8iEiIiJR5bnkY5rmYRCRNlq2JS/WIYh0aJ5LPrqkqWe6iLTNqm37Yx2CSIfmueRDg4yJiIh4m/eSj1gHICIiIm3iveRD2YeIiIineS75UN2HiIiIt3kw+RAREREvi1jyYWaDzGyuma0ys5Vm9vPwHDccRxEREZFYSYrgsSuAm5xzi82sC7DIzN5xzmlGOBERkQ4sYjUfzrltzrnF/vf5wCpgQNuPW/v+43V72no4ERERibJI1nzUMLOhwDjgkyDbpgJTATIzM8nOzm7yWPuLaidl+Gjh55RujspHaJOCgoJmP1c88Vq84L2YvRaviEg4RfyX28w6Ay8D1zvnGgwr6JybDkwHmDBhgsvKymr2mLfMewuA0WPGkDW2fzjDjYjs7GxC+Vzxwmvxgvdi9lq8IiLhFNGnXcwsGV/i8bRz7pVwH/+u11eG+5AiIiISYZF82sWAfwGrnHN/jsQ59haWReKwIiIiEkGRrPk4BvgBcJKZLfG/zorg+URERMQDItbnwzn3IRqOVEREROrRCKciIiISVUo+RKTDOuUbfWMdgkiHpORDRDqc608ZAcD0H0yIcSQiHZOSDxEREYkqJR8i0mFpokqR2FDyISIdTuAcUSISfUo+RKTDqc49TFUfIjGh5ENERESiyvPJx6KNe2Mdgoh4jdpdRGLK88nHtx79mJteWBrrMESkhczsDDNbY2Zrzey2INu/Z2bL/K/5ZnZYuM7t0PDLIrHkyeRjWLe6Yb+8OIfNe4tiFI2ItJSZJQIPA2cCY4CLzWxMvWJfAyc458YCvwGmRzdKEYkUTyYfSUFuWb7amR/9QESktSYCa51z651zZcBzwLmBBZxz851zuf7FBcDAcJ1crS4isRWxieUiaXDXBL7aV9Vg/YL1exg3uDupSYkxiEpEWmAAsDlgOQc4qonylwNvN7bRzKYCUwEyMzPJzs5u8uQbNpZhuGbLxZOCggLFG2Fei9lr8QbyZPJx8egU5myqqLPuhueXkldcziVHDea+8w+NUWQiEqJgXS6C1keY2Yn4ko9jGzuYc246/maZCRMmuKysrCZPvrB0Nfb1OporF0+ys7MVb4R5LWavxRvIm80uCQ2vW3nF5QA888km1mzPZ3teSbTDEpHQ5QCDApYHAlvrFzKzscA/gXOdc3vCdXI1u4jElidrPppz+gPzAPjteYdw2MDupCQlMOqALjGOSkQCLARGmNkwYAtwEXBJYAEzGwy8AvzAOfdlOE+u3EMkttpl8lHtl6+tqHl/25mj6dsllQuOCFufNRFpJedchZldC8wCEoF/O+dWmtlV/u1/B+4CegGP+EcirXDOhW0aWj1qKxI77Tr5CDTt7dUAnD9uADe9uJTvjB/EpOG9YhyVSMflnJsBzKi37u8B768ArojEuR/NXheJw4pIiDzZ5wPg9jNHt2q/iirHK4u3cPFjCwCoqnJc+u9PmfflLpxzrN1ZEM4wRUREpB7P1nwM6ZXeqv2ufmpRneWi8krmfbmLeV/uolunZPKKy3n8R0eSNaovAO9+sYPSiiqmjO3X5pjbg7lrdnLby8t4//9OJC1ZjzSLiEjLebbm45RvZLZqv3dX7ax5P/S2tzjk7lk1y9VPzKzeXjtg2RVPfMZPn1lc5xgVlVWUlFe26vyNGXPXTH7yxGdhPWYk/PbNL9ixv5ScXI0oKyIirePZ5CMpMYEfHTM0IsfOLSprsO7zTbmUV/oGNrvyyUWMvnMmn6zfw6yV20M+7vR56xh5R/BxkorKKnnnix3NHuONpVt5fcmWoNteWZwT9Bh5ReW8vXxbyHE2JR6eEnDO8daybVRWxUM0IiLSUp5NPgB+fMywiBz3H++v52fPfs7O/bVjhZz/yPyaTqtzVvtqTy6cvoArn1zE8py8kI5734zVlFU2HJk10ITfvsOYu2Y2WF9RWcXGPYX87NnP+flzS4Lue+MLS+vUnuwuKGVvYRnXPruYq59ezP2zVvPYvPUhxRpu+0vKufaZxeQWNkzsWuqlRTn89JnF/Hf+hrYHJiIiUefp5MMi+KzcG0u3MvG+OXXW/evDr9m4p7BB2XP+9mFNM0xZRRXXPL2ItTvz+XKHr/mmosrV1JoArN1ZwKY9RfzqjZVc+u9P2ZZXXLNtd0EZRWWVrN9VUGegtN+9vZoT7s9ucO7SCl+flWAm/PZdjvjNOzWT7j08dx33zlgV4jcQXk9+vJE3l21j+gdtT3525pcCsKugtM3HEhGR6PNsh9NYCZYAAIy+cyarf3MG5z38Eau35zNjua85ZsO0KdyYXUTJnNq+Jaf8+f06+0763XsNjnfSn3xl7j5nDGP6deWjtbuDnvfet1bxxMcbW/w5nl+4iXGDezAys+Hga/vLHI/NW88Vxw3DGs3wjDmrdlDl4NQxofe/aW5kye/9cwE90lP42yVHNHEM30GCDHQbtGxFlSM50dN5tohIu+LpK3KP9JRYh1DHsb9/r05nVYDzH/mI/WU029zSmHv+9wUXTl/Q4Lh3vb6CB+d8xYdBkpKLpy+oqe0IZl9RGbe+vJzT/jIv6PbHlpVy74xVLN/SeHNSTm4Rl//3szrNPIWlFZRWVLJ5bxGfbdgbdL83l/lG0M5es5OXFuU02P7R2j28uWxbnfJPLqibXFV39UgIkhgtz8lj7praTsUPz13LiDveZn9JeaOfpSkFpRVNfpciItJynq75yEhNYsO0KQy97a1YhwL4mkzq+3zTvoicq6najo/X7+G4P8xtdHtBae2kfGc8MI+UpATeuPZYVm7NY8qDH9ZsK690vL18Gz0yUujfrRODe6Wzfpev2emy/yysKferN1byf6eP4uC7Z3Fg7wzW7/aVefiSI+jbNZXxg3swf50vScrJLa6z/7fH1444+8bSBlN7cO0znwPwg6OH1Kyr8td8BKuVOedvvvg3TJsCwPOf+SZOzS0sIyMliSrXslqQbz86n9Xb82uOV1XluPXlZVx+3DCqqnzNXuMG96DKnxElNFIds3JrHkN7ZZCR6un/ciIiYdEuroTLfnUaY381O9ZhxK099Tp5PjRnbc37wBqVwMQD4FuPzq+zfPmxwTv4Pj5/A4/7O39WJx5AzSPK3dOT2VdUW/Owt148f37nS3ILy+rUcMz7chd/fqfudB5FZRUccvesmqaiwtIKisoqSE1KJLekqqY5ptqSzfvYvNeX7Lz/5S6e+WRTTSIxZ9UOOiUnMvmg3oCveabKQWJA8rB5b1Gd7+erHfkUlFbw4qIcFny9p+bY/bqlsc3fP2fDtCl8vbuQE/+YzaEDuvG/646ltKKSKQ9+yLEH9eapK2pnjb/j1eWs2Z7PC1dO4p7/reTSyUMZ3qdz0O+42l/f/YrhfTM4e2x/tuwrZuHXezlv3IAG5Soqq1iwfi9HH9iTJDU5iUicaRfJR9e0ZN687ljOfujD5gt3QPklFXWWq2sDWuqtZa17XDcw8QC4xD+6LMCu/FIenPNVg30u/fendZYfnruW+2etAWoTpn99+DX/+vDrmjKPr6tNli77z6dkr6ntiHvX6ytr3u8pKOXy//qai1b9+gw6pSTyi1dX8Oynmzjr0AO4bPIwJg7rWaf2qH7tWnXiAdQkHgCPZK/lDzN9cVY3W1X5W9zqN5E9/ckmAP44ew3//Xgjby3fzrDe6ZxzWH++2LqfLfuKefLyo9i8t4g5q3Zw2THD+Mu7voTs7LH9+c6j89maV8LZY/s1SDAO8j/SfdnkofzqmwcDkF9STnFZJX27piEiEkvtIvkAOGRAt1iH4GmhNF1tD3j0uC0CaxOOvPfdkPapTjyasnTzvpr3gYlHfeN/W3vOb9w1kzH9uvLFtv0AzFi+nRnLt7d6ELvqxKPakws28p2ApqVD757Fg5eMo7i8tpbmEf88I7sLStldUMrCDbk125xzXPiPj9maV0K39OQ6x65+6qfK+Zp/nIPUpAQenltbs/X4/A2cP24AN7ywhB15JRSWVdY0IYmIxEq7ST4A3r3xBNZsz2dIr3TVgkjIqhOPQO+uan7At1Dc+doK9gb0BcovreBHAf1lmvPYB+vZ6q9ZueH5pTXrA5PFN5Zu5eYXlzbYt9q5D39UZ7mwtEJ9T0QkptpVY/BBfTszZWw/DhnQjQ3TpjD1+ANjHZJITVNJa9w3Y3WzZZpKPIIJ1rFXRCSa2vXtz/+dPorTxmSS2TWtyac/RDqSCI7NJyISknZV81FfcmICE4b2jOhIqCIiItIy7Tr5qNYpYOr3k0f3rXn/zcP689TlRwXbRaTdUjIuIrHWIZKPXp1TeeHKSay453SuPGE4AEcO7cGDF4/j2BG9WXjHKWSN6sOcm06o2edX54yJVbgiIiLtWrvu8xFo4rCeAIzp35UDuqZx82mjarb16ZLK4z+aCMCKe04nKcFISUzgvx9v5CfHHcgvXl0ek5hFImHJ5jwuPDLWUYhIR9Zhko9qnVOTWPCLk5vcXm3uzVkAfKNfF4b1zuDwX78D+EaxXJazjwO6pdG3Sxp3vraCycN7cfXTi7ni2GH888Ov6ZKW1GBwr1+dM4anP9nEVzsLGj3/qMwurNmR3+h2kbZavmVfrEMQkQ6uwyUfrTFucA8AXr1mMr07pwIwdmD3mu2/Oe8QoHY+kV+e7WuyefqTjZxx8AHc8VQ2i/ckctkxw7jsmGHs3F/Cvz78mmtOPIhFG/dy0uhMrvjvZ7y7agcvXj2J0vIqkhONpMQE1mzPJye3iJ8/twSASycN4bsTBtWMY7Jh2hScc7z4WQ63vLysQez9u6XVjBMR6O/fH89VTy0KzxcknnL6mANiHYKIdHARTT7M7Azgr0Ai8E/n3LRIni/SqpOQUH3vKN9kaBeNTuXvWVk16/t2TeP2s74BwEmjfSNp/vOHE2p3DBj9evyQHowf0oNTvpFJcXklPdNTSEgwZl5/HOnJvr8+M+O7Rw7iu0cO4p0vdlBZ5cgtKuPzTbn8/ltj2bS3iEE90lnw9R4yUpIYmdmFTimJvHHtMfxx9peMPqAL5ZVVjOjbhd+9vYpuSZXkFNSdJwV8TVKLN+ayp7CUwwZ2JzU5kbeXb+O3b61q0fdS36WThvDExxsZfUAX/vvjiRx13xwA7j5nDPf874s2HVsaOnZE71iHICIdXMSSDzNLBB4GTgVygIVm9oZzTr8mrZCRmlRnVMrRB3QNWu7UMbXDgl88cTAAQ3plADB5eN0fnbEDu/PEjyfWWXfJUYPJzs4mKyuLnNwiemWk8tLiHPYUlNI5NYnjR/apU/6K4w5kf3E50z9Yzwe3nERuURl7Csro1imZMf19Ma7cmkeX1GQG90oHYNHGXLqmJTGgRyeSEhJISUrg1+ceUnPMm08byZmH9mN4n86MPqArQ3ql0797J95evo2rn17M3eeM4UfHDKO8sooLHpnPo98/grVLP2V90hDGDuzGhKE9a0YAHdwznTd/diwVlY6eGSlUVjmKyirokpbM9c99zmtLtvLI947gmqcXc8ERAzhicA8cvpFJAV7/6TH06+5rXttXVMb/lm7lzoB5Yh753hGMzOzCul0FfLUjn90FZeQWlfH6kq2kpyRSVFYJwHfGD2TS8F4c3L8bpz8wz/fd+ZvoANKSEygpr2LJXaeyK7+UU/8yL+DvqRvLcvLqfO9P/Hgilc7x0mc5vLW84Zw7ow/oUmcY+96dU+menszanQX0ykhtUF5EJKqccxF5AZOAWQHLtwO3N7XP+PHjXSjmzp0bUrl4oXgjr37MewpK3a78kpD3z8ktqnlfVVXlSssrW7RPS3y5fb+b9sw7ddaVlle63fXifW/VDvfYvHWuvKLSPfHxBre3oNTtKSgNesyZK7a5jbsL3fa84pp1xWUV7s+z17g12/c755wrq6h0q7ftDzlO4DMXoetDJF+hXEdmLNvqHnjhnWbLxROv/b/0WrzOeS9mL8Tb2HUkks0uA4DA6VNzgAaDapjZVGAqQGZmJtnZ2c0euKCgIKRy8ULxRl44Ym44t25k9gE4uEtJs/EacBDw4QebGAQsXfh1o2VTgfX+ufQCG8HGJcPWVVvZGrByW/Mjtrd7Zx7aj057mp+sUEQiI5LJR7ChjBp0JHDOTQemA0yYMMFlBfSNaEx1s4BXKN7I81rMXotXRCScIjnIWA4wKGB5IKAZrURERDq4SCYfC4ERZjbMzFKAi4A3Ing+ERER8YCINbs45yrM7FpgFr5Hbf/tnFvZzG4iIiLSzkV0nA/n3AxgRiTPISIiIt7SISaWExERkfih5ENERESiSsmHiIiIRJX5BiCLD2a2C9gYQtHewO4IhxNOijfyvBazF+Id4pzr03yx+KLrSNzwWrzgvZi9EG/Q60hcJR+hMrPPnHMTmi8ZHxRv5HktZq/F2x557e9A8Uae12L2WryB1OwiIiIiUaXkQ0RERKLKq8nH9FgH0EKKN/K8FrPX4m2PvPZ3oHgjz2sxey3eGp7s8yEiIiLe5dWaDxEREfEoJR8iIiISVZ5KPszsDDNbY2Zrzey2GMYxyMzmmtkqM1tpZj/3r+9pZu+Y2Vf+P3sE7HO7P+41ZnZ6wPrxZrbcv+1BM7MIxp1oZp+b2Zseibe7mb1kZqv93/WkeI7ZzG7w/3tYYWbPmllaPMfbUek60ua4dR3RdaTtnHOeeOGbGXcdcCCQAiwFxsQoln7AEf73XYAvgTHAH4Db/OtvA37vfz/GH28qMMz/ORL92z4FJgEGvA2cGcG4bwSeAd70L8d7vP8FrvC/TwG6x2vMwADga6CTf/kF4LJ4jbejvnQdCUvcuo7oOtL2zxrrAFrwlzIJmBWwfDtwe6zj8sfyOnAqsAbo51/XD1gTLFZglv/z9ANWB6y/GPhHhGIcCMwBTgq4aMRzvF39/wmt3vq4jNl/0dgM9MQ3W/SbwGnxGm9Hfek60uYYdR2JYMwd6TripWaX6r+Uajn+dTFlZkOBccAnQKZzbhuA/8++/mKNxT7A/77++kh4ALgFqApYF8/xHgjsAv7jr+L9p5llxGvMzrktwB+BTcA2IM85Nzte4+3AdB1pmwfQdSRiMXek64iXko9g7VUxfU7YzDoDLwPXO+f2N1U0yDrXxPqwMrOzgZ3OuUWh7hJkXdTi9UsCjgAedc6NAwrxVTc2JtbfcQ/gXHxVn/2BDDP7flO7NBJX3P07b2fi7vvVdaRmfSToOlJ3fdzwUvKRAwwKWB4IbI1RLJhZMr4LxtPOuVf8q3eYWT//9n7ATv/6xmLP8b+vvz7cjgG+aWYbgOeAk8zsqTiOtzqGHOfcJ/7ll/BdROI15lOAr51zu5xz5cArwOQ4jrej0nWk9XQdiXzMHeY64qXkYyEwwsyGmVkKcBHwRiwC8fca/hewyjn354BNbwA/9L//Ib423Or1F5lZqpkNA0YAn/qrz/LN7Gj/MS8N2CdsnHO3O+cGOueG4vve3nPOfT9e4/XHvB3YbGaj/KtOBr6I45g3AUebWbr/PCcDq+I43o5K15FW0nVE15GwinWnk5a8gLPw9QhfB9wRwziOxVeFtQxY4n+dBfTC1xnrK/+fPQP2ucMf9xoCeh0DE4AV/m1/o17HqAjEnkVtR7G4jhc4HPjM/z2/BvSI55iBe4DV/nM9ia8HetzG21Ffuo6EJXZdR3QdadNLw6uLiIhIVHmp2UVERETaASUfIiIiElVKPkRERCSqlHyIiIhIVCn5EBERkahS8tFBmVmB/8+hZnZJmI/9i3rL88N5fBGJD7qOSGsp+ZChQIsuGmaW2EyROhcN59zkFsYkIt4yFF1HpAWUfMg04DgzW2JmN5hZopndb2YLzWyZmV0JYGZZZjbXzJ4BlvvXvWZmi8xspZlN9a+bBnTyH+9p/7rquyPzH3uFmS03swsDjp1tZi+Z2Woze9o/Kh9mNs3MvvDH8seofzsiEgpdR6RlYj3KmV6xeQEF/j+z8I9U6F+eCvzS/z4V38iAw/zlCoFhAWV7+v/shG8kvV6Bxw5yrm8B7wCJQCa+oYT7+Y+dh2/+gQTgY3yjP/bEN2pf9WB43WP9vemll161L11H9GrtSzUfUt9pwKVmtgTf9N698M0XAL45A74OKPszM1sKLMA3udEImnYs8KxzrtI5twN4Hzgy4Ng5zrkqfMNMDwX2AyXAP83sAqCojZ9NRKJD1xFpkpIPqc+A65xzh/tfw5xzs/3bCmsKmWXhm4FxknPuMOBzIC2EYzemNOB9JZDknKsAJuKb9fM8YGYLPoeIxI6uI9IkJR+SD3QJWJ4FXG2+qb4xs5FmlhFkv25ArnOuyMxGA0cHbCuv3r+eecCF/vbgPsDxwKeNBWZmnYFuzrkZwPX4JogSkfij64i0SFKsA5CYWwZU+Ks9Hwf+iq+qcrG/s9YufHcL9c0ErjKzZfjaUxcEbJsOLDOzxc657wWsfxWYBCzFN5vnLc657f6LTjBdgNfNLA3f3c4NrfqEIhJpuo5Ii2hWWxEREYkqNbuIiIhIVCn5EBERkahS8iEiIiJRpeRDREREokrJh4iIiESVkg8RERGJKiUfIiIiElX/D9dCxHnU4XopAAAAAElFTkSuQmCC",
            "text/plain": [
              "<Figure size 648x288 with 2 Axes>"
            ]
          },
          "metadata": {
            "needs_background": "light"
          },
          "output_type": "display_data"
        }
      ],
      "source": [
        "########################################################################\n",
        "#             TODO: Define appropriate network dimensions.             #\n",
        "########################################################################\n",
        "\n",
        "in_features, hidden_dim, out_features = 784, 30, 10\n",
        "\n",
        "########################################################################\n",
        "#                          END OF YOUR CODE                            #\n",
        "########################################################################\n",
        "\n",
        "# Training parameters\n",
        "learning_rate = 5e-1  # step size for gradient descent\n",
        "epochs = 10  # how many times to iterate through the intire training set\n",
        "\n",
        "# Define and initialize layers\n",
        "layers = [Linear(in_features, hidden_dim),\n",
        "          Sigmoid(),\n",
        "          Linear(hidden_dim, out_features)]\n",
        "\n",
        "# Initialize network\n",
        "net = Net(layers)\n",
        "\n",
        "# Define list to store loss of each iteration\n",
        "train_losses = []\n",
        "train_accs = []\n",
        "\n",
        "for epoch in range(epochs):\n",
        "    # Training loop\n",
        "    for i, (x_batch, y_batch) in enumerate(train_loader):\n",
        "        # Flatten input to 1D tensor\n",
        "        x_batch = x_batch.flatten(start_dim=1)\n",
        "        # Convert labels to one-hot encoding\n",
        "        y_batch = nn.functional.one_hot(y_batch, num_classes=10)\n",
        "\n",
        "        # Perform forward pass\n",
        "        y_pred = net.forward(x_batch)\n",
        "\n",
        "        ########################################################################\n",
        "        #      TODO: Calculate Cross Entropy loss between prediction and       #\n",
        "        #                     labels and append to list.                       #\n",
        "        ########################################################################\n",
        "\n",
        "        loss, grad = CrossEntropyLoss(y_batch, y_pred)\n",
        "        train_losses.append(loss)\n",
        "\n",
        "        ########################################################################\n",
        "        #                          END OF YOUR CODE                            #\n",
        "        ########################################################################\n",
        "\n",
        "\n",
        "        ########################################################################\n",
        "        #            TODO: Perform backward pass and optimizer step.           #\n",
        "        ########################################################################\n",
        "        \n",
        "        net.backward(grad)\n",
        "        net.optimizer_step(learning_rate)\n",
        "\n",
        "        ########################################################################\n",
        "        #                          END OF YOUR CODE                            #\n",
        "        ########################################################################\n",
        "\n",
        "        # Calculate accuracy of prediction\n",
        "        correct = torch.argmax(y_pred, axis=1) == torch.argmax(y_batch, axis=1)\n",
        "        train_accs.append(torch.sum(correct)/len(y_pred))\n",
        "\n",
        "        # Print progress\n",
        "        if i % 200 == 0:\n",
        "            print('Train Epoch: {} [{}/{} ({:.0f}%)]\\tLoss: {:.6f}'.format(\n",
        "                epoch + 1, i * len(x_batch), len(train_loader.dataset),\n",
        "                100. * i / len(train_loader), loss))\n",
        "\n",
        "    # Validation loop    \n",
        "    test_loss = 0\n",
        "    total_correct = 0\n",
        "    for x_batch, y_batch in test_loader:\n",
        "        # Flatten input to 1D tensor\n",
        "        x_batch = x_batch.flatten(start_dim=1)\n",
        "        # Convert labels to one-hot encoding\n",
        "        y_batch = nn.functional.one_hot(y_batch, num_classes=10)\n",
        "\n",
        "        # Perform forward pass with x_xor as input and y_pred as output variables\n",
        "        y_pred = net.forward(x_batch)\n",
        "\n",
        "        ########################################################################\n",
        "        #   TODO: Calculate Cross Entropy loss between prediction and labels.  #\n",
        "        ########################################################################\n",
        "\n",
        "        loss, _ = CrossEntropyLoss(y_batch, y_pred)\n",
        "\n",
        "        ########################################################################\n",
        "        #                          END OF YOUR CODE                            #\n",
        "        ########################################################################\n",
        "\n",
        "        # Keep track of total loss over test set\n",
        "        test_loss += loss\n",
        "\n",
        "        # Calculate accuracy of prediction\n",
        "        correct = torch.argmax(y_pred, axis=1) == torch.argmax(y_batch, axis=1)\n",
        "        total_correct += torch.sum(correct)\n",
        "\n",
        "    test_loss /= len(test_loader.dataset)\n",
        "    test_acc = 100. * total_correct / len(test_loader.dataset)\n",
        "    print('\\nTest set: Average loss: {:.4f}, Accuracy: {}/{} ({:.0f}%)\\n'.format(\n",
        "        test_loss, total_correct, len(test_loader.dataset), test_acc))\n",
        "    \n",
        "# Plot training curves\n",
        "plt.figure(figsize=(9,4))\n",
        "plt.subplot(1,2,1)\n",
        "plt.xlabel('Iterations')\n",
        "plt.ylabel('Loss')\n",
        "plt.plot(train_losses)\n",
        "plt.grid()\n",
        "\n",
        "plt.subplot(1,2,2)\n",
        "plt.xlabel('Iterations')\n",
        "plt.ylabel('Accuracy (%)')\n",
        "plt.plot(train_accs)\n",
        "plt.grid()"
      ]
    },
    {
      "cell_type": "markdown",
      "metadata": {
        "id": "48obmfkSS956"
      },
      "source": [
        "Your network can now classify handwritten digits with reasonably high accuracy. Well done!\n",
        "\n",
        "This concludes the second assignment. From the next assignment onwards we will start using more and more built-in PyTorch functionality and will get to know some more advanced methods for training neural networks. Also, we will implement the most prominent building block of modern computer vision: the convolutional layer."
      ]
    }
  ],
  "metadata": {
    "colab": {
      "provenance": [],
      "toc_visible": true
    },
    "kernelspec": {
      "display_name": "tf_cpu",
      "language": "python",
      "name": "python3"
    },
    "language_info": {
      "codemirror_mode": {
        "name": "ipython",
        "version": 3
      },
      "file_extension": ".py",
      "mimetype": "text/x-python",
      "name": "python",
      "nbconvert_exporter": "python",
      "pygments_lexer": "ipython3",
      "version": "3.9.7"
    },
    "vscode": {
      "interpreter": {
        "hash": "c787a08491dcdbfffe458afac131a89c5dc65d7591e57c155dcecfdeb67d0cb5"
      }
    }
  },
  "nbformat": 4,
  "nbformat_minor": 0
}