notebbok_04_glove_vectors_sentiment_classifier.ipynb

{
  "cells": [
    {
      "cell_type": "markdown",
      "metadata": {
        "id": "view-in-github",
        "colab_type": "text"
      },
      "source": [
        "<a href=\"https://colab.research.google.com/github/julurisaichandu/nlp/blob/main/glove_vectors_sentiment_classifier.ipynb\" target=\"_parent\"><img src=\"https://colab.research.google.com/assets/colab-badge.svg\" alt=\"Open In Colab\"/></a>"
      ]
    },
    {
      "cell_type": "markdown",
      "metadata": {
        "id": "ngBQCUjjYeqf"
      },
      "source": [
        "Notebook 4: How to make a racist model (without trying)\n",
        "===============\n",
        "\n",
        "CS 6120 Natural Language Processing, Amir\n",
        "\n"
      ]
    },
    {
      "cell_type": "markdown",
      "source": [
        "\n",
        "Saichandu Juluri"
      ],
      "metadata": {
        "id": "lQ7g74OMi_QL"
      }
    },
    {
      "cell_type": "code",
      "source": [
        "!zip -r dataset.zip /content/data"
      ],
      "metadata": {
        "colab": {
          "base_uri": "https://localhost:8080/"
        },
        "id": "2WLWy6BVkb_9",
        "outputId": "b196bf02-690a-4042-834d-5398c768b00b"
      },
      "execution_count": null,
      "outputs": [
        {
          "output_type": "stream",
          "name": "stdout",
          "text": [
            "updating: content/data/ (stored 0%)\n",
            "  adding: content/data/negative-words.txt (deflated 68%)\n",
            "  adding: content/data/.ipynb_checkpoints/ (stored 0%)\n",
            "  adding: content/data/glove.6B.50d.txt (deflated 60%)\n",
            "  adding: content/data/positive-words.txt (deflated 67%)\n"
          ]
        }
      ]
    },
    {
      "cell_type": "markdown",
      "metadata": {
        "id": "0lS5fqV6Yeqj"
      },
      "source": [
        "Saving notebooks as pdfs\n",
        "----------\n",
        "\n",
        "Feel free to add cells to this notebook as you wish. Make sure to leave **code that you've written** and any **answers to questions** that you've written in your notebook. Turn in your notebook as a pdf at the end of lecture's day.\n",
        "\n",
        "\n",
        "To convert your notebook to a pdf for turn in, you'll do the following:\n",
        "1. Kernel -> Restart & Run All (clear your kernel's memory and run all cells)\n",
        "2. File -> Download As -> .html -> open in a browser -> print to pdf\n",
        "\n",
        "(The download as pdf option doesn't preserve formatting and output as nicely as taking the step \"through\" html, but will do if the above doesn't work for you.)"
      ]
    },
    {
      "cell_type": "markdown",
      "metadata": {
        "id": "2fEGKfcdk_NE"
      },
      "source": [
        "*This notebook is part of the [ConceptNet blog](http://blog.conceptnet.io), written by Robyn Speer from [Luminoso](http://luminoso.com).*\n",
        "\n",
        "## Let's make a sentiment classifier!\n",
        "\n",
        "Sentiment analysis is a very frequently-implemented task in NLP, and it's no surprise. Recognizing whether people are expressing positive or negative opinions about things has obvious business applications. It's used in social media monitoring, customer feedback, and even automatic stock trading (leading to bots that [buy Berkshire Hathaway when Anne Hathaway gets a good movie review](https://www.theatlantic.com/technology/archive/2011/03/does-anne-hathaway-news-drive-berkshire-hathaways-stock/72661/)).\n",
        "\n",
        "It's simplistic, sometimes too simplistic, but it's one of the easiest ways to get measurable results from NLP. In a few steps, you can put text in one end and get positive and negative scores out the other, and you never have to figure out what you should do with a parse tree or a graph of entities or any difficult representation like that.\n",
        "\n",
        "So that's what we're going to do here, following the path of least resistance at every step, obtaining a classifier that should look very familiar to anyone involved in current NLP. For example, you can find this model described in the [Deep Averaging Networks](http://cs.umd.edu/~miyyer/pubs/2015_acl_dan.pdf) paper (Iyyer et al., 2015). This model is not the point of that paper, so don't take this as an attack on their results; it was there as an example of a well-known way to use word vectors.\n",
        "\n",
        "Here's the outline of what we're going to do:\n",
        "\n",
        "* Acquire some typical **word embeddings** to represent the meanings of words\n",
        "* Acquire **training and test data**, with gold-standard examples of positive and negative words\n",
        "* **Train a classifier**, using gradient descent, to recognize other positive and negative words based on their word embeddings\n",
        "* Compute **sentiment scores** for sentences of text using this classifier\n",
        "* **Behold the monstrosity** that we have created\n",
        "\n",
        "And at that point we will have shown \"how to make a racist AI without really trying\". Of course that would be a terrible place to leave it, so afterward, we're going to:\n",
        "\n",
        "* **Measure the problem** statistically, so we can recognize if we're solving it\n",
        "* **Improve the data** to obtain a semantic model that's more accurate _and_ less racist"
      ]
    },
    {
      "cell_type": "markdown",
      "metadata": {
        "id": "48GvndXwk_NG"
      },
      "source": [
        "## Software dependencies\n",
        "\n",
        "This tutorial is written in Python, and relies on a typical Python machine-learning stack: `numpy` and `scipy` for numerical computing, `pandas` for managing our data, and `scikit-learn` for machine learning. Later on we'll graph some things with `matplotlib` and `seaborn`.\n",
        "\n",
        "You could also replace `scikit-learn` with TensorFlow or Keras or something like that, as they can also train classifiers using gradient descent. But there's no need for the deep-learning abstractions they provide, as it only takes a single layer of machine learning to solve this problem."
      ]
    },
    {
      "cell_type": "code",
      "execution_count": null,
      "metadata": {
        "collapsed": true,
        "id": "XJ8c6q25k_NH"
      },
      "outputs": [],
      "source": [
        "import numpy as np\n",
        "import pandas as pd\n",
        "import matplotlib\n",
        "import seaborn\n",
        "import re\n",
        "import statsmodels.formula.api\n",
        "\n",
        "from sklearn.linear_model import SGDClassifier\n",
        "from sklearn.model_selection import train_test_split\n",
        "from sklearn.metrics import accuracy_score"
      ]
    },
    {
      "cell_type": "code",
      "execution_count": null,
      "metadata": {
        "collapsed": true,
        "id": "ev0Zez5gk_NJ"
      },
      "outputs": [],
      "source": [
        "# Configure how graphs will show up in this notebook\n",
        "%matplotlib inline\n",
        "seaborn.set_context('notebook', rc={'figure.figsize': (10, 6)}, font_scale=1.5)"
      ]
    },
    {
      "cell_type": "code",
      "source": [],
      "metadata": {
        "id": "i5UMSznRwRp7"
      },
      "execution_count": null,
      "outputs": []
    },
    {
      "cell_type": "markdown",
      "metadata": {
        "id": "xBxl4Oxlk_NK"
      },
      "source": [
        "## Step 1: Word embeddings\n",
        "\n",
        "Word embeddings are frequently used to represent words as inputs to machine learning. The words become vectors in a multi-dimensional space, where nearby vectors represent similar meanings. With word embeddings, you can compare words by (roughly) what they mean, not just exact string matches.\n",
        "\n",
        "Successfully training word vectors requires starting from hundreds of gigabytes of input text. Fortunately, various machine-learning groups have already done this and provided pre-trained word embeddings that we can download.\n",
        "\n",
        "Two very well-known datasets of pre-trained English word embeddings are **word2vec**, pretrained on Google News data, and **GloVe**, pretrained on the Common Crawl of web pages. We would get similar results for either one, but here we'll use GloVe because its source of data is more transparent.\n",
        "\n",
        "> **Why does it matter that the word embeddings are \"well-known\"?**\n",
        ">\n",
        "> I'm glad you asked, hypothetical questioner! We're trying to do something extremely typical at each step, and for some reason, comparison-shopping for better word embeddings isn't typical yet. Read on, and I hope you'll come out of this tutorial with the desire to use [modern, high-quality word embeddings](https://github.com/commonsense/conceptnet-numberbatch), especially those that are aware of algorithmic bias and try to mitigate it. But that's getting ahead of things.\n"
      ]
    },
    {
      "cell_type": "markdown",
      "metadata": {
        "id": "KLWj2RmIlRgL"
      },
      "source": [
        "### TODO: Download GloVe Embeddings\n",
        "\n",
        "GloVe comes in three sizes: 6B, 42B, and 840B. The 840B size is powerful, but requires significant post-processing to use it in a way that's an improvement over 42B. The 42B version is pretty good and is also neatly trimmed to a vocabulary of 1 million words. We will be using the 6B version for the sake of boosting speed. But if you are interested in using more powerful embeddings, feel free to download the 42B version instead.\n",
        "\n",
        "\n",
        "Create a `data` folder under the same directory as this notebook's. Download glove.6B.zip from https://nlp.stanford.edu/data/glove.6B.zip, extract it and move the file `glove.6B.50d.txt` to the `data` folder. Next we define a function to read the simple format of its word vectors."
      ]
    },
    {
      "cell_type": "code",
      "source": [
        "# !wget https://huggingface.co/stanfordnlp/glove/resolve/main/glove.6B.zip\n",
        "!unzip glove.6B.zip"
      ],
      "metadata": {
        "colab": {
          "base_uri": "https://localhost:8080/"
        },
        "id": "xvvI7-OzZKfc",
        "outputId": "0e15a719-ecc4-43e1-ba6b-7ee53aab5bb6"
      },
      "execution_count": null,
      "outputs": [
        {
          "output_type": "stream",
          "name": "stdout",
          "text": [
            "Archive:  glove.6B.zip\n",
            "error [glove.6B.zip]:  missing 512753664 bytes in zipfile\n",
            "  (attempting to process anyway)\n",
            "error: invalid zip file with overlapped components (possible zip bomb)\n"
          ]
        }
      ]
    },
    {
      "cell_type": "code",
      "source": [
        "!mkdir data\n",
        "!mv glove.6B.50d.txt data/"
      ],
      "metadata": {
        "id": "UY1GaIoEdUvs",
        "colab": {
          "base_uri": "https://localhost:8080/"
        },
        "outputId": "913117ef-f562-4b23-b25c-69f865d487cc"
      },
      "execution_count": null,
      "outputs": [
        {
          "output_type": "stream",
          "name": "stdout",
          "text": [
            "mkdir: cannot create directory ‘data’: File exists\n",
            "mv: cannot stat 'glove.6B.50d.txt': No such file or directory\n"
          ]
        }
      ]
    },
    {
      "cell_type": "code",
      "execution_count": null,
      "metadata": {
        "id": "Y__UCUl-k_NL",
        "colab": {
          "base_uri": "https://localhost:8080/"
        },
        "outputId": "d9f6cc03-7d99-4da6-e466-8fa79fa773f9"
      },
      "outputs": [
        {
          "output_type": "execute_result",
          "data": {
            "text/plain": [
              "(400001, 50)"
            ]
          },
          "metadata": {},
          "execution_count": 5
        }
      ],
      "source": [
        "def load_embeddings(filename):\n",
        "    \"\"\"\n",
        "    Load a DataFrame from the generalized text format used by word2vec, GloVe,\n",
        "    fastText, and ConceptNet Numberbatch. The main point where they differ is\n",
        "    whether there is an initial line with the dimensions of the matrix.\n",
        "    \"\"\"\n",
        "    labels = []\n",
        "    rows = []\n",
        "    with open(filename, encoding='utf-8') as infile:\n",
        "        for i, line in enumerate(infile):\n",
        "            items = line.rstrip().split(' ')\n",
        "            if len(items) == 2:\n",
        "                # This is a header row giving the shape of the matrix\n",
        "                continue\n",
        "            labels.append(items[0])\n",
        "            values = np.array([float(x) for x in items[1:]], 'f')\n",
        "            rows.append(values)\n",
        "\n",
        "    arr = np.vstack(rows)\n",
        "    return pd.DataFrame(arr, index=labels, dtype='f')\n",
        "\n",
        "embeddings = load_embeddings('data/glove.6B.50d.txt')\n",
        "embeddings.shape"
      ]
    },
    {
      "cell_type": "markdown",
      "metadata": {
        "id": "C-S2E51_k_NN"
      },
      "source": [
        "## Step 2: A gold-standard sentiment lexicon\n",
        "\n",
        "We need some input about which words are positive and which words are negative. There are many sentiment lexicons you could use, but we're going to go with a very straightforward lexicon (Hu and Liu, 2004), the same one used by the Deep Averaging Networks paper.\n"
      ]
    },
    {
      "cell_type": "markdown",
      "metadata": {
        "id": "eNvWL-rtmMDC"
      },
      "source": [
        "### TODO: Download Sentiment Lexicon\n",
        "\n",
        "Download the lexicon from http://www.cs.uic.edu/~liub/FBS/opinion-lexicon-English.rar and extract it into `data/positive-words.txt` and `data/negative-words.txt`.\n",
        "\n",
        "Next we define how to read these files, and read them in as the `pos_words` and `neg_words` variables:"
      ]
    },
    {
      "cell_type": "code",
      "source": [
        "!wget http://www.cs.uic.edu/~liub/FBS/opinion-lexicon-English.rar"
      ],
      "metadata": {
        "colab": {
          "base_uri": "https://localhost:8080/"
        },
        "id": "PTtKQwLxdkS5",
        "outputId": "d966b1f8-421e-406b-fa6f-92a323bdaf3f"
      },
      "execution_count": null,
      "outputs": [
        {
          "output_type": "stream",
          "name": "stdout",
          "text": [
            "--2024-09-30 19:44:29--  http://www.cs.uic.edu/~liub/FBS/opinion-lexicon-English.rar\n",
            "Resolving www.cs.uic.edu (www.cs.uic.edu)... 131.193.32.29\n",
            "Connecting to www.cs.uic.edu (www.cs.uic.edu)|131.193.32.29|:80... connected.\n",
            "HTTP request sent, awaiting response... 301 Moved Permanently\n",
            "Location: https://www.cs.uic.edu/~liub/FBS/opinion-lexicon-English.rar [following]\n",
            "--2024-09-30 19:44:30--  https://www.cs.uic.edu/~liub/FBS/opinion-lexicon-English.rar\n",
            "Connecting to www.cs.uic.edu (www.cs.uic.edu)|131.193.32.29|:443... connected.\n",
            "HTTP request sent, awaiting response... 200 OK\n",
            "Length: 24020 (23K)\n",
            "Saving to: ‘opinion-lexicon-English.rar’\n",
            "\n",
            "opinion-lexicon-Eng 100%[===================>]  23.46K   130KB/s    in 0.2s    \n",
            "\n",
            "2024-09-30 19:44:30 (130 KB/s) - ‘opinion-lexicon-English.rar’ saved [24020/24020]\n",
            "\n"
          ]
        }
      ]
    },
    {
      "cell_type": "code",
      "source": [
        "!pip install unrar"
      ],
      "metadata": {
        "colab": {
          "base_uri": "https://localhost:8080/"
        },
        "id": "wg_L1-uIdyAf",
        "outputId": "5f37b986-07ea-4641-8c1f-0c71cf3f2f00"
      },
      "execution_count": null,
      "outputs": [
        {
          "output_type": "stream",
          "name": "stdout",
          "text": [
            "Collecting unrar\n",
            "  Downloading unrar-0.4-py3-none-any.whl.metadata (3.0 kB)\n",
            "Downloading unrar-0.4-py3-none-any.whl (25 kB)\n",
            "Installing collected packages: unrar\n",
            "Successfully installed unrar-0.4\n"
          ]
        }
      ]
    },
    {
      "cell_type": "code",
      "source": [
        "!unrar x opinion-lexicon-English.rar data/"
      ],
      "metadata": {
        "id": "rUOoRcpld1B3"
      },
      "execution_count": null,
      "outputs": []
    },
    {
      "cell_type": "code",
      "execution_count": null,
      "metadata": {
        "collapsed": true,
        "id": "M1mGurhHk_NO"
      },
      "outputs": [],
      "source": [
        "def load_lexicon(filename):\n",
        "    \"\"\"\n",
        "    Load a file from Bing Liu's sentiment lexicon\n",
        "    (https://www.cs.uic.edu/~liub/FBS/sentiment-analysis.html), containing\n",
        "    English words in Latin-1 encoding.\n",
        "\n",
        "    One file contains a list of positive words, and the other contains\n",
        "    a list of negative words. The files contain comment lines starting\n",
        "    with ';' and blank lines, which should be skipped.\n",
        "    \"\"\"\n",
        "    lexicon = []\n",
        "    with open(filename, encoding='latin-1') as infile:\n",
        "        for line in infile:\n",
        "            line = line.rstrip()\n",
        "            if line and not line.startswith(';'):\n",
        "                lexicon.append(line)\n",
        "    return lexicon\n",
        "\n",
        "pos_words = load_lexicon('data/positive-words.txt')\n",
        "neg_words = load_lexicon('data/negative-words.txt')"
      ]
    },
    {
      "cell_type": "markdown",
      "metadata": {
        "id": "cxzP6Azrk_NQ"
      },
      "source": [
        "## Step 3: Train a model to predict word sentiments"
      ]
    },
    {
      "cell_type": "markdown",
      "metadata": {
        "id": "VgCVsqzek_NR"
      },
      "source": [
        "Our data points here are the embeddings of these positive and negative words. We use the Pandas `.loc[]` operation to look up the embeddings of all the words.\n",
        "\n",
        "Some of these words are not in the GloVe vocabulary, particularly the misspellings such as \"fancinating\". Those words end up with rows full of `NaN` to indicate their missing embeddings, so we use `.dropna()` to remove them."
      ]
    },
    {
      "cell_type": "code",
      "execution_count": null,
      "metadata": {
        "id": "2RwLyapNk_NS"
      },
      "outputs": [],
      "source": [
        "pos_vectors = embeddings.loc[embeddings.index.isin(pos_words)].dropna()\n",
        "neg_vectors = embeddings.loc[embeddings.index.isin(neg_words)].dropna()"
      ]
    },
    {
      "cell_type": "markdown",
      "metadata": {
        "id": "WRotWIx4k_NS"
      },
      "source": [
        "Now we make arrays of the desired inputs and outputs. The inputs are the embeddings, and the outputs are 1 for positive words and -1 for negative words. We also make sure to keep track of the words they're labeled with, so we can interpret the results."
      ]
    },
    {
      "cell_type": "code",
      "execution_count": null,
      "metadata": {
        "collapsed": true,
        "id": "UDjCz7Cik_NT"
      },
      "outputs": [],
      "source": [
        "vectors = pd.concat([pos_vectors, neg_vectors])\n",
        "targets = np.array([1 for entry in pos_vectors.index] + [-1 for entry in neg_vectors.index])\n",
        "labels = list(pos_vectors.index) + list(neg_vectors.index)"
      ]
    },
    {
      "cell_type": "markdown",
      "metadata": {
        "id": "Xc-jUoGpk_NU"
      },
      "source": [
        "> **Hold on. Some words are neither positive nor negative, they're neutral. Shouldn't there be a third class for neutral words?**\n",
        ">\n",
        "> I think that having examples of neutral words would be quite beneficial, especially because the problems we're going to see come from assigning sentiment to words that shouldn't have sentiment. If we could reliably identify when words should be neutral, it would be worth the slight extra complexity of a 3-class classifier. It requires finding a source of examples of neutral words, because Liu's data only lists positive and negative words.\n",
        ">\n",
        "> So I tried a version of this notebook where I put in 800 examples of neutral words, and put a strong weight on predicting words to be neutral. But the end results were not much different from what you're about to see.\n",
        ">\n",
        "> **How is this list drawing the line between positive and negative anyway? Doesn't that depend on context?**\n",
        ">\n",
        "> Good question. Domain-general sentiment analysis isn't as straightforward as it sounds. The decision boundary we're trying to find is fairly arbitrary in places. In this list, \"audacious\" is marked as \"bad\" while \"ambitious\" is \"good\". \"Comical\" is bad, \"humorous\" is good. \"Refund\" is good, even though it's typically in bad situations that you have to request one or pay one.\n",
        ">\n",
        "> I think everyone knows that sentiment requires context, but when implementing an easy approach to sentiment analysis, you just have to kind of hope that you can ignore context and the sentiments will average out to the right trend."
      ]
    },
    {
      "cell_type": "markdown",
      "metadata": {
        "id": "K1oq2z-Ik_NU"
      },
      "source": [
        "Using the scikit-learn `train_test_split` function, we simultaneously separate the input vectors, output values, and labels into training and test data, with 10% of the data used for testing."
      ]
    },
    {
      "cell_type": "code",
      "execution_count": null,
      "metadata": {
        "collapsed": true,
        "id": "QTatslxak_NU"
      },
      "outputs": [],
      "source": [
        "train_vectors, test_vectors, train_targets, test_targets, train_labels, test_labels = \\\n",
        "    train_test_split(vectors, targets, labels, test_size=0.1, random_state=0)"
      ]
    },
    {
      "cell_type": "markdown",
      "metadata": {
        "id": "cx2943qDk_NV"
      },
      "source": [
        "Now we make our classifier, and train it by running the training vectors through it for 100 iterations. We use a logistic function as the loss, so that the resulting classifier can output the probability that a word is positive or negative."
      ]
    },
    {
      "cell_type": "code",
      "execution_count": null,
      "metadata": {
        "id": "NEkgUqOLk_NV"
      },
      "outputs": [],
      "source": [
        "model = SGDClassifier(loss='log_loss', random_state=0, max_iter=100)"
      ]
    },
    {
      "cell_type": "markdown",
      "metadata": {
        "id": "Da-8NhbDnRnv"
      },
      "source": [
        "### TODO: Train a SGD Classifier"
      ]
    },
    {
      "cell_type": "code",
      "execution_count": null,
      "metadata": {
        "id": "2_iRA6FRnaR5",
        "colab": {
          "base_uri": "https://localhost:8080/",
          "height": 80
        },
        "outputId": "63954377-a2a6-416a-c773-42159fdb0eac"
      },
      "outputs": [
        {
          "output_type": "execute_result",
          "data": {
            "text/plain": [
              "SGDClassifier(loss='log_loss', max_iter=100, random_state=0)"
            ],
            "text/html": [
              "<style>#sk-container-id-3 {\n",
              "  /* Definition of color scheme common for light and dark mode */\n",
              "  --sklearn-color-text: black;\n",
              "  --sklearn-color-line: gray;\n",
              "  /* Definition of color scheme for unfitted estimators */\n",
              "  --sklearn-color-unfitted-level-0: #fff5e6;\n",
              "  --sklearn-color-unfitted-level-1: #f6e4d2;\n",
              "  --sklearn-color-unfitted-level-2: #ffe0b3;\n",
              "  --sklearn-color-unfitted-level-3: chocolate;\n",
              "  /* Definition of color scheme for fitted estimators */\n",
              "  --sklearn-color-fitted-level-0: #f0f8ff;\n",
              "  --sklearn-color-fitted-level-1: #d4ebff;\n",
              "  --sklearn-color-fitted-level-2: #b3dbfd;\n",
              "  --sklearn-color-fitted-level-3: cornflowerblue;\n",
              "\n",
              "  /* Specific color for light theme */\n",
              "  --sklearn-color-text-on-default-background: var(--sg-text-color, var(--theme-code-foreground, var(--jp-content-font-color1, black)));\n",
              "  --sklearn-color-background: var(--sg-background-color, var(--theme-background, var(--jp-layout-color0, white)));\n",
              "  --sklearn-color-border-box: var(--sg-text-color, var(--theme-code-foreground, var(--jp-content-font-color1, black)));\n",
              "  --sklearn-color-icon: #696969;\n",
              "\n",
              "  @media (prefers-color-scheme: dark) {\n",
              "    /* Redefinition of color scheme for dark theme */\n",
              "    --sklearn-color-text-on-default-background: var(--sg-text-color, var(--theme-code-foreground, var(--jp-content-font-color1, white)));\n",
              "    --sklearn-color-background: var(--sg-background-color, var(--theme-background, var(--jp-layout-color0, #111)));\n",
              "    --sklearn-color-border-box: var(--sg-text-color, var(--theme-code-foreground, var(--jp-content-font-color1, white)));\n",
              "    --sklearn-color-icon: #878787;\n",
              "  }\n",
              "}\n",
              "\n",
              "#sk-container-id-3 {\n",
              "  color: var(--sklearn-color-text);\n",
              "}\n",
              "\n",
              "#sk-container-id-3 pre {\n",
              "  padding: 0;\n",
              "}\n",
              "\n",
              "#sk-container-id-3 input.sk-hidden--visually {\n",
              "  border: 0;\n",
              "  clip: rect(1px 1px 1px 1px);\n",
              "  clip: rect(1px, 1px, 1px, 1px);\n",
              "  height: 1px;\n",
              "  margin: -1px;\n",
              "  overflow: hidden;\n",
              "  padding: 0;\n",
              "  position: absolute;\n",
              "  width: 1px;\n",
              "}\n",
              "\n",
              "#sk-container-id-3 div.sk-dashed-wrapped {\n",
              "  border: 1px dashed var(--sklearn-color-line);\n",
              "  margin: 0 0.4em 0.5em 0.4em;\n",
              "  box-sizing: border-box;\n",
              "  padding-bottom: 0.4em;\n",
              "  background-color: var(--sklearn-color-background);\n",
              "}\n",
              "\n",
              "#sk-container-id-3 div.sk-container {\n",
              "  /* jupyter's `normalize.less` sets `[hidden] { display: none; }`\n",
              "     but bootstrap.min.css set `[hidden] { display: none !important; }`\n",
              "     so we also need the `!important` here to be able to override the\n",
              "     default hidden behavior on the sphinx rendered scikit-learn.org.\n",
              "     See: https://github.com/scikit-learn/scikit-learn/issues/21755 */\n",
              "  display: inline-block !important;\n",
              "  position: relative;\n",
              "}\n",
              "\n",
              "#sk-container-id-3 div.sk-text-repr-fallback {\n",
              "  display: none;\n",
              "}\n",
              "\n",
              "div.sk-parallel-item,\n",
              "div.sk-serial,\n",
              "div.sk-item {\n",
              "  /* draw centered vertical line to link estimators */\n",
              "  background-image: linear-gradient(var(--sklearn-color-text-on-default-background), var(--sklearn-color-text-on-default-background));\n",
              "  background-size: 2px 100%;\n",
              "  background-repeat: no-repeat;\n",
              "  background-position: center center;\n",
              "}\n",
              "\n",
              "/* Parallel-specific style estimator block */\n",
              "\n",
              "#sk-container-id-3 div.sk-parallel-item::after {\n",
              "  content: \"\";\n",
              "  width: 100%;\n",
              "  border-bottom: 2px solid var(--sklearn-color-text-on-default-background);\n",
              "  flex-grow: 1;\n",
              "}\n",
              "\n",
              "#sk-container-id-3 div.sk-parallel {\n",
              "  display: flex;\n",
              "  align-items: stretch;\n",
              "  justify-content: center;\n",
              "  background-color: var(--sklearn-color-background);\n",
              "  position: relative;\n",
              "}\n",
              "\n",
              "#sk-container-id-3 div.sk-parallel-item {\n",
              "  display: flex;\n",
              "  flex-direction: column;\n",
              "}\n",
              "\n",
              "#sk-container-id-3 div.sk-parallel-item:first-child::after {\n",
              "  align-self: flex-end;\n",
              "  width: 50%;\n",
              "}\n",
              "\n",
              "#sk-container-id-3 div.sk-parallel-item:last-child::after {\n",
              "  align-self: flex-start;\n",
              "  width: 50%;\n",
              "}\n",
              "\n",
              "#sk-container-id-3 div.sk-parallel-item:only-child::after {\n",
              "  width: 0;\n",
              "}\n",
              "\n",
              "/* Serial-specific style estimator block */\n",
              "\n",
              "#sk-container-id-3 div.sk-serial {\n",
              "  display: flex;\n",
              "  flex-direction: column;\n",
              "  align-items: center;\n",
              "  background-color: var(--sklearn-color-background);\n",
              "  padding-right: 1em;\n",
              "  padding-left: 1em;\n",
              "}\n",
              "\n",
              "\n",
              "/* Toggleable style: style used for estimator/Pipeline/ColumnTransformer box that is\n",
              "clickable and can be expanded/collapsed.\n",
              "- Pipeline and ColumnTransformer use this feature and define the default style\n",
              "- Estimators will overwrite some part of the style using the `sk-estimator` class\n",
              "*/\n",
              "\n",
              "/* Pipeline and ColumnTransformer style (default) */\n",
              "\n",
              "#sk-container-id-3 div.sk-toggleable {\n",
              "  /* Default theme specific background. It is overwritten whether we have a\n",
              "  specific estimator or a Pipeline/ColumnTransformer */\n",
              "  background-color: var(--sklearn-color-background);\n",
              "}\n",
              "\n",
              "/* Toggleable label */\n",
              "#sk-container-id-3 label.sk-toggleable__label {\n",
              "  cursor: pointer;\n",
              "  display: block;\n",
              "  width: 100%;\n",
              "  margin-bottom: 0;\n",
              "  padding: 0.5em;\n",
              "  box-sizing: border-box;\n",
              "  text-align: center;\n",
              "}\n",
              "\n",
              "#sk-container-id-3 label.sk-toggleable__label-arrow:before {\n",
              "  /* Arrow on the left of the label */\n",
              "  content: \"▸\";\n",
              "  float: left;\n",
              "  margin-right: 0.25em;\n",
              "  color: var(--sklearn-color-icon);\n",
              "}\n",
              "\n",
              "#sk-container-id-3 label.sk-toggleable__label-arrow:hover:before {\n",
              "  color: var(--sklearn-color-text);\n",
              "}\n",
              "\n",
              "/* Toggleable content - dropdown */\n",
              "\n",
              "#sk-container-id-3 div.sk-toggleable__content {\n",
              "  max-height: 0;\n",
              "  max-width: 0;\n",
              "  overflow: hidden;\n",
              "  text-align: left;\n",
              "  /* unfitted */\n",
              "  background-color: var(--sklearn-color-unfitted-level-0);\n",
              "}\n",
              "\n",
              "#sk-container-id-3 div.sk-toggleable__content.fitted {\n",
              "  /* fitted */\n",
              "  background-color: var(--sklearn-color-fitted-level-0);\n",
              "}\n",
              "\n",
              "#sk-container-id-3 div.sk-toggleable__content pre {\n",
              "  margin: 0.2em;\n",
              "  border-radius: 0.25em;\n",
              "  color: var(--sklearn-color-text);\n",
              "  /* unfitted */\n",
              "  background-color: var(--sklearn-color-unfitted-level-0);\n",
              "}\n",
              "\n",
              "#sk-container-id-3 div.sk-toggleable__content.fitted pre {\n",
              "  /* unfitted */\n",
              "  background-color: var(--sklearn-color-fitted-level-0);\n",
              "}\n",
              "\n",
              "#sk-container-id-3 input.sk-toggleable__control:checked~div.sk-toggleable__content {\n",
              "  /* Expand drop-down */\n",
              "  max-height: 200px;\n",
              "  max-width: 100%;\n",
              "  overflow: auto;\n",
              "}\n",
              "\n",
              "#sk-container-id-3 input.sk-toggleable__control:checked~label.sk-toggleable__label-arrow:before {\n",
              "  content: \"▾\";\n",
              "}\n",
              "\n",
              "/* Pipeline/ColumnTransformer-specific style */\n",
              "\n",
              "#sk-container-id-3 div.sk-label input.sk-toggleable__control:checked~label.sk-toggleable__label {\n",
              "  color: var(--sklearn-color-text);\n",
              "  background-color: var(--sklearn-color-unfitted-level-2);\n",
              "}\n",
              "\n",
              "#sk-container-id-3 div.sk-label.fitted input.sk-toggleable__control:checked~label.sk-toggleable__label {\n",
              "  background-color: var(--sklearn-color-fitted-level-2);\n",
              "}\n",
              "\n",
              "/* Estimator-specific style */\n",
              "\n",
              "/* Colorize estimator box */\n",
              "#sk-container-id-3 div.sk-estimator input.sk-toggleable__control:checked~label.sk-toggleable__label {\n",
              "  /* unfitted */\n",
              "  background-color: var(--sklearn-color-unfitted-level-2);\n",
              "}\n",
              "\n",
              "#sk-container-id-3 div.sk-estimator.fitted input.sk-toggleable__control:checked~label.sk-toggleable__label {\n",
              "  /* fitted */\n",
              "  background-color: var(--sklearn-color-fitted-level-2);\n",
              "}\n",
              "\n",
              "#sk-container-id-3 div.sk-label label.sk-toggleable__label,\n",
              "#sk-container-id-3 div.sk-label label {\n",
              "  /* The background is the default theme color */\n",
              "  color: var(--sklearn-color-text-on-default-background);\n",
              "}\n",
              "\n",
              "/* On hover, darken the color of the background */\n",
              "#sk-container-id-3 div.sk-label:hover label.sk-toggleable__label {\n",
              "  color: var(--sklearn-color-text);\n",
              "  background-color: var(--sklearn-color-unfitted-level-2);\n",
              "}\n",
              "\n",
              "/* Label box, darken color on hover, fitted */\n",
              "#sk-container-id-3 div.sk-label.fitted:hover label.sk-toggleable__label.fitted {\n",
              "  color: var(--sklearn-color-text);\n",
              "  background-color: var(--sklearn-color-fitted-level-2);\n",
              "}\n",
              "\n",
              "/* Estimator label */\n",
              "\n",
              "#sk-container-id-3 div.sk-label label {\n",
              "  font-family: monospace;\n",
              "  font-weight: bold;\n",
              "  display: inline-block;\n",
              "  line-height: 1.2em;\n",
              "}\n",
              "\n",
              "#sk-container-id-3 div.sk-label-container {\n",
              "  text-align: center;\n",
              "}\n",
              "\n",
              "/* Estimator-specific */\n",
              "#sk-container-id-3 div.sk-estimator {\n",
              "  font-family: monospace;\n",
              "  border: 1px dotted var(--sklearn-color-border-box);\n",
              "  border-radius: 0.25em;\n",
              "  box-sizing: border-box;\n",
              "  margin-bottom: 0.5em;\n",
              "  /* unfitted */\n",
              "  background-color: var(--sklearn-color-unfitted-level-0);\n",
              "}\n",
              "\n",
              "#sk-container-id-3 div.sk-estimator.fitted {\n",
              "  /* fitted */\n",
              "  background-color: var(--sklearn-color-fitted-level-0);\n",
              "}\n",
              "\n",
              "/* on hover */\n",
              "#sk-container-id-3 div.sk-estimator:hover {\n",
              "  /* unfitted */\n",
              "  background-color: var(--sklearn-color-unfitted-level-2);\n",
              "}\n",
              "\n",
              "#sk-container-id-3 div.sk-estimator.fitted:hover {\n",
              "  /* fitted */\n",
              "  background-color: var(--sklearn-color-fitted-level-2);\n",
              "}\n",
              "\n",
              "/* Specification for estimator info (e.g. \"i\" and \"?\") */\n",
              "\n",
              "/* Common style for \"i\" and \"?\" */\n",
              "\n",
              ".sk-estimator-doc-link,\n",
              "a:link.sk-estimator-doc-link,\n",
              "a:visited.sk-estimator-doc-link {\n",
              "  float: right;\n",
              "  font-size: smaller;\n",
              "  line-height: 1em;\n",
              "  font-family: monospace;\n",
              "  background-color: var(--sklearn-color-background);\n",
              "  border-radius: 1em;\n",
              "  height: 1em;\n",
              "  width: 1em;\n",
              "  text-decoration: none !important;\n",
              "  margin-left: 1ex;\n",
              "  /* unfitted */\n",
              "  border: var(--sklearn-color-unfitted-level-1) 1pt solid;\n",
              "  color: var(--sklearn-color-unfitted-level-1);\n",
              "}\n",
              "\n",
              ".sk-estimator-doc-link.fitted,\n",
              "a:link.sk-estimator-doc-link.fitted,\n",
              "a:visited.sk-estimator-doc-link.fitted {\n",
              "  /* fitted */\n",
              "  border: var(--sklearn-color-fitted-level-1) 1pt solid;\n",
              "  color: var(--sklearn-color-fitted-level-1);\n",
              "}\n",
              "\n",
              "/* On hover */\n",
              "div.sk-estimator:hover .sk-estimator-doc-link:hover,\n",
              ".sk-estimator-doc-link:hover,\n",
              "div.sk-label-container:hover .sk-estimator-doc-link:hover,\n",
              ".sk-estimator-doc-link:hover {\n",
              "  /* unfitted */\n",
              "  background-color: var(--sklearn-color-unfitted-level-3);\n",
              "  color: var(--sklearn-color-background);\n",
              "  text-decoration: none;\n",
              "}\n",
              "\n",
              "div.sk-estimator.fitted:hover .sk-estimator-doc-link.fitted:hover,\n",
              ".sk-estimator-doc-link.fitted:hover,\n",
              "div.sk-label-container:hover .sk-estimator-doc-link.fitted:hover,\n",
              ".sk-estimator-doc-link.fitted:hover {\n",
              "  /* fitted */\n",
              "  background-color: var(--sklearn-color-fitted-level-3);\n",
              "  color: var(--sklearn-color-background);\n",
              "  text-decoration: none;\n",
              "}\n",
              "\n",
              "/* Span, style for the box shown on hovering the info icon */\n",
              ".sk-estimator-doc-link span {\n",
              "  display: none;\n",
              "  z-index: 9999;\n",
              "  position: relative;\n",
              "  font-weight: normal;\n",
              "  right: .2ex;\n",
              "  padding: .5ex;\n",
              "  margin: .5ex;\n",
              "  width: min-content;\n",
              "  min-width: 20ex;\n",
              "  max-width: 50ex;\n",
              "  color: var(--sklearn-color-text);\n",
              "  box-shadow: 2pt 2pt 4pt #999;\n",
              "  /* unfitted */\n",
              "  background: var(--sklearn-color-unfitted-level-0);\n",
              "  border: .5pt solid var(--sklearn-color-unfitted-level-3);\n",
              "}\n",
              "\n",
              ".sk-estimator-doc-link.fitted span {\n",
              "  /* fitted */\n",
              "  background: var(--sklearn-color-fitted-level-0);\n",
              "  border: var(--sklearn-color-fitted-level-3);\n",
              "}\n",
              "\n",
              ".sk-estimator-doc-link:hover span {\n",
              "  display: block;\n",
              "}\n",
              "\n",
              "/* \"?\"-specific style due to the `<a>` HTML tag */\n",
              "\n",
              "#sk-container-id-3 a.estimator_doc_link {\n",
              "  float: right;\n",
              "  font-size: 1rem;\n",
              "  line-height: 1em;\n",
              "  font-family: monospace;\n",
              "  background-color: var(--sklearn-color-background);\n",
              "  border-radius: 1rem;\n",
              "  height: 1rem;\n",
              "  width: 1rem;\n",
              "  text-decoration: none;\n",
              "  /* unfitted */\n",
              "  color: var(--sklearn-color-unfitted-level-1);\n",
              "  border: var(--sklearn-color-unfitted-level-1) 1pt solid;\n",
              "}\n",
              "\n",
              "#sk-container-id-3 a.estimator_doc_link.fitted {\n",
              "  /* fitted */\n",
              "  border: var(--sklearn-color-fitted-level-1) 1pt solid;\n",
              "  color: var(--sklearn-color-fitted-level-1);\n",
              "}\n",
              "\n",
              "/* On hover */\n",
              "#sk-container-id-3 a.estimator_doc_link:hover {\n",
              "  /* unfitted */\n",
              "  background-color: var(--sklearn-color-unfitted-level-3);\n",
              "  color: var(--sklearn-color-background);\n",
              "  text-decoration: none;\n",
              "}\n",
              "\n",
              "#sk-container-id-3 a.estimator_doc_link.fitted:hover {\n",
              "  /* fitted */\n",
              "  background-color: var(--sklearn-color-fitted-level-3);\n",
              "}\n",
              "</style><div id=\"sk-container-id-3\" class=\"sk-top-container\"><div class=\"sk-text-repr-fallback\"><pre>SGDClassifier(loss=&#x27;log_loss&#x27;, max_iter=100, random_state=0)</pre><b>In a Jupyter environment, please rerun this cell to show the HTML representation or trust the notebook. <br />On GitHub, the HTML representation is unable to render, please try loading this page with nbviewer.org.</b></div><div class=\"sk-container\" hidden><div class=\"sk-item\"><div class=\"sk-estimator fitted sk-toggleable\"><input class=\"sk-toggleable__control sk-hidden--visually\" id=\"sk-estimator-id-3\" type=\"checkbox\" checked><label for=\"sk-estimator-id-3\" class=\"sk-toggleable__label fitted sk-toggleable__label-arrow fitted\">&nbsp;&nbsp;SGDClassifier<a class=\"sk-estimator-doc-link fitted\" rel=\"noreferrer\" target=\"_blank\" href=\"https://scikit-learn.org/1.5/modules/generated/sklearn.linear_model.SGDClassifier.html\">?<span>Documentation for SGDClassifier</span></a><span class=\"sk-estimator-doc-link fitted\">i<span>Fitted</span></span></label><div class=\"sk-toggleable__content fitted\"><pre>SGDClassifier(loss=&#x27;log_loss&#x27;, max_iter=100, random_state=0)</pre></div> </div></div></div></div>"
            ]
          },
          "metadata": {},
          "execution_count": 24
        }
      ],
      "source": [
        "model = SGDClassifier(loss='log_loss', random_state=0, max_iter=100)\n",
        "model.fit(train_vectors, train_targets)"
      ]
    },
    {
      "cell_type": "markdown",
      "metadata": {
        "id": "0stTRuwEk_NW"
      },
      "source": [
        "We evaluate the classifier on the test vectors. It predicts the correct sentiment for sentiment words outside of its training data 89% of the time. Not bad."
      ]
    },
    {
      "cell_type": "code",
      "source": [
        "accuracy_score(model.predict(test_vectors), test_targets)"
      ],
      "metadata": {
        "colab": {
          "base_uri": "https://localhost:8080/"
        },
        "id": "_PNQhIGLfo-7",
        "outputId": "84056022-3da1-4924-d099-6aeacf8f0b9a"
      },
      "execution_count": null,
      "outputs": [
        {
          "output_type": "execute_result",
          "data": {
            "text/plain": [
              "0.8942307692307693"
            ]
          },
          "metadata": {},
          "execution_count": 25
        }
      ]
    },
    {
      "cell_type": "code",
      "source": [
        "model = SGDClassifier(loss='squared_error', random_state=0, max_iter=100)\n",
        "model.fit(train_vectors, train_targets)"
      ],
      "metadata": {
        "colab": {
          "base_uri": "https://localhost:8080/",
          "height": 136
        },
        "id": "pi_j0KlFfiNJ",
        "outputId": "53bbbbd2-d9f4-4215-a436-978dba4e559c"
      },
      "execution_count": null,
      "outputs": [
        {
          "output_type": "stream",
          "name": "stderr",
          "text": [
            "/usr/local/lib/python3.10/dist-packages/sklearn/linear_model/_stochastic_gradient.py:744: ConvergenceWarning: Maximum number of iteration reached before convergence. Consider increasing max_iter to improve the fit.\n",
            "  warnings.warn(\n"
          ]
        },
        {
          "output_type": "execute_result",
          "data": {
            "text/plain": [
              "SGDClassifier(loss='squared_error', max_iter=100, random_state=0)"
            ],
            "text/html": [
              "<style>#sk-container-id-4 {\n",
              "  /* Definition of color scheme common for light and dark mode */\n",
              "  --sklearn-color-text: black;\n",
              "  --sklearn-color-line: gray;\n",
              "  /* Definition of color scheme for unfitted estimators */\n",
              "  --sklearn-color-unfitted-level-0: #fff5e6;\n",
              "  --sklearn-color-unfitted-level-1: #f6e4d2;\n",
              "  --sklearn-color-unfitted-level-2: #ffe0b3;\n",
              "  --sklearn-color-unfitted-level-3: chocolate;\n",
              "  /* Definition of color scheme for fitted estimators */\n",
              "  --sklearn-color-fitted-level-0: #f0f8ff;\n",
              "  --sklearn-color-fitted-level-1: #d4ebff;\n",
              "  --sklearn-color-fitted-level-2: #b3dbfd;\n",
              "  --sklearn-color-fitted-level-3: cornflowerblue;\n",
              "\n",
              "  /* Specific color for light theme */\n",
              "  --sklearn-color-text-on-default-background: var(--sg-text-color, var(--theme-code-foreground, var(--jp-content-font-color1, black)));\n",
              "  --sklearn-color-background: var(--sg-background-color, var(--theme-background, var(--jp-layout-color0, white)));\n",
              "  --sklearn-color-border-box: var(--sg-text-color, var(--theme-code-foreground, var(--jp-content-font-color1, black)));\n",
              "  --sklearn-color-icon: #696969;\n",
              "\n",
              "  @media (prefers-color-scheme: dark) {\n",
              "    /* Redefinition of color scheme for dark theme */\n",
              "    --sklearn-color-text-on-default-background: var(--sg-text-color, var(--theme-code-foreground, var(--jp-content-font-color1, white)));\n",
              "    --sklearn-color-background: var(--sg-background-color, var(--theme-background, var(--jp-layout-color0, #111)));\n",
              "    --sklearn-color-border-box: var(--sg-text-color, var(--theme-code-foreground, var(--jp-content-font-color1, white)));\n",
              "    --sklearn-color-icon: #878787;\n",
              "  }\n",
              "}\n",
              "\n",
              "#sk-container-id-4 {\n",
              "  color: var(--sklearn-color-text);\n",
              "}\n",
              "\n",
              "#sk-container-id-4 pre {\n",
              "  padding: 0;\n",
              "}\n",
              "\n",
              "#sk-container-id-4 input.sk-hidden--visually {\n",
              "  border: 0;\n",
              "  clip: rect(1px 1px 1px 1px);\n",
              "  clip: rect(1px, 1px, 1px, 1px);\n",
              "  height: 1px;\n",
              "  margin: -1px;\n",
              "  overflow: hidden;\n",
              "  padding: 0;\n",
              "  position: absolute;\n",
              "  width: 1px;\n",
              "}\n",
              "\n",
              "#sk-container-id-4 div.sk-dashed-wrapped {\n",
              "  border: 1px dashed var(--sklearn-color-line);\n",
              "  margin: 0 0.4em 0.5em 0.4em;\n",
              "  box-sizing: border-box;\n",
              "  padding-bottom: 0.4em;\n",
              "  background-color: var(--sklearn-color-background);\n",
              "}\n",
              "\n",
              "#sk-container-id-4 div.sk-container {\n",
              "  /* jupyter's `normalize.less` sets `[hidden] { display: none; }`\n",
              "     but bootstrap.min.css set `[hidden] { display: none !important; }`\n",
              "     so we also need the `!important` here to be able to override the\n",
              "     default hidden behavior on the sphinx rendered scikit-learn.org.\n",
              "     See: https://github.com/scikit-learn/scikit-learn/issues/21755 */\n",
              "  display: inline-block !important;\n",
              "  position: relative;\n",
              "}\n",
              "\n",
              "#sk-container-id-4 div.sk-text-repr-fallback {\n",
              "  display: none;\n",
              "}\n",
              "\n",
              "div.sk-parallel-item,\n",
              "div.sk-serial,\n",
              "div.sk-item {\n",
              "  /* draw centered vertical line to link estimators */\n",
              "  background-image: linear-gradient(var(--sklearn-color-text-on-default-background), var(--sklearn-color-text-on-default-background));\n",
              "  background-size: 2px 100%;\n",
              "  background-repeat: no-repeat;\n",
              "  background-position: center center;\n",
              "}\n",
              "\n",
              "/* Parallel-specific style estimator block */\n",
              "\n",
              "#sk-container-id-4 div.sk-parallel-item::after {\n",
              "  content: \"\";\n",
              "  width: 100%;\n",
              "  border-bottom: 2px solid var(--sklearn-color-text-on-default-background);\n",
              "  flex-grow: 1;\n",
              "}\n",
              "\n",
              "#sk-container-id-4 div.sk-parallel {\n",
              "  display: flex;\n",
              "  align-items: stretch;\n",
              "  justify-content: center;\n",
              "  background-color: var(--sklearn-color-background);\n",
              "  position: relative;\n",
              "}\n",
              "\n",
              "#sk-container-id-4 div.sk-parallel-item {\n",
              "  display: flex;\n",
              "  flex-direction: column;\n",
              "}\n",
              "\n",
              "#sk-container-id-4 div.sk-parallel-item:first-child::after {\n",
              "  align-self: flex-end;\n",
              "  width: 50%;\n",
              "}\n",
              "\n",
              "#sk-container-id-4 div.sk-parallel-item:last-child::after {\n",
              "  align-self: flex-start;\n",
              "  width: 50%;\n",
              "}\n",
              "\n",
              "#sk-container-id-4 div.sk-parallel-item:only-child::after {\n",
              "  width: 0;\n",
              "}\n",
              "\n",
              "/* Serial-specific style estimator block */\n",
              "\n",
              "#sk-container-id-4 div.sk-serial {\n",
              "  display: flex;\n",
              "  flex-direction: column;\n",
              "  align-items: center;\n",
              "  background-color: var(--sklearn-color-background);\n",
              "  padding-right: 1em;\n",
              "  padding-left: 1em;\n",
              "}\n",
              "\n",
              "\n",
              "/* Toggleable style: style used for estimator/Pipeline/ColumnTransformer box that is\n",
              "clickable and can be expanded/collapsed.\n",
              "- Pipeline and ColumnTransformer use this feature and define the default style\n",
              "- Estimators will overwrite some part of the style using the `sk-estimator` class\n",
              "*/\n",
              "\n",
              "/* Pipeline and ColumnTransformer style (default) */\n",
              "\n",
              "#sk-container-id-4 div.sk-toggleable {\n",
              "  /* Default theme specific background. It is overwritten whether we have a\n",
              "  specific estimator or a Pipeline/ColumnTransformer */\n",
              "  background-color: var(--sklearn-color-background);\n",
              "}\n",
              "\n",
              "/* Toggleable label */\n",
              "#sk-container-id-4 label.sk-toggleable__label {\n",
              "  cursor: pointer;\n",
              "  display: block;\n",
              "  width: 100%;\n",
              "  margin-bottom: 0;\n",
              "  padding: 0.5em;\n",
              "  box-sizing: border-box;\n",
              "  text-align: center;\n",
              "}\n",
              "\n",
              "#sk-container-id-4 label.sk-toggleable__label-arrow:before {\n",
              "  /* Arrow on the left of the label */\n",
              "  content: \"▸\";\n",
              "  float: left;\n",
              "  margin-right: 0.25em;\n",
              "  color: var(--sklearn-color-icon);\n",
              "}\n",
              "\n",
              "#sk-container-id-4 label.sk-toggleable__label-arrow:hover:before {\n",
              "  color: var(--sklearn-color-text);\n",
              "}\n",
              "\n",
              "/* Toggleable content - dropdown */\n",
              "\n",
              "#sk-container-id-4 div.sk-toggleable__content {\n",
              "  max-height: 0;\n",
              "  max-width: 0;\n",
              "  overflow: hidden;\n",
              "  text-align: left;\n",
              "  /* unfitted */\n",
              "  background-color: var(--sklearn-color-unfitted-level-0);\n",
              "}\n",
              "\n",
              "#sk-container-id-4 div.sk-toggleable__content.fitted {\n",
              "  /* fitted */\n",
              "  background-color: var(--sklearn-color-fitted-level-0);\n",
              "}\n",
              "\n",
              "#sk-container-id-4 div.sk-toggleable__content pre {\n",
              "  margin: 0.2em;\n",
              "  border-radius: 0.25em;\n",
              "  color: var(--sklearn-color-text);\n",
              "  /* unfitted */\n",
              "  background-color: var(--sklearn-color-unfitted-level-0);\n",
              "}\n",
              "\n",
              "#sk-container-id-4 div.sk-toggleable__content.fitted pre {\n",
              "  /* unfitted */\n",
              "  background-color: var(--sklearn-color-fitted-level-0);\n",
              "}\n",
              "\n",
              "#sk-container-id-4 input.sk-toggleable__control:checked~div.sk-toggleable__content {\n",
              "  /* Expand drop-down */\n",
              "  max-height: 200px;\n",
              "  max-width: 100%;\n",
              "  overflow: auto;\n",
              "}\n",
              "\n",
              "#sk-container-id-4 input.sk-toggleable__control:checked~label.sk-toggleable__label-arrow:before {\n",
              "  content: \"▾\";\n",
              "}\n",
              "\n",
              "/* Pipeline/ColumnTransformer-specific style */\n",
              "\n",
              "#sk-container-id-4 div.sk-label input.sk-toggleable__control:checked~label.sk-toggleable__label {\n",
              "  color: var(--sklearn-color-text);\n",
              "  background-color: var(--sklearn-color-unfitted-level-2);\n",
              "}\n",
              "\n",
              "#sk-container-id-4 div.sk-label.fitted input.sk-toggleable__control:checked~label.sk-toggleable__label {\n",
              "  background-color: var(--sklearn-color-fitted-level-2);\n",
              "}\n",
              "\n",
              "/* Estimator-specific style */\n",
              "\n",
              "/* Colorize estimator box */\n",
              "#sk-container-id-4 div.sk-estimator input.sk-toggleable__control:checked~label.sk-toggleable__label {\n",
              "  /* unfitted */\n",
              "  background-color: var(--sklearn-color-unfitted-level-2);\n",
              "}\n",
              "\n",
              "#sk-container-id-4 div.sk-estimator.fitted input.sk-toggleable__control:checked~label.sk-toggleable__label {\n",
              "  /* fitted */\n",
              "  background-color: var(--sklearn-color-fitted-level-2);\n",
              "}\n",
              "\n",
              "#sk-container-id-4 div.sk-label label.sk-toggleable__label,\n",
              "#sk-container-id-4 div.sk-label label {\n",
              "  /* The background is the default theme color */\n",
              "  color: var(--sklearn-color-text-on-default-background);\n",
              "}\n",
              "\n",
              "/* On hover, darken the color of the background */\n",
              "#sk-container-id-4 div.sk-label:hover label.sk-toggleable__label {\n",
              "  color: var(--sklearn-color-text);\n",
              "  background-color: var(--sklearn-color-unfitted-level-2);\n",
              "}\n",
              "\n",
              "/* Label box, darken color on hover, fitted */\n",
              "#sk-container-id-4 div.sk-label.fitted:hover label.sk-toggleable__label.fitted {\n",
              "  color: var(--sklearn-color-text);\n",
              "  background-color: var(--sklearn-color-fitted-level-2);\n",
              "}\n",
              "\n",
              "/* Estimator label */\n",
              "\n",
              "#sk-container-id-4 div.sk-label label {\n",
              "  font-family: monospace;\n",
              "  font-weight: bold;\n",
              "  display: inline-block;\n",
              "  line-height: 1.2em;\n",
              "}\n",
              "\n",
              "#sk-container-id-4 div.sk-label-container {\n",
              "  text-align: center;\n",
              "}\n",
              "\n",
              "/* Estimator-specific */\n",
              "#sk-container-id-4 div.sk-estimator {\n",
              "  font-family: monospace;\n",
              "  border: 1px dotted var(--sklearn-color-border-box);\n",
              "  border-radius: 0.25em;\n",
              "  box-sizing: border-box;\n",
              "  margin-bottom: 0.5em;\n",
              "  /* unfitted */\n",
              "  background-color: var(--sklearn-color-unfitted-level-0);\n",
              "}\n",
              "\n",
              "#sk-container-id-4 div.sk-estimator.fitted {\n",
              "  /* fitted */\n",
              "  background-color: var(--sklearn-color-fitted-level-0);\n",
              "}\n",
              "\n",
              "/* on hover */\n",
              "#sk-container-id-4 div.sk-estimator:hover {\n",
              "  /* unfitted */\n",
              "  background-color: var(--sklearn-color-unfitted-level-2);\n",
              "}\n",
              "\n",
              "#sk-container-id-4 div.sk-estimator.fitted:hover {\n",
              "  /* fitted */\n",
              "  background-color: var(--sklearn-color-fitted-level-2);\n",
              "}\n",
              "\n",
              "/* Specification for estimator info (e.g. \"i\" and \"?\") */\n",
              "\n",
              "/* Common style for \"i\" and \"?\" */\n",
              "\n",
              ".sk-estimator-doc-link,\n",
              "a:link.sk-estimator-doc-link,\n",
              "a:visited.sk-estimator-doc-link {\n",
              "  float: right;\n",
              "  font-size: smaller;\n",
              "  line-height: 1em;\n",
              "  font-family: monospace;\n",
              "  background-color: var(--sklearn-color-background);\n",
              "  border-radius: 1em;\n",
              "  height: 1em;\n",
              "  width: 1em;\n",
              "  text-decoration: none !important;\n",
              "  margin-left: 1ex;\n",
              "  /* unfitted */\n",
              "  border: var(--sklearn-color-unfitted-level-1) 1pt solid;\n",
              "  color: var(--sklearn-color-unfitted-level-1);\n",
              "}\n",
              "\n",
              ".sk-estimator-doc-link.fitted,\n",
              "a:link.sk-estimator-doc-link.fitted,\n",
              "a:visited.sk-estimator-doc-link.fitted {\n",
              "  /* fitted */\n",
              "  border: var(--sklearn-color-fitted-level-1) 1pt solid;\n",
              "  color: var(--sklearn-color-fitted-level-1);\n",
              "}\n",
              "\n",
              "/* On hover */\n",
              "div.sk-estimator:hover .sk-estimator-doc-link:hover,\n",
              ".sk-estimator-doc-link:hover,\n",
              "div.sk-label-container:hover .sk-estimator-doc-link:hover,\n",
              ".sk-estimator-doc-link:hover {\n",
              "  /* unfitted */\n",
              "  background-color: var(--sklearn-color-unfitted-level-3);\n",
              "  color: var(--sklearn-color-background);\n",
              "  text-decoration: none;\n",
              "}\n",
              "\n",
              "div.sk-estimator.fitted:hover .sk-estimator-doc-link.fitted:hover,\n",
              ".sk-estimator-doc-link.fitted:hover,\n",
              "div.sk-label-container:hover .sk-estimator-doc-link.fitted:hover,\n",
              ".sk-estimator-doc-link.fitted:hover {\n",
              "  /* fitted */\n",
              "  background-color: var(--sklearn-color-fitted-level-3);\n",
              "  color: var(--sklearn-color-background);\n",
              "  text-decoration: none;\n",
              "}\n",
              "\n",
              "/* Span, style for the box shown on hovering the info icon */\n",
              ".sk-estimator-doc-link span {\n",
              "  display: none;\n",
              "  z-index: 9999;\n",
              "  position: relative;\n",
              "  font-weight: normal;\n",
              "  right: .2ex;\n",
              "  padding: .5ex;\n",
              "  margin: .5ex;\n",
              "  width: min-content;\n",
              "  min-width: 20ex;\n",
              "  max-width: 50ex;\n",
              "  color: var(--sklearn-color-text);\n",
              "  box-shadow: 2pt 2pt 4pt #999;\n",
              "  /* unfitted */\n",
              "  background: var(--sklearn-color-unfitted-level-0);\n",
              "  border: .5pt solid var(--sklearn-color-unfitted-level-3);\n",
              "}\n",
              "\n",
              ".sk-estimator-doc-link.fitted span {\n",
              "  /* fitted */\n",
              "  background: var(--sklearn-color-fitted-level-0);\n",
              "  border: var(--sklearn-color-fitted-level-3);\n",
              "}\n",
              "\n",
              ".sk-estimator-doc-link:hover span {\n",
              "  display: block;\n",
              "}\n",
              "\n",
              "/* \"?\"-specific style due to the `<a>` HTML tag */\n",
              "\n",
              "#sk-container-id-4 a.estimator_doc_link {\n",
              "  float: right;\n",
              "  font-size: 1rem;\n",
              "  line-height: 1em;\n",
              "  font-family: monospace;\n",
              "  background-color: var(--sklearn-color-background);\n",
              "  border-radius: 1rem;\n",
              "  height: 1rem;\n",
              "  width: 1rem;\n",
              "  text-decoration: none;\n",
              "  /* unfitted */\n",
              "  color: var(--sklearn-color-unfitted-level-1);\n",
              "  border: var(--sklearn-color-unfitted-level-1) 1pt solid;\n",
              "}\n",
              "\n",
              "#sk-container-id-4 a.estimator_doc_link.fitted {\n",
              "  /* fitted */\n",
              "  border: var(--sklearn-color-fitted-level-1) 1pt solid;\n",
              "  color: var(--sklearn-color-fitted-level-1);\n",
              "}\n",
              "\n",
              "/* On hover */\n",
              "#sk-container-id-4 a.estimator_doc_link:hover {\n",
              "  /* unfitted */\n",
              "  background-color: var(--sklearn-color-unfitted-level-3);\n",
              "  color: var(--sklearn-color-background);\n",
              "  text-decoration: none;\n",
              "}\n",
              "\n",
              "#sk-container-id-4 a.estimator_doc_link.fitted:hover {\n",
              "  /* fitted */\n",
              "  background-color: var(--sklearn-color-fitted-level-3);\n",
              "}\n",
              "</style><div id=\"sk-container-id-4\" class=\"sk-top-container\"><div class=\"sk-text-repr-fallback\"><pre>SGDClassifier(loss=&#x27;squared_error&#x27;, max_iter=100, random_state=0)</pre><b>In a Jupyter environment, please rerun this cell to show the HTML representation or trust the notebook. <br />On GitHub, the HTML representation is unable to render, please try loading this page with nbviewer.org.</b></div><div class=\"sk-container\" hidden><div class=\"sk-item\"><div class=\"sk-estimator fitted sk-toggleable\"><input class=\"sk-toggleable__control sk-hidden--visually\" id=\"sk-estimator-id-4\" type=\"checkbox\" checked><label for=\"sk-estimator-id-4\" class=\"sk-toggleable__label fitted sk-toggleable__label-arrow fitted\">&nbsp;&nbsp;SGDClassifier<a class=\"sk-estimator-doc-link fitted\" rel=\"noreferrer\" target=\"_blank\" href=\"https://scikit-learn.org/1.5/modules/generated/sklearn.linear_model.SGDClassifier.html\">?<span>Documentation for SGDClassifier</span></a><span class=\"sk-estimator-doc-link fitted\">i<span>Fitted</span></span></label><div class=\"sk-toggleable__content fitted\"><pre>SGDClassifier(loss=&#x27;squared_error&#x27;, max_iter=100, random_state=0)</pre></div> </div></div></div></div>"
            ]
          },
          "metadata": {},
          "execution_count": 26
        }
      ]
    },
    {
      "cell_type": "code",
      "execution_count": null,
      "metadata": {
        "id": "BNd1jZl4k_NW",
        "colab": {
          "base_uri": "https://localhost:8080/"
        },
        "outputId": "7756cfec-ccd6-437f-a18b-99d55793dcfc"
      },
      "outputs": [
        {
          "output_type": "execute_result",
          "data": {
            "text/plain": [
              "0.875"
            ]
          },
          "metadata": {},
          "execution_count": 27
        }
      ],
      "source": [
        "accuracy_score(model.predict(test_vectors), test_targets)"
      ]
    },
    {
      "cell_type": "markdown",
      "metadata": {
        "id": "1uOcfqO9k_NW"
      },
      "source": [
        "### TODO: Compute Sentiments\n",
        "\n",
        "Let's define a function that we can use to see the sentiment that this classifier predicts for particular words, then use it to see some examples of its predictions on the test data."
      ]
    },
    {
      "cell_type": "code",
      "source": [
        "model = SGDClassifier(loss='log_loss', random_state=0, max_iter=100)\n",
        "model.fit(train_vectors, train_targets)"
      ],
      "metadata": {
        "colab": {
          "base_uri": "https://localhost:8080/",
          "height": 80
        },
        "id": "-ARwWNDlhvGY",
        "outputId": "2544661c-d6d3-4914-847c-91e1af151671"
      },
      "execution_count": null,
      "outputs": [
        {
          "output_type": "execute_result",
          "data": {
            "text/plain": [
              "SGDClassifier(loss='log_loss', max_iter=100, random_state=0)"
            ],
            "text/html": [
              "<style>#sk-container-id-5 {\n",
              "  /* Definition of color scheme common for light and dark mode */\n",
              "  --sklearn-color-text: black;\n",
              "  --sklearn-color-line: gray;\n",
              "  /* Definition of color scheme for unfitted estimators */\n",
              "  --sklearn-color-unfitted-level-0: #fff5e6;\n",
              "  --sklearn-color-unfitted-level-1: #f6e4d2;\n",
              "  --sklearn-color-unfitted-level-2: #ffe0b3;\n",
              "  --sklearn-color-unfitted-level-3: chocolate;\n",
              "  /* Definition of color scheme for fitted estimators */\n",
              "  --sklearn-color-fitted-level-0: #f0f8ff;\n",
              "  --sklearn-color-fitted-level-1: #d4ebff;\n",
              "  --sklearn-color-fitted-level-2: #b3dbfd;\n",
              "  --sklearn-color-fitted-level-3: cornflowerblue;\n",
              "\n",
              "  /* Specific color for light theme */\n",
              "  --sklearn-color-text-on-default-background: var(--sg-text-color, var(--theme-code-foreground, var(--jp-content-font-color1, black)));\n",
              "  --sklearn-color-background: var(--sg-background-color, var(--theme-background, var(--jp-layout-color0, white)));\n",
              "  --sklearn-color-border-box: var(--sg-text-color, var(--theme-code-foreground, var(--jp-content-font-color1, black)));\n",
              "  --sklearn-color-icon: #696969;\n",
              "\n",
              "  @media (prefers-color-scheme: dark) {\n",
              "    /* Redefinition of color scheme for dark theme */\n",
              "    --sklearn-color-text-on-default-background: var(--sg-text-color, var(--theme-code-foreground, var(--jp-content-font-color1, white)));\n",
              "    --sklearn-color-background: var(--sg-background-color, var(--theme-background, var(--jp-layout-color0, #111)));\n",
              "    --sklearn-color-border-box: var(--sg-text-color, var(--theme-code-foreground, var(--jp-content-font-color1, white)));\n",
              "    --sklearn-color-icon: #878787;\n",
              "  }\n",
              "}\n",
              "\n",
              "#sk-container-id-5 {\n",
              "  color: var(--sklearn-color-text);\n",
              "}\n",
              "\n",
              "#sk-container-id-5 pre {\n",
              "  padding: 0;\n",
              "}\n",
              "\n",
              "#sk-container-id-5 input.sk-hidden--visually {\n",
              "  border: 0;\n",
              "  clip: rect(1px 1px 1px 1px);\n",
              "  clip: rect(1px, 1px, 1px, 1px);\n",
              "  height: 1px;\n",
              "  margin: -1px;\n",
              "  overflow: hidden;\n",
              "  padding: 0;\n",
              "  position: absolute;\n",
              "  width: 1px;\n",
              "}\n",
              "\n",
              "#sk-container-id-5 div.sk-dashed-wrapped {\n",
              "  border: 1px dashed var(--sklearn-color-line);\n",
              "  margin: 0 0.4em 0.5em 0.4em;\n",
              "  box-sizing: border-box;\n",
              "  padding-bottom: 0.4em;\n",
              "  background-color: var(--sklearn-color-background);\n",
              "}\n",
              "\n",
              "#sk-container-id-5 div.sk-container {\n",
              "  /* jupyter's `normalize.less` sets `[hidden] { display: none; }`\n",
              "     but bootstrap.min.css set `[hidden] { display: none !important; }`\n",
              "     so we also need the `!important` here to be able to override the\n",
              "     default hidden behavior on the sphinx rendered scikit-learn.org.\n",
              "     See: https://github.com/scikit-learn/scikit-learn/issues/21755 */\n",
              "  display: inline-block !important;\n",
              "  position: relative;\n",
              "}\n",
              "\n",
              "#sk-container-id-5 div.sk-text-repr-fallback {\n",
              "  display: none;\n",
              "}\n",
              "\n",
              "div.sk-parallel-item,\n",
              "div.sk-serial,\n",
              "div.sk-item {\n",
              "  /* draw centered vertical line to link estimators */\n",
              "  background-image: linear-gradient(var(--sklearn-color-text-on-default-background), var(--sklearn-color-text-on-default-background));\n",
              "  background-size: 2px 100%;\n",
              "  background-repeat: no-repeat;\n",
              "  background-position: center center;\n",
              "}\n",
              "\n",
              "/* Parallel-specific style estimator block */\n",
              "\n",
              "#sk-container-id-5 div.sk-parallel-item::after {\n",
              "  content: \"\";\n",
              "  width: 100%;\n",
              "  border-bottom: 2px solid var(--sklearn-color-text-on-default-background);\n",
              "  flex-grow: 1;\n",
              "}\n",
              "\n",
              "#sk-container-id-5 div.sk-parallel {\n",
              "  display: flex;\n",
              "  align-items: stretch;\n",
              "  justify-content: center;\n",
              "  background-color: var(--sklearn-color-background);\n",
              "  position: relative;\n",
              "}\n",
              "\n",
              "#sk-container-id-5 div.sk-parallel-item {\n",
              "  display: flex;\n",
              "  flex-direction: column;\n",
              "}\n",
              "\n",
              "#sk-container-id-5 div.sk-parallel-item:first-child::after {\n",
              "  align-self: flex-end;\n",
              "  width: 50%;\n",
              "}\n",
              "\n",
              "#sk-container-id-5 div.sk-parallel-item:last-child::after {\n",
              "  align-self: flex-start;\n",
              "  width: 50%;\n",
              "}\n",
              "\n",
              "#sk-container-id-5 div.sk-parallel-item:only-child::after {\n",
              "  width: 0;\n",
              "}\n",
              "\n",
              "/* Serial-specific style estimator block */\n",
              "\n",
              "#sk-container-id-5 div.sk-serial {\n",
              "  display: flex;\n",
              "  flex-direction: column;\n",
              "  align-items: center;\n",
              "  background-color: var(--sklearn-color-background);\n",
              "  padding-right: 1em;\n",
              "  padding-left: 1em;\n",
              "}\n",
              "\n",
              "\n",
              "/* Toggleable style: style used for estimator/Pipeline/ColumnTransformer box that is\n",
              "clickable and can be expanded/collapsed.\n",
              "- Pipeline and ColumnTransformer use this feature and define the default style\n",
              "- Estimators will overwrite some part of the style using the `sk-estimator` class\n",
              "*/\n",
              "\n",
              "/* Pipeline and ColumnTransformer style (default) */\n",
              "\n",
              "#sk-container-id-5 div.sk-toggleable {\n",
              "  /* Default theme specific background. It is overwritten whether we have a\n",
              "  specific estimator or a Pipeline/ColumnTransformer */\n",
              "  background-color: var(--sklearn-color-background);\n",
              "}\n",
              "\n",
              "/* Toggleable label */\n",
              "#sk-container-id-5 label.sk-toggleable__label {\n",
              "  cursor: pointer;\n",
              "  display: block;\n",
              "  width: 100%;\n",
              "  margin-bottom: 0;\n",
              "  padding: 0.5em;\n",
              "  box-sizing: border-box;\n",
              "  text-align: center;\n",
              "}\n",
              "\n",
              "#sk-container-id-5 label.sk-toggleable__label-arrow:before {\n",
              "  /* Arrow on the left of the label */\n",
              "  content: \"▸\";\n",
              "  float: left;\n",
              "  margin-right: 0.25em;\n",
              "  color: var(--sklearn-color-icon);\n",
              "}\n",
              "\n",
              "#sk-container-id-5 label.sk-toggleable__label-arrow:hover:before {\n",
              "  color: var(--sklearn-color-text);\n",
              "}\n",
              "\n",
              "/* Toggleable content - dropdown */\n",
              "\n",
              "#sk-container-id-5 div.sk-toggleable__content {\n",
              "  max-height: 0;\n",
              "  max-width: 0;\n",
              "  overflow: hidden;\n",
              "  text-align: left;\n",
              "  /* unfitted */\n",
              "  background-color: var(--sklearn-color-unfitted-level-0);\n",
              "}\n",
              "\n",
              "#sk-container-id-5 div.sk-toggleable__content.fitted {\n",
              "  /* fitted */\n",
              "  background-color: var(--sklearn-color-fitted-level-0);\n",
              "}\n",
              "\n",
              "#sk-container-id-5 div.sk-toggleable__content pre {\n",
              "  margin: 0.2em;\n",
              "  border-radius: 0.25em;\n",
              "  color: var(--sklearn-color-text);\n",
              "  /* unfitted */\n",
              "  background-color: var(--sklearn-color-unfitted-level-0);\n",
              "}\n",
              "\n",
              "#sk-container-id-5 div.sk-toggleable__content.fitted pre {\n",
              "  /* unfitted */\n",
              "  background-color: var(--sklearn-color-fitted-level-0);\n",
              "}\n",
              "\n",
              "#sk-container-id-5 input.sk-toggleable__control:checked~div.sk-toggleable__content {\n",
              "  /* Expand drop-down */\n",
              "  max-height: 200px;\n",
              "  max-width: 100%;\n",
              "  overflow: auto;\n",
              "}\n",
              "\n",
              "#sk-container-id-5 input.sk-toggleable__control:checked~label.sk-toggleable__label-arrow:before {\n",
              "  content: \"▾\";\n",
              "}\n",
              "\n",
              "/* Pipeline/ColumnTransformer-specific style */\n",
              "\n",
              "#sk-container-id-5 div.sk-label input.sk-toggleable__control:checked~label.sk-toggleable__label {\n",
              "  color: var(--sklearn-color-text);\n",
              "  background-color: var(--sklearn-color-unfitted-level-2);\n",
              "}\n",
              "\n",
              "#sk-container-id-5 div.sk-label.fitted input.sk-toggleable__control:checked~label.sk-toggleable__label {\n",
              "  background-color: var(--sklearn-color-fitted-level-2);\n",
              "}\n",
              "\n",
              "/* Estimator-specific style */\n",
              "\n",
              "/* Colorize estimator box */\n",
              "#sk-container-id-5 div.sk-estimator input.sk-toggleable__control:checked~label.sk-toggleable__label {\n",
              "  /* unfitted */\n",
              "  background-color: var(--sklearn-color-unfitted-level-2);\n",
              "}\n",
              "\n",
              "#sk-container-id-5 div.sk-estimator.fitted input.sk-toggleable__control:checked~label.sk-toggleable__label {\n",
              "  /* fitted */\n",
              "  background-color: var(--sklearn-color-fitted-level-2);\n",
              "}\n",
              "\n",
              "#sk-container-id-5 div.sk-label label.sk-toggleable__label,\n",
              "#sk-container-id-5 div.sk-label label {\n",
              "  /* The background is the default theme color */\n",
              "  color: var(--sklearn-color-text-on-default-background);\n",
              "}\n",
              "\n",
              "/* On hover, darken the color of the background */\n",
              "#sk-container-id-5 div.sk-label:hover label.sk-toggleable__label {\n",
              "  color: var(--sklearn-color-text);\n",
              "  background-color: var(--sklearn-color-unfitted-level-2);\n",
              "}\n",
              "\n",
              "/* Label box, darken color on hover, fitted */\n",
              "#sk-container-id-5 div.sk-label.fitted:hover label.sk-toggleable__label.fitted {\n",
              "  color: var(--sklearn-color-text);\n",
              "  background-color: var(--sklearn-color-fitted-level-2);\n",
              "}\n",
              "\n",
              "/* Estimator label */\n",
              "\n",
              "#sk-container-id-5 div.sk-label label {\n",
              "  font-family: monospace;\n",
              "  font-weight: bold;\n",
              "  display: inline-block;\n",
              "  line-height: 1.2em;\n",
              "}\n",
              "\n",
              "#sk-container-id-5 div.sk-label-container {\n",
              "  text-align: center;\n",
              "}\n",
              "\n",
              "/* Estimator-specific */\n",
              "#sk-container-id-5 div.sk-estimator {\n",
              "  font-family: monospace;\n",
              "  border: 1px dotted var(--sklearn-color-border-box);\n",
              "  border-radius: 0.25em;\n",
              "  box-sizing: border-box;\n",
              "  margin-bottom: 0.5em;\n",
              "  /* unfitted */\n",
              "  background-color: var(--sklearn-color-unfitted-level-0);\n",
              "}\n",
              "\n",
              "#sk-container-id-5 div.sk-estimator.fitted {\n",
              "  /* fitted */\n",
              "  background-color: var(--sklearn-color-fitted-level-0);\n",
              "}\n",
              "\n",
              "/* on hover */\n",
              "#sk-container-id-5 div.sk-estimator:hover {\n",
              "  /* unfitted */\n",
              "  background-color: var(--sklearn-color-unfitted-level-2);\n",
              "}\n",
              "\n",
              "#sk-container-id-5 div.sk-estimator.fitted:hover {\n",
              "  /* fitted */\n",
              "  background-color: var(--sklearn-color-fitted-level-2);\n",
              "}\n",
              "\n",
              "/* Specification for estimator info (e.g. \"i\" and \"?\") */\n",
              "\n",
              "/* Common style for \"i\" and \"?\" */\n",
              "\n",
              ".sk-estimator-doc-link,\n",
              "a:link.sk-estimator-doc-link,\n",
              "a:visited.sk-estimator-doc-link {\n",
              "  float: right;\n",
              "  font-size: smaller;\n",
              "  line-height: 1em;\n",
              "  font-family: monospace;\n",
              "  background-color: var(--sklearn-color-background);\n",
              "  border-radius: 1em;\n",
              "  height: 1em;\n",
              "  width: 1em;\n",
              "  text-decoration: none !important;\n",
              "  margin-left: 1ex;\n",
              "  /* unfitted */\n",
              "  border: var(--sklearn-color-unfitted-level-1) 1pt solid;\n",
              "  color: var(--sklearn-color-unfitted-level-1);\n",
              "}\n",
              "\n",
              ".sk-estimator-doc-link.fitted,\n",
              "a:link.sk-estimator-doc-link.fitted,\n",
              "a:visited.sk-estimator-doc-link.fitted {\n",
              "  /* fitted */\n",
              "  border: var(--sklearn-color-fitted-level-1) 1pt solid;\n",
              "  color: var(--sklearn-color-fitted-level-1);\n",
              "}\n",
              "\n",
              "/* On hover */\n",
              "div.sk-estimator:hover .sk-estimator-doc-link:hover,\n",
              ".sk-estimator-doc-link:hover,\n",
              "div.sk-label-container:hover .sk-estimator-doc-link:hover,\n",
              ".sk-estimator-doc-link:hover {\n",
              "  /* unfitted */\n",
              "  background-color: var(--sklearn-color-unfitted-level-3);\n",
              "  color: var(--sklearn-color-background);\n",
              "  text-decoration: none;\n",
              "}\n",
              "\n",
              "div.sk-estimator.fitted:hover .sk-estimator-doc-link.fitted:hover,\n",
              ".sk-estimator-doc-link.fitted:hover,\n",
              "div.sk-label-container:hover .sk-estimator-doc-link.fitted:hover,\n",
              ".sk-estimator-doc-link.fitted:hover {\n",
              "  /* fitted */\n",
              "  background-color: var(--sklearn-color-fitted-level-3);\n",
              "  color: var(--sklearn-color-background);\n",
              "  text-decoration: none;\n",
              "}\n",
              "\n",
              "/* Span, style for the box shown on hovering the info icon */\n",
              ".sk-estimator-doc-link span {\n",
              "  display: none;\n",
              "  z-index: 9999;\n",
              "  position: relative;\n",
              "  font-weight: normal;\n",
              "  right: .2ex;\n",
              "  padding: .5ex;\n",
              "  margin: .5ex;\n",
              "  width: min-content;\n",
              "  min-width: 20ex;\n",
              "  max-width: 50ex;\n",
              "  color: var(--sklearn-color-text);\n",
              "  box-shadow: 2pt 2pt 4pt #999;\n",
              "  /* unfitted */\n",
              "  background: var(--sklearn-color-unfitted-level-0);\n",
              "  border: .5pt solid var(--sklearn-color-unfitted-level-3);\n",
              "}\n",
              "\n",
              ".sk-estimator-doc-link.fitted span {\n",
              "  /* fitted */\n",
              "  background: var(--sklearn-color-fitted-level-0);\n",
              "  border: var(--sklearn-color-fitted-level-3);\n",
              "}\n",
              "\n",
              ".sk-estimator-doc-link:hover span {\n",
              "  display: block;\n",
              "}\n",
              "\n",
              "/* \"?\"-specific style due to the `<a>` HTML tag */\n",
              "\n",
              "#sk-container-id-5 a.estimator_doc_link {\n",
              "  float: right;\n",
              "  font-size: 1rem;\n",
              "  line-height: 1em;\n",
              "  font-family: monospace;\n",
              "  background-color: var(--sklearn-color-background);\n",
              "  border-radius: 1rem;\n",
              "  height: 1rem;\n",
              "  width: 1rem;\n",
              "  text-decoration: none;\n",
              "  /* unfitted */\n",
              "  color: var(--sklearn-color-unfitted-level-1);\n",
              "  border: var(--sklearn-color-unfitted-level-1) 1pt solid;\n",
              "}\n",
              "\n",
              "#sk-container-id-5 a.estimator_doc_link.fitted {\n",
              "  /* fitted */\n",
              "  border: var(--sklearn-color-fitted-level-1) 1pt solid;\n",
              "  color: var(--sklearn-color-fitted-level-1);\n",
              "}\n",
              "\n",
              "/* On hover */\n",
              "#sk-container-id-5 a.estimator_doc_link:hover {\n",
              "  /* unfitted */\n",
              "  background-color: var(--sklearn-color-unfitted-level-3);\n",
              "  color: var(--sklearn-color-background);\n",
              "  text-decoration: none;\n",
              "}\n",
              "\n",
              "#sk-container-id-5 a.estimator_doc_link.fitted:hover {\n",
              "  /* fitted */\n",
              "  background-color: var(--sklearn-color-fitted-level-3);\n",
              "}\n",
              "</style><div id=\"sk-container-id-5\" class=\"sk-top-container\"><div class=\"sk-text-repr-fallback\"><pre>SGDClassifier(loss=&#x27;log_loss&#x27;, max_iter=100, random_state=0)</pre><b>In a Jupyter environment, please rerun this cell to show the HTML representation or trust the notebook. <br />On GitHub, the HTML representation is unable to render, please try loading this page with nbviewer.org.</b></div><div class=\"sk-container\" hidden><div class=\"sk-item\"><div class=\"sk-estimator fitted sk-toggleable\"><input class=\"sk-toggleable__control sk-hidden--visually\" id=\"sk-estimator-id-5\" type=\"checkbox\" checked><label for=\"sk-estimator-id-5\" class=\"sk-toggleable__label fitted sk-toggleable__label-arrow fitted\">&nbsp;&nbsp;SGDClassifier<a class=\"sk-estimator-doc-link fitted\" rel=\"noreferrer\" target=\"_blank\" href=\"https://scikit-learn.org/1.5/modules/generated/sklearn.linear_model.SGDClassifier.html\">?<span>Documentation for SGDClassifier</span></a><span class=\"sk-estimator-doc-link fitted\">i<span>Fitted</span></span></label><div class=\"sk-toggleable__content fitted\"><pre>SGDClassifier(loss=&#x27;log_loss&#x27;, max_iter=100, random_state=0)</pre></div> </div></div></div></div>"
            ]
          },
          "metadata": {},
          "execution_count": 32
        }
      ]
    },
    {
      "cell_type": "code",
      "execution_count": null,
      "metadata": {
        "id": "8tsLp0Nuk_NX",
        "colab": {
          "base_uri": "https://localhost:8080/",
          "height": 712
        },
        "outputId": "182c54f5-3481-4c9f-c219-3e07d281e16f"
      },
      "outputs": [
        {
          "output_type": "stream",
          "name": "stdout",
          "text": [
            "[-3.8643893  -0.02119884]\n",
            "[-1.68977889e-04 -8.68582717e+00]\n"
          ]
        },
        {
          "output_type": "execute_result",
          "data": {
            "text/plain": [
              "              sentiment\n",
              "well           3.843190\n",
              "killed        -8.685658\n",
              "great          5.661081\n",
              "free           3.185349\n",
              "better         5.268084\n",
              "fell          -0.493469\n",
              "victory        4.352416\n",
              "helped         2.650431\n",
              "intelligence  -0.446912\n",
              "concerns      -2.409386\n",
              "limited        4.064308\n",
              "premier        1.129882\n",
              "winner         5.691388\n",
              "denied        -3.400399\n",
              "criminal      -7.329353\n",
              "cancer         1.358638\n",
              "trust          3.791185\n",
              "criticism     -2.821326\n",
              "helping        2.836155\n",
              "trouble       -2.599560"
            ],
            "text/html": [
              "\n",
              "  <div id=\"df-2b8fd40b-902e-4296-b5c9-35748e15ff18\" class=\"colab-df-container\">\n",
              "    <div>\n",
              "<style scoped>\n",
              "    .dataframe tbody tr th:only-of-type {\n",
              "        vertical-align: middle;\n",
              "    }\n",
              "\n",
              "    .dataframe tbody tr th {\n",
              "        vertical-align: top;\n",
              "    }\n",
              "\n",
              "    .dataframe thead th {\n",
              "        text-align: right;\n",
              "    }\n",
              "</style>\n",
              "<table border=\"1\" class=\"dataframe\">\n",
              "  <thead>\n",
              "    <tr style=\"text-align: right;\">\n",
              "      <th></th>\n",
              "      <th>sentiment</th>\n",
              "    </tr>\n",
              "  </thead>\n",
              "  <tbody>\n",
              "    <tr>\n",
              "      <th>well</th>\n",
              "      <td>3.843190</td>\n",
              "    </tr>\n",
              "    <tr>\n",
              "      <th>killed</th>\n",
              "      <td>-8.685658</td>\n",
              "    </tr>\n",
              "    <tr>\n",
              "      <th>great</th>\n",
              "      <td>5.661081</td>\n",
              "    </tr>\n",
              "    <tr>\n",
              "      <th>free</th>\n",
              "      <td>3.185349</td>\n",
              "    </tr>\n",
              "    <tr>\n",
              "      <th>better</th>\n",
              "      <td>5.268084</td>\n",
              "    </tr>\n",
              "    <tr>\n",
              "      <th>fell</th>\n",
              "      <td>-0.493469</td>\n",
              "    </tr>\n",
              "    <tr>\n",
              "      <th>victory</th>\n",
              "      <td>4.352416</td>\n",
              "    </tr>\n",
              "    <tr>\n",
              "      <th>helped</th>\n",
              "      <td>2.650431</td>\n",
              "    </tr>\n",
              "    <tr>\n",
              "      <th>intelligence</th>\n",
              "      <td>-0.446912</td>\n",
              "    </tr>\n",
              "    <tr>\n",
              "      <th>concerns</th>\n",
              "      <td>-2.409386</td>\n",
              "    </tr>\n",
              "    <tr>\n",
              "      <th>limited</th>\n",
              "      <td>4.064308</td>\n",
              "    </tr>\n",
              "    <tr>\n",
              "      <th>premier</th>\n",
              "      <td>1.129882</td>\n",
              "    </tr>\n",
              "    <tr>\n",
              "      <th>winner</th>\n",
              "      <td>5.691388</td>\n",
              "    </tr>\n",
              "    <tr>\n",
              "      <th>denied</th>\n",
              "      <td>-3.400399</td>\n",
              "    </tr>\n",
              "    <tr>\n",
              "      <th>criminal</th>\n",
              "      <td>-7.329353</td>\n",
              "    </tr>\n",
              "    <tr>\n",
              "      <th>cancer</th>\n",
              "      <td>1.358638</td>\n",
              "    </tr>\n",
              "    <tr>\n",
              "      <th>trust</th>\n",
              "      <td>3.791185</td>\n",
              "    </tr>\n",
              "    <tr>\n",
              "      <th>criticism</th>\n",
              "      <td>-2.821326</td>\n",
              "    </tr>\n",
              "    <tr>\n",
              "      <th>helping</th>\n",
              "      <td>2.836155</td>\n",
              "    </tr>\n",
              "    <tr>\n",
              "      <th>trouble</th>\n",
              "      <td>-2.599560</td>\n",
              "    </tr>\n",
              "  </tbody>\n",
              "</table>\n",
              "</div>\n",
              "    <div class=\"colab-df-buttons\">\n",
              "\n",
              "  <div class=\"colab-df-container\">\n",
              "    <button class=\"colab-df-convert\" onclick=\"convertToInteractive('df-2b8fd40b-902e-4296-b5c9-35748e15ff18')\"\n",
              "            title=\"Convert this dataframe to an interactive table.\"\n",
              "            style=\"display:none;\">\n",
              "\n",
              "  <svg xmlns=\"http://www.w3.org/2000/svg\" height=\"24px\" viewBox=\"0 -960 960 960\">\n",
              "    <path d=\"M120-120v-720h720v720H120Zm60-500h600v-160H180v160Zm220 220h160v-160H400v160Zm0 220h160v-160H400v160ZM180-400h160v-160H180v160Zm440 0h160v-160H620v160ZM180-180h160v-160H180v160Zm440 0h160v-160H620v160Z\"/>\n",
              "  </svg>\n",
              "    </button>\n",
              "\n",
              "  <style>\n",
              "    .colab-df-container {\n",
              "      display:flex;\n",
              "      gap: 12px;\n",
              "    }\n",
              "\n",
              "    .colab-df-convert {\n",
              "      background-color: #E8F0FE;\n",
              "      border: none;\n",
              "      border-radius: 50%;\n",
              "      cursor: pointer;\n",
              "      display: none;\n",
              "      fill: #1967D2;\n",
              "      height: 32px;\n",
              "      padding: 0 0 0 0;\n",
              "      width: 32px;\n",
              "    }\n",
              "\n",
              "    .colab-df-convert:hover {\n",
              "      background-color: #E2EBFA;\n",
              "      box-shadow: 0px 1px 2px rgba(60, 64, 67, 0.3), 0px 1px 3px 1px rgba(60, 64, 67, 0.15);\n",
              "      fill: #174EA6;\n",
              "    }\n",
              "\n",
              "    .colab-df-buttons div {\n",
              "      margin-bottom: 4px;\n",
              "    }\n",
              "\n",
              "    [theme=dark] .colab-df-convert {\n",
              "      background-color: #3B4455;\n",
              "      fill: #D2E3FC;\n",
              "    }\n",
              "\n",
              "    [theme=dark] .colab-df-convert:hover {\n",
              "      background-color: #434B5C;\n",
              "      box-shadow: 0px 1px 3px 1px rgba(0, 0, 0, 0.15);\n",
              "      filter: drop-shadow(0px 1px 2px rgba(0, 0, 0, 0.3));\n",
              "      fill: #FFFFFF;\n",
              "    }\n",
              "  </style>\n",
              "\n",
              "    <script>\n",
              "      const buttonEl =\n",
              "        document.querySelector('#df-2b8fd40b-902e-4296-b5c9-35748e15ff18 button.colab-df-convert');\n",
              "      buttonEl.style.display =\n",
              "        google.colab.kernel.accessAllowed ? 'block' : 'none';\n",
              "\n",
              "      async function convertToInteractive(key) {\n",
              "        const element = document.querySelector('#df-2b8fd40b-902e-4296-b5c9-35748e15ff18');\n",
              "        const dataTable =\n",
              "          await google.colab.kernel.invokeFunction('convertToInteractive',\n",
              "                                                    [key], {});\n",
              "        if (!dataTable) return;\n",
              "\n",
              "        const docLinkHtml = 'Like what you see? Visit the ' +\n",
              "          '<a target=\"_blank\" href=https://colab.research.google.com/notebooks/data_table.ipynb>data table notebook</a>'\n",
              "          + ' to learn more about interactive tables.';\n",
              "        element.innerHTML = '';\n",
              "        dataTable['output_type'] = 'display_data';\n",
              "        await google.colab.output.renderOutput(dataTable, element);\n",
              "        const docLink = document.createElement('div');\n",
              "        docLink.innerHTML = docLinkHtml;\n",
              "        element.appendChild(docLink);\n",
              "      }\n",
              "    </script>\n",
              "  </div>\n",
              "\n",
              "\n",
              "<div id=\"df-5ecc4fc4-88ad-4bb4-99e8-78d839b6854e\">\n",
              "  <button class=\"colab-df-quickchart\" onclick=\"quickchart('df-5ecc4fc4-88ad-4bb4-99e8-78d839b6854e')\"\n",
              "            title=\"Suggest charts\"\n",
              "            style=\"display:none;\">\n",
              "\n",
              "<svg xmlns=\"http://www.w3.org/2000/svg\" height=\"24px\"viewBox=\"0 0 24 24\"\n",
              "     width=\"24px\">\n",
              "    <g>\n",
              "        <path d=\"M19 3H5c-1.1 0-2 .9-2 2v14c0 1.1.9 2 2 2h14c1.1 0 2-.9 2-2V5c0-1.1-.9-2-2-2zM9 17H7v-7h2v7zm4 0h-2V7h2v10zm4 0h-2v-4h2v4z\"/>\n",
              "    </g>\n",
              "</svg>\n",
              "  </button>\n",
              "\n",
              "<style>\n",
              "  .colab-df-quickchart {\n",
              "      --bg-color: #E8F0FE;\n",
              "      --fill-color: #1967D2;\n",
              "      --hover-bg-color: #E2EBFA;\n",
              "      --hover-fill-color: #174EA6;\n",
              "      --disabled-fill-color: #AAA;\n",
              "      --disabled-bg-color: #DDD;\n",
              "  }\n",
              "\n",
              "  [theme=dark] .colab-df-quickchart {\n",
              "      --bg-color: #3B4455;\n",
              "      --fill-color: #D2E3FC;\n",
              "      --hover-bg-color: #434B5C;\n",
              "      --hover-fill-color: #FFFFFF;\n",
              "      --disabled-bg-color: #3B4455;\n",
              "      --disabled-fill-color: #666;\n",
              "  }\n",
              "\n",
              "  .colab-df-quickchart {\n",
              "    background-color: var(--bg-color);\n",
              "    border: none;\n",
              "    border-radius: 50%;\n",
              "    cursor: pointer;\n",
              "    display: none;\n",
              "    fill: var(--fill-color);\n",
              "    height: 32px;\n",
              "    padding: 0;\n",
              "    width: 32px;\n",
              "  }\n",
              "\n",
              "  .colab-df-quickchart:hover {\n",
              "    background-color: var(--hover-bg-color);\n",
              "    box-shadow: 0 1px 2px rgba(60, 64, 67, 0.3), 0 1px 3px 1px rgba(60, 64, 67, 0.15);\n",
              "    fill: var(--button-hover-fill-color);\n",
              "  }\n",
              "\n",
              "  .colab-df-quickchart-complete:disabled,\n",
              "  .colab-df-quickchart-complete:disabled:hover {\n",
              "    background-color: var(--disabled-bg-color);\n",
              "    fill: var(--disabled-fill-color);\n",
              "    box-shadow: none;\n",
              "  }\n",
              "\n",
              "  .colab-df-spinner {\n",
              "    border: 2px solid var(--fill-color);\n",
              "    border-color: transparent;\n",
              "    border-bottom-color: var(--fill-color);\n",
              "    animation:\n",
              "      spin 1s steps(1) infinite;\n",
              "  }\n",
              "\n",
              "  @keyframes spin {\n",
              "    0% {\n",
              "      border-color: transparent;\n",
              "      border-bottom-color: var(--fill-color);\n",
              "      border-left-color: var(--fill-color);\n",
              "    }\n",
              "    20% {\n",
              "      border-color: transparent;\n",
              "      border-left-color: var(--fill-color);\n",
              "      border-top-color: var(--fill-color);\n",
              "    }\n",
              "    30% {\n",
              "      border-color: transparent;\n",
              "      border-left-color: var(--fill-color);\n",
              "      border-top-color: var(--fill-color);\n",
              "      border-right-color: var(--fill-color);\n",
              "    }\n",
              "    40% {\n",
              "      border-color: transparent;\n",
              "      border-right-color: var(--fill-color);\n",
              "      border-top-color: var(--fill-color);\n",
              "    }\n",
              "    60% {\n",
              "      border-color: transparent;\n",
              "      border-right-color: var(--fill-color);\n",
              "    }\n",
              "    80% {\n",
              "      border-color: transparent;\n",
              "      border-right-color: var(--fill-color);\n",
              "      border-bottom-color: var(--fill-color);\n",
              "    }\n",
              "    90% {\n",
              "      border-color: transparent;\n",
              "      border-bottom-color: var(--fill-color);\n",
              "    }\n",
              "  }\n",
              "</style>\n",
              "\n",
              "  <script>\n",
              "    async function quickchart(key) {\n",
              "      const quickchartButtonEl =\n",
              "        document.querySelector('#' + key + ' button');\n",
              "      quickchartButtonEl.disabled = true;  // To prevent multiple clicks.\n",
              "      quickchartButtonEl.classList.add('colab-df-spinner');\n",
              "      try {\n",
              "        const charts = await google.colab.kernel.invokeFunction(\n",
              "            'suggestCharts', [key], {});\n",
              "      } catch (error) {\n",
              "        console.error('Error during call to suggestCharts:', error);\n",
              "      }\n",
              "      quickchartButtonEl.classList.remove('colab-df-spinner');\n",
              "      quickchartButtonEl.classList.add('colab-df-quickchart-complete');\n",
              "    }\n",
              "    (() => {\n",
              "      let quickchartButtonEl =\n",
              "        document.querySelector('#df-5ecc4fc4-88ad-4bb4-99e8-78d839b6854e button');\n",
              "      quickchartButtonEl.style.display =\n",
              "        google.colab.kernel.accessAllowed ? 'block' : 'none';\n",
              "    })();\n",
              "  </script>\n",
              "</div>\n",
              "\n",
              "    </div>\n",
              "  </div>\n"
            ],
            "application/vnd.google.colaboratory.intrinsic+json": {
              "type": "dataframe",
              "summary": "{\n  \"name\": \"words_to_sentiment(test_labels)\",\n  \"rows\": 20,\n  \"fields\": [\n    {\n      \"column\": \"sentiment\",\n      \"properties\": {\n        \"dtype\": \"number\",\n        \"std\": 4.196390255085836,\n        \"min\": -8.68565819487111,\n        \"max\": 5.6913879133747445,\n        \"num_unique_values\": 20,\n        \"samples\": [\n          3.843190453200174,\n          -2.8213264726115885,\n          1.3586383081959061\n        ],\n        \"semantic_type\": \"\",\n        \"description\": \"\"\n      }\n    }\n  ]\n}"
            }
          },
          "metadata": {},
          "execution_count": 37
        }
      ],
      "source": [
        "def vecs_to_sentiment(vecs):\n",
        "    # predict_log_proba gives the log probability for each class\n",
        "    predictions = model.predict_log_proba(vecs)\n",
        "\n",
        "    # To see an overall positive vs. negative classification in one number,\n",
        "    # we take the log probability of positive sentiment minus the log\n",
        "    # probability of negative sentiment and return it.\n",
        "    print(predictions[0])\n",
        "    print(predictions[1])\n",
        "\n",
        "    # TODO: Write your code here\n",
        "    return predictions[:, 1] - predictions[:, 0]\n",
        "\n",
        "\n",
        "def words_to_sentiment(words):\n",
        "    vecs = embeddings.loc[embeddings.index.isin(words)].dropna()\n",
        "    log_odds = vecs_to_sentiment(vecs)\n",
        "    return pd.DataFrame({'sentiment': log_odds}, index=vecs.index)\n",
        "\n",
        "\n",
        "# Show 20 examples from the test set\n",
        "words_to_sentiment(test_labels).iloc[:20]"
      ]
    },
    {
      "cell_type": "markdown",
      "metadata": {
        "collapsed": true,
        "id": "gkikS4Qbk_NY"
      },
      "source": [
        "More than the accuracy number, this convinces us that the classifier is working. We can see that the classifier has learned to generalize sentiment to words outside of its training data."
      ]
    },
    {
      "cell_type": "markdown",
      "metadata": {
        "id": "dUJeb4jTk_NY"
      },
      "source": [
        "## Step 4: Get a sentiment score for text\n",
        "\n",
        "There are many ways to combine sentiments for word vectors into an overall sentiment score. Again, because we're following the path of least resistance, we're just going to average them."
      ]
    },
    {
      "cell_type": "markdown",
      "metadata": {
        "id": "Npksh1xwq8wQ"
      },
      "source": [
        "### TODO: Compute a Text Sentiment Score"
      ]
    },
    {
      "cell_type": "code",
      "execution_count": null,
      "metadata": {
        "id": "MOREpgc5k_NY"
      },
      "outputs": [],
      "source": [
        "import re\n",
        "TOKEN_RE = re.compile(r\"\\w.*?\\b\")\n",
        "# The regex above finds tokens that start with a word-like character (\\w), and continues\n",
        "# matching characters (.+?) until the next word break (\\b). It's a relatively simple\n",
        "# expression that manages to extract something very much like words from text.\n",
        "\n",
        "\n",
        "def text_to_sentiment(text):\n",
        "    tokens = [token.casefold() for token in TOKEN_RE.findall(text)]\n",
        "\n",
        "    # Compute sentiments for all the tokens and return the mean value\n",
        "\n",
        "    # TODO: Write your code here\n",
        "    print(vecs_to_sentiment(embeddings.loc[tokens]))\n",
        "    return vecs_to_sentiment(embeddings.loc[tokens]).mean()\n"
      ]
    },
    {
      "cell_type": "markdown",
      "metadata": {
        "id": "uHQHIkGDk_NZ"
      },
      "source": [
        "There are many things we could have done better:\n",
        "\n",
        "- Weight words by their inverse frequency, so that words like \"the\" and \"I\" don't cause big changes in sentiment\n",
        "- Adjust the averaging so that short sentences don't end up with the most extreme sentiment values\n",
        "- Take phrases into account\n",
        "- Use a more robust word-segmentation algorithm that isn't confused by apostrophes\n",
        "- Account for negations such as \"not happy\"\n",
        "\n",
        "But all of those would require extra code and wouldn't fundamentally change the results we're about to see. At least now we can roughly compare the relative positivity of different sentences:"
      ]
    },
    {
      "cell_type": "code",
      "execution_count": null,
      "metadata": {
        "id": "uajn0kMhk_Na",
        "colab": {
          "base_uri": "https://localhost:8080/"
        },
        "outputId": "58e0c537-e380-444c-ef3d-e8a5ff4fb17c"
      },
      "outputs": [
        {
          "output_type": "stream",
          "name": "stdout",
          "text": [
            "[-1.89791971 -0.16237787]\n",
            "[-2.23087092 -0.11365574]\n",
            "[1.73554184 2.11721518 2.00086882 0.46933224 0.53128114]\n",
            "[-1.89791971 -0.16237787]\n",
            "[-2.23087092 -0.11365574]\n"
          ]
        },
        {
          "output_type": "execute_result",
          "data": {
            "text/plain": [
              "1.3708478428409872"
            ]
          },
          "metadata": {},
          "execution_count": 41
        }
      ],
      "source": [
        "text_to_sentiment(\"this example is pretty cool\")"
      ]
    },
    {
      "cell_type": "code",
      "execution_count": null,
      "metadata": {
        "id": "QmUYmqvpk_Na",
        "colab": {
          "base_uri": "https://localhost:8080/"
        },
        "outputId": "b9fead35-5ad6-4db6-a2ef-f52c13b1585a"
      },
      "outputs": [
        {
          "output_type": "stream",
          "name": "stdout",
          "text": [
            "[-1.89791971 -0.16237787]\n",
            "[-2.23087092 -0.11365574]\n",
            "[1.73554184 2.11721518 2.00086882 0.14823695]\n",
            "[-1.89791971 -0.16237787]\n",
            "[-2.23087092 -0.11365574]\n"
          ]
        },
        {
          "output_type": "execute_result",
          "data": {
            "text/plain": [
              "1.5004656977937996"
            ]
          },
          "metadata": {},
          "execution_count": 42
        }
      ],
      "source": [
        "text_to_sentiment(\"this example is okay\")"
      ]
    },
    {
      "cell_type": "code",
      "execution_count": null,
      "metadata": {
        "id": "TalLb1qsk_Na",
        "colab": {
          "base_uri": "https://localhost:8080/"
        },
        "outputId": "3ce64040-d5c3-4cb0-e024-b396ccec4c56"
      },
      "outputs": [
        {
          "output_type": "stream",
          "name": "stdout",
          "text": [
            "[-0.10176308 -2.33555799]\n",
            "[-1.89791971 -0.16237787]\n",
            "[-2.23379491  1.73554184  2.11721518 -2.3381599 ]\n",
            "[-0.10176308 -2.33555799]\n",
            "[-1.89791971 -0.16237787]\n"
          ]
        },
        {
          "output_type": "execute_result",
          "data": {
            "text/plain": [
              "-0.17979944571988005"
            ]
          },
          "metadata": {},
          "execution_count": 43
        }
      ],
      "source": [
        "text_to_sentiment(\"meh, this example sucks\")"
      ]
    },
    {
      "cell_type": "markdown",
      "metadata": {
        "id": "gzJO5BAHk_Nb"
      },
      "source": [
        "## Step 5: Behold the monstrosity that we have created"
      ]
    },
    {
      "cell_type": "markdown",
      "metadata": {
        "id": "eLCzR7Duk_Nb"
      },
      "source": [
        "Not every sentence is going to contain obvious sentiment words. Let's see what it does with a few variations on a neutral sentence:"
      ]
    },
    {
      "cell_type": "code",
      "execution_count": null,
      "metadata": {
        "id": "wCTl0it3k_Nc",
        "colab": {
          "base_uri": "https://localhost:8080/"
        },
        "outputId": "8ba06b3f-3f83-438a-edab-6cdb782c7b91"
      },
      "outputs": [
        {
          "output_type": "stream",
          "name": "stdout",
          "text": [
            "[-0.85012816 -0.5574983 ]\n",
            "[-0.84389746 -0.5621737 ]\n",
            "[0.29262986 0.28172376 1.33190706 1.47649648 1.93636396 1.78200152]\n",
            "[-0.85012816 -0.5574983 ]\n",
            "[-0.84389746 -0.5621737 ]\n"
          ]
        },
        {
          "output_type": "execute_result",
          "data": {
            "text/plain": [
              "1.1835204380240416"
            ]
          },
          "metadata": {},
          "execution_count": 44
        }
      ],
      "source": [
        "text_to_sentiment(\"Let's go get Italian food\")"
      ]
    },
    {
      "cell_type": "code",
      "execution_count": null,
      "metadata": {
        "id": "cVvjOME9k_Nc",
        "colab": {
          "base_uri": "https://localhost:8080/"
        },
        "outputId": "89a1e537-5e15-4b36-96e1-7a0c02f4ba15"
      },
      "outputs": [
        {
          "output_type": "stream",
          "name": "stdout",
          "text": [
            "[-0.85012816 -0.5574983 ]\n",
            "[-0.84389746 -0.5621737 ]\n",
            "[0.29262986 0.28172376 1.33190706 1.47649648 3.70432927 1.78200152]\n",
            "[-0.85012816 -0.5574983 ]\n",
            "[-0.84389746 -0.5621737 ]\n"
          ]
        },
        {
          "output_type": "execute_result",
          "data": {
            "text/plain": [
              "1.4781813241527841"
            ]
          },
          "metadata": {},
          "execution_count": 45
        }
      ],
      "source": [
        "text_to_sentiment(\"Let's go get Chinese food\")"
      ]
    },
    {
      "cell_type": "code",
      "execution_count": null,
      "metadata": {
        "id": "32ZbUMKQk_Nd",
        "colab": {
          "base_uri": "https://localhost:8080/"
        },
        "outputId": "ce5bffc3-829d-4f24-92fe-3554c6ca543b"
      },
      "outputs": [
        {
          "output_type": "stream",
          "name": "stdout",
          "text": [
            "[-0.85012816 -0.5574983 ]\n",
            "[-0.84389746 -0.5621737 ]\n",
            "[ 0.29262986  0.28172376  1.33190706  1.47649648 -1.78758369  1.78200152]\n",
            "[-0.85012816 -0.5574983 ]\n",
            "[-0.84389746 -0.5621737 ]\n"
          ]
        },
        {
          "output_type": "execute_result",
          "data": {
            "text/plain": [
              "0.5628624973184243"
            ]
          },
          "metadata": {},
          "execution_count": 46
        }
      ],
      "source": [
        "text_to_sentiment(\"Let's go get Mexican food\")"
      ]
    },
    {
      "cell_type": "markdown",
      "metadata": {
        "id": "BAvs3c-_k_Nd"
      },
      "source": [
        "This is analogous to what I saw when I experimented with analyzing restaurant reviews using word embeddings, and found out that [all the Mexican restaurants were ending up with lower sentiment](https://blog.conceptnet.io/2017/04/24/conceptnet-numberbatch-17-04-better-less-stereotyped-word-vectors/) for no good reason.\n",
        "\n",
        "Word vectors are capable of representing subtle distinctions of meaning just by reading words in context. So they're also capable of representing less-subtle things like the biases of our society.\n",
        "\n",
        "Here are some other neutral statements:"
      ]
    },
    {
      "cell_type": "code",
      "execution_count": null,
      "metadata": {
        "id": "K8KeNvaWk_Ne",
        "colab": {
          "base_uri": "https://localhost:8080/"
        },
        "outputId": "f2b2ac99-60f6-4e19-da7b-0118ef0f1b32"
      },
      "outputs": [
        {
          "output_type": "stream",
          "name": "stdout",
          "text": [
            "[-2.23297199 -0.11340314]\n",
            "[-0.86299624 -0.54800164]\n",
            "[2.11956885 0.3149946  2.00086882 0.03311755]\n",
            "[-2.23297199 -0.11340314]\n",
            "[-0.86299624 -0.54800164]\n"
          ]
        },
        {
          "output_type": "execute_result",
          "data": {
            "text/plain": [
              "1.1171374537037189"
            ]
          },
          "metadata": {},
          "execution_count": 47
        }
      ],
      "source": [
        "text_to_sentiment(\"My name is Emily\")"
      ]
    },
    {
      "cell_type": "code",
      "execution_count": null,
      "metadata": {
        "id": "T1zzo4Ihk_Ne",
        "colab": {
          "base_uri": "https://localhost:8080/"
        },
        "outputId": "b95e43a6-4e87-4dfb-f927-fc3b9dd557b7"
      },
      "outputs": [
        {
          "output_type": "stream",
          "name": "stdout",
          "text": [
            "[-2.23297199 -0.11340314]\n",
            "[-0.86299624 -0.54800164]\n",
            "[ 2.11956885  0.3149946   2.00086882 -1.2321882 ]\n",
            "[-2.23297199 -0.11340314]\n",
            "[-0.86299624 -0.54800164]\n"
          ]
        },
        {
          "output_type": "execute_result",
          "data": {
            "text/plain": [
              "0.8008110143230732"
            ]
          },
          "metadata": {},
          "execution_count": 48
        }
      ],
      "source": [
        "text_to_sentiment(\"My name is Heather\")"
      ]
    },
    {
      "cell_type": "code",
      "execution_count": null,
      "metadata": {
        "id": "crzsmivOk_Nf",
        "colab": {
          "base_uri": "https://localhost:8080/"
        },
        "outputId": "2a71aa9e-5876-4736-df44-663b50baf323"
      },
      "outputs": [
        {
          "output_type": "stream",
          "name": "stdout",
          "text": [
            "[-2.23297199 -0.11340314]\n",
            "[-0.86299624 -0.54800164]\n",
            "[ 2.11956885  0.3149946   2.00086882 -0.96031902]\n",
            "[-2.23297199 -0.11340314]\n",
            "[-0.86299624 -0.54800164]\n"
          ]
        },
        {
          "output_type": "execute_result",
          "data": {
            "text/plain": [
              "0.8687783099697407"
            ]
          },
          "metadata": {},
          "execution_count": 49
        }
      ],
      "source": [
        "text_to_sentiment(\"My name is Yvette\")"
      ]
    },
    {
      "cell_type": "code",
      "execution_count": null,
      "metadata": {
        "id": "T5cITsd3k_Nf",
        "colab": {
          "base_uri": "https://localhost:8080/"
        },
        "outputId": "9dd816f6-7ce4-4c65-a8fe-0ccb1762fc84"
      },
      "outputs": [
        {
          "output_type": "stream",
          "name": "stdout",
          "text": [
            "[-2.23297199 -0.11340314]\n",
            "[-0.86299624 -0.54800164]\n",
            "[ 2.11956885  0.3149946   2.00086882 -1.77731774]\n",
            "[-2.23297199 -0.11340314]\n",
            "[-0.86299624 -0.54800164]\n"
          ]
        },
        {
          "output_type": "execute_result",
          "data": {
            "text/plain": [
              "0.6645286299274739"
            ]
          },
          "metadata": {},
          "execution_count": 50
        }
      ],
      "source": [
        "text_to_sentiment(\"My name is Shaniqua\")"
      ]
    },
    {
      "cell_type": "markdown",
      "metadata": {
        "id": "7awsCgLSUoAZ"
      },
      "source": [
        "### TODO: Experiment with your name and your teammates' names"
      ]
    },
    {
      "cell_type": "code",
      "execution_count": null,
      "metadata": {
        "id": "liaWmQzyamhn",
        "colab": {
          "base_uri": "https://localhost:8080/"
        },
        "outputId": "bb673615-f8e9-46c4-b102-98264bbfa8d1"
      },
      "outputs": [
        {
          "output_type": "stream",
          "name": "stdout",
          "text": [
            "[-2.23297199 -0.11340314]\n",
            "[-0.86299624 -0.54800164]\n",
            "[2.11956885 0.3149946  2.00086882 1.90759899 3.11139037 1.79505327]\n",
            "[-2.23297199 -0.11340314]\n",
            "[-0.86299624 -0.54800164]\n"
          ]
        },
        {
          "output_type": "execute_result",
          "data": {
            "text/plain": [
              "1.8749124822503707"
            ]
          },
          "metadata": {},
          "execution_count": 51
        }
      ],
      "source": [
        "# TODO: Fill in your name\n",
        "text_to_sentiment(\"My name is Steve Paul Jobs\")"
      ]
    },
    {
      "cell_type": "code",
      "execution_count": null,
      "metadata": {
        "id": "idIaIdjTamhn",
        "colab": {
          "base_uri": "https://localhost:8080/"
        },
        "outputId": "3a02e202-7378-4768-ad75-10d2ea879b0b"
      },
      "outputs": [
        {
          "output_type": "stream",
          "name": "stdout",
          "text": [
            "[-2.23297199 -0.11340314]\n",
            "[-0.86299624 -0.54800164]\n",
            "[ 2.11956885  0.3149946   2.00086882 -0.99775825]\n",
            "[-2.23297199 -0.11340314]\n",
            "[-0.86299624 -0.54800164]\n"
          ]
        },
        {
          "output_type": "execute_result",
          "data": {
            "text/plain": [
              "0.8594185031936941"
            ]
          },
          "metadata": {},
          "execution_count": 52
        }
      ],
      "source": [
        "# TODO: Fill in one of your teammates' name\n",
        "text_to_sentiment(\"My name is Narayan\")"
      ]
    },
    {
      "cell_type": "markdown",
      "metadata": {
        "id": "ma0UCasuk_Nf"
      },
      "source": [
        "Well, dang.\n",
        "\n",
        "The system has associated wildly different sentiments with people's names. You can look at these examples and many others and see that the sentiment is generally more positive for stereotypically-white names, and more negative for stereotypically-black names.\n",
        "\n",
        "This is the test that Caliskan, Bryson, and Narayanan used to conclude that [semantics derived automatically from language corpora contain human-like biases](http://opus.bath.ac.uk/55288/), a paper published in *Science* in April 2017, and we'll be using more of it shortly."
      ]
    },
    {
      "cell_type": "markdown",
      "metadata": {
        "id": "MDR6pTGHk_Ng"
      },
      "source": [
        "## Step 6: Measure the problem\n",
        "\n",
        "We want to learn how to not make something like this again. So let's put more data through it, and statistically measure how bad its bias is.\n",
        "\n",
        "Here we have four lists of names that tend to reflect different ethnic backgrounds, mostly from a United States perspective. The first two are lists of predominantly \"white\" and \"black\" names adapted from Caliskan et al.'s article. I also added typically Hispanic names, as well as Muslim names that come from Arabic or Urdu; these are two more distinct groupings of given names that tend to represent your background.\n",
        "\n",
        "This data is currently used as a bias-check in the ConceptNet build process, and can be found in the `conceptnet5.vectors.evaluation.bias` module. I'm interested in expanding this to more ethnic backgrounds, which may require looking at surnames and not just given names.\n",
        "\n",
        "Here are the lists:"
      ]
    },
    {
      "cell_type": "code",
      "execution_count": null,
      "metadata": {
        "id": "LkDzS93Rk_Ng"
      },
      "outputs": [],
      "source": [
        "NAMES_BY_ETHNICITY = {\n",
        "    # The first two lists are from the Caliskan et al. appendix describing the\n",
        "    # Word Embedding Association Test.\n",
        "    'White': [\n",
        "        'Adam', 'Chip', 'Harry', 'Josh', 'Roger', 'Alan', 'Frank', 'Ian', 'Justin',\n",
        "        'Ryan', 'Andrew', 'Fred', 'Jack', 'Matthew', 'Stephen', 'Brad', 'Greg', 'Jed',\n",
        "        'Paul', 'Todd', 'Brandon', 'Hank', 'Jonathan', 'Peter', 'Wilbur', 'Amanda',\n",
        "        'Courtney', 'Heather', 'Melanie', 'Sara', 'Amber', 'Crystal', 'Katie',\n",
        "        'Meredith', 'Shannon', 'Betsy', 'Donna', 'Kristin', 'Nancy', 'Stephanie',\n",
        "        'Bobbie-Sue', 'Ellen', 'Lauren', 'Peggy', 'Sue-Ellen', 'Colleen', 'Emily',\n",
        "        'Megan', 'Rachel', 'Wendy'\n",
        "    ],\n",
        "\n",
        "    'Black': [\n",
        "        'Alonzo', 'Jamel', 'Lerone', 'Percell', 'Theo', 'Alphonse', 'Jerome',\n",
        "        'Leroy', 'Rasaan', 'Torrance', 'Darnell', 'Lamar', 'Lionel', 'Rashaun',\n",
        "        'Tyree', 'Deion', 'Lamont', 'Malik', 'Terrence', 'Tyrone', 'Everol',\n",
        "        'Lavon', 'Marcellus', 'Terryl', 'Wardell', 'Aiesha', 'Lashelle', 'Nichelle',\n",
        "        'Shereen', 'Temeka', 'Ebony', 'Latisha', 'Shaniqua', 'Tameisha', 'Teretha',\n",
        "        'Jasmine', 'Latonya', 'Shanise', 'Tanisha', 'Tia', 'Lakisha', 'Latoya',\n",
        "        'Sharise', 'Tashika', 'Yolanda', 'Lashandra', 'Malika', 'Shavonn',\n",
        "        'Tawanda', 'Yvette'\n",
        "    ],\n",
        "\n",
        "    # This list comes from statistics about common Hispanic-origin names in the US.\n",
        "    'Hispanic': [\n",
        "        'Juan', 'José', 'Miguel', 'Luís', 'Jorge', 'Santiago', 'Matías', 'Sebastián',\n",
        "        'Mateo', 'Nicolás', 'Alejandro', 'Samuel', 'Diego', 'Daniel', 'Tomás',\n",
        "        'Juana', 'Ana', 'Luisa', 'María', 'Elena', 'Sofía', 'Isabella', 'Valentina',\n",
        "        'Camila', 'Valeria', 'Ximena', 'Luciana', 'Mariana', 'Victoria', 'Martina'\n",
        "    ],\n",
        "\n",
        "    # The following list conflates religion and ethnicity, I'm aware. So do given names.\n",
        "    #\n",
        "    # This list was cobbled together from searching baby-name sites for common Muslim names,\n",
        "    # as spelled in English. I did not ultimately distinguish whether the origin of the name\n",
        "    # is Arabic or Urdu or another language.\n",
        "    #\n",
        "    # I'd be happy to replace it with something more authoritative, given a source.\n",
        "    'Arab/Muslim': [\n",
        "        'Mohammed', 'Omar', 'Ahmed', 'Ali', 'Youssef', 'Abdullah', 'Yasin', 'Hamza',\n",
        "        'Ayaan', 'Syed', 'Rishaan', 'Samar', 'Ahmad', 'Zikri', 'Rayyan', 'Mariam',\n",
        "        'Jana', 'Malak', 'Salma', 'Nour', 'Lian', 'Fatima', 'Ayesha', 'Zahra', 'Sana',\n",
        "        'Zara', 'Alya', 'Shaista', 'Zoya', 'Yasmin'\n",
        "    ]\n",
        "}"
      ]
    },
    {
      "cell_type": "markdown",
      "metadata": {
        "id": "om62i3wrk_Nh"
      },
      "source": [
        "Now we'll use Pandas to make a table of these names, their predominant ethnic background, and the sentiment score we get for them:"
      ]
    },
    {
      "cell_type": "code",
      "execution_count": null,
      "metadata": {
        "id": "Xh8WQ8DSk_Nh",
        "colab": {
          "base_uri": "https://localhost:8080/"
        },
        "outputId": "398ab02f-7aaf-4564-e959-a73cccfe7e00"
      },
      "outputs": [
        {
          "output_type": "stream",
          "name": "stdout",
          "text": [
            "[-0.76892129 -0.62271257]\n",
            "[-0.12672713 -2.12841356]\n",
            "[-1.06046083 -0.42510092]\n",
            "[-0.05828261 -2.87145135]\n",
            "[-0.97774372 -0.47185953]\n",
            "[-2.26850883 -0.10921945]\n",
            "[-3.15496623 -0.04357562]\n",
            "[-2.55926062 -0.08051823]\n"
          ]
        }
      ],
      "source": [
        "def name_sentiment_table():\n",
        "    frames = []\n",
        "    for group, name_list in sorted(NAMES_BY_ETHNICITY.items()):\n",
        "        lower_names = [name.lower() for name in name_list]\n",
        "        sentiments = words_to_sentiment(lower_names)\n",
        "        sentiments['group'] = group\n",
        "        frames.append(sentiments)\n",
        "\n",
        "    # Put together the data we got from each ethnic group into one big table\n",
        "    return pd.concat(frames)\n",
        "\n",
        "name_sentiments = name_sentiment_table()"
      ]
    },
    {
      "cell_type": "markdown",
      "metadata": {
        "id": "cBcP2f1ek_Nh"
      },
      "source": [
        "A sample of the data:"
      ]
    },
    {
      "cell_type": "code",
      "execution_count": null,
      "metadata": {
        "id": "MXIPnBQFk_Ni",
        "colab": {
          "base_uri": "https://localhost:8080/",
          "height": 238
        },
        "outputId": "0398c369-e2e1-4421-be8f-5f55be9e3129"
      },
      "outputs": [
        {
          "output_type": "execute_result",
          "data": {
            "text/plain": [
              "        sentiment        group\n",
              "ali      0.146209  Arab/Muslim\n",
              "ayaan   -6.065590  Arab/Muslim\n",
              "malika  -3.107048        Black\n",
              "ana     -1.792622     Hispanic\n",
              "andrew   0.635782        White\n",
              "sara     2.416037        White"
            ],
            "text/html": [
              "\n",
              "  <div id=\"df-2f9f83da-3684-4692-8032-0b39b2f82fa1\" class=\"colab-df-container\">\n",
              "    <div>\n",
              "<style scoped>\n",
              "    .dataframe tbody tr th:only-of-type {\n",
              "        vertical-align: middle;\n",
              "    }\n",
              "\n",
              "    .dataframe tbody tr th {\n",
              "        vertical-align: top;\n",
              "    }\n",
              "\n",
              "    .dataframe thead th {\n",
              "        text-align: right;\n",
              "    }\n",
              "</style>\n",
              "<table border=\"1\" class=\"dataframe\">\n",
              "  <thead>\n",
              "    <tr style=\"text-align: right;\">\n",
              "      <th></th>\n",
              "      <th>sentiment</th>\n",
              "      <th>group</th>\n",
              "    </tr>\n",
              "  </thead>\n",
              "  <tbody>\n",
              "    <tr>\n",
              "      <th>ali</th>\n",
              "      <td>0.146209</td>\n",
              "      <td>Arab/Muslim</td>\n",
              "    </tr>\n",
              "    <tr>\n",
              "      <th>ayaan</th>\n",
              "      <td>-6.065590</td>\n",
              "      <td>Arab/Muslim</td>\n",
              "    </tr>\n",
              "    <tr>\n",
              "      <th>malika</th>\n",
              "      <td>-3.107048</td>\n",
              "      <td>Black</td>\n",
              "    </tr>\n",
              "    <tr>\n",
              "      <th>ana</th>\n",
              "      <td>-1.792622</td>\n",
              "      <td>Hispanic</td>\n",
              "    </tr>\n",
              "    <tr>\n",
              "      <th>andrew</th>\n",
              "      <td>0.635782</td>\n",
              "      <td>White</td>\n",
              "    </tr>\n",
              "    <tr>\n",
              "      <th>sara</th>\n",
              "      <td>2.416037</td>\n",
              "      <td>White</td>\n",
              "    </tr>\n",
              "  </tbody>\n",
              "</table>\n",
              "</div>\n",
              "    <div class=\"colab-df-buttons\">\n",
              "\n",
              "  <div class=\"colab-df-container\">\n",
              "    <button class=\"colab-df-convert\" onclick=\"convertToInteractive('df-2f9f83da-3684-4692-8032-0b39b2f82fa1')\"\n",
              "            title=\"Convert this dataframe to an interactive table.\"\n",
              "            style=\"display:none;\">\n",
              "\n",
              "  <svg xmlns=\"http://www.w3.org/2000/svg\" height=\"24px\" viewBox=\"0 -960 960 960\">\n",
              "    <path d=\"M120-120v-720h720v720H120Zm60-500h600v-160H180v160Zm220 220h160v-160H400v160Zm0 220h160v-160H400v160ZM180-400h160v-160H180v160Zm440 0h160v-160H620v160ZM180-180h160v-160H180v160Zm440 0h160v-160H620v160Z\"/>\n",
              "  </svg>\n",
              "    </button>\n",
              "\n",
              "  <style>\n",
              "    .colab-df-container {\n",
              "      display:flex;\n",
              "      gap: 12px;\n",
              "    }\n",
              "\n",
              "    .colab-df-convert {\n",
              "      background-color: #E8F0FE;\n",
              "      border: none;\n",
              "      border-radius: 50%;\n",
              "      cursor: pointer;\n",
              "      display: none;\n",
              "      fill: #1967D2;\n",
              "      height: 32px;\n",
              "      padding: 0 0 0 0;\n",
              "      width: 32px;\n",
              "    }\n",
              "\n",
              "    .colab-df-convert:hover {\n",
              "      background-color: #E2EBFA;\n",
              "      box-shadow: 0px 1px 2px rgba(60, 64, 67, 0.3), 0px 1px 3px 1px rgba(60, 64, 67, 0.15);\n",
              "      fill: #174EA6;\n",
              "    }\n",
              "\n",
              "    .colab-df-buttons div {\n",
              "      margin-bottom: 4px;\n",
              "    }\n",
              "\n",
              "    [theme=dark] .colab-df-convert {\n",
              "      background-color: #3B4455;\n",
              "      fill: #D2E3FC;\n",
              "    }\n",
              "\n",
              "    [theme=dark] .colab-df-convert:hover {\n",
              "      background-color: #434B5C;\n",
              "      box-shadow: 0px 1px 3px 1px rgba(0, 0, 0, 0.15);\n",
              "      filter: drop-shadow(0px 1px 2px rgba(0, 0, 0, 0.3));\n",
              "      fill: #FFFFFF;\n",
              "    }\n",
              "  </style>\n",
              "\n",
              "    <script>\n",
              "      const buttonEl =\n",
              "        document.querySelector('#df-2f9f83da-3684-4692-8032-0b39b2f82fa1 button.colab-df-convert');\n",
              "      buttonEl.style.display =\n",
              "        google.colab.kernel.accessAllowed ? 'block' : 'none';\n",
              "\n",
              "      async function convertToInteractive(key) {\n",
              "        const element = document.querySelector('#df-2f9f83da-3684-4692-8032-0b39b2f82fa1');\n",
              "        const dataTable =\n",
              "          await google.colab.kernel.invokeFunction('convertToInteractive',\n",
              "                                                    [key], {});\n",
              "        if (!dataTable) return;\n",
              "\n",
              "        const docLinkHtml = 'Like what you see? Visit the ' +\n",
              "          '<a target=\"_blank\" href=https://colab.research.google.com/notebooks/data_table.ipynb>data table notebook</a>'\n",
              "          + ' to learn more about interactive tables.';\n",
              "        element.innerHTML = '';\n",
              "        dataTable['output_type'] = 'display_data';\n",
              "        await google.colab.output.renderOutput(dataTable, element);\n",
              "        const docLink = document.createElement('div');\n",
              "        docLink.innerHTML = docLinkHtml;\n",
              "        element.appendChild(docLink);\n",
              "      }\n",
              "    </script>\n",
              "  </div>\n",
              "\n",
              "\n",
              "<div id=\"df-92d0c794-bcea-4021-85f7-de280be8d1ef\">\n",
              "  <button class=\"colab-df-quickchart\" onclick=\"quickchart('df-92d0c794-bcea-4021-85f7-de280be8d1ef')\"\n",
              "            title=\"Suggest charts\"\n",
              "            style=\"display:none;\">\n",
              "\n",
              "<svg xmlns=\"http://www.w3.org/2000/svg\" height=\"24px\"viewBox=\"0 0 24 24\"\n",
              "     width=\"24px\">\n",
              "    <g>\n",
              "        <path d=\"M19 3H5c-1.1 0-2 .9-2 2v14c0 1.1.9 2 2 2h14c1.1 0 2-.9 2-2V5c0-1.1-.9-2-2-2zM9 17H7v-7h2v7zm4 0h-2V7h2v10zm4 0h-2v-4h2v4z\"/>\n",
              "    </g>\n",
              "</svg>\n",
              "  </button>\n",
              "\n",
              "<style>\n",
              "  .colab-df-quickchart {\n",
              "      --bg-color: #E8F0FE;\n",
              "      --fill-color: #1967D2;\n",
              "      --hover-bg-color: #E2EBFA;\n",
              "      --hover-fill-color: #174EA6;\n",
              "      --disabled-fill-color: #AAA;\n",
              "      --disabled-bg-color: #DDD;\n",
              "  }\n",
              "\n",
              "  [theme=dark] .colab-df-quickchart {\n",
              "      --bg-color: #3B4455;\n",
              "      --fill-color: #D2E3FC;\n",
              "      --hover-bg-color: #434B5C;\n",
              "      --hover-fill-color: #FFFFFF;\n",
              "      --disabled-bg-color: #3B4455;\n",
              "      --disabled-fill-color: #666;\n",
              "  }\n",
              "\n",
              "  .colab-df-quickchart {\n",
              "    background-color: var(--bg-color);\n",
              "    border: none;\n",
              "    border-radius: 50%;\n",
              "    cursor: pointer;\n",
              "    display: none;\n",
              "    fill: var(--fill-color);\n",
              "    height: 32px;\n",
              "    padding: 0;\n",
              "    width: 32px;\n",
              "  }\n",
              "\n",
              "  .colab-df-quickchart:hover {\n",
              "    background-color: var(--hover-bg-color);\n",
              "    box-shadow: 0 1px 2px rgba(60, 64, 67, 0.3), 0 1px 3px 1px rgba(60, 64, 67, 0.15);\n",
              "    fill: var(--button-hover-fill-color);\n",
              "  }\n",
              "\n",
              "  .colab-df-quickchart-complete:disabled,\n",
              "  .colab-df-quickchart-complete:disabled:hover {\n",
              "    background-color: var(--disabled-bg-color);\n",
              "    fill: var(--disabled-fill-color);\n",
              "    box-shadow: none;\n",
              "  }\n",
              "\n",
              "  .colab-df-spinner {\n",
              "    border: 2px solid var(--fill-color);\n",
              "    border-color: transparent;\n",
              "    border-bottom-color: var(--fill-color);\n",
              "    animation:\n",
              "      spin 1s steps(1) infinite;\n",
              "  }\n",
              "\n",
              "  @keyframes spin {\n",
              "    0% {\n",
              "      border-color: transparent;\n",
              "      border-bottom-color: var(--fill-color);\n",
              "      border-left-color: var(--fill-color);\n",
              "    }\n",
              "    20% {\n",
              "      border-color: transparent;\n",
              "      border-left-color: var(--fill-color);\n",
              "      border-top-color: var(--fill-color);\n",
              "    }\n",
              "    30% {\n",
              "      border-color: transparent;\n",
              "      border-left-color: var(--fill-color);\n",
              "      border-top-color: var(--fill-color);\n",
              "      border-right-color: var(--fill-color);\n",
              "    }\n",
              "    40% {\n",
              "      border-color: transparent;\n",
              "      border-right-color: var(--fill-color);\n",
              "      border-top-color: var(--fill-color);\n",
              "    }\n",
              "    60% {\n",
              "      border-color: transparent;\n",
              "      border-right-color: var(--fill-color);\n",
              "    }\n",
              "    80% {\n",
              "      border-color: transparent;\n",
              "      border-right-color: var(--fill-color);\n",
              "      border-bottom-color: var(--fill-color);\n",
              "    }\n",
              "    90% {\n",
              "      border-color: transparent;\n",
              "      border-bottom-color: var(--fill-color);\n",
              "    }\n",
              "  }\n",
              "</style>\n",
              "\n",
              "  <script>\n",
              "    async function quickchart(key) {\n",
              "      const quickchartButtonEl =\n",
              "        document.querySelector('#' + key + ' button');\n",
              "      quickchartButtonEl.disabled = true;  // To prevent multiple clicks.\n",
              "      quickchartButtonEl.classList.add('colab-df-spinner');\n",
              "      try {\n",
              "        const charts = await google.colab.kernel.invokeFunction(\n",
              "            'suggestCharts', [key], {});\n",
              "      } catch (error) {\n",
              "        console.error('Error during call to suggestCharts:', error);\n",
              "      }\n",
              "      quickchartButtonEl.classList.remove('colab-df-spinner');\n",
              "      quickchartButtonEl.classList.add('colab-df-quickchart-complete');\n",
              "    }\n",
              "    (() => {\n",
              "      let quickchartButtonEl =\n",
              "        document.querySelector('#df-92d0c794-bcea-4021-85f7-de280be8d1ef button');\n",
              "      quickchartButtonEl.style.display =\n",
              "        google.colab.kernel.accessAllowed ? 'block' : 'none';\n",
              "    })();\n",
              "  </script>\n",
              "</div>\n",
              "\n",
              "    </div>\n",
              "  </div>\n"
            ],
            "application/vnd.google.colaboratory.intrinsic+json": {
              "type": "dataframe",
              "summary": "{\n  \"name\": \"name_sentiments\",\n  \"rows\": 6,\n  \"fields\": [\n    {\n      \"column\": \"sentiment\",\n      \"properties\": {\n        \"dtype\": \"number\",\n        \"std\": 3.028744616471011,\n        \"min\": -6.065589644761247,\n        \"max\": 2.416037342696028,\n        \"num_unique_values\": 6,\n        \"samples\": [\n          0.146208725123244,\n          -6.065589644761247,\n          2.416037342696028\n        ],\n        \"semantic_type\": \"\",\n        \"description\": \"\"\n      }\n    },\n    {\n      \"column\": \"group\",\n      \"properties\": {\n        \"dtype\": \"string\",\n        \"num_unique_values\": 4,\n        \"samples\": [\n          \"Black\",\n          \"White\",\n          \"Arab/Muslim\"\n        ],\n        \"semantic_type\": \"\",\n        \"description\": \"\"\n      }\n    }\n  ]\n}"
            }
          },
          "metadata": {},
          "execution_count": 55
        }
      ],
      "source": [
        "name_sentiments.iloc[::25]"
      ]
    },
    {
      "cell_type": "markdown",
      "metadata": {
        "id": "jTNPHYLlk_Ni"
      },
      "source": [
        "Now we can visualize the distribution of sentiment we get for each kind of name:"
      ]
    },
    {
      "cell_type": "code",
      "execution_count": null,
      "metadata": {
        "id": "AkVjFLqqk_Nj",
        "colab": {
          "base_uri": "https://localhost:8080/",
          "height": 499
        },
        "outputId": "8a86895b-5fd8-4f77-a295-eb338ac1551b"
      },
      "outputs": [
        {
          "output_type": "execute_result",
          "data": {
            "text/plain": [
              "(-10.0, 10.0)"
            ]
          },
          "metadata": {},
          "execution_count": 56
        },
        {
          "output_type": "display_data",
          "data": {
            "text/plain": [
              "<Figure size 640x480 with 1 Axes>"
            ],
            "image/png": "iVBORw0KGgoAAAANSUhEUgAAAnQAAAHQCAYAAAAoDPeqAAAAOXRFWHRTb2Z0d2FyZQBNYXRwbG90bGliIHZlcnNpb24zLjcuMSwgaHR0cHM6Ly9tYXRwbG90bGliLm9yZy/bCgiHAAAACXBIWXMAAA9hAAAPYQGoP6dpAABp/klEQVR4nO3deVxU9f4/8Newo+yo4IKyiGZcUzRxxX3JMvWapuJ6s9JrefVblnpLU7N7KzOtzMw0TcsFl66WKdcFLNQ0cbviBgIK4i6CIDuf3x/+ZmJg5szKcs68no8Hj/Isn/OBMzPnPZ/l/VEJIQSIiIiISLbsaroCRERERGQZBnREREREMseAjoiIiEjmGNARERERyRwDOiIiIiKZY0BHREREJHMM6IiIiIhkjgEdERERkcwxoCMiIiKSOQZ0RERERDJnkwHdzZs3sWHDBvzjH/9A586d4erqCpVKhZ49exo8t7i4GIsXL0abNm1Qt25deHt7o1evXtixY4dFdcrNzcW7776LJ554Aq6urqhfvz4GDRqEuLg4i8olIiIi5VPZ4lquy5Ytw//93/9V2t6jRw/JAKqgoAD9+vVDfHw87O3tERYWhry8PFy5cgUAMGvWLHz44Ycm1+fu3bvo1q0bLl26BGdnZzz55JO4c+cOMjIyoFKpsHz5ckydOtXkcomIiMg22GQLnYeHB/r27Ys5c+Zgx44dmDt3rlHnzZo1C/Hx8QgKCkJiYiLOnDmD5ORk7Ny5E87Ozvjoo4/w008/mVyfSZMm4dKlS2jfvj1SUlJw8uRJXLt2DV9//TWEEPjHP/6B06dPm1wuERER2QabbKGraPny5Zg2bZpkC92tW7fQtGlTFBUV4eDBg+jVq5fW/nnz5uH9999Hu3btkJCQYPS1T506hXbt2sHOzg6XLl1C8+bNtfaPHz8eGzZswLBhw7B9+3aTfzciIiJSPptsoTPHrl27UFRUhNDQ0ErBHABMnjwZAHDy5ElNF6wxtm3bBgDo3bt3pWCufLm//PIL8vLyzKk6ERERKRwDOiP9/vvvAIDIyEid+xs3boygoCCtY00pt3v37jr3R0REwNnZGQUFBex2JSIiIp0Y0Bnp8uXLAICQkBC9x6j3Xbp0yWrlOjo6IiAgwORyiYiIyHY41HQF5OL+/fsAAB8fH73HqPdlZWXVSLnBwcF696WlpcHZ2RkNGzY0um5ERERUs27cuAFnZ2c8ePBA8jgGdEYqKCgAADg5Oek9xtnZGQCQn59f4+VWJIRASUmJ2ecTERFR9TP22c2AzkguLi4AgKKiIr3HFBYWAgBcXV1NKvfRo0dWKTclJUXvPnXrndQxREREVLtI9b6VxzF0RvL29gbwZxepLup96mNrslwiIiKyHQzojNSiRQsAQHJyst5j1OlK1Mdao9zi4mJcu3bN5HKJiIjIdjCgM1KnTp0AAPHx8Tr3X79+HampqVrHmlLub7/9pnP/8ePHUVRUBBcXF7Rt29aEGhMREZGtYEBnpCFDhsDR0RFJSUmIjY2ttP/rr78GAISHh+tMEKzP8OHDAQCxsbE6W+nU5Q4cOBBubm7mVJ2IiIgUjgGdkfz8/DSrNqjXXlX76aef8PHHHwMA3nvvvUrnbtu2DYGBgQgMDKy0r127dhg0aBBKS0sxatQo3LhxA8DjWamrVq3Chg0bYGdnh3fffbcKfisiIiJSApuc5Zqeno7w8HDNv9WpQw4fPox69epptr/99tt4++23Nf/++OOPkZCQgKNHjyIsLAx/+ctfkJubqxk79+abb2LIkCGVrpebm4urV6/qrc+3336Lrl27IiEhAUFBQXjyySdx9+5dpKenQ6VSYdmyZWjXrp3FvzcREREpk00GdKWlpbh3716l7SUlJVrbHz16pLXf1dUVcXFxWLp0KX744QdcvnwZTk5O6NGjB6ZNm4YXXnjBrPrUr18fCQkJ+PDDD7Ft2zacP38edevWxcCBA/HWW2/pXDuWiIiISE0lhBA1XQmqesxDR0REJD/GPr85ho6IiIhI5hjQEREREckcAzoiIiIimWNAR0RERCRzDOiIiIiIZI4BHREREZHMMaAjIiIikjkGdEREREQyx4COiIiISOYY0BERERHJHAM6IiIiIpljQEdEREQkcwzoiIiIiGSOAR0RERGRzDGgIyIiIpI5BnREREREMseAjoiIiEjmGNARERERyRwDOiIiIiKZY0BHREREJHMM6IiIiIhkjgEdERERkcwxoCMiIiKSOQZ0RERERDLHgI6IiIhI5hjQEREREckcAzoiIiIimWNAR0RERCRzDOhMkJaWBpVKZdTP3/72N6PLnT9/vsHyVq5cWYW/GREREcmZQ01XQE5cXFzQtWtXvfsLCgqQkJAAAOjSpYvJ5Tdo0AChoaE69zVs2NDk8oiIiMg2MKAzgb+/P+Lj4/Xu/+677zBx4kS4urpi5MiRJpc/cOBArFu3zoIaEhERkS1il6sVqYOxYcOGwcPDo2YrQ0RERDaDAZ2VpKWl4dChQwCAiRMn1mxliIiIyKawy9VKvvvuOwgh0LRpU/Tu3dusMs6cOYOoqCjcvHkT7u7ueOqppzBq1CiEhYVZubZERESkJAzorEAIge+++w4AMH78eNjZmdfwefr0aZw+fVrz7127duGDDz7A9OnT8cknn8De3t4a1SUiIiKFYUBnBYcOHUJqaioA87pbGzVqhIULF2LAgAEIDg6Gu7s7Ll++jBUrVmDlypVYtmwZHB0d8fHHH0uWExwcrHdfeno6AgICTK4bERER1X4qIYSo6UrI3cSJE/Hdd98hMjISv/76q1XL/vjjjzFr1iw4ODggKSkJgYGBeo81JqBLSUmxav2IiIio6qif7Yae3wzoLJSbmwt/f3/k5eVhzZo1eOmll6xafmlpKZo2bYrMzEx89tln+Mc//mFWOca+IIiIiKj2MPb5zVmuFtq2bRvy8vJQp04djBgxwurl29vbo2PHjgCApKQkq5dPRERE8seAzkLq3HPDhw+Hu7t7lVzDyckJAFBSUlIl5RMREZG8MaCzQGpqqmbMXFXmnjt37hwAoEmTJlV2DSIiIpIvBnQWUOeeCwwMRM+ePavkGrt370ZiYiIAoH///lVyDSIiIpI3BnRmEkJg/fr1AIAJEyZApVJJHt+tWzcEBgZi2bJlWtsTExMxefJknDlzRmt7WVkZNm3ahKioKADAoEGD0KFDB+v9AkRERKQYzENnJnXuOZVKhQkTJhg8PiMjA1evXsWDBw+0thcXF2PVqlVYtWoVfHx80KxZMzg4OCA5ORlZWVkAgMjISGzYsKEqfg0iIiJSAAZ0ZlJPhujevTuCgoLMLicwMBCLFi3C0aNHceHCBSQnJ6OgoAA+Pj4YOHAgoqKiMHr0aK4SQURERHoxD52NYB46IiIi+WEeOiIiIiIbwYCOiIiISOYY0BERERHJHAM6IiIiIpljQEdEREQkcwzoiIiIiGSOAR0RERGRzDGgIyIiIpI5BnREREREMseAjoiIiEjmGNARERERyRwDOiIiIiKZY0BHREREJHMM6IiIiIhkjgEdERERkcwxoCMiIiKSOQZ0RERERDLHgI6IiIhI5hjQEREREckcAzoiIiIimWNAR0RERCRzDOiIiIiIZI4BHREREZHMMaAjIiIikjkGdEREREQyx4COiIiISOYY0BERERHJHAM6IiIiIpljQGei+fPnQ6VSSf6sXLnSrLJjY2MxaNAg1K9fH66urnjiiScwd+5c5OXlWfm3ICIiIiVxqOkKyFWDBg0QGhqqc1/Dhg1NLu+LL77A9OnTIYRAkyZNEBAQgPPnz2PRokXYvn074uPj4ePjY2m1iYiISIEY0Jlp4MCBWLdunVXKSkhIwIwZMwAAX3/9NV555RWoVCpkZmZi8ODBSEhIwCuvvILt27db5XpERESkLOxyrQXef/99lJWVYdy4cXj11VehUqkAAI0aNcKmTZtgZ2eHHTt24OzZszVcUyIiIqqNGNDVsNzcXOzduxcA8Oqrr1baHxoait69ewMAtm7dWq11IyIiInlgl6uZzpw5g6ioKNy8eRPu7u546qmnMGrUKISFhZlUzqlTp1BYWAhnZ2dEREToPCYyMhL79+/H77//bo2qExERkcIwoDPT6dOncfr0ac2/d+3ahQ8++ADTp0/HJ598Ant7e6PKuXz5MgCgadOmcHR01HlMSEgIAODSpUuWVZqIiIgUiQGdiRo1aoSFCxdiwIABCA4Ohru7Oy5fvowVK1Zg5cqVWLZsGRwdHfHxxx8bVd79+/cBQHIGq3pfVlaWZFnBwcF696WnpyMgIMCoOhEREZG8MKAzka5xbq1bt8ZXX32FoKAgzJo1C0uXLsXUqVMRGBhosLyCggIAgJOTk95jnJ2dAQD5+fnmVZqIiIgUjZMirOjNN99Eo0aNUFJSgl27dhl1jouLCwCgqKhI7zGFhYUAAFdXV8myUlJS9P6wdY6IiEi5GNBZkb29PTp27AgASEpKMuocb29vAH92veqi3qc+loiIiKg8BnRWpu46LSkpMer4Fi1aAACuXbuG4uJincdcuXJF61giIiKi8hjQWdm5c+cAAE2aNDHq+PDwcDg5OaGwsBDHjx/Xecxvv/0GAOjcubN1KklERESKwoDOinbv3o3ExEQAQP/+/Y06x93dHQMGDAAArFq1qtL+pKQkHDx4EAAwfPhwK9WUiIiIlIQBnQkSExMxefJknDlzRmt7WVkZNm3ahKioKADAoEGD0KFDB61junXrhsDAQCxbtqxSuXPnzoVKpcKGDRuwatUqCCEAADdu3MDo0aNRVlaGoUOHok2bNlXzixEREZGsMaAzQXFxMVatWoW2bdvC19cX7dq1Q0REBOrVq4eoqCjk5OQgMjISGzZsqHRuRkYGrl69igcPHlTa16FDB3z66acAgMmTJ6NZs2Zo164dgoKCkJCQgJYtW+Kbb76p6l+PiIiIZIp56EwQGBiIRYsW4ejRo7hw4QKSk5NRUFAAHx8fDBw4EFFRURg9erTRq0SUN2PGDLRu3RpLlizBsWPHcPv2bTRr1gzDhw/HnDlz4ObmVgW/ERGRPMUk3sSK2GRcvpWLFn5umNqrOQaE+dd0tYhqjEqo+/dI0dSrSKSkpNRwTYiILBOTeBOTNyRobVOpgJVj2zOoI8Ux9vnNLlciIpKVFbHJlbYJAayIu1IDtSGqHRjQERGRrFy+latze9Kth9VcE6Lag2PoiIhIVlr4ueFMRnal7aF+7jVQG6oOHDNpGFvoiIhIVqb2ag6VSnubSgW81jOkZipEVUo9ZvJMRjbyi0txJiMbU75PQEzizZquWq3CgI6IiGRlQJg/Vo5tjzYBXqjjZI82AV74emx79GeLjSJxzKRx2OVKRESyMyDMn11uNoJjJo3DFjoiIiKqtVr46c7DyjGT2hjQERERUa3FMZPGYUBHREREtRbHTBqHY+iIiIioVuOYScPYQkdEREQkcwzoiIiIiGSOAR0RERGRzDGgIyIiIpI5ToogIkXgWo9EZMvYQkdEsse1HonI1jGgIyLZ41qPRGTr2OVKRLLHtR6JbBuHXLCFjogUgGs9KkdM4k0MWR6PVnP3YsjyeHabk0EccvEYAzoikj2u9agMfDCTFH3BPodcPMaAjohkj2s9KoO5D2a26imfVLDPIRePcQwdESkC13qUP6kHs74xUuoHvZr6Qb9ybHu+HhREKthv4eeGMxnZlfbb2pALttARkSKwlUb+9I2FbODurLd1ht1tyqLvfSwV7HPIxWMWBXTBwcHo1KmT0cdHRkYiJMS2/sBEVPU49koZ9D2YdVEHbexuUw6p97HUxCcOuXjMoi7XtLQ0FBQUGH18RkYGrl27ZskliYgqkWqlYbebfKgfzCviriDp1kOE+rnjtZ4hmL75tM7jk249ZHebgki9j6f2ao4p3ydAiD/3lW+F45CLah5DV1JSAjs79vISkXWxlUY5dD2YpYK2qT1DJB/0JB9S72N9wb6ttcJJqbboKj8/H7dv34a7O781EZF1MQ+dskmNkWJ3m3JIvY81k2JuPkRoAzdMZTBXiUktdNeuXUNaWprWtqKiIvz2228Q5b8elSOEwIMHD/DDDz+guLgYrVu3NruyRES6GOqOIXnT1TrTJdgXX8YmY/rm0za7MoDS6Hsfdwn25UxmI6iEvkhMhwULFmDhwoWafwshoNI3YrUC9bEbNmxAVFSU6TUliwQHBwMAUlJSargmRFUjJvEmu2NsRMVUJcDjBz8f8PKn6338ZWyyzi73NgFe2Pla1xqoZfUy9vlt8hi68vGfSqXS2zJX/hgPDw/85S9/wZQpU2QdzAkhcPToUezatQvx8fG4cOECcnJy4OXlhfDwcEyYMAFRUVFGB7lqEydOxHfffSd5zJ49e/DMM89YUn0iReOgaNvBSTDKpet9LDUphv5kUkD33nvv4b333tP8287ODv7+/sjMzLR6xWqjgwcPom/fvpp/BwcHIygoCKmpqdi3bx/27duHTZs2Yfv27XB2dja5/ICAADRt2lTnPm9vb7PrTUSkJJwEY1s4k9k4Fs1yHT9+PLy8vKxUldpPCIGgoCDMmDEDo0aNQoMGDTT7NmzYgFdeeQW7d+/GvHnz8NFHH5lc/ksvvYT58+dbscZERMrDB7xt4RhZ41gU0K1bt85K1ZCHiIgIXLp0CY6OjpX2jRs3Dunp6XjnnXewevVq/Pvf/2aKFiKiKsAHvG2pqpQl+paTkyuu5WoCDw8Pyf0DBw7EO++8g/v37+POnTvw8/OrppoREdkO5iSzPdYeI6vENYCtEtA9fPgQP//8M86ePYv79++juLhY77EqlQpr1qyxxmVrnfz8fM3/u7q6mnx+bGwsEhMTce/ePXh5eaF9+/YYO3YsmjVrZs1qEhHJHifByFN1t4rpu54SJ9aYlLZEl3Xr1mH69OnIzf1zkKquItUzYlUqFUpLSy25ZK01bdo0LF++HG3atMHp06eNPk9qlqujoyMWLVqEt99+22A56qnNuqSnpyMgIIBpS4iIqEYYSjdj7WBP6nozNp9GfnHlWKSOkz3OL6xdGSWqLG1JeTExMZg0aRKEEHBxcUHnzp3RqFEjODjYXk9uQkICVq5cCQCYPXu2SeeGhoZiyZIl6N27NwIDA+Hs7IyzZ89iyZIl2Lp1K2bNmgU3NzdMnTq1KqpOpAhKGw9DpDRSrWIAJLtApd7f5rTCKXFijUUtdH369EFsbCw6d+6MnTt3ol69etasm2zcunULERERuHbtGv76179ix44dViv7tddew4oVK+Dl5YX09HS4ueleGsUQJhYmJWOiWaLar9XcvXpbxUIb6A6w2gR4YWrPEL3vbwBmtcItHdlW58Sa2rhsnLHPb4umYSYkJEClUmHdunU2G8xlZ2dj4MCBuHbtGtq3b2/1mb//+te/4OTkhAcPHuDgwYNWLZtIKQx98yeimie1VqtUbkGp97ehVjh911PiGsAW9Y2WlJTAzc0NoaGh1qqPrOTm5uKZZ57BqVOnEBYWhpiYGIMzYU3l6emJsLAwnDp1CklJSVYtm0gpmGhWmcztRmf3e+0klW5G3/JeoX7uuHxT9/s46dZD6OtjTLr1UG8rnDq9jdIm1ljUQhcSEoLCwkLFTnKQ8ujRIzz33HP4/fffERoaiv3798PX17dKruXk5ATgcQBNRJVJfRMneVJ3o5/JyEZ+calmTFVM4k3N/iHL49Fq7l4MWR6vtV3qPKo5Uq1iU3s1R8VVM9XBl9T729Za4aRY1EI3duxYzJ49G3v27MGgQYOsVadar6CgAIMHD8avv/6KZs2a4cCBA/D3r5oXSElJCS5evAgAaNKkSZVcg0jumGhWecwdQK/EdBRKoq9VTCq3oAD0vr+l9kldT4ksmhRRXFyMyMhIZGZm4sCBAzbR9VpcXIyhQ4fil19+QePGjfHrr79Kpgux1JdffonXX38dDg4OuHbtGho2bGhWOZwUQUoXk3iTiWYVxNwB9JdvPpRNOgp6zJgucqn3t9Lf+8Y+vy0K6NavX4+HDx9i3rx5yM/Px/Dhw9GxY0e4u0t3c4wfP97cS9ao0tJSjBo1Ctu2bYO/vz8OHTqEFi1aGHVuYGAgAOCTTz7B8OHDNdv37duH/fv34+WXX9YKiIuKivD1119j5syZKCoqwuuvv44vvvjC7LrbYkDHcTTKxPtqG4YsjzcraJMK9na+1rVK6krm4wx1w6oloLOzs4Pq/3d6q5MGG6JSqWQ7FmzTpk2IiooC8DhAa9y4sd5jv/jiC4SHh2v+rf7brF27FhMnTtRs/89//oO//vWvAAA/Pz9Nt+qlS5c0yZpfeOEFbNy4UTOWzhy2FtDxQ0KZeF9tR0ziTb1pJfQNoFenuJBLOgqSDtwZgD9WLYmFmzZtalQQpxSFhYWa/09LS0NaWpreY7OzK79AdWnfvj3mzp2Lo0ePIikpCRcvXkRxcTHq16+P/v37Y+LEiXj++ectrbrN4TgaZeJ9tR3mjqnqz3VeZcWUGeqGWudtvfXe4qW/SB5srYVOavwNx9HIF++rbSv/wPbzcAYA3H5YyKBNxoxtoTNm2TCltt5XSwsdUW2lxGVdSPq+2vq3c6Wr+MBOu/dIMQ9sW2bsDHVDrfNV2Xovl88Wi/LQEdVWUjmNSL703dcuIb7MPaZwXA2kdtKXD9BYunLFTenxONFw+TINdc1amlxcCXkNrdblevfuXcTGxuLq1at49OgR5s2bZ41iyUpsrcsVUP5Udlul675KDZLnwGplsKS7XS4tLHJTFd2c+sps5lMHafceVTpe/R63ZHKF1O+xohZ8tlRbl2tJSQlmzZqFFStWoKioSLO9fECXlZWF4OBg5Ofn4+LFi5oUHkRVyZYSStoSXfd1+ubTOo/l0l/KYWgYhb6greLDunwSYn4+WMZQN6c5gbS+MoHHQZa+rllLkotL/R5yWlbQ4i7XESNGYNmyZSgqKkJYWBgcHCrHiN7e3oiKikJRURGio6MtvSQRkRYu/aV8UsMopLrF2FVbdaSCHXOXbtNX5u2HhZLLeBmzzJep10y69VBWny0WBXSbN2/Gzp070aBBA5w4cQJnz56Fj4+PzmNHjBgBAIiNjbXkkkRElXDMpPJJPbCV0sIiN1LBjtQ9kQr2DK3NuvO1rji/8BnsfK1rpSE0UvvNvaacPlssCujWrl0LlUqFxYsXayXR1SUiIgIqlQrnz5+35JJk4ywdgEvKZGuLcNs0IR53qwkBde+aUlpY5EYq2JG6J1LBnqGWWEOf//qOMfeacvpssWhSRIMGDXDv3j3k5ubC1dUVANCwYUPcvn0bpaWVB6/6+PggPz8f+fn55teYzKKESRHGDMA1Z8wGB0zLB++V7TJ34DpXjqha+iafmbt02/mFz+gsUwBGff7rO2bG5tMmX7O2vD6qZekvZ2dn1K1bF/fv39dskwrovLy8UFRUhEePKs9UoaqlhIDO0Cwmc2ZcKTkZpdIYk1iUwZ5ySb3/DQVttflhrVTmLt2mb+aoMbNYpY6BEDU+W9Vc1TLL1dvbG3fu3EFBQQFcXFwkj71x4wZycnLQrFkzSy5JNszQWBhzEktyKSn5MDS4nTMZlUFfYC71/pdaJgzgjPeaYO7SbebcfzWpY5aObGv2LFi5sCiga9euHWJiYhAbG4uBAwdKHvvtt98CADp37mzJJcmGGUpbYGjGlbkfElQ7mDsmhw9y+ZBKMWLo/a8vaGPLbc3Rd0/0BXsVu1VNuf+A9DPCUNCvBBZNihgzZgyEEJg7dy5yc3V/2ALA3r178f7770OlUmHChAmWXJJsmKHZRvoGPzdwdzZrdhPVLlL3ioG5Mlh7sLycsvzbGl0zUs29/2qGjjE0S1buLArooqKiEBkZiZMnT6JTp05Yvny5Jrnwvn378M0332Dw4MEYNGgQioqKMGjQIAwYMMAqFSfbY2i2kb43sy6mfEhQ7SB1rxiYK4Mx3aoV3//qVh3moKt9TM1KYM79V4+RHLI8HjM2n0YznzoI9K1T62ekVgWLl/7KysrCX//6V/z6669Q6Xl6CiHQt29f7NixA25uuj94qWopYVKEMXQNfp4u49lNpE3fvZIagC0AdrnJhDnLN1kym5KqjjkTzgxNfDFmJRBjriM31TLLVa2srAw//PAD1qxZg2PHjqGwsBAA4ODggIiICLz66qsYO3Ys7OwsXpiCzKTUgM6Y8TGWrPFH8mFuqgOqPaQCc31fsqTWeA1toHtMFd/7Vc+cz119939K9xB8dUi7VbU2rbVa1ao1oCuvrKwM9+/fR2lpKXx9fXUuBUbVT4kBnbHfzMx5SJAyMJiXn/KBeQN3ZwDArZxCs76wMQddzZEKtKVaR3V9MZNKc2ILrbA1FtBR7aTEgM6YvHTq1js/j8cPhtsPC9mtqlC6WmsNJROl2staX9g4pKJmWPPLlDmtsIG+deDp6qi390ZOs58Z0JEWJQZ0Um/ypSPbsqvNhuh7+DfzqYO0e5UTmbOFrvbjFzZ5M3dcq65Ay9SVQHSpmIRcTs+HakksXF5mZibOnj2LrKwsFBcXSx47fvx4a12WbJhUziHmJbMt+u438PiDWsnJRJXKUF7J8g/ktHuPavUD2RaZk2sOevZN6R6Cs9ezdb6P++u4Tvajokpf5Mp//iv1+WBxQHfq1ClMnz4dhw8fNup4lUrFgI6sYmqv5nozf0/ffFrnOcxLpkz6Hv63HxYqPpmoUvELm/zpSiw8ZHl8peM0qWR0NLMJARxJuWfSSiCt5u7VWR/1579S81ZaFNCdOnUK3bt3x6NHjyCEgLOzM+rVq8eJEFQtpDJ/G5NVnJTDUIZ4qbV85TKOxtbwC5sySQVT+rpN1XnojH1vGvr8V+rzwaLI691330VeXh5CQkKwatUq9OjRg6lJqFrpe5NLPQwq4kO99jH1nphyv8tfg+u/1l78wqZMkvdOCLPua8XPiy4h9fR20QLmfV7IgUWTIry8vPDw4UOcOHEC4eHh1qwXWZkSJ0UYYszsNrkNjrUF5t4TU2czMqWJfDEVkXwZmixh6kQKfZ8XU7qH4EjKPb2fB3Ka/Vwts1w9PT0hhEBOTo65RVA1scWArjx9LT58qNc+VXFPmNJEecx9ILNFvuZJ3TtTE4QzsfCfLAro2rdvj/PnzyM3Nxf29vbmFkPVwJYDOqkWHz7Uax9zE5Lqw5QmpMYWeXmy9eXdjH1+WzTgbeLEiSgqKsLOnTstKYbIYlKLQEvNiOOi7rWP1D2Rus/69hlKaVKeEsbRKImpi7sbOk/qs4BqL6mJFPwM/5NFAd3UqVPRt29fTJ48GUePHrVWnYhMov7WfSYjG/nFpZrB7eoPcakPg6m9mvOhXsvouyddgn313mep14ChlCZtArxQx8kebQK8OAarFjH0vjbnPKWmq5AbUwN1qaCNn+F/smiWq729PX766SfMnDkT3bp1Q2RkJDp06AB3d+nIeN68eZZclkiLoZxUhlJaME9Z7aLvnnwp1bqiJ3+VuhXWnJQmVLMMva/1jYUz1CLP2bE1y5zZ5VKzUnUlFrbVz3CLl/765Zdf8PrrryMtLQ2qimGyHqWllfu75SY2NhZLlizBsWPHkJubi2bNmmHEiBGYPXs26tata1aZ27dvx/Lly3H69GkUFRUhNDQUY8eOxfTp0+Ho6GhRfZU8hs7QmCvOiFMGqfssBCSXgeP9lx9zl/aTGhfL10LNM2VJt4qzWW01aKuWSRG//fYb+vTpg9LSUggh0Lx5c/j5+RlMLBwbG2vuJWuFL774AtOnT4cQAk2aNEH9+vVx/vx5FBYWolWrVoiPj4ePj49JZc6cORNLliwBAISEhKBu3bpITExEaWkpunfvjv/+979wdnY2u85KDuiMmRVpyx8GSiF1n/Xlr9J6SPD+y4q595uvhdqtKtbgVvrM5WoJ6Pr164cDBw7g6aefxubNmzUXVbKEhARERERACIGVK1filVdegUqlQmZmJgYPHoyEhAQMGzYM27dvN7rMH3/8EcOGDYOzszOio6MxePBgAMDFixfx7LPPIjU1FW+88YYm4DOHkgM6fS1wU3qE4EjyXcW+yW2Nufmr+LCWJ6n7PZ2tcLJlyRczXWxh5nK1zHJNSEiASqXCDz/8YBPBHAC8//77KCsrw7hx4/Dqq69qupkbNWqETZs2wc7ODjt27MDZs2eNLnPBggUAgFmzZmmCOQB44oknsHr1agDAl19+iTt37ljxN1EO9Zir8oPbp/QIwVdxV0weUE21l677rH5IS+0jeZK6p1KD5PlaqN2kJjGYM2mFM5f/ZPFKEQDw4MEDK1WndsvNzUW9evVQWFiI+Ph4dO1a+RtDv379sH//frz77rt4//33DZaZlJSEFi1aAAAyMjLQuHHjSseEhoYiOTkZq1evxqRJk8yqu5Jb6HRhwmAi5eK4WHnT1+0t9bk9tWeIzm5Va+etrI2qpYWuVatWyM/PR2FhoSXFyMapU6dQWFgIZ2dnRERE6DwmMjISAPD7778bVab6uODgYJ3BnDllknSqEiKSN6lWOHNz11H1GRDmj52vdcX5hc9g52tdNUG43pRFIfpTFjEP3Z8sSlsyefJkvPTSS/j+++/NbjmSk8uXLwMAmjZtqnfWaUjI49w3ly5dMqlM9XmWlCnV7Z2eno6AgACj6qQETE9ApAz6BrzrSjljTkoMqj3MSVkkldLE1lgU0E2cOBGHDh3C9OnTUbduXYwaNcpa9aqV7t+/DwCSM1jV+7KysmqsTJLOW0TypPSZbFSZqQGaodx1VPvpCtSnbz6t89ikWw91BoFdQnzxZWwypm8+bVOfFRYFdC+99BJUKhWcnJwwZswYzJkzB08//bRkYmGVSoU1a9ZYctkaU1BQAABwcnLSe4w6tUh+fn61lynVv24rk1bUmDBYWdjyYptMDdA41EKZDPW4lA8CbfmzwqKAbt26dVCpVFDPq7h69SquXr2q81j1cXIO6FxcXAAARUVFeo9Rjyd0dXWtsTLpMa4CoByWtLywZU++pAI0XfeVQy3kydB71JQeF1tupbUooBs/frzRq0Mogbe3N4A/u0l1Ue9TH1sTZRIpjakPdn5bVwZ9AVoDd2ed93VK9xCcvZ7NoRYyYsx71JQeF1tupbW4hc6WqNOLXLt2DcXFxTonRly5ckXrWGPLTE6u/K3C3DKJlMbUB7v6YWDL39aVQF/LjC5CAEdS7nGohcwY+x7VNwmGrbR/sihtia0JDw+Hk5MTCgsLcfz4cZ3H/PbbbwCAzp07G1Vmp06dAACpqam4fv26VcokUhp96Qx0KZ9U1Ja/rSuBvvQkt3J0p8pSD5LXlRKDaidD71F9aWjULXsVU5l0CamnN3Gx0jGgM4G7uzsGDBgAAFi1alWl/UlJSTh48CAAYPjw4UaV2aJFC7Ru3VpvmQcPHkRycjKcnJy0VpEgsiXmPNgBMEeVAugK0HhflUPqXuoL2tQtcxWVb6W1xZVCGNCZaO7cuVCpVNiwYQNWrVqlmRBy48YNjB49GmVlZRg6dCjatGmjdV5gYCACAwOxbdu2SmW+9957AICPPvoIP/30k2b7pUuX8PLLLwMApk6divr161fVr0VU65nzYJdaZojki/dVOaTupVR3rFTLnq220hq99Ffv3r0BAM2aNcPatWu1tpl0QZUKBw4cMPm82mTZsmV44403IIRAQEAA6tWrh/Pnz6OwsBAtW7ZEfHw86tWrp3WOevLI2rVrMXHixEpl/t///R+WLVsG4HEiYTc3N5w7dw6lpaXo1q0b9u3bp5kRaw5bW/qLbIMxS0DpW2aI5I33VTn03UupZb1CG+geK6fE5R2NfX4bHdDZ2T1uzHviiSdw/vx5rW2mUKlUKC2tfIPk5sCBA1iyZAmOHTuGvLw8NGvWDMOHD8ecOXPg5la51cBQQAcAW7duxZdffonTp0+jqKgIzZs3x9ixY/F///d/elemMBYDOlIqPtiJlMnQ2q62sp6v1QO6BQsWAADq1auH1157TWubqdRdjFR9GNCR0jHfHJGyGGqBt5Uvc1YP6EjeGNCRklXMZQU8/uBnvjkieVMHbRdv5MBepUJJmUCrhu429YXN2Oc3J0UQUa2kL12BLlKDp4lIvgaE+WNqzxAUlpThUXEpikrLtGa70p8sCugWLlyITz/91OjjP//8cyxcuNCSSxJVCVOCB6p6UukKdGG+OSLl4hc241jU5WpnZwd/f39kZmYadXxQUBCuXbumiEkRcqOELldTx0gZezy762ofqcHQumawmXo8wDF3RHIhNdv1/MJnaqBG1YtdrqQoprbYmHI8v/3VPqZmj9eXHb5LsK9JWebZMktU+zCRtHGqNaC7f/++RbnUyHaZGnSZcjy762ofU7PHr/z1CqZ0D9HKDj+lewi+OnTFpCzzDOKJah8mkjZOtQV0W7duxcOHD9G0adPquiQpiKlBlynH89tf7WNO9vgjKfe0ssMfuXJX53GGsswTUe2ib+k/JaYosYSDKQd/9tln+Oyzz7S23blzR9O/q4sQAg8ePEBOTg5UKhWee+4582pKNq2Fn+6s4PqCLqnjK46d6hJSD2evZ1fKdcRvfzVH/QGuK8fU9M2ndZ5TMRiTCtpMfT0RUc0aEObPMa4GmBTQPXjwAGlpaVrbSktLK23Tp0+fPpg3b54plyQC8LjFRleCyS4hj8dIVRzYrvf4YF+tCRBnMrJx9no2pnQPwZGUe4pPUCkn+j7AjQ3GpI7Tl2WeQTwRyZVJs1zPnDmD06dPA3jc8vbSSy/B09NTswapLnZ2dvDw8MBf/vIXhITww7KmKGaWa7kWmy4hvviqwpin8rNTdWUR/zI22WbW/1Mqfdnjp3QPwZErd7VaXlf+esXms8wrFWcpk5rSXwvVslKEqWlLqOYoIaCryJxUFbY+/V0pKgX3wb746lDl4J4tr8rEVEPKZE5gZguvBWOf3yZ1uVZUVlZmyelEFjEmtUXFDweOnVKGit2xQ5bHVzqm/EQJUhapWcpKeYjbmoqBmXpWuqHAjK+FPzEPHcmWqaktpnyfoDdfGcdOyRtnrdoW3m/lMZRKSN9qPnwt/MmiFrryysrKkJSUhPv376O4uFjy2O7du1vrsmTD9E18UI+Vq0jdYqNv9iTJF1tebQvvt/JIBWZSrXd8LfzJ4oDuxo0bmDNnDrZt24b8/HyDx6tUKpSUlFh6WSKzU1tw+rvySAX3pDy838ojFZhJtd7xtfAniwK6zMxMdOzYEZmZmTB2boUFczCIKrE0tQUpg1RwT8rD+608UoGZoS/ofC08ZlFAN3/+fFy/fh3u7u744IMPMGTIEDRq1Aj29vbWqh+RWfitTZmkZsHpC+6VntLAVrGlXVmkAjNDX9D5WnjMorQlAQEByMzMxJYtWzB8+HBr1ousTIlpSwxhnjFlMSc9gS2kNCBSOn25J21l+a9qyUPn4uICIQTy8vLg4GC1+RVUBWwxoCNlMSfvoDnnEFHtY8tf0KslD12DBg2Qk5PDYI6Iqpw5eQeZ0oBIGditaphFeej69u2Lhw8fIikpyVr1ISLSyZy8g34eznrPIdugL38ZkdJYFND985//RN26dTFr1ixr1YeISKepvZrrTQqtL62B+hhd55By6Ava9AX6DOpIiSwK6Jo3b45du3bh0KFD6NevH2JjY5GXl2etuhERaahnwbUJ8EIdJ3u0CfDSDIrW17V6+2Gh3nNIGaSCNkOrDxApiUWD38qnJzl48CAOHjxo8BwmFqbagKks5MmcvIMce6NsUkEbx1CSLbGohU4IYdYPkbWYMz6G3TDKI9UdS8omFbRJjbskUhqLWuhiY2OtVQ8ik0mt7zcgzF9vK5zUN3q25MgTs8UrkzEt6VKts1N7hjDBuEKwV8Uwi/LQkXwoMQ+dVI6xqT1D9CaUnbH5NPKLSyudV8fJHucXPlNl9SUi4xmbFNpQ0llbzl+mFLaeILxa8tAR1SSprhapVjiu86psUt/k+S1fPoxtSTfUOssxlPLHXhXjWDSGriIhBO7evYtr165Zs9ha4cyZM1iwYAF69+4NPz8/ODo6wtvbG5GRkVixYgWKi4vNKnf+/PlQqVSSPytXrrTyb6MMUuNjpII9jreSP3PSVHDspLwYk0ha/RpYEZuMqT1DcH7hM9j5Wle2wCkMJ7cYxyotdCdPnsSiRYuwf/9+5OXlVZrJmpWVhdmzZ0OlUmHp0qVwdXW1xmWrzZUrV9C2bVvNvwMCAtC2bVtkZGQgPj4e8fHxWLduHWJiYuDt7W3WNRo0aIDQ0FCd+xo2bGhWmUo3tVdzveNjvoxNlpz1yPFW8iU1dlIyTYWO0SX8ll97SbWkGxo/S8rCXhXjWBzQbdiwAS+//LJkC5W3tzeuXLmC2NhY9OzZE6NGjbL0stVKCIEGDRrgH//4B8aOHYtmzZpp9v3yyy8YP348/vjjD0yZMgVbtmwx6xoDBw7EunXrrFRj2yAVmAlAcjA0u2Hky9w0FfpGC/Nbfu1k6AtbRQzOlUvqtUB/sqjL9fz583jllVdQXFyMf/zjHzhx4gTq1aun89gJEyZACIE9e/ZYcska0aRJE6SmpuKdd97RCuYA4Nlnn8Xnn38OANi+fTvu3btXE1W0WQPC/LHzta6VulqkktCSvJmbpoIpLOTFnETSDM6ViZ/nxrGohe7TTz9FUVERXnvtNSxbtgyAdrLh8vr06QMASEhI0Lm/NnNxcZHcP3DgQABAaWkpkpOT4evrWx3VIgPYCqdM5qapMNRqS7WPOYmkSZn4eW6YRS10sbGxUKlURq3l2qhRI7i6uiI9Pd2SS9ZK+fn5mv+vU6eOWWWcOXMGUVFR6N27N4YMGYK5c+ciMTHRWlW0SVyUW5mkJrVIfZPnt3zl4MQmososykPn6uoKR0dH5OTkaLY1bNgQt2/fRmlp5Txf9evXR3Z2NoqKisy9ZK20ZMkSzJw5E76+vrh165beVkpd5s+fjwULFujcp1KpMH36dHzyyScmlamLEvPQSbH1vEVKpyu3mACYksSGML8c2Qpjn98WBXReXl4oKChAfn4+VP//65K+gK6wsBBubm7w9vbG7du3zb1krXPt2jU89dRTyM7Oxr///W/Mnj3bpPNXrVqFW7duYcCAAQgODoa7uzsuX76MFStWaNKVvPXWW/j4448NlqW+6bqkp6cjICDAZgI6qaTDO1/rWgM1oqrEAJ6IlMrYgM6iLtfg4GAUFxfj8uXLBo+NiYlBaWkpwsLCLLlkrfLo0SMMHToU2dnZiIiIwJtvvmlyGa+++irmzp2LiIgI1KtXD87OzmjdujW++uorfPTRRwCApUuXIi0tzcq1VzYOmrYtkulKiIhsgEUB3bPPPgshhGZChD4PHz7U5KEbPHiwJZc02YwZMwwm7tX107NnT8lyi4qKMGzYMJw6dQqBgYHYsWMHHB0drVr3N998E40aNUJJSQl27dpl8PiUlBS9PwEBAVatW22hb5yc1IxGjq1THgbwRGTrLAroZsyYAU9PT6xatQpz587FgwcPtPbn5+djx44diIiIwMWLF+Hv749XX33VkkuazM3NDb6+vib/eHp66i2zpKQEI0eORExMDBo1aoQDBw6gcePGVq+7vb09OnbsCABISkqyevlyJ5X5X9+g6S4hvlwtQCZMCbzNSUnCwJ6IlMSiMXQAsH//fgwZMgQFBQVwcHBAWVkZysrK4Ofnh7t376K0tBRCCLi5uSEmJgadO3e2Vt1rRGlpKaKiohAdHY369evj0KFDaNWqVZVdb9SoUdiyZQumTJmCr776yuxylDgpwtA4OV2DpvWtIMGxdbWLqWPiDC3Qbmn5RFSzbHkd5moZQwcAffv2xe+//46ePXuiuLhYE8DdvHkTJSUlEEKgZ8+eOHr0qOyDubKyMvztb39DdHQ0vL29sW/fvioN5gDg3LlzAB4nNyZtRnWzCfH4IS8EhLHnUI0zdUycqSlJOOaOSD64DrNxrLKWa+vWrXHgwAFcvXoVhw8fRmZmJkpLS+Hv74+uXbuiefPm1rhMjZsyZQo2bNgADw8PxMTEoE2bNlV6vd27d2ty0fXv379KryVH5qz12MynDtLuPdJ5DtUe5gTepiQeZWBPJB9SX8BspZXOGFYJ6NSaNWtWaWkspXjjjTfwzTffwM3NDXv27EGHDh2MPrdbt27IyMjAjBkzMGPGDM32xMREfP7555g6dapWcFhWVqbpZgWAQYMGmXQ9W2HOWo/qY7haQO1W1SsBcKUBIvngFzDjWBTQFRUV4ebNm3BycoK/v3aUnJubi/nz52Pfvn2ws7PDoEGD8M9//hOurq4WVbgmHD16FEuXLgUAuLu74+2339Z77DvvvKNZCkwtIyMDV69erTRppLi4GKtWrcKqVavg4+ODZs2awcHBAcnJycjKygIAREZGYsOGDdb9hRRC3c2mK7no9M2ndZ5z+2Gh3nOo9rD2YtwVx990CamHs9ezGdgTyQC/gBnHooBu9erVmDZtGiZMmIBvv/1Wa99zzz2H+Ph4qOdcnD17Fr/99ptmuTA5KSws1Pz/jRs3cOPGDb3H3rp1y+hyAwMDsWjRIhw9ehQXLlxAcnIyCgoK4OPjg4EDByIqKgqjR4+2eJUIJTNnrUeuCVj76QvWBR5PhtE1MFrfoGld3e9nr2djSvcQHEm5x8CeqJaz9hc8pbJoluuQIUPw888/IyYmBn379tVs37VrF4YOHQo7OzuMHj0arq6uWL9+PYqLi7F27VqMHz/eKpUn4ylxlqsUU2c9Uu0nNTMVgN59KzizmUj2bHmpN2Of3xa10F24cAEA0L59e63tGzduhEqlwqxZs/DBBx8AANq1a4epU6di48aNDOioykl1x5I8Sc5M1fG9VL2P42+I5I89K4ZZ1ELn7e2NkpISPHyo/cGozkGXnJyMoKAgAEBeXh7c3d3h5+cn2WVJVcPWWuhIeVrN3Yv84tJK2+s42UMI6N0X2kB39ztb6IhIDqolD11eXh7s7LSLSEtLw507dxAQEKAJ5gCgbt268PLywv379y25JBHZKKnVIKT26Vs1hONviEhJLArofHx8kJubqzV78+DBgwCALl26VDq+pKQEbm66P3iJiKRIBWZS+0xNOkxEJEcWjaFr164dYmJisGbNGrz55psoKyvDmjVroFKp0KtXL61j79y5g9zc3CpfWYGIlMnQuEipfRx/Q0RKZ1FAN2HCBOzduxezZ8/G/v37cefOHZw8eRLu7u4YMWKE1rG//fYbADCgIyKzSQVmDNqIlMuW13I1lkVdriNHjsTEiRNRWlqKmJgYnDx5Ei4uLli5ciW8vLy0jt2yZYvOljsioqoWk3gTQ5bHo9XcvRiyPJ5rQBLJCNdyNY7FS399++23mDRpEo4cOQIvLy/06dNHMyNDraioCJ6enhg/fjyeffZZSy9JRDZK6lu6KYmFp3yfgJVj2/MbPpEMcC1X41iUtoTkg2lLSO6YWJjINkmlLDq/8JkaqFH1qpbEwkRE1YWJhYlsE9dyNY5FY+iIiKqLVGAmtU8qRx0R1X7MJWkcBnREJAtMLExkm5hL0jjsciUiWZjaqzmmfJ+g1buqDswEoHdff67rSyR7TEtkGCdF2AhOiiAliEm8qTcwk9pHRCRXxj6/GdDZCAZ0RERE8mPs85tj6IiIiIhkjmPoiIhIdrgUFJE2ttAREZGscCkoosoY0BERkaxIJpkmslEM6IiISFa4+gdRZQzoiIhIVrj6B1FlDOiIiEhWuPoHUWUM6IiISFa4FBRRZUxbQkREssOloIi0sYWOiIiISOYY0BERERHJHAM6IiIiIpljQEdEREQkcwzojLRu3TqoVCrJn9mzZ5td/qlTpzBy5Ej4+/vDxcUFwcHBmD59Ou7cuWPF34KIiIiUiLNcTeTh4YHWrVvr3BcYGGhWmTt27MCoUaNQXFyMBg0aICwsDJcuXcLnn3+OrVu3Ij4+HsHBwRbU2jZx8W4iIrIVDOhMFB4ejri4OKuVd/36dYwbNw7FxcWYO3cu5s2bBwcHB2RnZ2PUqFHYu3cvRo4ciePHj0NVMZMm6aVevFtNvXj3yrHtGdQREZHisMu1hi1evBiPHj1C9+7dsXDhQjg4PI6xPT09sXHjRnh6euLEiRP4+eefa7im8sLFu4mIyJYwoKth27ZtAwC8+uqrlfZ5e3tjxIgRAIDo6OhqrZfccfFuIiKyJQzoTHTt2jVMnDgRffr0waBBg/DWW2/h999/N6us9PR0XL9+HQDQvXt3ncdERkYCgNnXsFVcvJuIiGwJAzoTpaam4rvvvsPBgwexe/dufPLJJ+jcuTNGjx6NR48emVTW5cuXAQBOTk5o0qSJzmNCQh4vNp2SkoLi4mLLKm9DuHg3ERHZEk6KMJK3tzfefvttPP/882jevDl8fHyQlpaG9evX46OPPsLmzZtRWlpqUtfo/fv3NWXrm/Dg4+MDACgrK0NOTg58fX31lic1EzY9PR0BAQFG103u1It3r4i7gqRbDxHq547XeoZw8W4iIlIkBnRGGjJkCIYMGaK1rUWLFli0aBGeeuopjBw5UpNipFu3bkaVWVBQAOBxC50+zs7Omv/Pz883o+a2i4t3ExGRrVB8l+uMGTMMJgTW9dOzZ0+jr/Hiiy+iY8eOAB7nlDOWi4sLAKCoqEjvMYWFhZr/d3V1lSwvJSVF748ttc4RERHZGsW30Lm5uUl2U+rj6elp0vFdunTBsWPHkJSUZPQ53t7eAICsrCwIIXR2u6q7Ze3s7ODh4WFSnYiIiMg2KD6gW7RoERYtWlTl11F3m5aUlBh9TosWLQA8bqFLT09H06ZNKx1z5crjvGlBQUFwdHS0Qk2JiIhIaRTf5Vpdzp07BwB6Z6vq0rRpUzRq1AgA8Ntvv+k8Rr29c+fOFtaQiIiIlIoBnRWcPXsWe/fuBQD079/fpHNfeOEFAMCqVasq7cvKysLWrVsBQJNgmIiIiKgiBnRGyMnJwciRI3HkyBEIIbT2xcTEYODAgSgtLUWbNm0wbNiwSuePGjUKgYGBmDlzZqV9b731FlxdXfHrr79i3rx5KC0tBQBkZ2cjKioK2dnZCA8Px/PPP181vxwRERHJnuLH0FlDWVkZoqOjER0dDXd3dwQHB8PZ2RlXr17FrVu3AABhYWHYtWsX7O3tK51/8+ZNXL16FXfv3q20LyAgAOvXr8fo0aPx/vvv4+uvv0ZAQAAuXryIvLw8+Pn5ITo6Wm+eOiIiIiK20Bmhbt26WLx4MYYNG4aGDRvi6tWrOHnyJEpKStCrVy98+eWXOHHihM5JDcYYPnw4jh07huHDhwMA/ve//6F+/fp4/fXXcfbsWTRv3tyavw4REREpjEpU7EMkRVKvIpGSklLDNSEiIiJjGfv8ZpcrKUZM4k2siE3G5Vu5aOHnhqm9mnOlCCIisgnsciVFiEm8ickbEnAmIxv5xaU4k5GNKd8nICbxZk1XjYiIqMoxoCNFWBGbXGmbEMCKuCs1UBsiIqLqxYCOFOHyrVyd25NuPazmmhAREVU/BnSkCC383HRuD/Vzr+aaEBERVT8GdKQIU3s1R8VUfSoV8FrPkJqpEBERUTViQEeKMCDMHyvHtkebAC/UcbJHmwAvfD22PfpzlisREdkApi0hxRgQ5q+VpiQm8SaGLI9nGhMiIlI8ttCRIjGNCRER2RIGdKRITGNCRES2hAEdKRLTmBARkS1hQEeKxDQmRERkSxjQkSIxjQkREdkSBnSkSExjQkREtoRpS0ixKqYxISIiUiq20BERERHJHAM6IiIiIpljQEdEREQkcwzoiIiIiGSOAR0RERGRzDGgIyIiIpI5BnREREREMsc8dKRYMYk3sSI2GZdv5aKFnxum9mrOvHRERKRIbKEjRYpJvInJGxJwJiMb+cWlOJORjSnfJyAm8WZNV42IiMjqGNCRIq2ITa60TQhgRdyVGqgNERFR1WJAR4p0+Vauzu1Jtx5Wc02IiIiqHgM6UqQWfm46t4f6uVdzTYiIiKoeAzpSpKm9mkOl0t6mUgGv9QypmQoRERFVIQZ0RlKpVEb99OrVy6Ry161bZ7DM2bNnV9FvpVwDwvyxcmx7tAnwQh0ne7QJ8MLXY9ujP2e5EhGRAjFtiZG6du2qd58QAkeOHAEAdOnSxazyPTw80Lp1a537AgMDzSrT1g0I82eaEiIisgkM6IwUHx+vd9+hQ4fQs2dPAMDEiRPNKj88PBxxcXFmnUtERES2jV2uVrBu3ToAj1vxQkNDa7YyREREZHMY0FkoLy8P27ZtA2B+6xwRERGRJdjlaqFt27YhNzcXderUwYsvvmh2OdeuXcPEiRORnp4OV1dXtGrVCi+88AI6depkxdoSERGREjGgs5C6u3XYsGHw8PAwu5zU1FSkpqZq/r1792588sknGDVqFNasWYM6depYWlUiIiJSKAZ0FkhNTcWhQ4cAmN/d6u3tjbfffhvPP/88mjdvDh8fH6SlpWH9+vX46KOPsHnzZpSWliI6OtpgWcHBwXr3paenIyAgwKw6EhERUe2mEkKImq6EXM2fPx8LFixAs2bNkJqaClXFTLYWio6OxsiRIwEAv/32G7p16yZ5vDEBXUpKilXrSERERFVH/Ww39PxW/KSIGTNmGJ0UuPyPOg2JPkIIrF+/HgAwfvx4qwdzAPDiiy+iY8eOAIAdO3YYPD4lJUXvD1vniIiIlEvxXa5ubm7w9fU1+TxPT0/J/YcOHdK0yk2YMMHc6hnUpUsXHDt2DElJSVV2DSIiIpI3xQd0ixYtwqJFi6xernoyRGRkJEJCqm59UCcnJwBASUlJlV2DiIiI5E3xXa5VITc3t9pyz507dw4A0KRJkyq9DhEREckXAzozbNu2DXl5eahbty5GjBhRZdc5e/Ys9u7dCwDo379/lV2HiIiI5I0BnRnU3a3Dhw+Hm5ubweNHjRqFwMBAzJw5U2t7Tk4ORo4ciSNHjqDiZOOYmBgMHDgQpaWlaNOmDYYNG2a1+hMREZGyKH4MnbWlpqbi119/BWB8d+vNmzdx9epV3L17V2t7WVkZoqOjER0dDXd3dwQHB8PZ2RlXr17FrVu3AABhYWHYtWsX7O3trfp7EBERkXIwoDPRd999ByEEgoKC0KNHD4vKqlu3LhYvXoyjR4/i3LlzuHr1KnJzc+Hp6YlevXph+PDheOmll+Di4mKl2hMREZESMbGwjTA2MSERERHVHkwsTERERGQjGNARERERyRwDOiIiIiKZY0BHREREJHMM6IiIiIhkjgEdERERkcwxoCMiIiKSOQZ0RERERDLHgI6IiIhI5hjQEREREckcAzoiIiIimWNAR0RERCRzDOiIiIiIZI4BHREREZHMMaAjIiIikjkGdEREREQyx4COiIiISOYY0BERERHJHAM6IiIiIpljQEdEREQkcwzoiIiIiGSOAR0RERGRzDGgIyIiIpI5BnREREREMseAjoiIiEjmGNARERERyRwDOiIiIiKZY0BHREREJHMM6IiIiIhkzuYCugcPHiA6OhpvvfUWevbsCXd3d6hUKgQGBhp1vhACq1evRqdOneDh4QEPDw906tQJq1evhhDC7HoVFxdj8eLFaNOmDerWrQtvb2/06tULO3bsMLtMIiIisg0ONV2B6hYXF4eRI0eadW5ZWRlGjRqFrVu3AgCefPJJAMCxY8dw7NgxHDhwABs3boRKpTKp3IKCAvTr1w/x8fGwt7dHWFgY8vLyEBcXh7i4OMyaNQsffvihWXUmIiIi5bO5FjpXV1d0794db775JjZt2oQvv/zS6HM/++wzbN26FT4+Pjhy5AgSExORmJiII0eOwMfHB5s3b8by5ctNrtOsWbMQHx+PoKAgJCYm4syZM0hOTsbOnTvh7OyMjz76CD/99JPJ5RIREZFtUAlL+gkV4Oeff8bzzz+PZs2aIS0tTe9xxcXFaNSoEe7evYtvv/0Wf/vb37T2f/vtt5g0aRIaNGiA69evw8HBuMbPW7duoWnTpigqKsLBgwfRq1cvrf3z5s3D+++/j3bt2iEhIcHk308tODgYAJCSkmJ2GURERFS9jH1+21wLnbni4uJw9+5duLm5ISoqqtL+MWPGwM3NDbdv38ahQ4eMLnfXrl0oKipCaGhopWAOACZPngwAOHnyJK5cuWL+L0BERESKxYDOSL///jsAICIiAs7OzpX2Ozs7o0OHDlrHmlJuZGSkzv2NGzdGUFCQyeUSERGR7bC5SRHmunz5MgAgJCRE7zEhISGIjY3FpUuXrF5uamqqwXLVzbK6pKamwsHBQfIYIiIiql3S09ONGsbFFjoj3b9/HwDg4+Oj9xj1vqysrBovtyKVSmX0uD4lSU9PR3p6ek1Xg6oJ77dt4f22LbZ6vx0cHHT2DFY6rhrqoggFBQUAACcnJ73HqP/g+fn5NVIuJzxUxskgtoX327bwftsW3m9psmmhmzFjBlQqlck/PXv2tMr1XVxcAABFRUV6jyksLATwODVKTZdLREREtkM2LXRubm7w9fU1+TxPT0+rXN/b2xvAn12kuqj3qY+tyXKJiIjIdsgmoFu0aBEWLVpUY9dv0aIFACA5OVnvMeq0IupjjS338OHDVi+XiIiIbIdsulxrWqdOnQAAf/zxh6YLtLzCwkL88ccfAIDOnTubXG58fLzO/devX0dqaqrWsURERETlMaAzUq9eveDr64vc3Fxs3Lix0v4ffvgBubm5qF+/Pnr06GF0uUOGDIGjoyOSkpIQGxtbaf/XX38NAAgPD0fz5s3N/wWIiIhIsRjQGcnR0RFz5swBAMycORNHjx7V7Dt69CjeeustAMA777xTKT3I77//jsDAQAQGBiIjI0Nrn5+fn2Y1iEmTJmnlmvvpp5/w8ccfAwDee+896/9SREREpAg2uZZrvXr1NP9fXFyMnJwc2NnZaU06GD16NL744gut88rKyjBixAjs2LEDAPDkk08CAM6fPw8AGDFiBDZv3gw7O+04OS4uTrOsV2pqKgIDA7X25+fno0+fPjh69Cjs7e3xl7/8Bbm5uZqxc2+++SY++eQTK/zmREREpESymRRhTffu3au0raysTGv7w4cPKx1jZ2eHbdu24ZtvvsHq1atx4cIFAECHDh3wyiuv4OWXX4ZKpTK5Pq6uroiLi8PSpUvxww8/4PLly3ByckKPHj0wbdo0vPDCCyaXSURERLbDJlvoiIiIiJSEY+iIiIiIZI4BHREREZHMMaAjIiIikjkGdFQtevbsCZVKhXXr1tV0VWqFtLQ0qFSqSjOeASAwMBAqlQppaWnVXq/aQurvU13WrVtn1fWglWzixIk29/6Oi4vj68OK1H9Pc97ztvj604UBncKNHTsWKpUKKpUKu3fvrunqWOzvf/87VCoVfv75ZwB/fgiof2JiYiTPf/fdd7WOp6qlDuQr/ri4uCAoKAhjx47VrLBCNUd9n+bPny95XPn3G9me9PR0zf1PTEzUeczdu3dhZ2cHlUqF559/Xm9Zb731VpUHxKdPn8b8+fNtJtBjQKdgOTk5+PHHHzX/lvuLWgiBn376CXXq1EHfvn11HrN+/XrJ8zds2FBV1bOakJAQtGzZEo6OjjVdFasJCAhA165dNT9BQUG4ceMGfvjhB3Tq1Alr166t6SqSBRo2bIiWLVvC09OzpqtSberUqYOWLVuiadOmNV2VahMQEICgoCAAwK+//qrzmF9//RXq5BmHDx9GWVmZzuMOHToEAFYJ6PS9/k6fPo0FCxbI/tlnLAZ0ChYdHY1Hjx7By8sLwOOVJ7Kysmq2UhZISEjA9evX0a9fP7i4uGjt8/Lygp+fH3788UedOQQBIDY2FteuXcMTTzxRHdU124EDB3Dx4kU0bty4pqtiNS+99BLi4+M1PxcuXMD169cxfPhwlJWV4fXXX8f9+/druppkpn//+9+4ePEi/vrXv9Z0VapNREQELl68KPklUonUS1tKBXQAEBwcjKysLPzvf/+rdExubi5OnTqlVZ4lbPH1pwsDOgVTfyt566230KJFCxQWFmLTpk01WykL7Nq1CwAwePDgSvvs7e0xZswY5OfnY+vWrTrP/+677wAAEyZMqLpKktF8fX2xZs0a2NnZ4dGjRzh8+HBNV4mIDDAmoHNwcMAbb7yh97gjR46gpKQETk5O6NSpU9VV1sYwoFOo5ORkHD58GCqVCmPGjMG4ceMA/BnUVFRxQOqaNWvQsWNHeHh4QKVS4cGDBwAeL122ePFi9O7dG82aNYOzszO8vb0RGRmJ1atX621eLy81NRVjx45Fw4YN4eLigieeeAIffPABCgsLJc/buXMn7OzsMGjQIJ37x48fr/d3zM3Nxfbt29GgQQM888wzeq+hHksUFxenc7/UQOhLly5h4sSJCAwMhLOzM9zc3BAUFITnnnsOX331leTvVp6+SRHqgb/z58/HgwcPMGPGDAQGBsLFxQWhoaH48MMPNX//wsJCLFq0CK1atYKrqysaN26MGTNmIC8vz+h6VAcPDw/NkntFRUVGnXPu3DksWLAA3bp1Q5MmTeDk5IR69eqhf//+2L59u+S5ZWVl2LhxI5555hk0aNAAzs7OaNKkCfr27YuvvvrK4GtQrbi4GFFRUVCpVGjVqhXS09ONOk+ppAalb9q0CX369IGvry8cHR1Rr149hIWF4dVXX8WxY8e0jp0/fz5UKhUmTpyIvLw8zJ49G82bN4eLiwsaN26MyZMn4+bNmzrrYO7rQv1+i4uLQ3JyMsaMGQN/f3+4uLigVatWWLx4sc7PNUOTIoqKivDVV1+hZ8+e8PX1hbOzM5o1a4ZBgwbp/RyWA3VAl5mZieTkZK192dnZOHPmDNq1a6f5nFV3rZan3hYREQFXV1ed14mOjkaXLl3g7u4ODw8P9O3bV++XPl2vv8DAQPztb3/TXK/8GF5dEy/OnTuHl156CUFBQXBxcYGXlxd69uyJjRs3Sv9BahNBivTuu+8KAKJ79+5CCCFSU1OFSqUSAMT58+crHR8bGysAiGbNmom///3vAoBo3Lix6NChg/Dx8RFZWVlCCCEmTZokAIg6deqI5s2biw4dOojAwEABQAAQI0aM0FmfHj16CADinXfeEd7e3sLR0VGEh4eL0NBQzbk9e/YU+fn5Os9PS0sTAETnzp111tvX11cIIUSbNm2ESqUSKSkpWsetW7dOABAzZswQ//vf/zTX1FfP2NhYnfVQX69Hjx5a20+cOCHq1q0rAAhXV1cRFhYmwsPDRf369QUA4enpqXV8amqq5u9dUbNmzQQAkZqaqrV9woQJAoCYPn26aNmypeZv2LRpU83vM3XqVJGfny+6du0q7OzsRFhYmAgNDdXc+wEDBuj8vaqK+u/53nvv6dx/5coVTd3PnTun2S719+nTp48AIDw8PETLli3F008/LRo3bqwpZ+bMmTqv9fDhQ9GvXz/Ncf7+/qJDhw4iICBA2NnZVfqbr127Vue9zs3NFQMGDBAAREREhLh7966pf5Zax9B9UlO//iu+d9SvzbVr12ptnz17tuZ4Pz8/0b59e9GyZUtRp04dzWu5vPfee08AEKNGjRIdOnQQAETLli1F27ZthYODgwAgGjVqJK5cuVKpbua+LtTvt08//VR4eHgIV1dX0b59e9GkSRPNua+99prev0XF14cQQmRmZorw8HDN+U2bNhVPP/208Pf31/vZIyfqv82aNWu0tv/8889af+uGDRuKBg0aVDq/W7dumueBWvlnkPr51ahRI9GuXTvh7u4uAAgnJycRHx9fqTxdr7/hw4drni8eHh6ia9eump/hw4drnb9q1SrN68vd3V20adNGNGrUSHOvJk6caMmfq9rI+1VFOpWVlWke8t98841me/fu3QUAMWvWrErnqN9M9vb2wtXVVWzfvl2zr7CwUJSWlgohhPjll1/E4cOHNf9Wu3TpkujSpYsAIDZu3FipfPUDw9HRUXTr1k1kZmZq9h0+fFj4+voKAOKf//ynzt/p888/FwDEv//9b531Vgd0n376qQAgFixYoHVcr169BABx6tSpKgnoBg8eLACIcePGiZycHK19qamp4tNPP620zdyAztHRUXTv3l3rb7h+/XoBQNjZ2YmhQ4eKJ598Uly+fFmzf9++fcLR0VEAEDExMTp/t6qgL1C4d++e+O9//yueeuopAUD89a9/1dov9ffZunWrOHXqVKXtCQkJokWLFgKAOHz4cKX9I0eOFABEw4YNxX//+1+tfXfu3BGLFy8Wt2/f1mzTFdDdu3dPdOzYUQAQ/fr1E7m5uYb/CDJQFQHdnTt3hL29vXBwcBDbtm0TZWVlmn2lpaXiwIEDYteuXVrlqAM6R0dH4e/vL44fP67Zl56ergnyunTpUqlu5r4u1O83R0dH8fLLL4uHDx9q9m3ZskWoVCqhUqnEpUuXdP4tKn4WlJaWis6dOwsA4oknntD6HYQQ4tq1a2LevHmV6iEnY8aMEQDEhAkTtLa//fbbAoDYuXOnEEKIF198UQAQFy5c0ByTn58vnJ2dBQCt96H67+ng4CDq1q0rtm3bptn36NEjMXz4cL33Xt8XCn1fyso7ePCgsLOzE3Xr1hVr1qzRerbFxcWJhg0bVnqW1lYM6BRo//79AoBwcXERDx480Gz/5ptvNN96SkpKtM4p/0G9ePFis66blJQkAIhnnnmm0j71A8PJyUlcv3690n71G8/d3V3rA1VN/e07MTFRZ73VAd3NmzeFg4ODCAkJ0RyTlpYmVCqVaN26tRBCVElA17JlSwFAnD59Wud5FVkS0Lm6uor09PRK56kDDZVKJf74449K+9UfrhVbRaqS+u+p78fLy0t8+OGHoqioSOs8qb+PlH379gkAYsqUKVrbT548qQl4ExISjCqr4sMgPT1dtGrVSgAQI0eOFIWFhSbVrTYzdJ90/ZSn64F69OhRAUC0bdvW6HqoAzoAYuvWrZX2Jycna1pTDx06ZHS5+l4XQvz5fgsLC6v0uSjEn1/Wli5dqrVd32fBjh07NJ9l165dM7qOcrJq1SoBQAQGBmpt79ixo1CpVOLevXtCCCG++OILAUCsXLlSc0z5wK38F6Lyz6APP/yw0jVv3rwpnJycBABx//59rX2WBHQRERECgPj222917t+5c6cAIEJDQ/WWUVs4gBRHPT5j0KBBWtO4R4wYgWnTpiEzMxP79+/HgAEDdJ5vaNLAgwcPsGXLFhw5cgQ3btxAfn6+Zpo6AM3sJV2GDRuGRo0aVdoeFRWFN954A1lZWTh8+LBW3bKzs/Hrr7+iefPmePLJJyXr5ufnhwEDBmD37t2Ij49Ht27dsGHDBgghqnQyRNOmTXHp0iVs3boVTz31VJXm6XrmmWfQpEmTStvbtWuHY8eOoW3btnj66ad17o+OjsaVK1eqrG76BAQEaKV3yMvLQ1paGh48eIBVq1YhLCxM79hIXW7evIlNmzbhjz/+wO3bt1FQUAAAmjFwFV+DO3bsAAD069cP7dq1M7n+Fy9eRP/+/ZGeno6pU6fiiy++gJ2d8oYgV7xPFWVnZ+PcuXNGlaUu5/Llyzh16hTCw8ONrkejRo0wbNiwSttDQkIwcOBA7N69G3v27EH37t219pv6uijv5Zdfhr29faXtnTp1wq5du4x+36hfa1FRUQgICDDqHLlRj6NLS0tDRkYGmjRpgkePHuHkyZMICwuDj48PACAyMhLA44kRkydP1vw/ADz99NOoW7euzvLVx5bn5+eHwMBAXL58GSkpKWjfvr3Fv0dGRgaOHz+OOnXqYMyYMTqPefbZZ+Ho6IikpCRkZmbqfH7VFgzoFCY3N1fzgaKeCKHm6emJwYMHIzo6GuvWrdMZ0NWrVw/169fXW35cXBxGjBiBu3fv6j3m3r17evfpC8icnJzQvHlz/PHHH7h06ZJW3fbs2YPi4mKds1t1mTBhAnbv3o3169ejW7duWL9+vWYWbFWZOXMm9u/fjw8++ADr16/HM888gy5duqBXr15o1qyZVa8VEhKic7v6vunb36BBAwCPXyPV7aWXXqqUtLasrAzff/89Jk2ahKFDh2LPnj3o16+fwbK2bNmCSZMmSU7wqPgaPH/+PACgc+fOJtc9LS0NkZGRuHv3LubNm4cFCxaYXIZc6LpP5cXFxaFXr15GldWoUSNERUVh48aNaN++PTp37oxevXqha9eu6N69u96HOQA88cQTegPmJ598Ert378alS5e0tpvzuigvNDRU53Y/Pz8Axr9vLHmtyUWLFi3g7++Pmzdv4tChQxgzZgyOHDmC4uJirSC7devW8PT01Jrpqv5/felK6tWrp0m1VZGfnx8uX75stc+wM2fOaP6/d+/eeo9Tf0HPyMio1QGd8r5i2rjo6Gjk5eXB19cXAwcOrLRfPRP0P//5D7Kzsyvtl/qQzcnJ0QRzL774IuLj43H37l0UFxdDCIHS0lIAQElJid4y1EGFLuoPzop55Hbu3AlAd7oSXQYPHgxvb29ER0fjwIEDSEpKwoABA+Dv72/U+ebo378/9u3bh549eyIzMxPffPMN/va3vyEwMBBdu3bF77//brVr6btH6g8dQ/vLt6bWJDs7O4wfPx7Tpk1DaWkp5syZY/Cc1NRUjB8/Hnl5eXjttddw/PhxZGVloaSkBEIITStKcXGx1nk5OTkAYFbi2/v37+PevXuwt7c3qZWJgLVr1+KDDz5AYGAgjhw5gg8++ADPPvss6tevjylTpuj8DAJM/5ww93VRnr73jTqwNPZ9Y8lrTU7UAZl6xqr6v+UDOjs7O3Tt2hUZGRlISUlBcXExjh49qnV+RVLPIFPvhSHq7A3qtEn6ftQz8B89emSV61YVBnQKo562fe/ePTg5OVVackndrVVQUIAtW7aYVPYvv/yCu3fvIiIiAps2bULXrl3h6+sLBwcHzTUNuX37tt59t27dAgC4u7trtpWUlGDv3r3w8fFBt27djKqns7MzXnzxRWRnZ2PSpEkA/gxkDTEU9Eh9++/Tpw9iY2ORlZWFvXv3Ys6cOQgODsaRI0fQr18/pKamGlUHW9O1a1cAj7vDDKUN2bJlC4qKijB8+HAsX74cHTp0gJeXl6arTN9r0MPDAwD0BhBS2rVrhy+//BJlZWV48cUXNV8wyDAnJyf885//REpKClJSUrB+/XqMHTsWAPD1118jKipK53mmfk6Y+7qoCpa81uREHbipW9zU/1V3s6qV73Y9ceIEHj16BHt7e837via5ubkBeNzqKx7PKZD8qe3r9jKgU5CUlBTEx8cDePwN18/PT+eP+gPH1FxI6oCkW7duOrtDjGmFUndHVFRcXKz5Ft2yZUvN9kOHDuHBgwd49tlndY5v0Uc9Xu7q1avw9PTEkCFDjDpP/e1Q/dCo6PLlywbLcHd3x4ABA/Cvf/0LFy5cQNu2bZGbm4vvv//eyNrbFnWOr7KyMs03Zn3Ur8GKY6fU9L0G//KXvwCApnXAVH//+9/x5ZdfoqSkBCNGjGBQZ4agoCCMGzcOGzZs0OR5/OWXX3Tm8Lt06ZLenJYXLlwAoP05Ye7roipY+lqTC3UL26VLl3Dt2jUcP34cISEhlbokywd06qAvPDxc8xyqSobGMqvvVUpKCvLz86u8PlWNAZ2CfPfddxBCIDQ0FLdu3cLNmzd1/qibxo8cOYKkpCSjy69Tpw4A4MaNG5X2CSHwySefGCxjx44dOs/fuHEj7t+/Dzc3N61vbqZ2t6p17twZL774Ivr06YO333670lJh+qjH0ej6MC4pKcE333xjUj2cnJw0g3czMzNNOtdWqL+EeHh4oF69epLHSr0GCwoK8MUXX+g8Tz3Aft++fTh9+rRZ9fz73/+OFStWaII69colZLr27dvD2dkZgO73xfXr1/Gf//yn0vaUlBTs2bMHALQShJv7uqgK6tfaxo0bcf369Wq7bnV78sknNe/XxYsXo6CgoFLrHAB06NABLi4uOHTokObZY43lvoyhTlqsL1gLCQlBeHg4CgoK8OWXX1ZLnaoSAzqFEEJo1hQ01L3Ytm1bPPXUUwBMa6VTf/vdunUrdu/erdn+8OFDvPzyyzh+/LhR9Rw9erRWtvejR4/irbfeAgC8/vrrmmZw4PH6s05OTpKrO+izZcsW7N+/H//85z+NPkfdJb1mzRrExsZqtufk5OCVV16plBldbeTIkdi5c6dmVp3asWPHNEGpNWZlKUlZWRnWrl2LFStWAHg8icdQK6z6NbhixQr88ccfmu23b9/G8OHD9a7Y0KZNG4wePRplZWV47rnncODAAa39d+/exZIlS3Dnzh3J60+ZMgVfffUVSkpKMHz4cAZ1Eg4cOIA333wTZ8+e1dpeXFyMf/3rXygsLESdOnXQqlWrSuc6Ojpi2rRpSEhI0Gy7fv06oqKiUFpais6dO2sFBea+LqrC4MGD0bVrVzx8+BD9+/fX+h2AxwPrpSaeyIVKpdL83VevXg1Adwupk5MTIiIikJKSovlMra6ATj1B7Pz583rf25988gns7e0xZ84cLF68uNI4uQcPHuD777/XPKNqterMkUJV5+DBg5ocZGlpaQaPX7JkiQAeZzAvLS3VytItJSoqSpMrKCgoSLRv317UqVNHqFQqTc4fXS+riitFODk5ifDwcE3+NgAiMjJSPHr0SHPOmTNnBADRv39/vfWpmIfOGFJ56IQQYsiQIZq/ZVBQkGjXrp1wcXERnp6eYtmyZTrzGnl6emqSk7Zq1Up07NhRawWN3r17i+LiYs3xluSh05f8VZ3Dq2KyTzVjcjJZm/q+BwQEaGVqb9u2reZvBkB069ZNKyGzvr9PSUmJiIyM1NyfFi1aiPDwcOHo6CicnJw0+bF0/V1zcnI0+Qzx/xMMd+jQQTRt2tSklSKEEGLlypVCpVIJJycnTRJVOauKxMI//vij5lhvb2/Rrl07ER4eLry8vDT3r2Ky1vIrRTz99NMCgGjVqpUIDw/XZPL39/cXSUlJWudZ8rpQv9/05Z5Uvw4qvq8MrRTRtm1bze8fGBgoOnToIBo2bKhZtUUJPvvsM63chBXvi9o777yjOcbOzk6z8lB5xjyD9OUJ1ZeHrrS0VPOMcXd3Fx07dhQ9evQQI0eO1Dpu/fr1wsXFRQAQzs7Ook2bNiIiIkIEBQVp7ld1fm6aiy10CqFuaevRo4dRaTLGjBkDBwcHXLt2DQcPHjTpOh988AFCQ0ORkZGBtLQ0dO/eHfv378fEiRMNnt+8eXMkJCRgxIgRuHHjBlJTUxEaGoqFCxdi3759Wuv6mdvdaqktW7Zg3rx5CA4OxvXr15GRkYFhw4bh5MmTaNOmjc5z1q9fj7///e948skncffuXSQkJCA7OxuRkZH46quvEBMTo5k8YovS09O1Zo2dO3cOTk5O6Nu3L1avXo24uDitQe762NvbY8+ePZg5cyYCAgKQmpqKzMxMPP/88zh69Khk2hN3d3fExMRg3bp16N27NwoLC3HmzBmUlZWhd+/eWLlypdEpCSZPnoyVK1eiuLiY3a96REZG4osvvsDQoUPh4+ODy5cv49y5c3B3d8eoUaNw5MgRvPzyyzrPdXZ2RlxcHGbNmoXCwkIkJiaiXr16ePnll3Hy5Ek0b95c63hLXhdVoWHDhvj999/x2WefoUuXLsjKysLZs2fh7OyMQYMGYcOGDdVan6pSvqWtYcOGle6LWvmu2KeeekpvWhJrs7Ozwy+//IKRI0eibt26OHHiBA4dOlRpTOW4ceOQmJiI6dOnIyQkBMnJyTh58iTy8/PRt29ffPbZZ7IYA60SopbkMCCqoEOHDjhx4gSuXbum2ASdRPSn+fPnY8GCBZgwYYLWQutEZBhb6KhWyszMREJCAtq2bctgjoiIyADb7QOiWq1Ro0Z60xYQERGRNrbQEREREckcAzoiIiIimeOkCCIiIiKZYwsdERERkcwxoCMiIiKSOQZ0RERERDLHgI6IiIhI5hjQEREREckcAzoiIiIimWNAR0RERCRzDOiIiIiIZI4BHREREZHMMaAjIrKQEAJr165F586d4e7uDk9PT3Ts2BGrVq2CEAITJ06ESqXCxIkTtc4LDAyESqXCunXrkJubi3nz5qF169Zwd3eHSqVCWlqa5tjS0lJ8++236N27N+rVqwdnZ2c0btwYI0aMQFxcnN66lb+GPvrqV/H8hw8fYs6cOWjZsiVcXV1Rr149DB06FMeOHTPtD0ZEVudQ0xUgIpKz0tJSjBkzBlu2bAEAqFQqeHl54cSJEzh+/Dji4uLg5OQkWca9e/fQvn17XL58GU5OTqhTp47W/uzsbAwdOlQTuNnb28Pd3R03btzAtm3bsG3bNsycOROLFy+ukt8RALKystChQwdcunQJTk5OcHFxwb1797Bz50789NNP+Oabb/DSSy9V2fWJSBpb6IiILLB48WJNMPfGG2/gzp07uH//PrKysvCvf/0Lmzdvxq5duyTLmD9/PnJycvDjjz8iNzcXWVlZSE9PR4MGDQAAkyZN0gSGn3/+OXJycpCVlYXMzExNEPXJJ59g5cqVVfZ7LliwALdv30Z0dDTy8vKQnZ2N8+fPo0ePHigrK8PkyZNx8uTJKrs+EUljQEdEZKa8vDz8+9//BvA46FqyZAl8fX0BAB4eHpgzZw7mzZuHrKwsyXLy8/Pxyy+/YOjQoXB0dAQANGnSBHXq1MGxY8ewfft2AMAXX3yBadOmaVrw/P39sWbNGrzwwgsAgLlz56KgoKBKftfs7Gxs3boVI0aMgIPD486dVq1aYc+ePQgNDUVJSQnmzp1bJdcmIsMY0BERmem///0vcnJyAADvvPOOzmPefPPNSl2oFT3zzDMIDw/XuU/d+tekSRO8/PLLOo95//33AQB3797Fvn37jKq7qbp27Yo+ffpU2u7q6oq33noLALB3715kZ2dXyfWJSBoDOiIiM6m7GJs2bYqgoCCdx7i7u6N9+/aS5XTt2lXvvhMnTgAAevXqBTs73R/ZrVq1QuPGjbWOt7bevXsb3FdWVsZuV6IawoCOiMhMd+7cAQA0atRI8jh1sKWPeqycLrdv3zaqjCZNmmgdb21S1y+/r6quT0TSGNAREVlIpVJZdL69vb2VakJEtooBHRGRmerXrw8AyMzMlDzu+vXrZl9D3XqXkZEheZx6f8XWPvUEBqnJEsaMe5P6Hcrvk2ptJKKqw4COiMhM7dq1AwBcvXpVKwlwebm5uUhISDD7Gk8//TQAIDY2FmVlZTqPuXjxoiao6tChg9Y+b29vAEB6errOc8vKyowadxcbG2twn52dnd7JHURUtRjQERGZqX///vDw8AAA/Otf/9J5zNKlS/Ho0SOzrzFq1CgAj1vBVq9erfOYefPmAQDq1auHvn37au1r06YNAODHH3+EEKLSud99953B1j8AiI+P17kiRUFBAZYsWQIAGDBgALy8vAyWRUTWx4COiMhMdevWxaxZswAA33zzDd5++23cv38fAPDw4UN89NFHmD9/vqaVzBwRERGaPHPTpk3D8uXLNQHizZs38corr2Dr1q0AHqcvcXFx0Tp/9OjRAIALFy7g1Vdfxb179wAAOTk5WLp0KaZMmQIfHx+D9fD09MQLL7yAbdu2oaSkBMDjlsHnnnsOFy9ehL29PRYuXGj270lEFhJERGS24uJiMXz4cAFAABB2dnbC29tb2NvbCwBi3LhxYvz48QKAmDx5sta5zZo1EwDE2rVrJa/x4MED0aNHD801HBwchLe3t1CpVJptM2fO1Hv+uHHjNMcBEF5eXsLOzk4AENOmTRMTJkwQAMSECRMqnauu46effipatmwpAAhnZ2fh6empKU+lUolVq1aZ8+cjIithCx0RkQUcHBwQHR2N1atXIyIiAq6urigpKcHTTz+N1atXY/369Xjw4AEAmN0d6enpiQMHDmDNmjXo2bMn3N3dkZubC39/f7zwwguIjY2VXMd13bp1+Oyzz9C2bVu4urqirKwMXbt2RXR0ND7//HOj6uDt7Y3jx49j9uzZaNq0KQoLC+Hj44Pnn38ehw8fxiuvvGLW70ZE1qESQsegCiIisgohBJo2bYqMjAysX78e48aNq+kqmSQwMBBXr17F2rVrMXHixJquDhHpwRY6IqIqtGHDBmRkZMDBwaHShAUiImthQEdEZKHRo0dj27ZtuHv3rmbbrVu38OGHH2q6IsePH4+GDRvWVBWJSOEcaroCRERyt2fPHmzevBkAUKdOHTg6Omol642MjMTSpUtrqnpEZAMY0BERWejzzz/Hnj17cOrUKdy+fRu5ubmoX78+2rZti1GjRmHcuHFwdHSs6WoSkYJxUgQRERGRzHEMHREREZHMMaAjIiIikjkGdEREREQyx4COiIiISOYY0BERERHJHAM6IiIiIpljQEdEREQkcwzoiIiIiGSOAR0RERGRzP0/d5ZzE+foHYwAAAAASUVORK5CYII=\n"
          },
          "metadata": {}
        }
      ],
      "source": [
        "plot = seaborn.swarmplot(x='group', y='sentiment', data=name_sentiments)\n",
        "plot.set_ylim([-10, 10])"
      ]
    },
    {
      "cell_type": "markdown",
      "metadata": {
        "id": "fm31Upx4k_Nj"
      },
      "source": [
        "We can see that as a bar-plot, too, showing the 95% confidence intervals of the means."
      ]
    },
    {
      "cell_type": "code",
      "execution_count": null,
      "metadata": {
        "id": "9Qjk5wSFk_Nk",
        "colab": {
          "base_uri": "https://localhost:8080/",
          "height": 472
        },
        "outputId": "aa11ccd3-fe7a-431b-f2f8-d0a48686eacc"
      },
      "outputs": [
        {
          "output_type": "display_data",
          "data": {
            "text/plain": [
              "<Figure size 640x480 with 1 Axes>"
            ],
            "image/png": "iVBORw0KGgoAAAANSUhEUgAAAmYAAAHHCAYAAAAVhJRcAAAAOXRFWHRTb2Z0d2FyZQBNYXRwbG90bGliIHZlcnNpb24zLjcuMSwgaHR0cHM6Ly9tYXRwbG90bGliLm9yZy/bCgiHAAAACXBIWXMAAA9hAAAPYQGoP6dpAABT0ElEQVR4nO3dd1hUV/4G8PdSBpCO2FC6WOJaUMEYRYkYjYltI8YWyxpbzGbll2iMMRgLyWbXWKK7KajRyEYjWKKJUWMBYy/YYkMUKYLGBihIGZjz+8OdWUdmcGAGuDDv53nmUe4999zvFGZezr1zriSEECAiIiKiGmdR0wUQERER0WMMZkREREQywWBGREREJBMMZkREREQywWBGREREJBMMZkREREQywWBGREREJBMMZkREREQyYVXTBVDFuLi4oKioCE2aNKnpUoiIiMhAN2/ehI2NDXJycsptx2BWyxQVFaGkpKSmyyAiIqIKMPSzm8GsllGPlKWkpNRwJURERGQoPz8/g9rxHDMiIiIimWAwIyIiIpIJswxmt27dQkxMDP72t7+ha9eusLOzgyRJCA0NrXSfc+fOhSRJ5d6+/vpr090JIiIiqnPM8hyzH374Af/3f/9XJX03bNgQAQEBOtfxm5RERERUHrMMZk5OTujduzeCgoIQFBSE06dPY8GCBSbpu1+/flizZo1J+iIiIiLzYpbBbPz48Rg/frzm58zMzBqshoiIiOgxszzHjIiIiEiOzHLErCqdPXsWI0eOxK1bt+Do6Ih27dph+PDhaNOmTU2XRkRERDLHYGZiZ86cwZkzZzQ/b9u2DZ988gmmTZuGzz//HJaWls/so7xJ6DIyMuDp6WmKUomIiEhmeCjTRDw8PDB//nwcO3YMd+7cQWFhIc6dO4cpU6ZACIGlS5di1qxZNV0mERERyRhHzExk0qRJZZa1bdsWX331FXx9fTFz5kwsWbIEU6dOhY+PT7l9lXe5JUMv6UBERES1D0fMqsF7770HDw8PlJSUYNu2bTVdDhEREckUR8yqgaWlJbp06YItW7YgOTm5psshIiKqMkII5OfnG7TO3t4ekiSVaadvuTlgMKsmCoUCAFBSUlLDlRARVS9+UJuX/Px8DBo0yKg+tm7dCgcHBxNVVLswmFWT8+fPAwCaNWtWw5UQEVUvflATGY7nmFWD7du348KFCwCAPn361HA1REREJFccMauA7t2748aNG4iIiEBERIRm+YULF7Bs2TJMnToV7du31yxXqVTYsGEDpkyZAgDo378/goKCqrtsIiKiamNvb4+tW7fqXJeXl4dRo0Zpfv7+++91joTa29tXWX1yZ5bBLCMjA4GBgZqfCwsLAQCHDh2Cu7u7Zvn777+P999/X/PzjRs3kJaWhpycHK3+lEoloqOjER0dDTc3N3h7e8PKygpXr15FdnY2ACAkJAQxMTFVeK+IiOSJH9TmRZIkgw87Ozg48BD1U8wymJWWluLevXtllpeUlGgtf/TokUH9+fj4ICoqCkeOHMGlS5dw9epVFBYWws3NDf369cPIkSMxYsQIg2b9JyKqa/hBTWQ4swxmPj4+EEJUeLvU1FSdy11cXDB79mwjqyIiIiJzx5P/iYiIiGSCwYyIiIhIJhjMiIiIiGSCwYyIiIhIJhjMiIiIiGSCwYyIiIhIJhjMiIiIiGSCwYyIiIhIJhjMiIiIiGSCwYyIiIhIJhjMiIiIiGSCwYyIiIhIJhjMiIiIiGSCwYyIiIhIJhjMiIiIiGSCwYyIiIhIJhjMiIiIiGSCwYyIiIhIJhjMiIiIiGSCwYyIiIhIJhjMiIiIiGSCwYyIiIhIJhjMiIiIiGSCwYyIiIhIJhjMiIiIiGSCwYyIiIhIJhjMiIiIiGSCwYyIiIhIJswymN26dQsxMTH429/+hq5du8LOzg6SJCE0NNTovuPj49G/f380aNAAdnZ2aNWqFSIjI5Gfn2984URERFSnWdV0ATXhhx9+wP/93/+ZvN/ly5dj2rRpEEKgWbNm8PT0xMWLFxEVFYVNmzbh4MGDcHNzM/l+iYiIqG4wyxEzJycn9O7dG7NmzcLmzZsRGRlpdJ+JiYmIiIgAAHzzzTdIT0/HqVOnkJKSgk6dOuHSpUuYOHGi0fshIiKiusssR8zGjx+P8ePHa37OzMw0us8FCxZApVJhzJgxmDRpkma5h4cH1q9fj1atWmHz5s04d+4c2rVrZ/T+iIiIqO4xyxEzU8vLy8POnTsBQCuUqQUEBKBXr14AgLi4uGqtjYiIiGoPBjMTOH36NIqKimBjY4Pg4GCdbUJCQgAAR48erc7SiIiIqBZhMDOBK1euAAC8vLxgbW2ts42/vz8AICkpqdrqIiIiotrFLM8xM7X79+8DQLnfuFSvy87OfmZ/fn5+etdlZGTA09OzghUSERFRbcARMxMoLCwEACgUCr1tbGxsAAAFBQXVUhMRERHVPhwxMwFbW1sAQHFxsd42RUVFAAA7O7tn9peSkqJ3XXmjaUREVLd1mrG2pkswilRSDOcnfg6N/AHCSv+ghpwlLhxTJf1yxMwEXF1dAfzvkKYu6nXqtkRERERPYzAzgRYtWgAA0tPToVQqdba5du2aVlsiIiKipzGYmUBgYCAUCgWKiopw/PhxnW0OHDgAAOjatWt1lkZERES1CIOZCTg6OqJv374AgOjo6DLrk5OTsW/fPgBAeHh4tdZGREREtQeDWQV0794dPj4+WLp0aZl1kZGRkCQJMTExiI6OhhACAHDz5k2MGDECKpUKgwcPRvv27au5aiIiIqotzDKYZWRkwN3dXXP74IMPAACHDh3SWv7Pf/5Ta7sbN24gLS0NOTk5ZfoMCgrC4sWLAQCTJ0+Gt7c3OnbsCF9fXyQmJqJly5ZYsWJFld83IiIiqr3McrqM0tJS3Lt3r8zykpISreWPHj2qUL8RERFo27YtFi1ahGPHjuH27dvw9vZGeHg4Zs2aBQcHB6NrJyIiorrLLIOZj4+P5lBjRaSmpj6zTVhYGMLCwipRFREREZk7szyUSURERCRHDGZEREREMsFgRkRERCQTDGZEREREMsFgRkRERCQTDGZEREREMsFgRkRERCQTDGZEREREMsFgRkRERCQTDGZEREREMmGWl2QiIqorOs1YW9MlGEUqKYbzEz+HRv4AYaWosXqMkbhwTE2XQHUAR8yIiIiIZILBjIiIiEgmGMyIiIiIZILBjIiIiEgmGMyIiIiIZILBjIiIiEgmGMyIiIiIZILBjIiIiEgmGMyIiIiIZILBjIiIiEgmGMyIiIiIZILBjIiIiEgmGMyIiIiIZILBjIiIiEgmGMyIiIiIZILBjIiIiEgmGMyIiIiIZILBjIiIiEgmzDqYxcfHo3///mjQoAHs7OzQqlUrREZGIj8/v8J9jRs3DpIklXvbuXNnFdwLIiIiqiusarqAmrJ8+XJMmzYNQgg0a9YMnp6euHjxIqKiorBp0yYcPHgQbm5uFe7X09MTXl5eOte5uroaWzYREZG8CQGpVKlzlVRaXO7Pmi4srQFJMnlptYFZBrPExEREREQAAL755htMnDgRkiQhKysLAwcORGJiIiZOnIhNmzZVuO/x48dj7ty5pi2YiIiolpBKlXA+t96gtk4XdH/O5rYbAWGlMGVZtYZZHspcsGABVCoVRo8ejUmTJkH6byr38PDA+vXrYWFhgc2bN+PcuXM1XCkRERGZE7MLZnl5eZpzvSZNmlRmfUBAAHr16gUAiIuLq9baiIiIyLyZ3aHM06dPo6ioCDY2NggODtbZJiQkBHv27MHRo0cr3H98fDwuXLiAe/fuwcXFBZ06dcIbb7wBb29vY0snIiKSPWFpjdx2I/St1Tr/TFhaAyh7Ltnj5ebJ7ILZlStXAABeXl6wttb9xPv7+wMAkpKSKtz/b7/9pvXzli1bMG/ePERFReH99983qA8/Pz+96zIyMuDp6VnhuoiIiKqFJJV7fpiwsqnGYmofszuUef/+fQAo9xuX6nXZ2dkG9xsQEIBFixbh9OnTyM7OxqNHj3D06FEMHToUSqUSM2fOxJdffmlc8URERFSnmd2IWWFhIQBAodCf5m1sHqf5goICg/udPXt2mWVdunRBbGws3n77bXz55ZeYPXs2xowZAwcHh3L7SklJ0buuvNE0IiIiqt3MbsTM1tYWAFBcrHvuFAAoKioCANjZ2Zlkn59++ikUCgVycnKwb98+k/RJREREdY/ZBTP1JK/qQ5q6qNeZakJYZ2dntGnTBgCQnJxskj6JiIio7jG7YNaiRQsAQHp6OpRK3TMTX7t2TautKagPnZaUlJisTyIiIqpbzC6YBQYGQqFQoKioCMePH9fZ5sCBAwCArl27mmSfJSUluHz5MgCgWbNmJumTiIiI6h6zC2aOjo7o27cvACA6OrrM+uTkZM15YOHh4SbZ5zfffIPc3FxYWVlpJq8lIiIieprZBTMAiIyMhCRJiImJQXR0NIQQAICbN29ixIgRUKlUGDx4MNq3b6+1nY+PD3x8fLBx40at5bt378bMmTPLnD9WXFyM5cuX49133wUATJkyBU2aNKnCe0ZERES1mVkGs6CgICxevBgAMHnyZHh7e6Njx47w9fVFYmIiWrZsiRUrVpTZLi0tDWlpacjLy9Nanp+fj3/+859o0aIFGjdujM6dO6Nz586oX78+/va3v6G4uBhDhgzBokWLquX+ERERUe1klsEMACIiIrB7927069cP+fn5uHjxIry9vfHhhx/i5MmTcHd3N7ivTp06ITIyEr1794atrS0uX76M33//Hc7Oznjttdewbds2bNy4sdy504iIiIjMboLZJ4WFhSEsLMzg9upDnk/z9PTE/PnzTVUWERERmSmzHTEjIiIikhsGMyIiIiKZYDAjIiIikgkGMyIiIiKZYDAjIiIikgkGMyIiIiKZYDAjIiIikgkGMyIiIiKZYDAjIiIikgkGMyIiIiKZMCqY+fn54fnnnze4fUhICPz9/Y3ZJREREVGdZdS1MlNTU1FYWGhw+xs3biA9Pd2YXRIRERHVWdV6KLOkpAQWFjx6SkRERKRLtaWkgoIC3L59G46OjtW1SyIiIqJapUKHMtPT05Gamqq1rLi4GAcOHIAQQuc2Qgjk5OTg+++/h1KpRNu2bStdLBEREVFdVqFgtnr1asyfP19rWXZ2NkJDQ5+5rRACkiRh8uTJFSqQiIiIyFxU+OT/J0fGJEnSO1L2ZBsnJyf86U9/wpQpUzBy5MiKV0lERERkBioUzD7++GN8/PHHmp8tLCzQuHFjZGVlmbwwIiIiInNj1HQZY8aMgYuLi4lKISIiIjJvRgWzNWvWmKgMIiIiIuKkYkREREQyYdSImdrDhw/x888/49y5c7h//z6USqXetpIkYdWqVabYLREREVGdYnQwW7NmDaZNm4a8vDzNMl3f1FR/g5PBjIiIiEg3o4LZrl278Oabb0IIAVtbW3Tt2hUeHh6wsjLJQBwRERGRWTEqQf3zn/+EEAJdu3bF1q1b4e7ubqq6iIiIiMyOUSf/JyYmQpIkrFmzhqGMiIiIyEhGBbOSkhI4ODggICDAVPUQERERmS2jgpm/vz+KiopQWlpqqnqIiIiIzJZRweyNN96AUqnEjh07TFVPtYqPj0f//v3RoEED2NnZoVWrVoiMjER+fn6l+9y0aRNefPFFuLq6wt7eHh06dMDnn39e7hQiRERERICRwSwiIgJBQUGYOnUqkpOTTVVTtVi+fDnCwsKwfft22NraonXr1khNTUVUVBSCgoJw//79Cvc5ffp0hIeHIyEhAfXr10fz5s1x/vx5zJgxA71790ZRUVEV3BMiIpkTAlJJse5babFWU6lUdzvomIaJqC4y6luZ69evx+jRozFnzhy0b98e4eHh6NKlCxwdHcvdbsyYMcbs1miJiYmIiIgAAHzzzTeYOHEiJElCVlYWBg4ciMTEREycOBGbNm0yuM8tW7Zg0aJFsLGxQWxsLAYOHAgAuHz5Ml555RX89ttv+PDDD7Fo0aKquEtERLIllSrhfG69QW2dLuh+381tNwLCSmHKsohkSRK6ZoM1kIWFBSRJAgDN5LHP3KEkoaSkpLK7NInBgwdj69atGDNmDL777jutdcnJyWjVqhVUKhXOnj2Ldu3aGdRnhw4dcPbsWcyZMwfz5s3TWrdv3z6EhYXBxsYGGRkZaNCgQaVr9/PzAwCkpKRUug8iqjs6zVhb0yU8k1RSbHAw06c2BLPEhVU/6FAbnm9zUdHn29DPb6NGzLy8vAwKY3KSl5eHnTt3AgAmTZpUZn1AQAB69eqFPXv2IC4uzqBglpycjLNnz+rts1evXmjevDmuXr2Kbdu24c033zTyXhAREVFdZFQwS01NNVEZ1ef06dMoKiqCjY0NgoODdbYJCQnBnj17cPToUYP6VLfz8/ND06ZN9fZ59epVHD16lMGMiMyKsLRGbrsR+tZCKlVqtQXK/sH/eDlR3Wd21066cuUKgMejfdbWun/R/f39AQBJSUkV6lO9nSn6JCKqMySp3MOQwsqmGoshkjezC2bqb1u6ubnpbaNel52dXSN9qo9D65KRkQFPT0+D6iIiIqLaxWTB7O7du4iPj0daWhoePXqEOXPmmKprkyosLAQAKBT6/3qzsXn811tBQUGN9WlKPFlUPnhysHmpjue7OvZB8sHnu+4zOpiVlJRg5syZ+PLLL1Fc/L/5aJ4MZtnZ2fDz80NBQQEuX74MHx8fY3dbaba2tgCgVevT1PON2dnZ1Uif5X1jo7zRNCIiIqrdjJpgFgCGDh2KpUuXori4GG3atIGVVdms5+rqipEjR6K4uBixsbHG7tIorq6uAFDuBLLqdeq2NdEnERERmR+jgtkPP/yArVu3omHDhjh58iTOnTun9zyroUOHAnh8GaSa1KJFCwBAenq63sskXbt2TautoX1evXpVb5uK9klERETmx6hgtnr1akiShIULFyIwMLDctsHBwZAkCRcvXjRml0YLDAyEQqFAUVERjh8/rrPNgQMHAABdu3Y1qM/nn38eAHD9+nVkZmaapE8iIiIyP0YFs9OnTwMAhgwZ8sy29erVg7OzM27fvm3MLo3m6OiIvn37AgCio6PLrE9OTsa+ffsAAOHh4Qb12aJFC7Rt21Zvn/v27cPVq1ehUCg0l2oiIiIieppRwSw3NxfOzs4GnySvUqlkcaWAyMhISJKEmJgYREdHQ31Vqps3b2LEiBFQqVQYPHgw2rdvr7Wdj48PfHx8sHHjxjJ9fvzxxwCAf/zjH/jpp580y5OSkjBhwgQAwNSpU426HBMRERHVbUYFM1dXV+Tm5mqmiyjPzZs38eDBAzRq1MiYXZpEUFAQFi9eDACYPHkyvL290bFjR/j6+iIxMREtW7bEihUrymyXlpaGtLQ05OXllVk3ZMgQREREoKioCAMHDkTz5s3RoUMHtGnTBtevX0f37t3x97//vcrvGxEREdVeRgWzjh07AjDshP5vv/0WgHzOsYqIiMDu3bvRr18/5Ofn4+LFi/D29saHH36IkydPwt3dvcJ9LlmyBLGxsejZsyfu3r2LK1eu4LnnnsM//vEP7Nu3TzOtBpHZEwJSSbGeWxEsivI0N6mkSGc7/Hekm4ioLjFqHrNRo0Zh586diIyMREhICBwcHHS227lzJxYsWABJkjB27FhjdmlSYWFhCAsLM7i9MOCDYOjQoZpvoBKRblKpEs7n1hvVR267EeVe5oeIqDYyKpiNHDkS0dHROHDgAJ5//nlMmTJFM8nq7t27kZqaip9++gm//PILVCoVBgwYoDnxnoiIiIi0GRXMJEnCjz/+iD//+c/47bffMG3aNM26l19+WfN/IQR69+6N77//3pjdEREREdVpRl+SydXVFfv27cP333+PVatW4dixY5rLD1lZWSE4OBiTJk3CG2+8AQsLoy80QER1gLC0Rm67ETrXSaXFcLqwSfPzgzZDICzLHrIUltZVVh8RUU0xyUXMLSwsMHr0aIwePRoqlQr3799HaWkp6tevr/MSTURk5iTJ4PPDhKWC55IRkdkweWqysLCo1DcaiYiIiMwdjy0SERERyYTJRsyysrJw7tw5ZGdn6704uNqYMWNMtVsiIiKiOsPoYHb69GlMmzYNhw4dMqi9JEkMZkREREQ6GBXMTp8+jR49euDRo0cQQsDGxgbu7u484Z+IiIioEoxKUB999BHy8/Ph7++P6Oho9OzZk1NiEBEREVWSUcHs0KFDkCQJsbGxCAwMNFVNRERERGbJqOEtIQTs7e0ZyoiIiIhMwKhg1rx5cyiVSpSWlpqqHiIiIiKzZVQwGzduHIqLi7F161ZT1UNERERktowKZlOnTkXv3r0xefJkHDlyxFQ1EREREZklo07+t7S0xE8//YTp06eje/fuCAkJQVBQEBwdHcvdbs6cOcbsloiIiKhOMnrCsT179uDnn3+GEAIHDhzAgQMHnrkNgxkRERFRWUYFswMHDmDw4MGak//9/f3RqFEjTjBLREREVAlGJaj58+ejpKQEnTt3xg8//AA/Pz9T1UVERERkdow6+T8xMRGSJOH7779nKCMiIiIyklHBTKVSwdHREQEBAaaqh4iIiMhsGRXMWrdujYKCAhQVFZmqHiIiIiKzZVQwmzx5MpRKJf7zn/+Yqh4iIiIis2XUyf/jxo3D/v37MW3aNNjb22P48OGmqouIiIjI7BgVzMaPHw9JkqBQKDBq1CjMmjULnTt3LneCWUmSsGrVKmN2S0RERFQnGRXM1qxZA0mSIIQAAKSlpSEtLU1nW3U7BjMiIiIi3YwKZmPGjIEkSaaqhYiIiMisGT1iRkRERESmwWsnkTwIAalUqW+l1jphaQ2g7EitsLQGOIJLRES1mNkGs9OnT+Ozzz7D/v37kZOTAw8PDwwYMAAfffQRGjRoUOH+5s6di3nz5pXb5quvvsKUKVMqW3KdJpUq4XxuvVF95LYbAWGlMFFFRERE1c8sg9nmzZsxfPhwKJVKNGzYEG3atEFSUhKWLVuGuLg4HDx4sNKXmGrYsKHeKyE0adLEmLKJiIiojjM4mPXq1QsA4O3tjdWrV2stqwhJkrB3794Kb2cqmZmZGD16NJRKJSIjIzFnzhxYWVkhNzcXw4cPx86dOzFs2DAcP368Ul9s6NevH8+9IyIiokoxOJglJCQAAFq1alVmWUXU9Lc4Fy5ciEePHqFHjx6YP3++ZrmzszPWrVsHX19fnDx5Ej///DMGDBhQg5WaF2Fpjdx2I3Suk0qL4XRhk+bnB22GQFiWPWT5+NwzIiKi2svgYPbxxx8DANzd3cssq002btwIAJg0aVKZda6urhg6dChWrlyJ2NhYBrPqJEkGnx8mLBU8l4yIiOqkCgezZy2Ts4yMDGRmZgIAevToobNNSEgIVq5ciaNHj1ZqH2fPnsXIkSNx69YtODo6ol27dhg+fDjatGlT6bqJiIjIPJjVyf9XrlwBACgUCjRr1kxnG39/fwBASkoKlEolrK0rdnjszJkzOHPmjObnbdu24ZNPPsG0adPw+eefw9LSsnLFExERUZ1nYczG8+fPx+LFiw1uv2zZMq3zuqrb/fv3ATw+ZKnvXDc3NzcAgEqlwoMHDwzu28PDA/Pnz8exY8dw584dFBYW4ty5c5gyZQqEEFi6dClmzZplUF9+fn56bxkZGQbXRERERLWLUSNmc+fORePGjfHuu+8a1H7JkiVIT0/HnDlzjNltpRUWFgJ4PGKmj42Njeb/BQUFBvet65y1tm3b4quvvoKvry9mzpyJJUuWYOrUqfDx8TG8aCIiIjIbRo2YVaeIiAhIklThW2hoqKYPW1tbAEBxcbHe/RQVFWn+b2dnZ5La33vvPXh4eKCkpATbtm17ZvuUlBS9N09PT5PURERERPJTreeY3b9/XxOOKsrBwQH169ev8HbOzs6a/7u6ugIAsrOzIYTQeThTfbjTwsICTk5Olar1aZaWlujSpQu2bNmC5ORkk/RJREREdU+1BbO4uDg8fPgQLVu2rNT2UVFRiIqKMqqGFi1aAHg8YpaRkQEvL68yba5duwYA8PX1rfCJ/+VRHz4tKSkxWZ9ERERUt1QomH3xxRf44osvtJbduXOn3MsXCSGQk5ODBw8eQJIkvPrqq5Wr1AS8vLzg4eGBrKwsHDhwAKNGjSrT5sCBAwCArl27mnTf58+fBwC93wYlIiIiqlAwy8nJQWpqqtay0tLSMsv0CQsLq7ET/9WGDBmC5cuXIzo6ukwwy87ORlxcHABg6NChJtvn9u3bceHCBQBAnz59TNYvERER1S0VCmaDBw/WfKNQCIHx48fD2dkZS5cu1buN+lytP/3pT5o5wmrSjBkzsHLlSvz222+YM2cOPv74Y1haWiI3NxcjR45Ebm4uAgMDdc763717d9y4cQMRERGIiIjQLL9w4QKWLVuGqVOnon379prlKpUKGzZswJQpUwAA/fv3R1BQUJXfRyIiIqqdKhTM2rdvrxU8xo8fDzs7O4wdO9bkhVUVT09PrF27FiNGjMCCBQvwzTffwNPTE5cvX0Z+fj4aNWqE2NhYnV8MuHHjBtLS0pCTk6O1XKlUIjo6GtHR0XBzc4O3tzesrKxw9epVZGdnA3h8RYGYmJjquItERERUSxk1XYZKpUJWVpapaqk24eHhOHbsGMLDwwEAv//+Oxo0aIC//vWvOHfuHJo3b16h/nx8fBAVFYVXX30VLi4uuHr1Ks6cOQOFQoF+/fohJiYG8fHxcHFxqYJ7Q0RERHWFWV2S6UkdO3bUnE9mKH3n0rm4uGD27NkmqIqIiIjMmcmCmUqlQnJyMu7fvw+lUlluW30XECciIiIyZ0YHs5s3b2LWrFnYuHGjQZcwkiSJc3kRERER6WBUMMvKykKXLl2QlZUFIYRB2xjajoiIiMjcGHXy/9y5c5GZmQkHBwcsW7YMaWlpUCqVUKlU5d6IiIiIqCyjRsx27NgBSZKwatUqzTcciYiIiKhyjBoxu3PnDqysrDB48GATlUNERERkvowKZg0bNoSdnR2srMx21g0iIiIikzEqUfXu3RvfffcdkpOTERAQYKqaiMgIiQvH1HQJRsnLy8OgQes1PycsGA4HB4carIiIqPoYNWL24Ycfwt7eHjNnzjRVPURERERmy6hg1rx5c2zbtg379+/HSy+9hPj4eOTn55uqNiIiIiKzYtShTEtLS83/9+3bh3379j1zG04wS0RERKSbUcGMk8USERERmY5RwSw+Pt5UdRARERGZPaOCWc+ePU1VBxEREZHZM+rkfyIiIiIyHZMGMyEE7t69i/T0dFN2S0RERGQWTBLMTp06hddeew3Ozs5o1KgR/Pz8tNZnZ2dj8uTJmDJlCgoKCkyxSyIiIqI6x+hrKcXExGDChAlQKpV627i6uuLatWuIj49HaGgohg8fbuxuiYiIiOoco0bMLl68iIkTJ0KpVOJvf/sbTp48CXd3d51tx44dCyEEduzYYcwuiYiIiOoso0bMFi9ejOLiYrz99ttYunQpAO1JZ58UFhYGAEhMTDRml0RERER1llEjZvHx8ZAkyaBrZXp4eMDOzg4ZGRnG7JKIiIiozjIqmGVlZcHe3h7NmjUzqH29evV48j8RERGRHkYFMxsbGxQXFxt0aaaioiLk5OTAxcXFmF0SERER1VlGBTM/Pz8olUpcuXLlmW137dqF0tJStGnTxphdEhEREdVZRgWzV155BUIIzYn/+jx8+BAffPABJEnCwIEDjdklERERUZ1lVDCLiIiAs7MzoqOjERkZiZycHK31BQUF2Lx5M4KDg3H58mU0btwYkyZNMmaXRERERHWWUcHM3d0dcXFxsLW1xaeffopGjRrh7t27AB5/C9PZ2RlDhw5FUlISHBwcsHHjRtjb25ukcCIiIqK6xuhLMvXu3RtHjx5FaGgolEolSktLIYTArVu3UFJSAiEEQkNDceTIEXTt2tUUNRMRERHVSUZfkgkA2rZti7179yItLQ2HDh1CVlYWSktL0bhxY3Tr1g3Nmzc3xW5MIicnB7/++itOnDiBEydOIDExEXl5efD29kZqaqrR/Z8+fRqfffYZ9u/fj5ycHHh4eGDAgAH46KOP0KBBA+PvABEREdVZJglmat7e3vD29jZllyaXkJCAYcOGVUnfmzdvxvDhw6FUKtGwYUO0adMGSUlJWLZsGeLi4nDw4MEyF3gnIiIiUjPqUGZxcTHS09Nx69atMuvy8vIwffp0tG/fHoGBgYiMjJTF5LJ2dnbo0aMH3nvvPaxfvx7//ve/TdJvZmYmRo8eDaVSicjISGRmZiIxMRGZmZl4+eWXcfPmTQwbNsygOd+IiIjIPBk1YrZy5Uq88847GDt2LL799lutda+++ioOHjyoCSLnzp3DgQMHNJdxqil9+/ZF3759NT///PPPJul34cKFePToEXr06IH58+drljs7O2PdunXw9fXFyZMn8fPPP2PAgAEm2ScRERHVLUaNmO3atQsAMHLkSK3l27Ztw4EDByBJEkaNGoUJEybA2toaBw4cQExMjDG7lK2NGzcCgM7pQFxdXTF06FAAQGxsbLXWRURERLWHUcHs0qVLAIBOnTppLV+3bp3m4uYxMTGIjo7G0qVLIYTAunXrjNmlLGVkZCAzMxMA0KNHD51tQkJCAABHjx6ttrqIiIiodjEqmN25cwf16tWDq6ur1vL4+HgAwIQJEzTLRo8eDQA4e/asMbuUJfUlqRQKhd4Luvv7+wMAUlJSoFQqq602IiIiqj2MOscsPz8fdnZ2WstSU1Nx584deHl5wdfXV7Pc3t4eLi4uuH//vjG7lCX1fXJ1ddV7/pybmxsAQKVS4cGDB6hfv77e/sr75mZGRgY8PT2NqJaIiIjkyqgRMzc3N+Tl5Wldimnfvn0AgBdeeKFM+5KSEjg4OBizS1kqLCwE8HjETB8bGxvN/+Xw7VQiIiKSH6OCWceOHQEAq1atAvB4NGjVqlWQJAkvvviiVts7d+4gLy8PjRs3rtS+IiIiIElShW+hoaHG3EWD2NraAng8fYg+RUVFmv8/Pcr4tJSUFL03jpYRERHVXUYdyhw7dix27tyJDz74AHv27MGdO3dw6tQpODo6ar6FqHbgwAEAQOvWrSu1LwcHh3IP/+nj7Oxcqf1VhPocu+zsbAghdB7OVB/utLCwgJOTU5XXRERERLWPUcFs2LBh2LVrF9asWaOZOsPW1hZff/01XFxctNpu2LBB50iaoaKiohAVFWVMuVWmRYsWAB6PmGVkZMDLy6tMm2vXrgEAfH19YW1tXa31ERERUe1g9CWZvv32W7z55ps4fPgwXFxcEBYWVubk9eLiYjg7O2PMmDF45ZVXjN2l7Hh5ecHDwwNZWVk4cOAARo0aVaaNesSQF3InIiIifUxyrcxu3bqhW7duetcrFApER0ebYleyNWTIECxfvhzR0dFlgll2djbi4uIAoMwhXiIiIiI1o07+NzfDhw+Hj48Ppk+fXmbdjBkzYGdnh99++w1z5sxBaWkpACA3NxcjR45Ebm4uAgMDeTkmIiIi0sssg5m7u7vmph7dysjI0Fr+zjvvlNnu1q1bSEtLw927d8us8/T0xNq1a2FlZYUFCxbAw8MDnTt3RtOmTbFz5040atQIsbGxNXqdUCIiIpI3swxm9+7d09wePHgA4PFUH08uf/jwYYX7DQ8Px7FjxxAeHg4A+P3339GgQQP89a9/xblz59C8eXOT3g8iIiKqW0xyjlltI4So1HYJCQnPbNOxY0fN+WREREREFWGWI2ZEREREcsRgRkRERCQTDGZEREREMsFgRkRERCQTDGZEREREMsFgRkRERCQTDGZEREREMsFgRkRERCQTDGZEREREMsFgRkRERCQTDGZEREREMsFgRkRERCQTDGZEREREMsFgRkRERCQTDGZEREREMmFV0wVQ1UtcOKamSzBKXl4eBg1ar/k5YcFwODg41GBFREREVYMjZkREREQywREzIqp2Qgjk5+frXJeXl1fuz2r29vaQJMnktRER1SQGMyKqdvn5+Rg0aJBBbUeNGqVz+datW3lIm4jqHB7KJCIiIpIJBjMiIiIimeChTCKqdvb29ti6davOdU+ff6bvXDJ7e/sqq4+IqKYwmBFRtZMkqdzzwxwdHauxGiIi+eChTCIiIiKZYDAjIiIikgkGMyIiIiKZYDAjIiIikgkGMyIiIiKZYDAjIiIikgmzC2Y5OTmIjY3FjBkzEBoaCkdHR0iSBB8fH6P6XbNmDSRJKvf2wQcfmOZOEBERUZ1kdvOYJSQkYNiwYVXWv5OTE9q2batznbHhj4iIiOo2swtmdnZ26NGjB4KCgtC5c2fcv38fb7/9tsn6DwwMREJCgsn6IyIiIvNhdsGsb9++6Nu3r+bnn3/+uQarISIiIvofszvHjIiIiEiuzG7ErKqlp6dj3LhxyMjIgJ2dHVq3bo0hQ4bg+eefr+nSiIiISOYYzEzs+vXruH79uubn7du34/PPP8fw4cOxatUq1KtXrwarIyIiIjljMDMRV1dXvP/++xgwYACaN28ONzc3pKamYu3atfjHP/6BH374AaWlpYiNjX1mX35+fnrXZWRkwNPT05SlExERkUwwmJnIoEGDMGjQIK1lLVq0QFRUFNq1a4dhw4YhLi4OBw8eRPfu3WuoSiIiIpKzWnPyf0RExDMncNV1Cw0NrenS8frrr6NLly4AgM2bNz+zfUpKit4bR8uIiIjqrlozYubg4ID69etXeDtnZ+cqqKbiXnjhBRw7dgzJyck1XQoRERHJVK0JZlFRUYiKiqrpMipNoVAAAEpKSmq4EiIiIpKrWnMos7Y7f/48AKBZs2Y1XAkRERHJFYNZNTh37hx27twJAOjTp08NV0NERERyxWBWAcOHD4ePjw+mT5+utfzBgwcYNmwYDh8+DCGE1rpdu3ahX79+KC0tRfv27fHaa69VZ8lERERUi9Sac8xMyd3dXfN/pVIJ4PH8YE8uHzFiBJYvX6613a1bt5CWloa7d+9qLVepVIiNjUVsbCwcHR3h5+cHGxsbpKWl4Y8//gAAtGnTBtu2bYOlpWVV3S0iIiKq5cwymN27d6/MMpVKpbX84cOHBvdnb2+PhQsX4siRIzh//jzS0tKQl5cHZ2dnvPjiiwgPD8f48eNha2trkvqJiIiobjLLYPb04UZDJSQk6FxubW1d5vAmERERUUXxHDMiIiIimWAwIyIiIpIJBjMiIiIimWAwIyIiIpIJBjMiIiIimWAwIyIiIpIJBjMiIiIimWAwIyIiIpIJBjMiIiIimWAwIyIiIpIJBjMiIiIimWAwIyIiIpIJBjMiIiIimWAwIyIiIpIJBjMiIiIimWAwIyIiIpIJBjMiIiIimWAwIyIiIpIJBjMiIiIimWAwIyIiIpIJBjMiIiIimWAwIyIiIpIJBjMiIiIimWAwIyIiIpIJBjMiIiIimWAwIyIiIpIJBjMiIiIimWAwIyIiIpIJBjMiIiIimTC7YHb27FnMmzcPvXr1QqNGjWBtbQ1XV1eEhITgyy+/hFKpNKr/+Ph49O/fHw0aNICdnR1atWqFyMhI5Ofnm+geEBERUV0lCSFETRdRXa5du4bmzZtrfvb09ESjRo1w48YN3Lp1CwAQFBSEXbt2wdXVtcL9L1++HNOmTYMQAs2aNUODBg1w8eJFFBUVoXXr1jh48CDc3NyMug9+fn4AgJSUFKP6qU3y8vIwaNAgzc9bt26Fg4NDDVZERERUMYZ+fpvViJkQAg0bNkRUVBRSU1ORnp6OEydO4ObNm9i+fTvq16+PEydOYMqUKRXuOzExEREREQCAb775Bunp6Th16hRSUlLQqVMnXLp0CRMnTjTxPSIiIqK6xKyCWbNmzXD9+nXMnj0b3t7eWuteeeUVLFu2DACwadMm3Lt3r0J9L1iwACqVCqNHj8akSZMgSRIAwMPDA+vXr4eFhQU2b96Mc+fOmebOEBERUZ1jVsHM1tYW9erV07u+X79+AIDS0lJcvXrV4H7z8vKwc+dOAMCkSZPKrA8ICECvXr0AAHFxcRUpmYiIiMyIWQWzZykoKND8v7wA97TTp0+jqKgINjY2CA4O1tkmJCQEAHD06FHjiiQiIqI6y6qmC5CT9evXAwDq16+P5557zuDtrly5AgDw8vKCtbW1zjb+/v4AgKSkpGf2pz5BUJeMjAx4enoaXBsRERHVHhwx+6/09HQsWLAAADB9+nRYWloavO39+/cBoNxvXKrXZWdnG1ElERER1WUcMQPw6NEjDB48GLm5uQgODsZ7771Xoe0LCwsBAAqFQm8bGxsbANqHS/Up76u05Y2mERERUe1Wa0bMIiIiIElShW+hoaHl9ltcXIzXXnsNp0+fho+PDzZv3qz3cKQ+tra2mr70KSoqAgDY2dlVqG8iIiIyH7VmxMzBwQH169ev8HbOzs5615WUlGDYsGHYtWsXPDw8sHfvXjRt2rTC+1BPRqs+pKmLel1lJq4lIiIi81BrgllUVBSioqJM1l9paSlGjRqFH3/8EQ0aNMCePXsqfZiwRYsWAB6fp6ZUKnWOuF27dk2rLREREdHTas2hTFNSqVT4y1/+gtjYWLi6umL37t1o3bp1pfsLDAyEQqFAUVERjh8/rrPNgQMHAABdu3at9H6IiIiobjPLYDZlyhTExMTAyckJu3btQvv27Y3qz9HREX379gUAREdHl1mfnJyMffv2AQDCw8ON2hcRERHVXWYXzN59912sWLECDg4O2LFjB4KCggzetnv37vDx8cHSpUvLrIuMjIQkSYiJiUF0dDTU14a/efMmRowYAZVKhcGDBxsdAomIiKjuqjXnmJnCkSNHsGTJEgCPR7nef/99vW1nz56tuUST2o0bN5CWloacnJwy7YOCgrB48WK8++67mDx5MqKiouDu7o6LFy+iqKgILVu2xIoVK0x6f4iIiKhuMatgpp6yAng8knXz5k29bf/4448K9x8REYG2bdti0aJFOHbsGG7fvg1vb2+Eh4dj1qxZcHBwqFTdREREZB7MKpiFhoZqDjFWRmpq6jPbhIWFISwsrNL7ICIiIvNldueYEREREckVgxkRERGRTJjVoUySLyEE8vPzda7Ly8sr92c1e3t7SJJk8tqIiIiqC4MZyUJ+fj4GDRpkUNtRo0bpXL5161Z+wYKIiGo1HsokIiIikgkGMyIiIiKZ4KFMkgV7e3ts3bpV57qnzz/Tdy6Zvb19ldVHRERUHRjMSBYkSSr3/DBHR8dqrIaIiKhm8FAmERERkUwwmBERERHJBIMZERERkUwwmBERERHJBIMZERERkUwwmBERERHJBIMZERERkUwwmBERERHJBIMZERERkUwwmBERERHJBIMZERERkUwwmBERERHJhCSEEDVdBBnOzs4OJSUl8PT0rOlSiIiIyEAZGRmwsrJCQUFBue2sqqkeMhEbG5uaLqFGZGRkAAADqZng821e+HybF3N9vq2srAz6DOeIGdUKfn5+AICUlJQaroSqA59v88Ln27zw+S4fzzEjIiIikgkGMyIiIiKZYDAjIiIikgkGMyIiIiKZYDAjIiIikgkGMyIiIiKZ4HQZRERERDLBETMiIiIimWAwIyIiIpIJBjMiIiIimWAwIyIiIpIJBjMiIiIimWAwI4OFhoZCkiSsWbOmpkuRhdTUVEiSBB8fnzLrfHx8IEkSUlNTq70uOSnvMaoua9asgSRJCA0NrbEaaotx48aZ3e94QkICXx8mpH48K/M7b46vP10YzGqBN954A5IkQZIkbN++vabLMdpbb70FSZLw888/A/jfL7L6tmvXrnK3/+ijj7TaU9VTh/Knb7a2tvD19cUbb7yBEydO1HSZZk/9PM2dO7fcdk/+zpH5ycjI0Dz/Fy5c0Nnm7t27sLCwgCRJGDBggN6+ZsyYUeXB9syZM5g7d67ZBDYGM5l78OABtmzZovm5tr8whRD46aefUK9ePfTu3Vtnm7Vr15a7fUxMTFWVZzL+/v5o2bIlrK2ta7oUk/L09ES3bt00N19fX9y8eRPff/89nn/+eaxevbqmSyQjNGnSBC1btoSzs3NNl1Jt6tWrh5YtW8LLy6umS6k2np6e8PX1BQD89ttvOtv89ttvUE9zeujQIahUKp3t9u/fDwAmCWb6Xn9nzpzBvHnzav3nn6EYzGQuNjYWjx49gouLCwDgp59+QnZ2ds0WZYTExERkZmbipZdegq2trdY6FxcXNGrUCFu2bMHDhw91bh8fH4/09HS0atWqOsqttL179+Ly5cto2rRpTZdiUuPHj8fBgwc1t0uXLiEzMxPh4eFQqVT461//ivv379d0mVRJf//733H58mX8+c9/rulSqk1wcDAuX75c7h+EdVHPnj0BlB/MAMDPzw/Z2dn4/fffy7TJy8vD6dOntfozhjm+/nRhMJM59V8IM2bMQIsWLVBUVIT169fXbFFG2LZtGwBg4MCBZdZZWlpi1KhRKCgoQFxcnM7tv/vuOwDA2LFjq65IqpD69etj1apVsLCwwKNHj3Do0KGaLomInsGQYGZlZYV3331Xb7vDhw+jpKQECoUCzz//fNUVa2YYzGTs6tWrOHToECRJwqhRozB69GgA/wsnT3v6pMtVq1ahS5cucHJygiRJyMnJAQBcv34dCxcuRK9eveDt7Q0bGxu4uroiJCQEK1eu1Dtk/aTr16/jjTfeQJMmTWBra4tWrVrhk08+QVFRUbnbbd26FRYWFujfv7/O9WPGjNF7H/Py8rBp0yY0bNgQL7/8st59qM+zSUhI0Lm+vJN9k5KSMG7cOPj4+MDGxgYODg7w9fXFq6++iq+++qrc+/YkfSf/q09unTt3LnJychAREQEfHx/Y2toiICAAn332mebxLyoqQlRUFFq3bg07Ozs0bdoUERERyM/PN7iO6uLk5ARXV1cAQHFxsUHbnD9/HvPmzUP37t3RrFkzKBQKuLu7o0+fPti0aVO526pUKqxbtw4vv/wyGjZsCBsbGzRr1gy9e/fGV1999czXoZpSqcTIkSMhSRJat26NjIwMg7arq8o7+Xr9+vUICwtD/fr1YW1tDXd3d7Rp0waTJk3CsWPHtNrOnTsXkiRh3LhxyM/PxwcffIDmzZvD1tYWTZs2xeTJk3Hr1i2dNVT2daH+nUtISMDVq1cxatQoNG7cGLa2tmjdujUWLlyo873tWSf/FxcX46uvvkJoaCjq168PGxsbeHt7o3///nrfi2sDdTDLysrC1atXtdbl5ubi7Nmz6Nixo+a9Vn3I8knqZcHBwbCzs9O5n9jYWLzwwgtwdHSEk5MTevfurfePN12vPx8fH/zlL3/R7O/Jc1x1fcHg/PnzGD9+PHx9fWFrawsXFxeEhoZi3bp15T8gciJItj766CMBQPTo0UMIIcT169eFJEkCgLh48WKZ9vHx8QKA8Pb2Fm+99ZYAIJo2bSqCgoKEm5ubyM7OFkII8eabbwoAol69eqJ58+YiKChI+Pj4CAACgBg6dKjOenr27CkAiNmzZwtXV1dhbW0tAgMDRUBAgGbb0NBQUVBQoHP71NRUAUB07dpVZ93169cXQgjRvn17IUmSSElJ0Wq3Zs0aAUBERESI33//XbNPfXXGx8frrEO9v549e2otP3nypLC3txcAhJ2dnWjTpo0IDAwUDRo0EACEs7OzVvvr169rHu+neXt7CwDi+vXrWsvHjh0rAIhp06aJli1bah5DLy8vzf2ZOnWqKCgoEN26dRMWFhaiTZs2IiAgQPPc9+3bV+f9qkrqx/Tjjz/Wuf7atWua+s+fP69ZXt5jFBYWJgAIJycn0bJlS9G5c2fRtGlTTT/Tp0/Xua+HDx+Kl156SdOucePGIigoSHh6egoLC4syj/vq1at1Pt95eXmib9++AoAIDg4Wd+/erejDIjvPep7U1L8DT//+qF+fq1ev1lr+wQcfaNo3atRIdOrUSbRs2VLUq1dP83p+0scffywAiOHDh4ugoCABQLRs2VJ06NBBWFlZCQDCw8NDXLt2rUxtlX1dqH/nFi9eLJycnISdnZ3o1KmTaNasmWbbt99+W+9j8fTrQwghsrKyRGBgoGZ7Ly8v0blzZ9G4cWO97z+1ifqxWbVqldbyn3/+WeuxbtKkiWjYsGGZ7bt37675TFB78nNI/Rnm4eEhOnbsKBwdHQUAoVAoxMGDB8v0p+v1Fx4ervmMcXJyEt26ddPcwsPDtbaPjo7WvL4cHR1F+/bthYeHh+a5GjdunDEPV7Wp3a+qOkylUmk+rFesWKFZ3qNHDwFAzJw5s8w26l8IS0tLYWdnJzZt2qRZV1RUJEpLS4UQQvzyyy/i0KFDmp/VkpKSxAsvvCAAiHXr1pXpX/2mb21tLbp37y6ysrI06w4dOiTq168vAIgPP/xQ531atmyZACD+/ve/66xbHcwWL14sAIh58+ZptXvxxRcFAHH69OkqCWYDBw4UAMTo0aPFgwcPtNZdv35dLF68uMyyygYza2tr0aNHD63HcO3atQKAsLCwEIMHDxbPPfecuHLlimb97t27hbW1tQAgdu3apfO+VRV9H/j37t0Tv/76q2jXrp0AIP785z9rrS/vMYqLixOnT58uszwxMVG0aNFCABCHDh0qs37YsGECgGjSpIn49ddftdbduXNHLFy4UNy+fVuzTFcwu3fvnujSpYsAIF566SWRl5f37AehFqiKYHbnzh1haWkprKysxMaNG4VKpdKsKy0tFXv37hXbtm3T6kcdzKytrUXjxo3F8ePHNesyMjI0Ye2FF14oU1tlXxfq3zlra2sxYcIE8fDhQ826DRs2CEmShCRJIikpSedj8fT7QWlpqejatasAIFq1aqV1H4QQIj09XcyZM6dMHbXJqFGjBAAxduxYreXvv/++ACC2bt0qhBDi9ddfFwDEpUuXNG0KCgqEjY2NAKD1e6h+PK2srIS9vb3YuHGjZt2jR49EeHi43ude3x8G+v64etK+ffuEhYWFsLe3F6tWrdL6fEtISBBNmjQp83kqVwxmMrVnzx4BQNja2oqcnBzN8hUrVmj+AikpKdHa5sk324ULF1Zqv8nJyQKAePnll8usU7/pKxQKkZmZWWa9+pfH0dFR601RTf2X8IULF3TWrQ5mt27dElZWVsLf31/TJjU1VUiSJNq2bSuEEFUSzFq2bCkAiDNnzujc7mnGBDM7OzuRkZFRZjt1WJAkSZw4caLMevUb5NMjFFVN/Zjqu7m4uIjPPvtMFBcXa21X3mNUnt27dwsAYsqUKVrLT506pQmviYmJBvX19Jt6RkaGaN26tQAghg0bJoqKiipUm5w963nSdXuSrg/GI0eOCACiQ4cOBtehDmYARFxcXJn1V69e1Yxu7t+/3+B+9b0uhPjf71ybNm3KvDcK8b8/vJYsWaK1XN/7webNmzXvZ+np6QbXWJtER0cLAMLHx0dreZcuXYQkSeLevXtCCCGWL18uAIivv/5a0+bJAPbkHzZPfg599tlnZfZ569YtoVAoBABx//59rXXGBLPg4GABQHz77bc612/dulUAEAEBAXr7kAsrkCypz13o37+/1leHhw4dinfeeQdZWVnYs2cP+vbtq3P7Z50cn5OTgw0bNuDw4cO4efMmCgoKNF+NBqD5po0ur732Gjw8PMosHzlyJN59911kZ2fj0KFDWrXl5ubit99+Q/PmzfHcc8+VW1ujRo3Qt29fbN++HQcPHkT37t0RExMDIUSVnvTv5eWFpKQkxMXFoV27dlU6x9PLL7+MZs2alVnesWNHHDt2DB06dEDnzp11ro+NjcW1a9eqrLbyeHp6ak0rkJ+fj9TUVOTk5CA6Ohpt2rTRe/6gLrdu3cL69etx4sQJ3L59G4WFhQCgOUfs6dfh5s2bAQAvvfQSOnbsWOH6L1++jD59+iAjIwNTp07F8uXLYWFR9061ffp5elpubi7Onz9vUF/qfq5cuYLTp08jMDDQ4Do8PDzw2muvlVnu7++Pfv36Yfv27dixYwd69Oihtb6ir4snTZgwAZaWlmWWP//889i2bZvBvzvq19rIkSPh6elp0Da1jfo8s9TUVNy4cQPNmjXDo0ePcOrUKbRp0wZubm4AgJCQEACPvwAwefJkzf8BoHPnzrC3t9fZv7rtkxo1agQfHx9cuXIFKSkp6NSpk9H348aNGzh+/Djq1auHUaNG6WzzyiuvwNraGsnJycjKytL5GSYXDGYylJeXp3lTUJ/wr+bs7IyBAwciNjYWa9as0RnM3N3d0aBBA739JyQkYOjQobh7967eNvfu3dO7Tl+wUigUaN68OU6cOIGkpCSt2nbs2AGlUqnz25i6jB07Ftu3b8fatWvRvXt3rF27VvOtzaoyffp07NmzB5988gnWrl2Ll19+GS+88AJefPFFeHt7m3Rf/v7+Opernzd96xs2bAjg8WukJowfP77M5KUqlQr/+c9/8Oabb2Lw4MHYsWMHXnrppWf2tWHDBrz55pvlfpnh6dfhxYsXAQBdu3atcO2pqakICQnB3bt3MWfOHMybN6/CfdQWup6nJyUkJODFF180qC8PDw+MHDkS69atQ6dOndC1a1e8+OKL6NatG3r06KH3QxkAWrVqpTf4Pvfcc9i+fTuSkpK0llfmdfGkgIAAncsbNWoEwPDfHWNea7VFixYt0LhxY9y6dQv79+/HqFGjcPjwYSiVSq2w3LZtWzg7O2t9M1P9f33TZLi7u2umeXpao0aNcOXKFZO9j509e1bz/169eultp/5j+8aNG7IOZnXvT8U6IDY2Fvn5+ahfvz769etXZr36m4s//vgjcnNzy6wv743ywYMHmlD2+uuv4+DBg7h79y6USiWEECgtLQUAlJSU6O1DHQ50Ub/5PT0P2datWwHoniZDl4EDB8LV1RWxsbHYu3cvkpOT0bdvXzRu3Nig7SujT58+2L17N0JDQ5GVlYUVK1bgL3/5C3x8fNCtWzccPXrUZPvS9xyp3zietf7J0c2aZmFhgTFjxuCdd95BaWkpZs2a9cxtrl+/jjFjxiA/Px9vv/02jh8/juzsbJSUlEAIoRnVUCqVWts9ePAAACo1Aer9+/dx7949WFpaVmjUh4DVq1fjk08+gY+PDw4fPoxPPvkEr7zyCho0aIApU6bofB8CKv5eUdnXxZP0/e6oA6KhvzvGvNZqE3WwUn/DUv3vk8HMwsIC3bp1w40bN5CSkgKlUokjR45obf+08j6HKvpcPIt6xgH1dD36bupvjD969Mgk+60qDGYypP6q8L1796BQKMpcBkd9qKiwsBAbNmyoUN+//PIL7t69i+DgYKxfvx7dunVD/fr1YWVlpdnns9y+fVvvuj/++AMA4OjoqFlWUlKCnTt3ws3NDd27dzeoThsbG7z++uvIzc3Fm2++CeB/gfRZnhVeyvtLPCwsDPHx8cjOzsbOnTsxa9Ys+Pn54fDhw3jppZdw/fp1g2owR926dQPw+DDTs6ar2LBhA4qLixEeHo5//etfCAoKgouLi+YQlL7XoZOTEwDoDQLl6dixI/79739DpVLh9ddf1/yxQM+mUCjw4YcfIiUlBSkpKVi7di3eeOMNAMA333yDkSNH6tyuou8VlX1dVAVjXmu1iTqAqUfA1P+qD1+qPXk48+TJk3j06BEsLS01v/c1ycHBAcDjUVjx+Nz5cm9yvy4qg5nMpKSk4ODBgwAe/7XZqFEjnTf1m0ZF59FRB4vu3bvrPMRgyKiQeoj/aUqlUvMXbcuWLTXL9+/fj5ycHLzyyis6z/3QR30+WVpaGpydnTFo0CCDtlP/paZ+43/alStXntmHo6Mj+vbti08//RSXLl1Chw4dkJeXh//85z8GVm9+1HNEqVQqzV+w+qhfh0+fW6Sm73X4pz/9CQA0f61X1FtvvYV///vfKCkpwdChQxnOKsHX1xejR49GTEyMZq7AX375RecccElJSXrnRbx06RIA7feKyr4uqoKxr7XaQj3ilZSUhPT0dBw/fhz+/v5lDvU9GczU4S0wMFDzWVSVnnW+r/q5SklJQUFBQZXXU9UYzGTmu+++gxACAQEB+OOPP3Dr1i2dN/Vw8+HDh5GcnGxw//Xq1QMA3Lx5s8w6IQQ+//zzZ/axefNmnduvW7cO9+/fh4ODg9ZfURU9jKnWtWtXvP766wgLC8P7779f5hJO+qjPMdH1hlpSUoIVK1ZUqA6FQqE5QTUrK6tC25oT9R8UTk5OcHd3L7dtea/DwsJCLF++XOd26hPJd+/ejTNnzlSqzrfeegtffvmlJpypr0ZBFdepUyfY2NgA0P27kZmZiR9//LHM8pSUFOzYsQMAtCaLruzroiqoX2vr1q1DZmZmte23uj333HOa39eFCxeisLCwzGgZAAQFBcHW1hb79+/XfP6Y4jJMhlBPXqsvdPn7+yMwMBCFhYX497//XS01VSUGMxkRQmiu1/asw3YdOnRAu3btAFRs1Ez9l2hcXBy2b9+uWf7w4UNMmDABx48fN6jOESNGaM3cfeTIEcyYMQMA8Ne//lUztAw8vr6nQqEod7Z+fTZs2IA9e/bgww8/NHgb9aHeVatWIT4+XrP8wYMHmDhxYplZrtWGDRuGrVu3ar4Bpnbs2DFNuDTFN4jqGpVKhdWrV+PLL78E8PgLK88aGVW/Dr/88kucOHFCs/z27dsIDw/XOwN/+/btMWLECKhUKrz66qvYu3ev1vq7d+9i0aJFuHPnTrn7nzJlCr766iuUlJQgPDyc4awce/fuxXvvvYdz585pLVcqlfj0009RVFSEevXqoXXr1mW2tba2xjvvvIPExETNsszMTIwcORKlpaXo2rWr1od7ZV8XVWHgwIHo1q0bHj58iD59+mjdB+DxCeTlfcGitpAkSfO4r1y5EoDuEUuFQoHg4GCkpKRo3lerK5ipvwx18eJFvb/bn3/+OSwtLTFr1iwsXLiwzHlkOTk5+M9//qP5nJK16pybg8q3b98+zRxWqampz2y/aNEiATyejbq0tFRrxuXyjBw5UjPPjK+vr+jUqZOoV6+ekCRJM1+MrpfG0zP/KxQKERgYqJn/C4AICQkRjx490mxz9uxZAUD06dNHbz1Pz2NmiPLmMRNCiEGDBmkeS19fX9GxY0dha2srnJ2dxdKlS3XOiePs7KyZoLJ169aiS5cuWldE6NWrl1AqlZr2xsxjpm8CUPX8T09P+KhmyHw+VUH93Ht6emrNvN2hQwfN4wZAdO/eXWtyXn2PUUlJiQgJCdE8Ry1atBCBgYHC2tpaKBQKzfxKuh7bBw8eaObEw38nmg0KChJeXl4VmvlfCCG+/vprIUmSUCgUmsk0a7OqmGB2y5Ytmraurq6iY8eOIjAwULi4uGiev6cn7Xxy5v/OnTsLAKJ169YiMDBQMzN748aNRXJystZ2xrwu1L9z+uYvVL8Onv7detbM/x06dNDcfx8fHxEUFCSaNGmiuRJHXfDFF19ozW339POiNnv2bE0bCwsLzdVknmTI55C+uSb1zWNWWlqq+ZxxdHQUXbp0ET179hTDhg3Tard27Vpha2srAAgbGxvRvn17ERwcLHx9fTXPV3W/d1YGR8xkRD3y1bNnT4OmZxg1ahSsrKyQnp6Offv2VWg/n3zyCQICAnDjxg2kpqaiR48e2LNnD8aNG/fM7Zs3b47ExEQMHToUN2/exPXr1xEQEID58+dj9+7dWtdMq+xhTGNt2LABc+bMgZ+fHzIzM3Hjxg289tprOHXqFNq3b69zm7Vr1+Ktt97Cc889h7t37yIxMRG5ubkICQnBV199hV27dmm+JGGuMjIytL7ldP78eSgUCvTu3RsrV65EQkKC1snc+lhaWmLHjh2YPn06PD09cf36dWRlZWHAgAE4cuRIudNtODo6YteuXVizZg169eqFoqIinD17FiqVCr169cLXX39t8FfhJ0+ejK+//hpKpZKHNfUICQnB8uXLMXjwYLi5ueHKlSs4f/48HB0dMXz4cBw+fBgTJkzQua2NjQ0SEhIwc+ZMFBUV4cKFC3B3d8eECRNw6tQpNG/eXKu9Ma+LqtCkSRMcPXoUX3zxBV544QVkZ2fj3LlzsLGxQf/+/RETE1Ot9VSVJ0e+mjRpUuZ5UXvyEGe7du30TodhahYWFvjll18wbNgw2Nvb4+TJk9i/f3+Zcw5Hjx6NCxcuYNq0afD398fVq1dx6tQpFBQUoHfv3vjiiy9qxXnCkhAy+t491TlBQUE4efIk0tPT6+wkjUSkbe7cuZg3bx7Gjh2r84LoRKQfR8yoymRlZSExMREdOnRgKCMiIjKAeR+XoSrl4eGh96vyREREVBZHzIiIiIhkgsGMiIiISCZ48j8RERGRTHDEjIiIiEgmGMyIiIiIZILBjIiIiEgmGMyIiIiIZILBjIiIiEgmGMyIiIiIZILBjIiIiEgmGMyIiIiIZILBjIiIiEgmGMyIiP5LCIHVq1eja9eucHR0hLOzM7p06YLo6GgIITBu3DhIkoRx48Zpbefj4wNJkrBmzRrk5eVhzpw5aNu2LRwdHSFJElJTUzVtS0tL8e2336JXr15wd3eHjY0NmjZtiqFDhyIhIUFvbU/uQx999T29/cOHDzFr1iy0bNkSdnZ2cHd3x+DBg3Hs2LGKPWBEZHJWNV0AEZEclJaWYtSoUdiwYQMAQJIkuLi44OTJkzh+/DgSEhKgUCjK7ePevXvo1KkTrly5AoVCgXr16mmtz83NxeDBgzUBzNLSEo6Ojrh58yY2btyIjRs3Yvr06Vi4cGGV3EcAyM7ORlBQEJKSkqBQKGBra4t79+5h69at+Omnn7BixQqMHz++yvZPROXjiBkREYCFCxdqQtm7776LO3fu4P79+8jOzsann36KH374Adu2bSu3j7lz5+LBgwfYsmUL8vLykJ2djYyMDDRs2BAA8Oabb2oC3rJly/DgwQNkZ2cjKytLE4Y+//xzfP3111V2P+fNm4fbt28jNjYW+fn5yM3NxcWLF9GzZ0+oVCpMnjwZp06dqrL9E1H5GMyIyOzl5+fj73//O4DH4WnRokWoX78+AMDJyQmzZs3CnDlzkJ2dXW4/BQUF+OWXXzB48GBYW1sDAJo1a4Z69erh2LFj2LRpEwBg+fLleOeddzQjao0bN8aqVaswZMgQAEBkZCQKCwur5L7m5uYiLi4OQ4cOhZXV44MmrVu3xo4dOxAQEICSkhJERkZWyb6J6NkYzIjI7P3666948OABAGD27Nk627z33ntlDk0+7eWXX0ZgYKDOderRuGbNmmHChAk62yxYsAAAcPfuXezevdug2iuqW7duCAsLK7Pczs4OM2bMAADs3LkTubm5VbJ/IiofgxkRmT31oTsvLy/4+vrqbOPo6IhOnTqV20+3bt30rjt58iQA4MUXX4SFhe633tatW6Np06Za7U2tV69ez1ynUql4OJOohjCYEZHZu3PnDgDAw8Oj3Hbq0KSP+lwyXW7fvm1QH82aNdNqb2rl7f/JdVW1fyIqH4MZEdF/SZJk1PaWlpYmqoSIzBWDGRGZvQYNGgAAsrKyym2XmZlZ6X2oR9Nu3LhRbjv1+qdH39Qn6pf3pQBDzgsr7z48ua680T8iqjoMZkRk9jp27AgASEtL05oM9kl5eXlITEys9D46d+4MAIiPj4dKpdLZ5vLly5pwFBQUpLXO1dUVAJCRkaFzW5VKZdB5afHx8c9cZ2FhofdLDERUtRjMiMjs9enTB05OTgCATz/9VGebJUuW4NGjR5Xex/DhwwE8HpVauXKlzjZz5swBALi7u6N3795a69q3bw8A2LJlC4QQZbb97rvvnjkaBwAHDx7UeYWBwsJCLFq0CADQt29fuLi4PLMvIjI9BjMiMnv29vaYOXMmAGDFihV4//33cf/+fQDAw4cP8Y9//ANz587VjFpVRnBwsGaesnfeeQf/+te/NEHv1q1bmDhxIuLi4gA8njbD1tZWa/sRI0YAAC5duoRJkybh3r17AIAHDx5gyZIlmDJlCtzc3J5Zh7OzM4YMGYKNGzeipKQEwOORuldffRWXL1+GpaUl5s+fX+n7SURGEkREJJRKpQgPDxcABABhYWEhXF1dhaWlpQAgRo8eLcaMGSMAiMmTJ2tt6+3tLQCI1atXl7uPnJwc0bNnT80+rKyshKurq5AkSbNs+vTpercfPXq0ph0A4eLiIiwsLAQA8c4774ixY8cKAGLs2LFltlXXuHjxYtGyZUsBQNjY2AhnZ2dNf5Ikiejo6Mo8fERkIhwxIyLC45PrY2NjsXLlSgQHB8POzg4lJSXo3LkzVq5cibVr1yInJwcAKn2Yz9nZGXv37sWqVasQGhoKR0dH5OXloXHjxhgyZAji4+PLvU7mmjVr8MUXX6BDhw6ws7ODSqVCt27dEBsbi2XLlhlUg6urK44fP44PPvgAXl5eKCoqgpubGwYMGIBDhw5h4sSJlbpvRGQakhA6TlYgIiItQgh4eXnhxo0bWLt2LUaPHl3TJVWIj48P0tLSsHr1aowbN66myyEiPThiRkRkgJiYGNy4cQNWVlZlTswnIjIVBjMiov8aMWIENm7ciLt372qW/fHHH/jss880h/jGjBmDJk2a1FSJRFTHWdV0AUREcrFjxw788MMPAIB69erB2tpaa9LWkJAQLFmypKbKIyIzwGBGRPRfy5Ytw44dO3D69Gncvn0beXl5aNCgATp06IDhw4dj9OjRsLa2rukyiagO48n/RERERDLBc8yIiIiIZILBjIiIiEgmGMyIiIiIZILBjIiIiEgmGMyIiIiIZILBjIiIiEgmGMyIiIiIZILBjIiIiEgmGMyIiIiIZOL/AR0UyK9eH+5qAAAAAElFTkSuQmCC\n"
          },
          "metadata": {}
        }
      ],
      "source": [
        "plot = seaborn.barplot(x='group', y='sentiment', data=name_sentiments, capsize=.1)"
      ]
    },
    {
      "cell_type": "markdown",
      "metadata": {
        "id": "5toY9syHk_Nk"
      },
      "source": [
        "And finally we can break out the serious statistical machinery, using the [statsmodels](http://www.statsmodels.org/stable/index.html) package, to tell us how big of an effect this is (along with a bunch of other statistics)."
      ]
    },
    {
      "cell_type": "code",
      "execution_count": null,
      "metadata": {
        "id": "-_Hqn7REk_Nl",
        "colab": {
          "base_uri": "https://localhost:8080/",
          "height": 229
        },
        "outputId": "36ca15d1-88f7-4efd-8921-f9d06ac7fd17"
      },
      "outputs": [
        {
          "output_type": "execute_result",
          "data": {
            "text/plain": [
              "<class 'statsmodels.iolib.table.SimpleTable'>"
            ],
            "text/html": [
              "<table class=\"simpletable\">\n",
              "<caption>OLS Regression Results</caption>\n",
              "<tr>\n",
              "  <th>Dep. Variable:</th>        <td>sentiment</td>    <th>  R-squared:         </th> <td>   0.196</td>\n",
              "</tr>\n",
              "<tr>\n",
              "  <th>Model:</th>                   <td>OLS</td>       <th>  Adj. R-squared:    </th> <td>   0.178</td>\n",
              "</tr>\n",
              "<tr>\n",
              "  <th>Method:</th>             <td>Least Squares</td>  <th>  F-statistic:       </th> <td>   11.35</td>\n",
              "</tr>\n",
              "<tr>\n",
              "  <th>Date:</th>             <td>Mon, 30 Sep 2024</td> <th>  Prob (F-statistic):</th> <td>1.04e-06</td>\n",
              "</tr>\n",
              "<tr>\n",
              "  <th>Time:</th>                 <td>20:05:59</td>     <th>  Log-Likelihood:    </th> <td> -297.53</td>\n",
              "</tr>\n",
              "<tr>\n",
              "  <th>No. Observations:</th>      <td>   144</td>      <th>  AIC:               </th> <td>   603.1</td>\n",
              "</tr>\n",
              "<tr>\n",
              "  <th>Df Residuals:</th>          <td>   140</td>      <th>  BIC:               </th> <td>   614.9</td>\n",
              "</tr>\n",
              "<tr>\n",
              "  <th>Df Model:</th>              <td>     3</td>      <th>                     </th>     <td> </td>   \n",
              "</tr>\n",
              "<tr>\n",
              "  <th>Covariance Type:</th>      <td>nonrobust</td>    <th>                     </th>     <td> </td>   \n",
              "</tr>\n",
              "</table>"
            ],
            "text/latex": "\\begin{center}\n\\begin{tabular}{lclc}\n\\toprule\n\\textbf{Dep. Variable:}    &    sentiment     & \\textbf{  R-squared:         } &    0.196  \\\\\n\\textbf{Model:}            &       OLS        & \\textbf{  Adj. R-squared:    } &    0.178  \\\\\n\\textbf{Method:}           &  Least Squares   & \\textbf{  F-statistic:       } &    11.35  \\\\\n\\textbf{Date:}             & Mon, 30 Sep 2024 & \\textbf{  Prob (F-statistic):} & 1.04e-06  \\\\\n\\textbf{Time:}             &     20:05:59     & \\textbf{  Log-Likelihood:    } &  -297.53  \\\\\n\\textbf{No. Observations:} &         144      & \\textbf{  AIC:               } &    603.1  \\\\\n\\textbf{Df Residuals:}     &         140      & \\textbf{  BIC:               } &    614.9  \\\\\n\\textbf{Df Model:}         &           3      & \\textbf{                     } &           \\\\\n\\textbf{Covariance Type:}  &    nonrobust     & \\textbf{                     } &           \\\\\n\\bottomrule\n\\end{tabular}\n%\\caption{OLS Regression Results}\n\\end{center}"
          },
          "metadata": {},
          "execution_count": 58
        }
      ],
      "source": [
        "ols_model = statsmodels.formula.api.ols('sentiment ~ group', data=name_sentiments).fit()\n",
        "ols_model.summary().tables[0]"
      ]
    },
    {
      "cell_type": "markdown",
      "metadata": {
        "id": "kWbYfC7Kk_Nl"
      },
      "source": [
        "The F-statistic is the ratio of the variation between groups to the variation within groups, which we can take as a measure of overall ethnic bias.\n",
        "\n",
        "The probability, right below that, is the probability that we would see this high of an F-statistic given the null hypothesis: that is, given data where there was no difference between ethnicities. The probability is very, very low. If this were a paper, we'd get to call the result \"highly statistically significant\".\n",
        "\n",
        "Out of all these numbers, the F-value is the one we really want to improve. A lower F-value is better."
      ]
    },
    {
      "cell_type": "code",
      "execution_count": null,
      "metadata": {
        "id": "MLKXo4LKk_Nm",
        "colab": {
          "base_uri": "https://localhost:8080/",
          "height": 141
        },
        "outputId": "10aca5ab-2495-4e82-8f3a-11055aa43033"
      },
      "outputs": [
        {
          "output_type": "error",
          "ename": "NameError",
          "evalue": "name 'ols_model' is not defined",
          "traceback": [
            "\u001b[0;31m---------------------------------------------------------------------------\u001b[0m",
            "\u001b[0;31mNameError\u001b[0m                                 Traceback (most recent call last)",
            "\u001b[0;32m<ipython-input-1-4a7ddd24b6e3>\u001b[0m in \u001b[0;36m<cell line: 1>\u001b[0;34m()\u001b[0m\n\u001b[0;32m----> 1\u001b[0;31m \u001b[0mols_model\u001b[0m\u001b[0;34m.\u001b[0m\u001b[0mfvalue\u001b[0m\u001b[0;34m\u001b[0m\u001b[0;34m\u001b[0m\u001b[0m\n\u001b[0m",
            "\u001b[0;31mNameError\u001b[0m: name 'ols_model' is not defined"
          ]
        }
      ],
      "source": [
        "ols_model.fvalue"
      ]
    },
    {
      "cell_type": "markdown",
      "metadata": {
        "id": "ukGovxCxk_Nn"
      },
      "source": [
        "### What about another dataset?\n",
        "\n",
        "You may think this is a problem that only GloVe has. If the system weren't trained on all of the Common Crawl (which contains lots of unsavory sites and like 20 copies of Urban Dictionary), maybe it wouldn't have gone bad. What about good old word2vec, trained on Google News?\n",
        "\n",
        "Turns out word2vec is even worse. With an F-value over 15, it has even larger differences in sentiment between groups.\n",
        "\n",
        "In retrospect, expecting _news_ to be safe from algorithmic bias was rather a lot to hope for."
      ]
    },
    {
      "cell_type": "markdown",
      "metadata": {
        "id": "XTVRJa05k_Nq"
      },
      "source": [
        "### There is no trade-off\n",
        "\n",
        "Some people expect that fighting algorithmic racism is going to come with some sort of trade-off. There's no trade-off here. You can have data that's better and less racist. You can have data that's better *because* it's less racist. There was never anything \"accurate\" about the overt racism that word2vec and GloVe learned."
      ]
    },
    {
      "cell_type": "markdown",
      "metadata": {
        "collapsed": true,
        "id": "YD36k7qpk_Nq"
      },
      "source": [
        "## Other approaches\n",
        "\n",
        "This is of course only one way to do sentiment analysis. All the steps we used are common, but you probably object that you wouldn't do it that way. But if you have your own process, I urge you to see if your process is encoding prejudices and biases in the model it learns.\n",
        "\n",
        "Instead of or in addition to changing your source of word vectors, you could try to fix this problem in the output directly. It may help, for example, to build a stronger model of whether sentiment should be assigned to words at all, designed to specifically exclude names and groups of people.\n",
        "\n",
        "You could abandon the idea of inferring sentiment for words, and only count the sentiment of words that appear exactly in the list. This is perhaps the most common form of sentiment analysis -- the kind that includes no machine learning at all. Its results will be no more biased than whoever made the list. But the lack of machine learning means that this approach has low recall, and the only way to adapt it to your data set is to edit the list manually.\n",
        "\n",
        "As a hybrid approach, you could produce a large number of inferred sentiments for words, and have a human annotator patiently look through them, making a list of exceptions whose sentiment should be set to 0. The downside of this is that it's extra work; the upside is that you take the time to actually see what your data is doing. And that's something that I think should happen more often in machine learning anyway.\n"
      ]
    },
    {
      "cell_type": "code",
      "source": [],
      "metadata": {
        "id": "rDG24jCbjasF"
      },
      "execution_count": null,
      "outputs": []
    }
  ],
  "metadata": {
    "colab": {
      "collapsed_sections": [
        "XTVRJa05k_Nq",
        "YD36k7qpk_Nq"
      ],
      "provenance": [],
      "include_colab_link": true
    },
    "kernelspec": {
      "display_name": "Python 3",
      "language": "python",
      "name": "python3"
    },
    "language_info": {
      "codemirror_mode": {
        "name": "ipython",
        "version": 3
      },
      "file_extension": ".py",
      "mimetype": "text/x-python",
      "name": "python",
      "nbconvert_exporter": "python",
      "pygments_lexer": "ipython3",
      "version": "3.8.12"
    }
  },
  "nbformat": 4,
  "nbformat_minor": 0
}