06 - Day - 6 Advanced NLP and its Applications/05 - Exploring_Word_Embeddings_with_Deep_Learning_Models.ipynb

{
 "cells": [
  {
   "cell_type": "markdown",
   "metadata": {
    "colab_type": "text",
    "id": "JS9QXHRoRiqc"
   },
   "source": [
    "# Text Representation with Feature Engineering\n",
    "\n",
    "### Project 7 - Exploring Word Embeddings with New Deep Learning Models\n",
    "\n",
    "We have discussed in the previous sub-unit that Feature Engineering is the secret sauce to creating superior and better performing machine learning models. \n",
    "\n",
    "Traditional (count-based) feature engineering strategies for textual data involve models belonging to a family of models popularly known as the Bag of Words model. This includes term frequencies, TF-IDF (term frequency-inverse document frequency), N-grams and so on. While they are effective methods for extracting features from text, due to the inherent nature of the model being just a bag of unstructured words, we lose additional information like the semantics, structure, sequence and context around nearby words in each text document. \n",
    "\n",
    "This forms as enough motivation for us to explore more sophisticated models which can capture this information and give us features which are vector representation of words, popularly known as embeddings.\n",
    "\n",
    "Here we will explore the following feature engineering techniques:\n",
    "\n",
    "- Word2Vec\n",
    "- GloVe\n",
    "- FastText\n",
    "\n",
    "Predictive methods like Neural Network based language models try to predict words from its neighboring words looking at word sequences in the corpus and in the process it learns distributed representations giving us dense word embeddings. We will be focusing on these predictive methods in this article."
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {
    "colab_type": "text",
    "id": "wYUGSn3dRiqd"
   },
   "source": [
    "# Prepare a Sample Corpus\n",
    "\n",
    "Let’s now take a sample corpus of documents on which we will run most of our analyses in this article. A corpus is typically a collection of text documents usually belonging to one or more subjects or domains."
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 1,
   "metadata": {
    "colab": {
     "base_uri": "https://localhost:8080/",
     "height": 297
    },
    "colab_type": "code",
    "id": "KH4ZwBgtRiqe",
    "outputId": "aedbc229-8ecc-4251-f5b0-0a6eb04d16ff"
   },
   "outputs": [
    {
     "data": {
      "text/html": [
       "<div>\n",
       "<style scoped>\n",
       "    .dataframe tbody tr th:only-of-type {\n",
       "        vertical-align: middle;\n",
       "    }\n",
       "\n",
       "    .dataframe tbody tr th {\n",
       "        vertical-align: top;\n",
       "    }\n",
       "\n",
       "    .dataframe thead th {\n",
       "        text-align: right;\n",
       "    }\n",
       "</style>\n",
       "<table border=\"1\" class=\"dataframe\">\n",
       "  <thead>\n",
       "    <tr style=\"text-align: right;\">\n",
       "      <th></th>\n",
       "      <th>Document</th>\n",
       "      <th>Category</th>\n",
       "    </tr>\n",
       "  </thead>\n",
       "  <tbody>\n",
       "    <tr>\n",
       "      <th>0</th>\n",
       "      <td>The sky is blue and beautiful.</td>\n",
       "      <td>weather</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>1</th>\n",
       "      <td>Love this blue and beautiful sky!</td>\n",
       "      <td>weather</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>2</th>\n",
       "      <td>The quick brown fox jumps over the lazy dog.</td>\n",
       "      <td>animals</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>3</th>\n",
       "      <td>A king's breakfast has sausages, ham, bacon, eggs, toast and beans</td>\n",
       "      <td>food</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>4</th>\n",
       "      <td>I love green eggs, ham, sausages and bacon!</td>\n",
       "      <td>food</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>5</th>\n",
       "      <td>The brown fox is quick and the blue dog is lazy!</td>\n",
       "      <td>animals</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>6</th>\n",
       "      <td>The sky is very blue and the sky is very beautiful today</td>\n",
       "      <td>weather</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>7</th>\n",
       "      <td>The dog is lazy but the brown fox is quick!</td>\n",
       "      <td>animals</td>\n",
       "    </tr>\n",
       "  </tbody>\n",
       "</table>\n",
       "</div>"
      ],
      "text/plain": [
       "                                                             Document Category\n",
       "0                                      The sky is blue and beautiful.  weather\n",
       "1                                   Love this blue and beautiful sky!  weather\n",
       "2                        The quick brown fox jumps over the lazy dog.  animals\n",
       "3  A king's breakfast has sausages, ham, bacon, eggs, toast and beans     food\n",
       "4                         I love green eggs, ham, sausages and bacon!     food\n",
       "5                    The brown fox is quick and the blue dog is lazy!  animals\n",
       "6            The sky is very blue and the sky is very beautiful today  weather\n",
       "7                         The dog is lazy but the brown fox is quick!  animals"
      ]
     },
     "execution_count": 1,
     "metadata": {},
     "output_type": "execute_result"
    }
   ],
   "source": [
    "import pandas as pd\n",
    "import numpy as np\n",
    "\n",
    "pd.options.display.max_colwidth = 200\n",
    "\n",
    "corpus = ['The sky is blue and beautiful.',\n",
    "          'Love this blue and beautiful sky!',\n",
    "          'The quick brown fox jumps over the lazy dog.',\n",
    "          \"A king's breakfast has sausages, ham, bacon, eggs, toast and beans\",\n",
    "          'I love green eggs, ham, sausages and bacon!',\n",
    "          'The brown fox is quick and the blue dog is lazy!',\n",
    "          'The sky is very blue and the sky is very beautiful today',\n",
    "          'The dog is lazy but the brown fox is quick!'    \n",
    "]\n",
    "labels = ['weather', 'weather', 'animals', 'food', 'food', 'animals', 'weather', 'animals']\n",
    "\n",
    "corpus = np.array(corpus)\n",
    "corpus_df = pd.DataFrame({'Document': corpus, \n",
    "                          'Category': labels})\n",
    "corpus_df = corpus_df[['Document', 'Category']]\n",
    "corpus_df"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {
    "colab_type": "text",
    "id": "FLojJzK0Riqi"
   },
   "source": [
    "Let's go ahead and pre-process our text data now"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {
    "colab_type": "text",
    "id": "7KR8w3qbRiqi"
   },
   "source": [
    "# Simple Text Pre-processing\n",
    "\n",
    "Since the focus of this unit is on feature engineering, we will build a simple text pre-processor which focuses on removing special characters, extra whitespaces, digits, stopwords and lower casing the text corpus."
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 2,
   "metadata": {
    "colab": {
     "base_uri": "https://localhost:8080/",
     "height": 119
    },
    "colab_type": "code",
    "id": "5YHHrL7VRiqj",
    "outputId": "9f5da9ac-ffec-4d91-b4ef-28791e90aea1"
   },
   "outputs": [
    {
     "data": {
      "text/plain": [
       "array(['sky blue beautiful', 'love blue beautiful sky',\n",
       "       'quick brown fox jumps lazy dog',\n",
       "       'kings breakfast sausages ham bacon eggs toast beans',\n",
       "       'love green eggs ham sausages bacon',\n",
       "       'brown fox quick blue dog lazy', 'sky blue sky beautiful today',\n",
       "       'dog lazy brown fox quick'], dtype='<U51')"
      ]
     },
     "execution_count": 2,
     "metadata": {},
     "output_type": "execute_result"
    }
   ],
   "source": [
    "import nltk\n",
    "import re\n",
    "\n",
    "stop_words = nltk.corpus.stopwords.words('english')\n",
    "\n",
    "def normalize_document(doc):\n",
    "    # lower case and remove special characters\\whitespaces\n",
    "    doc = re.sub(r'[^a-zA-Z\\s]', '', doc, re.I|re.A)\n",
    "    doc = doc.lower()\n",
    "    doc = doc.strip()\n",
    "    # tokenize document\n",
    "    tokens = nltk.word_tokenize(doc)\n",
    "    # filter stopwords out of document\n",
    "    filtered_tokens = [token for token in tokens if token not in stop_words]\n",
    "    # re-create document from filtered tokens\n",
    "    doc = ' '.join(filtered_tokens)\n",
    "    return doc\n",
    "\n",
    "normalize_corpus = np.vectorize(normalize_document)\n",
    "\n",
    "norm_corpus = normalize_corpus(corpus)\n",
    "norm_corpus"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {
    "colab_type": "text",
    "id": "2JCyCJ6GRiql"
   },
   "source": [
    "# The Word2Vec Model\n",
    "\n",
    "This model was created by Google in 2013 and is a predictive deep learning based model to compute and generate high quality, distributed and continuous dense vector representations of words, which capture contextual and semantic similarity. Essentially these are unsupervised models which can take in massive textual corpora, create a vocabulary of possible words and generate dense word embeddings for each word in the vector space representing that vocabulary. \n",
    "\n",
    "Usually you can specify the size of the word embedding vectors and the total number of vectors are essentially the size of the vocabulary. This makes the dimensionality of this dense vector space much lower than the high-dimensional sparse vector space built using traditional Bag of Words models.\n",
    "\n",
    "There are two different model architectures which can be leveraged by Word2Vec to create these word embedding representations. These include,\n",
    "\n",
    "- The Continuous Bag of Words (CBOW) Model\n",
    "- The Skip-gram Model\n",
    "\n",
    "## The Continuous Bag of Words (CBOW) Model\n",
    "\n",
    "The CBOW model architecture tries to predict the current target word (the center word) based on the source context words (surrounding words). \n",
    "\n",
    "Considering a simple sentence, ___“the quick brown fox jumps over the lazy dog”___, this can be pairs of __(context_window, target_word)__ where if we consider a context window of size 2, we have examples like __([quick, fox], brown)__, __([the, brown], quick)__, __([the, dog], lazy)__ and so on. \n",
    "\n",
    "Thus the model tries to predict the __`target_word`__ based on the __`context_window`__ words.\n",
    "\n",
    "![](cbow_arch.png)\n",
    "\n",
    "\n",
    "## The Skip-gram Model\n",
    "\n",
    "The Skip-gram model architecture usually tries to achieve the reverse of what the CBOW model does. It tries to predict the source context words (surrounding words) given a target word (the center word). \n",
    "\n",
    "Considering our simple sentence from earlier, ___“the quick brown fox jumps over the lazy dog”___. If we used the CBOW model, we get pairs of __(context_window, target_word)__ where if we consider a context window of size 2, we have examples like __([quick, fox], brown)__, __([the, brown], quick)__, __([the, dog], lazy)__ and so on. \n",
    "\n",
    "Now considering that the skip-gram model’s aim is to predict the context from the target word, the model typically inverts the contexts and targets, and tries to predict each context word from its target word. Hence the task becomes to predict the context __[quick, fox]__ given target word __‘brown’__ or __[the, brown]__ given target word __‘quick’__ and so on. \n",
    "\n",
    "Thus the model tries to predict the context_window words based on the target_word.\n",
    "\n",
    "![](skipgram_arch.png)"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {
    "colab_type": "text",
    "id": "BHD8pUx7Riqm"
   },
   "source": [
    "# Robust Word2Vec Model with Gensim\n",
    "\n",
    "The __`gensim`__ framework, created by Radim Řehůřek consists of a robust, efficient and scalable implementation of the Word2Vec model. We will leverage the same on our sample toy corpus. In our workflow, we will tokenize our normalized corpus and then focus on the following four parameters in the Word2Vec model to build it.\n",
    "\n",
    "- __`size`:__ The word embedding dimensionality\n",
    "- __`window`:__ The context window size\n",
    "- __`min_count`:__ The minimum word count\n",
    "- __`sample`:__ The downsample setting for frequent words\n",
    "- __`sg`:__ Training model, 1 for skip-gram otherwise CBOW\n",
    "\n",
    "We will build a simple Word2Vec model on the corpus and visualize the embeddings."
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 3,
   "metadata": {
    "colab": {
     "base_uri": "https://localhost:8080/",
     "height": 34
    },
    "colab_type": "code",
    "id": "-tKqisDuRiqm",
    "outputId": "f7106f35-82d8-4e0d-8940-02d524b4879a"
   },
   "outputs": [
    {
     "data": {
      "text/plain": [
       "<gensim.models.word2vec.Word2Vec at 0x109c0deb8>"
      ]
     },
     "execution_count": 3,
     "metadata": {},
     "output_type": "execute_result"
    }
   ],
   "source": [
    "import nltk\n",
    "from gensim.models import word2vec\n",
    "\n",
    "tokenized_corpus = [nltk.word_tokenize(doc) for doc in norm_corpus]\n",
    "\n",
    "# Set values for various parameters\n",
    "feature_size = 15    # Word vector dimensionality  \n",
    "window_context = 20  # Context window size                                                                                    \n",
    "min_word_count = 1   # Minimum word count                        \n",
    "sample = 1e-3        # Downsample setting for frequent words\n",
    "sg = 1               # skip-gram model\n",
    "\n",
    "w2v_model = word2vec.Word2Vec(tokenized_corpus, size=feature_size, \n",
    "                              window=window_context, min_count = min_word_count,\n",
    "                              sg=sg, sample=sample, iter=5000)\n",
    "w2v_model"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 9,
   "metadata": {},
   "outputs": [
    {
     "name": "stderr",
     "output_type": "stream",
     "text": [
      "/anaconda3/lib/python3.6/site-packages/ipykernel_launcher.py:1: DeprecationWarning: Call to deprecated `__getitem__` (Method will be removed in 4.0.0, use self.wv.__getitem__() instead).\n",
      "  \"\"\"Entry point for launching an IPython kernel.\n"
     ]
    },
    {
     "data": {
      "text/plain": [
       "array([ 0.02641913, -0.8048629 , -0.35478404, -0.22569372,  0.76352954,\n",
       "        0.1202753 ,  0.8450837 ,  0.09746587, -0.14705269, -0.47812614,\n",
       "        0.52777976, -0.73928857, -0.4105191 , -0.07248587,  0.41778702],\n",
       "      dtype=float32)"
      ]
     },
     "execution_count": 9,
     "metadata": {},
     "output_type": "execute_result"
    }
   ],
   "source": [
    "w2v_model['blue']"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "# T-SNE\n",
    "https://www.youtube.com/watch?v=NEaUSP4YerM"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 4,
   "metadata": {
    "colab": {
     "base_uri": "https://localhost:8080/",
     "height": 378
    },
    "colab_type": "code",
    "id": "6NY4l-zaRiqo",
    "outputId": "bd4ef9b9-421e-494a-9a0f-8e3985fba0a4"
   },
   "outputs": [
    {
     "data": {
      "image/png": "iVBORw0KGgoAAAANSUhEUgAAAtEAAAFpCAYAAABauHSCAAAABHNCSVQICAgIfAhkiAAAAAlwSFlz\nAAALEgAACxIB0t1+/AAAADl0RVh0U29mdHdhcmUAbWF0cGxvdGxpYiB2ZXJzaW9uIDMuMC4zLCBo\ndHRwOi8vbWF0cGxvdGxpYi5vcmcvnQurowAAIABJREFUeJzs3Xucz3X+///byyGSU0U60aivHOeA\nGdkcUiraRGIpbGHxK5FtN9tJrZS2Wvupj1JKIhubXUqlTZtiUaoZzDiEj7RDyRYbMg1leP3+eI/3\nDhGvDDOT2/Vymcu8X8f34/V+d+lyn6fn6/EKwjBEkiRJ0uErVdQFSJIkSSWNIVqSJEmKyBAtSZIk\nRWSIliRJkiIyREuSJEkRGaIlSZKkiAzRkiRJUkSGaEmSJCkiQ7QkSZIUkSFakiRJiqhMYZwkCIKq\nwLNAIyAE+gKrgalAApANdAvDcEsQBAHwv8DPgVygdxiGi3/o/NWqVQsTEhIKo1RJkiTpoBYtWrQ5\nDMPqh9qvUEI0sVA8KwzDrkEQnABUAO4C3g7D8KEgCO4A7gBuB64A6uT/XAA8lf/7oBISEsjIyCik\nUiVJkqQDC4Jg3eHsd8TTOYIgqAK0BsYDhGH4XRiGW4FOwPP5uz0PXJ3/uhMwKYx5H6gaBMEZR1qH\nJEmSdKwUxpzo2sAmYEIQBEuCIHg2CIKTgBphGG7M3+ffQI3812cBnxY4/rP8dZIkSVKJUBghugzQ\nBHgqDMPGwDfEpm7EhWEYEpsrfdiCIBgQBEFGEAQZmzZtKoQyJUmSpMJRGCH6M+CzMAw/yF+eRixU\nf7F3mkb+7y/zt28AahY4/uz8dfsIw/CZMAxTwzBMrV79kHO7JUmSpGPmiEN0GIb/Bj4NgqBu/qq2\nwEfAq8AN+etuAF7Jf/0qcH0Q0xzYVmDah46RCy+8sKhLkCRJKrEKqzvHYGByfmeOT4A+xAL6X4Mg\n+BWwDuiWv+/fibW3+5hYi7s+hVSDInjvvfeKugRJkqQSq1AethKGYWb+1IukMAyvDsNwSxiG/wnD\nsG0YhnXCMLw0DMOv8vcNwzC8OQzD88IwTAzDsET0rqtYsWJRl1CoKlasyNy5c+nQoUN83aBBg5g4\ncSIQayt45513kpKSQmpqKosXL6Zdu3acd955jB07FoC5c+fSunVrrrzySurWrcuNN97Inj172L17\nN71796ZRo0YkJiby6KOPFsUlSpIkHTWFNRKtn6BatWqRmZnJrbfeSu/evXn33XfZuXMnjRo14sYb\nbwTgww8/5KOPPuKcc86hffv2vPTSS9SuXZsNGzawfPlyALZu3VqUlyFJklTofOx3RDk5ObRt25Ym\nTZqQmJjIK6/EpnqPHTuWlJQUUlJSqF27NhdffDHPPfccv/71r+PHjhs3jltvvbWoSo+sY8eOACQm\nJnLBBRdQqVIlqlevTrly5eLBuFmzZpx77rmULl2a6667jgULFnDuuefyySefMHjwYGbNmkXlypWL\n8jIkSZIKnSE6ovLly/Pyyy+zePFi5syZw29/+1vCMOTGG28kMzOT9PR0zj77bH7zm9/QrVs3Xnvt\nNXbt2gXAhAkT6Nu377EvespkqJcApUvFfk+ZDECZMmXYs2dPfLedO3fuc1i5cuUAKFWqVPz13uW8\nvDwAYk9x/68gCDj55JPJysqiTZs2jB07ln79+h2Fi5IkSSo6huiIwjDkrrvuIikpiUsvvZQNGzbw\nxRdfxLcPGTKESy65hKuuuoqKFStyySWXMHPmTFatWsWuXbtITEw8tgVPmQy3DYAu62BCGPt92wDI\ny+Occ87ho48+4ttvv2Xr1q28/fbbkU//4Ycf8q9//Ys9e/YwdepUWrZsyebNm9mzZw9dunThgQce\nYPHixUfhwiRJkoqOc6IPZspkGHE3rFkPdWpB/sjr5MmT2bRpE4sWLaJs2bIkJCTER3AnTpzIunXr\neOKJJ+Kn6devHw8++CD16tWjT58iaEQy4m7okwsN85cbAn1yCf4QULNmTbp160ajRo2oXbs2jRs3\njnz6tLQ0Bg0axMcff8zFF19M586dWbZsGX369ImPcv/hD38ovOuRJEkqBoLYwwSLt9TU1DAj4xg2\n8dg7etsnF+oCq6HiHyDnhRf4302b+fjjj3n88ceZM2cOl1xyCf/617/4z3/+ww033MD8+fM5+eST\n9zldkyZN2LRpE0uXLv3etqOudKnYCHSBP5f+sxWa3AzrjvC7nzt3LqNGjWLmzJlHWKQkSVLxEATB\nojAMUw+1nyPRB3Kg0dsysfU9F2Rw1VVXkZiYSGpqKvXq1QPgiSee4KuvvuLiiy8GIDU1lWeffRaA\nbt26kZmZeewDNMRG0Vevi1/L51ugzb1w22lFUIskSdJPhCPRB3KA0VvygD4B7N5zsKMOqkOHDtx6\n6620bdu20Eo8bAcYVWdCBRj1DPToeezrkSRJKsYOdyTaGwsPpE6tWNgsaHX++gi2bt3K+eefz4kn\nnlg0ARpiQXnUMzD9nNgfAdPPMUBLkiQdIUeiD8TRW0mSpOOSc6KPxN6gXLA7x6iRJSJADx8+nIoV\nK3LbbbcVdSmSJEk/WU7nOJgePWFVdmwO9KrsEhGgi7PRo0dTv359evb0c5QkSSWfIfonYOTIkZx/\n/vm0bNmS1atjk7kzMzNp3rw5SUlJdO7cmS1btgCQnp5OUlISKSkpDB06lEaNGh2TGp988kneeust\nJk+efEzeT5Ik6WgyRJdwixYt4sUXXyQzM5O///3vpKenA3D99dfz8MMPs3TpUhITE7nvvvsA6NOn\nD08//TSZmZmULl36mNR444038sknn3DFFVfwpz/9iauvvpqkpCSaN2/O0qVLgdiTHkeMGAHAm2++\nSevWrfd5JLkkSVJxYogu4ebPn0/nzp2pUKEClStXpmPHjnzzzTds3bqViy66CIAbbriBefPmsXXr\nVrZv387PfvYzAHr06HFMahw7dixnnnkmc+bMITs7m8aNG7N06VIefPBBrr/+eiD2VMOpU6cyZ84c\nbrnlFiZMmECpUv7nKUmSiidTylGSkZHBLbfc8oP7VKxYMfqJp0yGegmxXtb1EmDRoh9VX1FZsGAB\nv/zlLwG45JJL+M9//sPXX39NhQoVGDduHJdddhmDBg3ivPPOK+JKJUmSDs4QfZSkpqYyevTowj3p\n3tZ7XdbFHgbTZR2tZ01jxsSJ7Nixg+3bt/Paa69x0kkncfLJJzN//nwA/vznP3PRRRdRtWpVKlWq\nxAcffADAiy++WLj17V9rwbD/zTeHPGTZsmWceuqpfP7550evLkmSpEJgiI6g4A181113HaNGjaJN\nmzbs7WG9efNmEhISAJg7dy4dOnQAICcnhz59+pCYmEhSUhLTp0/f57ybN2/mZz/7Ga+//voPF1Dw\nceRlgIbQZMBOuu/YTnJyMldccQVpaWkAPP/88wwdOpSkpCQyMzO59957ARg/fjz9+/cnJSWFb775\nhipVqhTa5xN3gLDPlv/A9Gm0atUqfnPh3LlzqVatGpUrV2bdunX86U9/YsmSJbzxxhvxoC9JklQc\n2Sf6MBW8gS8vL48mTZrQtGnTwzr2/vvvp0qVKixbtgwg3ikD4IsvvqBjx4488MADXHbZZT98ojXr\nYw9/Kagu3P3VNu7etOV7u7///vvfW9ewYcP4zXwPPfQQqamH7CUeXcGwD7HfFUP440iGf5hF3759\nSUpKokKFCjz//POEYcivfvUrRo0axZlnnsn48ePp3bs36enplC9fvvDrkyRJOkKG6MNU8AY+gI4d\nOx72sbNnz95n6sTJJ58MwK5du2jbti1jxoyJ3wT4g+rUgtXr/htOIfLjyF9//XX+8Ic/kJeXxznn\nnMPEiRMP+9jDdoCwnz0a6LMBTjmFGTNmfO+Q2bNnx183bdo0/geHJElSceR0jh9ScF7vgyNg+feD\nXZkyZeKt2Hbu3Bnp9GXKlKFp06a8+eabh3fAvSNjjx9fAeQR+z2hQmz9YerevTuZmZksX76c119/\nnerVq0eq+bDUqRUL9wVFDPuSJEnFmSH6YPab19v6qi3MeO0VdkycEL+BDyAhIYFF+R0ypk2bdsBT\nXXbZZYwZMya+vHc6RxAEPPfcc6xatYqHH3740DX16AmjnoHp50CfIPZ71DPF72mKhRD2JUmSijND\n9MHsdxNfk7bQvUVI8oAB+9zAd9ttt/HUU0/RuHFjNm/efMBTDRs2jC1bttCoUSOSk5OZM2dOfFvp\n0qX5y1/+wjvvvMOTTz556LpKwuPIS0rYlyRJ+pGCMAyLuoZDSk1NDfd2wDhmSpeKdZYoOGs8j1go\n3L2H4cOHU7FiRW677bZjW9dhyM7OpkOHDixfvryoS5EkSSpRgiBYFIbhITsvOBJ9MD/xeb27d+8u\n6hIkSZJKLEP0wRxiXu/w4cOL5Sj0Xnl5efTs2ZP69evTtWtXcnNzSUhI4Pbbb6dJkyb87W9/IzMz\nk+bNm5OUlETnzp3ZsmULX375Zbx1X1ZWFkEQsH79egDOO+88cnNz6d27N7fccgsXXngh55577kHn\ngkuSJP1UGaIPpoTP6129ejUDBw5k5cqVVK5cOT7f+tRTT2Xx4sVce+21XH/99Tz88MMsXbqUxMRE\n7rvvPk477TR27tzJ119/zfz580lNTWX+/PmsW7eO0047Ld7ib+PGjSxYsICZM2dyxx13FOWlSpIk\nHXP2if4hPXqWmNC8v5o1a9KiRQsAevXqFX8Eeffu3QHYtm0bW7dujfenvuGGG/jFL34BwIUXXsi7\n777LvHnzuOuuu5g1axZhGNKqVav4+a+++mpKlSpFgwYN+OKLL47lpUmSJBU5R6J/Cgr2s66XAK/M\nIAiCfXbZu3zSSScd8nStW7eOjz536tSJrKwsFixYsE+ILleuXPx1Sbg5VZIkqTAZoku6/fpZ02Ud\njLyT9evXs3DhwtguU6bQsmXLfQ6rUqUKJ598MvPnzwfgz3/+c3xUulWrVrzwwgvUqVOHUqVKccop\np/D3v//9e+eQJEk6XhmiS7r9+lnTEPjFTuqeUIYxY8ZQv359tmzZwk033fS9Q59//nmGDh1KUlIS\nmZmZ3HvvvUDsATJhGNK6dWsAWrZsSdWqVeOPK5ckSTre2Se6pDtEP2tJkiQdPvtEHy9+4v2sJUmS\niiNDdEl3iH7WkiRJKny2uCvp9rbgG3E3rFkfG4EeNbLEtuaTJEkqCQzRPwUluJ+1JElSSeR0DkmS\nJCkiQ7QkSZIUkSFakiRJisgQLUmSJEVkiJYkSZIiMkRLkiRJERmiJUmSpIgM0ZIkSVJEhmhJkiQp\nIkO0JEmSFJEhWpIkSYrIEC1JkiRFZIiWJEmSIjJES5IkSREZoiVJkqSIDNGSJElSRIUWooMgKB0E\nwZIgCGbmL9cOguCDIAg+DoJgahAEJ+SvL5e//HH+9oTCqkGSJEk6FgpzJHoIsLLA8sPAo2EY/j9g\nC/Cr/PW/Arbkr380fz9JkiSpxCiUEB0EwdnAlcCz+csBcAkwLX+X54Gr8193yl8mf3vb/P0lSZKk\nEqGwRqIfA34H7MlfPhXYGoZhXv7yZ8BZ+a/PAj4FyN++LX9/SZIkqUQ44hAdBEEH4MswDBcVQj0F\nzzsgCIKMIAgyNm3aVJinliRJko5IYYxEtwA6BkGQDbxIbBrH/wJVgyAok7/P2cCG/NcbgJoA+dur\nAP/Z/6RhGD4ThmFqGIap1atXL4QyJUmSpMJxxCE6DMM7wzA8OwzDBOBa4J0wDHsCc4Cu+bvdALyS\n//rV/GXyt78ThmF4pHVIkiRJx8rR7BN9O/CbIAg+JjbneXz++vHAqfnrfwPccRRrkCRJkgpdmUPv\ncvjCMJwLzM1//QnQ7AD77AR+UZjvK0mSJB1LPrFQkiRJisgQLUmSJEVkiJYkSZIiMkRLkiRJERmi\nJUmSpIgM0ZIkSVJEhmhJkiQpIkO0JEmSFJEhWpIkSYrIEC1JkiRFZIiWJEmSIjJES5IkSREZoiVJ\nkqSIDNGSJElSRIZoSZIkKSJDtCRJkhSRIVqSJEmKyBAtSZIkRWSIliRJkiIyREuSJEkRGaIlSZKk\niAzRkiRJUkSGaEmSJCkiQ7QkSZIUkSFakiRJisgQLUmSJEVkiJYkSZIiMkRLkiRJERmiJUmSpIgM\n0ZIkSVJEhmhJkiQpIkO0JEmSFJEhWpIkSYrIEC1JkiRFZIiWJEmSIjJES5IkSREZoiVJkqSIDNGS\nJElSRIZoSZIkKSJDtCRJkhSRIVqSJEmKyBAtSZIkRWSIliRJkiIyREuSJEkRGaIlSZKkiAzRkiRJ\nUkSGaEmSJCkiQ7QkSZIUkSFakiRJisgQLUmSJEVkiJYkSZIiMkRLkiRJER1xiA6CoGYQBHOCIPgo\nCIIVQRAMyV9/ShAEbwVBsCb/98n564MgCEYHQfBxEARLgyBocqQ1SJIkScdSYYxE5wG/DcOwAdAc\nuDkIggbAHcDbYRjWAd7OXwa4AqiT/zMAeKoQapAkSZKOmSMO0WEYbgzDcHH+6+3ASuAsoBPwfP5u\nzwNX57/uBEwKY94HqgZBcMaR1iFJkiQdK4U6JzoIggSgMfABUCMMw435m/4N1Mh/fRbwaYHDPstf\nJ0mSJJUIhRaigyCoCEwHfh2G4dcFt4VhGAJhxPMNCIIgIwiCjE2bNhVWmZIkSdIRK5QQHQRBWWIB\nenIYhi/lr/5i7zSN/N9f5q/fANQscPjZ+ev2EYbhM2EYpoZhmFq9evXCKFOSJEkqFIXRnSMAxgMr\nwzD8nwKbXgVuyH99A/BKgfXX53fpaA5sKzDtQ5IkSSr2yhTCOVoAvwSWBUGQmb/uLuAh4K9BEPwK\nWAd0y9/2d+DnwMdALtCnEGqQJEmSjpkjDtFhGC4AgoNsbnuA/UPg5iN9X0mSJKmo+MRCSZIkKSJD\ntCRJkhSRIVqSJEmKyBAtSZIkRWSIliRJkiIyREuSJEkRGaIlSZKkiAzRkiRJUkSGaEmSJCkiQ7Qk\nSZIUkSFakiRJisgQLUmSJEVkiJYkSZIiMkRLkiRJERmiJUmSpIgM0ZIkSVJEhmhJkiQpIkO0JEmS\nFJEhWpIkSYrIEC1JkiRFZIiWJEmSIjJES5IkSREZoiVJkqSIDNGSJElSRIZoSZIkKSJDtCRJkhSR\nIVqSJEmKyBAtSZIkRWSIliRJkiIyREuSJEkRGaIlSZKkiAzRkiRJUkSGaEmSJCkiQ7QkSZIUkSFa\nkiRJisgQLUmSJEVkiJYkSZIiMkRLkiRJERmiJUmSpIgM0ZIkSVJEhmhJkiQpIkO0JEmSFJEhWpIk\nSYrIEC1JkiRFZIiWJEmSIjJES5IkSREZoiVJkqSIDNGSJElSRIZoSZIkKSJDtCRJkhSRIVqSJEmK\nyBAtSZIkRVRkIToIgvZBEKwOguDjIAjuKKo6JEmSpKiKJEQHQVAaGANcATQArguCoEFR1CJJkiRF\nVVQj0c2Aj8Mw/CQMw++AF4FORVSLJEmSFElRheizgE8LLH+Wv06SJEkq9ortjYVBEAwIgiAjCIKM\nTZs2FXU5kiRJUlxRhegNQM0Cy2fnr4sLw/CZMAxTwzBMrV69+jEtTpIkSfohRRWi04E6QRDUDoLg\nBOBa4NUiqkWSJEmKpExRvGkYhnlBEAwC3gRKA8+FYbiiKGqRJEmSoiqSEA0QhuHfgb8X1ftLkiRJ\nP1axvbFQkiRJKq4M0ZIkSVJEhmhJkiQpIkO0JEmSFJEhWpIkSYrIEC1JkiRFZIiWJEmSIjJES5Ik\nSREZoiVJkqSIDNGSJElSRIZoSZIkKSJDtCRJkhSRIVqSJEmKyBAtSZIkRWSIliRJkiIyREuSJEkR\nGaIlSZKkiAzRkiRJUkSGaEmSJCkiQ7QkSZIUkSFakiRJisgQLUmSJEVkiJYkSZIiMkRLkiRJERmi\nJUmSpIgM0ZIkSVJEhmhJkiQpIkO0JEmSFJEhWpIkSYrIEC1JkiRFZIiWJEmSIjJES5IkSREZoiX9\nJGRnZ9OoUaOiLkOSdJwwREuSJEkRGaIl/WTk5eXRs2dP6tevT9euXcnNzWXRokVcdNFFNG3alHbt\n2rFx40YAxo0bR1paGsnJyXTp0oXc3FwAevfuzS233MKFF17Iueeey7Rp0wDYuHEjrVu3JiUlhUaN\nGjF//vwiu05JUtEzREv6yVi9ejUDBw5k5cqVVK5cmTFjxjB48GCmTZvGokWL6Nu3L3fffTcA11xz\nDenp6WRlZVG/fn3Gjx8fP8/GjRtZsGABM2fO5I477gBgypQptGvXjszMTLKyskhJSSmSa5QkFQ9l\niroASSosNWvWpEWLFgD06tWLBx98kOXLl3PZZZcBsHv3bs444wwAli9fzrBhw9i6dSs5OTm0a9cu\nfp6rr76aUqVK0aBBA7744gsA0tLS6Nu3L7t27eLqq682REvScc6RaEkl05TJUC8BSpeK/X5lBkEQ\n7LNLpUqVaNiwIZmZmWRmZrJs2TL+8Y9/ALFpG0888QTLli3j97//PTt37owfV65cufjrMAwBaN26\nNfPmzeOss86id+/eTJo06ahfoiSp+DJESyp5pkyG2wZAl3UwIYz9Hnkn69evZ+HChbFdpkyhefPm\nbNq0Kb5u165drFixAoDt27dzxhlnsGvXLiZPnnzIt1y3bh01atSgf//+9OvXj8WLFx+965MkFXuG\naEklz4i7oU8uNCQ2Ka0h8Iud1D2hDGPGjKF+/fps2bIlPh/69ttvJzk5mZSUFN577z0A7r//fi64\n4AJatGhBvXr1DvmWc+fOJTk5mcaNGzN16lSGDBlyVC9RklS8BXv/qbI4S01NDTMyMoq6DEkHsXXr\nVqZMmcLAgQML7ZyPPfYYAwYMoEKFCt/fWLpUbAS64F0deUCfAHbvKbQaJEnHnyAIFoVhmHqo/RyJ\nlnTEtm7dypNPPlmo53zsscfibee+p04tWL3futX56yVJOgbsziHpiN1xxx2sXbuWlJSUeCeMN954\ngyAIGDZsGN27dycnJ4dOnTqxZcsWdu3axQMPPECnTp345ptv6NatG5999hm7d+/mnnvu4YsvvuDz\nzz/n4osvplq1asyZM2ffN7x3ZGxOdJ9cqEssQE+oAKNGHvNrlyQdn5zOIemIZWdn06FDB5YvX870\n6dMZO3Yss2bNYvPmzaSlpfHBBx9QvXp1cnNzqVy5Mps3b6Z58+asWbOGl156iVmzZjFu3DgAtm3b\nRpUqVUhISCAjI4Nq1aod+E2nTI7NjV6zPjYCfe9I6NHziGrfKyMjg0mTJjF69Ogf9XlIkkoup3NI\nKhILFizguuuuo3Tp0tSoUYOLLrqI9PR0wjDkrrvuIikpiUsvvZQNGzbwxRdfkJiYyFtvvcXtt9/O\n/PnzqVKlyuG9UY+esCo7Ngd6VfaPCtAHk5qaaoCWJP0gQ7SkH6dgn+a2LWHbth/cffLkyWzatIlF\nixaRmZlJjRo12LlzJ+effz6LFy8mMTGRYcOGMWLEiGNT/wF88sknNG7cmD/+8Y906NABgOHDh9O3\nb1/atGnDueeeu0+4vv/++6lbty4tW7bkuuuuY9SoUQCMHj2aBg0akJSUxLXXXlsk1yJJOroM0ZKi\n269Pc6WOG9i+cQNMmUyrVq2YOnUqu3fvZtOmTcybN49mzZqxbds2TjvtNMqWLcucOXNYt24dAJ9/\n/jkVKlSgV69eDB06NN5/uVKlSmzfvv2YXdLq1avp0qULEydOJC0tbZ9tq1at4s033+TDDz/kvvvu\nY9euXaSnpzN9+nSysrJ44403KDjl7KGHHmLJkiUsXbqUsWPHHrNrkCQdO4ZoSdHt16f51DRoUTek\nUZ8+LFy4kKSkJJKTk7nkkkt45JFHOP300+nZsycZGRkkJiYyadKkeG/mZcuW0axZM1JSUrjvvvsY\nNmwYAAMGDKB9+/ZcfPHFR/1yNm3aRKdOnZg8eTLJycnf237llVdSrlw5qlWrxmmnncYXX3zBu+++\nS6dOnShfvjyVKlXiqquuiu+flJREz549eeGFFyhTxvu3C8rOzqZRo0ZFXUYkJbFmSUef/3eXFN2a\n9bGuGAVMuR3okwd//CMAf8z/vVe1atXiTw4sKCEhgXbt2n1v/eDBgxk8eHChlfzfQve7IfGmX1Ol\nShVq1arFggULaNCgwfcOKfgY8NKlS5OXl/eDb/H6668zb948XnvtNUaOHMmyZcsM05L0E+NItKTo\nSmqf5oM8LvyEHTt4+eWXmTRpElOmTDmsU7Vo0YLXXnuNnTt3kpOTw8yZMwHYs2cPn376KRdffDEP\nP/ww27ZtIycn52heVYmze/du+vfvT8OGDbn88svZsWMH48aNIy0tjeTkZLp06RLvEd67d29uuukm\nmjdvzrnnnsvcuXPp27cv9evXp3fv3ses5ry8PHr27En9+vXp2rUrubm5jBgxgrS0NBo1asSAAQPY\n2+3q448/5tJLLyU5OZkmTZqwdu1awjBk6NChNGrUiMTERKZOnQrEnoTZpk0bunbtSr169ejZsycl\noWuWJEO0pB/j3pGxvswriD0pcAWx5XuLeZ/mgzwunC+/4KSTTmLmzJk8+uijfP3114c8VVpaGh07\ndiQpKYkrrriCxMREqlSpwu7du+nVqxeJiYk0btyYW265hapVqx7tKytR1qxZw80338yKFSuoWrUq\n06dP55prriE9PZ2srCzq16/P+PHj4/tv2bKFhQsX8uijj9KxY0duvfVWVqxYwbJly8jMzDwmNa9e\nvZqBAweycuVKKleuzJNPPsmgQYNIT09n+fLl7NixI/6HVM+ePbn55pvJysrivffe44wzzuCll14i\nMzOTrKwsZs+ezdChQ9m4cSMAS5Ys4bHHHuOjjz7ik08+4d133z0m1yTpyPjvi5Ki29tOruC0iFE/\nrk/zMXWAaSgJF8DysbHpGVWrViU9PR2Ajh07ArHuHAUV7Cd92223MXz4cHJzc2ndujVNmzalbNmy\nLFiw4Ohdw09A7dq1SUlJAaBp06ZkZ2ezfPlyhg0bxtatW8nJydlnis9VV11FEAQkJiZSo0YNEhMT\nAWjYsCHZ2dnxcx1NNWvWpEWLFgD06tWL0aNHU7t2bR555BFyc3P56quvaNiwIW3atGHDhg107twZ\ngPLlywMHb/1YuXJlmjVrxtmpOzjoAAAgAElEQVRnnw1ASkoK2dnZtGzZ8qhfk6Qjc0Qj0UEQ/DEI\nglVBECwNguDlIAiqFth2ZxAEHwdBsDoIgnYF1rfPX/dxEAR3HMn7SypCR7FP81FTyNNQBgwYQEpK\nCk2aNKFLly40adLkiEv8ySnYCrFeArwy44BzzHv37s0TTzzBsmXL+P3vf8/OnTvj++zdv1SpUvsc\nW6pUqUPOTy+smoMg2GeXIAgYOHAg06ZNY9myZfTv33+fmqOIOudeUvFwpNM53gIahWGYBPwfcCdA\nEAQNgGuJ/WNpe+DJIAhKB0FQGhgDXAE0AK7L31fST8TR6mQwfPjweB/mglatWkVKSgqNGzdm7dq1\nP3yS/aahTPwLfP5s+R89DWXKlClkZmayatUq7rzzzh91jp+0g8xBP1BP8e3bt3PGGWewa9cuJk+e\nXATF5jtIzevXr4/fGDtlypT4SHG1atXIyclh2rRpQKw149lnn82MGTMA+Pbbb8nNzT1o60dJJdcR\nhegwDP8RhuHeP5nfB87Of90JeDEMw2/DMPwX8DHQLP/n4zAMPwnD8Dvgxfx9JR1Hdu/eXWjnmjFj\nBl27dmXJkiWcd955P7xzj54w6hmYfg70CZj4djk+H3xXyRhFL4l+YA76/u6//34uuOACWrRoEW9/\nWCQOUnPdE8owZswY6tevz5YtW7jpppvo378/jRo1ol27dvv0Fv/zn//M6NGjSUpK4sILL+Tf//43\nnTt3PmDrR0klV1BYdwEHQfAaMDUMwxeCIHgCeD8Mwxfyt40H3sjftX0Yhv3y1/8SuCAMw0EHON8A\nYABArVq1mu59MIOk4i07O5v27dvTtGlTFi9eTMOGDZk0aRINGjSge/fuvPXWW/zud78jLS2Nm2++\nmU2bNlGhQgXGjRtHvXr1eO2113jggQf47rvvOPXUU5k8eTI1atRg+PDhVKxYkdtuu41x48bx0ksv\nMWjQIH71q19RunRpzj//fObMmcPVV1/Np59+ys6dOxkyZAgDBgxg9+7d/OpXvyIjI4MgCOjbty81\na9akd+/enHXWWZx44oksXLiQE088sag/vp+W0qVio7kF777JA/oEsWlAxVFJrFlSoQqCYFEYhqmH\n2u+QNxYGQTAbONCfy3eHYfhK/j53E/vfTKH9G1wYhs8AzwCkpqba70cqQVavXs348eNp0aIFffv2\n5cknnwTg1FNPjT+RsG3btowdO5Y6derwwQcfMHDgQN555x1atmzJ+++/TxAEPPvsszzyyCP86U9/\nip/7iSee4K233mLGjNjc2htvvDEergGee+45TjnlFHbs2EFaWhpdunQhOzubDRs2xG8K3Lp1K1Wr\nVuWJJ55g1KhRpKYe8v+V+jHq1ILV62KjuXsV91aIJbFmSUXikCE6DMNLf2h7EAS9gQ5A2/C/w9ob\ngJoFdjs7fx0/sF7ST8SBOhkAdO/eHYCcnBzee+89fvGLX8SP+fbbbwH47LPP6N69Oxs3buS7776j\ndu3a8X0mTZpEzZo1mTFjBmXLlj3ge48ePZqXX34ZgE8//ZQ1a9ZQt25dPvnkEwYPHsyVV17J5Zdf\nXvgXre+7d2RsfnGf3FhXlNXE5qSPKsatEEtizZKKxJF252gP/A7oGIZhboFNrwLXBkFQLgiC2kAd\n4EMgHagTBEHtIAhOIHbz4atHUoOkInaYnQwATjrpJCD2QJKqVauSmZkZ/1m5ciUQe1LhoEGDWLZs\nGU8//fQ+HQ8SExPJzs7ms88+O2Apc+fOZfbs2SxcuJCsrCwaN27Mzp07Ofnkk8nKyqJNmzaMHTuW\nfv36Ff7noO/bbw4608+JLRfnOeglsWZJReJIu3M8AVQC3gqCIDMIgrEAYRiuAP4KfATMAm4Ow3B3\n/k2Ig4A3gZXAX/P3lVQSRexksFflypWpXbs2f/vb3wAIw5CsrCwAtm3bxllnnQXA888/v89xjRs3\n5umnn6Zjx458/vnn3ytn27ZtnHzyyVSoUIFVq1bx/vvvA7B582b27NlDly5deOCBB+JTSipVqsT2\n7dsL8QPR95TEVoglsWZJx9yRduf4f2EY1gzDMCX/58YC20aGYXheGIZ1wzB8o8D6v4dheH7+Nv99\nTCrJInQy2N/kyZMZP348ycnJNGzYkFdeeQWItbL7xS9+QdOmTalWrdr3jmvZsiWjRo3iyiuvZPPm\nzftsa9++PXl5edSvX5877riD5s2bA7BhwwbatGlDSkoKvXr14g9/+AMQe6T0jTfeSEpKCjt27CjU\nj0aS9NNWaN05jqbU1NQwIyOjqMuQtD87GRx3XnjhBUaPHs13333HBRdcwJNPPsnEiRN5+OGHqVq1\nKsnJyZQrV44nnniCtWvX0rNnT7755hs6derEY489Rk5ODhs3bqR79+58/fXX5OXl8dRTT9GqVavI\ntXzzzTd069aNzz77jN27d3PPPfewevVqXnvtNXbs2MGFF17I008/TRAEtGnTJn4T6ebNm0lNTSU7\nO5sVK1bQp08fvvvuO/bs2cP06dOpU6fOAbu8AIwfP/6A17pp0yZuvPFG1q9fD8Bjjz1GixYt+Oc/\n/8mQIUOA2LSmefPmUalSpcL7QiQVusPtznGk0zkkHc8K+QmAKt5WrlzJ1KlTeffdd8nMzKR06dJM\nnjyZ+++/n/fff593332XVatWxfcfMmQIQ4YMYdmyZfHHWkNsik+7du3IzMwkKyvrRz+2e9asWZx5\n5plkZWWxfPly2rdvz6BBg0hPT2f58uXs2LGDmTNn/uA5xo4dy5AhQ8jMzCQjIyNe53PPPceiRYvI\nyMhg9OjR/Oc//+Hzzz//wWu99dZbSU9PZ/r06fF596NGjWLMmDFkZmYyf/582yhKPyGGaEk/3n5P\nAGQFseUf+QRAFW9vv/02ixYtIi0tjZSUFN5++23+53/+h4suuohTTjmFsmXL7tNxZeHChfHlHj16\nxNenpaUxYcIEhg8fzrJly370yGxiYiJvvfUWt99+O/Pnz6dKlSrMmTOHCy64gMTERN555x1WrPjh\n225+9rOf8eCDD/Lwww+zbt26eMgdPXo0ycnJNG/ePN7l5cMPPzzotc6ePZtBgwaRkpJCx44d+frr\nr8nJyaFFixb85je/YfTo0WzdupUyZQ7ZFEtSCWGIlvTj2cngp22/zithejo33HBDvKPK6tWrGT58\neOTTtm7dmnnz5nHWWWfRu3dvJk2a9KNqOr/j5Sy+604SExMZNmwYI0aMYODAgUybNo1ly5bRv3//\neHeXMmXKsGdPbIpRwY4vPXr04NVXX+XEE0/k5z//Oe+8885Bu7z8kD179vD+++/HP5sNGzZQsWJF\n7rjjDp599ll27NhBixYt9hm9llSyGaIlHRk7Gfw0HaDzStu//5VpEybw5ZdfAvDVV1/RuHFj/vnP\nf7Jlyxby8vKYPn16/BTNmzePL7/44ovx9evWraNGjRr079+ffv36xbulRK3p88vXUeH3t9KrVMDQ\noUPj56lWrRo5OTlMmzYtfmhCQgKLFi0C2Gf9J598wrnnnsstt9xCp06dWLp06UG7vKSlpR30Wi+/\n/HIef/zx+HJmZiYAa9euJTExkdtvv520tDRDtPQT4r8rSZK+r2DnFYCG0GDATh6YUJHLL7+cPXv2\nULZsWcaMGcNdd91Fs2bNOOWUU6hXrx5VqlQBYjfX9erVi5EjR9K+ffv4+rlz5/LHP/6RsmXLUrFi\nxcMfid6vpmUVYOjuHZTq24eyick89dRTzJgxg0aNGnH66aeTlpYWP/S2226jW7duPPPMM1x55ZXx\n9X/961/585//TNmyZTn99NO56667OOmkkxg7diz169enbt268S4vZ5111kGvdfTo0dx8880kJSWR\nl5dH69atGTt2LI899hhz5syhVKlSNGzYkCuuuOIIvhRJxYndOSRJ3xeh80pOTg4VK1YkLy+Pzp07\n07dvXzp37kxubi4nnngiQRDw4osv8pe//CXeyvBo13S0HOxaJf10HG53DkeiJUnfV6cWrF7335Fo\nOGjnleHDhzN79mx27tzJ5ZdfztVXXw3AokWLGDRoEGEYUrVqVZ577rljVtPRcrBrlXT8cSRakvR9\ne+cf98mFusTC6oQKRXvjaHGsqRDl5eXZvUMqBhyJliT9eHtD6Yi7Yc362GjvqJFFG1aLY00R3H//\n/bzwwgtUr16dmjVr0rRpU2bOnElKSgoLFizguuuu4/rrrz/gQ1u++eYbBg8ezPLly9m1axfDhw+n\nU6dOTJw4kVdffZXc3FzWrl1L586deeSRR4r4SqXjgyFaknRgPXoWv4BaHGs6DHsfwpKVlcWuXbto\n0qQJTZs2BeC7775j77+29ujRg1tvvZWWLVuyfv162rVrx8qVKxk5ciSXXHIJzz33HFu3bqVZs2Zc\neumlQKwTyJIlSyhXrhx169Zl8ODB1KxZs8iuVTpeGKIlSTrK3n33XTp16kT58uUpX748V111VXxb\n9+7d469nz57NRx99FF/e+9CWf/zjH7z66quMGjUKiPW63jta3bZt23iXkAYNGrBu3TpDtHQMGKIl\nSSpCJ510Uvz13oe2lC9ffp99wjBk+vTp1K1bd5/1H3zwAeXKlYsvly5dmry8vKNbsCTAh61IknR0\nFHi6YotHH+a1Sc+zc+dOcnJymDlz5gEPOdhDW9q1a8fjjz/O3mYAS5YsOerlS/phhmhJkgrbfk9X\nTOv1bzpu3kBS7dpcccUVJCYmxqdgFDR69GgyMjJISkqiQYMGjB07FoB77rmHXbt2kZSURMOGDbnn\nnnuO9RVJ2o8t7iRJKmz1EmIBukBP65wlUPG1c8hd/BGtW7fmmWeeoUmTJkVWIkDFihXJyckp0hqk\n4sYWd5IkFZU162O9rAsYsAA+Wr2OnU2acMMNNxR5gJZ0ZJzOIUlSYatTK/YwmAKmXAKZdc9h1apV\n3HnnnUVT10GEYcjQoUNp1KgRiYmJTJ06FYBrr72W119/Pb5f7969mTZtGrt372bo0KGkpaWRlJTE\n008/XVSlS0XGEC1JUmG7d2TsaYorgDxivydUiK0vhl566SUyMzPJyspi9uzZDB06lI0bN9K9e3f+\n+te/ArF+1m+//TZXXnkl48ePp0qVKqSnp5Oens64ceP417/+VcRXIR1bTueQJKmwlbCnK+59YmLp\n0qWpUaMGF110Eenp6VxxxRUMGTKEb7/9llmzZtG6dWtOPPFE/vGPf7B06VKmTZsGwLZt21izZg21\na9cu4iuRjh1DtCRJR0NxfLrilMn7BvtD9JQuX748bdq04c0332Tq1Klce+21QGz6x+OPP067du2O\nRdVSseR0DkmSjgf7td2jyzr47luYMplWrVoxdepUdu/ezaZNm5g3bx7NmjUDYk9UnDBhAvPnz6d9\n+/ZArG/1U089xa5duwD4v//7P7755psiuzSpKDgSLUnS8WDE3dAn979t9xoSSwEj7qbzyn+xcOFC\nkpOTCYKARx55hNNPPx2IPQDml7/8JZ06deKEE04AoF+/fmRnZ9OkSRPCMKR69erMmDGjSC5LKir2\niZYk6XhQulRsBLrg8Fke0CeA3XuKqiqp2DncPtFO55Ak6XhwgLZ7rM5fLykyQ7QkSceDEtZ2Tyru\nnBMtSdLxoIS13ZOKO0O0JEnHi+LYdk8qoZzOIUmSJEVkiJYkSZIiMkRLkiRJERmiJUmSpIgM0ZIk\nSVJEhmhJkiQpIkO0JEmSFJEhWpIkSWRnZ9OoUaOjev4pU6bElzMyMrjlllsA+Pbbb7n00ktJSUlh\n6tSpBz3HxIkTGTRo0FGrMQoftiJJkqSjbm+I7tGjBwCpqamkpqYCsGTJEgAyMzOLrL6oHImWJEkS\nAHl5efTs2ZP69evTtWtXcnNzWbRoERdddBFNmzalXbt2bNy4EYBx48aRlpZGcnIyXbp0ITc3F4De\nvXszbdq0+DkrVqwIwB133MH8+fNJSUnh0UcfZe7cuXTo0IEvv/ySXr16kZ6eTkpKCmvXriUhIYHN\nmzcDsRHrNm3aHNsP4jAYoiVJkgTA6tWrGThwICtXrqRy5cqMGTOGwYMHM23aNBYtWkTfvn25++67\nAbjmmmtIT08nKyuL+vXrM378+B8890MPPUSrVq3IzMzk1ltvja8/7bTTePbZZ+PbzjvvvKN6jYXF\n6RySJEkCoGbNmrRo0QKAXr168eCDD7J8+XIuu+wyAHbv3s0ZZ5wBwPLlyxk2bBhbt24lJyeHdu3a\nFVndRcEQLUmSdDyaMhlG3A1r1kOdWnDTrwmCYJ9dKlWqRMOGDVm4cOH3Du/duzczZswgOTmZiRMn\nMnfuXADKlCnDnj17ANizZw/fffdd5NIKnmPnzp2Rjz8WnM4hSZJ0vJkyGW4bAF3WwYQw9nvknaxf\nvz4emKdMmULz5s3ZtGlTfN2uXbtYsWIFANu3b+eMM85g165dTJ48OX7qhIQEFi1aBMCrr77Krl27\ngFgg3759+2GVt/cc2dnZdOrU6Xvb27RpQ0ZGxo+//kJgiJYkSSomtm7dypNPPhnpmP1v5DssI+6G\nPrnQkNi8hIbAL3ZS94QyjBkzhvr167Nly5b4fOjbb7+d5ORkUlJSeO+99wC4//77ueCCC2jRogX1\n6tWLn7p///7885//JDk5mYULF3LSSScBkJSUROnSpUlOTubRRx/9wfJ+//vfM2TIEDp27Pi90fHi\nIgjDsKhrOKTU1NSwqP/akCRJOtqys7Pp0KEDy5cvP+xjevfuTYcOHejatevhv1HpUrER6IITe/OA\nPgHs3nP45znKsrOzad++PU2bNmXx4sU0bNiQSZMm8fOf/5xRo0aRmppKxYoVycnJAWDatGnMnDmT\niRMnsmnTJm688UbWr18PwGOPPRaf7/1DgiBYFIZh6qH2cyRakiSpmLjjjjtYu3YtKSkpDB06lKFD\nh9KoUSMSExPjDyEJw5BBgwZRt25dLr30Ur788sv48SNGjCAtLY1GjRoxYMAAwjBk7dq1NGnSJL7P\nmjVraFK2LKze781XE5sbXczs3zHkcEfqhwwZwq233kp6ejrTp0+nX79+hVqXIVqSJKmYeOihhzjv\nvPPIzMykefPmZGZmkpWVxezZsxk6dCgbN27k5ZdfZvXq1Xz00UdMmjQpPr0CYNCgQaSnp7N8+XJ2\n7NjBzJkzOe+886hSpUr8QSYTJkygT/drYUIFWEFsBHoFseV7RxbJdf+Q/TuGLFiw4LCOmz17NoMG\nDSIlJYWOHTvy9ddfx0esC4PdOSRJkoqhBQsWcN1111G6dGlq1KjBRRddRHp6OvPmzYuvP/PMM7nk\nkkvix8yZM4dHHnmE3NxcvvrqKxo2bMhVV11Fv379mDBhAv/zP//D1KlT+fDDD+HNy/ftzjFqJPTo\nWYRXfGD7z4n+oeWCnTz27NnD+++/T/ny5Y9KXY5ES5IkFZUpk6FeQmyOcr0EeGXGjz7Vzp07GThw\nINOmTWPZsmX0798/Hiq7dOnCG2+8wcyZM2natCmnnnpqLDCvyo7NgV6VXXwCdMHPpG3L73UMadmy\n5T6716hRg5UrV7Jnzx5efvnl+PrLL7+cxx9/PL5c2I8UN0RLkiQVhQO0mav0h7vY/u9/A9CqVSum\nTp3K7t272bRpE/PmzaNZs2a0bt06vn7jxo3MmTMH+O8obLVq1cjJydmnY0f58uVp164dN910E336\n9Dn213q49v9M2m+gbumAMUNvi3cMuemmm/Y55KGHHqJDhw5ceOGF8QfBAIwePZqMjAySkpJo0KAB\nY8eOLdRS7c4hSZJUFOolxMJiwwLrVkCPxyuwtFZtrrjiCgDeeOMNgiBg2LBhdO/enTAMGTx4MG+9\n9Ra1atWibNmy9O3bl65duzJs2DD+8pe/cPrpp3P++edzzjnnMHz4cADef/99unbtyrp16yhduvQx\nv9zDcpDPhOnnxEbLj4HD7c5hiJYkSSoKx7jN3KhRo9i2bRv333//IfdNSEggIyODatWqFXodP6gY\ntN47pi3ugiD4bRAEYRAE1fKXgyAIRgdB8HEQBEuDIGhSYN8bgiBYk/9zQ2G8vyRJUolTp9YxazPX\nuXNnJk2axJAhQwr93IXqGH4mR+qIu3MEQVATuBxYX2D1FUCd/J8LgKeAC4IgOAX4PZAKhMCiIAhe\nDcNwy5HWIUmSVKLcOzI2/7dPLtQlFhYnVIh1yShkBW+4298333xDt27d+Oyzz9i9ezf33HNPfNuO\nHTu45ppruOaaa/j000855ZRT+PWvfw3A3XffzWmnnVa4wfwYfiZHqjBGoh8FfkcsFO/VCZgUxrwP\nVA2C4AygHfBWGIZf5Qfnt4D2hVCDJElSydKjJ4x6Jjbft08Q+z3qmWPeJWPWrFmceeaZZGVlsXz5\nctq3j0WznJwcrrrqKq677jr69+9P3759mTRpEhBrH/fiiy/Sq1evwi2mmHwmh+OIRqKDIOgEbAjD\nMGu/nn1nAZ8WWP4sf93B1kuSJB1/evQs8oCYmJjIb3/7W26//XY6dOhAq1atAOjUqRO/+93v6Nkz\nVl9CQgKnnnoqS5Ys4YsvvqBx48axVnmFrRh8JofjkCE6CILZwOkH2HQ3cBexqRyFLgiCAcAAgFq1\nit88GEmSpBJryuT4g1bOr1OLxXfdyd8rnMSwYcNo27YtAC1atGDWrFn06NEj/kCTfv36MXHiRP79\n73/Tt2/foryCIveju3MEQZAIvA3k5q86G/gcaAbcB8wNw/Av+fuuBtrs/QnD8P/LX/90wf0Oxu4c\nkiRJhWRvL+b8ecefZ8ApfzuR8n8ax8zKVXj22WfJzMwkIyODESNGkJeXx5NPPgnAd999R2JiIrt2\n7WLNmjXFt1XeETjq3TnCMFwWhuFpYRgmhGGYQGxqRpMwDP8NvApcn9+lozmwLQzDjcCbwOVBEJwc\nBMHJxEax3/yxNUiSJCmiEXfHAnRDoAwsqwDNdu8gpW8f7rvvPoYNGxbf9X//93/ZsWMHv/vd7wA4\n4YQTuPjii+nWrdtPMkBHUWh9ooMgyAZSwzDcHMTG/J8gdtNgLtAnDMOM/P36EpsGAjAyDMMJhzq3\nI9GSJEmF5Ah6Me/Zs4cmTZrwt7/9jTp16hzVMovK4Y5EH3GLu73yR6P3vg6Bmw+y33PAc4X1vpIk\nSYqgTi1Yvd9TAQ+jF/NHH31Ehw4d6Ny58082QEdRaCFakiRJJcCP7MXcoEEDPvnkk2NSYklgiJYk\nSTqe7G0fl9+dgzq1YgG6BLSVK04M0ZIkScebEtKLuTgrjCcWSpIkSccVQ7QkSZIUkSFakiRJisgQ\nLUmSJEVkiJYkSZIiMkRLkiRJERmiJUmSpIgM0ZIkSVJEhmhJkiQpIkO0JEmSFFEQhmFR13BIQRBs\nAtYVdR1ANWBzURehOL+P4sfvpHjx+yhe/D6KF7+P4qU4fR/nhGFY/VA7lYgQXVwEQZARhmFqUdeh\nGL+P4sfvpHjx+yhe/D6KF7+P4qUkfh9O55AkSZIiMkRLkiRJERmio3mmqAvQPvw+ih+/k+LF76N4\n8fsoXvw+/v/27i7EqioM4/j/yc8gyjIxcQSFhLAoiwjDG9HCyUQrLIwoKyECA4OgNC8i6KIIsu9u\nSrKQTCxRrChTwSs1SjM/sqYiMqyBUisEw3y72O/UZnB0Do5nzznn+cFh9nrXvlis58zMmr3XPtO/\nNFwe3hNtZmZmZlYjX4k2MzMzM6uRF9E9kPSspK8l7ZK0RtKwUt9iSR2S9kuaXqq3Z61D0qJqRt6c\nJN0uaY+kE5Ku7dbnPCrmua6GpGWSOiXtLtUukrRB0rf59cKsS9KLmdEuSddUN/LmI2mMpM2S9ubP\nqoVZdx4VkTRU0nZJX2YmT2Z9nKRtOffvShqc9SHZ7sj+sVWOvxlJGiBph6T12W7oLLyI7tkG4IqI\nuBL4BlgMIGkCMBe4HGgHXs03xQDgFeAmYAJwZ55rfWM3cBuwpVx0HtXzXFfqTYr3fdkiYGNEjAc2\nZhuKfMbn6wHgtTqNsVUcBx6JiAnAJGBBfh84j+ocA6ZGxFXARKBd0iTgGWBpRFwKHALm5/nzgUNZ\nX5rnWd9aCOwrtRs6Cy+iexARn0TE8WxuBdryeDawMiKORcQPQAdwXb46IuL7iPgbWJnnWh+IiH0R\nsf8kXc6jep7rikTEFuD3buXZwPI8Xg7cUqq/FYWtwDBJo+oz0uYXEQcj4os8/pNioTAa51GZnNu/\nsjkoXwFMBVZnvXsmXVmtBqZJUp2G2/QktQE3A69nWzR4Fl5E9879wEd5PBr4qdR3IGs91e3sch7V\n81z3LyMj4mAe/wKMzGPnVCd56/lqYBvOo1J5Z3In0Elxh/k74HDpIll53v/LJPuPAMPrO+Km9jzw\nKHAi28Np8CwGVj2AKkn6FLjkJF1LImJtnrOE4jbdinqOrRX1Jg8z672ICEn+CKY6knQe8B7wcET8\nUb545jzqLyL+ASbmc01rgMsqHlJLkjQT6IyIzyVNqXo8faWlF9ERccOp+iXdC8wEpsX/nwX4MzCm\ndFpb1jhF3XrhdHn0wHlU71QZWP39KmlURBzM7QGdWXdOZ5mkQRQL6BUR8X6WnUc/EBGHJW0GrqfY\nOjMwr3CW570rkwOSBgIXAL9VMuDmMxmYJWkGMBQ4H3iBBs/C2zl6IKmd4rbDrIg4WupaB8zNJ0fH\nUTwUsh34DBifT5oOpnjYbV29x92CnEf1PNf9yzpgXh7PA9aW6vfkp0JMAo6UthnYGcr9mm8A+yLi\nuVKX86iIpBF5BRpJ5wI3UuxV3wzMydO6Z9KV1RxgU+kCmp2BiFgcEW0RMZbid8SmiLiLBs+ipa9E\nn8bLwBBgQ96O2xoRD0bEHkmrgL0U2zwW5O0iJD0EfAwMAJZFxJ5qht58JN0KvASMAD6QtDMipjuP\n6kXEcc91NSS9A0wBLpZ0AHgCeBpYJWk+8CNwR57+ITCD4uHbo8B9dR9wc5sM3A18lXtwAR7HeVRp\nFLA8P0HoHGBVRKyXtN+RvKMAAABwSURBVBdYKekpYAfFHz/k17cldVA8sDu3ikG3mMdo4Cz8HwvN\nzMzMzGrk7RxmZmZmZjXyItrMzMzMrEZeRJuZmZmZ1ciLaDMzMzOzGnkRbWZmZmZWIy+izczMzMxq\n5EW0mZmZmVmNvIg2MzMzM6vRvwEyPu4Vjj3QAAAAAElFTkSuQmCC\n",
      "text/plain": [
       "<Figure size 864x432 with 1 Axes>"
      ]
     },
     "metadata": {
      "tags": []
     },
     "output_type": "display_data"
    }
   ],
   "source": [
    "import matplotlib.pyplot as plt\n",
    "\n",
    "%matplotlib inline\n",
    "\n",
    "# visualize embeddings\n",
    "from sklearn.manifold import TSNE\n",
    "\n",
    "words = w2v_model.wv.index2word\n",
    "wvs = w2v_model.wv[words]\n",
    "\n",
    "tsne = TSNE(n_components=2, random_state=42, n_iter=5000, perplexity=5)\n",
    "np.set_printoptions(suppress=True)\n",
    "T = tsne.fit_transform(wvs)\n",
    "labels = words\n",
    "\n",
    "plt.figure(figsize=(12, 6))\n",
    "plt.scatter(T[:, 0], T[:, 1], c='orange', edgecolors='r')\n",
    "for label, x, y in zip(labels, T[:, 0], T[:, 1]):\n",
    "    plt.annotate(label, xy=(x+1, y+1), xytext=(0, 0), textcoords='offset points')"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 5,
   "metadata": {
    "colab": {
     "base_uri": "https://localhost:8080/",
     "height": 85
    },
    "colab_type": "code",
    "id": "V7h2GmytRiqq",
    "outputId": "1a4fac8b-d815-4444-da85-cfbde23d1482"
   },
   "outputs": [
    {
     "data": {
      "text/plain": [
       "(array([-0.8185685 ,  0.3990876 ,  0.02527559, -0.46214026,  0.20842107,\n",
       "         0.75595075, -0.6801817 , -0.65128547,  0.23382275,  0.45040986,\n",
       "         0.97311556,  1.0209339 ,  1.1465698 ,  0.6203446 ,  1.0008206 ],\n",
       "       dtype=float32), (15,))"
      ]
     },
     "execution_count": 5,
     "metadata": {
      "tags": []
     },
     "output_type": "execute_result"
    }
   ],
   "source": [
    "w2v_model.wv['sky'], w2v_model.wv['sky'].shape"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 6,
   "metadata": {
    "colab": {
     "base_uri": "https://localhost:8080/",
     "height": 669
    },
    "colab_type": "code",
    "id": "3INpkJK4Riqs",
    "outputId": "bebb8557-724d-40c7-bf92-bbeb5e0bf737"
   },
   "outputs": [
    {
     "data": {
      "text/html": [
       "<div>\n",
       "<style scoped>\n",
       "    .dataframe tbody tr th:only-of-type {\n",
       "        vertical-align: middle;\n",
       "    }\n",
       "\n",
       "    .dataframe tbody tr th {\n",
       "        vertical-align: top;\n",
       "    }\n",
       "\n",
       "    .dataframe thead th {\n",
       "        text-align: right;\n",
       "    }\n",
       "</style>\n",
       "<table border=\"1\" class=\"dataframe\">\n",
       "  <thead>\n",
       "    <tr style=\"text-align: right;\">\n",
       "      <th></th>\n",
       "      <th>0</th>\n",
       "      <th>1</th>\n",
       "      <th>2</th>\n",
       "      <th>3</th>\n",
       "      <th>4</th>\n",
       "      <th>5</th>\n",
       "      <th>6</th>\n",
       "      <th>7</th>\n",
       "      <th>8</th>\n",
       "      <th>9</th>\n",
       "      <th>10</th>\n",
       "      <th>11</th>\n",
       "      <th>12</th>\n",
       "      <th>13</th>\n",
       "      <th>14</th>\n",
       "    </tr>\n",
       "  </thead>\n",
       "  <tbody>\n",
       "    <tr>\n",
       "      <th>sky</th>\n",
       "      <td>-0.818569</td>\n",
       "      <td>0.399088</td>\n",
       "      <td>0.025276</td>\n",
       "      <td>-0.462140</td>\n",
       "      <td>0.208421</td>\n",
       "      <td>0.755951</td>\n",
       "      <td>-0.680182</td>\n",
       "      <td>-0.651285</td>\n",
       "      <td>0.233823</td>\n",
       "      <td>0.450410</td>\n",
       "      <td>0.973116</td>\n",
       "      <td>1.020934</td>\n",
       "      <td>1.146570</td>\n",
       "      <td>0.620345</td>\n",
       "      <td>1.000821</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>blue</th>\n",
       "      <td>-0.347705</td>\n",
       "      <td>0.484645</td>\n",
       "      <td>-0.123974</td>\n",
       "      <td>0.018303</td>\n",
       "      <td>0.018064</td>\n",
       "      <td>0.105478</td>\n",
       "      <td>-0.628725</td>\n",
       "      <td>-0.625935</td>\n",
       "      <td>0.337193</td>\n",
       "      <td>0.179394</td>\n",
       "      <td>0.567942</td>\n",
       "      <td>0.625656</td>\n",
       "      <td>0.772077</td>\n",
       "      <td>0.308862</td>\n",
       "      <td>0.901417</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>beautiful</th>\n",
       "      <td>-0.631715</td>\n",
       "      <td>0.489223</td>\n",
       "      <td>-0.039816</td>\n",
       "      <td>-0.560539</td>\n",
       "      <td>0.027471</td>\n",
       "      <td>0.600807</td>\n",
       "      <td>-0.557207</td>\n",
       "      <td>-0.453279</td>\n",
       "      <td>0.333005</td>\n",
       "      <td>0.194662</td>\n",
       "      <td>0.700585</td>\n",
       "      <td>1.144448</td>\n",
       "      <td>1.131276</td>\n",
       "      <td>0.647305</td>\n",
       "      <td>0.927159</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>quick</th>\n",
       "      <td>-0.377969</td>\n",
       "      <td>0.875323</td>\n",
       "      <td>0.505043</td>\n",
       "      <td>0.503531</td>\n",
       "      <td>-0.134367</td>\n",
       "      <td>-0.698876</td>\n",
       "      <td>-0.583188</td>\n",
       "      <td>0.053357</td>\n",
       "      <td>0.380772</td>\n",
       "      <td>0.780073</td>\n",
       "      <td>0.057065</td>\n",
       "      <td>-0.184094</td>\n",
       "      <td>0.876488</td>\n",
       "      <td>-0.264486</td>\n",
       "      <td>1.617072</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>brown</th>\n",
       "      <td>-0.426562</td>\n",
       "      <td>0.552525</td>\n",
       "      <td>0.424001</td>\n",
       "      <td>0.568211</td>\n",
       "      <td>-0.203049</td>\n",
       "      <td>-0.795935</td>\n",
       "      <td>-0.452301</td>\n",
       "      <td>-0.161856</td>\n",
       "      <td>0.344370</td>\n",
       "      <td>0.999754</td>\n",
       "      <td>0.189382</td>\n",
       "      <td>-0.306823</td>\n",
       "      <td>0.981535</td>\n",
       "      <td>-0.455129</td>\n",
       "      <td>1.527002</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>fox</th>\n",
       "      <td>-0.544743</td>\n",
       "      <td>0.937319</td>\n",
       "      <td>0.375954</td>\n",
       "      <td>0.619939</td>\n",
       "      <td>-0.003503</td>\n",
       "      <td>-0.792852</td>\n",
       "      <td>-0.575107</td>\n",
       "      <td>0.010681</td>\n",
       "      <td>0.432367</td>\n",
       "      <td>0.760273</td>\n",
       "      <td>0.179154</td>\n",
       "      <td>-0.037524</td>\n",
       "      <td>0.739188</td>\n",
       "      <td>-0.455503</td>\n",
       "      <td>1.631031</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>lazy</th>\n",
       "      <td>-0.323250</td>\n",
       "      <td>0.807996</td>\n",
       "      <td>0.387009</td>\n",
       "      <td>0.391866</td>\n",
       "      <td>0.059279</td>\n",
       "      <td>-0.911020</td>\n",
       "      <td>-0.712572</td>\n",
       "      <td>-0.355619</td>\n",
       "      <td>0.513344</td>\n",
       "      <td>1.131328</td>\n",
       "      <td>0.164242</td>\n",
       "      <td>-0.431585</td>\n",
       "      <td>0.981779</td>\n",
       "      <td>-0.382720</td>\n",
       "      <td>1.228837</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>dog</th>\n",
       "      <td>-0.497215</td>\n",
       "      <td>0.946547</td>\n",
       "      <td>0.381737</td>\n",
       "      <td>0.588996</td>\n",
       "      <td>-0.123029</td>\n",
       "      <td>-0.604750</td>\n",
       "      <td>-0.472004</td>\n",
       "      <td>-0.014517</td>\n",
       "      <td>0.493909</td>\n",
       "      <td>0.639742</td>\n",
       "      <td>0.004018</td>\n",
       "      <td>-0.108050</td>\n",
       "      <td>0.898489</td>\n",
       "      <td>-0.554202</td>\n",
       "      <td>1.574298</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>love</th>\n",
       "      <td>-0.631920</td>\n",
       "      <td>-0.159586</td>\n",
       "      <td>0.422561</td>\n",
       "      <td>-0.758301</td>\n",
       "      <td>-0.140733</td>\n",
       "      <td>0.286956</td>\n",
       "      <td>-0.247963</td>\n",
       "      <td>-0.187684</td>\n",
       "      <td>0.390543</td>\n",
       "      <td>0.290638</td>\n",
       "      <td>0.500850</td>\n",
       "      <td>0.830710</td>\n",
       "      <td>0.013555</td>\n",
       "      <td>0.353955</td>\n",
       "      <td>0.172023</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>sausages</th>\n",
       "      <td>-0.576832</td>\n",
       "      <td>-0.458122</td>\n",
       "      <td>1.026463</td>\n",
       "      <td>-1.154116</td>\n",
       "      <td>-0.650626</td>\n",
       "      <td>-0.082031</td>\n",
       "      <td>-0.189536</td>\n",
       "      <td>0.369872</td>\n",
       "      <td>0.995566</td>\n",
       "      <td>0.419234</td>\n",
       "      <td>0.693259</td>\n",
       "      <td>0.424213</td>\n",
       "      <td>-0.044790</td>\n",
       "      <td>0.079687</td>\n",
       "      <td>0.145574</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>ham</th>\n",
       "      <td>-0.877566</td>\n",
       "      <td>-0.482393</td>\n",
       "      <td>0.929203</td>\n",
       "      <td>-1.328698</td>\n",
       "      <td>-0.354967</td>\n",
       "      <td>-0.043935</td>\n",
       "      <td>-0.003959</td>\n",
       "      <td>0.160286</td>\n",
       "      <td>1.060482</td>\n",
       "      <td>-0.005815</td>\n",
       "      <td>0.600583</td>\n",
       "      <td>0.441404</td>\n",
       "      <td>0.137963</td>\n",
       "      <td>-0.205029</td>\n",
       "      <td>0.152584</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>bacon</th>\n",
       "      <td>-0.976038</td>\n",
       "      <td>-0.463793</td>\n",
       "      <td>0.790353</td>\n",
       "      <td>-0.859319</td>\n",
       "      <td>-0.761272</td>\n",
       "      <td>-0.040525</td>\n",
       "      <td>-0.344847</td>\n",
       "      <td>0.335068</td>\n",
       "      <td>1.191478</td>\n",
       "      <td>0.181390</td>\n",
       "      <td>0.441230</td>\n",
       "      <td>0.578118</td>\n",
       "      <td>-0.040014</td>\n",
       "      <td>-0.233115</td>\n",
       "      <td>0.004666</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>eggs</th>\n",
       "      <td>-0.840217</td>\n",
       "      <td>-0.629078</td>\n",
       "      <td>1.017721</td>\n",
       "      <td>-1.310373</td>\n",
       "      <td>-0.434270</td>\n",
       "      <td>-0.297177</td>\n",
       "      <td>-0.330215</td>\n",
       "      <td>0.229346</td>\n",
       "      <td>0.703244</td>\n",
       "      <td>0.318045</td>\n",
       "      <td>0.607417</td>\n",
       "      <td>0.403582</td>\n",
       "      <td>-0.140879</td>\n",
       "      <td>-0.105237</td>\n",
       "      <td>0.229215</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>jumps</th>\n",
       "      <td>-0.341378</td>\n",
       "      <td>0.670975</td>\n",
       "      <td>0.437849</td>\n",
       "      <td>0.494568</td>\n",
       "      <td>-0.176995</td>\n",
       "      <td>-0.914049</td>\n",
       "      <td>-0.521373</td>\n",
       "      <td>-0.026236</td>\n",
       "      <td>0.475310</td>\n",
       "      <td>0.806331</td>\n",
       "      <td>-0.020645</td>\n",
       "      <td>-0.340928</td>\n",
       "      <td>0.754797</td>\n",
       "      <td>-0.486747</td>\n",
       "      <td>1.407213</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>kings</th>\n",
       "      <td>-0.893495</td>\n",
       "      <td>-0.469336</td>\n",
       "      <td>1.210493</td>\n",
       "      <td>-1.258101</td>\n",
       "      <td>-0.816817</td>\n",
       "      <td>-0.789102</td>\n",
       "      <td>-0.231874</td>\n",
       "      <td>0.428963</td>\n",
       "      <td>1.099132</td>\n",
       "      <td>0.218836</td>\n",
       "      <td>0.067266</td>\n",
       "      <td>0.314867</td>\n",
       "      <td>0.015383</td>\n",
       "      <td>-0.180145</td>\n",
       "      <td>0.182565</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>breakfast</th>\n",
       "      <td>-0.772558</td>\n",
       "      <td>-0.558312</td>\n",
       "      <td>1.258790</td>\n",
       "      <td>-0.937237</td>\n",
       "      <td>-0.905406</td>\n",
       "      <td>-0.658782</td>\n",
       "      <td>-0.175029</td>\n",
       "      <td>0.624610</td>\n",
       "      <td>1.118971</td>\n",
       "      <td>0.501862</td>\n",
       "      <td>0.534945</td>\n",
       "      <td>0.137323</td>\n",
       "      <td>-0.297624</td>\n",
       "      <td>-0.415096</td>\n",
       "      <td>0.298166</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>toast</th>\n",
       "      <td>-0.753567</td>\n",
       "      <td>-0.569372</td>\n",
       "      <td>1.248701</td>\n",
       "      <td>-1.184956</td>\n",
       "      <td>-0.870067</td>\n",
       "      <td>-0.782575</td>\n",
       "      <td>-0.137969</td>\n",
       "      <td>0.573266</td>\n",
       "      <td>1.246748</td>\n",
       "      <td>0.434166</td>\n",
       "      <td>0.161741</td>\n",
       "      <td>0.153767</td>\n",
       "      <td>-0.128329</td>\n",
       "      <td>-0.177872</td>\n",
       "      <td>0.331763</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>beans</th>\n",
       "      <td>-0.924956</td>\n",
       "      <td>-0.503358</td>\n",
       "      <td>1.221505</td>\n",
       "      <td>-1.280991</td>\n",
       "      <td>-0.901741</td>\n",
       "      <td>-0.725451</td>\n",
       "      <td>-0.042456</td>\n",
       "      <td>0.466820</td>\n",
       "      <td>1.198279</td>\n",
       "      <td>0.321750</td>\n",
       "      <td>0.184301</td>\n",
       "      <td>0.354623</td>\n",
       "      <td>-0.019580</td>\n",
       "      <td>-0.172424</td>\n",
       "      <td>0.229652</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>green</th>\n",
       "      <td>-0.496436</td>\n",
       "      <td>-0.483039</td>\n",
       "      <td>0.496584</td>\n",
       "      <td>-1.251115</td>\n",
       "      <td>-0.488516</td>\n",
       "      <td>-0.087176</td>\n",
       "      <td>-0.477904</td>\n",
       "      <td>0.081075</td>\n",
       "      <td>0.932781</td>\n",
       "      <td>-0.092416</td>\n",
       "      <td>0.358710</td>\n",
       "      <td>0.611974</td>\n",
       "      <td>0.158456</td>\n",
       "      <td>0.380635</td>\n",
       "      <td>0.238793</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>today</th>\n",
       "      <td>-0.682214</td>\n",
       "      <td>0.454401</td>\n",
       "      <td>0.012207</td>\n",
       "      <td>-0.301444</td>\n",
       "      <td>-0.004128</td>\n",
       "      <td>0.539051</td>\n",
       "      <td>-0.560557</td>\n",
       "      <td>-0.669468</td>\n",
       "      <td>0.236689</td>\n",
       "      <td>0.297953</td>\n",
       "      <td>0.785535</td>\n",
       "      <td>1.206358</td>\n",
       "      <td>0.864446</td>\n",
       "      <td>0.635667</td>\n",
       "      <td>0.865603</td>\n",
       "    </tr>\n",
       "  </tbody>\n",
       "</table>\n",
       "</div>"
      ],
      "text/plain": [
       "                 0         1         2   ...        12        13        14\n",
       "sky       -0.818569  0.399088  0.025276  ...  1.146570  0.620345  1.000821\n",
       "blue      -0.347705  0.484645 -0.123974  ...  0.772077  0.308862  0.901417\n",
       "beautiful -0.631715  0.489223 -0.039816  ...  1.131276  0.647305  0.927159\n",
       "quick     -0.377969  0.875323  0.505043  ...  0.876488 -0.264486  1.617072\n",
       "brown     -0.426562  0.552525  0.424001  ...  0.981535 -0.455129  1.527002\n",
       "fox       -0.544743  0.937319  0.375954  ...  0.739188 -0.455503  1.631031\n",
       "lazy      -0.323250  0.807996  0.387009  ...  0.981779 -0.382720  1.228837\n",
       "dog       -0.497215  0.946547  0.381737  ...  0.898489 -0.554202  1.574298\n",
       "love      -0.631920 -0.159586  0.422561  ...  0.013555  0.353955  0.172023\n",
       "sausages  -0.576832 -0.458122  1.026463  ... -0.044790  0.079687  0.145574\n",
       "ham       -0.877566 -0.482393  0.929203  ...  0.137963 -0.205029  0.152584\n",
       "bacon     -0.976038 -0.463793  0.790353  ... -0.040014 -0.233115  0.004666\n",
       "eggs      -0.840217 -0.629078  1.017721  ... -0.140879 -0.105237  0.229215\n",
       "jumps     -0.341378  0.670975  0.437849  ...  0.754797 -0.486747  1.407213\n",
       "kings     -0.893495 -0.469336  1.210493  ...  0.015383 -0.180145  0.182565\n",
       "breakfast -0.772558 -0.558312  1.258790  ... -0.297624 -0.415096  0.298166\n",
       "toast     -0.753567 -0.569372  1.248701  ... -0.128329 -0.177872  0.331763\n",
       "beans     -0.924956 -0.503358  1.221505  ... -0.019580 -0.172424  0.229652\n",
       "green     -0.496436 -0.483039  0.496584  ...  0.158456  0.380635  0.238793\n",
       "today     -0.682214  0.454401  0.012207  ...  0.864446  0.635667  0.865603\n",
       "\n",
       "[20 rows x 15 columns]"
      ]
     },
     "execution_count": 6,
     "metadata": {
      "tags": []
     },
     "output_type": "execute_result"
    }
   ],
   "source": [
    "vec_df = pd.DataFrame(wvs, index=words)\n",
    "vec_df"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {
    "colab_type": "text",
    "id": "gTCiDguVRiqu"
   },
   "source": [
    "### Looking at term semantic similarity"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 7,
   "metadata": {
    "colab": {
     "base_uri": "https://localhost:8080/",
     "height": 689
    },
    "colab_type": "code",
    "id": "S63FJhi2Riqv",
    "outputId": "ca368c74-ffbc-4b7a-9d48-25c2b3aa9358"
   },
   "outputs": [
    {
     "data": {
      "text/html": [
       "<div>\n",
       "<style scoped>\n",
       "    .dataframe tbody tr th:only-of-type {\n",
       "        vertical-align: middle;\n",
       "    }\n",
       "\n",
       "    .dataframe tbody tr th {\n",
       "        vertical-align: top;\n",
       "    }\n",
       "\n",
       "    .dataframe thead th {\n",
       "        text-align: right;\n",
       "    }\n",
       "</style>\n",
       "<table border=\"1\" class=\"dataframe\">\n",
       "  <thead>\n",
       "    <tr style=\"text-align: right;\">\n",
       "      <th></th>\n",
       "      <th>sky</th>\n",
       "      <th>blue</th>\n",
       "      <th>beautiful</th>\n",
       "      <th>quick</th>\n",
       "      <th>brown</th>\n",
       "      <th>fox</th>\n",
       "      <th>lazy</th>\n",
       "      <th>dog</th>\n",
       "      <th>love</th>\n",
       "      <th>sausages</th>\n",
       "      <th>ham</th>\n",
       "      <th>bacon</th>\n",
       "      <th>eggs</th>\n",
       "      <th>jumps</th>\n",
       "      <th>kings</th>\n",
       "      <th>breakfast</th>\n",
       "      <th>toast</th>\n",
       "      <th>beans</th>\n",
       "      <th>green</th>\n",
       "      <th>today</th>\n",
       "    </tr>\n",
       "  </thead>\n",
       "  <tbody>\n",
       "    <tr>\n",
       "      <th>sky</th>\n",
       "      <td>1.000000</td>\n",
       "      <td>0.916322</td>\n",
       "      <td>0.978884</td>\n",
       "      <td>0.431173</td>\n",
       "      <td>0.405290</td>\n",
       "      <td>0.432529</td>\n",
       "      <td>0.406296</td>\n",
       "      <td>0.419792</td>\n",
       "      <td>0.723005</td>\n",
       "      <td>0.348153</td>\n",
       "      <td>0.375051</td>\n",
       "      <td>0.323178</td>\n",
       "      <td>0.333458</td>\n",
       "      <td>0.307980</td>\n",
       "      <td>0.151495</td>\n",
       "      <td>0.101826</td>\n",
       "      <td>0.105531</td>\n",
       "      <td>0.163778</td>\n",
       "      <td>0.473998</td>\n",
       "      <td>0.980911</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>blue</th>\n",
       "      <td>0.916322</td>\n",
       "      <td>1.000000</td>\n",
       "      <td>0.912599</td>\n",
       "      <td>0.624923</td>\n",
       "      <td>0.593610</td>\n",
       "      <td>0.637191</td>\n",
       "      <td>0.604366</td>\n",
       "      <td>0.624508</td>\n",
       "      <td>0.529950</td>\n",
       "      <td>0.203664</td>\n",
       "      <td>0.220718</td>\n",
       "      <td>0.215074</td>\n",
       "      <td>0.185698</td>\n",
       "      <td>0.529882</td>\n",
       "      <td>0.091409</td>\n",
       "      <td>0.044772</td>\n",
       "      <td>0.052484</td>\n",
       "      <td>0.091067</td>\n",
       "      <td>0.362248</td>\n",
       "      <td>0.929598</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>beautiful</th>\n",
       "      <td>0.978884</td>\n",
       "      <td>0.912599</td>\n",
       "      <td>1.000000</td>\n",
       "      <td>0.415492</td>\n",
       "      <td>0.364876</td>\n",
       "      <td>0.410171</td>\n",
       "      <td>0.361448</td>\n",
       "      <td>0.408251</td>\n",
       "      <td>0.726956</td>\n",
       "      <td>0.365698</td>\n",
       "      <td>0.398261</td>\n",
       "      <td>0.350296</td>\n",
       "      <td>0.334364</td>\n",
       "      <td>0.285894</td>\n",
       "      <td>0.197300</td>\n",
       "      <td>0.112035</td>\n",
       "      <td>0.140627</td>\n",
       "      <td>0.206921</td>\n",
       "      <td>0.533898</td>\n",
       "      <td>0.981693</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>quick</th>\n",
       "      <td>0.431173</td>\n",
       "      <td>0.624923</td>\n",
       "      <td>0.415492</td>\n",
       "      <td>1.000000</td>\n",
       "      <td>0.974776</td>\n",
       "      <td>0.986562</td>\n",
       "      <td>0.948987</td>\n",
       "      <td>0.985003</td>\n",
       "      <td>0.078848</td>\n",
       "      <td>0.147220</td>\n",
       "      <td>0.095342</td>\n",
       "      <td>0.139736</td>\n",
       "      <td>0.140857</td>\n",
       "      <td>0.980632</td>\n",
       "      <td>0.238964</td>\n",
       "      <td>0.268426</td>\n",
       "      <td>0.265857</td>\n",
       "      <td>0.229278</td>\n",
       "      <td>0.076854</td>\n",
       "      <td>0.411802</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>brown</th>\n",
       "      <td>0.405290</td>\n",
       "      <td>0.593610</td>\n",
       "      <td>0.364876</td>\n",
       "      <td>0.974776</td>\n",
       "      <td>1.000000</td>\n",
       "      <td>0.966186</td>\n",
       "      <td>0.966826</td>\n",
       "      <td>0.964126</td>\n",
       "      <td>0.059448</td>\n",
       "      <td>0.150536</td>\n",
       "      <td>0.103589</td>\n",
       "      <td>0.145033</td>\n",
       "      <td>0.156364</td>\n",
       "      <td>0.983229</td>\n",
       "      <td>0.239937</td>\n",
       "      <td>0.290021</td>\n",
       "      <td>0.274447</td>\n",
       "      <td>0.236794</td>\n",
       "      <td>0.050416</td>\n",
       "      <td>0.374005</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>fox</th>\n",
       "      <td>0.432529</td>\n",
       "      <td>0.637191</td>\n",
       "      <td>0.410171</td>\n",
       "      <td>0.986562</td>\n",
       "      <td>0.966186</td>\n",
       "      <td>1.000000</td>\n",
       "      <td>0.940885</td>\n",
       "      <td>0.988680</td>\n",
       "      <td>0.086827</td>\n",
       "      <td>0.125054</td>\n",
       "      <td>0.094065</td>\n",
       "      <td>0.146798</td>\n",
       "      <td>0.134135</td>\n",
       "      <td>0.973922</td>\n",
       "      <td>0.219031</td>\n",
       "      <td>0.260623</td>\n",
       "      <td>0.242012</td>\n",
       "      <td>0.211370</td>\n",
       "      <td>0.051543</td>\n",
       "      <td>0.421023</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>lazy</th>\n",
       "      <td>0.406296</td>\n",
       "      <td>0.604366</td>\n",
       "      <td>0.361448</td>\n",
       "      <td>0.948987</td>\n",
       "      <td>0.966826</td>\n",
       "      <td>0.940885</td>\n",
       "      <td>1.000000</td>\n",
       "      <td>0.931963</td>\n",
       "      <td>0.059554</td>\n",
       "      <td>0.134683</td>\n",
       "      <td>0.088038</td>\n",
       "      <td>0.118733</td>\n",
       "      <td>0.141515</td>\n",
       "      <td>0.965676</td>\n",
       "      <td>0.224570</td>\n",
       "      <td>0.251142</td>\n",
       "      <td>0.249651</td>\n",
       "      <td>0.210059</td>\n",
       "      <td>0.055752</td>\n",
       "      <td>0.368139</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>dog</th>\n",
       "      <td>0.419792</td>\n",
       "      <td>0.624508</td>\n",
       "      <td>0.408251</td>\n",
       "      <td>0.985003</td>\n",
       "      <td>0.964126</td>\n",
       "      <td>0.988680</td>\n",
       "      <td>0.931963</td>\n",
       "      <td>1.000000</td>\n",
       "      <td>0.052253</td>\n",
       "      <td>0.108274</td>\n",
       "      <td>0.095622</td>\n",
       "      <td>0.145849</td>\n",
       "      <td>0.104835</td>\n",
       "      <td>0.972904</td>\n",
       "      <td>0.215023</td>\n",
       "      <td>0.242703</td>\n",
       "      <td>0.233813</td>\n",
       "      <td>0.207681</td>\n",
       "      <td>0.041935</td>\n",
       "      <td>0.404676</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>love</th>\n",
       "      <td>0.723005</td>\n",
       "      <td>0.529950</td>\n",
       "      <td>0.726956</td>\n",
       "      <td>0.078848</td>\n",
       "      <td>0.059448</td>\n",
       "      <td>0.086827</td>\n",
       "      <td>0.059554</td>\n",
       "      <td>0.052253</td>\n",
       "      <td>1.000000</td>\n",
       "      <td>0.799470</td>\n",
       "      <td>0.792994</td>\n",
       "      <td>0.767679</td>\n",
       "      <td>0.796566</td>\n",
       "      <td>0.005466</td>\n",
       "      <td>0.625080</td>\n",
       "      <td>0.576993</td>\n",
       "      <td>0.582872</td>\n",
       "      <td>0.644243</td>\n",
       "      <td>0.841480</td>\n",
       "      <td>0.741229</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>sausages</th>\n",
       "      <td>0.348153</td>\n",
       "      <td>0.203664</td>\n",
       "      <td>0.365698</td>\n",
       "      <td>0.147220</td>\n",
       "      <td>0.150536</td>\n",
       "      <td>0.125054</td>\n",
       "      <td>0.134683</td>\n",
       "      <td>0.108274</td>\n",
       "      <td>0.799470</td>\n",
       "      <td>1.000000</td>\n",
       "      <td>0.944656</td>\n",
       "      <td>0.940454</td>\n",
       "      <td>0.962446</td>\n",
       "      <td>0.147289</td>\n",
       "      <td>0.910787</td>\n",
       "      <td>0.924428</td>\n",
       "      <td>0.919423</td>\n",
       "      <td>0.930842</td>\n",
       "      <td>0.897956</td>\n",
       "      <td>0.339160</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>ham</th>\n",
       "      <td>0.375051</td>\n",
       "      <td>0.220718</td>\n",
       "      <td>0.398261</td>\n",
       "      <td>0.095342</td>\n",
       "      <td>0.103589</td>\n",
       "      <td>0.094065</td>\n",
       "      <td>0.088038</td>\n",
       "      <td>0.095622</td>\n",
       "      <td>0.792994</td>\n",
       "      <td>0.944656</td>\n",
       "      <td>1.000000</td>\n",
       "      <td>0.934962</td>\n",
       "      <td>0.951132</td>\n",
       "      <td>0.101450</td>\n",
       "      <td>0.899074</td>\n",
       "      <td>0.864812</td>\n",
       "      <td>0.875990</td>\n",
       "      <td>0.912917</td>\n",
       "      <td>0.900396</td>\n",
       "      <td>0.355246</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>bacon</th>\n",
       "      <td>0.323178</td>\n",
       "      <td>0.215074</td>\n",
       "      <td>0.350296</td>\n",
       "      <td>0.139736</td>\n",
       "      <td>0.145033</td>\n",
       "      <td>0.146798</td>\n",
       "      <td>0.118733</td>\n",
       "      <td>0.145849</td>\n",
       "      <td>0.767679</td>\n",
       "      <td>0.940454</td>\n",
       "      <td>0.934962</td>\n",
       "      <td>1.000000</td>\n",
       "      <td>0.919470</td>\n",
       "      <td>0.156260</td>\n",
       "      <td>0.912529</td>\n",
       "      <td>0.907225</td>\n",
       "      <td>0.897633</td>\n",
       "      <td>0.924267</td>\n",
       "      <td>0.876585</td>\n",
       "      <td>0.331912</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>eggs</th>\n",
       "      <td>0.333458</td>\n",
       "      <td>0.185698</td>\n",
       "      <td>0.334364</td>\n",
       "      <td>0.140857</td>\n",
       "      <td>0.156364</td>\n",
       "      <td>0.134135</td>\n",
       "      <td>0.141515</td>\n",
       "      <td>0.104835</td>\n",
       "      <td>0.796566</td>\n",
       "      <td>0.962446</td>\n",
       "      <td>0.951132</td>\n",
       "      <td>0.919470</td>\n",
       "      <td>1.000000</td>\n",
       "      <td>0.155002</td>\n",
       "      <td>0.926607</td>\n",
       "      <td>0.917920</td>\n",
       "      <td>0.915359</td>\n",
       "      <td>0.931424</td>\n",
       "      <td>0.888642</td>\n",
       "      <td>0.315476</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>jumps</th>\n",
       "      <td>0.307980</td>\n",
       "      <td>0.529882</td>\n",
       "      <td>0.285894</td>\n",
       "      <td>0.980632</td>\n",
       "      <td>0.983229</td>\n",
       "      <td>0.973922</td>\n",
       "      <td>0.965676</td>\n",
       "      <td>0.972904</td>\n",
       "      <td>0.005466</td>\n",
       "      <td>0.147289</td>\n",
       "      <td>0.101450</td>\n",
       "      <td>0.156260</td>\n",
       "      <td>0.155002</td>\n",
       "      <td>1.000000</td>\n",
       "      <td>0.282232</td>\n",
       "      <td>0.319914</td>\n",
       "      <td>0.315454</td>\n",
       "      <td>0.269876</td>\n",
       "      <td>0.055042</td>\n",
       "      <td>0.286493</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>kings</th>\n",
       "      <td>0.151495</td>\n",
       "      <td>0.091409</td>\n",
       "      <td>0.197300</td>\n",
       "      <td>0.238964</td>\n",
       "      <td>0.239937</td>\n",
       "      <td>0.219031</td>\n",
       "      <td>0.224570</td>\n",
       "      <td>0.215023</td>\n",
       "      <td>0.625080</td>\n",
       "      <td>0.910787</td>\n",
       "      <td>0.899074</td>\n",
       "      <td>0.912529</td>\n",
       "      <td>0.926607</td>\n",
       "      <td>0.282232</td>\n",
       "      <td>1.000000</td>\n",
       "      <td>0.951472</td>\n",
       "      <td>0.985816</td>\n",
       "      <td>0.994538</td>\n",
       "      <td>0.839887</td>\n",
       "      <td>0.158965</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>breakfast</th>\n",
       "      <td>0.101826</td>\n",
       "      <td>0.044772</td>\n",
       "      <td>0.112035</td>\n",
       "      <td>0.268426</td>\n",
       "      <td>0.290021</td>\n",
       "      <td>0.260623</td>\n",
       "      <td>0.251142</td>\n",
       "      <td>0.242703</td>\n",
       "      <td>0.576993</td>\n",
       "      <td>0.924428</td>\n",
       "      <td>0.864812</td>\n",
       "      <td>0.907225</td>\n",
       "      <td>0.917920</td>\n",
       "      <td>0.319914</td>\n",
       "      <td>0.951472</td>\n",
       "      <td>1.000000</td>\n",
       "      <td>0.977887</td>\n",
       "      <td>0.963628</td>\n",
       "      <td>0.751828</td>\n",
       "      <td>0.097428</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>toast</th>\n",
       "      <td>0.105531</td>\n",
       "      <td>0.052484</td>\n",
       "      <td>0.140627</td>\n",
       "      <td>0.265857</td>\n",
       "      <td>0.274447</td>\n",
       "      <td>0.242012</td>\n",
       "      <td>0.249651</td>\n",
       "      <td>0.233813</td>\n",
       "      <td>0.582872</td>\n",
       "      <td>0.919423</td>\n",
       "      <td>0.875990</td>\n",
       "      <td>0.897633</td>\n",
       "      <td>0.915359</td>\n",
       "      <td>0.315454</td>\n",
       "      <td>0.985816</td>\n",
       "      <td>0.977887</td>\n",
       "      <td>1.000000</td>\n",
       "      <td>0.990308</td>\n",
       "      <td>0.805278</td>\n",
       "      <td>0.105260</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>beans</th>\n",
       "      <td>0.163778</td>\n",
       "      <td>0.091067</td>\n",
       "      <td>0.206921</td>\n",
       "      <td>0.229278</td>\n",
       "      <td>0.236794</td>\n",
       "      <td>0.211370</td>\n",
       "      <td>0.210059</td>\n",
       "      <td>0.207681</td>\n",
       "      <td>0.644243</td>\n",
       "      <td>0.930842</td>\n",
       "      <td>0.912917</td>\n",
       "      <td>0.924267</td>\n",
       "      <td>0.931424</td>\n",
       "      <td>0.269876</td>\n",
       "      <td>0.994538</td>\n",
       "      <td>0.963628</td>\n",
       "      <td>0.990308</td>\n",
       "      <td>1.000000</td>\n",
       "      <td>0.838045</td>\n",
       "      <td>0.171681</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>green</th>\n",
       "      <td>0.473998</td>\n",
       "      <td>0.362248</td>\n",
       "      <td>0.533898</td>\n",
       "      <td>0.076854</td>\n",
       "      <td>0.050416</td>\n",
       "      <td>0.051543</td>\n",
       "      <td>0.055752</td>\n",
       "      <td>0.041935</td>\n",
       "      <td>0.841480</td>\n",
       "      <td>0.897956</td>\n",
       "      <td>0.900396</td>\n",
       "      <td>0.876585</td>\n",
       "      <td>0.888642</td>\n",
       "      <td>0.055042</td>\n",
       "      <td>0.839887</td>\n",
       "      <td>0.751828</td>\n",
       "      <td>0.805278</td>\n",
       "      <td>0.838045</td>\n",
       "      <td>1.000000</td>\n",
       "      <td>0.484399</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>today</th>\n",
       "      <td>0.980911</td>\n",
       "      <td>0.929598</td>\n",
       "      <td>0.981693</td>\n",
       "      <td>0.411802</td>\n",
       "      <td>0.374005</td>\n",
       "      <td>0.421023</td>\n",
       "      <td>0.368139</td>\n",
       "      <td>0.404676</td>\n",
       "      <td>0.741229</td>\n",
       "      <td>0.339160</td>\n",
       "      <td>0.355246</td>\n",
       "      <td>0.331912</td>\n",
       "      <td>0.315476</td>\n",
       "      <td>0.286493</td>\n",
       "      <td>0.158965</td>\n",
       "      <td>0.097428</td>\n",
       "      <td>0.105260</td>\n",
       "      <td>0.171681</td>\n",
       "      <td>0.484399</td>\n",
       "      <td>1.000000</td>\n",
       "    </tr>\n",
       "  </tbody>\n",
       "</table>\n",
       "</div>"
      ],
      "text/plain": [
       "                sky      blue  beautiful  ...     beans     green     today\n",
       "sky        1.000000  0.916322   0.978884  ...  0.163778  0.473998  0.980911\n",
       "blue       0.916322  1.000000   0.912599  ...  0.091067  0.362248  0.929598\n",
       "beautiful  0.978884  0.912599   1.000000  ...  0.206921  0.533898  0.981693\n",
       "quick      0.431173  0.624923   0.415492  ...  0.229278  0.076854  0.411802\n",
       "brown      0.405290  0.593610   0.364876  ...  0.236794  0.050416  0.374005\n",
       "fox        0.432529  0.637191   0.410171  ...  0.211370  0.051543  0.421023\n",
       "lazy       0.406296  0.604366   0.361448  ...  0.210059  0.055752  0.368139\n",
       "dog        0.419792  0.624508   0.408251  ...  0.207681  0.041935  0.404676\n",
       "love       0.723005  0.529950   0.726956  ...  0.644243  0.841480  0.741229\n",
       "sausages   0.348153  0.203664   0.365698  ...  0.930842  0.897956  0.339160\n",
       "ham        0.375051  0.220718   0.398261  ...  0.912917  0.900396  0.355246\n",
       "bacon      0.323178  0.215074   0.350296  ...  0.924267  0.876585  0.331912\n",
       "eggs       0.333458  0.185698   0.334364  ...  0.931424  0.888642  0.315476\n",
       "jumps      0.307980  0.529882   0.285894  ...  0.269876  0.055042  0.286493\n",
       "kings      0.151495  0.091409   0.197300  ...  0.994538  0.839887  0.158965\n",
       "breakfast  0.101826  0.044772   0.112035  ...  0.963628  0.751828  0.097428\n",
       "toast      0.105531  0.052484   0.140627  ...  0.990308  0.805278  0.105260\n",
       "beans      0.163778  0.091067   0.206921  ...  1.000000  0.838045  0.171681\n",
       "green      0.473998  0.362248   0.533898  ...  0.838045  1.000000  0.484399\n",
       "today      0.980911  0.929598   0.981693  ...  0.171681  0.484399  1.000000\n",
       "\n",
       "[20 rows x 20 columns]"
      ]
     },
     "execution_count": 7,
     "metadata": {
      "tags": []
     },
     "output_type": "execute_result"
    }
   ],
   "source": [
    "import numpy as np\n",
    "from sklearn.metrics.pairwise import cosine_similarity\n",
    "\n",
    "similarity_matrix = cosine_similarity(vec_df.values)\n",
    "similarity_df = pd.DataFrame(similarity_matrix, index=words, columns=words)\n",
    "similarity_df"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 8,
   "metadata": {
    "colab": {
     "base_uri": "https://localhost:8080/",
     "height": 374
    },
    "colab_type": "code",
    "id": "ngIHnn__Riqx",
    "outputId": "d05dd28c-5955-4313-ce19-029b87b8dfba"
   },
   "outputs": [
    {
     "data": {
      "text/plain": [
       "sky           [today, beautiful, blue]\n",
       "blue           [today, sky, beautiful]\n",
       "beautiful           [today, sky, blue]\n",
       "quick                [fox, dog, jumps]\n",
       "brown             [jumps, quick, lazy]\n",
       "fox                [dog, quick, jumps]\n",
       "lazy             [brown, jumps, quick]\n",
       "dog                [fox, quick, jumps]\n",
       "love           [green, sausages, eggs]\n",
       "sausages            [eggs, ham, bacon]\n",
       "ham            [eggs, sausages, bacon]\n",
       "bacon           [sausages, ham, beans]\n",
       "eggs            [sausages, ham, beans]\n",
       "jumps              [brown, quick, fox]\n",
       "kings        [beans, toast, breakfast]\n",
       "breakfast        [toast, beans, kings]\n",
       "toast        [beans, kings, breakfast]\n",
       "beans        [kings, toast, breakfast]\n",
       "green            [ham, sausages, eggs]\n",
       "today           [beautiful, sky, blue]\n",
       "dtype: object"
      ]
     },
     "execution_count": 8,
     "metadata": {
      "tags": []
     },
     "output_type": "execute_result"
    }
   ],
   "source": [
    "feature_names = np.array(words)\n",
    "similarity_df.apply(lambda row: feature_names[np.argsort(-row.values)[1:4]], \n",
    "                    axis=1)"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {
    "colab_type": "text",
    "id": "rTUAkYxQRiqz"
   },
   "source": [
    "# The GloVe Model\n",
    "\n",
    "The GloVe model stands for Global Vectors which is an unsupervised learning model which can be used to obtain dense word vectors similar to Word2Vec. However the technique is different and training is performed on an aggregated global word-word co-occurrence matrix, giving us a vector space with meaningful sub-structures. This method was invented in Stanford by Pennington et al. and I recommend you to read the original paper on GloVe, _[‘GloVe: Global Vectors for Word Representation’ by Pennington et al.](https://nlp.stanford.edu/pubs/glove.pdf)_ which is an excellent read to get some perspective on how this model works.\n",
    "\n",
    "The basic methodology of the GloVe model is to first create a huge word-context co-occurence matrix consisting of (word, context) pairs such that each element in this matrix represents how often a word occurs with the context (which can be a sequence of words). The idea then is to apply matrix factorization to approximate this matrix as depicted in the following figure.\n",
    "\n",
    "![](glove_arch.png)\n",
    "\n",
    "Considering the __Word-Context (WC)__ matrix, __Word-Feature (WF)__ matrix and __Feature-Context (FC)__ matrix, we try to factorize __WC = WF x FC__\n",
    "\n",
    "Such that we we aim to reconstruct __WC__ from __WF__ and __FC__ by multiplying them. For this, we typically initialize __WF__ and __FC__ with some random weights and attempt to multiply them to get __WC'__ (an approximation of __WC__) and measure how close it is to __WC__. We do this multiple times using Stochastic Gradient Descent (SGD) to minimize the error. Finally, the __Word-Feature matrix (WF)__ gives us the word embeddings for each word where __F__ can be preset to a specific number of dimensions"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {
    "colab_type": "text",
    "id": "_UFyeFqfRiq0"
   },
   "source": [
    "# Robust Glove Model with SpaCy\n",
    "\n",
    "Let’s try and leverage GloVe based embeddings for our document clustering task. The very popular spacy framework comes with capabilities to leverage GloVe embeddings based on different language models. You can also get pre-trained word vectors and load them up as needed using gensim or spacy.\n",
    "\n",
    "If you have spacy installed, we will be using the __[`en_vectors_web_lg`](https://spacy.io/models/en#en_vectors_web_lg)__ model which consists of 300-dimensional word vectors trained on [Common Crawl](http://commoncrawl.org) with GloVe.\n",
    "\n",
    "__Install Instructions:__\n",
    "\n",
    "```\n",
    "# Use the following command to install spaCy\n",
    "> pip install -U spacy\n",
    "OR\n",
    "> conda install -c conda-forge spacy\n",
    "\n",
    "C:\\WINDOWS\\system32>python -m spacy download en_vectors_web_lg\n",
    "Collecting en_vectors_web_lg==2.0.0 from https://github.com/explosion/spacy-models/releases/download/en_vectors_web_lg-2.0.0/en_vectors_web_lg-2.0.0.tar.gz#egg=en_vectors_web_lg==2.0.0\n",
    "  Downloading https://github.com/explosion/spacy-models/releases/download/en_vectors_web_lg-2.0.0/en_vectors_web_lg-2.0.0.tar.gz (661.8MB)\n",
    "    100% |████████████████████████████████| 661.8MB 392kB/s\n",
    "Installing collected packages: en-vectors-web-lg\n",
    "  Running setup.py install for en-vectors-web-lg ... done\n",
    "Successfully installed en-vectors-web-lg-2.0.0\n",
    "You are using pip version 10.0.1, however version 18.0 is available.\n",
    "You should consider upgrading via the 'python -m pip install --upgrade pip' command.\n",
    "\n",
    "    Linking successful\n",
    "    C:\\Anaconda3\\lib\\site-packages\\en_vectors_web_lg -->\n",
    "    C:\\Anaconda3\\lib\\site-packages\\spacy\\data\\en_vectors_web_lg\n",
    "\n",
    "    You can now load the model via spacy.load('en_vectors_web_lg')\n",
    "```"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 2,
   "metadata": {
    "colab": {
     "base_uri": "https://localhost:8080/",
     "height": 156
    },
    "colab_type": "code",
    "id": "geEhPWzgTA7K",
    "outputId": "925b33eb-8de8-417f-d596-da42af328273"
   },
   "outputs": [
    {
     "name": "stdout",
     "output_type": "stream",
     "text": [
      "Requirement already satisfied: en_core_web_sm==2.1.0 from https://github.com/explosion/spacy-models/releases/download/en_core_web_sm-2.1.0/en_core_web_sm-2.1.0.tar.gz#egg=en_core_web_sm==2.1.0 in /anaconda3/lib/python3.6/site-packages\n",
      "\u001b[33mYou are using pip version 9.0.3, however version 19.2.3 is available.\n",
      "You should consider upgrading via the 'pip install --upgrade pip' command.\u001b[0m\n",
      "\u001b[38;5;2m✔ Download and installation successful\u001b[0m\n",
      "You can now load the model via spacy.load('en_core_web_sm')\n",
      "\u001b[38;5;2m✔ Linking successful\u001b[0m\n",
      "/anaconda3/lib/python3.6/site-packages/en_core_web_sm -->\n",
      "/anaconda3/lib/python3.6/site-packages/spacy/data/en\n",
      "You can now load the model via spacy.load('en')\n"
     ]
    }
   ],
   "source": [
    "!python -m spacy download en"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 3,
   "metadata": {
    "colab": {
     "base_uri": "https://localhost:8080/",
     "height": 88
    },
    "colab_type": "code",
    "id": "sz4gggzKTkYx",
    "outputId": "1b8aa213-261e-46d5-96a6-3ab1d2ac32d2"
   },
   "outputs": [
    {
     "name": "stdout",
     "output_type": "stream",
     "text": [
      "Collecting en_vectors_web_lg==2.1.0 from https://github.com/explosion/spacy-models/releases/download/en_vectors_web_lg-2.1.0/en_vectors_web_lg-2.1.0.tar.gz#egg=en_vectors_web_lg==2.1.0\n",
      "  Downloading https://github.com/explosion/spacy-models/releases/download/en_vectors_web_lg-2.1.0/en_vectors_web_lg-2.1.0.tar.gz (661.8MB)\n",
      "\u001b[K    100% |████████████████████████████████| 661.8MB 7.8MB/s eta 0:00:011.4MB/s eta 0:00:05        | 215.6MB 99.1MB/s eta 0:00:05    38% |████████████▏                   | 251.7MB 101.9MB/s eta 0:00:0539% |████████████▋                   | 261.2MB 100.1MB/s eta 0:00:05 eta 0:00:09/s eta 0:00:05     | 292.8MB 98.7MB/s eta 0:00:04    45% |██████████████▍                 | 298.7MB 64.2MB/s eta 0:00:06    51% |████████████████▍               | 340.0MB 13.4MB/s eta 0:00:24MB 38.6MB/s eta 0:00:09██████████               | 353.4MB 105.6MB/s eta 0:00:03��████▊              | 367.5MB 30.6MB/s eta 0:00:10�███              | 370.9MB 8.4MB/s eta 0:00:35�██████████▏             | 376.2MB 24.4MB/s eta 0:00:12    57% |██████████████████▍             | 379.4MB 15.6MB/s eta 0:00:19| 419.5MB 101.6MB/s eta 0:00:03��████▍           | 421.4MB 27.4MB/s eta 0:00:09    64% |████████████████████▋           | 426.1MB 1.2MB/s eta 0:03:24451.5MB 30.0MB/s eta 0:00:08██████████████████          | 456.7MB 28.1MB/s eta 0:00:08��█████████████▍         | 462.2MB 79.4MB/s eta 0:00:03 | 468.2MB 29.9MB/s eta 0:00:07��███████▊         | 470.2MB 50.2MB/s eta 0:00:04�███████████▉         | 471.7MB 28.0MB/s eta 0:00:07��██████████████         | 476.8MB 73.4MB/s eta 0:00:03:08��████████████▎        | 482.3MB 98.0MB/s eta 0:00:024% |███████████████████████▊        | 489.9MB 22.8MB/s eta 0:00:08��▉        | 493.3MB 31.7MB/s eta 0:00:06██████████████████████       | 519.2MB 91.7MB/s eta 0:00:02��██████████████▋      | 528.8MB 74.7MB/s eta 0:00:02��      | 530.1MB 73.5MB/s eta 0:00:0280% |█████████████████████████▊      | 533.1MB 75.7MB/s eta 0:00:02█████████████████▉      | 534.2MB 21.4MB/s eta 0:00:06█████████████████████▉      | 535.4MB 89.8MB/s eta 0:00:02MB/s eta 0:00:05MB/s eta 0:00:06��█████████████████████▌     | 548.9MB 70.6MB/s eta 0:00:02B/s eta 0:00:02██████▋     | 551.2MB 21.0MB/s eta 0:00:06ta 0:00:06�██████████████████████▋    | 570.0MB 23.3MB/s eta 0:00:04████▋    | 571.3MB 18.2MB/s eta 0:00:05�██▊    | 573.8MB 90.3MB/s eta 0:00:01   87% |████████████████████████████    | 578.6MB 67.7MB/s eta 0:00:02�███████▍   | 587.9MB 21.0MB/s eta 0:00:04    89% |████████████████████████████▋   | 592.4MB 4.3MB/s eta 0:00:17�██████████▊   | 594.1MB 56.1MB/s eta 0:00:02�█▉   | 596.0MB 57.0MB/s eta 0:00:02��███████████████   | 599.6MB 15.5MB/s eta 0:00:05��███████████████████████████   | 601.7MB 63.1MB/s eta 0:00:01�█████████████▏  | 603.3MB 54.8MB/s eta 0:00:02████████████████████████████▏  | 604.4MB 16.9MB/s eta 0:00:04 |█████████████████████████████▍  | 608.3MB 57.1MB/s eta 0:00:01.9MB/s eta 0:00:03B/s eta 0:00:03███████▋  | 611.4MB 14.9MB/s eta 0:00:04��███████▊  | 613.8MB 58.6MB/s eta 0:00:01███████▊  | 615.5MB 14.0MB/s eta 0:00:04    93% |█████████████████████████████▉  | 616.1MB 8.4MB/s eta 0:00:06��██████████████████████████  | 618.6MB 14.4MB/s eta 0:00:03    93% |██████████████████████████████  | 620.1MB 14.3MB/s eta 0:00:03███████████████████████████  | 620.8MB 19.6MB/s eta 0:00:03��██████████████████████████  | 622.3MB 14.3MB/s eta 0:00:03��████████████▏ | 623.6MB 16.6MB/s eta 0:00:03�███████████▍ | 627.9MB 20.2MB/s eta 0:00:02 19.7MB/s eta 0:00:02    95% |██████████████████████████████▌ | 629.7MB 496kB/s eta 0:01:05��█████████████████▋ | 632.8MB 17.0MB/s eta 0:00:02��████████████▋ | 634.2MB 16.4MB/s eta 0:00:02��████████████▊ | 635.8MB 15.5MB/s eta 0:00:02��████████████▉ | 636.7MB 15.7MB/s eta 0:00:02████████████████ | 640.2MB 17.9MB/s eta 0:00:02�████████████ | 641.6MB 100.4MB/s eta 0:00:01��█████████████ | 642.5MB 16.8MB/s eta 0:00:02��█████████████████▎| 645.9MB 103.0MB/s eta 0:00:01�██████████████████▎| 647.4MB 16.7MB/s eta 0:00:01�██████████████████▍| 649.5MB 16.8MB/s eta 0:00:01�█▌| 651.7MB 14.9MB/s eta 0:00:01███████████████████████████▋| 653.4MB 55.9MB/s eta 0:00:01��██████████▋| 654.3MB 58.5MB/s eta 0:00:01�██████████████████▉| 658.2MB 15.5MB/s eta 0:00:01��███████████| 660.2MB 16.9MB/s eta 0:00:01\n",
      "\u001b[?25hInstalling collected packages: en-vectors-web-lg\n",
      "  Running setup.py install for en-vectors-web-lg ... \u001b[?25ldone\n",
      "\u001b[?25hSuccessfully installed en-vectors-web-lg-2.1.0\n",
      "\u001b[33mYou are using pip version 9.0.3, however version 19.2.3 is available.\n",
      "You should consider upgrading via the 'pip install --upgrade pip' command.\u001b[0m\n",
      "\u001b[38;5;2m✔ Download and installation successful\u001b[0m\n",
      "You can now load the model via spacy.load('en_vectors_web_lg')\n"
     ]
    }
   ],
   "source": [
    "!python -m spacy download en_vectors_web_lg"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 1,
   "metadata": {
    "colab": {
     "base_uri": "https://localhost:8080/",
     "height": 34
    },
    "colab_type": "code",
    "id": "xCfnNf5fRiq0",
    "outputId": "32218af6-78f9-42f1-fcb4-d46673e47c1d"
   },
   "outputs": [
    {
     "ename": "OSError",
     "evalue": "[E050] Can't find model 'en_vectors_web_lg'. It doesn't seem to be a shortcut link, a Python package or a valid path to a data directory.",
     "output_type": "error",
     "traceback": [
      "\u001b[0;31m---------------------------------------------------------------------------\u001b[0m",
      "\u001b[0;31mOSError\u001b[0m                                   Traceback (most recent call last)",
      "\u001b[0;32m<ipython-input-1-9d0bfbece33a>\u001b[0m in \u001b[0;36m<module>\u001b[0;34m\u001b[0m\n\u001b[1;32m      1\u001b[0m \u001b[0;32mimport\u001b[0m \u001b[0mspacy\u001b[0m\u001b[0;34m\u001b[0m\u001b[0m\n\u001b[1;32m      2\u001b[0m \u001b[0;34m\u001b[0m\u001b[0m\n\u001b[0;32m----> 3\u001b[0;31m \u001b[0mnlp\u001b[0m \u001b[0;34m=\u001b[0m \u001b[0mspacy\u001b[0m\u001b[0;34m.\u001b[0m\u001b[0mload\u001b[0m\u001b[0;34m(\u001b[0m\u001b[0;34m'en_vectors_web_lg'\u001b[0m\u001b[0;34m)\u001b[0m\u001b[0;34m\u001b[0m\u001b[0m\n\u001b[0m\u001b[1;32m      4\u001b[0m \u001b[0mtotal_vectors\u001b[0m \u001b[0;34m=\u001b[0m \u001b[0mlen\u001b[0m\u001b[0;34m(\u001b[0m\u001b[0mnlp\u001b[0m\u001b[0;34m.\u001b[0m\u001b[0mvocab\u001b[0m\u001b[0;34m.\u001b[0m\u001b[0mvectors\u001b[0m\u001b[0;34m)\u001b[0m\u001b[0;34m\u001b[0m\u001b[0m\n\u001b[1;32m      5\u001b[0m \u001b[0;34m\u001b[0m\u001b[0m\n",
      "\u001b[0;32m/anaconda3/lib/python3.6/site-packages/spacy/__init__.py\u001b[0m in \u001b[0;36mload\u001b[0;34m(name, **overrides)\u001b[0m\n\u001b[1;32m     25\u001b[0m     \u001b[0;32mif\u001b[0m \u001b[0mdepr_path\u001b[0m \u001b[0;32mnot\u001b[0m \u001b[0;32min\u001b[0m \u001b[0;34m(\u001b[0m\u001b[0;32mTrue\u001b[0m\u001b[0;34m,\u001b[0m \u001b[0;32mFalse\u001b[0m\u001b[0;34m,\u001b[0m \u001b[0;32mNone\u001b[0m\u001b[0;34m)\u001b[0m\u001b[0;34m:\u001b[0m\u001b[0;34m\u001b[0m\u001b[0m\n\u001b[1;32m     26\u001b[0m         \u001b[0mdeprecation_warning\u001b[0m\u001b[0;34m(\u001b[0m\u001b[0mWarnings\u001b[0m\u001b[0;34m.\u001b[0m\u001b[0mW001\u001b[0m\u001b[0;34m.\u001b[0m\u001b[0mformat\u001b[0m\u001b[0;34m(\u001b[0m\u001b[0mpath\u001b[0m\u001b[0;34m=\u001b[0m\u001b[0mdepr_path\u001b[0m\u001b[0;34m)\u001b[0m\u001b[0;34m)\u001b[0m\u001b[0;34m\u001b[0m\u001b[0m\n\u001b[0;32m---> 27\u001b[0;31m     \u001b[0;32mreturn\u001b[0m \u001b[0mutil\u001b[0m\u001b[0;34m.\u001b[0m\u001b[0mload_model\u001b[0m\u001b[0;34m(\u001b[0m\u001b[0mname\u001b[0m\u001b[0;34m,\u001b[0m \u001b[0;34m**\u001b[0m\u001b[0moverrides\u001b[0m\u001b[0;34m)\u001b[0m\u001b[0;34m\u001b[0m\u001b[0m\n\u001b[0m\u001b[1;32m     28\u001b[0m \u001b[0;34m\u001b[0m\u001b[0m\n\u001b[1;32m     29\u001b[0m \u001b[0;34m\u001b[0m\u001b[0m\n",
      "\u001b[0;32m/anaconda3/lib/python3.6/site-packages/spacy/util.py\u001b[0m in \u001b[0;36mload_model\u001b[0;34m(name, **overrides)\u001b[0m\n\u001b[1;32m    137\u001b[0m     \u001b[0;32melif\u001b[0m \u001b[0mhasattr\u001b[0m\u001b[0;34m(\u001b[0m\u001b[0mname\u001b[0m\u001b[0;34m,\u001b[0m \u001b[0;34m\"exists\"\u001b[0m\u001b[0;34m)\u001b[0m\u001b[0;34m:\u001b[0m  \u001b[0;31m# Path or Path-like to model data\u001b[0m\u001b[0;34m\u001b[0m\u001b[0m\n\u001b[1;32m    138\u001b[0m         \u001b[0;32mreturn\u001b[0m \u001b[0mload_model_from_path\u001b[0m\u001b[0;34m(\u001b[0m\u001b[0mname\u001b[0m\u001b[0;34m,\u001b[0m \u001b[0;34m**\u001b[0m\u001b[0moverrides\u001b[0m\u001b[0;34m)\u001b[0m\u001b[0;34m\u001b[0m\u001b[0m\n\u001b[0;32m--> 139\u001b[0;31m     \u001b[0;32mraise\u001b[0m \u001b[0mIOError\u001b[0m\u001b[0;34m(\u001b[0m\u001b[0mErrors\u001b[0m\u001b[0;34m.\u001b[0m\u001b[0mE050\u001b[0m\u001b[0;34m.\u001b[0m\u001b[0mformat\u001b[0m\u001b[0;34m(\u001b[0m\u001b[0mname\u001b[0m\u001b[0;34m=\u001b[0m\u001b[0mname\u001b[0m\u001b[0;34m)\u001b[0m\u001b[0;34m)\u001b[0m\u001b[0;34m\u001b[0m\u001b[0m\n\u001b[0m\u001b[1;32m    140\u001b[0m \u001b[0;34m\u001b[0m\u001b[0m\n\u001b[1;32m    141\u001b[0m \u001b[0;34m\u001b[0m\u001b[0m\n",
      "\u001b[0;31mOSError\u001b[0m: [E050] Can't find model 'en_vectors_web_lg'. It doesn't seem to be a shortcut link, a Python package or a valid path to a data directory."
     ]
    }
   ],
   "source": [
    "import spacy\n",
    "\n",
    "nlp = spacy.load('en_vectors_web_lg')\n",
    "total_vectors = len(nlp.vocab.vectors)\n",
    "\n",
    "print('Total word vectors:', total_vectors)"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {
    "colab_type": "text",
    "id": "5Gf4dcBuRiq2"
   },
   "source": [
    "This validates that everything is working and in order. Let’s get the GloVe embeddings for each of our words now in our toy corpus."
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 12,
   "metadata": {
    "colab": {
     "base_uri": "https://localhost:8080/",
     "height": 718
    },
    "colab_type": "code",
    "id": "sd68_DV0Riq3",
    "outputId": "0beef2e4-3e0e-4290-deee-4163bb2681bc"
   },
   "outputs": [
    {
     "data": {
      "text/html": [
       "<div>\n",
       "<style scoped>\n",
       "    .dataframe tbody tr th:only-of-type {\n",
       "        vertical-align: middle;\n",
       "    }\n",
       "\n",
       "    .dataframe tbody tr th {\n",
       "        vertical-align: top;\n",
       "    }\n",
       "\n",
       "    .dataframe thead th {\n",
       "        text-align: right;\n",
       "    }\n",
       "</style>\n",
       "<table border=\"1\" class=\"dataframe\">\n",
       "  <thead>\n",
       "    <tr style=\"text-align: right;\">\n",
       "      <th></th>\n",
       "      <th>0</th>\n",
       "      <th>1</th>\n",
       "      <th>2</th>\n",
       "      <th>3</th>\n",
       "      <th>4</th>\n",
       "      <th>5</th>\n",
       "      <th>6</th>\n",
       "      <th>7</th>\n",
       "      <th>8</th>\n",
       "      <th>9</th>\n",
       "      <th>10</th>\n",
       "      <th>11</th>\n",
       "      <th>12</th>\n",
       "      <th>13</th>\n",
       "      <th>14</th>\n",
       "      <th>15</th>\n",
       "      <th>16</th>\n",
       "      <th>17</th>\n",
       "      <th>18</th>\n",
       "      <th>19</th>\n",
       "      <th>20</th>\n",
       "      <th>21</th>\n",
       "      <th>22</th>\n",
       "      <th>23</th>\n",
       "      <th>24</th>\n",
       "      <th>25</th>\n",
       "      <th>26</th>\n",
       "      <th>27</th>\n",
       "      <th>28</th>\n",
       "      <th>29</th>\n",
       "      <th>30</th>\n",
       "      <th>31</th>\n",
       "      <th>32</th>\n",
       "      <th>33</th>\n",
       "      <th>34</th>\n",
       "      <th>35</th>\n",
       "      <th>36</th>\n",
       "      <th>37</th>\n",
       "      <th>38</th>\n",
       "      <th>39</th>\n",
       "      <th>...</th>\n",
       "      <th>260</th>\n",
       "      <th>261</th>\n",
       "      <th>262</th>\n",
       "      <th>263</th>\n",
       "      <th>264</th>\n",
       "      <th>265</th>\n",
       "      <th>266</th>\n",
       "      <th>267</th>\n",
       "      <th>268</th>\n",
       "      <th>269</th>\n",
       "      <th>270</th>\n",
       "      <th>271</th>\n",
       "      <th>272</th>\n",
       "      <th>273</th>\n",
       "      <th>274</th>\n",
       "      <th>275</th>\n",
       "      <th>276</th>\n",
       "      <th>277</th>\n",
       "      <th>278</th>\n",
       "      <th>279</th>\n",
       "      <th>280</th>\n",
       "      <th>281</th>\n",
       "      <th>282</th>\n",
       "      <th>283</th>\n",
       "      <th>284</th>\n",
       "      <th>285</th>\n",
       "      <th>286</th>\n",
       "      <th>287</th>\n",
       "      <th>288</th>\n",
       "      <th>289</th>\n",
       "      <th>290</th>\n",
       "      <th>291</th>\n",
       "      <th>292</th>\n",
       "      <th>293</th>\n",
       "      <th>294</th>\n",
       "      <th>295</th>\n",
       "      <th>296</th>\n",
       "      <th>297</th>\n",
       "      <th>298</th>\n",
       "      <th>299</th>\n",
       "    </tr>\n",
       "  </thead>\n",
       "  <tbody>\n",
       "    <tr>\n",
       "      <th>sausages</th>\n",
       "      <td>-0.174290</td>\n",
       "      <td>-0.064869</td>\n",
       "      <td>-0.046976</td>\n",
       "      <td>0.287420</td>\n",
       "      <td>-0.128150</td>\n",
       "      <td>0.647630</td>\n",
       "      <td>0.056315</td>\n",
       "      <td>-0.240440</td>\n",
       "      <td>-0.025094</td>\n",
       "      <td>0.502220</td>\n",
       "      <td>0.029936</td>\n",
       "      <td>0.871290</td>\n",
       "      <td>-0.270070</td>\n",
       "      <td>-0.854380</td>\n",
       "      <td>0.241580</td>\n",
       "      <td>-0.278750</td>\n",
       "      <td>0.479100</td>\n",
       "      <td>0.132680</td>\n",
       "      <td>0.284740</td>\n",
       "      <td>0.323430</td>\n",
       "      <td>-0.171420</td>\n",
       "      <td>0.324220</td>\n",
       "      <td>0.137840</td>\n",
       "      <td>-0.357310</td>\n",
       "      <td>0.288720</td>\n",
       "      <td>-0.357420</td>\n",
       "      <td>-0.602930</td>\n",
       "      <td>-0.186040</td>\n",
       "      <td>-0.271860</td>\n",
       "      <td>-0.829750</td>\n",
       "      <td>0.270550</td>\n",
       "      <td>0.225760</td>\n",
       "      <td>0.352370</td>\n",
       "      <td>-0.196700</td>\n",
       "      <td>1.051100</td>\n",
       "      <td>0.210840</td>\n",
       "      <td>0.115040</td>\n",
       "      <td>0.283120</td>\n",
       "      <td>0.700670</td>\n",
       "      <td>1.452400</td>\n",
       "      <td>...</td>\n",
       "      <td>0.145720</td>\n",
       "      <td>-0.327310</td>\n",
       "      <td>-0.594870</td>\n",
       "      <td>0.679110</td>\n",
       "      <td>0.213380</td>\n",
       "      <td>1.053200</td>\n",
       "      <td>-0.032428</td>\n",
       "      <td>0.105010</td>\n",
       "      <td>-0.622310</td>\n",
       "      <td>-0.151530</td>\n",
       "      <td>0.418720</td>\n",
       "      <td>-0.380260</td>\n",
       "      <td>0.401980</td>\n",
       "      <td>-0.007394</td>\n",
       "      <td>-0.399710</td>\n",
       "      <td>0.896790</td>\n",
       "      <td>-0.081613</td>\n",
       "      <td>-0.019213</td>\n",
       "      <td>0.360840</td>\n",
       "      <td>0.013679</td>\n",
       "      <td>0.437120</td>\n",
       "      <td>-0.148730</td>\n",
       "      <td>-0.078819</td>\n",
       "      <td>-0.435240</td>\n",
       "      <td>0.249950</td>\n",
       "      <td>0.663400</td>\n",
       "      <td>0.297820</td>\n",
       "      <td>-0.332740</td>\n",
       "      <td>-0.254690</td>\n",
       "      <td>-0.018595</td>\n",
       "      <td>0.302240</td>\n",
       "      <td>0.195470</td>\n",
       "      <td>-0.653980</td>\n",
       "      <td>-0.291150</td>\n",
       "      <td>-0.684290</td>\n",
       "      <td>-0.266370</td>\n",
       "      <td>0.304310</td>\n",
       "      <td>-0.806830</td>\n",
       "      <td>0.619540</td>\n",
       "      <td>0.201200</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>lazy</th>\n",
       "      <td>-0.353320</td>\n",
       "      <td>-0.299710</td>\n",
       "      <td>-0.176230</td>\n",
       "      <td>-0.321940</td>\n",
       "      <td>-0.385640</td>\n",
       "      <td>0.586110</td>\n",
       "      <td>0.411160</td>\n",
       "      <td>-0.418680</td>\n",
       "      <td>0.073093</td>\n",
       "      <td>1.486500</td>\n",
       "      <td>0.178510</td>\n",
       "      <td>-0.364790</td>\n",
       "      <td>-0.318890</td>\n",
       "      <td>-0.186660</td>\n",
       "      <td>-0.176970</td>\n",
       "      <td>-0.185470</td>\n",
       "      <td>-0.287790</td>\n",
       "      <td>0.579630</td>\n",
       "      <td>0.413430</td>\n",
       "      <td>-0.142170</td>\n",
       "      <td>-0.122250</td>\n",
       "      <td>0.093341</td>\n",
       "      <td>-0.117550</td>\n",
       "      <td>-0.436210</td>\n",
       "      <td>0.416610</td>\n",
       "      <td>-0.172810</td>\n",
       "      <td>0.080734</td>\n",
       "      <td>-0.534250</td>\n",
       "      <td>0.790970</td>\n",
       "      <td>-0.175410</td>\n",
       "      <td>-0.503800</td>\n",
       "      <td>0.092905</td>\n",
       "      <td>0.000162</td>\n",
       "      <td>-0.636440</td>\n",
       "      <td>0.064689</td>\n",
       "      <td>0.262830</td>\n",
       "      <td>0.266300</td>\n",
       "      <td>0.183250</td>\n",
       "      <td>-0.397460</td>\n",
       "      <td>0.109490</td>\n",
       "      <td>...</td>\n",
       "      <td>-0.185390</td>\n",
       "      <td>0.121400</td>\n",
       "      <td>-0.077576</td>\n",
       "      <td>-0.188900</td>\n",
       "      <td>0.394630</td>\n",
       "      <td>-0.383130</td>\n",
       "      <td>-0.288160</td>\n",
       "      <td>-0.359380</td>\n",
       "      <td>0.642020</td>\n",
       "      <td>0.084683</td>\n",
       "      <td>-0.288520</td>\n",
       "      <td>-0.443680</td>\n",
       "      <td>-0.141780</td>\n",
       "      <td>-0.207710</td>\n",
       "      <td>0.006349</td>\n",
       "      <td>-0.368950</td>\n",
       "      <td>-0.082058</td>\n",
       "      <td>-0.195290</td>\n",
       "      <td>-0.067179</td>\n",
       "      <td>0.001303</td>\n",
       "      <td>0.468360</td>\n",
       "      <td>-0.175710</td>\n",
       "      <td>0.198760</td>\n",
       "      <td>0.467780</td>\n",
       "      <td>0.593930</td>\n",
       "      <td>-0.335560</td>\n",
       "      <td>-0.072169</td>\n",
       "      <td>-0.369840</td>\n",
       "      <td>0.114630</td>\n",
       "      <td>0.430670</td>\n",
       "      <td>0.402310</td>\n",
       "      <td>-0.038554</td>\n",
       "      <td>-0.288670</td>\n",
       "      <td>-0.244130</td>\n",
       "      <td>0.460990</td>\n",
       "      <td>0.514170</td>\n",
       "      <td>0.136260</td>\n",
       "      <td>0.344190</td>\n",
       "      <td>-0.845300</td>\n",
       "      <td>-0.077383</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>beautiful</th>\n",
       "      <td>0.171200</td>\n",
       "      <td>0.534390</td>\n",
       "      <td>-0.348540</td>\n",
       "      <td>-0.097234</td>\n",
       "      <td>0.101800</td>\n",
       "      <td>-0.170860</td>\n",
       "      <td>0.295650</td>\n",
       "      <td>-0.041816</td>\n",
       "      <td>-0.516550</td>\n",
       "      <td>2.117200</td>\n",
       "      <td>0.179450</td>\n",
       "      <td>0.264130</td>\n",
       "      <td>-0.231180</td>\n",
       "      <td>-0.328040</td>\n",
       "      <td>-0.219140</td>\n",
       "      <td>-0.043589</td>\n",
       "      <td>-0.492380</td>\n",
       "      <td>1.642500</td>\n",
       "      <td>0.243910</td>\n",
       "      <td>-0.196320</td>\n",
       "      <td>-0.257400</td>\n",
       "      <td>0.071950</td>\n",
       "      <td>-0.027151</td>\n",
       "      <td>-0.065426</td>\n",
       "      <td>-0.224650</td>\n",
       "      <td>0.159030</td>\n",
       "      <td>0.545230</td>\n",
       "      <td>-0.178430</td>\n",
       "      <td>-0.233200</td>\n",
       "      <td>0.475470</td>\n",
       "      <td>-0.073868</td>\n",
       "      <td>0.261460</td>\n",
       "      <td>-0.195230</td>\n",
       "      <td>-0.084660</td>\n",
       "      <td>0.128970</td>\n",
       "      <td>-0.460530</td>\n",
       "      <td>0.164740</td>\n",
       "      <td>-0.450340</td>\n",
       "      <td>-0.340450</td>\n",
       "      <td>-0.214240</td>\n",
       "      <td>...</td>\n",
       "      <td>0.527940</td>\n",
       "      <td>0.122210</td>\n",
       "      <td>0.367880</td>\n",
       "      <td>0.142500</td>\n",
       "      <td>0.039286</td>\n",
       "      <td>-0.418430</td>\n",
       "      <td>0.020668</td>\n",
       "      <td>-0.153280</td>\n",
       "      <td>-0.852450</td>\n",
       "      <td>0.400710</td>\n",
       "      <td>-1.663700</td>\n",
       "      <td>-0.472610</td>\n",
       "      <td>-0.488350</td>\n",
       "      <td>-0.506310</td>\n",
       "      <td>-0.205750</td>\n",
       "      <td>0.100800</td>\n",
       "      <td>0.075323</td>\n",
       "      <td>-0.022721</td>\n",
       "      <td>0.207690</td>\n",
       "      <td>-0.112650</td>\n",
       "      <td>0.589970</td>\n",
       "      <td>-0.206480</td>\n",
       "      <td>-0.314750</td>\n",
       "      <td>-0.220790</td>\n",
       "      <td>-0.126200</td>\n",
       "      <td>-0.028778</td>\n",
       "      <td>-0.176830</td>\n",
       "      <td>0.180990</td>\n",
       "      <td>0.232540</td>\n",
       "      <td>-0.199140</td>\n",
       "      <td>-0.285540</td>\n",
       "      <td>0.104670</td>\n",
       "      <td>0.126310</td>\n",
       "      <td>0.120040</td>\n",
       "      <td>0.254380</td>\n",
       "      <td>0.247400</td>\n",
       "      <td>0.207670</td>\n",
       "      <td>0.172580</td>\n",
       "      <td>0.063875</td>\n",
       "      <td>0.350990</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>blue</th>\n",
       "      <td>0.129450</td>\n",
       "      <td>0.036518</td>\n",
       "      <td>0.032298</td>\n",
       "      <td>-0.060034</td>\n",
       "      <td>0.399840</td>\n",
       "      <td>-0.103020</td>\n",
       "      <td>-0.507880</td>\n",
       "      <td>0.076630</td>\n",
       "      <td>-0.422920</td>\n",
       "      <td>0.815730</td>\n",
       "      <td>-0.182540</td>\n",
       "      <td>-0.072278</td>\n",
       "      <td>-0.358110</td>\n",
       "      <td>-0.428660</td>\n",
       "      <td>0.108800</td>\n",
       "      <td>0.151030</td>\n",
       "      <td>0.240360</td>\n",
       "      <td>2.115100</td>\n",
       "      <td>-0.116390</td>\n",
       "      <td>-1.058300</td>\n",
       "      <td>-0.053679</td>\n",
       "      <td>-0.340660</td>\n",
       "      <td>-0.387930</td>\n",
       "      <td>0.220610</td>\n",
       "      <td>-0.345300</td>\n",
       "      <td>-0.294830</td>\n",
       "      <td>-0.377170</td>\n",
       "      <td>0.040785</td>\n",
       "      <td>0.155960</td>\n",
       "      <td>0.378200</td>\n",
       "      <td>-0.230140</td>\n",
       "      <td>0.106140</td>\n",
       "      <td>-0.083818</td>\n",
       "      <td>-0.121110</td>\n",
       "      <td>-0.498090</td>\n",
       "      <td>-0.311230</td>\n",
       "      <td>0.168420</td>\n",
       "      <td>0.005281</td>\n",
       "      <td>-0.281660</td>\n",
       "      <td>-0.335470</td>\n",
       "      <td>...</td>\n",
       "      <td>0.158920</td>\n",
       "      <td>0.486020</td>\n",
       "      <td>-0.087139</td>\n",
       "      <td>0.601080</td>\n",
       "      <td>-0.000770</td>\n",
       "      <td>-0.082692</td>\n",
       "      <td>-0.120790</td>\n",
       "      <td>0.113970</td>\n",
       "      <td>-0.492130</td>\n",
       "      <td>-0.134840</td>\n",
       "      <td>-0.493600</td>\n",
       "      <td>0.351470</td>\n",
       "      <td>-0.019813</td>\n",
       "      <td>-0.075597</td>\n",
       "      <td>0.369570</td>\n",
       "      <td>-0.309500</td>\n",
       "      <td>-0.218600</td>\n",
       "      <td>0.335340</td>\n",
       "      <td>0.106540</td>\n",
       "      <td>-0.123870</td>\n",
       "      <td>-0.140150</td>\n",
       "      <td>-0.312630</td>\n",
       "      <td>-0.524710</td>\n",
       "      <td>-0.285340</td>\n",
       "      <td>-0.277620</td>\n",
       "      <td>0.080487</td>\n",
       "      <td>0.060031</td>\n",
       "      <td>0.539010</td>\n",
       "      <td>0.554740</td>\n",
       "      <td>-0.147810</td>\n",
       "      <td>-0.501280</td>\n",
       "      <td>0.169010</td>\n",
       "      <td>0.548250</td>\n",
       "      <td>-0.319380</td>\n",
       "      <td>-0.072887</td>\n",
       "      <td>0.382950</td>\n",
       "      <td>0.237410</td>\n",
       "      <td>0.052289</td>\n",
       "      <td>0.182060</td>\n",
       "      <td>0.412640</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>fox</th>\n",
       "      <td>-0.348680</td>\n",
       "      <td>-0.077720</td>\n",
       "      <td>0.177750</td>\n",
       "      <td>-0.094953</td>\n",
       "      <td>-0.452890</td>\n",
       "      <td>0.237790</td>\n",
       "      <td>0.209440</td>\n",
       "      <td>0.037886</td>\n",
       "      <td>0.035064</td>\n",
       "      <td>0.899010</td>\n",
       "      <td>-0.517950</td>\n",
       "      <td>-0.547580</td>\n",
       "      <td>-0.673110</td>\n",
       "      <td>-0.131650</td>\n",
       "      <td>-0.585420</td>\n",
       "      <td>0.312030</td>\n",
       "      <td>-0.094121</td>\n",
       "      <td>0.140050</td>\n",
       "      <td>0.028915</td>\n",
       "      <td>-0.242290</td>\n",
       "      <td>-0.091525</td>\n",
       "      <td>0.133210</td>\n",
       "      <td>0.114760</td>\n",
       "      <td>-0.545550</td>\n",
       "      <td>-0.046149</td>\n",
       "      <td>-0.022279</td>\n",
       "      <td>-0.994870</td>\n",
       "      <td>0.240050</td>\n",
       "      <td>-0.113700</td>\n",
       "      <td>-0.117760</td>\n",
       "      <td>-0.478400</td>\n",
       "      <td>0.278980</td>\n",
       "      <td>0.659960</td>\n",
       "      <td>-0.082358</td>\n",
       "      <td>0.075643</td>\n",
       "      <td>-0.308380</td>\n",
       "      <td>0.134520</td>\n",
       "      <td>-0.122870</td>\n",
       "      <td>-0.243410</td>\n",
       "      <td>-0.010298</td>\n",
       "      <td>...</td>\n",
       "      <td>0.520700</td>\n",
       "      <td>0.152850</td>\n",
       "      <td>0.196420</td>\n",
       "      <td>-0.176430</td>\n",
       "      <td>0.433690</td>\n",
       "      <td>-0.492050</td>\n",
       "      <td>0.044667</td>\n",
       "      <td>0.195000</td>\n",
       "      <td>-0.431390</td>\n",
       "      <td>0.167560</td>\n",
       "      <td>-0.896450</td>\n",
       "      <td>0.551010</td>\n",
       "      <td>0.004203</td>\n",
       "      <td>0.328360</td>\n",
       "      <td>0.343580</td>\n",
       "      <td>0.185590</td>\n",
       "      <td>-0.146440</td>\n",
       "      <td>0.156990</td>\n",
       "      <td>-0.507790</td>\n",
       "      <td>0.071018</td>\n",
       "      <td>0.156870</td>\n",
       "      <td>0.230150</td>\n",
       "      <td>0.087828</td>\n",
       "      <td>0.270050</td>\n",
       "      <td>0.266410</td>\n",
       "      <td>0.291140</td>\n",
       "      <td>-0.132860</td>\n",
       "      <td>-0.254160</td>\n",
       "      <td>-0.095441</td>\n",
       "      <td>-0.174040</td>\n",
       "      <td>-0.283050</td>\n",
       "      <td>0.270240</td>\n",
       "      <td>-0.654800</td>\n",
       "      <td>0.105300</td>\n",
       "      <td>-0.068738</td>\n",
       "      <td>-0.534750</td>\n",
       "      <td>0.061783</td>\n",
       "      <td>0.123610</td>\n",
       "      <td>-0.553700</td>\n",
       "      <td>-0.544790</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>jumps</th>\n",
       "      <td>-0.334840</td>\n",
       "      <td>0.215990</td>\n",
       "      <td>-0.350440</td>\n",
       "      <td>-0.260020</td>\n",
       "      <td>0.411070</td>\n",
       "      <td>0.154010</td>\n",
       "      <td>-0.386110</td>\n",
       "      <td>0.206380</td>\n",
       "      <td>0.386700</td>\n",
       "      <td>1.460500</td>\n",
       "      <td>0.355940</td>\n",
       "      <td>-0.057770</td>\n",
       "      <td>-0.310030</td>\n",
       "      <td>0.533790</td>\n",
       "      <td>-0.016381</td>\n",
       "      <td>0.366800</td>\n",
       "      <td>0.360580</td>\n",
       "      <td>0.362340</td>\n",
       "      <td>-0.101930</td>\n",
       "      <td>-0.365060</td>\n",
       "      <td>-0.546650</td>\n",
       "      <td>0.563370</td>\n",
       "      <td>0.229860</td>\n",
       "      <td>-0.757940</td>\n",
       "      <td>0.320240</td>\n",
       "      <td>0.132530</td>\n",
       "      <td>0.343050</td>\n",
       "      <td>-0.240840</td>\n",
       "      <td>0.044138</td>\n",
       "      <td>-0.535110</td>\n",
       "      <td>-0.257930</td>\n",
       "      <td>0.346190</td>\n",
       "      <td>-0.071962</td>\n",
       "      <td>-0.213440</td>\n",
       "      <td>0.332550</td>\n",
       "      <td>-0.404350</td>\n",
       "      <td>0.107330</td>\n",
       "      <td>-0.027624</td>\n",
       "      <td>0.289630</td>\n",
       "      <td>-0.225150</td>\n",
       "      <td>...</td>\n",
       "      <td>0.134910</td>\n",
       "      <td>0.343390</td>\n",
       "      <td>0.048773</td>\n",
       "      <td>-0.517680</td>\n",
       "      <td>0.212040</td>\n",
       "      <td>0.051897</td>\n",
       "      <td>0.142010</td>\n",
       "      <td>0.225710</td>\n",
       "      <td>0.352020</td>\n",
       "      <td>0.398190</td>\n",
       "      <td>-0.202120</td>\n",
       "      <td>-0.097922</td>\n",
       "      <td>-0.024712</td>\n",
       "      <td>-0.100950</td>\n",
       "      <td>0.000120</td>\n",
       "      <td>-0.443560</td>\n",
       "      <td>0.097503</td>\n",
       "      <td>-0.128980</td>\n",
       "      <td>0.123670</td>\n",
       "      <td>-0.291170</td>\n",
       "      <td>-0.302130</td>\n",
       "      <td>-0.337530</td>\n",
       "      <td>-0.372290</td>\n",
       "      <td>0.201640</td>\n",
       "      <td>0.419570</td>\n",
       "      <td>0.020038</td>\n",
       "      <td>-0.171160</td>\n",
       "      <td>0.433560</td>\n",
       "      <td>0.179710</td>\n",
       "      <td>0.089282</td>\n",
       "      <td>-0.107030</td>\n",
       "      <td>-0.279480</td>\n",
       "      <td>-0.186200</td>\n",
       "      <td>-0.543140</td>\n",
       "      <td>-0.479980</td>\n",
       "      <td>-0.284680</td>\n",
       "      <td>0.036022</td>\n",
       "      <td>0.190290</td>\n",
       "      <td>0.692290</td>\n",
       "      <td>-0.071501</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>dog</th>\n",
       "      <td>-0.057120</td>\n",
       "      <td>0.052685</td>\n",
       "      <td>0.003026</td>\n",
       "      <td>-0.048517</td>\n",
       "      <td>0.007043</td>\n",
       "      <td>0.041856</td>\n",
       "      <td>-0.024704</td>\n",
       "      <td>-0.039783</td>\n",
       "      <td>0.009614</td>\n",
       "      <td>0.308416</td>\n",
       "      <td>-0.089130</td>\n",
       "      <td>0.041381</td>\n",
       "      <td>-0.095640</td>\n",
       "      <td>0.033153</td>\n",
       "      <td>-0.048714</td>\n",
       "      <td>0.026033</td>\n",
       "      <td>0.071408</td>\n",
       "      <td>0.151969</td>\n",
       "      <td>0.020897</td>\n",
       "      <td>-0.064305</td>\n",
       "      <td>-0.059467</td>\n",
       "      <td>-0.022701</td>\n",
       "      <td>0.038028</td>\n",
       "      <td>-0.069476</td>\n",
       "      <td>0.051839</td>\n",
       "      <td>-0.006171</td>\n",
       "      <td>-0.034795</td>\n",
       "      <td>-0.059360</td>\n",
       "      <td>0.012666</td>\n",
       "      <td>-0.036328</td>\n",
       "      <td>-0.079183</td>\n",
       "      <td>0.017406</td>\n",
       "      <td>-0.011875</td>\n",
       "      <td>0.078330</td>\n",
       "      <td>0.051765</td>\n",
       "      <td>0.021839</td>\n",
       "      <td>0.079245</td>\n",
       "      <td>-0.128954</td>\n",
       "      <td>-0.006980</td>\n",
       "      <td>0.054850</td>\n",
       "      <td>...</td>\n",
       "      <td>0.005369</td>\n",
       "      <td>-0.053469</td>\n",
       "      <td>-0.016607</td>\n",
       "      <td>-0.095284</td>\n",
       "      <td>-0.006578</td>\n",
       "      <td>0.055140</td>\n",
       "      <td>-0.004591</td>\n",
       "      <td>-0.007719</td>\n",
       "      <td>-0.064524</td>\n",
       "      <td>0.027798</td>\n",
       "      <td>-0.041898</td>\n",
       "      <td>0.120860</td>\n",
       "      <td>0.014708</td>\n",
       "      <td>0.013792</td>\n",
       "      <td>0.016121</td>\n",
       "      <td>0.056161</td>\n",
       "      <td>0.008394</td>\n",
       "      <td>0.031247</td>\n",
       "      <td>0.026793</td>\n",
       "      <td>-0.022593</td>\n",
       "      <td>-0.014645</td>\n",
       "      <td>0.047150</td>\n",
       "      <td>0.008740</td>\n",
       "      <td>-0.042436</td>\n",
       "      <td>0.063281</td>\n",
       "      <td>0.067289</td>\n",
       "      <td>0.037409</td>\n",
       "      <td>-0.026295</td>\n",
       "      <td>0.020831</td>\n",
       "      <td>-0.004480</td>\n",
       "      <td>0.003257</td>\n",
       "      <td>-0.036864</td>\n",
       "      <td>-0.043878</td>\n",
       "      <td>0.000249</td>\n",
       "      <td>-0.026959</td>\n",
       "      <td>0.077895</td>\n",
       "      <td>0.044350</td>\n",
       "      <td>0.035107</td>\n",
       "      <td>0.042551</td>\n",
       "      <td>-0.010643</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>breakfast</th>\n",
       "      <td>0.073378</td>\n",
       "      <td>0.227670</td>\n",
       "      <td>0.208420</td>\n",
       "      <td>-0.456790</td>\n",
       "      <td>-0.078219</td>\n",
       "      <td>0.601960</td>\n",
       "      <td>-0.024494</td>\n",
       "      <td>-0.467980</td>\n",
       "      <td>0.054627</td>\n",
       "      <td>2.283700</td>\n",
       "      <td>-0.681900</td>\n",
       "      <td>-0.037457</td>\n",
       "      <td>-0.324390</td>\n",
       "      <td>0.442410</td>\n",
       "      <td>0.197600</td>\n",
       "      <td>-0.508460</td>\n",
       "      <td>-0.196110</td>\n",
       "      <td>1.313400</td>\n",
       "      <td>0.190610</td>\n",
       "      <td>-0.050723</td>\n",
       "      <td>-0.458790</td>\n",
       "      <td>0.169650</td>\n",
       "      <td>-0.265080</td>\n",
       "      <td>-0.439940</td>\n",
       "      <td>0.294480</td>\n",
       "      <td>-0.395270</td>\n",
       "      <td>-0.476240</td>\n",
       "      <td>-0.436840</td>\n",
       "      <td>-0.561440</td>\n",
       "      <td>-0.191850</td>\n",
       "      <td>0.183180</td>\n",
       "      <td>0.157190</td>\n",
       "      <td>0.281240</td>\n",
       "      <td>0.033112</td>\n",
       "      <td>0.064599</td>\n",
       "      <td>0.271460</td>\n",
       "      <td>-0.337520</td>\n",
       "      <td>-0.229610</td>\n",
       "      <td>-0.539340</td>\n",
       "      <td>0.577800</td>\n",
       "      <td>...</td>\n",
       "      <td>-0.199320</td>\n",
       "      <td>-0.344430</td>\n",
       "      <td>0.108090</td>\n",
       "      <td>-0.078482</td>\n",
       "      <td>-0.140850</td>\n",
       "      <td>0.065754</td>\n",
       "      <td>-0.335080</td>\n",
       "      <td>0.058111</td>\n",
       "      <td>-1.530400</td>\n",
       "      <td>0.123050</td>\n",
       "      <td>-0.637070</td>\n",
       "      <td>-0.152480</td>\n",
       "      <td>0.308960</td>\n",
       "      <td>-0.317620</td>\n",
       "      <td>-0.101160</td>\n",
       "      <td>0.791200</td>\n",
       "      <td>0.111700</td>\n",
       "      <td>-0.449060</td>\n",
       "      <td>0.336830</td>\n",
       "      <td>0.906860</td>\n",
       "      <td>0.409920</td>\n",
       "      <td>-0.610050</td>\n",
       "      <td>-0.246430</td>\n",
       "      <td>0.067408</td>\n",
       "      <td>0.344840</td>\n",
       "      <td>0.285870</td>\n",
       "      <td>0.543030</td>\n",
       "      <td>0.127340</td>\n",
       "      <td>-0.486970</td>\n",
       "      <td>0.968110</td>\n",
       "      <td>0.647710</td>\n",
       "      <td>0.373820</td>\n",
       "      <td>0.019931</td>\n",
       "      <td>-0.033672</td>\n",
       "      <td>-0.073184</td>\n",
       "      <td>0.296830</td>\n",
       "      <td>0.340420</td>\n",
       "      <td>-0.599390</td>\n",
       "      <td>-0.061114</td>\n",
       "      <td>0.232200</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>today</th>\n",
       "      <td>-0.156570</td>\n",
       "      <td>0.594890</td>\n",
       "      <td>-0.031445</td>\n",
       "      <td>-0.077586</td>\n",
       "      <td>0.278630</td>\n",
       "      <td>-0.509210</td>\n",
       "      <td>-0.066350</td>\n",
       "      <td>-0.081890</td>\n",
       "      <td>-0.047986</td>\n",
       "      <td>2.803600</td>\n",
       "      <td>-0.131820</td>\n",
       "      <td>-0.627100</td>\n",
       "      <td>0.253880</td>\n",
       "      <td>-0.472910</td>\n",
       "      <td>-0.428750</td>\n",
       "      <td>0.143520</td>\n",
       "      <td>-0.118180</td>\n",
       "      <td>0.679740</td>\n",
       "      <td>-0.006229</td>\n",
       "      <td>-0.001527</td>\n",
       "      <td>-0.163790</td>\n",
       "      <td>-0.034067</td>\n",
       "      <td>0.271020</td>\n",
       "      <td>0.054245</td>\n",
       "      <td>-0.011752</td>\n",
       "      <td>0.324470</td>\n",
       "      <td>-0.321400</td>\n",
       "      <td>-0.003704</td>\n",
       "      <td>-0.205390</td>\n",
       "      <td>0.057008</td>\n",
       "      <td>0.177460</td>\n",
       "      <td>-0.166030</td>\n",
       "      <td>0.505320</td>\n",
       "      <td>-0.215200</td>\n",
       "      <td>0.010503</td>\n",
       "      <td>0.598390</td>\n",
       "      <td>0.021347</td>\n",
       "      <td>0.103650</td>\n",
       "      <td>0.023387</td>\n",
       "      <td>0.097183</td>\n",
       "      <td>...</td>\n",
       "      <td>0.553160</td>\n",
       "      <td>0.100970</td>\n",
       "      <td>-0.386630</td>\n",
       "      <td>0.012178</td>\n",
       "      <td>-0.181380</td>\n",
       "      <td>-0.381290</td>\n",
       "      <td>-0.200480</td>\n",
       "      <td>-0.347470</td>\n",
       "      <td>0.702780</td>\n",
       "      <td>0.299910</td>\n",
       "      <td>-0.198150</td>\n",
       "      <td>0.041330</td>\n",
       "      <td>-0.169990</td>\n",
       "      <td>0.346500</td>\n",
       "      <td>-0.135520</td>\n",
       "      <td>0.162350</td>\n",
       "      <td>-0.145150</td>\n",
       "      <td>0.289770</td>\n",
       "      <td>0.095377</td>\n",
       "      <td>-0.113840</td>\n",
       "      <td>0.372630</td>\n",
       "      <td>-0.116170</td>\n",
       "      <td>0.091593</td>\n",
       "      <td>0.126660</td>\n",
       "      <td>-0.035243</td>\n",
       "      <td>0.261140</td>\n",
       "      <td>0.235310</td>\n",
       "      <td>-0.323920</td>\n",
       "      <td>-0.338050</td>\n",
       "      <td>0.134510</td>\n",
       "      <td>-0.326580</td>\n",
       "      <td>-0.413380</td>\n",
       "      <td>0.367910</td>\n",
       "      <td>-0.262630</td>\n",
       "      <td>-0.203690</td>\n",
       "      <td>-0.296560</td>\n",
       "      <td>-0.014873</td>\n",
       "      <td>-0.250060</td>\n",
       "      <td>-0.115940</td>\n",
       "      <td>0.083741</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>green</th>\n",
       "      <td>-0.072368</td>\n",
       "      <td>0.233200</td>\n",
       "      <td>0.137260</td>\n",
       "      <td>-0.156630</td>\n",
       "      <td>0.248440</td>\n",
       "      <td>0.349870</td>\n",
       "      <td>-0.241700</td>\n",
       "      <td>-0.091426</td>\n",
       "      <td>-0.530150</td>\n",
       "      <td>1.341300</td>\n",
       "      <td>-0.867850</td>\n",
       "      <td>-0.131830</td>\n",
       "      <td>-0.596790</td>\n",
       "      <td>-0.344150</td>\n",
       "      <td>-0.161210</td>\n",
       "      <td>-0.000925</td>\n",
       "      <td>0.532670</td>\n",
       "      <td>2.132900</td>\n",
       "      <td>0.021933</td>\n",
       "      <td>-0.519330</td>\n",
       "      <td>0.365570</td>\n",
       "      <td>-0.012978</td>\n",
       "      <td>-0.271540</td>\n",
       "      <td>0.004896</td>\n",
       "      <td>-0.118490</td>\n",
       "      <td>-0.383380</td>\n",
       "      <td>-0.489440</td>\n",
       "      <td>0.491470</td>\n",
       "      <td>0.136640</td>\n",
       "      <td>-0.096163</td>\n",
       "      <td>-0.028429</td>\n",
       "      <td>0.003963</td>\n",
       "      <td>0.155420</td>\n",
       "      <td>-0.296800</td>\n",
       "      <td>-0.148950</td>\n",
       "      <td>-0.055311</td>\n",
       "      <td>0.300030</td>\n",
       "      <td>0.163760</td>\n",
       "      <td>-0.169410</td>\n",
       "      <td>-0.101660</td>\n",
       "      <td>...</td>\n",
       "      <td>-0.063701</td>\n",
       "      <td>0.268550</td>\n",
       "      <td>0.125370</td>\n",
       "      <td>0.603330</td>\n",
       "      <td>0.340680</td>\n",
       "      <td>-0.364250</td>\n",
       "      <td>-0.353150</td>\n",
       "      <td>-0.432980</td>\n",
       "      <td>-0.420860</td>\n",
       "      <td>0.157040</td>\n",
       "      <td>-0.255520</td>\n",
       "      <td>0.168950</td>\n",
       "      <td>0.079552</td>\n",
       "      <td>-0.315130</td>\n",
       "      <td>0.085769</td>\n",
       "      <td>-0.079049</td>\n",
       "      <td>0.000499</td>\n",
       "      <td>0.415510</td>\n",
       "      <td>0.130620</td>\n",
       "      <td>0.218690</td>\n",
       "      <td>0.170560</td>\n",
       "      <td>-0.236900</td>\n",
       "      <td>-0.390740</td>\n",
       "      <td>0.059123</td>\n",
       "      <td>-0.080229</td>\n",
       "      <td>0.119570</td>\n",
       "      <td>0.372940</td>\n",
       "      <td>0.389800</td>\n",
       "      <td>0.427670</td>\n",
       "      <td>-0.112340</td>\n",
       "      <td>-0.405170</td>\n",
       "      <td>0.243570</td>\n",
       "      <td>0.437300</td>\n",
       "      <td>-0.461520</td>\n",
       "      <td>-0.352710</td>\n",
       "      <td>0.336250</td>\n",
       "      <td>0.069899</td>\n",
       "      <td>-0.111550</td>\n",
       "      <td>0.532930</td>\n",
       "      <td>0.712680</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>kings</th>\n",
       "      <td>0.259230</td>\n",
       "      <td>-0.854690</td>\n",
       "      <td>0.360010</td>\n",
       "      <td>-0.642000</td>\n",
       "      <td>0.568530</td>\n",
       "      <td>-0.321420</td>\n",
       "      <td>0.173250</td>\n",
       "      <td>0.133030</td>\n",
       "      <td>-0.089720</td>\n",
       "      <td>1.528600</td>\n",
       "      <td>-0.289030</td>\n",
       "      <td>-0.658740</td>\n",
       "      <td>0.179310</td>\n",
       "      <td>-0.077445</td>\n",
       "      <td>0.086629</td>\n",
       "      <td>-0.132770</td>\n",
       "      <td>0.189590</td>\n",
       "      <td>0.243270</td>\n",
       "      <td>0.194470</td>\n",
       "      <td>0.256190</td>\n",
       "      <td>0.115750</td>\n",
       "      <td>0.441050</td>\n",
       "      <td>-0.328350</td>\n",
       "      <td>0.151940</td>\n",
       "      <td>-0.868790</td>\n",
       "      <td>0.175560</td>\n",
       "      <td>-0.190540</td>\n",
       "      <td>0.382940</td>\n",
       "      <td>-0.124040</td>\n",
       "      <td>0.784500</td>\n",
       "      <td>0.111830</td>\n",
       "      <td>0.642220</td>\n",
       "      <td>-0.268260</td>\n",
       "      <td>0.053329</td>\n",
       "      <td>-0.055495</td>\n",
       "      <td>0.750070</td>\n",
       "      <td>-0.644130</td>\n",
       "      <td>-0.276970</td>\n",
       "      <td>-0.108700</td>\n",
       "      <td>0.445280</td>\n",
       "      <td>...</td>\n",
       "      <td>0.756250</td>\n",
       "      <td>0.262880</td>\n",
       "      <td>-0.039965</td>\n",
       "      <td>0.037196</td>\n",
       "      <td>-0.183640</td>\n",
       "      <td>-0.360540</td>\n",
       "      <td>0.515670</td>\n",
       "      <td>0.445350</td>\n",
       "      <td>-0.235570</td>\n",
       "      <td>0.452830</td>\n",
       "      <td>-0.297770</td>\n",
       "      <td>0.626260</td>\n",
       "      <td>-0.544090</td>\n",
       "      <td>-0.003886</td>\n",
       "      <td>-0.076988</td>\n",
       "      <td>-0.623640</td>\n",
       "      <td>0.102650</td>\n",
       "      <td>0.178460</td>\n",
       "      <td>-0.097651</td>\n",
       "      <td>-0.290180</td>\n",
       "      <td>0.530120</td>\n",
       "      <td>0.071785</td>\n",
       "      <td>-0.056542</td>\n",
       "      <td>-0.011450</td>\n",
       "      <td>0.053002</td>\n",
       "      <td>0.323720</td>\n",
       "      <td>0.379260</td>\n",
       "      <td>-0.312000</td>\n",
       "      <td>0.007405</td>\n",
       "      <td>-0.572380</td>\n",
       "      <td>-0.470090</td>\n",
       "      <td>0.063743</td>\n",
       "      <td>-0.545210</td>\n",
       "      <td>-0.192310</td>\n",
       "      <td>-0.301020</td>\n",
       "      <td>1.068500</td>\n",
       "      <td>0.231160</td>\n",
       "      <td>-0.147330</td>\n",
       "      <td>0.662490</td>\n",
       "      <td>-0.577420</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>sky</th>\n",
       "      <td>0.312550</td>\n",
       "      <td>-0.303080</td>\n",
       "      <td>0.019587</td>\n",
       "      <td>-0.354940</td>\n",
       "      <td>0.100180</td>\n",
       "      <td>-0.141530</td>\n",
       "      <td>-0.514270</td>\n",
       "      <td>0.886110</td>\n",
       "      <td>-0.530540</td>\n",
       "      <td>1.556600</td>\n",
       "      <td>-0.001385</td>\n",
       "      <td>-0.214260</td>\n",
       "      <td>-0.524280</td>\n",
       "      <td>0.104250</td>\n",
       "      <td>-0.339810</td>\n",
       "      <td>0.186400</td>\n",
       "      <td>0.022981</td>\n",
       "      <td>1.186600</td>\n",
       "      <td>-0.180270</td>\n",
       "      <td>-0.891930</td>\n",
       "      <td>0.084658</td>\n",
       "      <td>0.173110</td>\n",
       "      <td>-0.380410</td>\n",
       "      <td>0.311870</td>\n",
       "      <td>-0.259610</td>\n",
       "      <td>-0.097019</td>\n",
       "      <td>-0.014994</td>\n",
       "      <td>0.051264</td>\n",
       "      <td>0.068153</td>\n",
       "      <td>0.035770</td>\n",
       "      <td>-0.123790</td>\n",
       "      <td>0.201050</td>\n",
       "      <td>-0.246020</td>\n",
       "      <td>-0.070083</td>\n",
       "      <td>-0.265050</td>\n",
       "      <td>-0.012957</td>\n",
       "      <td>-0.184630</td>\n",
       "      <td>-0.129890</td>\n",
       "      <td>0.084696</td>\n",
       "      <td>-0.021024</td>\n",
       "      <td>...</td>\n",
       "      <td>0.263900</td>\n",
       "      <td>0.108530</td>\n",
       "      <td>0.030066</td>\n",
       "      <td>-0.175170</td>\n",
       "      <td>0.155300</td>\n",
       "      <td>-0.880140</td>\n",
       "      <td>-0.195720</td>\n",
       "      <td>0.157680</td>\n",
       "      <td>-0.557130</td>\n",
       "      <td>0.342250</td>\n",
       "      <td>-0.722380</td>\n",
       "      <td>0.326730</td>\n",
       "      <td>-0.051512</td>\n",
       "      <td>0.090080</td>\n",
       "      <td>0.378730</td>\n",
       "      <td>0.013059</td>\n",
       "      <td>-0.239590</td>\n",
       "      <td>-0.368900</td>\n",
       "      <td>0.010188</td>\n",
       "      <td>-0.158160</td>\n",
       "      <td>0.291280</td>\n",
       "      <td>-0.962140</td>\n",
       "      <td>-0.925610</td>\n",
       "      <td>-0.122720</td>\n",
       "      <td>-0.022322</td>\n",
       "      <td>-0.152050</td>\n",
       "      <td>0.595670</td>\n",
       "      <td>0.397470</td>\n",
       "      <td>0.487380</td>\n",
       "      <td>-0.147080</td>\n",
       "      <td>-0.667050</td>\n",
       "      <td>0.279110</td>\n",
       "      <td>0.500970</td>\n",
       "      <td>-0.277580</td>\n",
       "      <td>-0.143720</td>\n",
       "      <td>0.342710</td>\n",
       "      <td>0.287580</td>\n",
       "      <td>0.537740</td>\n",
       "      <td>0.363490</td>\n",
       "      <td>0.496920</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>brown</th>\n",
       "      <td>-0.374120</td>\n",
       "      <td>-0.076264</td>\n",
       "      <td>0.109260</td>\n",
       "      <td>0.186620</td>\n",
       "      <td>0.029943</td>\n",
       "      <td>0.182700</td>\n",
       "      <td>-0.631980</td>\n",
       "      <td>0.133060</td>\n",
       "      <td>-0.128980</td>\n",
       "      <td>0.603430</td>\n",
       "      <td>-0.680410</td>\n",
       "      <td>-0.142220</td>\n",
       "      <td>-0.133570</td>\n",
       "      <td>-0.659430</td>\n",
       "      <td>0.052402</td>\n",
       "      <td>0.167450</td>\n",
       "      <td>0.639230</td>\n",
       "      <td>1.768000</td>\n",
       "      <td>0.346190</td>\n",
       "      <td>-0.624800</td>\n",
       "      <td>-0.128670</td>\n",
       "      <td>-0.197030</td>\n",
       "      <td>-0.374500</td>\n",
       "      <td>0.330640</td>\n",
       "      <td>0.046832</td>\n",
       "      <td>-0.653510</td>\n",
       "      <td>-0.561360</td>\n",
       "      <td>0.227370</td>\n",
       "      <td>0.229230</td>\n",
       "      <td>-0.415810</td>\n",
       "      <td>-0.167670</td>\n",
       "      <td>0.335430</td>\n",
       "      <td>0.097227</td>\n",
       "      <td>-0.467030</td>\n",
       "      <td>-0.026857</td>\n",
       "      <td>-0.067665</td>\n",
       "      <td>-0.192110</td>\n",
       "      <td>-0.133740</td>\n",
       "      <td>0.016268</td>\n",
       "      <td>-0.208270</td>\n",
       "      <td>...</td>\n",
       "      <td>0.117450</td>\n",
       "      <td>0.055016</td>\n",
       "      <td>0.236820</td>\n",
       "      <td>0.898190</td>\n",
       "      <td>-0.409690</td>\n",
       "      <td>0.075218</td>\n",
       "      <td>-0.110300</td>\n",
       "      <td>-0.409910</td>\n",
       "      <td>-0.957180</td>\n",
       "      <td>0.527490</td>\n",
       "      <td>-0.042655</td>\n",
       "      <td>0.266180</td>\n",
       "      <td>0.305260</td>\n",
       "      <td>-0.519010</td>\n",
       "      <td>-0.460440</td>\n",
       "      <td>-0.093834</td>\n",
       "      <td>0.130060</td>\n",
       "      <td>0.019268</td>\n",
       "      <td>0.010198</td>\n",
       "      <td>0.007583</td>\n",
       "      <td>0.295510</td>\n",
       "      <td>0.231560</td>\n",
       "      <td>-0.034880</td>\n",
       "      <td>-0.116940</td>\n",
       "      <td>-0.327330</td>\n",
       "      <td>0.204910</td>\n",
       "      <td>0.475000</td>\n",
       "      <td>0.513100</td>\n",
       "      <td>-0.145760</td>\n",
       "      <td>-0.185080</td>\n",
       "      <td>-0.015404</td>\n",
       "      <td>0.392890</td>\n",
       "      <td>-0.034826</td>\n",
       "      <td>-0.720300</td>\n",
       "      <td>-0.365320</td>\n",
       "      <td>0.740510</td>\n",
       "      <td>0.108390</td>\n",
       "      <td>-0.365760</td>\n",
       "      <td>-0.288190</td>\n",
       "      <td>0.114630</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>bacon</th>\n",
       "      <td>-0.430730</td>\n",
       "      <td>-0.016025</td>\n",
       "      <td>0.484620</td>\n",
       "      <td>0.101390</td>\n",
       "      <td>-0.299200</td>\n",
       "      <td>0.761820</td>\n",
       "      <td>-0.353130</td>\n",
       "      <td>-0.325290</td>\n",
       "      <td>0.156730</td>\n",
       "      <td>0.873210</td>\n",
       "      <td>-0.078280</td>\n",
       "      <td>0.277670</td>\n",
       "      <td>-0.524210</td>\n",
       "      <td>0.103800</td>\n",
       "      <td>-0.018345</td>\n",
       "      <td>-0.397500</td>\n",
       "      <td>0.230810</td>\n",
       "      <td>0.940140</td>\n",
       "      <td>0.275650</td>\n",
       "      <td>0.314490</td>\n",
       "      <td>0.157910</td>\n",
       "      <td>0.219310</td>\n",
       "      <td>-0.328710</td>\n",
       "      <td>-0.187760</td>\n",
       "      <td>-0.209180</td>\n",
       "      <td>-0.528110</td>\n",
       "      <td>-0.425460</td>\n",
       "      <td>0.020278</td>\n",
       "      <td>0.124110</td>\n",
       "      <td>-0.553420</td>\n",
       "      <td>-0.566570</td>\n",
       "      <td>0.768710</td>\n",
       "      <td>0.329540</td>\n",
       "      <td>-0.153310</td>\n",
       "      <td>0.631360</td>\n",
       "      <td>0.108580</td>\n",
       "      <td>-0.126160</td>\n",
       "      <td>0.102500</td>\n",
       "      <td>0.291810</td>\n",
       "      <td>1.129400</td>\n",
       "      <td>...</td>\n",
       "      <td>0.011195</td>\n",
       "      <td>-0.129620</td>\n",
       "      <td>-0.518830</td>\n",
       "      <td>0.015317</td>\n",
       "      <td>0.249340</td>\n",
       "      <td>0.594640</td>\n",
       "      <td>-0.209520</td>\n",
       "      <td>-0.364450</td>\n",
       "      <td>-0.853380</td>\n",
       "      <td>-0.196230</td>\n",
       "      <td>0.425870</td>\n",
       "      <td>-0.343410</td>\n",
       "      <td>0.622890</td>\n",
       "      <td>0.374680</td>\n",
       "      <td>-0.768440</td>\n",
       "      <td>0.632930</td>\n",
       "      <td>0.026989</td>\n",
       "      <td>-0.281060</td>\n",
       "      <td>-0.010866</td>\n",
       "      <td>0.182310</td>\n",
       "      <td>0.437670</td>\n",
       "      <td>-0.278490</td>\n",
       "      <td>0.109020</td>\n",
       "      <td>-0.177370</td>\n",
       "      <td>0.025408</td>\n",
       "      <td>0.480750</td>\n",
       "      <td>0.586230</td>\n",
       "      <td>0.424540</td>\n",
       "      <td>-0.247770</td>\n",
       "      <td>0.165690</td>\n",
       "      <td>0.304240</td>\n",
       "      <td>0.413440</td>\n",
       "      <td>-0.540730</td>\n",
       "      <td>-0.035930</td>\n",
       "      <td>-0.429450</td>\n",
       "      <td>-0.246590</td>\n",
       "      <td>0.161490</td>\n",
       "      <td>-1.065400</td>\n",
       "      <td>-0.244940</td>\n",
       "      <td>0.269540</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>eggs</th>\n",
       "      <td>-0.417810</td>\n",
       "      <td>-0.035192</td>\n",
       "      <td>-0.126150</td>\n",
       "      <td>-0.215930</td>\n",
       "      <td>-0.669740</td>\n",
       "      <td>0.513250</td>\n",
       "      <td>-0.797090</td>\n",
       "      <td>-0.068611</td>\n",
       "      <td>0.634660</td>\n",
       "      <td>1.256300</td>\n",
       "      <td>-0.809100</td>\n",
       "      <td>0.012227</td>\n",
       "      <td>-0.261690</td>\n",
       "      <td>0.260560</td>\n",
       "      <td>0.260520</td>\n",
       "      <td>-0.185840</td>\n",
       "      <td>-0.285730</td>\n",
       "      <td>1.359900</td>\n",
       "      <td>-0.115380</td>\n",
       "      <td>-0.019430</td>\n",
       "      <td>-0.033052</td>\n",
       "      <td>0.386340</td>\n",
       "      <td>-0.520640</td>\n",
       "      <td>-0.325880</td>\n",
       "      <td>-0.255210</td>\n",
       "      <td>-0.614430</td>\n",
       "      <td>-0.366000</td>\n",
       "      <td>0.133900</td>\n",
       "      <td>0.049880</td>\n",
       "      <td>-1.093600</td>\n",
       "      <td>-0.436250</td>\n",
       "      <td>-0.279800</td>\n",
       "      <td>0.525020</td>\n",
       "      <td>-0.450040</td>\n",
       "      <td>0.196330</td>\n",
       "      <td>0.611860</td>\n",
       "      <td>0.004073</td>\n",
       "      <td>-0.381950</td>\n",
       "      <td>0.357830</td>\n",
       "      <td>0.753130</td>\n",
       "      <td>...</td>\n",
       "      <td>-0.016694</td>\n",
       "      <td>0.168550</td>\n",
       "      <td>-0.135030</td>\n",
       "      <td>0.072960</td>\n",
       "      <td>-0.065344</td>\n",
       "      <td>0.768680</td>\n",
       "      <td>-0.194850</td>\n",
       "      <td>-0.313310</td>\n",
       "      <td>-0.552650</td>\n",
       "      <td>-0.488490</td>\n",
       "      <td>0.470120</td>\n",
       "      <td>-0.396550</td>\n",
       "      <td>0.688490</td>\n",
       "      <td>-0.017080</td>\n",
       "      <td>-1.037700</td>\n",
       "      <td>0.081078</td>\n",
       "      <td>-0.414330</td>\n",
       "      <td>0.188600</td>\n",
       "      <td>0.216510</td>\n",
       "      <td>0.603600</td>\n",
       "      <td>0.943960</td>\n",
       "      <td>-0.288970</td>\n",
       "      <td>-0.023022</td>\n",
       "      <td>-0.616510</td>\n",
       "      <td>-0.180130</td>\n",
       "      <td>0.405020</td>\n",
       "      <td>0.043301</td>\n",
       "      <td>-0.438280</td>\n",
       "      <td>-0.684250</td>\n",
       "      <td>0.209900</td>\n",
       "      <td>-0.232860</td>\n",
       "      <td>-0.139740</td>\n",
       "      <td>-0.681080</td>\n",
       "      <td>-0.370920</td>\n",
       "      <td>-0.545510</td>\n",
       "      <td>0.073728</td>\n",
       "      <td>0.111620</td>\n",
       "      <td>-0.324700</td>\n",
       "      <td>0.059721</td>\n",
       "      <td>0.159160</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>beans</th>\n",
       "      <td>-0.423290</td>\n",
       "      <td>-0.264500</td>\n",
       "      <td>0.200870</td>\n",
       "      <td>0.082187</td>\n",
       "      <td>0.066944</td>\n",
       "      <td>1.027600</td>\n",
       "      <td>-0.989140</td>\n",
       "      <td>-0.259950</td>\n",
       "      <td>0.145960</td>\n",
       "      <td>0.766450</td>\n",
       "      <td>-0.581010</td>\n",
       "      <td>0.317560</td>\n",
       "      <td>-0.043719</td>\n",
       "      <td>-0.847330</td>\n",
       "      <td>0.188070</td>\n",
       "      <td>-0.716880</td>\n",
       "      <td>0.009522</td>\n",
       "      <td>1.124100</td>\n",
       "      <td>0.437220</td>\n",
       "      <td>0.371400</td>\n",
       "      <td>-0.208100</td>\n",
       "      <td>-0.380830</td>\n",
       "      <td>-0.282530</td>\n",
       "      <td>0.159310</td>\n",
       "      <td>-0.096079</td>\n",
       "      <td>-0.959680</td>\n",
       "      <td>-0.222080</td>\n",
       "      <td>0.299910</td>\n",
       "      <td>0.163690</td>\n",
       "      <td>-0.710410</td>\n",
       "      <td>0.242250</td>\n",
       "      <td>-0.072045</td>\n",
       "      <td>0.802380</td>\n",
       "      <td>-0.477040</td>\n",
       "      <td>-0.313830</td>\n",
       "      <td>0.332220</td>\n",
       "      <td>0.277570</td>\n",
       "      <td>0.260040</td>\n",
       "      <td>0.278750</td>\n",
       "      <td>0.600180</td>\n",
       "      <td>...</td>\n",
       "      <td>-0.157950</td>\n",
       "      <td>-0.233430</td>\n",
       "      <td>0.009833</td>\n",
       "      <td>0.106710</td>\n",
       "      <td>-0.324040</td>\n",
       "      <td>0.290390</td>\n",
       "      <td>-0.099283</td>\n",
       "      <td>-0.294040</td>\n",
       "      <td>-0.564550</td>\n",
       "      <td>0.230860</td>\n",
       "      <td>0.647260</td>\n",
       "      <td>-0.244820</td>\n",
       "      <td>0.223060</td>\n",
       "      <td>-0.353680</td>\n",
       "      <td>-0.633440</td>\n",
       "      <td>0.286330</td>\n",
       "      <td>-0.448500</td>\n",
       "      <td>0.002610</td>\n",
       "      <td>0.687900</td>\n",
       "      <td>0.707310</td>\n",
       "      <td>0.353040</td>\n",
       "      <td>-0.451420</td>\n",
       "      <td>0.177150</td>\n",
       "      <td>-0.599130</td>\n",
       "      <td>0.276450</td>\n",
       "      <td>0.776900</td>\n",
       "      <td>0.776190</td>\n",
       "      <td>0.449290</td>\n",
       "      <td>0.172610</td>\n",
       "      <td>-0.184870</td>\n",
       "      <td>0.048760</td>\n",
       "      <td>0.351680</td>\n",
       "      <td>-0.786260</td>\n",
       "      <td>-0.368790</td>\n",
       "      <td>-0.528640</td>\n",
       "      <td>0.287650</td>\n",
       "      <td>-0.273120</td>\n",
       "      <td>-1.114000</td>\n",
       "      <td>0.064322</td>\n",
       "      <td>0.223620</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>toast</th>\n",
       "      <td>0.130740</td>\n",
       "      <td>-0.193730</td>\n",
       "      <td>0.253270</td>\n",
       "      <td>0.090102</td>\n",
       "      <td>-0.272580</td>\n",
       "      <td>-0.030571</td>\n",
       "      <td>0.096945</td>\n",
       "      <td>-0.115060</td>\n",
       "      <td>0.484000</td>\n",
       "      <td>0.848380</td>\n",
       "      <td>-0.076659</td>\n",
       "      <td>0.018191</td>\n",
       "      <td>-0.241220</td>\n",
       "      <td>0.283660</td>\n",
       "      <td>-0.104840</td>\n",
       "      <td>0.246110</td>\n",
       "      <td>-0.143910</td>\n",
       "      <td>0.665090</td>\n",
       "      <td>-0.140930</td>\n",
       "      <td>-0.278840</td>\n",
       "      <td>-0.205080</td>\n",
       "      <td>-0.419600</td>\n",
       "      <td>-0.690090</td>\n",
       "      <td>-0.053260</td>\n",
       "      <td>-0.144340</td>\n",
       "      <td>-0.039000</td>\n",
       "      <td>0.298490</td>\n",
       "      <td>-0.279020</td>\n",
       "      <td>-0.355020</td>\n",
       "      <td>-0.702110</td>\n",
       "      <td>-0.238980</td>\n",
       "      <td>0.428790</td>\n",
       "      <td>0.333940</td>\n",
       "      <td>0.089014</td>\n",
       "      <td>0.005472</td>\n",
       "      <td>0.178270</td>\n",
       "      <td>0.014409</td>\n",
       "      <td>-0.317270</td>\n",
       "      <td>0.134030</td>\n",
       "      <td>0.863840</td>\n",
       "      <td>...</td>\n",
       "      <td>-0.284800</td>\n",
       "      <td>0.227920</td>\n",
       "      <td>0.139690</td>\n",
       "      <td>0.382060</td>\n",
       "      <td>-0.355480</td>\n",
       "      <td>0.238640</td>\n",
       "      <td>-0.201450</td>\n",
       "      <td>0.166890</td>\n",
       "      <td>-0.696860</td>\n",
       "      <td>0.601810</td>\n",
       "      <td>0.053797</td>\n",
       "      <td>-0.024351</td>\n",
       "      <td>0.917220</td>\n",
       "      <td>0.472390</td>\n",
       "      <td>-0.595590</td>\n",
       "      <td>0.690710</td>\n",
       "      <td>-0.247270</td>\n",
       "      <td>0.163100</td>\n",
       "      <td>0.335320</td>\n",
       "      <td>0.524700</td>\n",
       "      <td>0.196020</td>\n",
       "      <td>-0.398660</td>\n",
       "      <td>0.400470</td>\n",
       "      <td>0.066376</td>\n",
       "      <td>-0.183670</td>\n",
       "      <td>0.483620</td>\n",
       "      <td>0.701590</td>\n",
       "      <td>0.019826</td>\n",
       "      <td>-0.162880</td>\n",
       "      <td>0.915870</td>\n",
       "      <td>0.142080</td>\n",
       "      <td>0.481910</td>\n",
       "      <td>0.045167</td>\n",
       "      <td>0.057151</td>\n",
       "      <td>-0.149520</td>\n",
       "      <td>-0.495130</td>\n",
       "      <td>-0.086677</td>\n",
       "      <td>-0.569040</td>\n",
       "      <td>-0.359290</td>\n",
       "      <td>0.097443</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>love</th>\n",
       "      <td>0.139490</td>\n",
       "      <td>0.534530</td>\n",
       "      <td>-0.252470</td>\n",
       "      <td>-0.125650</td>\n",
       "      <td>0.048748</td>\n",
       "      <td>0.152440</td>\n",
       "      <td>0.199060</td>\n",
       "      <td>-0.065970</td>\n",
       "      <td>0.128830</td>\n",
       "      <td>2.055900</td>\n",
       "      <td>-0.443390</td>\n",
       "      <td>-0.101740</td>\n",
       "      <td>0.271180</td>\n",
       "      <td>-0.124290</td>\n",
       "      <td>-0.457530</td>\n",
       "      <td>0.055878</td>\n",
       "      <td>-0.303950</td>\n",
       "      <td>0.795430</td>\n",
       "      <td>-0.290600</td>\n",
       "      <td>0.519080</td>\n",
       "      <td>0.169090</td>\n",
       "      <td>-0.327430</td>\n",
       "      <td>0.206060</td>\n",
       "      <td>-0.083885</td>\n",
       "      <td>0.196960</td>\n",
       "      <td>0.328470</td>\n",
       "      <td>0.087838</td>\n",
       "      <td>-0.127840</td>\n",
       "      <td>0.309790</td>\n",
       "      <td>0.206920</td>\n",
       "      <td>-0.085855</td>\n",
       "      <td>0.547900</td>\n",
       "      <td>-0.423440</td>\n",
       "      <td>0.345730</td>\n",
       "      <td>0.229210</td>\n",
       "      <td>-0.215380</td>\n",
       "      <td>0.091481</td>\n",
       "      <td>-0.278010</td>\n",
       "      <td>-0.341830</td>\n",
       "      <td>-0.130670</td>\n",
       "      <td>...</td>\n",
       "      <td>0.055732</td>\n",
       "      <td>0.319250</td>\n",
       "      <td>-0.262930</td>\n",
       "      <td>0.306830</td>\n",
       "      <td>0.094407</td>\n",
       "      <td>0.344400</td>\n",
       "      <td>0.303240</td>\n",
       "      <td>0.417650</td>\n",
       "      <td>-0.104270</td>\n",
       "      <td>0.418500</td>\n",
       "      <td>-0.678930</td>\n",
       "      <td>-0.160720</td>\n",
       "      <td>-0.820230</td>\n",
       "      <td>-0.014762</td>\n",
       "      <td>-0.091018</td>\n",
       "      <td>-0.008955</td>\n",
       "      <td>-0.026122</td>\n",
       "      <td>-0.134550</td>\n",
       "      <td>0.192630</td>\n",
       "      <td>0.331130</td>\n",
       "      <td>-0.152970</td>\n",
       "      <td>-0.368220</td>\n",
       "      <td>-0.146490</td>\n",
       "      <td>-0.315470</td>\n",
       "      <td>0.139630</td>\n",
       "      <td>0.298850</td>\n",
       "      <td>0.671980</td>\n",
       "      <td>0.220490</td>\n",
       "      <td>-0.182790</td>\n",
       "      <td>-0.248110</td>\n",
       "      <td>-0.124380</td>\n",
       "      <td>0.178440</td>\n",
       "      <td>-0.099469</td>\n",
       "      <td>0.008682</td>\n",
       "      <td>0.089213</td>\n",
       "      <td>-0.075513</td>\n",
       "      <td>-0.049069</td>\n",
       "      <td>-0.015228</td>\n",
       "      <td>0.088408</td>\n",
       "      <td>0.302170</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>ham</th>\n",
       "      <td>-0.773320</td>\n",
       "      <td>-0.282540</td>\n",
       "      <td>0.580760</td>\n",
       "      <td>0.841480</td>\n",
       "      <td>0.258540</td>\n",
       "      <td>0.585210</td>\n",
       "      <td>-0.021890</td>\n",
       "      <td>-0.463680</td>\n",
       "      <td>0.139070</td>\n",
       "      <td>0.658720</td>\n",
       "      <td>-0.168670</td>\n",
       "      <td>0.192260</td>\n",
       "      <td>-0.398010</td>\n",
       "      <td>-0.600910</td>\n",
       "      <td>-0.241560</td>\n",
       "      <td>-0.088710</td>\n",
       "      <td>0.099011</td>\n",
       "      <td>0.838680</td>\n",
       "      <td>0.397170</td>\n",
       "      <td>0.288050</td>\n",
       "      <td>0.477210</td>\n",
       "      <td>-0.160270</td>\n",
       "      <td>-0.011555</td>\n",
       "      <td>0.400100</td>\n",
       "      <td>-0.178950</td>\n",
       "      <td>-0.941120</td>\n",
       "      <td>-0.538650</td>\n",
       "      <td>0.284690</td>\n",
       "      <td>0.204900</td>\n",
       "      <td>-0.603190</td>\n",
       "      <td>-0.767880</td>\n",
       "      <td>0.331020</td>\n",
       "      <td>0.508820</td>\n",
       "      <td>-0.166050</td>\n",
       "      <td>0.729980</td>\n",
       "      <td>0.247200</td>\n",
       "      <td>-0.409570</td>\n",
       "      <td>-0.030922</td>\n",
       "      <td>-0.147350</td>\n",
       "      <td>1.449700</td>\n",
       "      <td>...</td>\n",
       "      <td>-0.083141</td>\n",
       "      <td>-0.412700</td>\n",
       "      <td>0.057152</td>\n",
       "      <td>0.079565</td>\n",
       "      <td>-0.370370</td>\n",
       "      <td>0.430860</td>\n",
       "      <td>-0.237530</td>\n",
       "      <td>-0.541830</td>\n",
       "      <td>-0.926910</td>\n",
       "      <td>0.089818</td>\n",
       "      <td>0.516110</td>\n",
       "      <td>-0.174690</td>\n",
       "      <td>0.482730</td>\n",
       "      <td>0.580170</td>\n",
       "      <td>-0.693060</td>\n",
       "      <td>0.632340</td>\n",
       "      <td>-0.079333</td>\n",
       "      <td>-0.008798</td>\n",
       "      <td>0.058371</td>\n",
       "      <td>0.006332</td>\n",
       "      <td>-0.053509</td>\n",
       "      <td>-0.074803</td>\n",
       "      <td>0.139680</td>\n",
       "      <td>0.391960</td>\n",
       "      <td>0.584610</td>\n",
       "      <td>0.275520</td>\n",
       "      <td>0.755370</td>\n",
       "      <td>0.418950</td>\n",
       "      <td>-0.183280</td>\n",
       "      <td>0.452320</td>\n",
       "      <td>0.464470</td>\n",
       "      <td>0.481400</td>\n",
       "      <td>-0.829200</td>\n",
       "      <td>0.354910</td>\n",
       "      <td>0.224530</td>\n",
       "      <td>-0.493920</td>\n",
       "      <td>0.456930</td>\n",
       "      <td>-0.649100</td>\n",
       "      <td>-0.131930</td>\n",
       "      <td>0.372040</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>quick</th>\n",
       "      <td>-0.445630</td>\n",
       "      <td>0.191510</td>\n",
       "      <td>-0.249210</td>\n",
       "      <td>0.465900</td>\n",
       "      <td>0.161950</td>\n",
       "      <td>0.212780</td>\n",
       "      <td>-0.046480</td>\n",
       "      <td>0.021170</td>\n",
       "      <td>0.417660</td>\n",
       "      <td>1.686900</td>\n",
       "      <td>-0.451860</td>\n",
       "      <td>-0.011807</td>\n",
       "      <td>0.072409</td>\n",
       "      <td>-0.114820</td>\n",
       "      <td>-0.534210</td>\n",
       "      <td>-0.150700</td>\n",
       "      <td>-0.352960</td>\n",
       "      <td>1.711500</td>\n",
       "      <td>-0.211920</td>\n",
       "      <td>-0.400470</td>\n",
       "      <td>-0.420410</td>\n",
       "      <td>0.076042</td>\n",
       "      <td>0.023444</td>\n",
       "      <td>-0.744810</td>\n",
       "      <td>0.006499</td>\n",
       "      <td>0.443510</td>\n",
       "      <td>0.484180</td>\n",
       "      <td>-0.541020</td>\n",
       "      <td>0.375070</td>\n",
       "      <td>-0.595550</td>\n",
       "      <td>0.097029</td>\n",
       "      <td>0.119870</td>\n",
       "      <td>-0.079122</td>\n",
       "      <td>-0.053397</td>\n",
       "      <td>-0.028888</td>\n",
       "      <td>0.157510</td>\n",
       "      <td>0.164100</td>\n",
       "      <td>0.288250</td>\n",
       "      <td>-0.065758</td>\n",
       "      <td>-0.240000</td>\n",
       "      <td>...</td>\n",
       "      <td>-0.185420</td>\n",
       "      <td>-0.064913</td>\n",
       "      <td>0.242120</td>\n",
       "      <td>-0.674920</td>\n",
       "      <td>0.394260</td>\n",
       "      <td>-0.304280</td>\n",
       "      <td>0.103000</td>\n",
       "      <td>0.336920</td>\n",
       "      <td>0.300430</td>\n",
       "      <td>0.349690</td>\n",
       "      <td>0.353660</td>\n",
       "      <td>0.017008</td>\n",
       "      <td>-0.170840</td>\n",
       "      <td>-0.286230</td>\n",
       "      <td>-0.106920</td>\n",
       "      <td>0.313400</td>\n",
       "      <td>0.307010</td>\n",
       "      <td>-0.103920</td>\n",
       "      <td>0.231590</td>\n",
       "      <td>0.027126</td>\n",
       "      <td>0.169710</td>\n",
       "      <td>0.010259</td>\n",
       "      <td>-0.171380</td>\n",
       "      <td>0.448950</td>\n",
       "      <td>-0.293820</td>\n",
       "      <td>-0.312280</td>\n",
       "      <td>-0.054189</td>\n",
       "      <td>0.069783</td>\n",
       "      <td>-0.104720</td>\n",
       "      <td>-0.124150</td>\n",
       "      <td>-0.329460</td>\n",
       "      <td>0.421860</td>\n",
       "      <td>-0.039543</td>\n",
       "      <td>0.150180</td>\n",
       "      <td>0.338220</td>\n",
       "      <td>0.049554</td>\n",
       "      <td>0.149420</td>\n",
       "      <td>-0.038789</td>\n",
       "      <td>-0.019069</td>\n",
       "      <td>0.348650</td>\n",
       "    </tr>\n",
       "  </tbody>\n",
       "</table>\n",
       "<p>20 rows × 300 columns</p>\n",
       "</div>"
      ],
      "text/plain": [
       "                0         1         2    ...       297       298       299\n",
       "sausages  -0.174290 -0.064869 -0.046976  ... -0.806830  0.619540  0.201200\n",
       "lazy      -0.353320 -0.299710 -0.176230  ...  0.344190 -0.845300 -0.077383\n",
       "beautiful  0.171200  0.534390 -0.348540  ...  0.172580  0.063875  0.350990\n",
       "blue       0.129450  0.036518  0.032298  ...  0.052289  0.182060  0.412640\n",
       "fox       -0.348680 -0.077720  0.177750  ...  0.123610 -0.553700 -0.544790\n",
       "jumps     -0.334840  0.215990 -0.350440  ...  0.190290  0.692290 -0.071501\n",
       "dog       -0.057120  0.052685  0.003026  ...  0.035107  0.042551 -0.010643\n",
       "breakfast  0.073378  0.227670  0.208420  ... -0.599390 -0.061114  0.232200\n",
       "today     -0.156570  0.594890 -0.031445  ... -0.250060 -0.115940  0.083741\n",
       "green     -0.072368  0.233200  0.137260  ... -0.111550  0.532930  0.712680\n",
       "kings      0.259230 -0.854690  0.360010  ... -0.147330  0.662490 -0.577420\n",
       "sky        0.312550 -0.303080  0.019587  ...  0.537740  0.363490  0.496920\n",
       "brown     -0.374120 -0.076264  0.109260  ... -0.365760 -0.288190  0.114630\n",
       "bacon     -0.430730 -0.016025  0.484620  ... -1.065400 -0.244940  0.269540\n",
       "eggs      -0.417810 -0.035192 -0.126150  ... -0.324700  0.059721  0.159160\n",
       "beans     -0.423290 -0.264500  0.200870  ... -1.114000  0.064322  0.223620\n",
       "toast      0.130740 -0.193730  0.253270  ... -0.569040 -0.359290  0.097443\n",
       "love       0.139490  0.534530 -0.252470  ... -0.015228  0.088408  0.302170\n",
       "ham       -0.773320 -0.282540  0.580760  ... -0.649100 -0.131930  0.372040\n",
       "quick     -0.445630  0.191510 -0.249210  ... -0.038789 -0.019069  0.348650\n",
       "\n",
       "[20 rows x 300 columns]"
      ]
     },
     "execution_count": 12,
     "metadata": {
      "tags": []
     },
     "output_type": "execute_result"
    }
   ],
   "source": [
    "unique_words = list(set([word for sublist in tokenized_corpus for word in sublist]))\n",
    "\n",
    "word_glove_vectors = np.array([nlp(word).vector for word in unique_words])\n",
    "vec_df = pd.DataFrame(word_glove_vectors, index=unique_words)\n",
    "vec_df"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {
    "colab_type": "text",
    "id": "fSKGVbkeRiq5"
   },
   "source": [
    "We can now use t-SNE to visualize these embeddings similar to what we did using our Word2Vec embeddings."
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 13,
   "metadata": {
    "colab": {
     "base_uri": "https://localhost:8080/",
     "height": 378
    },
    "colab_type": "code",
    "id": "j6aSesHGRiq5",
    "outputId": "50fb644e-cbd4-4937-ac07-25b83d2f331c"
   },
   "outputs": [
    {
     "data": {
      "image/png": "iVBORw0KGgoAAAANSUhEUgAAAtEAAAFpCAYAAABauHSCAAAABHNCSVQICAgIfAhkiAAAAAlwSFlz\nAAALEgAACxIB0t1+/AAAADl0RVh0U29mdHdhcmUAbWF0cGxvdGxpYiB2ZXJzaW9uIDMuMC4zLCBo\ndHRwOi8vbWF0cGxvdGxpYi5vcmcvnQurowAAIABJREFUeJzs3XucznX+//HHZ2jT5FQpnTC0JcwJ\nM6hxikp9VUgljS1ZbAelvr+Uol1ba7f62q1VSlol7fRNUdrV2UbOZUYjilbaGZItdrGmoYz5/P6Y\ncX3Hobgy5hrjcb/d5nZdn/fncL3eo9HTe96f9ycIwxBJkiRJBy4u1gVIkiRJhxtDtCRJkhQlQ7Qk\nSZIUJUO0JEmSFCVDtCRJkhQlQ7QkSZIUJUO0JEmSFCVDtCRJkhQlQ7QkSZIUJUO0JEmSFKXqsS7g\nQNSrVy9MSEiIdRmSJEmq4nJycjaGYXji/o47LEJ0QkIC2dnZsS5DkiRJVVwQBPkHcpzTOSRJkqQo\nGaIlSZKkKBmiJUmSpCgZoiVJkqQoGaIlSZKkKBmiJUmSpCgZoiVJkiqhb775hu7du5OSkkJiYiJT\npkzhvvvuIz09ncTERAYPHkwYhgB07tw5shzwxo0b2fV8jY8//pg2bdqQmppKcnIyq1atAqBnz560\nbt2aFi1aMGHChMhnTpw4kbPOOos2bdowaNAghgwZAsCGDRvo3bs36enppKenM3/+fADee+89UlNT\nSU1NpWXLlmzdurWivj0xd1isEy1JknSkefPNNzn11FN57bXXANiyZQsXXHABv/zlLwH42c9+xowZ\nM7j00ku/9xrjx49n6NChZGZm8t1337Fz504Ann76aY4//ni2bdtGeno6vXv35ttvv+X+++9nyZIl\n1KpViy5dupCSkgLA0KFDuf3222nfvj1r1qyhW7durFixgjFjxjBu3DgyMjIoKCigRo0ah/i7Unk4\nEi1JklQJJSUl8c4773DXXXcxd+5c6tSpw6xZs2jbti1JSUm8++67fPzxxz94jXPOOYff/va3PPjg\ng+Tn53PMMccAMHbsWFJSUmjXrh1r165l1apVfPDBB3Tq1Injjz+eo446iiuvvDJynZkzZzJkyBBS\nU1O57LLL+M9//kNBQQEZGRn893//N2PHjmXz5s1Ur37kjM8eOT2VJEmq7LKyYMQIWLOGsxo2ZMnd\nd/P6sccycuRIunbtyrhx48jOzqZBgwaMGjWK7du3A1C9enWKi4sBIm0A11xzDW3btuW1117jv/7r\nv3jyySeJi4tj5syZLFy4kPj4eDp37rzbOftSXFzMokWL9hppHj58ON27d+f1118nIyODt956i7PP\nPrucvymVkyPRkiRJlUFWFgweDPn5EIZ8mZ9P/O230y8IGDZsGEuWLAGgXr16FBQUMHXq1MipCQkJ\n5OTkAOzW/vnnn9OkSRNuvfVWevTowUcffcSWLVs47rjjiI+PZ+XKlSxatAiA9PR03nvvPTZt2kRR\nURHTpk2LXOfCCy/k0UcfjWzn5uYCsHr1apKSkrjrrrtIT09n5cqVh+77U8k4Ei1JklQZjBgBhYWR\nzWXAsG3biLv+eo5KSeGJJ55g+vTpJCYmcvLJJ5Oenh459o477uCqq65iwoQJdO/ePdL+4osv8txz\nz3HUUUdx8sknc88993Dssccyfvx4mjVrRtOmTWnXrh0Ap512Gvfccw9t2rTh+OOP5+yzz6ZOnTpA\nyfSPm2++meTkZIqKiujYsSPjx4/nkUceYdasWcTFxdGiRQsuvvjiivleVQLBrrs6K7O0tLRw1x2n\nkiRJVVJcHOwrlwUBlE7VONQKCgqoWbMmRUVF9OrViwEDBtCrV68K+ezKIgiCnDAM0/Z3nNM5JEmS\nKoOGDaNrPwRGjRpFamoqiYmJNG7cmJ49e1bYZx9unM4hSZJUGYweXTInusyUDuLjS9oryJgxYyrs\nsw53jkRLkiRVBpmZMGECNGpUMoWjUaOS7czMWFemfXAkWpIkqbLIzDQ0HyYciZYkSZKiZIiWJEmS\nomSIliRJkqJkiJYkSVKllJeXR2JiYqzL2CdDtCRJkhQlQ7QkSarSKvNopvavqKiIzMxMmjVrxhVX\nXEFhYSE5OTl06tSJ1q1b061bN9avXw/AU089RXp6OikpKfTu3ZvC0jW3+/fvz6233sq5555LkyZN\nmDp1KgDr16+nY8eOkQfMzJ0794DrMkRLkiSp0vr000+56aabWLFiBbVr12bcuHHccsstTJ06lZyc\nHAYMGMCIESMAuPzyy1m8eDFLly6lWbNmTJw4MXKd9evXM2/ePGbMmMHw4cMBeP755+nWrRu5ubks\nXbqU1NTUA67LdaIlSVKVt3PnTgYNGsSCBQs47bTTePXVV/nzn//MhAkT+O677/jpT3/Kc889R3x8\nPP379+eYY47hww8/5Ouvv+bpp59m8uTJLFy4kLZt2zJp0qRYd+eI0qBBAzIyMgDo168fv/3tb1m+\nfDkXXHABUPJne8oppwCwfPlyRo4cyebNmykoKKBbt26R6/Ts2ZO4uDiaN2/OV199BUB6ejoDBgxg\nx44d9OzZM6oQ7Ui0JEmq8latWsXNN9/Mxx9/TN26dZk2bdoPjlpu2rSJhQsX8vDDD3PZZZdx++23\n8/HHH7Ns2TJyc3Nj2JMjTxAEu23XqlWLFi1akJubS25uLsuWLePtt98GSqZtPPbYYyxbtoxf/epX\nbN++PXLe0UcfHXkfhiEAHTt2ZM6cOZx22mn079+fyZMnH3BdhmhJklTlNW7cODLK2Lp1a/Ly8li+\nfDkdOnQgKSmJrKwsPv7448jxl156KUEQkJSURP369UlKSiIuLo4WLVqQl5cXo14cIbKyICEB4uKg\nfXvWrFnDwoULgZLpF+3atWPDhg2Rth07dkT+7LZu3copp5zCjh07yMrK2u9H5efnU79+fQYNGsTA\ngQNZsmTJAZdpiJYkSVXPHkHs6DIjktWqVaOoqOiARi3j4uJ2G8GMi4ujqKiowrpxxMnKgsGDIT8f\nwhDWraNpEDDujjto1qwZmzZtisyHvuuuu0hJSSE1NZUFCxYAcP/999O2bVsyMjI4++yz9/txs2fP\nJiUlhZYtWzJlyhSGDh16wKU6J1qSJFUtu4JY6coMrFsHQVDSnpkZOWzPUcvTTjstJuXm5eVxySWX\nsHz58ph8fqUyYsT//bkBCcDK0jBNmd8ApKamMmfOnL1Ov/HGG7nxxhv3at9zHntBQQEA1113Hddd\nd92PKtWRaEmSVLXsEcSAklHN0hUcdol21FIVYM2a6NpjKNg1sboyS0tLC7Ozs2NdhiRJOhzExZWE\n5j0FARQXV3w9+5GXl8dFF11E69atWbJkCS1atGDy5MmMGTOGv/71r2zbto1zzz2XJ598kiAI+Oyz\nz7jhhhvYsGED1apV46WXXqJJkybceeedvPHGGwRBwMiRI+nTpw+zZ89m1KhR1KtXj+XLl9O6dWv+\n/Oc/73WzXqWRkFAylWNPjRrtNhJ9KAVBkBOGYdr+jnMkWpIkVS0NG0bXXgnsuRby448/zpAhQ1i8\neDHLly9n27ZtzJgxA4DMzExuvvlmli5dyoIFCzjllFN4+eWXI2sdz5w5k2HDhkUeQPLhhx/yyCOP\n8Mknn/D5558zf/78WHb1h40eDfHxu7fFx5e0VzKGaEmSVLUcRkFslz3XQp43bx6zZs2ibdu2JCUl\n8e677/Lxxx+zdetW1q1bR69evQCoUaMG8fHxzJs3j759+1KtWjXq169Pp06dWLx4MQBt2rTh9NNP\nJy4ujtTU1Mq9ukhmJkyYUDLyHAQlrxMm7DaXvbLwxkJJklS17ApcI0aUzKVt2LAkQFfCILbLntMr\ngiDgpptuIjs7mwYNGjBq1KjdVg+JRtnVRXatTFKpZWZW6j+rXRyJliRJVU9mZskc2uLiktfKFsr2\nsxZy+/btAahXrx4FBQVMnToVKHnQyOmnn8706dMB+PbbbyksLKRDhw5MmTKFnTt3smHDBubMmUOb\nNm1i0rUjhSPRkiRJFWkfS/DtWgt5wL//TfPmzbnxxhvZtGkTiYmJnHzyyaSnp0dOf+655/jFL37B\nL3/5S4466iheeuklevXqxcKFC0lJSSEIAh566CFOPvlkVq5cGaNOVn2uziFJklSRKsEKFPp+Fbo6\nRxAEdYMgmBoEwcogCFYEQXBOEATHB0HwThAEq0pfjys9NgiCYGwQBJ8FQfBREAStyqMGSZKkw8Jh\ntBayvl95zYn+I/BmGIZnAynACmA48LcwDM8E/la6DXAxcGbp12DgiXKqQZIkqfI7DJfg094OOkQH\nQVAH6AhMBAjD8LswDDcDPYBnSw97FuhZ+r4HMDkssQioGwTBKQdbhyRJ0mHhMFyCT3srj5HoxsAG\n4JkgCD4MguBPQRAcC9QPw3B96TH/BOqXvj8NWFvm/C9K2yRJkqq+w2gtZH2/8gjR1YFWwBNhGLYE\nvuH/pm4AEJbcvRjVHYxBEAwOgiA7CILsDRs2lEOZkiRJlURlX4JP+1UeIfoL4IswDN8v3Z5KSaj+\natc0jdLXr0v3rwMalDn/9NK23YRhOCEMw7QwDNNOPPHEcihTkiRJKh8HHaLDMPwnsDYIgqalTV2B\nT4C/ANeVtl0HvFr6/i/AtaWrdLQDtpSZ9iFJkiRVeuX1sJVbgKwgCH4CfA5cT0lAfzEIgp8D+cBV\npce+DvwX8BlQWHqsJEmSdNgolxAdhmEusK9Fqbvu49gQuLk8PleSJEmKhfJaJ1qSJEk6YhiiJUmS\npCgZoiVJkqQoGaIlSZKkKBmiJUmSpCgZoiVJkqQoGaIlSZKkKBmiJUmSdED+/Oc/06ZNG1JTU/nF\nL37Bzp07mThxImeddRZt2rRh0KBBDBkyBIDVq1fTrl07kpKSGDlyJDVr1gRg/fr1dOzYkdTUVBIT\nE5k7d24su/SjGaIlSZK0XytWrGDKlCnMnz+f3NxcqlWrRlZWFvfffz+LFi1i/vz5rFy5MnL80KFD\nGTp0KMuWLeP000+PtD///PN069aN3Nxcli5dSmpqaiy6c9DK67HfkiRJqsL+9re/kZOTQ3p6OgDb\ntm1jwYIFdOrUieOPPx6AK6+8kr///e8ALFy4kOnTpwNwzTXXcMcddwCQnp7OgAED2LFjBz179jxs\nQ7Qj0ZIkSdq3rCxISIC4OMJf/pLr0tLIzc0lNzeXTz/9lFGjRkV9yY4dOzJnzhxOO+00+vfvz+TJ\nk8u97IpgiJYkSdLesrJg8GDIz4cwpOumTUydMYOvH38cgH//+9+0bNmS9957j02bNlFUVMS0adMi\np7dr1y6y/cILL0Ta8/PzqV+/PoMGDWLgwIEsWbKkYvtVTgzRkiRJ2tuIEVBYGNlsDvwmDLnwtttI\nTk7mggsuYP369dxzzz20adOGjIwMEhISqFOnDgCPPPIIf/jDH0hOTuazzz6LtM+ePZuUlBRatmzJ\nlClTGDp0aCx6d9CCMAxjXcN+paWlhdnZ2bEuQ5Ik6cgRFwf7yolBAMXFkc2CggJq1qxJUVERvXr1\nYsCAAfTq1YvCwkKOOeYYgiDghRde4H//93959dVXK7ADP04QBDlhGKbt7zhvLJQkSdLeGjYsmcqx\nr/YyRo0axcyZM9m+fTsXXnghPXv2BCAnJ4chQ4YQhiF169bl6aefroiqK4wj0ZIkSdrbrjnRZaZ0\nEB8PEyZAZmbs6jrEDnQk2jnRkiRJ2ltmZklgbtSoZApHo0ZVPkBHw+kckiRJ2rfMTEPz93AkWpIk\nSYqSIVqSJEmKkiFakiRJipIhWpIkVTmbN2/m8dIn65WXRx55hMKyK1XoiGaIliRJVY4hWoeaq3NI\nkqQqZ/jw4axevZrU1FQuuOACAN544w2CIGDkyJH06dOHgoICevTowaZNm9ixYwe/+c1v6NGjB998\n8w1XXXUVX3zxBTt37uTee+/lq6++4ssvv+S8886jXr16zJo1K8Y9VKz5sBVJklTl5OXlcckll7B8\n+XKmTZvG+PHjefPNN9m4cSPp6em8//77nHjiiRQWFlK7dm02btxIu3btWLVqFS+//DJvvvkmTz31\nFABbtmyhTp06JCQkkJ2dTb169WLcOx1KPmxFkiQJmDdvHn379qVatWrUr1+fTp06sXjxYsIw5J57\n7iE5OZnzzz+fdevW8dVXX5GUlMQ777zDXXfdxdy5c6lTp06su6BKyBAtSZKqhqwsSEiAuDho3x62\nbNnP4Vls2LCBnJwccnNzqV+/Ptu3b+ess85iyZIlJCUlMXLkSO67776KqV+HFUO0JEk6/GVlweDB\nkJ8PYUitdevYum4dZGXRoUMHpkyZws6dO9mwYQNz5syhTZs2bNmyhZNOOomjjjqKWbNmkZ+fD8CX\nX35JfHw8/fr1Y9iwYSxZsgSAWrVqsXXr1lj2UpWINxZKkqTD34gRUGbljBOAjDAk8frruXjoUJKT\nk0lJSSEIAh566CFOPvlkMjMzufTSS0lKSiItLY2zzz4bgGXLljFs2DDi4uI46qijeOKJJwAYPHgw\nF110Eaeeeqo3FsobCyVJUhUQFwf7yjRBAMXFFV+PDlveWChJko4cDRtG1y4dJEO0JEk6/I0eDfHx\nu7fFx5e0S4eAIVqSJB3+MjNhwgRo1KhkCkejRiXbmZmxrkxVlDcWSpKkqiEz09CsCuNItCRJkhQl\nQ7QkSZIUJUO0JEmSFCVDtCRJkhQlQ7QkSZIUJUO0JEmSFCVDtCRJkhQlQ7QkSZIUJUO0JEmSFCVD\ntCRJkhQlQ7QkSZIUJUO0JEmSFCVDtCRJkhQlQ7QkSSpXeXl5JCYmlvt1R40axZgxY/ZqX7lyJamp\nqbRs2ZLVq1dHdc1Jkybx5ZdflleJOoIYoiVJUoXbuXNnuV1r+vTpXHHFFXz44YecccYZUZ1riNaP\nZYiWJEnlrqioiMzMTJo1a8YVV1xBYWEhCQkJ3HXXXbRq1YqXXnqJ1atXc9FFF9G6dWs6dOjAypUr\nAfjrX/9K27ZtadmyJeeffz5fffXVXtd/6qmnuPjii3nttdd45JFHeOKJJzjvvPMA6NmzJ61bt6ZF\nixZMmDABKAnt/fv3JzExkaSkJB5++GGmTp1KdnY2mZmZpKamsm3btor7BumwVz3WBUiSpKrn008/\nZeLEiWRkZDBgwAAef/xxAE444QSWLFkCQNeuXRk/fjxnnnkm77//PjfddBPvvvsu7du3Z9GiRQRB\nwJ/+9Cceeughfv/730eu/dhjj/HOO+8wffp0jj76aG644QZq1qzJHXfcAcDTTz/N8ccfz7Zt20hP\nT6d3797k5eWxbt06li9fDsDmzZupW7cujz32GGPGjCEtLa2Cv0M63BmiJUlSuWvQoAEZGRkA9OvX\nj7FjxwLQp08fAAoKCliwYAFXXnll5Jxvv/0WgC+++II+ffqwfv16vvvuOxo3bhw5ZvLkyTRo0IDp\n06dz1FFH7fOzx44dyyuvvALA2rVrWbVqFU2bNuXzzz/nlltuoXv37lx44YXl32kdUZzOIUmSDl5W\nFiQkQFwctG9PUFi42+4gCAA49thjASguLqZu3brk5uZGvlasWAHALbfcwpAhQ1i2bBlPPvkk27dv\nj1wnKSmJvLw8vvjii32WMXv2bGbOnMnChQtZunQpLVu2ZPv27Rx33HEsXbqUzp07M378eAYOHHgI\nvgk6khiiJUnSwcnKgsGDIT8fwhDWrWPNv/7FwlGjAHj++edp3779bqfUrl2bxo0b89JLLwEQhiFL\nly4FYMuWLZx22mkAPPvss7ud17JlS5588kkuu+yyfd4QuGXLFo477jji4+NZuXIlixYtAmDjxo0U\nFxfTu3dvfvOb30SmlNSqVYutW7eW3/dCRwxDtCRJOjgjRsAeI89NgXFjxtCsWTM2bdrEjTfeuNdp\nWVlZTJw4kZSUFFq0aMGrr74KlCxld+WVV9K6dWvq1au313nt27dnzJgxdO/enY0bN+6276KLLqKo\nqIhmzZoxfPhw2rVrB8C6devo3Lkzqamp9OvXj9/97ncA9O/fnxtuuMEbCxW1IAzDWNewX2lpaWF2\ndnasy5AkSfsSF1cyAr2nIIDi4oqvRzoIQRDkhGG43ztNHYmWJEkHp2HD6NqlKqDcQnQQBNWCIPgw\nCIIZpduNgyB4PwiCz4IgmBIEwU9K248u3f6sdH9CedUgSZJiYPRoiI/fvS0+vqRdqqLKcyR6KLCi\nzPaDwMNhGP4U2AT8vLT958Cm0vaHS4+TJEmHq8xMmDABGjUqmcLRqFHJdmZmrCuTDplyCdFBEJwO\ndAf+VLodAF2AqaWHPAv0LH3fo3Sb0v1dg13r3kiSpMNTZibk5ZXMgc7LM0CryiuvkehHgDuBXXcP\nnABsDsOwqHT7C+C00venAWsBSvdvKT1+N0EQDA6CIDsIguwNGzaUU5mSJEnSwTvoEB0EwSXA12EY\n5pRDPRFhGE4IwzAtDMO0E088sTwvLUmSJB2U8njsdwZwWRAE/wXUAGoDfwTqBkFQvXS0+XRgXenx\n64AGwBdBEFQH6gD/Koc6JEmSpApx0CPRYRjeHYbh6WEYJgBXA++GYZgJzAKuKD3sOuDV0vd/Kd2m\ndP+74eGwWLUkSZJU6lCuE30X8N9BEHxGyZzniaXtE4ETStv/Gxh+CGuQJEmSyl15TOeICMNwNjC7\n9P3nQJt9HLMduLI8P1eSJEmqSD6xUJIkSYqSIVqSJEmKkiFakiRJipIhWpIkSYqSIVqSJEmKkiFa\nkiRJipIhWpIkSYqSIVqSJEmKkiFakiRJipIhWpIkSYqSIVqSJEmKkiFakiRJipIhWpIkSYqSIVqS\nJEmKkiFakiRJipIhWpIkSYqSIVqSJEmKkiFakiRJipIhWpIkSYqSIVqSJEmKkiFakiRJipIhWpIk\nSYqSIVqSJEmKkiFakiRJipIhWpIkSYqSIVqSJEmKkiFakiRJipIhWpIkSYqSIVqSJEmKkiFakiRJ\nipIhWpIkSYqSIVqSJEmKkiFakiRJipIhWpIkSYqSIVqSJEmKkiFakiRJipIhWpIkSYqSIVqSJEmK\nkiFakiRJipIhWpIkSYqSIVqSJEmKkiFakiRJipIhWpIkSYqSIVqSJEmKkiFakiRJipIhWpIkSYqS\nIVqSJEmKkiFakiRJipIhWpIkSYqSIVqSJEmKkiFakiRJipIhWpIkSYqSIVqSJEmKkiFakiRJipIh\nWpIkSYqSIVqSJEmK0kGH6CAIGgRBMCsIgk+CIPg4CIKhpe3HB0HwThAEq0pfjyttD4IgGBsEwWdB\nEHwUBEGrg61BkiRJqkjlMRJdBPy/MAybA+2Am4MgaA4MB/4WhuGZwN9KtwEuBs4s/RoMPFEONUiS\nJEkV5qBDdBiG68MwXFL6fiuwAjgN6AE8W3rYs0DP0vc9gMlhiUVA3SAITjnYOiRJkqSKUq5zooMg\nSABaAu8D9cMwXF+6659A/dL3pwFry5z2RWmbJEmSdFgotxAdBEFNYBpwWxiG/ym7LwzDEAijvN7g\nIAiygyDI3rBhQ3mVKUmSJB20cgnRQRAcRUmAzgrD8OXS5q92TdMoff26tH0d0KDM6aeXtu0mDMMJ\nYRimhWGYduKJJ5ZHmZIkSVK5KI/VOQJgIrAiDMM/lNn1F+C60vfXAa+Wab+2dJWOdsCWMtM+JEmS\npEqvejlcIwP4GbAsCILc0rZ7gAeAF4Mg+DmQD1xVuu914L+Az4BC4PpyqEGSJEmqMAcdosMwnAcE\n37O76z6OD4GbD/ZzJUmSpFjxiYWSJElSlAzRkiRJUpQM0ZIkSVKUDNGSJElSlAzRkqRD6txzz411\nCZJU7gzRkqRDasGCBbEuQZLKnSFakqKUl5dHYmLibm3Z2dnceuutMaqocqtZsyazZ8/mkksuibQN\nGTKESZMmAZCQkMDdd99NamoqaWlpLFmyhG7dunHGGWcwfvx4AGbPnk3Hjh3p3r07TZs25YYbbqC4\nuJidO3fSv39/EhMTSUpK4uGHH45FFyUdgcrjYSuSdMRLS0sjLS0t1mUctho2bEhubi633347/fv3\nZ/78+Wzfvp3ExERuuOEGAD744AM++eQTGjVqxEUXXcTLL79M48aNWbduHcuXLwdg8+bNseyGpCOI\nI9GSdBA+//xzWrZsyf/8z/9ERlpHjRrFgAED6Ny5M02aNGHs2LGR4++//36aNm1K+/bt6du3L2PG\njAFg7NixNG/enOTkZK6++uqY9CWWLrvsMgCSkpJo27YttWrV4sQTT+Too4+OBOM2bdrQpEkTqlWr\nRt++fZk3bx5NmjTh888/55ZbbuHNN9+kdu3aseyGpCOII9GS9CN9+umnXH311UyaNIlNmzbx3nvv\nRfatXLmSWbNmsXXrVpo2bcqNN95Ibm4u06ZNY+nSpezYsYNWrVrRunVrAB544AH+8Y9/7BYaD1tZ\nWTBiBKxZAw0bQlER1atXp7i4OHLI9u3bdzvl6KOPBiAuLi7yftd2UVERAEGw+8NxgyDguOOOY+nS\npbz11luMHz+eF198kaeffvpQ9UySIhyJlqQfYcOGDfTo0YOsrCxSUlL22t+9e3eOPvpo6tWrx0kn\nncRXX33F/Pnz6dGjBzVq1KBWrVpceumlkeOTk5PJzMzkz3/+M9WrH8bjG1lZMHgw5OdDGJa8fvst\njZYs4ZNPPuHbb79l8+bN/O1vf4v60h988AH/+Mc/KC4uZsqUKbRv356NGzdSXFxM7969+c1vfsOS\nJUsOQackaW+GaEnan6wsSEiAuLiS1+nTqVOnDg0bNmTevHn7PKXsaGq1atUio6nf57XXXuPmm29m\nyZIlpKen7/f4SmvECCgs3K0pABr84Q9cddVVJCYmctVVV9GyZcuoL52ens6QIUNo1qwZjRs3plev\nXqxbt47OnTuTmppKv379+N3vfldOHZGkH3YYD3dIUgXYNbK6Kxjm58Pdd/OTevV45ZVX6NatGzVr\n1uTUU0/d76UyMjL4xS9+wd13301RUREzZsxg8ODBFBcXs3btWs477zzat2/PCy+8QEFBAXXr1j3E\nnTsE1qzZbfNfwPGl7Q899BAPPfTQXqfk5eVF3vfv35/+/fvvc1/t2rWZMWPGbuempKQ4+iwpJgzR\nkvRD9jGyyvbt8NVXHHvsscxsORHcAAAgAElEQVSYMYMLLriAe++9d7+XSk9P57LLLiM5OZn69euT\nlJREnTp12LlzJ/369WPLli2EYcitt956eAZoKJkDnZ8PwJdAZ+COXe2SVIUEYRjGuob9SktLC7Oz\ns2NdhqQjUVxcydzePQUBlLlR7kAVFBRQs2ZNCgsL6dixIxMmTKBVq1blUGglsefIPUB8PEyYAJmZ\nsatLkg5QEAQ5YRjud81S50RL0g/5vhHUHzmyOnjwYFJTU2nVqhW9e/euWgEaSoLyhAnQqFHJPzQa\nNTJAS6qSHImWpB/iyKokHVEciZak8uDIqiRpH7yxUJL2JzPT0CxJ2o0j0ZIkSVKUDNGSJElSlAzR\nkiRJUpQM0ZIkSVKUDNGSJElSlAzRkiRJUpQM0ZIkSVKUDNGSJElSlAzRkiRJUpQM0ZIkSVKUDNGS\nJElSlAzRkiRJUpQM0ZIkSVKUDNGSJElSlAzRkiRJUpQM0ZIkSVKUDNGSJElSlAzRkiRJUpQM0ZJ0\nhBg1ahRjxoyJdRmSVCUYoiVJkqQoGaIlqQobPXo0Z511Fu3bt+fTTz8FIDc3l3bt2pGcnEyvXr3Y\ntGkTAIsXLyY5OZnU1FSGDRtGYmJiLEuXpErNEC1JVVROTg4vvPACubm5vP766yxevBiAa6+9lgcf\nfJCPPvqIpKQkfv3rXwNw/fXX8+STT5Kbm0u1atViWfpByc7O5tZbb/3BY2rWrFlB1UiqqgzRklRF\nzZ07l169ehEfH0/t2rW57LLL+Oabb9i8eTOdOnUC4LrrrmPOnDls3ryZrVu3cs455wBwzTXXxLL0\ng5KWlsbYsWNjXYakKs4QLUlVSVYWJCRAXBzcdx8sWxbrispF2Wkpffv2ZcyYMXTu3Jns7GwANm7c\nSEJCAgCzZ8/mkksuAaCgoIDrr7+epKQkkpOTmTZt2m7X3bhxI+eccw6vvfZahfZH0uHPEC1JVUVW\nFgweDPn5EIZ03LSJ6a++yrZnnmHr1q389a9/5dhjj+W4445j7ty5ADz33HN06tSJunXrUqtWLd5/\n/30AXnjhhVj2ZDffNy3lQNx///3UqVOHZcuW8dFHH9GlS5fIvq+++oru3btz33330b1790NRuqQq\nrHqsC5CkAzF27FieeOIJWrVqRVZWVqzLqZxGjIDCwshmK6BPGJIyeDAnTZxIeno6AM8++yw33HAD\nhYWFNGnShGeeeQaAiRMnMmjQIOLi4ujUqRN16tSJRS/2UnZaCsBll112wOfOnDlzt38QHHfccQDs\n2LGDrl27Mm7cuMjUFkmKhiFa0mHh8ccfZ+bMmZx++umxLqXyWrNmr6YRwIidO2HevN3aFy1atNex\nLVq04KOPPgLggQceIC0t7ZCUWV6qV69OcXExANu3b4/63NatW/PWW28ZoiX9KE7nkFTp3XDDDXz+\n+edcfPHF/P73v6dnz54kJyfTrl27SOgbOnQo9913HwBvvfUWHTt2jASsI0bDhtG17+G1114jNTWV\nxMRE5s6dy8iRI8uxuCiVmdvd8cEHmT5pEtu2bYtMSwFISEggJycHgKlTp+7zMhdccAHjxo2LbO9a\nzi8IAp5++mlWrlzJgw8+eGj7IqlKMkRLqvTGjx/PqaeeyqxZs8jLy6Nly5Z89NFH/Pa3v+Xaa68F\n4He/+x1Tpkxh1qxZ3HrrrTzzzDPExR1hf8WNHg2lUx4i4uNL2g9Anz59yM3NZfny5bz22muceOKJ\nh6DIA7DH3O5W//wnff75T1KaNOHiiy+OTEu54447eOKJJ2jZsiUbN27c56VGjhzJpk2bSExMJCUl\nhVmzZkX2VatWjf/93//l3Xff5fHHH6+QrkmqOoIwDGNdw36lpaWFu+7AlnRkSkhIIDs7mwsuuIBp\n06bRpEkTABo0aMDHH39M7dq1WbBgAR07duThhx/mlltuiXHFMZKVVTI3es2akhHo0aMhMzPWVUUn\nIaEkQO+pUSPIy2PUqFHUrFmTO+64o8JLk1T1BUGQE4bhfuezOSdaUpWxbNkyTjjhBL788stYlxI7\nmZmHX2je0z7mdv9guyTFwBH2u05Jh42y6x0nJMA33wDQoUOHyOocs2fPpl69etSuXZv8/Hx+//vf\n8+GHH/LGG29ElmrTYWg/c7tHjRrlKLSkmDNES6p89pgTS34+/OtfMHUqo0aNIicnh+TkZIYPH86z\nzz5LGIb8/Oc/Z8yYMZx66qlMnDiRgQMHRr1igyqJg5zbLUkVwTnRkiqf/cyJ1RGgKsztlnRYOtA5\n0YZoSZVPXFzJCPSeggCOtGXrJEkV6kBDtNM5JFU+B7nesSRJh5ohWlLl45xYSVIlZ4iWVPlkZsKE\nCSVzoIOg5HXCBOfESpIqjZiF6CAILgqC4NMgCD4LgmB4rOqQVEllZpbcRFhcXPJqgD5gmzdvjvoJ\nfP379//eR2dLkvYWkxAdBEE1YBxwMdAc6BsEQfNY1HKkKyoqinUJksrZjwnRkqToxGokug3wWRiG\nn4dh+B3wAtAjRrVUaffffz9Nmzalffv29O3blzFjxtC5c2duu+020tLS+OMf/8iGDRvo3bs36enp\npKenM3/+fAC++eYbBgwYQJs2bWjZsiWvvvoqAJMmTeLyyy/noosu4swzz+TOO++MZRcl7WH48OGs\nXr2a1NRUhg0bxrBhw0hMTCQpKYkpU6YAEIYhQ4YMoWnTppx//vl8/fXXkfPvu+8+0tPTSUxMZPDg\nwYRhyOrVq2nVqlXkmFWrVu22LUlHmliF6NOAtWW2vyhtUzlavHgx06ZNY+nSpbzxxhuUXSbwu+++\nIzs7m//3//4fQ4cO5fbbb48cP3DgQABGjx5Nly5d+OCDD5g1axbDhg3jm9KnxuXm5jJlyhSWLVvG\nlClTWLt27T5rkFTxHnjgAc444wxyc3Np164dubm5LF26lJkzZzJs2DDWr1/PK6+8wqeffsonn3zC\n5MmTWbBgQeT8IUOGsHjxYpYvX862bduYMWMGZ5xxBnXq1CE3NxeAZ555huuvvz5WXZSkmKse6wK+\nTxAEg4HBAA1d1upHmT9/Pj169KBGjRrUqFGDSy+9NLKvT58+kfczZ87kk08+iWz/5z//oaCggLff\nfpu//OUvjBkzBoDt27ezZs0aALp27UqdOnUAaN68Ofn5+TRo0KAiuiUpCvPmzaNv375Uq1aN+vXr\n06lTJxYvXsycOXMi7aeeeipdunSJnDNr1iweeughCgsL+fe//02LFi249NJLGThwIM888wx/+MMf\nmDJlCh988EEMeyZJsRWrEL0OKJu4Ti9tiwjDcAIwAUoetlJxpR3myj7lq25d6Nx5n4cde+yxkffF\nxcUsWrSIGjVq7HZMGIZMmzaNpk2b7tb+/vvvc/TRR0e2q1Wr5txqKdbK/uyfemrJqiY/wvbt27np\nppvIzs6mQYMGjBo1KvL49N69e/PrX/+aLl260Lp1a0444YTy7IEkHVZiNZ1jMXBmEASNgyD4CXA1\n8JcY1VJ1ZGXB4MElj0sOQzI2beKv06ez/ZlnKCgoYMaMGfs87cILL+TRRx+NbO/6dW23bt149NFH\n2fVUyw8//PDQ90FS9Pb42a+1bh1b162DrCw6dOjAlClT2LlzJxs2bGDOnDm0adOGjh07RtrXr1/P\nrFmzACKBuV69ehQUFOy2YkeNGjXo1q0bN954o1M5JB3xYhKiwzAsAoYAbwErgBfDMPw4FrVUKSNG\nQGFhZDMduCwMSR48mIsvvpikpKTIFIyyxo4dS3Z2NsnJyTRv3pzx48cDcO+997Jjxw6Sk5Np0aIF\n9957b0X1RFI09vjZPwHICEMSr7+ehQsXkpycTEpKCl26dOGhhx7i5JNPplevXpx55pk0b96ca6+9\nlnPOOQeAunXrMmjQIBITE+nWrRvp6em7fVRmZiZxcXFceOGFFdlDSap0gl2jjJVZWlpaWPamOH2P\nuDjY48+zAKgZBBQWFNCxY0cmTJjgHfVSVbOPn32gZEpHcXG5ftSYMWPYsmUL999/f7leV5IqiyAI\ncsIwTNvfcZX2xkL9CA0blvw6t4zBwCfVq7O9VSuuu+46A7RUFe3jZz/SXo569erF6tWreffdd8v1\nupJ0ODJEVyWjR5fMiyzza93n4+N9XLJU1e3jZ5/4+JL2cvTKK6+U6/Uk6XAWs8d+6xDIzCwJzI0a\nlfwat1EjA7R0JPBnX5IqnHOiJUmSpFIHOifakWhJkiQpSoZoSZIkKUqGaEmSfkDNmjVjXYKkSsgQ\nLUmSJEXJEH0A8vLySExMjHUZkqQYKigooGvXrrRq1YqkpCReffVVAMaPH09qaiqpqak0btyY8847\nj6effprbbrstcu5TTz3F7bffHqvSJR0Crs5xAPLy8rjkkktYvnz59x6zc+dOqlWrVoFVSZIqQs2a\nNSkoKKCoqIjCwkJq167Nxo0badeuHatWrSIIAgB27NhBly5duPPOOznvvPNISUlh5cqVHHXUUZx7\n7rk8+eSTJCUlxbg3kvbH1TnKWVFREZmZmTRr1owrrriCwsJCEhISuOuuu2jVqhUvvfQSubm5tGvX\njuTkZHr16sWmTZv4+uuvad26NQBLly4lCALWrFkDwBlnnEFhYSH9+/fn1ltv5dxzz6VJkyZMnTo1\nll2VJO1DGIbcc889JCcnc/7557Nu3Tq++uqryP6hQ4fSpUsXLr30UmrWrEmXLl2YMWMGK1euZMeO\nHQZoqYoxRB+gTz/9lJtuuokVK1ZQu3ZtHn/8cQBOOOEElixZwtVXX821117Lgw8+yEcffURSUhK/\n/vWvOemkk9i+fTv/+c9/mDt3LmlpacydO5f8/HxOOukk4uPjAVi/fj3z5s1jxowZDB8+/JD14/um\npnTu3BnX4pakUllZkJAAcXElT4LMyiIrK4sNGzaQk5NDbm4u9evXZ/v27QBMmjSJ/Px8fvWrX0Uu\nMXDgQCZNmsQzzzzD9ddfH6OOSDpUfOz3AWrQoAEZGRkA9OvXj7FjxwLQp08fALZs2cLmzZvp1KkT\nANdddx1XXnklAOeeey7z589nzpw53HPPPbz55puEYUiHDh0i1+/ZsydxcXE0b958t5ENSVIFy8ra\n+zHqgwez5fLLOemkkzjqqKOYNWsW+fn5AOTk5DBmzBjmzp1LXNz/jU21bduWtWvXsmTJEj766KOK\n7oWkQ8yR6O9TdhSifXuCsn+ZQmQO3LHHHrvfS3Xs2DEy+tyjRw+WLl3KvHnzdgvRRx99dOT9oZ6n\nvq+pKWWVXc5p6tSp9O/fH4ANGzbQu3dv0tPTSU9PZ/78+Ye0TkmKiREjdg/QAIWFZM6eTXZ2NklJ\nSUyePJmzzz4bgMcee4x///vfnHfeeaSmpjJw4MDIaVdddRUZGRkcd9xxFdkDSRXAkeh92XMUYt06\n1gALR43inFGjeP7552nfvj0ffvhh5JQ6depw3HHHMXfuXDp06MBzzz0XGZXu0KEDI0aMoGPHjsTF\nxXH88cfz+uuv87vf/S4GnSuZmjJx4kQyMjIYMGBAZGrK/gwdOpTbb7+d9u3bs2bNGrp168aKFSsO\ncbWSVMFK71vZpaD0td66dSxcu3avw5955pnvvdS8efNclUOqohyJ3pd9jEI0BcaNGUOzZs3YtGkT\nN954416nPfvsswwbNozk5GRyc3P55S9/CUBCQgJhGNKxY0cA2rdvT926dWM2MrHn1JR58+Yd0Hkz\nZ85kyJAhpKamctlll/Gf//yHgoKC/Z8oSYeThg2ja9+HzZs3c9ZZZ3HMMcfQtWvXcipMUmXiSPS+\n7DEKkQCshJJgXWbkNS8vb7fjUlNTWbRo0T4vubbM6MU999zDPffcE9meNGnSbseWazDNyir5R8Ga\nNSX/A7jttshUlF1+aHvXTTMAxcXFLFq0iBo1apRffZJU2Ywevfec6Pj4kvYDVLduXf7+978fguIk\nVRaORO9LOYxCVAq7pqXk50MYlrzefTdr1qxh4cKFAJGpKWXVr1+fFStWUFxczCuvvBJpv/DCC3n0\n0Ucj27m5uRXTD0mqSJmZMGECNGoEQVDyOmFCSbsklTJE78vo0SWjDmVFOQpRKezr5pjt22lavTrj\nxo373qkpDzzwAJdccgnnnnsup5xySqR97NixZGdnk5ycTPPmzRk/fnxF9EKSKl5mJuTlQXFxyasB\nWtIefGLh99lzGsTo0YffX6JxcSUj0HsKgpL/MUiSJGk3B/rEQudEf5/MzMMvNO+pYcOSKRz7apck\nSdKP5nSOqqyqTEuRJEmqZAzRVZk3x0iSJB0STueo6qrCtBRJkqRKxpFoSZIkKUqGaEmSJClKhmhJ\nkiQpSoZoSZIkKUqGaEmSJClKhmhJkiQpSoZoVUoJCQls3Lgx1mVIkiTtkyFakiRJipIPW1HMffPN\nN1x11VV88cUX7Ny5k3vvvTeyb9u2bVx++eVcfvnlrF27luOPP57bbrsNgBEjRnDSSScxdOjQWJUu\nSZKOUI5EK+befPNNTj31VJYuXcry5cu56KKLACgoKODSSy+lb9++DBo0iAEDBjB58mQAiouLeeGF\nF+jXr18sS5ckSUcoQ7RiLikpiXfeeYe77rqLuXPnUqdOHQB69OjB9ddfz7XXXguUzJM+4YQT+PDD\nD3n77bdp2bIlJ5xwQixLlyRJRyincyg2srJgxAhYs4azGjZkyd138/qxxzJy5Ei6du0KQEZGBm++\n+SbXXHMNQRAAMHDgQCZNmsQ///lPBgwYEMseSJKkI1gQhmGsa9ivtLS0MDs7O9ZlqLxkZcHgwVBY\nCMCXwPHHHEONp55iRp06/OlPfyI3N5fs7Gzuu+8+ioqKePzxxwH47rvvSEpKYseOHaxatYpq1arF\nsCOSJKmqCYIgJwzDtP0d53QOVbwRIyIBGmAZ0GbbNlKvv55f//rXjBw5MrLvj3/8I9u2bePOO+8E\n4Cc/+QnnnXceV111lQFah52aNWvGugRJUjlxJFoVLy4O9vXfXRBAcfEPnlpcXEyrVq146aWXOPPM\nMw9RgdKhUbNmTQoKCmJdhiTpBzgSrcqrYcPo2kt98skn/PSnP6Vr164GaB3WwjBk2LBhJCYmkpSU\nxJQpUwC4+uqree211yLH9e/fn6lTp7Jz506GDRtGeno6ycnJPPnkk7EqXZJUyhsLVfFGj95tTjQA\n8fEl7T+gefPmfP7554e4OOnQe/nll8nNzWXp0qVs3LiR9PR0OnbsSJ8+fXjxxRfp3r073333HX/7\n29944oknmDhxInXq1GHx4sV8++23ZGRkcOGFF9K4ceNYd0WSjliORKviZWbChAnQqFHJFI5GjUq2\nMzNjXZlUIebNm0ffvn2pVq0a9evXp1OnTixevJiLL76YWbNm8e233/LGG2/QsWNHjjnmGN5++20m\nT55Mamoqbdu25V//+herVq2KdTck6YjmSLRiIzPT0Kyqr8xSjjRsCEVFP3h4jRo16Ny5M2+99RZT\npkzh6quvBkqmfzz66KN069atIqqWJB0AR6Il6VDYtZRjfn7JjbT5+fDtt5CVRYcOHZgyZQo7d+5k\nw4YNzJkzhzZt2gDQp08fnnnmGebOnRt5eme3bt144okn2LFjBwB///vf+eabb2LWNUmSI9GSdGjs\nsZRj2fZe//gHCxcuJCUlhSAIeOihhzj55JMBuPDCC/nZz35Gjx49+MlPfgKUPGQoLy+PVq1aEYYh\nJ554ItOnT6/I3kiS9uASd5J0KBzEUo6SpNhxiTtJiqUfuZSjJOnwYIiWpENh9OiSpRvLOoClHCVJ\nhwdDtCQdCi7lKElVmjcWStKh4lKOklRlORItSZIkRckQLUmSJEXJEC1JkiRFyRAtSZIkRckQLUmS\nJEXJEC1JkiRFyRAtSZIkRemgQnQQBP8TBMHKIAg+CoLglSAI6pbZd3cQBJ8FQfBpEATdyrRfVNr2\nWRAEww/m8yVJkqRYONiR6HeAxDAMk4G/A3cDBEHQHLgaaAFcBDweBEG1IAiqAeOAi4HmQN/SYyWp\n0svLyyMxMfGQXv/555+PbGdnZ3PrrbcC8O2333L++eeTmprKlClTvvcakyZNYsiQIYesRklSiYN6\nYmEYhm+X2VwEXFH6vgfwQhiG3wL/CILgM6BN6b7PwjD8HCAIghdKj/3kYOqQpKpgV4i+5pprAEhL\nSyMtLQ2ADz/8EIDc3NyY1SdJ+j/lOSd6APBG6fvTgLVl9n1R2vZ97ZJ0WCgqKiIzM5NmzZpxxRVX\nUFhYSE5ODp06daJ169Z069aN9evXA/DUU0+Rnp5OSkoKvXv3prCwEID+/fszderUyDVr1qwJwPDh\nw5k7dy6pqak8/PDDzJ49m0suuYSvv/6afv36sXjxYlJTU1m9ejUJCQls3LgRKBmx7ty5c8V+IyTp\nCLffEB0EwcwgCJbv46tHmWNGAEVAVnkVFgTB4CAIsoMgyN6wYUN5XVaSDsqnn37KTTfdxIoVK6hd\nuzbjxo3jlltuYerUqeTk5DBgwABGjBgBwOWXX87ixYtZunQpzZo1Y+LEiT947QceeIAOHTqQm5vL\n7bffHmk/6aST+NOf/hTZd8YZZxzSPkqS9m+/0znCMDz/h/YHQdAfuAToGoZhWNq8DmhQ5rDTS9v4\ngfY9P3cCMAEgLS0t3NcxklTRGjRoQEZGBgD9+vXjt7/9LcuXL+eCCy4AYOfOnZxyyikALF++nJEj\nR7J582YKCgro1q3b915XknR4Oag50UEQXATcCXQKw7CwzK6/AM8HQfAH4FTgTOADIADODIKgMSXh\n+WrgmoOpQZIOqawsGDEC1qyBU08l2L59t921atWiRYsWLFy4cK9T+/fvz/Tp00lJSWHSpEnMnj0b\ngOrVq1NcXAxAcXEx3333XdRllb3G9j1qkiQdegc7J/oxoBbwThAEuUEQjAcIw/Bj4EVKbhh8E7g5\nDMOdYRgWAUOAt4AVwIulx0pS5ZOVBYMHQ34+hCGsW8eaf/2LhaNGAfD888/Trl07NmzYEAnRO3bs\n4OOPS/5a27p1K6eccgo7duwgK+v/ZrslJCSQk5MDwF/+8hd27NgBlATyrVu3HlBpZa8xbdq0cumu\nJOnAHVSIDsPwp2EYNgjDMLX064Yy+0aHYXhGGIZNwzB8o0z762EYnlW6b/TBfL4kHVIjRkBh4W5N\nTYFxY8bQrFkzNm3aFJkPfdddd5GSkkJqaioLFiwA4P7776dt27ZkZGRw9tlnR64xaNAg3nvvPVJS\nUli4cCHHHnvs/2/vfkPurOs4jr8/3mvGiNJlmbi5LZoPZm0kt0uQQDPmNHE9kFjc1KpBJBYLhHL6\nYFAJ9odW0R8YNbAQlpjliKKWWc+mTU1t2p+bwnLMzLYsEJTltwfXb+10u3vzsNPOfc7eL7jZub7X\nj/Ebn/u6znfX9bvOAWDlypVMTEywatUqtm7desypbdmyhU2bNjE5OcnExMRg/92SpOPKkWXMc9fk\n5GTt2bNn2NOQdKo57bTuCvRMCbSlFJKk8ZLkgaqaPN44v/ZbkmZz3nn91SVJpwybaEmazS23wIIF\n/1tbsKCrS5JOaTbRkjSbqSnYtg2WLOmWcCxZ0m1PTQ17ZpKkITuhj7iTpLE3NWXTLEl6Ca9ES5Ik\nSX2yiZYkSZL6ZBMtSZIk9ckmWpIkSeqTTbQkSZLUJ5toSZIkqU820ZIkSVKfbKIlSZKkPtlES5Ik\nSX2yiZYkSZL6lKoa9hyOK8nfgCeGPY8RdhbwzLAnoYEwy/FgjuPDLMeDOY6PQWS5pKped7xBI9FE\n68Qk2VNVk8Oeh06cWY4HcxwfZjkezHF8nMwsXc4hSZIk9ckmWpIkSeqTTfSpYduwJ6CBMcvxYI7j\nwyzHgzmOj5OWpWuiJUmSpD55JVqSJEnqk030GErysSS/TbI3yed66puTTCf5XZIreuprW206yY3D\nmbWOJskNSSrJWW07Sb7SsnokyYU9Yzck+UP72TC8WatXks+34/GRJN9PckbPPo/JEWVGoyXJ4iT3\nJnmsvTduavWFSXa18+auJGe2+qznWg1fkokkDyX5YdteluS+ltd3k8xv9dPb9nTbv3SQ87CJHjNJ\nLgPWAauq6gLgC62+AlgPXACsBb7efgkngK8BVwIrgPe2sRqyJIuBNcCfe8pXAsvbz4eBb7SxC4Et\nwNuA1cCWw28GGrpdwJuraiXwe2AzeEyOMjMaSYeAG6pqBXAxcH3L7EbgnqpaDtzTtmGWc63mjE3A\n4z3bnwW2VtWbgIPAxlbfCBxs9a1t3MDYRI+f64Bbq+p5gKp6utXXATuq6vmq+hMwTddsrQamq+qP\nVfUCsKON1fBtBT4B9D64sA74dnV2A2ckOQe4AthVVQeq6iBd47b2pM9YL1FVP62qQ21zN7CovfaY\nHF1mNGKqan9VPdhe/4uuATuXLrfb2rDbgHe317OdazVkSRYB7wK+2bYDvAO4sw2ZmePhfO8ELm/j\nB8ImevycD7y93bb4ZZKLWv1c4C89455stdnqGqIk64B9VfXwjF3mONo+BPy4vTbL0WVGI6zd0n8r\ncB9wdlXtb7ueAs5ur8147voS3QWmF9v2a4F/9Fys6M3qvzm2/c+28QMxb1B/kU6eJD8D3nCUXTfT\nZbqQ7nbVRcAdSd54Eqenl+k4Od5Et5RDI+BYWVbV3W3MzXS3lG8/mXOTdESSVwHfAz5eVf/svShZ\nVZXEjyybw5JcDTxdVQ8kuXTY87GJHkFV9c7Z9iW5Drirus8uvD/Ji3TfI78PWNwzdFGrcYy6/o9m\nyzHJW4BlwMPtBL8IeDDJambPcR9w6Yz6LwY+aR3VsY5JgCQfAK4GLq8jnyvqMTm6jpWd5qgkr6Br\noG+vqrta+a9Jzqmq/W25xuElkGY8N10CXJPkKuCVwKuBL9Mtt5nXrjb3ZnU4xyeTzANeA/x9UJNx\nOcf4+QFwGUCS84H5wDPATmB9e1J1Gd3DEvcDvwKWtydb59M96LRzKDMXAFX1aFW9vqqWVtVSultT\nF1bVU3TZvL89OX4x8C4yP8sAAAE3SURBVGy7FfkTYE2SM9sDhWtaTUOWZC3drcdrquq5nl0ek6PL\njEZMWwf7LeDxqvpiz66dwOFPM9oA3N1TP9q5VkNUVZuralF7b1wP/LyqpoB7gWvbsJk5Hs732jZ+\nYHcbvBI9frYD25P8BngB2NB+YfYmuQN4jO6W8vVV9W+AJB+la7gmgO1VtXc4U9fL8CPgKrqH0J4D\nPghQVQeSfJruzR3gU1V1YDhT1AxfBU4HdrU7C7ur6iNV5TE5oqrqkBmNnEuA9wGPJvl1q90E3Eq3\n7HEj8ATwnrbvqOdazVmfBHYk+QzwEN1/mGh/fifJNHCArvEeGL+xUJIkSeqTyzkkSZKkPtlES5Ik\nSX2yiZYkSZL6ZBMtSZIk9ckmWpIkSeqTTbQkSZLUJ5toSZIkqU820ZIkSVKf/gPaPplGScU8bAAA\nAABJRU5ErkJggg==\n",
      "text/plain": [
       "<Figure size 864x432 with 1 Axes>"
      ]
     },
     "metadata": {
      "tags": []
     },
     "output_type": "display_data"
    }
   ],
   "source": [
    "tsne = TSNE(n_components=2, random_state=42, n_iter=5000, perplexity=3)\n",
    "np.set_printoptions(suppress=True)\n",
    "T = tsne.fit_transform(word_glove_vectors)\n",
    "labels = unique_words\n",
    "\n",
    "plt.figure(figsize=(12, 6))\n",
    "plt.scatter(T[:, 0], T[:, 1], c='red', edgecolors='r')\n",
    "for label, x, y in zip(labels, T[:, 0], T[:, 1]):\n",
    "    plt.annotate(label, xy=(x+1, y+1), xytext=(0, 0), textcoords='offset points')"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {
    "colab_type": "text",
    "id": "Ce6BV5bFRiq7"
   },
   "source": [
    "### Looking at term semantic similarity"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 14,
   "metadata": {
    "colab": {
     "base_uri": "https://localhost:8080/",
     "height": 689
    },
    "colab_type": "code",
    "id": "fyO3JzS2Riq8",
    "outputId": "9b88675d-eb0f-4693-b318-a43bddcaeaf2"
   },
   "outputs": [
    {
     "data": {
      "text/html": [
       "<div>\n",
       "<style scoped>\n",
       "    .dataframe tbody tr th:only-of-type {\n",
       "        vertical-align: middle;\n",
       "    }\n",
       "\n",
       "    .dataframe tbody tr th {\n",
       "        vertical-align: top;\n",
       "    }\n",
       "\n",
       "    .dataframe thead th {\n",
       "        text-align: right;\n",
       "    }\n",
       "</style>\n",
       "<table border=\"1\" class=\"dataframe\">\n",
       "  <thead>\n",
       "    <tr style=\"text-align: right;\">\n",
       "      <th></th>\n",
       "      <th>sausages</th>\n",
       "      <th>lazy</th>\n",
       "      <th>beautiful</th>\n",
       "      <th>blue</th>\n",
       "      <th>fox</th>\n",
       "      <th>jumps</th>\n",
       "      <th>dog</th>\n",
       "      <th>breakfast</th>\n",
       "      <th>today</th>\n",
       "      <th>green</th>\n",
       "      <th>kings</th>\n",
       "      <th>sky</th>\n",
       "      <th>brown</th>\n",
       "      <th>bacon</th>\n",
       "      <th>eggs</th>\n",
       "      <th>beans</th>\n",
       "      <th>toast</th>\n",
       "      <th>love</th>\n",
       "      <th>ham</th>\n",
       "      <th>quick</th>\n",
       "    </tr>\n",
       "  </thead>\n",
       "  <tbody>\n",
       "    <tr>\n",
       "      <th>sausages</th>\n",
       "      <td>1.000000</td>\n",
       "      <td>0.168205</td>\n",
       "      <td>0.114540</td>\n",
       "      <td>0.097443</td>\n",
       "      <td>0.133348</td>\n",
       "      <td>0.083306</td>\n",
       "      <td>0.247529</td>\n",
       "      <td>0.415231</td>\n",
       "      <td>0.094879</td>\n",
       "      <td>0.233169</td>\n",
       "      <td>0.070071</td>\n",
       "      <td>0.061753</td>\n",
       "      <td>0.289625</td>\n",
       "      <td>0.729162</td>\n",
       "      <td>0.548310</td>\n",
       "      <td>0.546647</td>\n",
       "      <td>0.498062</td>\n",
       "      <td>0.146917</td>\n",
       "      <td>0.622770</td>\n",
       "      <td>0.162966</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>lazy</th>\n",
       "      <td>0.168205</td>\n",
       "      <td>1.000000</td>\n",
       "      <td>0.294308</td>\n",
       "      <td>0.230513</td>\n",
       "      <td>0.267240</td>\n",
       "      <td>0.222112</td>\n",
       "      <td>0.301678</td>\n",
       "      <td>0.319941</td>\n",
       "      <td>0.286802</td>\n",
       "      <td>0.230945</td>\n",
       "      <td>0.190515</td>\n",
       "      <td>0.259361</td>\n",
       "      <td>0.256875</td>\n",
       "      <td>0.264376</td>\n",
       "      <td>0.236769</td>\n",
       "      <td>0.245672</td>\n",
       "      <td>0.231329</td>\n",
       "      <td>0.339280</td>\n",
       "      <td>0.222571</td>\n",
       "      <td>0.400060</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>beautiful</th>\n",
       "      <td>0.114540</td>\n",
       "      <td>0.294308</td>\n",
       "      <td>1.000000</td>\n",
       "      <td>0.461366</td>\n",
       "      <td>0.210050</td>\n",
       "      <td>0.142554</td>\n",
       "      <td>0.280658</td>\n",
       "      <td>0.314472</td>\n",
       "      <td>0.314184</td>\n",
       "      <td>0.392392</td>\n",
       "      <td>0.158615</td>\n",
       "      <td>0.428081</td>\n",
       "      <td>0.355700</td>\n",
       "      <td>0.151157</td>\n",
       "      <td>0.214437</td>\n",
       "      <td>0.165268</td>\n",
       "      <td>0.189482</td>\n",
       "      <td>0.594738</td>\n",
       "      <td>0.110936</td>\n",
       "      <td>0.289178</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>blue</th>\n",
       "      <td>0.097443</td>\n",
       "      <td>0.230513</td>\n",
       "      <td>0.461366</td>\n",
       "      <td>1.000000</td>\n",
       "      <td>0.371178</td>\n",
       "      <td>0.187263</td>\n",
       "      <td>0.314065</td>\n",
       "      <td>0.197103</td>\n",
       "      <td>0.180088</td>\n",
       "      <td>0.764083</td>\n",
       "      <td>0.207656</td>\n",
       "      <td>0.627800</td>\n",
       "      <td>0.683011</td>\n",
       "      <td>0.242987</td>\n",
       "      <td>0.297546</td>\n",
       "      <td>0.261304</td>\n",
       "      <td>0.232264</td>\n",
       "      <td>0.364961</td>\n",
       "      <td>0.185452</td>\n",
       "      <td>0.220331</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>fox</th>\n",
       "      <td>0.133348</td>\n",
       "      <td>0.267240</td>\n",
       "      <td>0.210050</td>\n",
       "      <td>0.371178</td>\n",
       "      <td>1.000000</td>\n",
       "      <td>0.250834</td>\n",
       "      <td>0.485855</td>\n",
       "      <td>0.165064</td>\n",
       "      <td>0.150072</td>\n",
       "      <td>0.323800</td>\n",
       "      <td>0.226735</td>\n",
       "      <td>0.303459</td>\n",
       "      <td>0.406912</td>\n",
       "      <td>0.210082</td>\n",
       "      <td>0.247314</td>\n",
       "      <td>0.118537</td>\n",
       "      <td>0.163371</td>\n",
       "      <td>0.255050</td>\n",
       "      <td>0.209454</td>\n",
       "      <td>0.192686</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>jumps</th>\n",
       "      <td>0.083306</td>\n",
       "      <td>0.222112</td>\n",
       "      <td>0.142554</td>\n",
       "      <td>0.187263</td>\n",
       "      <td>0.250834</td>\n",
       "      <td>1.000000</td>\n",
       "      <td>0.307961</td>\n",
       "      <td>0.103865</td>\n",
       "      <td>0.142099</td>\n",
       "      <td>0.150752</td>\n",
       "      <td>0.086659</td>\n",
       "      <td>0.278595</td>\n",
       "      <td>0.110400</td>\n",
       "      <td>0.091021</td>\n",
       "      <td>0.150622</td>\n",
       "      <td>0.095544</td>\n",
       "      <td>0.119099</td>\n",
       "      <td>0.185114</td>\n",
       "      <td>0.046432</td>\n",
       "      <td>0.301096</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>dog</th>\n",
       "      <td>0.247529</td>\n",
       "      <td>0.301678</td>\n",
       "      <td>0.280658</td>\n",
       "      <td>0.314065</td>\n",
       "      <td>0.485855</td>\n",
       "      <td>0.307961</td>\n",
       "      <td>1.000000</td>\n",
       "      <td>0.295957</td>\n",
       "      <td>0.223730</td>\n",
       "      <td>0.272693</td>\n",
       "      <td>0.156674</td>\n",
       "      <td>0.219087</td>\n",
       "      <td>0.341204</td>\n",
       "      <td>0.295123</td>\n",
       "      <td>0.291202</td>\n",
       "      <td>0.230778</td>\n",
       "      <td>0.182994</td>\n",
       "      <td>0.358715</td>\n",
       "      <td>0.262579</td>\n",
       "      <td>0.299892</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>breakfast</th>\n",
       "      <td>0.415231</td>\n",
       "      <td>0.319941</td>\n",
       "      <td>0.314472</td>\n",
       "      <td>0.197103</td>\n",
       "      <td>0.165064</td>\n",
       "      <td>0.103865</td>\n",
       "      <td>0.295957</td>\n",
       "      <td>1.000000</td>\n",
       "      <td>0.288788</td>\n",
       "      <td>0.288437</td>\n",
       "      <td>0.161841</td>\n",
       "      <td>0.205429</td>\n",
       "      <td>0.277158</td>\n",
       "      <td>0.487737</td>\n",
       "      <td>0.431108</td>\n",
       "      <td>0.378215</td>\n",
       "      <td>0.513436</td>\n",
       "      <td>0.245735</td>\n",
       "      <td>0.371688</td>\n",
       "      <td>0.358221</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>today</th>\n",
       "      <td>0.094879</td>\n",
       "      <td>0.286802</td>\n",
       "      <td>0.314184</td>\n",
       "      <td>0.180088</td>\n",
       "      <td>0.150072</td>\n",
       "      <td>0.142099</td>\n",
       "      <td>0.223730</td>\n",
       "      <td>0.288788</td>\n",
       "      <td>1.000000</td>\n",
       "      <td>0.246754</td>\n",
       "      <td>0.204743</td>\n",
       "      <td>0.256770</td>\n",
       "      <td>0.147418</td>\n",
       "      <td>0.158949</td>\n",
       "      <td>0.183048</td>\n",
       "      <td>0.147652</td>\n",
       "      <td>0.174257</td>\n",
       "      <td>0.371650</td>\n",
       "      <td>0.104115</td>\n",
       "      <td>0.370023</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>green</th>\n",
       "      <td>0.233169</td>\n",
       "      <td>0.230945</td>\n",
       "      <td>0.392392</td>\n",
       "      <td>0.764083</td>\n",
       "      <td>0.323800</td>\n",
       "      <td>0.150752</td>\n",
       "      <td>0.272693</td>\n",
       "      <td>0.288437</td>\n",
       "      <td>0.246754</td>\n",
       "      <td>1.000000</td>\n",
       "      <td>0.151519</td>\n",
       "      <td>0.488385</td>\n",
       "      <td>0.646850</td>\n",
       "      <td>0.386664</td>\n",
       "      <td>0.392072</td>\n",
       "      <td>0.462498</td>\n",
       "      <td>0.287608</td>\n",
       "      <td>0.327302</td>\n",
       "      <td>0.290761</td>\n",
       "      <td>0.287504</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>kings</th>\n",
       "      <td>0.070071</td>\n",
       "      <td>0.190515</td>\n",
       "      <td>0.158615</td>\n",
       "      <td>0.207656</td>\n",
       "      <td>0.226735</td>\n",
       "      <td>0.086659</td>\n",
       "      <td>0.156674</td>\n",
       "      <td>0.161841</td>\n",
       "      <td>0.204743</td>\n",
       "      <td>0.151519</td>\n",
       "      <td>1.000000</td>\n",
       "      <td>0.263785</td>\n",
       "      <td>0.193487</td>\n",
       "      <td>0.146716</td>\n",
       "      <td>0.118818</td>\n",
       "      <td>0.092759</td>\n",
       "      <td>0.125741</td>\n",
       "      <td>0.263805</td>\n",
       "      <td>0.127651</td>\n",
       "      <td>0.127679</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>sky</th>\n",
       "      <td>0.061753</td>\n",
       "      <td>0.259361</td>\n",
       "      <td>0.428081</td>\n",
       "      <td>0.627800</td>\n",
       "      <td>0.303459</td>\n",
       "      <td>0.278595</td>\n",
       "      <td>0.219087</td>\n",
       "      <td>0.205429</td>\n",
       "      <td>0.256770</td>\n",
       "      <td>0.488385</td>\n",
       "      <td>0.263785</td>\n",
       "      <td>1.000000</td>\n",
       "      <td>0.408844</td>\n",
       "      <td>0.172964</td>\n",
       "      <td>0.223585</td>\n",
       "      <td>0.160064</td>\n",
       "      <td>0.216372</td>\n",
       "      <td>0.351084</td>\n",
       "      <td>0.174496</td>\n",
       "      <td>0.192515</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>brown</th>\n",
       "      <td>0.289625</td>\n",
       "      <td>0.256875</td>\n",
       "      <td>0.355700</td>\n",
       "      <td>0.683011</td>\n",
       "      <td>0.406912</td>\n",
       "      <td>0.110400</td>\n",
       "      <td>0.341204</td>\n",
       "      <td>0.277158</td>\n",
       "      <td>0.147418</td>\n",
       "      <td>0.646850</td>\n",
       "      <td>0.193487</td>\n",
       "      <td>0.408844</td>\n",
       "      <td>1.000000</td>\n",
       "      <td>0.432891</td>\n",
       "      <td>0.466903</td>\n",
       "      <td>0.453427</td>\n",
       "      <td>0.347578</td>\n",
       "      <td>0.331943</td>\n",
       "      <td>0.335111</td>\n",
       "      <td>0.217009</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>bacon</th>\n",
       "      <td>0.729162</td>\n",
       "      <td>0.264376</td>\n",
       "      <td>0.151157</td>\n",
       "      <td>0.242987</td>\n",
       "      <td>0.210082</td>\n",
       "      <td>0.091021</td>\n",
       "      <td>0.295123</td>\n",
       "      <td>0.487737</td>\n",
       "      <td>0.158949</td>\n",
       "      <td>0.386664</td>\n",
       "      <td>0.146716</td>\n",
       "      <td>0.172964</td>\n",
       "      <td>0.432891</td>\n",
       "      <td>1.000000</td>\n",
       "      <td>0.620539</td>\n",
       "      <td>0.617909</td>\n",
       "      <td>0.622701</td>\n",
       "      <td>0.292268</td>\n",
       "      <td>0.738816</td>\n",
       "      <td>0.265370</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>eggs</th>\n",
       "      <td>0.548310</td>\n",
       "      <td>0.236769</td>\n",
       "      <td>0.214437</td>\n",
       "      <td>0.297546</td>\n",
       "      <td>0.247314</td>\n",
       "      <td>0.150622</td>\n",
       "      <td>0.291202</td>\n",
       "      <td>0.431108</td>\n",
       "      <td>0.183048</td>\n",
       "      <td>0.392072</td>\n",
       "      <td>0.118818</td>\n",
       "      <td>0.223585</td>\n",
       "      <td>0.466903</td>\n",
       "      <td>0.620539</td>\n",
       "      <td>1.000000</td>\n",
       "      <td>0.585053</td>\n",
       "      <td>0.495935</td>\n",
       "      <td>0.254177</td>\n",
       "      <td>0.489116</td>\n",
       "      <td>0.245487</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>beans</th>\n",
       "      <td>0.546647</td>\n",
       "      <td>0.245672</td>\n",
       "      <td>0.165268</td>\n",
       "      <td>0.261304</td>\n",
       "      <td>0.118537</td>\n",
       "      <td>0.095544</td>\n",
       "      <td>0.230778</td>\n",
       "      <td>0.378215</td>\n",
       "      <td>0.147652</td>\n",
       "      <td>0.462498</td>\n",
       "      <td>0.092759</td>\n",
       "      <td>0.160064</td>\n",
       "      <td>0.453427</td>\n",
       "      <td>0.617909</td>\n",
       "      <td>0.585053</td>\n",
       "      <td>1.000000</td>\n",
       "      <td>0.449284</td>\n",
       "      <td>0.249891</td>\n",
       "      <td>0.495773</td>\n",
       "      <td>0.251534</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>toast</th>\n",
       "      <td>0.498062</td>\n",
       "      <td>0.231329</td>\n",
       "      <td>0.189482</td>\n",
       "      <td>0.232264</td>\n",
       "      <td>0.163371</td>\n",
       "      <td>0.119099</td>\n",
       "      <td>0.182994</td>\n",
       "      <td>0.513436</td>\n",
       "      <td>0.174257</td>\n",
       "      <td>0.287608</td>\n",
       "      <td>0.125741</td>\n",
       "      <td>0.216372</td>\n",
       "      <td>0.347578</td>\n",
       "      <td>0.622701</td>\n",
       "      <td>0.495935</td>\n",
       "      <td>0.449284</td>\n",
       "      <td>1.000000</td>\n",
       "      <td>0.274246</td>\n",
       "      <td>0.500586</td>\n",
       "      <td>0.292624</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>love</th>\n",
       "      <td>0.146917</td>\n",
       "      <td>0.339280</td>\n",
       "      <td>0.594738</td>\n",
       "      <td>0.364961</td>\n",
       "      <td>0.255050</td>\n",
       "      <td>0.185114</td>\n",
       "      <td>0.358715</td>\n",
       "      <td>0.245735</td>\n",
       "      <td>0.371650</td>\n",
       "      <td>0.327302</td>\n",
       "      <td>0.263805</td>\n",
       "      <td>0.351084</td>\n",
       "      <td>0.331943</td>\n",
       "      <td>0.292268</td>\n",
       "      <td>0.254177</td>\n",
       "      <td>0.249891</td>\n",
       "      <td>0.274246</td>\n",
       "      <td>1.000000</td>\n",
       "      <td>0.218128</td>\n",
       "      <td>0.292446</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>ham</th>\n",
       "      <td>0.622770</td>\n",
       "      <td>0.222571</td>\n",
       "      <td>0.110936</td>\n",
       "      <td>0.185452</td>\n",
       "      <td>0.209454</td>\n",
       "      <td>0.046432</td>\n",
       "      <td>0.262579</td>\n",
       "      <td>0.371688</td>\n",
       "      <td>0.104115</td>\n",
       "      <td>0.290761</td>\n",
       "      <td>0.127651</td>\n",
       "      <td>0.174496</td>\n",
       "      <td>0.335111</td>\n",
       "      <td>0.738816</td>\n",
       "      <td>0.489116</td>\n",
       "      <td>0.495773</td>\n",
       "      <td>0.500586</td>\n",
       "      <td>0.218128</td>\n",
       "      <td>1.000000</td>\n",
       "      <td>0.191665</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>quick</th>\n",
       "      <td>0.162966</td>\n",
       "      <td>0.400060</td>\n",
       "      <td>0.289178</td>\n",
       "      <td>0.220331</td>\n",
       "      <td>0.192686</td>\n",
       "      <td>0.301096</td>\n",
       "      <td>0.299892</td>\n",
       "      <td>0.358221</td>\n",
       "      <td>0.370023</td>\n",
       "      <td>0.287504</td>\n",
       "      <td>0.127679</td>\n",
       "      <td>0.192515</td>\n",
       "      <td>0.217009</td>\n",
       "      <td>0.265370</td>\n",
       "      <td>0.245487</td>\n",
       "      <td>0.251534</td>\n",
       "      <td>0.292624</td>\n",
       "      <td>0.292446</td>\n",
       "      <td>0.191665</td>\n",
       "      <td>1.000000</td>\n",
       "    </tr>\n",
       "  </tbody>\n",
       "</table>\n",
       "</div>"
      ],
      "text/plain": [
       "           sausages      lazy  beautiful  ...      love       ham     quick\n",
       "sausages   1.000000  0.168205   0.114540  ...  0.146917  0.622770  0.162966\n",
       "lazy       0.168205  1.000000   0.294308  ...  0.339280  0.222571  0.400060\n",
       "beautiful  0.114540  0.294308   1.000000  ...  0.594738  0.110936  0.289178\n",
       "blue       0.097443  0.230513   0.461366  ...  0.364961  0.185452  0.220331\n",
       "fox        0.133348  0.267240   0.210050  ...  0.255050  0.209454  0.192686\n",
       "jumps      0.083306  0.222112   0.142554  ...  0.185114  0.046432  0.301096\n",
       "dog        0.247529  0.301678   0.280658  ...  0.358715  0.262579  0.299892\n",
       "breakfast  0.415231  0.319941   0.314472  ...  0.245735  0.371688  0.358221\n",
       "today      0.094879  0.286802   0.314184  ...  0.371650  0.104115  0.370023\n",
       "green      0.233169  0.230945   0.392392  ...  0.327302  0.290761  0.287504\n",
       "kings      0.070071  0.190515   0.158615  ...  0.263805  0.127651  0.127679\n",
       "sky        0.061753  0.259361   0.428081  ...  0.351084  0.174496  0.192515\n",
       "brown      0.289625  0.256875   0.355700  ...  0.331943  0.335111  0.217009\n",
       "bacon      0.729162  0.264376   0.151157  ...  0.292268  0.738816  0.265370\n",
       "eggs       0.548310  0.236769   0.214437  ...  0.254177  0.489116  0.245487\n",
       "beans      0.546647  0.245672   0.165268  ...  0.249891  0.495773  0.251534\n",
       "toast      0.498062  0.231329   0.189482  ...  0.274246  0.500586  0.292624\n",
       "love       0.146917  0.339280   0.594738  ...  1.000000  0.218128  0.292446\n",
       "ham        0.622770  0.222571   0.110936  ...  0.218128  1.000000  0.191665\n",
       "quick      0.162966  0.400060   0.289178  ...  0.292446  0.191665  1.000000\n",
       "\n",
       "[20 rows x 20 columns]"
      ]
     },
     "execution_count": 14,
     "metadata": {
      "tags": []
     },
     "output_type": "execute_result"
    }
   ],
   "source": [
    "import numpy as np\n",
    "from sklearn.metrics.pairwise import cosine_similarity\n",
    "\n",
    "similarity_matrix = cosine_similarity(vec_df.values)\n",
    "similarity_df = pd.DataFrame(similarity_matrix, index=unique_words, columns=unique_words)\n",
    "similarity_df"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 15,
   "metadata": {
    "colab": {
     "base_uri": "https://localhost:8080/",
     "height": 374
    },
    "colab_type": "code",
    "id": "JC-aQ1IERiq9",
    "outputId": "caead4fe-3923-423f-9b0f-fbe71bcc6097"
   },
   "outputs": [
    {
     "data": {
      "text/plain": [
       "sausages           [bacon, ham, eggs]\n",
       "lazy         [quick, love, breakfast]\n",
       "beautiful           [love, blue, sky]\n",
       "blue              [green, brown, sky]\n",
       "fox                [dog, brown, blue]\n",
       "jumps               [dog, quick, sky]\n",
       "dog                [fox, love, brown]\n",
       "breakfast        [toast, bacon, eggs]\n",
       "today        [love, quick, beautiful]\n",
       "green              [blue, brown, sky]\n",
       "kings                [love, sky, fox]\n",
       "sky          [blue, green, beautiful]\n",
       "brown             [blue, green, eggs]\n",
       "bacon          [ham, sausages, toast]\n",
       "eggs         [bacon, beans, sausages]\n",
       "beans         [bacon, eggs, sausages]\n",
       "toast         [bacon, breakfast, ham]\n",
       "love         [beautiful, today, blue]\n",
       "ham          [bacon, sausages, toast]\n",
       "quick        [lazy, today, breakfast]\n",
       "dtype: object"
      ]
     },
     "execution_count": 15,
     "metadata": {
      "tags": []
     },
     "output_type": "execute_result"
    }
   ],
   "source": [
    "feature_names = np.array(unique_words)\n",
    "similarity_df.apply(lambda row: feature_names[np.argsort(-row.values)[1:4]], \n",
    "                    axis=1)"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {
    "colab_type": "text",
    "id": "pU1BAEUWRiq_"
   },
   "source": [
    "# The FastText Model\n",
    "\n",
    "The FastText model was first introduced by Facebook in 2016 as an extension and supposedly improvement of the vanilla Word2Vec model. Based on the original paper titled _[‘Enriching Word Vectors with Subword Information’](https://arxiv.org/pdf/1607.04606.pdf)_ by Mikolov et al. which is an excellent read to gain an in-depth understanding of how this model works. Overall, FastText is a framework for learning word representations and also performing robust, fast and accurate text classification. The framework is open-sourced by Facebook on [GitHub](https://github.com/facebookresearch/fastText) and claims to have the following.\n",
    "\n",
    "- Recent state-of-the-art English word vectors.\n",
    "- Word vectors for 157 languages trained on Wikipedia and Crawl.\n",
    "- Models for language identification and various supervised tasks.\n",
    "\n",
    "Though I haven't implemented this model from scratch, based on the research paper, following is what I learnt about how the model works. In general, predictive models like the Word2Vec model typically considers each word as a distinct entity (e.g. where) and generates a dense embedding for the word. However this poses to be a serious limitation with languages having massive vocabularies and many rare words which may not occur a lot in different corpora. \n",
    "\n",
    "The Word2Vec model typically ignores the morphological structure of each word and considers a word as a single entity. The FastText model ___considers each word as a Bag of Character n-grams___. This is also called as a ___subword model___ in the paper.\n",
    "\n",
    "We add special boundary symbols __<__ and __>__ at the beginning and end of words. This enables us to distinguish prefixes and suffixes from other character sequences. We also include the word __w__ itself in the set of its n-grams, to learn a representation for each word (in addition to its character n-grams). \n",
    "\n",
    "Taking the word where and __n=3 (tri-grams)__ as an example, it will be represented by the __character n-grams__: __<wh, whe, her, ere, re>__ and the special sequence __< where >__ representing the whole word. Note that the sequence , corresponding to the word __< her >__ is different from the tri-gram __her__ from the word __where__.\n",
    "\n",
    "In practice, the paper recommends in extracting all the n-grams for __n ≥ 3__ and __n ≤ 6__. This is a very simple approach, and different sets of n-grams could be considered, for example taking all prefixes and suffixes. We typically associate a vector representation (embedding) to each n-gram for a word. \n",
    "\n",
    "Thus, we can represent a word by the sum of the vector representations of its n-grams or the average of the embedding of these n-grams. Thus, due to this effect of leveraging n-grams from individual words based on their characters, there is a higher chance for rare words to get a good representation since their character based n-grams should occur across other words of the corpus."
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {
    "colab_type": "text",
    "id": "ZTpq6CyxRiq_"
   },
   "source": [
    "# Robust FastText Model with Gensim\n",
    "\n",
    "The __`gensim`__ package has nice wrappers providing us interfaces to leverage the FastText model available under the `gensim.models.fasttext` module. Let’s apply this once again on our toy corpus."
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 16,
   "metadata": {
    "colab": {
     "base_uri": "https://localhost:8080/",
     "height": 34
    },
    "colab_type": "code",
    "id": "qzrr2AiJRirA",
    "outputId": "953c4f62-98e4-42fc-f68d-dda2bc6c5989"
   },
   "outputs": [
    {
     "data": {
      "text/plain": [
       "<gensim.models.fasttext.FastText at 0x7fa2752d01d0>"
      ]
     },
     "execution_count": 16,
     "metadata": {
      "tags": []
     },
     "output_type": "execute_result"
    }
   ],
   "source": [
    "from gensim.models.fasttext import FastText\n",
    "\n",
    "# Set values for various parameters\n",
    "feature_size = 15    # Word vector dimensionality  \n",
    "window_context = 20  # Context window size                                                                                    \n",
    "min_word_count = 1   # Minimum word count                        \n",
    "sample = 1e-3        # Downsample setting for frequent words\n",
    "sg = 1               # skip-gram model\n",
    "\n",
    "ft_model = FastText(tokenized_corpus, size=feature_size, \n",
    "                     window=window_context, min_count = min_word_count,\n",
    "                     sg=sg, sample=sample, iter=5000)\n",
    "ft_model"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 17,
   "metadata": {
    "colab": {
     "base_uri": "https://localhost:8080/",
     "height": 378
    },
    "colab_type": "code",
    "id": "iR6sYaASRirB",
    "outputId": "233613d4-3851-4b02-dc33-1ff52f081948"
   },
   "outputs": [
    {
     "data": {
      "image/png": "iVBORw0KGgoAAAANSUhEUgAAAuAAAAFpCAYAAADdiZ2EAAAABHNCSVQICAgIfAhkiAAAAAlwSFlz\nAAALEgAACxIB0t1+/AAAADl0RVh0U29mdHdhcmUAbWF0cGxvdGxpYiB2ZXJzaW9uIDMuMC4zLCBo\ndHRwOi8vbWF0cGxvdGxpYi5vcmcvnQurowAAIABJREFUeJzs3XlclWX+//H3BZaGe7lkKRw0N+QA\nIqi5oaaZk4pLi4rj9lMnzdFxyq8WzkQWrc7YWJpLjU6FX/2mZWVlM06SaFaAHfcsF6DMKS1MCTWB\n6/cHeEZNywXuw/J6Ph48POc697nvz33P6cybm2sx1loBAAAAcIafrwsAAAAAKhICOAAAAOAgAjgA\nAADgIAI4AAAA4CACOAAAAOAgAjgAAADgIAI4AAAA4CACOAAAAOAgAjgAAADgIAI4AAAA4KBKvi7g\nYtWpU8e6XC5flwEAAIByLD09/bC1tm5JHqPMBHCXy6W0tDRflwEAAIByzBiTWdLHoAsKAAAA4CAC\nOAAAAOAgAjgAAADgIAI4AAAolTIyMhQaGurrMoBiRwAHAAAAHEQABwAApVZ+fr7Gjh2rVq1a6dZb\nb9Xx48e1aNEiRUdHKzw8XIMGDVJubq4kaeTIkRo/frzat2+vxo0bKzk5WaNHj1bLli01cuRI354I\ncAYCOABUUCX95/2MjAwtXbrU+zwtLU2TJk2SJJ08eVI9evRQRESEli9ffsF9LFmyRBMnTiyxGlH6\nffHFF7r33nu1Y8cO1apVSytXrtTAgQOVmpqqLVu2qGXLlnrxxRe922dnZ2vTpk2aPXu2+vXrpylT\npmjHjh3atm2bPB6PD88E+K8yMw84AKBsOR3Ahw4dKkmKiopSVFSUJOnTTz+VJAIRflVwcLAiIiIk\nSW3atFFGRoa2b9+uGTNm6MiRI8rJyVGvXr282/ft21fGGLndbtWvX19ut1uS1KpVK2VkZHj3BfgS\nd8ABoALLy8tTXFycWrZsqTvuuEO5ublKT09XTEyM2rRpo169eungwYOS9It/9l+xYoV3n9WqVZMk\nTZ8+XSkpKYqIiNDs2bOVnJysPn366Ntvv9WwYcOUmpqqiIgI7d27Vy6XS4cPH5ZUeKe8a9euzl4I\nlFqVK1f2Pvb391deXp5Gjhyp5557Ttu2bdNDDz2kEydO/Gx7Pz+/s97r5+envLw85woHfgEBHAAq\nsN27d2vChAnatWuXatSooblz5+r3v/+9VqxYofT0dI0ePVrx8fGS9It/9j+fJ554Qp07d5bH49GU\nKVO87fXq1dMLL7zgfa1JkyYleo4oO5KWJsnVzCU/fz+5mrm06o1V593u2LFjatCggU6dOqWkpCSH\nqwSuHF1QAKACa9SokTp27ChJGjZsmB577DFt375dPXv2lFQ4AK5BgwaS9It/9geuVNLSJI2bMk65\nvXOlwVJmVqYeeOQB1bmmzs+2feSRR9SuXTvVrVtX7dq107Fjx3xQMXD5COAAUIEZY856Xr16dbVq\n1UqbNm362bYjR47UqlWrFB4eriVLlig5OVmSVKlSJRUUFEiSCgoK9NNPP11yHWfu48zuBKg44hPi\nC8N3cFFDsHSizwn5f+jv3eb+++/3Ph4/fvzP9rFkyRLvY5fLpe3bt5/3NcDX6IICABVYVlaWN2wv\nXbpU7du316FDh7xtp06d0o4dOyRd+M/+LpdL6enpkqQ333xTp06dklQY5i/2zuSZ+1i5cmXxnBzK\nlKy9WVLgOY2BRe1AOVPiAdwYk2GM2WaM8Rhj0orarjXG/MsY80XRv7VLug4AqMjO7VubtLQwQDdv\n3lxz585Vy5YtlZ2d7e3/PW3aNIWHhysiIkIffvihpP/+2b9jx45q0aKFd99jx47VBx98oPDwcG3a\ntElVq1aVJIWFhcnf31/h4eGaPXv2L9b30EMPafLkyYqKipK/v/8vbovyKbBJoHRu1s4qagfKGWOt\nLdkDGJMhKcpae/iMtqckfW+tfcIYM11SbWvttF/aT1RUlE1LSyvRWgGgPDqrb22gpCwp4N0ALZy9\nUHFD43xdHiCJzylKD2NMurU2qkSP4aMAvltSV2vtQWNMA0nJ1trmv7QfAjgAXB5XM5cyO2T+t2+t\nJO2Xgj4MUsbnGb4qC/iZpKVJik+IV9beLAU2CVRiQiLhG44rLwF8v6RsSVbSAmvtQmPMEWttraLX\njaTs088vhAAOAJfHz99PNt5KZ/bsyJdMolFBfoHP6gKA0siJAO7EIMxO1tpISb0l3WuM6XLmi7bw\nN4Dz/hZgjBlnjEkzxqQdOnTIgVIBwLeOHDmiefPmXdJ7zl0I51z0rQWA0qXEA7i19kDRv99Kel1S\nW0nfFHU9UdG/317gvQuttVHW2qi6deuWdKkA4HOXE8B/TWJCogLeDZD2S8qXtL+wb21iQmKxHgdA\n+ZCRkaHQ0FBfl1GulWgAN8ZUNcZUP/1Y0q2Stkt6U9KIos1GSHqjJOsAgLJi+vTp2rt3ryIiIjR1\n6lRNnTpVoaGhcrvdWr58uSTJWquJEyeqefPm6tGjh7799r/3MGbOnKno6GiFhoZq3Lhxstaqfbv2\nqhtQV0EfBskkGt2QfIPqBtSlby0A+EhJ3wGvL2mDMWaLpE8kvW2tXSPpCUk9jTFfSOpR9BwAKrwn\nnnhCTZo0kcfjUfv27eXxeLRlyxatXbtWU6dO1cGDB/X6669r9+7d2rlzp1566SXvNIGSNHHiRKWm\npmr79u06fvy4Vq9erSZNmijYFaxV/7dKBfkFGjF0hO77430+PEsApV1eXp7i4uLUsmVL3XHHHcrN\nzVV6erpiYmLUpk0b9erVSwcPHpQkLVq0SNHR0QoPD9egQYOUm5srqbB73KRJk9ShQwc1btzY21Xu\n4MGD6tKliyIiIhQaGqqUlBSfnaevlGgAt9bus9aGF/20stYmFrV/Z629xVrb1Frbw1r7fUnWAQBl\n0YYNGzRkyBD5+/urfv36iomJUWpqqtavX+9tv+GGG9S9e3fve9atW6d27drJ7Xbr/fff9y6iM2bM\nGC1evFj5+flavny5hg4d6qvTAlAG7N69WxMmTNCuXbtUo0YNzZ0717tOQHp6ukaPHq34+HhJ0sCB\nA5WamqotW7aoZcuWevHFF737OXjwoDZs2KDVq1dr+vTpkgoX/erVq5f3BkNERIRPztGXWIoeAMqJ\nEydOaMKECUpLS1OjRo2UkJDgXdZ90KBBevjhh9W9e3e1adNG1113nY+rxS+pVq2acnJyfF0GKrBG\njRqpY8eOkqRhw4bpscce0/bt29WzZ09JUn5+vho0aCBJ2r59u2bMmKEjR44oJydHvXr18u6nf//+\n8vPzU0hIiL755htJUnR0tEaPHq1Tp06pf//+FTKAsxQ9AJQiZy7f3rlzZy1fvlz5+fk6dOiQ1q9f\nr7Zt26pLly7e9oMHD2rdunWS5A3bderUUU5Ozlkzo1SpUkW9evXS+PHjNWrUKOdPDECZUjhL9H9V\nr15drVq1ksfjkcfj0bZt2/TPf/5TUmFXk+eee07btm3TQw895P0ukqTKlSt7H5+e+rpLly5av369\nbrzxRo0cOVIvvfSSA2dUuhDAAcCHzl0ifs17a9SxY0eFhoZq06ZNCgsLU3h4uLp3766nnnpK119/\nvQYMGKCmTZsqJCREw4cP18033yxJqlWrlsaOHavQ0FD16tVL0dHRZx0rLi5Ofn5+uvXWW31xqrgM\n1trzDsQdPHiw3n77be92p6eizM/P19SpUxUdHa2wsDAtWLDAsVrz8vIcOxaK17nfQ6veWKWsrCxt\n2rRJUmGXkfbt2+vQoUPetlOnTnm7uB07dkwNGjTQqVOnlJSU9KvHy8zMVP369TV27FiNGTNGmzdv\nLrmTK62stWXip02bNhYAypNXkl6xAfUCrEbI6k+yGiEbUC/AvpL0Sokc7+mnn7YzZswokX2jeFWt\nWtVaa+2KFStsjx49bF5env3Pf/5jGzVqZL/++mv72muv2eHDh1trrT158qRt2LChzc3NtQsWLLCP\nPPKItdbaEydO2DZt2th9+/YVS00zZ860zZo1sx07drSDBw+2Tz/9tI2JibGTJ0+2bdq0sbNmzbLf\nfvutHThwoI2KirJRUVF2w4YN1lprc3Jy7KhRo2x0dLSNiIiwq1atstZau3jxYjtgwADbq1cve9NN\nN9mpU6cWS61XKicnx/7mN7+xYWFhtlWrVnbZsmX24YcftlFRUbZVq1Z27NixtqCgwFprbUxMjE1N\nTbXWWnvo0CEbFBRkrbV2+/btNjo62oaHh1u3220///xza621sbGxNjIy0oaEhNgFCxZ4j/nCCy/Y\npk2b2ujoaDtmzBh77733WmvtBa9pcnKyDQ8Pt+Hh4TYiIsIePXr0ss71fN9DVa6rYhs0aGDj4uJs\nixYt7MCBA+2PP/5oP/30U9u5c2cbFhZmQ0JC7MKFC6211s6bN8+6XC4bHR1tJ06caEeMGGGttXbE\niBH21Vdf9R7r9Od6yZIltlWrVjYiIsJ26tSp2D6jxUVSmi3hXOvzYH2xPwRwAOVNUNOgwv/TSzjj\nZ4RsUNOgYj9W//79rdvttocOHSr2faP4nQ4qf/jDH+yLL77obR82bJh944037PHjx22jRo3siRMn\n7KpVq+zQoUOttdYOGjTINm3a1BvMXC6Xfe+99664nk8++cSGh4fb48eP26NHj9qbbrrJG8DHjx/v\n3W7IkCE2JSXFWmttZmambdGihbXW2gceeMC+/PLL1lprs7OzbdOmTW1OTo5dvHixDQ4OtkeOHLHH\njx+3gYGBNisr64rrvVIrVqywY8aM8T4/cuSI/e6777zPhw0bZt98801r7YUD+MSJE+0rrxT+Mn3y\n5Embm5trrbXe/eTm5tpWrVrZw4cP2wMHDtigoCD73Xff2Z9++sl26tTJG8AvdE379OnjDePHjh2z\np06duqxzdfJ7qKxwIoAzCBMAfCRrb5Y0+JzGQCnr5XOXrbxyr7/+erHvE75TpUoVde3aVe+9956W\nL1+uwYMLP0jWWj377LNnDYIrDhs3blRsbKyqVKmiKlWqqG/fvt7X7r77bu/jtWvXaufOnd7nR48e\nVU5Ojv75z3/qzTff1KxZsyQVjlfIyir8nN9yyy2qWbOmJCkkJESZmZlq1KhRsdZ/qdxut+677z5N\nmzZNffr0UefOnbVy5Uo99dRTys3N1ffff69WrVqddR3OdfPNNysxMVFfffWVBg4cqKZNm0qS5syZ\n4/3v8csvv9QXX3yh//znP4qJidG1114rSbrzzjv1+eefS7rwNe3YsaP++Mc/Ki4uTgMHDlTDhg0v\n61yd/B66XBkZGerTp4+2b99erPtNSEhQtWrVdP/995/7UhVjjEeFK7XfYa3de7H7NMaMlPRPa+3X\nv7QdfcABwEdYIh7Sz/vfJi39bx/aCw3ElQqD7+LFi5WSkqLbbrtNktSrVy89//zzOnXqlCTp888/\n148//lii9VetWtX7uKCgQB999JF3oN6BAwdUrVo1WWu1cuVKb3tWVpZatmwp6exBev7+/qWiL3mz\nZs20efNmud1uzZgxQzNnztSECRO0YsUKbdu2TWPHjvUONKxUqZIKCgok6azBh0OHDtWbb76pa665\nRr/5zW/0/vvvKzk5WWvXrtWmTZu0ZcsWtW7d+qz3nM+Frun06dP1wgsv6Pjx4+rYsaM+++yzyzrX\n8vI9lJ+fX5y7qyVphbW29aWE7yIjJd3waxsRwAHAR8rSEvEul0uHDx/2dRnlTtLSJI2bMk6ZHTJl\n460yO2Rq3JRx3hA6YMCA8w7ElaRbb71VH3zwgXr06KGrr75aUuF87yEhIYqMjFRoaKh+97vfXVag\nPfeXgqPHjuqtt97SiRMnlJOTo9WrV5/3fbfeequeffZZ73OPxyOp8BeDZ599trDvq6RPP/30kmsq\nKef7Bejrr79WQECAhg0bpqlTp3oHCZ5vhiGXy6X09HRJOqt93759aty4sSZNmqTY2Fht3bpVP/zw\ng2rXrq2AgAB99tln+uijjyQVTsv3wQcfKDs7W3l5eVq5cqV3Pxe6pnv37pXb7da0adMUHR192QG8\nrHwPnW9hIJfLpWnTpikyMlKvvvqq9u7dq9tuu01t2rRR586dvdfkrbfeUrt27dS6dWv16NHDOx3i\nmRYtWqTevXufHtxcX9J4Y8w6STLGrDLGpBtjdhhjxhW1+Rtjlhhjthtjthljphhj7pAUJSnJGOMx\nxlxzwRMq6T4uxfVDH3AA5dErSa/YoKZB1vgZG9Q0qMQGYF6poKAg+o+XgNLY//ZCg4MHDBxgmzZt\najt16mQHDhxoFy5ceFb/Z2sL+0Dfdddd1u1225YtW9rf/e531trC/s7jxo2zoaGhNiQkxN5+++3W\n2sJBmKf7Oltr7e23327XrVvn83P9n2n/Y91utw0PD7dRUVE2NTXVxsfH28aNG9sOHTrYkSNH2oce\neshaa+2uXbus2+22ERERNj4+3tsH/PHHH7chISE2PDzc9urVy3733Xf2xIkT9rbbbrMtWrSwsbGx\nNiYmxnu+CxYssDfddJNt27atHT58uH3wwQettRe+phMnTrStWrWybrfbDh482J44ceKKrkNp/h7a\nv3+/leTt8z5q1Cj79NNP26CgIPvkk096t+vevbt3sOtHH31ku3XrZq219vvvv/cOml20aJH94x//\naK219qGHHrJPP/20ffbZZ22/fv2811DS15Lut6cHTErXFv17jaTtkq6T1EbSv87YplbRv8mSouyv\n5FpTtHGpFxUVZdPS0nxdBgCUez/++KPuuusuffXVV8rPz9ef/vQnTZs2TWlpaapataoGDhyogQMH\n6ssvv9S1116rP/zhD5Kk+Ph41atXT5MnT/bxGZQdfv5+svFW8j+jMV8yiUYF+QU+qcnVzKXMDplS\n8BmN+6VGGxopa0+WcnNz1aVLFy1cuFCRkZE+qbG4XOhcgz4MUsbnGY7WkpOTo2rVqikvL08DBgzQ\n6NGjNWDAAEdrKK0yMjLUpUsX77iB999/X3PmzJHH49EHH3ygoKAg5eTkqG7dumrevLn3fSdPntSu\nXbu0bds23XfffTp48KB++uknBQcHa82aNUpISNBrr72mRo0aadWqVbrqqqskScaYg5L+aq2dVfQ8\nQdLp/zFcknpJ2i0pTdI7kt5WYb/vAmNMsgrD+y+GVrqgAADOsmbNGt1www3asmWLtm/f7u1fnJOT\no759+2rIkCEaO3asRo8e7V1Ao6CgQMuWLdOwYcN8WXqZUxr732btzZLOPXyg9OXeLxUREaHIyEgN\nGjSozIdv6cLnmrXX+QGICQkJioiIUGhoqIKDg9W/f3/HayjNzl0Y6PTz02MQCgoKVKtWLW9feY/H\no127dkmSfv/732vixInatm2bFixYcFa/e7fbrYyMDH311VcXOm5XST0k3WytDZf0qaQq1tpsSeEq\nvON9j6QXLuV8COAAgLO43W7961//0rRp05SSkuKdoSI2NlajRo3S8OHDJRX2fb3uuuv06aef6p//\n/Kdat27NEveXqDT2v73QLwVBTYPk8Xj02Wef6YEHHvBJbcWtNP0CNGvWLO/1nTNnzs8CZ0VyMQsD\nderU6az31KhRQ8HBwXr11VclFXax3rJliyTphx9+0I033ihJ+sc//nHW+1q3bq0FCxaoX79++vrr\n805cUlNStrU21xjTQlJ7STLG1JHkZ61dKWmGpNO/kR6TVP3XzpEADgA4y/lmgJCkjh07as2aNTqz\n6+KYMWO0ZMkSLV68WKNHj/ZVyWVW3NA4LZy9UEEfBskkGgV9GKSFsxcqbmicz2oqTb8UZGRkKDQ0\ntMT2X5rOFYXONzD5gUceUIMGDTR37ly1bNlS2dnZGj9+/M/fm5SkF198UeHh4WrVqpXeeOMNSYV/\nXbjzzjvVpk0b1alT52fv69Spk2bNmqXbb7/9fIPN10iqZIzZJekJSR8Vtd8oKblousJXJJ3+rXSJ\npPm/NgiTPuAAUMElLU1SfEK8svZmKbBJoO6ffL/G/L8xqlKlilavXq0XXnhBHo9HaWlpmjlzpvLy\n8jRv3jxJ0k8//SS3261Tp07piy++kL+//68cDWXBuZ+JxIREn/xSUFLzP5+ptJwrCpWGfvnGmHRr\nbVRJHoM74ABQgZ3vbtP98ferWbNmioiI0MMPP6wZM2Z4t//b3/6m48eP63/+538kSVdffbW6deum\nu+66i/BdjsQNjVPG5xkqyC9QxucZPg2k55t+bubMmYqOjlZoaKjGjRvn/avMnj171KNHD4WHhysy\nMlJ79+6VtVZTp05VaGio3G63li9fLklKTk5W165d9fprr6uKXxUNGTxE+3fvJ3z7WGnql1+SuAMO\nABXYld5tKigo8M7Be3qlP6C4ZGRkKDg4WBs2bFDHjh01evRohYSEaPTo0d5VI3/729/qrrvuUt++\nfdWuXTtNnz5dAwYM0IkTJ1RQUKB3331X8+fP15o1a3T48GFFR0fr448/1u7duxUbG6sdO3bohhtu\nUMeOHfX000//rG8xnMUdcABAuXcld5t27typm266SbfccgvhGyWmUaNG6tixoyRp2LBh2rBhg9at\nW6d27drJ7Xbr/fff144dO3Ts2DEdOHDAO3VflSpVFBAQoA0bNmjIkCHy9/dX/fr1FRMTo9TUVElS\n27Zt1bBhQ/n5+SkiIkIZGRm+Ok0UqSj98iv5ugAAgO8ENglUZtY5d5suchaIkJAQ7du3r+SKA3T+\n6ecmTJigtLQ0NWrUSAkJCb+6nPuFVK5c2fvY39//slYNRfE63QUoPiFeWS8X9cufXf765XMHHAAq\nsIpytwllw6VMP3fusvDVq1dXw4YNtWrVKkmFi7Dk5uaqc+fOWr58ufLz83Xo0CGtX79ebdu29c0J\n4qKUpjEIJYU74ABQgVWUu00o/U4PCM7tnSsNljKzzp5+7nT/7/Hjxys7O1uhoaG6/vrrFR0d7d3H\nyy+/rN/97nf685//rKuuukqvvvqqBgwYoE2bNik8PFzGGD311FO6/vrr9dlnn/nwbFHRMQgTAEqx\nC03D1rVrV82aNUtRUSU6TghwTGkYfFccXnnlFc2ZM0c//fST2rVrp3nz5mnJkiV68sknVatWLYWH\nh6ty5cp67rnntHfvXsXFxenHH39UbGysnnnmGeXk5OjgwYO6++67dfToUeXl5en5559X586dfX1q\nFQaDMAEAQIVQHqaf27Vrl5YvX66NGzfK4/HI399fSUlJeuSRR/TRRx9p48aNZ915nzx5siZPnqxt\n27apYcOG3valS5eqV69e8ng82rJliyIiInxxOmc536JIaWlpmjRpko8qKtsI4ABQyp1vHuQzVatW\nzft4xYoVGjlypCTp0KFDGjRokKKjoxUdHa2NGzc6WTZwSUrTsvCX69///rfS09MVHR2tiIgI/fvf\n/9Zf//pXxcTE6Nprr9VVV12lO++807v9pk2bvM+HDh3qbY+OjtbixYuVkJCgbdu2qXr1X13Z3Cei\noqI0Z84cX5dRJhHAAaCU2717tyZMmKBdu3apRo0a3lUof83kyZM1ZcoUpaamauXKlRozZkwJVwpc\nvvIwINhaqxEjRsjj8cjj8Wj37t1KSEi45P106dJF69ev14033qiRI0fqpZdeKv5ir8C+ffvUunVr\nPf300+rTp4+kwuXeR48era5du6px48ZnBfNHHnlEzZs3V6dOnTRkyBDNmjVLkjRnzhyFhIQoLCxM\ngwcP9sm5+AqDMAGglDt3HuSLveO0du1a7dy50/v86NGjysnJOeuOOVBalIcBwbfccotiY2M1ZcoU\n1atXT99//71at26tP/zhD8rOzlb16tW1cuVKud1uSVL79u21cuVK3X333Vq2bJl3P5mZmWrYsKHG\njh2rkydPavPmzRo+fLivTussu3fv1uDBg7VkyRJlZ2frgw8+8L722Wefad26dTp27JiaN2+u8ePH\ny+PxaOXKldqyZYtOnTqlyMhItWnTRpL0xBNPaP/+/apcubKOHDniq1PyCe6AA0Apd755kC/0/Mz5\nkAsKCvTRRx9578YdOHCA8I1SrSxNP3fulIlJS5MUEhKiRx99VLfeeqvCwsLUs2dPHTx4UA8++KDa\ntm2rjh07yuVyqWbNmpKkZ555Rn/9618VFhamPXv2eNuTk5MVHh6u1q1ba/ny5Zo8ebIvT9Xr0KFD\nio2NVVJSksLDw3/2+u23367KlSurTp06qlevnr755htt3LhRsbGxqlKliqpXr66+fft6tw8LC1Nc\nXJxeeeUVVapUse4JE8ABoJS70DzIp9WvX1+7du1SQUGBXn/9dW/7rbfeqmeffdb73OPxOFMwUM6d\nnjIxs0OmbLxVZodMjZsyTklLk3T33XfL4/Fo69atSk9PV/v27TV06FB98cUX2rhxo77//nvv7EU3\n3nijPvroI23dulWRkZHe9hEjRmj79u369NNPlZKSouDg4F8qxzE1a9ZUYGCgNmzYcN7XL3Vho7ff\nflv33nuvNm/erOjo6Aq1EBIBHABKkfMtRNK8eXPNnTtXLVu2VHZ2tsaPH3/We5544gn16dNHHTp0\nUIMGDbztc+bMUVpamsLCwhQSEqL58+c7fTpAuRSfEF84X3mwJH9JwVJu71zFJ8Sfd/uEhARFREQo\nNDRUwcHB6t+/vyQpPT1dERERCgsL07x58/SXv/zFuZP4Fef7Lrr66qv1+uuv66WXXtLSpUsvaj8d\nO3bUW2+9pRMnTignJ0erV6+WVPgXui+//FLdunXTk08+qR9++EE5OTkleUqlSsW63w8Apdj5FiKJ\nfyxeC2cv/Nmf4pOTk72P77jjDt1xxx0/21+dOnW0fPnyki4bqHCy9mZJ544ZDJSyXj7/lImnBx2e\nq3PnztqyZUsxV3flLrQoUp1r6qhq1apavXq1evbsqT/96U+/uq/o6Gj169dPYWFhql+/vtxut2rW\nrKn8/HwNGzZMP/zwg6y1mjRpkmrVqnXWe48cOaKlS5dqwoQJxXZuzzzzjMaNG6eAgIBi2+flYCEe\nACglystCJEB5V97/Wy3u8zs9+Ds3N1ddunTRwoULFRkZ+avvu9BCZFfC5XIpLS1NderUueA2LMQD\nABVIeViIBKgIysOUib+kuL9sE8eRAAAfTklEQVSLxo0bp4iICEVGRmrQoEEXFb4lafr06dq7d68i\nIiI0depUTZ06VaGhoXK73d6/7uXk5OiWW25RZGSk3G633njjDUnSjz/+qNtvv13h4eEKDQ3V8uXL\nNWfOHH399dfq1q2bunXrdlnnUly4Aw4ApUR5v6sGlCdJS5MKp0zcWzRlYkLZmjLxl5SW76Iz74Cv\nXLlS8+fP15o1a3T48GFFR0fr448/Vt26dZWbm6saNWro8OHDat++vb744gu99tprWrNmjRYtWiRJ\n+uGHH1SzZk3ugAMAzlbe76oB5UlZmjLxUpXG76INGzZoyJAh8vf3V/369RUTE6PU1FRZa/Xggw8q\nLCxMPXr00IEDB/TNN9/I7XbrX//6l6ZNm6aUlBTvFI+lBQEcAEqJuKFxWjh7oYI+DJJJNAr6MOi8\nAzABoCSVpe+ipKQkHTp0SOnp6fJ4PKpfv75OnDihZs2aafPmzXK73ZoxY4Zmzpzp61LPQgAHgFKk\nPN9VA1B2+OK76NypD9e8t0bHjh2TVDhjzPLly5Wfn69Dhw5p/fr1atu2rX744QfVq1dPV111ldat\nW6fMzExJ0tdff62AgAANGzZMU6dO1ebNmyVJ1atX9+7Tl5iGEAAAAD51vqkP7/vzfQpvUjiIsnfv\n3goLC1N4eLiMMXrqqad0/fXXKy4uTn379pXb7VZUVJRatGghSdq2bZumTp0qPz8/XXXVVXr++ecl\nFQ4Ive2223TDDTdo3bp1PjtfBmECAADAp0rLwE+JQZgAAACoACraNKwEcAAAAPhUYJNA6dysnVXU\nXg4RwAEAAOBTpXHqw5LEIEwAAAD41OlZVuIT4pX1ctHiRrPLz+JG52IQJgAAAFCEQZgAAABAOUMA\nBwAAABxEAAcAAAAcRAAHAAAAHEQABwAAABxEAAcAAAAcRAAHAAAAHEQABwAAABxEAAcAAAAcRAAH\nAAAAHEQABwAAABxEAAcAAAAcRAAHAAAAHEQABwAAABxEAAcAAAAcRAAHAAAAHOSzAG6Muc0Ys9sY\ns8cYM91XdQAAAABO8kkAN8b4S5orqbekEElDjDEhvqgFAAAAcJKv7oC3lbTHWrvPWvuTpGWSYn1U\nCwAAAOAYXwXwGyV9ecbzr4raAAAAgHKtVA/CNMaMM8akGWPSDh065OtyAAAAgCvmqwB+QFKjM543\nLGo7i7V2obU2ylobVbduXceKAwAAAEqKrwJ4qqSmxphgY8zVkgZLetNHtQAAAACOqeSLg1pr84wx\nEyW9J8lf0t+ttTt8UQsAAADgJJ8EcEmy1r4j6R1fHR8AAADwhVI9CBMAAAAobwjgAAAAgIMI4AAA\nAICDCOAAAACAgwjgAAAAgIMI4AAAAICDCOAAAACAgwjgAAAAgIMI4AAAAICDCOAAAACAgwjgAAAA\ngIMI4AAAAICDCOAAAACAgwjgAAAAgIMI4AAAAICDCOAAAACAgwjgAAAAgIMI4AAAAICDCOAAAACA\ngwjgAAAAgIMI4AAAAICDCOAAAACAgwjgAAAAgIMI4AAAAICDCOAAAACAgwjgAAAAgIMI4AAAAICD\nCOAAAACAgwjgAAAAgIMI4AAAAICDCOAAAACAgwjgAAAAgIMI4AAAAICDCOAAAACAgwjgAAAAgIMI\n4AAAAICDCOAAAACAgwjgAAAAgIMI4AAAAICDCOAAAACAgwjgAAAAgIMI4AAAAICDCOAAAACAgwjg\nAAAAgIMI4AAAAICDCOAAAACAgwjgAAAAgIMI4AAAAICDCOAAAACAgwjgAAAAgIMI4AAAAICDCOAA\nAACAgwjgAAAAgIMI4AAAAICDCOAAAACAgwjgAAAAgIMI4AAAAICDCOAAAACAgwjgAAAAgINKLIAb\nYxKMMQeMMZ6in9+c8doDxpg9xpjdxpheJVUDAAAAUNpUKuH9z7bWzjqzwRgTImmwpFaSbpC01hjT\nzFqbX8K1AAAAAD7niy4osZKWWWtPWmv3S9ojqa0P6gAAAAAcV9IBfKIxZqsx5u/GmNpFbTdK+vKM\nbb4qagMAAADKvSsK4MaYtcaY7ef5iZX0vKQmkiIkHZT0l8vY/zhjTJoxJu3QoUNXUioAAABQKlxR\nH3BrbY+L2c4Ys0jS6qKnByQ1OuPlhkVt59v/QkkLJSkqKspefqUAAABA6VCSs6A0OOPpAEnbix6/\nKWmwMaayMSZYUlNJn5RUHQAAAEBpUpKzoDxljImQZCVlSPqdJFlrdxhj/k/STkl5ku5lBhQAAABU\nFCUWwK21v/2F1xIlJZbUsQEAAIDSipUwAQAAAAcRwAEAAAAHEcABAAAABxHAAQAAAAcRwAEAAAAH\nEcABAAAABxHAAQAAAAcRwAEAAAAHEcABAAAABxHAAQAAAAcRwAEAAAAHEcABAAAABxHAAQAAAAcR\nwAEAAAAHEcABAAAABxHAAQAAAAcRwAEAAAAHEcABAAAABxHAAQAAAAcRwAEAAAAHEcABAAAABxHA\nAQAAAAcRwAEAAAAHEcABAAAABxHAAQAAAAcRwAEAAAAHEcABAAAABxHAAQAAAAcRwAEAAAAHEcAB\nAAAABxHAAQAAAAcRwAEAAAAHEcABAAAABxHAAQAAAAcRwAEAAAAHEcABAAAABxHAAQAAAAcRwAEA\nAAAHEcABAAAABxHAAQAAAAcRwAEAAAAHEcABAAAABxHAAQAAAAcRwAEAAAAHEcABAAAABxHAAQAA\nAAcRwAEAAAAHEcABAAAABxHAAQAAAAcRwAEAAAAHEcABAAAABxHAAQAAAAcRwAEAAAAHEcABAAAA\nBxHAAQAAAAcRwAEAAAAHEcABAAAABxHAAQAAAAddUQA3xtxpjNlhjCkwxkSd89oDxpg9xpjdxphe\nZ7TfVtS2xxgz/UqODwAAAJQ1V3oHfLukgZLWn9lojAmRNFhSK0m3SZpnjPE3xvhLmiupt6QQSUOK\ntgUAAAAqhEpX8mZr7S5JMsac+1KspGXW2pOS9htj9khqW/TaHmvtvqL3LSvadueV1AEAAACUFSXV\nB/xGSV+e8fyrorYLtQMAAAAVwq/eATfGrJV0/XleirfWvlH8JZ117HGSxklSYGBgSR4KAAAAcMSv\nBnBrbY/L2O8BSY3OeN6wqE2/0H6+Yy+UtFCSoqKi7GXUAQAAAJQqJdUF5U1Jg40xlY0xwZKaSvpE\nUqqkpsaYYGPM1SocqPlmCdUAAAAAlDpXNAjTGDNA0rOS6kp62xjjsdb2stbuMMb8nwoHV+ZJutda\nm1/0nomS3pPkL+nv1todV3QGAAAAQBlirC0bPTuioqJsWlqar8sAAABAOWaMSbfWRv36lpePlTAB\nAAAABxHAAQAAAAcRwAEAAAAHEcABAAAABxHAAQAAAAcRwAEAAAAHEcABAAAABxHAAQAAAAcRwAEA\nAAAHEcABAAAABxHAAQAAAAcRwAEAAAAHEcABAAAABxHAAQAAAAcRwAEAAAAHEcABAAAABxHAAQAA\nAAcRwAEAAAAHEcABAAAABxHAAQAAAAcRwAEAAAAHEcABAAAABxHAAQAAAAcRwAEAAAAHEcABAAAA\nBxHAAQAAAAcRwAEAAAAHEcABAAAABxHAAQAAAAcRwAEAAAAHEcABAAAABxHAi1FCQoJmzZrl6zJ+\nZs6cOWrZsqXi4uJ8XQoAAECFV8nXBaDkzZs3T2vXrlXDhg19XQoAAECFxx3wK5SYmKhmzZqpU6dO\n2r17tyTJ4/Goffv2CgsL04ABA5SdnS1JSk1NVVhYmCIiIjR16lSFhoaWeH333HOP9u3bp969e+sv\nf/mL+vfvr7CwMLVv315bt26VJE2ePFkzZ86UJL333nvq0qWLCgoKSrw2AACAiogAfgXS09O1bNky\neTwevfPOO0pNTZUkDR8+XE8++aS2bt0qt9uthx9+WJI0atQoLViwQB6PR/7+/o7UOH/+fN1www1a\nt26dMjIy1Lp1a23dulWPPfaYhg8fLkl6/PHHtXz5cq1bt06TJk3S4sWL5efHRwMAAKAkkLKuQEpK\nigYMGKCAgADVqFFD/fr1048//qgjR44oJiZGkjRixAitX79eR44c0bFjx3TzzTdLkoYOHep4vRs2\nbNBvf/tbSVL37t313Xff6ejRowoICNCiRYvUs2dPTZw4UU2aNHG8NgAAgIqCAA5J0rZt23Tdddfp\n66+/9nUpAAAA5RoB/BIkLU2Sq5lLfv5+cjVzKefHHK1atUrHjx/XsWPH9NZbb6lq1aqqXbu2UlJS\nJEkvv/yyYmJiVKtWLVWvXl0ff/yxJGnZsmWO1Ji0NMn7WufOnZWUVPg8OTlZderUUY0aNZSZmam/\n/OUv+vTTT/Xuu+96awQAAEDxM9ZaX9dwUaKiomxaWprPjp+0NEnjpoxTbu9cKVBSlhTwboBuj7ld\nHo9H9erVU2BgoCIjI9WjRw/dc889ys3NVePGjbV48WLVrl1bH3/8scaOHSs/Pz/FxMQoLS1NGzdu\nLPEaAwoCtGvXLvn5+Wn06NHat2+fAgICtHDhQrndbvXs2VOTJk1Sv379lJ6erpEjRyo1NVVVqlQp\nttoAAADKAmNMurU2qkSPQQC/OK5mLmV2yJSCz2jcLwV9GKSMzzMuah85OTmqVq2aJOmJJ57QwYMH\n9be//a1U1QgAAFCRORHA6YJykbL2ZhXeVT5TYFH7RXr77bcVERGh0NBQpaSkaMaMGaWuRgAAAJQs\nFuK5SIFNApWZdc7d5azC9ot199136+677y7+4ooUR40AAAAoWdwBv0iJCYkKeDdA2i8pX9L+wv7V\niQmJvi7NqyzUCAAAUNFxB/wixQ2NkyTFJ8Qr6+UsBTYJVOLsRG97aVAWagQAAKjoGIQJAAAAFGEQ\nJgAAAFDOEMABAAAABxHAAQAAAAcRwAEAAAAHEcABAAAABxHAAQAAAAcRwMuBtLQ0TZo06Re3qVat\nmkPVAAAA4JewEE85EBUVpaioEp2uEgAAAMWEO+ClVGJiopo1a6ZOnTppyJAhmjVrlrp27arTixEd\nPnxYLpdLkpScnKw+ffpIknJycjRq1Ci53W6FhYVp5cqVZ+338OHDuvnmm/X22287ej4AAAAoxB3w\nUig9PV3Lli2Tx+NRXl6eIiMj1aZNm4t67yOPPKKaNWtq27ZtkqTs7Gzva99884369eunRx99VD17\n9iyR2gEAAPDLCOClUEpKigYMGKCAgABJUr9+/S76vWvXrtWyZcu8z2vXri1JOnXqlG655RbNnTtX\nMTExxVswAAAALhpdUMqQSpUqqaCgQJJ04sSJS35vmzZt9N5775VEaQAAALhIBPBSIGlpklzNXPLz\n95OrmUs5P+Zo1apVOn78uI4dO6a33npLkuRyuZSeni5JWrFixXn31bNnT82dO9f7/HQXFGOM/v73\nv+uzzz7Tk08+eVF1MXMKAABA8buiAG6MudMYs8MYU2CMiTqj3WWMOW6M8RT9zD/jtTbGmG3GmD3G\nmDnGGHMlNZR1SUuTNG7KOGV2yJSNt8rskKnH5zyuFi1aKDw8XL1791Z0dLQk6f7779fzzz+v1q1b\n6/Dhw+fd34wZM5Sdna3Q0FCFh4dr3bp13tf8/f31v//7v3r//fc1b948R84PAAAAZzPW2st/szEt\nJRVIWiDpfmttWlG7S9Jqa23oed7ziaRJkj6W9I6kOdbad3/tWFFRUfb0DCDliauZS5kdMqXgMxr3\nS0EfBinj8wxJUkJCgqpVq6b777/f0dqqVaumnJwc5eTkKDY2VtnZ2Tp16pQeffRRxcbGav78+Zo/\nv/B3qx9++EEul0u//e1vtXXrVj3zzDOSpEWLFmnnzp2aPXu2o7UDAABcDmNMurW2ROd3vqI74Nba\nXdba3Re7vTGmgaQa1tqPbGHyf0lS/yupoazL2pslBZ7TGFjUXkpUqVJFr7/+ujZv3qx169bpvvvu\nk7VW99xzjzwej1JTU9WwYUP98Y9/1F133aW33npLp06dkiQtXrxYo0eP9vEZAAAAlB4lOQtKsDHm\nU0lHJc2w1qZIulHSV2ds81VRW4UV2CRQmVnn3AHPKmw/LSEhwfG6zmSt1YMPPqj169fLz89PBw4c\n0DfffKPrr79ekjR58mR1795dffv2lSR1795dq1evVsuWLXXq1Cm53W5flg8AAFCq/GoAN8aslXT9\neV6Kt9a+cYG3HZQUaK39zhjTRtIqY0yrSy3OGDNO0jhJCgw89zZx+ZCYkKhxU8Ypt3du4Z3wLCng\n3QAlzk70dWleSUlJOnTokNLT03XVVVfJ5XJ5Z2FZsmSJMjMz9dxzz3m3HzNmjB577DG1aNFCo0aN\n8lXZAAAApdKvBnBrbY9L3am19qSkk0WP040xeyU1k3RAUsMzNm1Y1Hah/SyUtFAq7AN+qXWUBXFD\n4yRJ8Qnxyno5S4FNApU4O9Hb7qSkpUmFdewtrCMvL09SYf/uevXq6aqrrtK6deuUmZkpqXDBoFmz\nZiklJUV+fv/tzdSuXTt9+eWX2rx5s7Zu3er4eQAAAJRmJdIFxRhTV9L31tp8Y0xjSU0l7bPWfm+M\nOWqMaa/CQZjDJT1bEjWUhIyMDPXp00fbt28v1v3GDY3zSeA+0+nZWHJ750qDVdgtZk9he1xcnPr2\n7Su3262oqCi1aNFCkvTcc8/p+++/V7du3SRJUVFReuGFFyRJd911lzwej3chIAAAABS60llQBqgw\nQNeVdESSx1rbyxgzSNJMSadUOEvKQ9bat4reEyVpiaRrJL0r6ff2IoooDbOgXEwAz8/Pl7+/v4NV\nFY+LmY3lUvTp00dTpkzRLbfcUmw1AgAAlLSyMAvK69bahtbaytba+tbaXkXtK621ray1EdbayNPh\nu+i1NGttqLW2ibV24sWE79IkLy9PcXFxatmype644w7l5ubK5XJp2rRpioyM1KuvviqPx6P27dsr\nLCxMAwYMUHZ2tr799lu1adNGkrRlyxYZY5SVVTjTSZMmTZSbm6uRI0dq0qRJ6tChgxo3bnzBxXZK\nQnHNxnLkyBE1a9ZM11xzDeEbAADgPFgJ8xLt3r1bEyZM0K5du1SjRg3vgjbXXXedNm/erMGDB2v4\n8OF68skntXXrVrndbj388MOqV6+eTpw4oaNHjyolJUVRUVFKSUlRZmam6tWrp4CAAEnSwYMHtWHD\nBq1evVrTp0937LwCmwRK52btc2ZjuRi1atXS559/rldffbX4igMAAChHCOCXqFGjRurYsaMkadiw\nYdqwYYMk6e6775ZUOGDxyJEjiomJkSSNGDFC69evlyR16NBBGzdu1Pr1673T+qWkpKhz587e/ffv\n319+fn4KCQnRN99849h5JSYkKuDdAGm/pHxJ+4tmY0koPbOxAAAAlAcE8EtkjDnv86pVq/7qe7t0\n6eK96x0bG6stW7Zow4YNZwXwypUrex872TsnbmicFs5eqKAPg2QSjYI+DNLC2Qt9PjgUAACgvCGA\nX6KsrCxt2rRJkrR06VJ16tTprNdr1qyp2rVrKyUlRZL08ssve++Gd+7cWa+88oqaNm0qPz8/XXvt\ntXrnnXd+tg9fiRsap4zPM1SQX6CMzzMI3wAAACWAAH4BSUuT5Grmkp+/n1zNXEpamiRJat68uebO\nnauWLVsqOztb48eP/9l7//GPf2jq1KkKCwuTx+PRn//8Z0mSy+WStVZdunSRJHXq1Em1atViqj4A\nAIAK5IqmIXSSk9MQnjUn9hmrU9IlAwAAoHxzYhpCAvh5FPec2AAAACgbSv084OVVcc2JDQAAAJyL\nAH4exTUnNgAAAHAuAvh5lIU5sTt06ODrEgAAAHAZKvm6gNLo9EDL+IR4Zb2cpcAmgUqcnViqBmB+\n+OGHvi4BAAAAl4E74BdQ2ufErlatmpKTk9WnTx9v28SJE7VkyRJJhVMePvDAA4qIiFBUVJQ2b96s\nXr16qUmTJpo/f74kKTk5WV26dNHtt9+u5s2b65577lFBQYHy8/M1cuRIhYaGyu12a/bs2b44RQAA\ngHKJO+DlWGBgoDwej6ZMmaKRI0dq48aNOnHihEJDQ3XPPfdIkj755BPt3LlTQUFBuu222/Taa68p\nODhYBw4c0Pbt2yVJR44c8eVpAAAAlCvcAS/H+vXrJ0lyu91q166dqlevrrp166py5creUN22bVs1\nbtxY/v7+GjJkiDZs2KDGjRtr3759+v3vf681a9aoRo0avjwNAACAcoUAXoZVqlRJBQUF3ucnTpw4\n6/XKlStLkvz8/LyPTz/Py8uTJBljznqPMUa1a9fWli1b1LVrV82fP19jxowpqVMAAACocAjgpVzS\n0iS5mrnk5+8nVzOXkpYmeV8LCgrSzp07dfLkSR05ckT//ve/L3n/n3zyifbv36+CggItX75cnTp1\n0uHDh1VQUKBBgwbp0Ucf1ebNm4vzlAAAACo0+oCXYklLkzRuyjjl9s6VBkuZWZkaN2WcpMI71Y0a\nNdJdd92l0NBQBQcHq3Xr1pd8jOjoaE2cOFF79uxRt27dNGDAAG3btk2jRo3y3l1//PHHi/W8AAAA\nKjKWoi/FXM1cyuyQKQWf0bhfapjSUH55fsrMzLyi/ScnJ2vWrFlavXr1lRUKAABQTrAUfQWXtTdL\nOnfxzdrSV/u+0v333++TmgAAAHBluANeil3oDnjQh0HK+DzDV2UBAACUW9wBr+ASExIV8G6AtF9S\nvqT9UsC7AUpMSPR1aQAAALhMDMIsxU6vvhmfEK+sl7MU2CRQibMTS92qnAAAALh4dEEBAAAAitAF\nBQAAAChnCOAAAACAgwjgAAAAgIMI4AAAAICDCOAAAACAgwjgAAAAgIMI4AAAAICDCOAAAACAgwjg\nAAAAgIMI4AAAAICDysxS9MaYQ5IyfV3HBdSRdNjXRZQzXNPixzUtflzT4sc1LX5c0+LHNS1+pema\nBllr65bkAcpMAC/NjDFp1tooX9dRnnBNix/XtPhxTYsf17T4cU2LH9e0+FW0a0oXFAAAAMBBBHAA\nAADAQQTw4rHQ1wWUQ1zT4sc1LX5c0+LHNS1+XNPixzUtfhXqmtIHHAAAAHAQd8ABAAAABxHAL5Mx\nJsEYc8AY4yn6+c0Zrz1gjNljjNltjOnlyzrLGmPMbUXXbY8xZrqv6ymrjDEZxphtRZ/NtKK2a40x\n/zLGfFH0b21f11maGWP+boz51hiz/Yy2815DU2hO0ed2qzEm0neVl14XuKZ8l14BY0wjY8w6Y8xO\nY8wOY8zkonY+q5fpF64pn9XLZIypYoz5xBizpeiaPlzUHmyM+bjo2i03xlxd1F656Pmeotddvqy/\nJBDAr8xsa21E0c87kmSMCZE0WFIrSbdJmmeM8fdlkWVF0XWaK6m3pBBJQ4quJy5Pt6LP5ulpnaZL\n+re1tqmkfxc9x4UtUeF/w2e60DXsLalp0c84Sc87VGNZs0Q/v6YS36VXIk/SfdbaEEntJd1bdO34\nrF6+C11Tic/q5Topqbu1NlxShKTbjDHtJT2pwmt6k6RsSf+vaPv/Jym7qH120XblCgG8+MVKWmat\nPWmt3S9pj6S2Pq6prGgraY+1dp+19idJy1R4PVE8YiX9o+jxPyT192EtpZ61dr2k789pvtA1jJX0\nki30kaRaxpgGzlRadlzgml4I36UX4f+3dz+vVlVhGMe/T16DUDGouIg2EAkcWqMiCal00CQCCRuY\nSFADHTRu0qRBBPUPRIJEP7iY4kWkX+TYxBRMnSQUXbl5IdCURsrTYK1Tu8s919jn3L098XzgcPbZ\new1eXt6zWGevtfaxPW/7h3p8E7gMbCS12toyOR0mtXoXtd5u1Y+r68vAs8CRen5xnQ7q9wjwnCR1\nFG4nMgAfzcE6hXeoMZ2/Efi10WaO5b+48Y/kbnwMfC3prKTX67lp2/P1+Ddgup/QJtqwHKZ2R5O+\ndAzqNP3jwGlSq2OxKKeQWm1N0ipJ54EF4BvgCnDd9u3apJm3v3Nar98AHuo24pWVAfgyJH0r6ccl\nXi9Spu22UKZS5oH3ew024t+2236CMt18QNIzzYsujz/KI5BGkByOTfrSMZC0FvgCeNP2H81rqdV2\nlshpanUEtu/Y3gZsoswQbO05pF5N9R3Avcz28/+lnaQPgRP141Xg0cblTfVc3F1yNya2r9b3BUnH\nKJ3dNUkbbM/XKeeFXoOcTMNymNptyfa1wXH60nYkraYMFD+xfbSeTq2OYKmcplbHw/Z1SaeApyhL\noKbqXe5m3gY5nZM0BawHfu8l4BWSO+AtLVoz9xIw2NU/C+ypO3g3Uza6fN91fBPqDPBY3RV9P2VT\ny2zPMU0cSWskrRscA7so9TkL7KvN9gHH+4lwog3L4Szwan3CxJPAjcb0fywjfelo6rrYj4DLtj9o\nXEqttjQsp6nV9iQ9IunBevwAsJOytv4UsLs2W1yng/rdDXzn/9kf1+QOeHvvSdpGmdb7GXgDwPZF\nSTPAJcpO6gO27/QW5QSxfVvSQeArYBVwyPbFnsOaRNPAsbpfZQr41PaXks4AM5JeA34BXu4xxnue\npM+AHcDDkuaAt4F3WTqHJ4EXKJuv/gT2dx7wBBiS0x3pS0fyNLAXuFDX1wK8RWp1FMNy+kpqtbUN\nwOH6dJj7gBnbJyRdAj6X9A5wjvLDh/r+saSfKBu39/QR9ErKP2FGRERERHQoS1AiIiIiIjqUAXhE\nRERERIcyAI+IiIiI6FAG4BERERERHcoAPCIiIiKiQxmAR0RERER0KAPwiIiIiIgOZQAeEREREdGh\nvwBoT//nYxH0QgAAAABJRU5ErkJggg==\n",
      "text/plain": [
       "<Figure size 864x432 with 1 Axes>"
      ]
     },
     "metadata": {
      "tags": []
     },
     "output_type": "display_data"
    }
   ],
   "source": [
    "# visualize embeddings\n",
    "from sklearn.manifold import TSNE\n",
    "\n",
    "words = ft_model.wv.index2word\n",
    "wvs = ft_model.wv[words]\n",
    "\n",
    "tsne = TSNE(n_components=2, random_state=42, n_iter=5000, perplexity=5)\n",
    "np.set_printoptions(suppress=True)\n",
    "T = tsne.fit_transform(wvs)\n",
    "labels = words\n",
    "\n",
    "plt.figure(figsize=(12, 6))\n",
    "plt.scatter(T[:, 0], T[:, 1], c='green', edgecolors='k')\n",
    "for label, x, y in zip(labels, T[:, 0], T[:, 1]):\n",
    "    plt.annotate(label, xy=(x+1, y+1), xytext=(0, 0), textcoords='offset points')"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 18,
   "metadata": {
    "colab": {
     "base_uri": "https://localhost:8080/",
     "height": 85
    },
    "colab_type": "code",
    "id": "_LplYuHBRirE",
    "outputId": "00a2d159-159b-4e65-c398-464ffdd3f53c"
   },
   "outputs": [
    {
     "data": {
      "text/plain": [
       "(array([-1.1745292 ,  1.3008002 , -0.62582475,  0.14681922, -0.46031192,\n",
       "        -0.65553826, -0.1245426 , -0.09819014, -0.34571466,  0.23251319,\n",
       "         0.03760778,  0.8506686 , -0.5574059 ,  0.03467001, -0.45169955],\n",
       "       dtype=float32), (15,))"
      ]
     },
     "execution_count": 18,
     "metadata": {
      "tags": []
     },
     "output_type": "execute_result"
    }
   ],
   "source": [
    "ft_model.wv['sky'], ft_model.wv['sky'].shape"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 19,
   "metadata": {
    "colab": {
     "base_uri": "https://localhost:8080/",
     "height": 105
    },
    "colab_type": "code",
    "id": "40z0JRH5RirG",
    "outputId": "4a3f3b54-7f35-41c5-f7b2-5738421a029b"
   },
   "outputs": [
    {
     "name": "stdout",
     "output_type": "stream",
     "text": [
      "0.38875625\n",
      "0.96545506\n"
     ]
    },
    {
     "name": "stderr",
     "output_type": "stream",
     "text": [
      "/usr/local/lib/python3.6/dist-packages/gensim/matutils.py:737: FutureWarning: Conversion of the second argument of issubdtype from `int` to `np.signedinteger` is deprecated. In future, it will be treated as `np.int64 == np.dtype(int).type`.\n",
      "  if np.issubdtype(vec.dtype, np.int):\n"
     ]
    }
   ],
   "source": [
    "print(ft_model.wv.similarity(w1='ham', w2='sky'))\n",
    "print(ft_model.wv.similarity(w1='ham', w2='sausages'))"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 20,
   "metadata": {
    "colab": {
     "base_uri": "https://localhost:8080/",
     "height": 139
    },
    "colab_type": "code",
    "id": "7iwDNr-lRirI",
    "outputId": "c8988302-e591-4264-d174-5086d703b0ac"
   },
   "outputs": [
    {
     "name": "stdout",
     "output_type": "stream",
     "text": [
      "Odd one out for [ dog fox ham ]: ham\n",
      "Odd one out for [ bacon ham sky sausages ]: sky\n"
     ]
    },
    {
     "name": "stderr",
     "output_type": "stream",
     "text": [
      "/usr/local/lib/python3.6/dist-packages/gensim/models/keyedvectors.py:895: FutureWarning: arrays to stack must be passed as a \"sequence\" type such as list or tuple. Support for non-sequence iterables such as generators is deprecated as of NumPy 1.16 and will raise an error in the future.\n",
      "  vectors = vstack(self.word_vec(word, use_norm=True) for word in used_words).astype(REAL)\n",
      "/usr/local/lib/python3.6/dist-packages/gensim/matutils.py:737: FutureWarning: Conversion of the second argument of issubdtype from `int` to `np.signedinteger` is deprecated. In future, it will be treated as `np.int64 == np.dtype(int).type`.\n",
      "  if np.issubdtype(vec.dtype, np.int):\n"
     ]
    }
   ],
   "source": [
    "st1 = \"dog fox ham\"\n",
    "print('Odd one out for [',st1, ']:',  \n",
    "      ft_model.wv.doesnt_match(st1.split()))\n",
    "\n",
    "st2 = \"bacon ham sky sausages\"\n",
    "print('Odd one out for [',st2, ']:', \n",
    "      ft_model.wv.doesnt_match(st2.split()))"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {
    "colab_type": "text",
    "id": "QxZeE7w8RirK"
   },
   "source": [
    "### Getting document level embeddings\n",
    "\n",
    "Now suppose we wanted to cluster the eight documents from our toy corpus, we would need to get the document level embeddings from each of the words present in each document. One strategy would be to average out the word embeddings for each word in a document. This is an extremely useful strategy and you can adopt the same for your own problems. Let’s apply this now on our corpus to get features for each document."
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 0,
   "metadata": {
    "colab": {},
    "colab_type": "code",
    "id": "q8ETS-PxRirK"
   },
   "outputs": [],
   "source": [
    "def average_word_vectors(words, model, vocabulary, num_features):\n",
    "    \n",
    "    feature_vector = np.zeros((num_features,),dtype=\"float64\")\n",
    "    nwords = 0.\n",
    "    \n",
    "    for word in words:\n",
    "        if word in vocabulary: \n",
    "            nwords = nwords + 1.\n",
    "            feature_vector = np.add(feature_vector, model.wv[word])\n",
    "    \n",
    "    if nwords:\n",
    "        feature_vector = np.divide(feature_vector, nwords)\n",
    "        \n",
    "    return feature_vector\n",
    "\n",
    "\n",
    "def averaged_word_vectorizer(corpus, model, num_features):\n",
    "    vocabulary = set(model.wv.index2word)\n",
    "    features = [average_word_vectors(tokenized_sentence, model, vocabulary, num_features)\n",
    "                    for tokenized_sentence in corpus]\n",
    "    return np.array(features)"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 22,
   "metadata": {
    "colab": {
     "base_uri": "https://localhost:8080/",
     "height": 297
    },
    "colab_type": "code",
    "id": "MPFoB8y5RirM",
    "outputId": "c6073b7d-b71e-4496-e325-805d3ccacad4"
   },
   "outputs": [
    {
     "data": {
      "text/html": [
       "<div>\n",
       "<style scoped>\n",
       "    .dataframe tbody tr th:only-of-type {\n",
       "        vertical-align: middle;\n",
       "    }\n",
       "\n",
       "    .dataframe tbody tr th {\n",
       "        vertical-align: top;\n",
       "    }\n",
       "\n",
       "    .dataframe thead th {\n",
       "        text-align: right;\n",
       "    }\n",
       "</style>\n",
       "<table border=\"1\" class=\"dataframe\">\n",
       "  <thead>\n",
       "    <tr style=\"text-align: right;\">\n",
       "      <th></th>\n",
       "      <th>0</th>\n",
       "      <th>1</th>\n",
       "      <th>2</th>\n",
       "      <th>3</th>\n",
       "      <th>4</th>\n",
       "      <th>5</th>\n",
       "      <th>6</th>\n",
       "      <th>7</th>\n",
       "      <th>8</th>\n",
       "      <th>9</th>\n",
       "      <th>10</th>\n",
       "      <th>11</th>\n",
       "      <th>12</th>\n",
       "      <th>13</th>\n",
       "      <th>14</th>\n",
       "    </tr>\n",
       "  </thead>\n",
       "  <tbody>\n",
       "    <tr>\n",
       "      <th>0</th>\n",
       "      <td>-1.017834</td>\n",
       "      <td>1.219471</td>\n",
       "      <td>-0.471404</td>\n",
       "      <td>0.100632</td>\n",
       "      <td>-0.485411</td>\n",
       "      <td>-0.580870</td>\n",
       "      <td>-0.183611</td>\n",
       "      <td>-0.032766</td>\n",
       "      <td>-0.266565</td>\n",
       "      <td>0.289460</td>\n",
       "      <td>0.074620</td>\n",
       "      <td>0.748789</td>\n",
       "      <td>-0.517958</td>\n",
       "      <td>0.038283</td>\n",
       "      <td>-0.297274</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>1</th>\n",
       "      <td>-1.051389</td>\n",
       "      <td>0.996752</td>\n",
       "      <td>-0.461333</td>\n",
       "      <td>0.187398</td>\n",
       "      <td>-0.399002</td>\n",
       "      <td>-0.535357</td>\n",
       "      <td>-0.133190</td>\n",
       "      <td>-0.102741</td>\n",
       "      <td>-0.235152</td>\n",
       "      <td>0.255751</td>\n",
       "      <td>0.070397</td>\n",
       "      <td>0.575714</td>\n",
       "      <td>-0.376731</td>\n",
       "      <td>0.007694</td>\n",
       "      <td>-0.241070</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>2</th>\n",
       "      <td>-0.080685</td>\n",
       "      <td>1.394714</td>\n",
       "      <td>1.015425</td>\n",
       "      <td>0.091486</td>\n",
       "      <td>-1.111383</td>\n",
       "      <td>-0.360878</td>\n",
       "      <td>-0.166006</td>\n",
       "      <td>0.346778</td>\n",
       "      <td>-0.375867</td>\n",
       "      <td>0.003422</td>\n",
       "      <td>0.676978</td>\n",
       "      <td>-0.001734</td>\n",
       "      <td>-0.380349</td>\n",
       "      <td>0.563245</td>\n",
       "      <td>0.395930</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>3</th>\n",
       "      <td>-1.443467</td>\n",
       "      <td>0.329564</td>\n",
       "      <td>-0.008377</td>\n",
       "      <td>1.037905</td>\n",
       "      <td>-0.312026</td>\n",
       "      <td>-0.233052</td>\n",
       "      <td>0.184552</td>\n",
       "      <td>-0.440906</td>\n",
       "      <td>-0.320137</td>\n",
       "      <td>-0.371566</td>\n",
       "      <td>0.666171</td>\n",
       "      <td>-0.895322</td>\n",
       "      <td>0.647324</td>\n",
       "      <td>-0.199162</td>\n",
       "      <td>-0.239623</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>4</th>\n",
       "      <td>-1.307474</td>\n",
       "      <td>0.324607</td>\n",
       "      <td>-0.140498</td>\n",
       "      <td>0.826300</td>\n",
       "      <td>-0.173477</td>\n",
       "      <td>-0.265190</td>\n",
       "      <td>0.072099</td>\n",
       "      <td>-0.486643</td>\n",
       "      <td>-0.283610</td>\n",
       "      <td>-0.157870</td>\n",
       "      <td>0.412292</td>\n",
       "      <td>-0.508161</td>\n",
       "      <td>0.427139</td>\n",
       "      <td>-0.166446</td>\n",
       "      <td>-0.225815</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>5</th>\n",
       "      <td>-0.189542</td>\n",
       "      <td>1.352578</td>\n",
       "      <td>0.803024</td>\n",
       "      <td>0.068530</td>\n",
       "      <td>-1.020865</td>\n",
       "      <td>-0.378753</td>\n",
       "      <td>-0.193295</td>\n",
       "      <td>0.301214</td>\n",
       "      <td>-0.350197</td>\n",
       "      <td>0.057726</td>\n",
       "      <td>0.584805</td>\n",
       "      <td>0.126631</td>\n",
       "      <td>-0.409361</td>\n",
       "      <td>0.484519</td>\n",
       "      <td>0.307555</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>6</th>\n",
       "      <td>-1.085396</td>\n",
       "      <td>1.229123</td>\n",
       "      <td>-0.536072</td>\n",
       "      <td>0.100034</td>\n",
       "      <td>-0.476581</td>\n",
       "      <td>-0.611018</td>\n",
       "      <td>-0.181043</td>\n",
       "      <td>-0.026112</td>\n",
       "      <td>-0.259882</td>\n",
       "      <td>0.283033</td>\n",
       "      <td>0.046487</td>\n",
       "      <td>0.772775</td>\n",
       "      <td>-0.532793</td>\n",
       "      <td>0.030967</td>\n",
       "      <td>-0.318839</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>7</th>\n",
       "      <td>-0.088025</td>\n",
       "      <td>1.415853</td>\n",
       "      <td>1.003639</td>\n",
       "      <td>0.080209</td>\n",
       "      <td>-1.119665</td>\n",
       "      <td>-0.375660</td>\n",
       "      <td>-0.176112</td>\n",
       "      <td>0.350506</td>\n",
       "      <td>-0.377900</td>\n",
       "      <td>0.002104</td>\n",
       "      <td>0.662523</td>\n",
       "      <td>0.023472</td>\n",
       "      <td>-0.398187</td>\n",
       "      <td>0.561428</td>\n",
       "      <td>0.385074</td>\n",
       "    </tr>\n",
       "  </tbody>\n",
       "</table>\n",
       "</div>"
      ],
      "text/plain": [
       "         0         1         2   ...        12        13        14\n",
       "0 -1.017834  1.219471 -0.471404  ... -0.517958  0.038283 -0.297274\n",
       "1 -1.051389  0.996752 -0.461333  ... -0.376731  0.007694 -0.241070\n",
       "2 -0.080685  1.394714  1.015425  ... -0.380349  0.563245  0.395930\n",
       "3 -1.443467  0.329564 -0.008377  ...  0.647324 -0.199162 -0.239623\n",
       "4 -1.307474  0.324607 -0.140498  ...  0.427139 -0.166446 -0.225815\n",
       "5 -0.189542  1.352578  0.803024  ... -0.409361  0.484519  0.307555\n",
       "6 -1.085396  1.229123 -0.536072  ... -0.532793  0.030967 -0.318839\n",
       "7 -0.088025  1.415853  1.003639  ... -0.398187  0.561428  0.385074\n",
       "\n",
       "[8 rows x 15 columns]"
      ]
     },
     "execution_count": 22,
     "metadata": {
      "tags": []
     },
     "output_type": "execute_result"
    }
   ],
   "source": [
    "# get document level embeddings\n",
    "ft_doc_features = averaged_word_vectorizer(corpus=tokenized_corpus, model=ft_model,\n",
    "                                             num_features=feature_size)\n",
    "pd.DataFrame(ft_doc_features)"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {
    "colab_type": "text",
    "id": "mgrKl_vqRirN"
   },
   "source": [
    "### Trying out document clustering\n",
    "\n",
    "Now that we have our features for each document, let’s cluster these documents using the Affinity Propagation algorithm, which is a clustering algorithm based on the concept of “message passing” between data points and does not need the number of clusters as an explicit input which is often required by partition-based clustering algorithms."
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 23,
   "metadata": {
    "colab": {
     "base_uri": "https://localhost:8080/",
     "height": 297
    },
    "colab_type": "code",
    "id": "n9C0aRLwRirO",
    "outputId": "cfe31134-897f-4f74-cae9-1f9529c09786"
   },
   "outputs": [
    {
     "data": {
      "text/html": [
       "<div>\n",
       "<style scoped>\n",
       "    .dataframe tbody tr th:only-of-type {\n",
       "        vertical-align: middle;\n",
       "    }\n",
       "\n",
       "    .dataframe tbody tr th {\n",
       "        vertical-align: top;\n",
       "    }\n",
       "\n",
       "    .dataframe thead th {\n",
       "        text-align: right;\n",
       "    }\n",
       "</style>\n",
       "<table border=\"1\" class=\"dataframe\">\n",
       "  <thead>\n",
       "    <tr style=\"text-align: right;\">\n",
       "      <th></th>\n",
       "      <th>Document</th>\n",
       "      <th>Category</th>\n",
       "      <th>ClusterLabel</th>\n",
       "    </tr>\n",
       "  </thead>\n",
       "  <tbody>\n",
       "    <tr>\n",
       "      <th>0</th>\n",
       "      <td>The sky is blue and beautiful.</td>\n",
       "      <td>weather</td>\n",
       "      <td>0</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>1</th>\n",
       "      <td>Love this blue and beautiful sky!</td>\n",
       "      <td>weather</td>\n",
       "      <td>0</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>2</th>\n",
       "      <td>The quick brown fox jumps over the lazy dog.</td>\n",
       "      <td>animals</td>\n",
       "      <td>2</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>3</th>\n",
       "      <td>A king's breakfast has sausages, ham, bacon, eggs, toast and beans</td>\n",
       "      <td>food</td>\n",
       "      <td>1</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>4</th>\n",
       "      <td>I love green eggs, ham, sausages and bacon!</td>\n",
       "      <td>food</td>\n",
       "      <td>1</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>5</th>\n",
       "      <td>The brown fox is quick and the blue dog is lazy!</td>\n",
       "      <td>animals</td>\n",
       "      <td>2</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>6</th>\n",
       "      <td>The sky is very blue and the sky is very beautiful today</td>\n",
       "      <td>weather</td>\n",
       "      <td>0</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>7</th>\n",
       "      <td>The dog is lazy but the brown fox is quick!</td>\n",
       "      <td>animals</td>\n",
       "      <td>2</td>\n",
       "    </tr>\n",
       "  </tbody>\n",
       "</table>\n",
       "</div>"
      ],
      "text/plain": [
       "                                                             Document  ... ClusterLabel\n",
       "0                                      The sky is blue and beautiful.  ...            0\n",
       "1                                   Love this blue and beautiful sky!  ...            0\n",
       "2                        The quick brown fox jumps over the lazy dog.  ...            2\n",
       "3  A king's breakfast has sausages, ham, bacon, eggs, toast and beans  ...            1\n",
       "4                         I love green eggs, ham, sausages and bacon!  ...            1\n",
       "5                    The brown fox is quick and the blue dog is lazy!  ...            2\n",
       "6            The sky is very blue and the sky is very beautiful today  ...            0\n",
       "7                         The dog is lazy but the brown fox is quick!  ...            2\n",
       "\n",
       "[8 rows x 3 columns]"
      ]
     },
     "execution_count": 23,
     "metadata": {
      "tags": []
     },
     "output_type": "execute_result"
    }
   ],
   "source": [
    "from sklearn.cluster import AffinityPropagation\n",
    "\n",
    "ap = AffinityPropagation()\n",
    "ap.fit(ft_doc_features)\n",
    "\n",
    "cluster_labels = ap.labels_\n",
    "cluster_labels = pd.DataFrame(cluster_labels, \n",
    "                              columns=['ClusterLabel'])\n",
    "\n",
    "pd.concat([corpus_df, cluster_labels], axis=1)"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {
    "colab_type": "text",
    "id": "bB6pQQTPRirQ"
   },
   "source": [
    "We can see that our algorithm has clustered each document into the right group based on our Word2Vec features. Pretty neat! We can also visualize how each document in positioned in each cluster by using [_Principal Component Analysis (PCA)_](https://en.wikipedia.org/wiki/Principal_component_analysis) to reduce the feature dimensions to 2-D and then visualizing the same (by color coding each cluster)."
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 24,
   "metadata": {
    "colab": {
     "base_uri": "https://localhost:8080/",
     "height": 378
    },
    "colab_type": "code",
    "id": "3VA6oJnIRirR",
    "outputId": "05242d98-b335-44c6-f234-d3c067a16bfa"
   },
   "outputs": [
    {
     "data": {
      "image/png": "iVBORw0KGgoAAAANSUhEUgAAAe8AAAFpCAYAAAC1YKAIAAAABHNCSVQICAgIfAhkiAAAAAlwSFlz\nAAALEgAACxIB0t1+/AAAADl0RVh0U29mdHdhcmUAbWF0cGxvdGxpYiB2ZXJzaW9uIDMuMC4zLCBo\ndHRwOi8vbWF0cGxvdGxpYi5vcmcvnQurowAAIABJREFUeJzt3X9w1NX97/HXmwiRWCrwDV8L1bDY\nL2iaHyaw0gQJQ0NRqA4g6lRMx1qxadqrt2WmTp0JKDDG+v2WtnfCqBm0NvhtoNwb7JV6oQIlKFh/\nsEkBwy9BmgTBWggKEogl9dw/soRIEwhks5uTPB8zGfZz9rPnvPdkySufz579xJxzAgAA/ugT6wIA\nAMDFIbwBAPAM4Q0AgGcIbwAAPEN4AwDgGcIbAADPEN4AAHiG8AYAwDOENwAAniG8AQDwzGWxLqA9\niYmJLhAIxLoMAACiprKy8ohzbsiF9uu24R0IBBQKhWJdBgAAUWNmtR3Zj9PmAAB4hvAGAMAzhDcA\nAJ4hvAEA8AzhDQCAZwhvAAA8Q3gD8F5paakOHTrUsh0IBHTkyJEYVgR0LcIbgPfODe/OaGpqikg/\nQFcivAFE3c9//nMVFxdLkubMmaPc3FxJ0oYNG5SXl6e1a9cqOztbo0eP1l133aUTJ05IkhYuXKgb\nb7xRqampys/Pl3NO5eXlCoVCysvLU0ZGhk6dOiVJWrx4sUaPHq20tDTt3r1bktTQ0KD7779fY8eO\nVWZmpl566SVJzeE/bdo05ebmatKkSdGeDuCiEd4Aoi4nJ0ebNm2SJIVCIZ04cUKnT5/Wpk2blJ6e\nrscff1zr169XVVWVgsGgfvnLX0qSHnzwQW3ZskXV1dU6deqUXn75Zd15550KBoMqKyvT1q1b1b9/\nf0lSYmKiqqqq9IMf/ECLFi2SJBUVFSk3N1dvv/22Kioq9PDDD6uhoUGSVFVVpfLycr366qsxmBHg\n4hDeAKJuzJgxqqys1PHjxxUfH6/s7GyFQiFt2rRJ/fv3186dO3XTTTcpIyNDS5cuVW1t8xUjKyoq\n9LWvfU1paWnasGGDduzY0e4YM2fObBmrpqZGkrR27Vo9+eSTysjI0MSJE9XY2Ki6ujpJ0uTJkzV4\n8OCufeJAhHTba5sD6Ln69u2rESNGqLS0VOPGjVN6eroqKiq0b98+jRgxQpMnT9by5cs/95jGxkb9\n8Ic/VCgU0jXXXKP58+ersbGx3THi4+MlSXFxcS3vYzvntHLlSl133XWf2/ett97SFVdcEeFnCXQd\njrwBRMXyZWVKvT6guLg+Sr0+oIEDr9SiRYs0YcIE5eTkqKSkRJmZmcrKytLrr7+uffv2SWp+n/rd\nd99tCerExESdOHFC5eXlLX0PGDBAn3zyyQVruOWWW7R48WI55yRJf/nLX7rgmQJdjyNvAF1u+bIy\nFf4kX7/+7kmNv07avKdW95T8TYePNSk7O1tXXHGFLr/8cuXk5GjIkCEqLS3VrFmz9Omnn0qSHn/8\ncY0aNUrf+973lJqaqi996Uu68cYbW/q/7777VFBQoP79++uNN95ot4558+bpxz/+sdLT0/XZZ59p\nxIgRevnll7v8+QORZmd+A+1ugsGg40+CAj1D6vUBLb6jVl9POdtWsUN6aOVwVe+uiVldQHdjZpXO\nueCF9uO0OYAut2tvncZ//m1mjb+uub2juBALcBbhDaDLJY9M0uY9n2/bvKe5vaO4EAtwFuENoEu0\nvhBL0ohkfXNRH1XskNZul77xhDSrJF6f/rMvF2IBLgHhDaBLtL4QyycnTmjY1QE9WJ6kKf8lvfPh\nQA0aEtDWrVu5EAtwCQhvAF3i3Aux3HbbbXqu9HfKzZ2kwnkLdPjIES7EAlwiPioGoEtwIRag63Dk\nDSAizr0Iy/JlZcrJyeFCLEAXILwBdNqZi7AsvqNWjb9xWnxHrQp/kq/T//hUH3zwgbKzs3XVVVe1\neSGW9PR0ZWdna/fu3Ro4cGDLhVhuueWWNi/E0nrBWlvmzZun06dPKz09XSkpKZo3b140pgCIKi7S\nAqDTuAgLEBlcpAVA1ETiIiwAOo7wBtBpkbgIC4COI7wBdFrho0Wa/ZsEVeyQTjc1nzKf/ZsEFT5a\nFOvSgB6Jj4oB6LRZ9+RJkh5aWKhde+uUPDJJRYuKWtoBRBYL1gAA6CZYsAYAQA9FeAMA4BnCGwAA\nzxDeAAB4hvAGAMAzhDcAAJ4hvAEA8AzhDQCAZwhvAAA8E5HwNrPnzezvZlbdzv1mZsVmts/MtpvZ\n6EiMCwBAbxSpI+9SSVPOc/9USSPDX/mSnonQuAAA9DoRCW/n3GuSjp5nl+mSXnDN3pQ00MyGRmJs\nAAB6m2i95/1lSQdabb8fbgMAABepWy1YM7N8MwuZWejw4cOxLgcAgG4pWuF9UNI1rbavDrd9jnNu\niXMu6JwLDhkyJEqlAQDgl2iF9ypJ94ZXnWdJOuac+yBKYwPAJSstLdWhQ4datgOBgI4cORLDigDp\nskh0YmbLJU2UlGhm70t6TFJfSXLOlUhaLembkvZJOinpu5EYFwC6WmlpqVJTUzVs2LBO99XU1KTL\nLovIj130cpFabT7LOTfUOdfXOXe1c+7XzrmScHArvMr8fzjnvuKcS3POhSIxLgCc6+c//7mKi4sl\nSXPmzFFubq4kacOGDcrLy9PatWuVnZ2t0aNH66677tKJEyckSQsXLtSNN96o1NRU5efnyzmn8vJy\nhUIh5eXlKSMjQ6dOnZIkLV68WKNHj1ZaWpp2794tSWpoaND999+vsWPHKjMzUy+99JKk5vCfNm2a\ncnNzNWnSpGhPB3qobrVgDQA6KycnR5s2bZIkhUIhnThxQqdPn9amTZuUnp6uxx9/XOvXr1dVVZWC\nwaB++ctfSpIefPBBbdmyRdXV1Tp16pRefvll3XnnnQoGgyorK9PWrVvVv39/SVJiYqKqqqr0gx/8\nQIsWLZIkFRUVKTc3V2+//bYqKir08MMPq6GhQZJUVVWl8vJyvfrqqzGYEfREhDeAHmXMmDGqrKzU\n8ePHFR8fr+zsbIVCIW3atEn9+/fXzp07ddNNNykjI0NLly5VbW2tJKmiokJf+9rXlJaWpg0bNmjH\njh3tjjFz5syWsWpqaiRJa9eu1ZNPPqmMjAxNnDhRjY2NqqurkyRNnjxZgwcP7tonjl6FN18A9Ch9\n+/bViBEjVFpaqnHjxik9PV0VFRXat2+fRowYocmTJ2v58uWfe0xjY6N++MMfKhQK6ZprrtH8+fPV\n2NjY7hjx8fGSpLi4ODU1NUmSnHNauXKlrrvuus/t+9Zbb+mKK66I8LNEb8eRNwCvLV9WptTrA4qL\n66PU6wNavqxMOTk5WrRokSZMmKCcnByVlJQoMzNTWVlZev3117Vv3z5Jze9Tv/vuuy1BnZiYqBMn\nTqi8vLyl/wEDBuiTTz65YB233HKLFi9eLOecJOkvf/lLFzxboBnhDcBby5eVqfAn+Vp8R60af+O0\n+I5aFf4kX6f/8ak++OADZWdn66qrrtLll1+unJwcDRkyRKWlpZo1a5bS09OVnZ2t3bt3a+DAgfre\n976n1NRU3XLLLbrxxhtbxrjvvvtUUFDwuQVrbZk3b55Onz6t9PR0paSkaN68edGYAvRSdua3xO4m\nGAy6UIhF6QDal3p9QIvvqNXXU862VeyQHlo5XNW7a2JWF3CpzKzSORe80H4ceQPw1q69dRr/+beY\nNf665nagJyO8AXgreWSSNu/5fNvmPc3tQE9GeAPwVuGjRZr9mwRV7JBONzWfMp/9mwQVPloU69KA\nLsVHxQB4a9Y9eZKkhxYWatfeOiWPTFLRoqKWdqCnYsEaAADdBAvWAADooQhvAAA8Q3gDAOAZwhsA\nAM8Q3gAAeIbwBgDAM4Q3AACeIbwBAPAM4Q0AgGcIbwAAPEN4AwDgGcIbAADPEN4AAHiG8AYAwDOE\nNwAAniG8AQDwDOENAIBnCG8AADxDeAMA4BnCGwAAzxDeAAB4hvAGAMAzhDcAAJ4hvAEA8AzhDQCA\nZwhvAAA8Q3gDAOAZwhsAAM8Q3gAAeIbwBgDAM4Q3AACeIbwBAPAM4Q0AgGcIbwAAPEN4AwDgGcIb\nAADPEN4AAHiG8AYAwDOENwAAniG8AQDwTETC28ymmNkeM9tnZo+0cf99ZnbYzLaGvx6IxLgAAPRG\nl3W2AzOLk/SUpMmS3pe0xcxWOed2nrPrCufcg50dDwCA3i4SR95jJe1zzu13zv1D0u8kTY9AvwAA\noA2RCO8vSzrQavv9cNu57jCz7WZWbmbXRGBcAAB6pWgtWPuDpIBzLl3SOklL29rJzPLNLGRmocOH\nD0epNAAA/BKJ8D4oqfWR9NXhthbOuXrn3KfhzeckjWmrI+fcEudc0DkXHDJkSARKAwCg54lEeG+R\nNNLMRphZP0l3S1rVegczG9pqc5qkXREYFwCAXqnTq82dc01m9qCkVyTFSXreObfDzBZKCjnnVkn6\nn2Y2TVKTpKOS7uvsuAAA9FbmnIt1DW0KBoMuFArFugwAAKLGzCqdc8EL7ccV1gAA8AzhDQCAZwhv\nAAA8Q3gDAOAZwhsAAM8Q3gAAeIbwBgDAM4Q3AACeIbwBAPAM4Q0AgGcIbwAAPEN4AwDgGcIbAADP\nEN4AAHiG8AYAwDOENwAAniG8AQDwDOENAIBnCG8AADxDeAMA4BnCGwAAzxDeAAB4hvAGAMAzhDcA\nAJ4hvAEA8AzhDQCAZwhvAAA8Q3gDAOAZwhsAAM8Q3gAAeIbwBgDAM4Q3AACeIbwBAPAM4Q0AgGcI\nbwAAPEN4AwDgGcIbAADPEN4AAHiG8AYAwDOENwAAniG8AQDwDOENAIBnCG8AADxDeAMA4BnCGwAA\nzxDeAAB4hvAGAMAzhDcAAJ4hvAEA8AzhDQCAZwhvAAA8E5HwNrMpZrbHzPaZ2SNt3B9vZivC979l\nZoFIjAsAQG/U6fA2szhJT0maKumrkmaZ2VfP2W22pI+cc/8h6VeS/rOz4wIA0FtF4sh7rKR9zrn9\nzrl/SPqdpOnn7DNd0tLw7XJJk8zMIjA2AAC9TiTC+8uSDrTafj/c1uY+zrkmScck/VsExgYAoNfp\nVgvWzCzfzEJmFjp8+HCsywEAoFuKRHgflHRNq+2rw21t7mNml0m6UlL9uR0555Y454LOueCQIUMi\nUBoAAD1PJMJ7i6SRZjbCzPpJulvSqnP2WSXpO+Hbd0ra4JxzERgbAIBe57LOduCcazKzByW9IilO\n0vPOuR1mtlBSyDm3StKvJf23me2TdFTNAQ8AAC5Bp8NbkpxzqyWtPqft0Va3GyXdFYmxAADo7brV\ngjUAAHBhhDcAAJ4hvAEA8AzhDQCAZwhvAAA8Q3gDAOAZwhsAAM8Q3gAAeIbwBgDAM4Q3AACeIbwB\nAPAM4Q0AgGcIbwAAPEN4AwDgGcIbAADPEN4AAHiG8AYAwDOENwAAniG8AQDwDOENAIBnCG8AADxD\neAMA4BnCGwAAzxDeHigpKdELL7wQkb4CgYCOHDkSkb4AALFxWawLwIUVFBTEugQAQDfCkXeMzJgx\nQ2PGjFFKSoqWLFkiSfrCF76gwsJC3XDDDcrKytKHH34oSZo/f74WLVokSZo4caLmzJmjYDCo5ORk\nbdmyRTNnztTIkSM1d+7c8/bfWkNDg2699VbdcMMNSk1N1YoVK6LwrAEAkUB4x8jzzz+vyspKhUIh\nFRcXq76+Xg0NDcrKytK2bds0YcIEPfvss20+tl+/fgqFQiooKND06dP11FNPqbq6WqWlpaqvr2+3\n/9b++Mc/atiwYdq2bZuqq6s1ZcqULn/OAIDIILxjpLi4uOUI+8CBA9q7d6/69eun2267TZI0ZswY\n1dTUtPnYadOmSZLS0tKUkpKioUOHKj4+Xtdee60OHDjQbv+tpaWlad26dfrpT3+qTZs26corr+y6\nJwsAiCjCOwY2btyo9evX64033tC2bduUmZmpxsZG9e3bV2YmSYqLi1NTU1Obj4+Pj5ck9enTp+X2\nme2mpqZ2+29t1KhRqqqqUlpamubOnauFCxd20bMFAEQaC9Zi4NixYxo0aJASEhK0e/duvfnmm1Hv\n/9ChQxo8eLC+/e1va+DAgXruueciWgMAoOtw5B0FZcvKFBgVUJ+4PgqMCuijjz9SU1OTkpOT9cgj\njygrKyui402ZMuWC/b/zzjsaO3asMjIytGDBgs8tdgMAdG/mnIt1DW0KBoMuFArFuoxOK1tWpvw5\n+To59aSUJKlOSliToCW/WqK8e/JiXR4AoBsxs0rnXPCC+xHeXSswKqDacbXSiFaNf5WG/3m4at6t\niVVZAIBuqKPhzWnzLlb3Xl3zEXdrSeF2AAAuAeHdxZK+kiSdm9N14XYAAC4B4d3FiuYXKWFNgvRX\nSf+U9Nfm97yL5hfFujQAgKf4qFgXO7MorXB+oer+u05JX0lS0a+KWKwGALhkLFgDAKCbYMEaAAA9\nFOENAIBnCG8AADxDeAMA4BnCGwAAzxDeAAB4hvAGAMAzhDcAAJ4hvAEA8AzhDQCAZwhvAAA8Q3gD\nAOAZwvsilZSU6IUXXohIX4FAQEeOHIlIXwCA3oM/CXqRCgoKYl0CAKCX69SRt5kNNrN1ZrY3/O+g\ndvb7p5ltDX+t6syYXWHGjBkaM2aMUlJStGTJEknSF77wBRUWFuqGG25QVlaWPvzwQ0nS/PnztWjR\nIknSxIkTNWfOHAWDQSUnJ2vLli2aOXOmRo4cqblz5563/9YaGhp066236oYbblBqaqpWrFgRhWcN\nAPBVZ0+bPyLpT865kZL+FN5uyynnXEb4a1onx4y4559/XpWVlQqFQiouLlZ9fb0aGhqUlZWlbdu2\nacKECXr22WfbfGy/fv0UCoVUUFCg6dOn66mnnlJ1dbVKS0tVX1/fbv+t/fGPf9SwYcO0bds2VVdX\na8qUKV3+nAEA/upseE+XtDR8e6mkGZ3sLyaKi4tbjrAPHDigvXv3ql+/frrtttskSWPGjFFNTU2b\nj502rfl3kbS0NKWkpGjo0KGKj4/XtddeqwMHDrTbf2tpaWlat26dfvrTn2rTpk268soru+7JAgC8\n19nwvso590H49t8kXdXOfpebWcjM3jSzdgPezPLD+4UOHz7cydI6ZuPGjVq/fr3eeOMNbdu2TZmZ\nmWpsbFTfvn1lZpKkuLg4NTU1tTzmjTfeaFm0Fh8fL0nq06dPy+0z201NTe32f0ZmZqYGDx6sqqoq\npaWlae7cuVq4cGE0njoAwFMXDG8zW29m1W18TW+9n3POSXLtdDPcOReUdI+k/2VmX2lrJ+fcEudc\n0DkXHDJkyMU+lw4pW1amwKiA+sT1UWBUQKtWrdKgQYOUkJCg3bt3680337xgH9nZ2br33ns7NN6x\nY8cu2P/f/vY3JSQk6Nvf/rYefvhhVVVVXfTzAgD0Hhdcbe6c+0Z795nZh2Y21Dn3gZkNlfT3dvo4\nGP53v5ltlJQp6b1LK/nSlS0rU/6cfJ2celK6W6qtq1XJb0s0cuhIDRgwQHFxcYqLi9Mf/vAHSc2L\n1n70ox/pt7/9rU6ePNmyaG3t2rUtfX7/+9/XhAkTtHr1ah08eFBbtmzRz372M7311lt6+umn9cwz\nz6ikpORf+p84cWJLHzt37tTdd9+turo6NTU1aejQoVqxYoW+9a1vRXV+AAB+6Oxp81WSvhO+/R1J\nL527g5kNMrP48O1ESTdJ2tnJcS9J4fzC5uAeISlO0gjp1DdP6dipY6qtrdXHH3+sI0eO6JVXXlFt\nbW3LorXa2lp997vf1bPPPqv58+dr3LhxkppPuQ8YMED9+vXTnj17VFRU1LJo7eOPP9batWt14sQJ\nrVmz5l/6r6+vV01NjeLi4pSbm6sFCxboW9/6lk6ePKn33nuPRWsAgHZ1NryflDTZzPZK+kZ4W2YW\nNLPnwvskSwqZ2TZJFZKedM7FJLzr3quTks5pTGpuZ9EaAMAXnbpIi3OuXtKkNtpDkh4I3/6zpLTO\njBMpSV9JUm1dbfOR9xl10r8P/feWRWUJCQmaOHHiBRettXYxi9Za99/aqFGjVFVVpdWrV2vu3Lma\nNGmSHn300chOAACgR+hVl0ctml+khDUJ0l8l/VPSXyV70ZSZnnnRi9YuRkcWrR06dIhFawCADulV\nl0fNuydPr7/+ukqeL5H71EmJkstwejX0qkYOHank5GRdd911ysrKiui4U6ZMUUlJyXn7f+edd/Tw\nww+rT58+6tu3r5555pmI1gAA6Dms+RNe3U8wGHShUCji/QZGBVQ77pxT53+Vhv95uGrerYn4eAAA\ndJSZVYY/Wn1eveq0uXT+RWsAAPig14V30leSpHNzui7cDgDo8YqLi5WcnKy8vLxO9RPLP+vc68K7\nrUVrCWsSVDS/KNalAQCi4Omnn9a6detUVlYW61IuWa8L77x78rTkV0s0/M/DZUWm4X8eriW/WqK8\nezr3GxgAoPsrKCjQ/v37NXXqVP3iF7/QjBkzlJ6erqysLG3fvl2SdPTo0Tbb6+vrdfPNNyslJUUP\nPPCAYrlmrNeFt9Qc4DXv1uizf36mmndrCG4A6CVKSko0bNgwVVRUqKamRpmZmdq+fbueeOKJlr9Z\n8dhjj7XZvmDBAo0fP147duzQ7bffrrq62K2V6lUfFQMA4IzNmzdr5cqVkqTc3FzV19fr+PHj7ba/\n9tprevHFFyVJt956qwYNGhSz2nvlkTcAAD4jvAEAPVZZ2XIFAqnq0ydOgUCqysqWt9yXk5PTsmht\n48aNSkxM1Be/+MV22ydMmKBly5ZJktasWaOPPvoo+k8orNddpAUA0DuUlS1Xfn6hTp78taTxkjYr\nIWG2EhI+0a5du9SnTx/df//92r9/vxISErRkyRKlp6fr6NGjbbbX19dr1qxZOnjwoMaNG6e1a9eq\nsrJSiYmJEau5oxdpIbwBAD1SIJCq2trFkr7eqrVCw4c/pJqa6liVdV5cYQ0A0KvV1e1S8xF3a+PD\n7X4jvAEAPVJSUrKkzee0bg63+43wBgD0SEVFhUpImC2pQtJpSRVKSJitoqLCGFfWeXzOGwDQI+Xl\nzZIkFRY+pLq6XUpKSlZRUVFLu89YsAYAQDfBgjUAAHoowhsAAM8Q3gAAeIbwBgDAM4Q3AACeIbwB\nAPAM4Q0AgGcIbwAAPEN4AwDgGcIbAADPEN4AAHiG8AYAwDOENwAAniG8AQDwDOENAIBnCG8AADxD\neAMA4BnCGwAAzxDeAAB4hvAGAMAzhDcAAJ4hvAEA8AzhDQCAZwhvAAA8Q3gDAOAZwhsAAM8Q3gAA\neIbwBgDAM4Q3AACeIbwBAPAM4Q0AgGcIbwAAPEN4AwDgmU6Ft5ndZWY7zOwzMwueZ78pZrbHzPaZ\n2SOdGRMAgN6us0fe1ZJmSnqtvR3MLE7SU5KmSvqqpFlm9tVOjgsAQK91WWce7JzbJUlmdr7dxkra\n55zbH973d5KmS9rZmbEBAOitovGe95clHWi1/X64DQAAXIILHnmb2XpJX2rjrkLn3EuRLMbM8iXl\nS1JSUlIkuwYAoMe4YHg7577RyTEOSrqm1fbV4ba2xloiaYkkBYNB18lxAQDokaJx2nyLpJFmNsLM\n+km6W9KqKIwLAECP1NmPit1uZu9Lypb0/8zslXD7MDNbLUnOuSZJD0p6RdIuSf/bObejc2UDANB7\ndXa1+e8l/b6N9kOSvtlqe7Wk1Z0ZCwAANOMKawAAeIbwBgDAM4Q3AACeIbwBAPAM4Q0AgGcIbwAA\nPEN4AwDgGcIbAADPEN4AAHiG8AYAwDOENwAAl6i4uFjJycnKy8vrVD+BQEBHjhzp8P6durY5AAC9\n2dNPP63169fr6quvjuq4HHkDAHAJCgoKtH//fk2dOlW/+MUvNGPGDKWnpysrK0vbt2+XJB09erTN\n9vr6et18881KSUnRAw88IOfcRY1NeAMAcAlKSko0bNgwVVRUqKamRpmZmdq+fbueeOIJ3XvvvZKk\nxx57rM32BQsWaPz48dqxY4duv/121dXVXdTYnDYHAKCTNm/erJUrV0qScnNzVV9fr+PHj7fb/tpr\nr+nFF1+UJN16660aNGjQRY3HkTcAAJ4hvAEA6KCysuUKBFLVp0+cAoFUNTQ0SJJycnJUVlYmSdq4\ncaMSExP1xS9+sd32CRMmaNmyZZKkNWvW6KOPPrqoOjhtDgBAB5SVLVd+fqFOnvy1pPGqrd0ss5tV\nXr5S8+fP1/3336/09HQlJCRo6dKlktRu+2OPPaZZs2YpJSVF48aNU1JS0kXVYhe7wi1agsGgC4VC\nsS4DAABJUiCQqtraxZK+3qq1QsOHP6SamuqIjGFmlc654IX247Q5AAAdUFe3S9L4c1rHh9uji/AG\nAKADkpKSJW0+p3VzuD26CG8AADqgqKhQCQmzJVVIOi2pQgkJs1VUVBj1WliwBgBAB+TlzZIkFRY+\npLq6XUpKSlZRUVFLezSxYA0AgG6CBWsAAPRQhDcAAJ4hvAEA8AzhDQCAZwhvAAA8Q3gDAOAZwhsA\nAM8Q3gAAeIbwBgDAM4Q3AACeIbwBAPAM4Q0AgGe67R8mMbPDkmo7sGuipCNdXI4vmIuzmIuzmIuz\nmIuzmIuzutNcDHfODbnQTt02vDvKzEId+QssvQFzcRZzcRZzcRZzcRZzcZaPc8FpcwAAPEN4AwDg\nmZ4Q3ktiXUA3wlycxVycxVycxVycxVyc5d1ceP+eNwAAvU1POPIGAKBX8S68zewuM9thZp+ZWbur\nA82sxszeMbOtZhaKZo3RchFzMcXM9pjZPjN7JJo1RouZDTazdWa2N/zvoHb2+2f4NbHVzFZFu86u\ndKHvs5nFm9mK8P1vmVkg+lV2vQ7Mw31mdrjV6+CBWNQZDWb2vJn93cyq27nfzKw4PFfbzWx0tGuM\nlg7MxUQzO9bqdfFotGu8GN6Ft6RqSTMlvdaBfb/unMvw7SMAF+GCc2FmcZKekjRV0lclzTKzr0an\nvKh6RNKfnHMjJf0pvN2WU+HXRIZzblr0yutaHfw+z5b0kXPuPyT9StJ/RrfKrncRr/cVrV4Hz0W1\nyOgqlTTlPPdPlTQy/JUv6Zko1BQrpTr/XEjSplavi4VRqOmSeRfezrldzrk9sa6jO+jgXIyVtM85\nt9859w9Jv5M0veuri7rpkpaeCmIAAAAC5klEQVSGby+VNCOGtcRCR77PreeoXNIkM7Mo1hgNveX1\n3iHOudckHT3PLtMlveCavSlpoJkNjU510dWBufCKd+F9EZyktWZWaWb5sS4mhr4s6UCr7ffDbT3N\nVc65D8K3/ybpqnb2u9zMQmb2ppn1pIDvyPe5ZR/nXJOkY5L+LSrVRU9HX+93hE8Tl5vZNdEprVvq\nLT8fOirbzLaZ2RozS4l1MedzWawLaIuZrZf0pTbuKnTOvdTBbsY75w6a2b9LWmdmu8O/eXklQnPR\nI5xvLlpvOOecmbX3MYrh4dfFtZI2mNk7zrn3Il0rurU/SFrunPvUzL6v5rMRuTGuCbFXpeafDyfM\n7JuS/q+a307olrpleDvnvhGBPg6G//27mf1ezafTvAvvCMzFQUmtjyyuDrd553xzYWYfmtlQ59wH\n4dN+f2+njzOvi/1mtlFSpqSeEN4d+T6f2ed9M7tM0pWS6qNTXtRccB6cc62f83OS/isKdXVXPebn\nQ2c55463ur3azJ42s0TnXHe55vnn9MjT5mZ2hZkNOHNb0s1qXtzVG22RNNLMRphZP0l3S+pRq6zD\nVkn6Tvj2dyT9y1kJMxtkZvHh24mSbpK0M2oVdq2OfJ9bz9Gdkja4nnehhwvOwznv6U6TtCuK9XU3\nqyTdG151niXpWKu3n3oVM/vSmTUgZjZWzfnYfX+5dc559SXpdjW/L/OppA8lvRJuHyZpdfj2tZK2\nhb92qPkUc8xrj8VchLe/KeldNR9h9tS5+Dc1rzLfK2m9pMHh9qCk58K3x0l6J/y6eEfS7FjXHeE5\n+Jfvs6SFkqaFb18u6f9I2ifpbUnXxrrmGM3Dz8I/F7ZJqpB0faxr7sK5WC7pA0mnwz8rZksqkFQQ\nvt/UvDr/vfD/iWCsa47hXDzY6nXxpqRxsa75fF9cYQ0AAM/0yNPmAAD0ZIQ3AACeIbwBAPAM4Q0A\ngGcIbwAAPEN4AwDgGcIbAADPEN4AAHjm/wPdx/JV3GcdSgAAAABJRU5ErkJggg==\n",
      "text/plain": [
       "<Figure size 576x432 with 1 Axes>"
      ]
     },
     "metadata": {
      "tags": []
     },
     "output_type": "display_data"
    }
   ],
   "source": [
    "from sklearn.decomposition import PCA\n",
    "\n",
    "pca = PCA(n_components=2, random_state=42)\n",
    "pcs = pca.fit_transform(ft_doc_features)\n",
    "labels = ap.labels_\n",
    "categories = list(corpus_df['Category'])\n",
    "plt.figure(figsize=(8, 6))\n",
    "\n",
    "for i in range(len(labels)):\n",
    "    label = labels[i]\n",
    "    color = 'orange' if label == 0 else 'blue' if label == 1 else 'green'\n",
    "    annotation_label = categories[i]\n",
    "    x, y = pcs[i]\n",
    "    plt.scatter(x, y, c=color, edgecolors='k')\n",
    "    plt.annotate(annotation_label, xy=(x+1e-2, y+1e-2), xytext=(0, 0), \n",
    "                 textcoords='offset points')"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {
    "colab_type": "text",
    "id": "PvFiGdkMRirS"
   },
   "source": [
    "Everything looks to be in order as documents in each cluster are closer to each other and far apart from other clusters."
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 0,
   "metadata": {
    "colab": {},
    "colab_type": "code",
    "id": "tr_ejT8_Tsq3"
   },
   "outputs": [],
   "source": []
  }
 ],
 "metadata": {
  "colab": {
   "collapsed_sections": [],
   "include_colab_link": true,
   "name": "Exploring Word Embeddings with Deep Learning Models.ipynb",
   "provenance": [],
   "version": "0.3.2"
  },
  "kernelspec": {
   "display_name": "Python 3",
   "language": "python",
   "name": "python3"
  },
  "language_info": {
   "codemirror_mode": {
    "name": "ipython",
    "version": 3
   },
   "file_extension": ".py",
   "mimetype": "text/x-python",
   "name": "python",
   "nbconvert_exporter": "python",
   "pygments_lexer": "ipython3",
   "version": "3.6.4"
  }
 },
 "nbformat": 4,
 "nbformat_minor": 1
}