UN_Data_Analysis.ipynb

{
 "cells": [
  {
   "cell_type": "markdown",
   "id": "c3b933de",
   "metadata": {},
   "source": [
    "# United Nations Parallel Corpora Analysis\n",
    "#### Kinan Al-Mouk  - kim47@pitt.edu - March 16th 2022"
   ]
  },
  {
   "cell_type": "markdown",
   "id": "39fc58e9",
   "metadata": {},
   "source": [
    "# Table of Contents\n",
    "[Uploading and Analyzing Raw Data](#Uploading-and-Analyzing-Raw-Data)\n",
    "\n",
    "- [English](#English)\n",
    "\n",
    "- [Spanish](#Spanish)\n",
    "\n",
    "- [French](#French)\n",
    "\n",
    "- [Russian](#Russian)\n",
    "\n",
    "- [Arabic](#Arabic)\n",
    "\n",
    "- [Mandarin](#Mandarin)\n",
    "\n",
    "[Creating Initial DataFrame for Analysis](#DataFrame-Construction)\n",
    "\n",
    "- [Analysis](#Sixway-Analysis)"
   ]
  },
  {
   "cell_type": "markdown",
   "id": "7687c17d",
   "metadata": {},
   "source": [
    "# Uploading and Analyzing Raw Data"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 1,
   "id": "e1ae879c",
   "metadata": {},
   "outputs": [],
   "source": [
    "import nltk \n",
    "import pickle\n",
    "from time import time\n",
    "import numpy as np \n",
    "import pandas as pd "
   ]
  },
  {
   "cell_type": "markdown",
   "id": "50101cfc",
   "metadata": {},
   "source": [
    "# English"
   ]
  },
  {
   "cell_type": "markdown",
   "id": "6575d513",
   "metadata": {},
   "source": [
    "### Reading in English File"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 2,
   "id": "48d62b50",
   "metadata": {},
   "outputs": [
    {
     "name": "stdout",
     "output_type": "stream",
     "text": [
      "Data loaded in: 0.20158886909484863 seconds.\n"
     ]
    }
   ],
   "source": [
    "start = time()\n",
    "f = open('data/sixway/english.100k', 'r') # Reading in English File\n",
    "english100 = f.read()\n",
    "print(\"Data loaded in:\", (time()-start), \"seconds.\")\n",
    "f.close()"
   ]
  },
  {
   "cell_type": "markdown",
   "id": "fb73d9e7",
   "metadata": {},
   "source": [
    "### Word Tokenizing English File"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 3,
   "id": "933a9347",
   "metadata": {},
   "outputs": [
    {
     "name": "stdout",
     "output_type": "stream",
     "text": [
      "English word tokenized in: 18.570607900619507 seconds.\n",
      "Word Token Count for English File: 3105868\n"
     ]
    }
   ],
   "source": [
    "# Word Tokenization\n",
    "start = time()\n",
    "small_en_words = nltk.word_tokenize(english100)\n",
    "print(\"English word tokenized in:\", (time()-start), \"seconds.\")\n",
    "small_en_words_len = len(small_en_words)\n",
    "print(\"Word Token Count for English File:\", small_en_words_len)"
   ]
  },
  {
   "cell_type": "markdown",
   "id": "0c2de4e2",
   "metadata": {},
   "source": [
    "### Sentence Tokenizing English File"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 4,
   "id": "6bdd49a9",
   "metadata": {},
   "outputs": [
    {
     "name": "stdout",
     "output_type": "stream",
     "text": [
      "English sentence tokenized in: 5.034231901168823 seconds.\n",
      "Sentence Token Count for English File: 108375\n"
     ]
    }
   ],
   "source": [
    "# Sentence Tokenization\n",
    "start = time()\n",
    "small_en_sents = nltk.sent_tokenize(english100)\n",
    "print(\"English sentence tokenized in:\", (time()-start), \"seconds.\")\n",
    "small_en_sents_len = len(small_en_sents)\n",
    "print(\"Sentence Token Count for English File:\", small_en_sents_len)\n"
   ]
  },
  {
   "cell_type": "markdown",
   "id": "47870855",
   "metadata": {},
   "source": [
    "# Spanish"
   ]
  },
  {
   "cell_type": "markdown",
   "id": "fac689b4",
   "metadata": {},
   "source": [
    "### Reading in Spanish File"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 5,
   "id": "069f7ebd",
   "metadata": {},
   "outputs": [
    {
     "name": "stdout",
     "output_type": "stream",
     "text": [
      "Data loaded in: 0.18303704261779785 seconds.\n"
     ]
    }
   ],
   "source": [
    "start = time()\n",
    "f = open('data/sixway/spanish.100k', 'r') # Reading in Spanish File\n",
    "spanish100 = f.read()\n",
    "print(\"Data loaded in:\", (time()-start), \"seconds.\")\n",
    "f.close()"
   ]
  },
  {
   "cell_type": "markdown",
   "id": "b50c1532",
   "metadata": {},
   "source": [
    "### Word Tokenizing Spanish File"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 6,
   "id": "dccd47a7",
   "metadata": {},
   "outputs": [
    {
     "name": "stdout",
     "output_type": "stream",
     "text": [
      "Spanish word tokenized in: 20.98197078704834 seconds.\n",
      "Word Token Count for Spanish File: 3504309\n"
     ]
    }
   ],
   "source": [
    "start = time()\n",
    "small_es_words = nltk.word_tokenize(spanish100)\n",
    "print(\"Spanish word tokenized in:\", (time()-start), \"seconds.\")\n",
    "small_es_words_len = len(small_es_words)\n",
    "print(\"Word Token Count for Spanish File:\", small_es_words_len)"
   ]
  },
  {
   "cell_type": "markdown",
   "id": "49eb9410",
   "metadata": {},
   "source": [
    "### Sentence Tokenizing Spanish File"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 7,
   "id": "433ea6fb",
   "metadata": {},
   "outputs": [
    {
     "name": "stdout",
     "output_type": "stream",
     "text": [
      "Spanish sentence tokenized in: 5.2910449504852295 seconds.\n",
      "Sentence Token Count for English File: 102386\n"
     ]
    }
   ],
   "source": [
    "# Sentence Tokenization\n",
    "start = time()\n",
    "small_es_sents = nltk.sent_tokenize(spanish100)\n",
    "print(\"Spanish sentence tokenized in:\", (time()-start), \"seconds.\")\n",
    "small_es_sents_len = len(small_es_sents)\n",
    "print(\"Sentence Token Count for English File:\", small_es_sents_len)\n"
   ]
  },
  {
   "cell_type": "markdown",
   "id": "3b1e4ccb",
   "metadata": {},
   "source": [
    "# French"
   ]
  },
  {
   "cell_type": "markdown",
   "id": "304fab5d",
   "metadata": {},
   "source": [
    "### Reading in French File"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 8,
   "id": "e377ba29",
   "metadata": {},
   "outputs": [
    {
     "name": "stdout",
     "output_type": "stream",
     "text": [
      "Data loaded in: 0.25301671028137207 seconds.\n"
     ]
    }
   ],
   "source": [
    "start = time()\n",
    "f = open('data/sixway/french.100k', 'r') # Reading in French File\n",
    "french100 = f.read()\n",
    "print(\"Data loaded in:\", (time()-start), \"seconds.\")\n",
    "f.close()"
   ]
  },
  {
   "cell_type": "markdown",
   "id": "1142a6fc",
   "metadata": {},
   "source": [
    "### Word Tokenizing French File"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 9,
   "id": "258e5839",
   "metadata": {},
   "outputs": [
    {
     "name": "stdout",
     "output_type": "stream",
     "text": [
      "French word tokenized in: 23.894285202026367 seconds.\n",
      "Word Token Count for French File: 3456688\n"
     ]
    }
   ],
   "source": [
    "start = time()\n",
    "small_fr_words = nltk.word_tokenize(french100)\n",
    "print(\"French word tokenized in:\", (time()-start), \"seconds.\")\n",
    "small_fr_words_len = len(small_fr_words)\n",
    "print(\"Word Token Count for French File:\", small_fr_words_len)"
   ]
  },
  {
   "cell_type": "markdown",
   "id": "73ca46f8",
   "metadata": {},
   "source": [
    "### Sentence Tokenizing French File"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 10,
   "id": "7e33be35",
   "metadata": {},
   "outputs": [
    {
     "name": "stdout",
     "output_type": "stream",
     "text": [
      "French sentence tokenized in: 6.414879083633423 seconds.\n",
      "Sentence Token Count for French File: 107730\n"
     ]
    }
   ],
   "source": [
    "# Sentence Tokenization\n",
    "start = time()\n",
    "small_fr_sents = nltk.sent_tokenize(french100)\n",
    "print(\"French sentence tokenized in:\", (time()-start), \"seconds.\")\n",
    "small_fr_sents_len = len(small_fr_sents)\n",
    "print(\"Sentence Token Count for French File:\", small_fr_sents_len)\n"
   ]
  },
  {
   "cell_type": "markdown",
   "id": "6445a0b9",
   "metadata": {},
   "source": [
    "# Russian"
   ]
  },
  {
   "cell_type": "markdown",
   "id": "5cd20723",
   "metadata": {},
   "source": [
    "### Reading in Russian File"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 11,
   "id": "a744f798",
   "metadata": {},
   "outputs": [
    {
     "name": "stdout",
     "output_type": "stream",
     "text": [
      "Data loaded in: 0.1800708770751953 seconds.\n"
     ]
    }
   ],
   "source": [
    "start = time()\n",
    "f = open('data/sixway/russian.100k', 'r') # Reading in Russian File\n",
    "russian100 = f.read()\n",
    "print(\"Data loaded in:\", (time()-start), \"seconds.\")\n",
    "f.close()"
   ]
  },
  {
   "cell_type": "markdown",
   "id": "af737e74",
   "metadata": {},
   "source": [
    "### Word Tokenizing Russian File"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 12,
   "id": "81f80a64",
   "metadata": {},
   "outputs": [
    {
     "name": "stdout",
     "output_type": "stream",
     "text": [
      "Russian word tokenized in: 27.272552013397217 seconds.\n",
      "Word Token Count for Russian File: 2857554\n"
     ]
    }
   ],
   "source": [
    "start = time()\n",
    "small_ru_words = nltk.word_tokenize(russian100)\n",
    "print(\"Russian word tokenized in:\", (time()-start), \"seconds.\")\n",
    "small_ru_words_len = len(small_ru_words)\n",
    "print(\"Word Token Count for Russian File:\", small_ru_words_len)"
   ]
  },
  {
   "cell_type": "markdown",
   "id": "01bbf0a4",
   "metadata": {},
   "source": [
    "### Sentence Tokenizing Russian File"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 13,
   "id": "6b761fe7",
   "metadata": {},
   "outputs": [
    {
     "name": "stdout",
     "output_type": "stream",
     "text": [
      "Russian sentence tokenized in: 7.302166938781738 seconds.\n",
      "Sentence Token Count for Russian File: 108311\n"
     ]
    }
   ],
   "source": [
    "# Sentence Tokenization\n",
    "start = time()\n",
    "small_ru_sents = nltk.sent_tokenize(russian100)\n",
    "print(\"Russian sentence tokenized in:\", (time()-start), \"seconds.\")\n",
    "small_ru_sents_len = len(small_ru_sents)\n",
    "print(\"Sentence Token Count for Russian File:\", small_ru_sents_len)"
   ]
  },
  {
   "cell_type": "markdown",
   "id": "0defdf51",
   "metadata": {},
   "source": [
    "# Arabic"
   ]
  },
  {
   "cell_type": "markdown",
   "id": "2b729b43",
   "metadata": {},
   "source": [
    "### Reading in Arabic File"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 14,
   "id": "33600ebe",
   "metadata": {},
   "outputs": [
    {
     "name": "stdout",
     "output_type": "stream",
     "text": [
      "Data loaded in: 0.1367042064666748 seconds.\n"
     ]
    }
   ],
   "source": [
    "start = time()\n",
    "f = open('data/sixway/arabic.100k', 'r') # Reading in Arabic File\n",
    "arabic100 = f.read()\n",
    "print(\"Data loaded in:\", (time()-start), \"seconds.\")\n",
    "f.close()"
   ]
  },
  {
   "cell_type": "markdown",
   "id": "cd4e0310",
   "metadata": {},
   "source": [
    "### Word Tokenizing Arabic File"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 15,
   "id": "a3412deb",
   "metadata": {},
   "outputs": [
    {
     "name": "stdout",
     "output_type": "stream",
     "text": [
      "Arabic word tokenized in: 17.179253101348877 seconds.\n",
      "Word Token Count for Arabic File: 2564054\n"
     ]
    }
   ],
   "source": [
    "start = time()\n",
    "small_ar_words = nltk.word_tokenize(arabic100)\n",
    "print(\"Arabic word tokenized in:\", (time()-start), \"seconds.\")\n",
    "small_ar_words_len = len(small_ar_words)\n",
    "print(\"Word Token Count for Arabic File:\", small_ar_words_len)"
   ]
  },
  {
   "cell_type": "markdown",
   "id": "fbe5e6ab",
   "metadata": {},
   "source": [
    "### Sentence Tokenizing Arabic File"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 16,
   "id": "d211cc29",
   "metadata": {},
   "outputs": [
    {
     "name": "stdout",
     "output_type": "stream",
     "text": [
      "Arabic sentence tokenized in: 4.092132091522217 seconds.\n",
      "Sentence Token Count for Arabic File: 78687\n"
     ]
    }
   ],
   "source": [
    "# Sentence Tokenization\n",
    "start = time()\n",
    "small_ar_sents = nltk.sent_tokenize(arabic100)\n",
    "print(\"Arabic sentence tokenized in:\", (time()-start), \"seconds.\")\n",
    "small_ar_sents_len = len(small_ar_sents)\n",
    "print(\"Sentence Token Count for Arabic File:\", small_ar_sents_len)"
   ]
  },
  {
   "cell_type": "markdown",
   "id": "d3372e73",
   "metadata": {},
   "source": [
    "# Mandarin"
   ]
  },
  {
   "cell_type": "markdown",
   "id": "0e9b6a33",
   "metadata": {},
   "source": [
    "### Reading in Mandarin File"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 17,
   "id": "6914247d",
   "metadata": {},
   "outputs": [
    {
     "name": "stdout",
     "output_type": "stream",
     "text": [
      "Data loaded in: 0.08935904502868652 seconds.\n"
     ]
    }
   ],
   "source": [
    "start = time()\n",
    "f = open('data/sixway/mandarin.100k', 'r') # Reading in Mandarin File\n",
    "mandarin100 = f.read()\n",
    "print(\"Data loaded in:\", (time()-start), \"seconds.\")\n",
    "f.close()"
   ]
  },
  {
   "cell_type": "markdown",
   "id": "f43c6239",
   "metadata": {},
   "source": [
    "### Word Tokenizing Mandarin File"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 18,
   "id": "1791f0d2",
   "metadata": {},
   "outputs": [
    {
     "name": "stdout",
     "output_type": "stream",
     "text": [
      "Mandarin word tokenized in: 10.04600477218628 seconds.\n",
      "Word Token Count for Mandarin File: 375501\n"
     ]
    }
   ],
   "source": [
    "start = time()\n",
    "small_zh_words = nltk.word_tokenize(mandarin100)\n",
    "print(\"Mandarin word tokenized in:\", (time()-start), \"seconds.\")\n",
    "small_zh_words_len = len(small_zh_words)\n",
    "print(\"Word Token Count for Mandarin File:\", small_zh_words_len)"
   ]
  },
  {
   "cell_type": "markdown",
   "id": "c109b9c5",
   "metadata": {},
   "source": [
    "### Sentence Tokenizing Mandarin File"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 19,
   "id": "77cedb81",
   "metadata": {},
   "outputs": [
    {
     "name": "stdout",
     "output_type": "stream",
     "text": [
      "Mandarin sentence tokenized in: 5.594507932662964 seconds.\n",
      "Sentence Token Count for Mandarin File: 27314\n"
     ]
    }
   ],
   "source": [
    "# Sentence Tokenization\n",
    "start = time()\n",
    "small_zh_sents = nltk.sent_tokenize(mandarin100)\n",
    "print(\"Mandarin sentence tokenized in:\", (time()-start), \"seconds.\")\n",
    "small_zh_sents_len = len(small_zh_sents)\n",
    "print(\"Sentence Token Count for Mandarin File:\", small_zh_sents_len)"
   ]
  },
  {
   "cell_type": "markdown",
   "id": "2fa8e077",
   "metadata": {},
   "source": [
    "# DataFrame Construction"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 36,
   "id": "be75f7eb",
   "metadata": {},
   "outputs": [],
   "source": [
    "data = {'Language': ['English', 'Spanish', 'French', 'Russian', 'Arabic', 'Mandarin'], \n",
    "        'Text': [english100, spanish100, french100, russian100, arabic100, mandarin100],\n",
    "        'Word_Tokens' : [small_en_words, small_es_words, small_fr_words, small_ru_words, small_ar_words, small_zh_words],\n",
    "        'Word_Tokens_Len' : [small_en_words_len, small_es_words_len, small_fr_words_len, small_ru_words_len, small_ar_words_len, small_zh_words_len],  \n",
    "        'Sentence_Tokens' : [small_en_sents, small_es_sents, small_fr_sents, small_ru_sents, small_ar_sents, small_zh_sents],  \n",
    "        'Sentence_Tokens_Len' : [small_en_sents_len, small_es_sents_len, small_fr_sents_len, small_ru_sents_len, small_ar_sents_len, small_zh_sents_len] } \n"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 37,
   "id": "ee9e4b89",
   "metadata": {},
   "outputs": [],
   "source": [
    "sixway_df = pd.DataFrame(data)"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 39,
   "id": "4cc57134",
   "metadata": {},
   "outputs": [
    {
     "data": {
      "text/html": [
       "<div>\n",
       "<style scoped>\n",
       "    .dataframe tbody tr th:only-of-type {\n",
       "        vertical-align: middle;\n",
       "    }\n",
       "\n",
       "    .dataframe tbody tr th {\n",
       "        vertical-align: top;\n",
       "    }\n",
       "\n",
       "    .dataframe thead th {\n",
       "        text-align: right;\n",
       "    }\n",
       "</style>\n",
       "<table border=\"1\" class=\"dataframe\">\n",
       "  <thead>\n",
       "    <tr style=\"text-align: right;\">\n",
       "      <th></th>\n",
       "      <th>Language</th>\n",
       "      <th>Text</th>\n",
       "      <th>Word_Tokens</th>\n",
       "      <th>Word_Tokens_Len</th>\n",
       "      <th>Sentence_Tokens</th>\n",
       "      <th>Sentence_Tokens_Len</th>\n",
       "      <th>Average_Sentence_Length</th>\n",
       "      <th>Type_Count</th>\n",
       "      <th>TTR</th>\n",
       "    </tr>\n",
       "  </thead>\n",
       "  <tbody>\n",
       "    <tr>\n",
       "      <th>0</th>\n",
       "      <td>English</td>\n",
       "      <td>RESOLUTION 918 (1994)\\nAdopted by the Security...</td>\n",
       "      <td>[RESOLUTION, 918, (, 1994, ), Adopted, by, the...</td>\n",
       "      <td>3105868</td>\n",
       "      <td>[RESOLUTION 918 (1994)\\nAdopted by the Securit...</td>\n",
       "      <td>108375</td>\n",
       "      <td>28.658528</td>\n",
       "      <td>52685</td>\n",
       "      <td>0.016963</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>1</th>\n",
       "      <td>Spanish</td>\n",
       "      <td>RESOLUCIÓN 918 (1994)\\nAprobada por el Consejo...</td>\n",
       "      <td>[RESOLUCIÓN, 918, (, 1994, ), Aprobada, por, e...</td>\n",
       "      <td>3504309</td>\n",
       "      <td>[RESOLUCIÓN 918 (1994)\\nAprobada por el Consej...</td>\n",
       "      <td>102386</td>\n",
       "      <td>34.226447</td>\n",
       "      <td>63904</td>\n",
       "      <td>0.018236</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>2</th>\n",
       "      <td>French</td>\n",
       "      <td>RESOLUTION 918 (1994)\\nAdoptée par le Conseil ...</td>\n",
       "      <td>[RESOLUTION, 918, (, 1994, ), Adoptée, par, le...</td>\n",
       "      <td>3456688</td>\n",
       "      <td>[RESOLUTION 918 (1994)\\nAdoptée par le Conseil...</td>\n",
       "      <td>107730</td>\n",
       "      <td>32.086587</td>\n",
       "      <td>59693</td>\n",
       "      <td>0.017269</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>3</th>\n",
       "      <td>Russian</td>\n",
       "      <td>РЕЗОЛЮЦИЯ 918 (1994),\\nпринятая Советом Безопа...</td>\n",
       "      <td>[РЕЗОЛЮЦИЯ, 918, (, 1994, ), ,, принятая, Сове...</td>\n",
       "      <td>2857554</td>\n",
       "      <td>[РЕЗОЛЮЦИЯ 918 (1994),\\nпринятая Советом Безоп...</td>\n",
       "      <td>108311</td>\n",
       "      <td>26.382860</td>\n",
       "      <td>100809</td>\n",
       "      <td>0.035278</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>4</th>\n",
       "      <td>Arabic</td>\n",
       "      <td>القرار ٨١٩ )٤٩٩١(\\nالذي اتخذه مجلس اﻷمن في جلس...</td>\n",
       "      <td>[القرار, ٨١٩, ), ٤٩٩١, (, الذي, اتخذه, مجلس, ا...</td>\n",
       "      <td>2564054</td>\n",
       "      <td>[القرار ٨١٩ )٤٩٩١(\\nالذي اتخذه مجلس اﻷمن في جل...</td>\n",
       "      <td>78687</td>\n",
       "      <td>32.585484</td>\n",
       "      <td>128763</td>\n",
       "      <td>0.050219</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>5</th>\n",
       "      <td>Mandarin</td>\n",
       "      <td>第918(1994)号决议\\n1994年5月17日安全理事会第3377次会议通过\\n安全理事...</td>\n",
       "      <td>[第918, (, 1994, ), 号决议, 1994年5月17日安全理事会第3377次会...</td>\n",
       "      <td>375501</td>\n",
       "      <td>[第918(1994)号决议\\n1994年5月17日安全理事会第3377次会议通过\\n安全理...</td>\n",
       "      <td>27314</td>\n",
       "      <td>13.747565</td>\n",
       "      <td>168982</td>\n",
       "      <td>0.450017</td>\n",
       "    </tr>\n",
       "  </tbody>\n",
       "</table>\n",
       "</div>"
      ],
      "text/plain": [
       "   Language                                               Text  \\\n",
       "0   English  RESOLUTION 918 (1994)\\nAdopted by the Security...   \n",
       "1   Spanish  RESOLUCIÓN 918 (1994)\\nAprobada por el Consejo...   \n",
       "2    French  RESOLUTION 918 (1994)\\nAdoptée par le Conseil ...   \n",
       "3   Russian  РЕЗОЛЮЦИЯ 918 (1994),\\nпринятая Советом Безопа...   \n",
       "4    Arabic  القرار ٨١٩ )٤٩٩١(\\nالذي اتخذه مجلس اﻷمن في جلس...   \n",
       "5  Mandarin  第918(1994)号决议\\n1994年5月17日安全理事会第3377次会议通过\\n安全理事...   \n",
       "\n",
       "                                         Word_Tokens  Word_Tokens_Len  \\\n",
       "0  [RESOLUTION, 918, (, 1994, ), Adopted, by, the...          3105868   \n",
       "1  [RESOLUCIÓN, 918, (, 1994, ), Aprobada, por, e...          3504309   \n",
       "2  [RESOLUTION, 918, (, 1994, ), Adoptée, par, le...          3456688   \n",
       "3  [РЕЗОЛЮЦИЯ, 918, (, 1994, ), ,, принятая, Сове...          2857554   \n",
       "4  [القرار, ٨١٩, ), ٤٩٩١, (, الذي, اتخذه, مجلس, ا...          2564054   \n",
       "5  [第918, (, 1994, ), 号决议, 1994年5月17日安全理事会第3377次会...           375501   \n",
       "\n",
       "                                     Sentence_Tokens  Sentence_Tokens_Len  \\\n",
       "0  [RESOLUTION 918 (1994)\\nAdopted by the Securit...               108375   \n",
       "1  [RESOLUCIÓN 918 (1994)\\nAprobada por el Consej...               102386   \n",
       "2  [RESOLUTION 918 (1994)\\nAdoptée par le Conseil...               107730   \n",
       "3  [РЕЗОЛЮЦИЯ 918 (1994),\\nпринятая Советом Безоп...               108311   \n",
       "4  [القرار ٨١٩ )٤٩٩١(\\nالذي اتخذه مجلس اﻷمن في جل...                78687   \n",
       "5  [第918(1994)号决议\\n1994年5月17日安全理事会第3377次会议通过\\n安全理...                27314   \n",
       "\n",
       "   Average_Sentence_Length  Type_Count       TTR  \n",
       "0                28.658528       52685  0.016963  \n",
       "1                34.226447       63904  0.018236  \n",
       "2                32.086587       59693  0.017269  \n",
       "3                26.382860      100809  0.035278  \n",
       "4                32.585484      128763  0.050219  \n",
       "5                13.747565      168982  0.450017  "
      ]
     },
     "execution_count": 39,
     "metadata": {},
     "output_type": "execute_result"
    }
   ],
   "source": [
    "sixway_df['Average_Sentence_Length'] = sixway_df['Word_Tokens_Len']/sixway_df['Sentence_Tokens_Len']\n",
    "sixway_df['Type_Count'] = sixway_df.Word_Tokens.map(lambda x : len(set(x)))\n",
    "sixway_df['TTR'] = sixway_df.Type_Count / sixway_df.Word_Tokens_Len\n",
    "sixway_df"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 23,
   "id": "23cda15e",
   "metadata": {},
   "outputs": [],
   "source": [
    "sixway_df.to_pickle(\"sixway_df.pkl\")"
   ]
  },
  {
   "cell_type": "markdown",
   "id": "867ae1b2",
   "metadata": {},
   "source": [
    "## Word Tokens Length Comparsion"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 32,
   "id": "88ad2c83",
   "metadata": {},
   "outputs": [
    {
     "data": {
      "image/png": "iVBORw0KGgoAAAANSUhEUgAAAXQAAAERCAYAAABrWly6AAAAOXRFWHRTb2Z0d2FyZQBNYXRwbG90bGliIHZlcnNpb24zLjQuMywgaHR0cHM6Ly9tYXRwbG90bGliLm9yZy/MnkTPAAAACXBIWXMAAAsTAAALEwEAmpwYAAAeC0lEQVR4nO3de5gU1b3u8e8rjGJEMcokUUAgW5QIIuiAgqhsLwGj8RaNmK2EYMLRXNxGY5KzPceoO3l2LsSTI14IMWrIVpOghrDxflQiOIoCgoJgNiqaURJHohhACZff+aPWYDv0TPdAD8OU7+d5+pnqqtWrVnV1v1O9umq1IgIzM2v/dmrrBpiZWWU40M3McsKBbmaWEw50M7OccKCbmeWEA93MLCfaNNAl3SzpDUmLyiz/eUnPS1os6fbWbp+ZWXuitjwPXdLRwGpgSkT0L1G2D/A74NiIeEvSxyLije3RTjOz9qBNj9Aj4jHgb4XzJP2TpPslzZM0S1LftOgrwPUR8VZ6rMPczKzAjtiHPhn4RkQcBnwLuCHNPwA4QNLjkp6UNKrNWmhmtgPq2NYNKCSpMzAMmCqpYfYu6W9HoA8wAugOzJLUPyLe3s7NNDPbIe1QgU72ieHtiBhYZFkd8GRErAdelvQCWcA/vR3bZ2a2w9qhulwi4h2ysD4LQJlD0uJpwD+n+V3JumBeaot2mpntiNr6tMU7gCeAAyXVSTof+BfgfEkLgcXAqan4A8BKSc8DjwKXRcTKtmi3mdmOqE1PWzQzs8rZobpczMxs67XZl6Jdu3aNXr16tdXqzczapXnz5r0ZEdXFlrVZoPfq1Yu5c+e21erNzNolSa80tcxdLmZmOeFANzPLCQe6mVlO7GhXippZC61fv566ujree++9tm6KVVCnTp3o3r07VVVVZT/GgW7WztXV1bH77rvTq1cvCsZAsnYsIli5ciV1dXX07t277Me5y8WsnXvvvffYe++9HeY5Iom99967xZ+6Sga6pE6SnpK0MP1S0FVFyoyQtErSgnS7okWtMLNt4jDPn63Zp+V0uawj+5Wg1ZKqgNmS7ouIJxuVmxURJ7e4BWZmVhElAz2ywV5Wp7tV6eYBYMx2UL2+e09F61v+w5OaXf7Nb36Tnj17cvHFFwMwcuRIevTowU033QTApZdeSrdu3bjkkktavO6ZM2cyYcIEZsyYsXneAw88wHe+8x0Ali1bRrdu3dh1110ZMGAAU6ZM+WDbly/n5JNPZtGisn62eJuMGDGCCRMmUFNT0+rrakpZX4pK6gDMA/Yn+xm4OUWKDU0jJL4OfCsiFhepZzwwHmC//fbb6kbnWaXfjKWUerOalTJs2DCmTp3KxRdfzKZNm3jzzTd55513Ni+vra3lZz/7WVl1bdy4kQ4dOjRbZuTIkYwcORLYMUJ0R1LWl6IRsTH96ER3YIikxj/oPB/oGRGHABPJxi4vVs/kiKiJiJrq6qJDEZhZO3PkkUdSW1sLwOLFi+nfvz+77747b731FuvWrWPJkiUMGjSIhx9+mEGDBnHwwQczbtw41q1bB2TDgFx99dUMHz6cqVOncv/999O3b1+GDx/O3XffXXY7rrnmGvr370///v2L/gN56aWXGDRoEE8//TQvvvgio0aN4rDDDuOoo45i6dKlAIwdO5aLLrqIYcOG8clPfpI777wTgBUrVnD00UczcOBA+vfvz6xZs8pq05o1axg3bhyDBw9m0KBB/OEPfwDg1ltv5YwzzmDUqFH06dOHb3/722VvZ3NadNpiRLwtaSYwClhUMP+dgul7Jd0gqWtEvFmRVprZDmvfffelY8eOvPrqq9TW1jJ06FBee+01nnjiCbp06cKAAQPYtGkTY8eO5eGHH+aAAw5gzJgx3HjjjZu7aTp16sTs2bN577336NOnD4888gj7778/Z599dlltmDdvHrfccgtz5swhIjj88MM55phj+OhHPwrACy+8wOjRo7nlllsYOHAgxx13HJMmTaJPnz7MmTOHr371qzzyyCNAFt6zZ89m6dKlnHLKKZx55pncfvvtjBw5kssvv5yNGzeydu3astr1gx/8gGOPPZabb76Zt99+myFDhnD88ccDsGDBAp555hl22WUXDjzwQL7xjW/Qo0ePFj77H1Qy0CVVA+tTmO8KHA/8qFGZTwB/jYiQNITsyN8/PmFbcJdSPjUcpdfW1nLJJZfw2muvUVtbS5cuXRg2bBgvvPACvXv35oADDgDgi1/8Itdff/3mQG8I7qVLl9K7d2/69OkDwLnnnsvkyZNLrn/27Nmcfvrp7LbbbgCcccYZzJo1i1NOOYX6+npOPfVU7rrrLvr168fq1aupra3lrLPO2vz4hk8LAKeddho77bQTBx10EH/9618BGDx4MOPGjWP9+vWcdtppDBw4sKzn5cEHH2T69OlMmDAByE4xffXVVwE47rjj6NKlCwAHHXQQr7zySusHOrAP8KvUj74T8LuImCHpAoCImAScCVwoaQPwLjA6/MsZZh8aw4YNo7a2lueee47+/fvTo0cPfvrTn7LHHnswbtw4SsVBQxDD1p2u11z9Xbp0oUePHjz++OP069ePTZs2seeee7JgwYKi5XfZZZfN0w31Hn300Tz22GPcc889nHfeeVx22WWMGTOmrHbdddddHHjggR+YP2fOnA+sp0OHDmzYsKFkfaWU7EOPiGcjYlBEDIiI/hFxdZo/KYU5EXFdRPSLiEMi4oiIqN3mlplZu3HkkUcyY8YM9tprLzp06MBee+3F22+/zRNPPMHQoUPp27cvy5cvZ9myZQD8+te/5phjjtminr59+/Lyyy/z4osvAnDHHXeUtf6jjz6aadOmsXbtWtasWcPvf/97jjrqKAB23nlnpk2bxpQpU7j99tvZY4896N27N1OnTgWy0F24cGGz9b/yyit87GMf4ytf+Qrnn38+8+fPL6tdI0eOZOLEiZv/MTzzzDNlPW5r+dJ/s5xpi26mgw8+mDfffJMvfOELH5i3evVqunbtCsAtt9zCWWedxYYNGxg8eDAXXHDBFvV06tSJyZMnc9JJJ9G1a1eGDx9e1imHhx56KGPHjmXIkCEAfPnLX2bQoEEsX74cyD4BzJgxgxNOOIHddtuN2267jQsvvJDvf//7rF+/ntGjR3PIIYc0Wf/MmTP5yU9+QlVVFZ07d97i9MgGJ5100uaxV4YOHcqUKVO4+OKLGTBgABFBr169PnAKZqW12W+K1tTUhH/gYkt572PO+/a1hSVLlvCpT32qrZthraDYvpU0LyKKnqfpsVzMzHLCgW5mlhMOdLMc8Ell+bM1+9SBbtbOderUiZUrVzrUc6RhPPROnTq16HE+y8WsnevevTt1dXXU19e3dVOsghp+saglHOhm7VxVVVWLftXG8stdLmZmOeFANzPLCQe6mVlOONDNzHLCgW5mlhMOdDOznHCgm5nlhAPdzCwnHOhmZjnR7q4U9XjaZmbF+QjdzCwnHOhmZjlRMtAldZL0lKSFkhZLuqpIGUm6VtIySc9KOrR1mmtmZk0ppw99HXBsRKyWVAXMlnRfRDxZUOZEoE+6HQ7cmP6amdl2UvIIPTKr092qdGs8kv6pwJRU9klgT0n7VLapZmbWnLL60CV1kLQAeAN4KCLmNCrSDfhzwf26NM/MzLaTsgI9IjZGxECgOzBEUv9GRVTsYY1nSBovaa6kuf51FTOzymrRWS4R8TYwExjVaFEd0KPgfnfg9SKPnxwRNRFRU11d3bKWmplZs8o5y6Va0p5pelfgeGBpo2LTgTHpbJcjgFURsaLSjTUzs6aVc5bLPsCvJHUg+wfwu4iYIekCgIiYBNwLfAZYBqwFvtRK7TUzsyaUDPSIeBYYVGT+pILpAL5W2aaZmVlL+EpRM7OccKCbmeWEA93MLCfa3fC5Zjuy7Tm8s4d2tsZ8hG5mlhMOdDOznHCgm5nlhAPdzCwnHOhmZjnhQDczywkHuplZTjjQzcxywoFuZpYTDnQzs5xwoJuZ5YQD3cwsJxzoZmY54UA3M8sJB7qZWU440M3McsI/cGFmZdmeP94B/gGPrVHyCF1SD0mPSloiabGkfy1SZoSkVZIWpNsVrdNcMzNrSjlH6BuASyNivqTdgXmSHoqI5xuVmxURJ1e+iWZmVo6SR+gRsSIi5qfpvwNLgG6t3TAzM2uZFn0pKqkXMAiYU2TxUEkLJd0nqV8Tjx8vaa6kufX19S1vrZmZNansQJfUGbgLuDgi3mm0eD7QMyIOASYC04rVERGTI6ImImqqq6u3sslmZlZMWYEuqYoszG+LiLsbL4+IdyJidZq+F6iS1LWiLTUzs2aVc5aLgF8CSyLimibKfCKVQ9KQVO/KSjbUzMyaV85ZLkcC5wHPSVqQ5v0bsB9AREwCzgQulLQBeBcYHRFR+eaamVlTSgZ6RMwGVKLMdcB1lWqUmZm1nC/9NzPLCQe6mVlOONDNzHLCgW5mlhMOdDOznHCgm5nlhAPdzCwnHOhmZjnhQDczywkHuplZTjjQzcxywoFuZpYTDnQzs5xwoJuZ5YQD3cwsJxzoZmY54UA3M8sJB7qZWU440M3McsKBbmaWEyUDXVIPSY9KWiJpsaR/LVJGkq6VtEzSs5IObZ3mmplZUzqWUWYDcGlEzJe0OzBP0kMR8XxBmROBPul2OHBj+mtmZttJySP0iFgREfPT9N+BJUC3RsVOBaZE5klgT0n7VLy1ZmbWpBb1oUvqBQwC5jRa1A34c8H9OrYMfTMza0VlB7qkzsBdwMUR8U7jxUUeEkXqGC9prqS59fX1LWupmZk1q6xAl1RFFua3RcTdRYrUAT0K7ncHXm9cKCImR0RNRNRUV1dvTXvNzKwJ5ZzlIuCXwJKIuKaJYtOBMelslyOAVRGxooLtNDOzEso5y+VI4DzgOUkL0rx/A/YDiIhJwL3AZ4BlwFrgSxVvqZmZNatkoEfEbIr3kReWCeBrlWqUmZm1nK8UNTPLCQe6mVlOONDNzHLCgW5mlhMOdDOznHCgm5nlhAPdzCwnHOhmZjnhQDczywkHuplZTjjQzcxywoFuZpYTDnQzs5xwoJuZ5YQD3cwsJxzoZmY54UA3M8sJB7qZWU440M3McsKBbmaWEw50M7OcKBnokm6W9IakRU0sHyFplaQF6XZF5ZtpZmaldCyjzK3AdcCUZsrMioiTK9IiMzPbKiWP0CPiMeBv26EtZma2DSrVhz5U0kJJ90nq11QhSeMlzZU0t76+vkKrNjMzqEygzwd6RsQhwERgWlMFI2JyRNRERE11dXUFVm1mZg22OdAj4p2IWJ2m7wWqJHXd5paZmVmLbHOgS/qEJKXpIanOldtar5mZtUzJs1wk3QGMALpKqgO+B1QBRMQk4EzgQkkbgHeB0RERrdZiMzMrqmSgR8Q5JZZfR3Zao5mZtSFfKWpmlhMOdDOznHCgm5nlhAPdzCwnHOhmZjnhQDczywkHuplZTjjQzcxywoFuZpYTDnQzs5xwoJuZ5YQD3cwsJxzoZmY54UA3M8sJB7qZWU440M3McsKBbmaWEw50M7OccKCbmeWEA93MLCcc6GZmOVEy0CXdLOkNSYuaWC5J10paJulZSYdWvplmZlZKOUfotwKjmll+ItAn3cYDN257s8zMrKVKBnpEPAb8rZkipwJTIvMksKekfSrVQDMzK08l+tC7AX8uuF+X5m1B0nhJcyXNra+vr8CqzcysQSUCXUXmRbGCETE5Imoioqa6uroCqzYzswaVCPQ6oEfB/e7A6xWo18zMWqASgT4dGJPOdjkCWBURKypQr5mZtUDHUgUk3QGMALpKqgO+B1QBRMQk4F7gM8AyYC3wpdZqrJmZNa1koEfEOSWWB/C1irXIzMy2iq8UNTPLCQe6mVlOONDNzHLCgW5mlhMOdDOznHCgm5nlhAPdzCwnHOhmZjnhQDczywkHuplZTjjQzcxywoFuZpYTDnQzs5xwoJuZ5YQD3cwsJxzoZmY54UA3M8sJB7qZWU440M3McsKBbmaWE2UFuqRRkl6QtEzSd4ssHyFplaQF6XZF5ZtqZmbN6ViqgKQOwPXACUAd8LSk6RHxfKOisyLi5FZoo5mZlaGcI/QhwLKIeCki/gH8Bji1dZtlZmYtVU6gdwP+XHC/Ls1rbKikhZLuk9SvWEWSxkuaK2lufX39VjTXzMyaUk6gq8i8aHR/PtAzIg4BJgLTilUUEZMjoiYiaqqrq1vUUDMza145gV4H9Ci43x14vbBARLwTEavT9L1AlaSuFWulmZmVVE6gPw30kdRb0s7AaGB6YQFJn5CkND0k1buy0o01M7OmlTzLJSI2SPo68ADQAbg5IhZLuiAtnwScCVwoaQPwLjA6Ihp3y5iZWSsqGeiwuRvl3kbzJhVMXwdcV9mmmZlZS/hKUTOznHCgm5nlRFldLmZmedfru/ds1/Ut/+FJFa/TR+hmZjnhQDczywkHuplZTjjQzcxywoFuZpYTDnQzs5xwoJuZ5YQD3cwsJxzoZmY54UA3M8sJB7qZWU440M3McsKBbmaWEw50M7OccKCbmeWEA93MLCcc6GZmOeFANzPLibICXdIoSS9IWibpu0WWS9K1afmzkg6tfFPNzKw5JQNdUgfgeuBE4CDgHEkHNSp2ItAn3cYDN1a4nWZmVkI5R+hDgGUR8VJE/AP4DXBqozKnAlMi8ySwp6R9KtxWMzNrRscyynQD/lxwvw44vIwy3YAVhYUkjSc7ggdYLemFFrV223QF3mzpg/SjVmhJ6/D2FdFOti/P2wbevqK2Yft6NrWgnEBXkXmxFWWIiMnA5DLWWXGS5kZETVuse3vw9rVfed428PZtT+V0udQBPQrudwde34oyZmbWisoJ9KeBPpJ6S9oZGA1Mb1RmOjAmne1yBLAqIlY0rsjMzFpPyS6XiNgg6evAA0AH4OaIWCzpgrR8EnAv8BlgGbAW+FLrNXmrtUlXz3bk7Wu/8rxt4O3bbhSxRVe3mZm1Q75S1MwsJxzoZmY50S4CXdJGSQsKblsMP9CCulanv/tKurOZcr0kLdra9bSgPZdLWpyGTFggqfE5/ttaf22J5asrub5m1tN4H/ZqpfWMkDSjNeousq6GbVok6b8k7VnBum8qckV2m5N0uqSQ1HcrHlv0tSbpaknHb3vrSq4/JP264H5HSfWVer1IGivpuhY+pkbStZVYP5R3HvqO4N2IGFjJCiPideDMStbZUpKGAicDh0bEOkldgZ0ruY6IGFbJ+rZBk/tQksi+z9m0fZu0zTZvk6RfAV8DflCJiiPiy5WopxWcA8wmO9vtysIFkjpExMaWVhgRV1SmaSWtAfpL2jUi3gVOAF7bTuvegqSOETEXmFupOtvFEXpTJC2XdJWk+ZKeazhqkFQt6aE0/+eSXklhWfjYzUfgkvpJeiodbT0rqU8q1kHSL9IR9IOSdq3wJuwDvBkR6wAi4s2IeD1t149Sm56StH9q52clzZH0jKT/J+njaf6Vkm6WNFPSS5IuKtjOhk8k+0h6rOCI8qiCMj+QtFDSkw11trb0/C+RdAMwH+gh6TJJT6d9cFWjclvsB0n7p+dhYdrX/5Sq7yzpTklLJd2W/mG0tifIro4m7YeaNN1V0vI0vcXrTNJuku5J27BI0tlF6rhR0ty0/VcVPIdFX/+tRVJn4EjgfLJAb/hE9Kik24Hn0rxpkual9o5vVMdPU3sfllSd5t0q6cw0PVhSbXo+npK0e4U34z7gpDR9DnBHQduGpHU/k/4emOaPlXS3pPsl/bekHxc85kuS/iTpj+m5aZjf3Ht1sqQHgSkq+ETZ3Pu4bBGxw9+AjcCCgtvZaf5y4Btp+qvATWn6OuB/pulRZFetdk33V6e/vYBFaXoi8C9pemdg17R8AzAwzf8dcG6Ft6tz2p4/ATcAxxRs1+VpegwwI01/lPfPTPoy8NM0fSVQC+xCdhnySqCq0fZeWlBnB2D3NB3AZ9P0j4H/tR324e/T87sJOCIt/zTZ6V8iO9CYARzd3H4A5gCnp+lOwEeAEcAqsovbdiIL2uGttE0Nz20HYCowKt2fCdSk6a7A8mZeZ58DflFQZ5cidexVsJ6ZwIDmXv+t+D48F/hlmq4FDk3P9xqgd0G5hvbuCiwC9i54rTVs/xXAdWn6VrJPyzsDLwGD0/w9gI6V3F/AAODO9HpZkNo/o/H6gOOBu9L02NSuLulxr5BdSLkP8CpQndr+eME2NfdenQfsmu4Xrv9Kmngfl3vLQ5fL3envPOCMND0cOB0gIu6X9FaJ+p8ALpfUHbg7Iv47HdS9HBELCurvtVWtb0JErJZ0GHAU8M/Ab/X+9wN3FPz9P2m6eyqzD9kL6OWC6u6J7Eh/naQ3gI+TXcHb4GngZklVwLSC7foHWXhCto0nVGr7GvnAPlTWh/5KZIO5QRbonwaeSfc7k43e+SpF9kM6cusWEb8HiIj3Ur0AT0VEXbq/gGy/zW6Fbdq1oP55wEMlyhd7nT0HTJD0I7I39qwij/t8OtLtSBYiBwHPpmXFXv+t5RzgZ2n6N+n+PWTPd+Fr8SJJp6fpHmT7cSXZP/Dfpvn/yfttb3AgsCIingaIiHcqvQER8Wx67Z1Ddv1MoS7Ar5R9Qg+gqmDZwxGxCkDS82TjqXQFZkZEfZr/W+CAVL659+r0yLp8iin1Pm5Wu+5ySdalvxt5/zuBFn3EjojbgVOAd4EHJB3bqO7G9VdMRGyMiJkR8T3g62RHbPDBsXAapieSHQEcDPwPsqOFBs22NSIeIzvifQ34taQxadH6SIcHxR7XytYUTAv4j4gYmG77R8Qv07Ji29bcPm71/ZY0/JPqSfam/Vqav4H331ub91Gx11lE/Ak4jKy74j8kfaA/WVJv4FvAcRExgCxAi+33Vt13kvYGjgVuSl1IlwFnk+2HNQXlRpAd3Q6NiEPI/kF3orhiY0JtjwtjpgMTKOhuSf4deDQi+gOfpbz3V1Ptbe69uqb4Q5pdT1nyEOjFzAY+DyDp02Qff5ok6ZPASxFxLdnOHtDqLczWe6De768HGEj2cQ6yN0vD3yfSdBfe/xLniy1cV0/gjYj4BfBLso/LO5IHgHGpnxZJ3SR9rKnC6eitTtJpqfwukj6yXVq6ZVtWARcB30qfgJaThTQUfPFe7HUmaV9gbUT8J1nINN4ve5AFwKrUD3tia25LM84kGyK7Z0T0iogeZEedwxuV6wK8FRFrU5/+EQXLduL95+MLbPmpaSmwr6TBAJJ2l9Qa/6RuBq6OiOcazS98f40to545wAhJe6f9flYTdbXovbot2kuXS8NH2wb3R0Rzpy5eBdyRvmD6I9kwvn9vpvzZwLmS1gN/Aa4meyO1ts7ARGWnu20gGzphPNmZL7tImkP2Jjgnlb8SmCrpNeBJoHcL1jUCuCxt42qyvvkdRkQ8KOlTwBOp22Q1WZ9tc2dNnAf8XNLVwHo++IbariLiGUkLyb4snAD8TtJ5wCMFxYq9zgYDP5G0iWwbLmxU70JJzwCLyfpxH2/1jSnuHOCHjebdRdbeFwvm3Q9cIOlZ4AWy12mDNUA/SfPIvuc4u2AZEfGP9J6dqOyL73fJjvYrempt6o77v0UW/Zisy+USPrjfmqpnhaQryQ64VpB9ud8hLb6SrX+vbrVcXvovaRdgY2Tj0AwFbmymD36Hkz7S1kREi8dYNrMPr/ZyhN5S+5EdIe1E9qXfV9q4PWZmrS6XR+hmZh9Gef1S1MzsQ8eBbmaWEw50M7OccKBbu6TtNEqkWXviQDczywkHuuVGiRHumhqN8n8rG5XxIUl3SPpWmt/UiIm9JM1SNmLgfEnD0vydJN2gbITBGZLu1fsjCB4m6Y/KRiB8II3vYVZxDnTLk9lkozcOIhs86tsFy/oCI4EhwPckVaXA/hwwiGxgq5oy1vEGcEJEHEp2pWPDjxOcQTZI18Fko+sNBUiXhE8EzoyIw8guO6/ImOlmjeX1wiL7cGrpaJTDgT80jHwn6b/KWEcVcJ2kgWTDEjSMrjccmBrZj3T8RdKjaf6BQH/goTSkQQeyy8TNKs6BbnkyEbgmIqanUf+uLFjW0hEbi46YCHwT+CtwSFr+XprfVF0CFkfE0NLNN9s27nKxPGnpCHezgc9K6pRGeTypYNlyioyYmNaxIh2Jn8f7gzHNBj6X+tI/TjYYGmQDVFWnMYVIXT39WrRVZmVyoFt79RFJdQW3S3h/hLtZQMmBzdIPKUwHFpL92MJcslEAIRsx8UJlP7Jd+POFNwBflPQkWXdLw9jWd5H9EMEi4OdkQ6uuioh/kP1D+FEajXEBsKP8zqvljMdysQ81SZ3TL0d9BHgMGB8R87exrr2Bp4AjI+IvlWyvWXPch24fdpMlHUTWT/6rrQ3zZEYa235n4N8d5ra9+QjdzCwn3IduZpYTDnQzs5xwoJuZ5YQD3cwsJxzoZmY58f8Bzt2J1M2as4QAAAAASUVORK5CYII=\n",
      "text/plain": [
       "<Figure size 432x288 with 1 Axes>"
      ]
     },
     "metadata": {
      "needs_background": "light"
     },
     "output_type": "display_data"
    }
   ],
   "source": [
    "sixway_df.plot.bar(x='Language', y='Word_Tokens_Len', rot=0)\n",
    "plt.savefig(\"Word_Tokens_Comparison.png\")"
   ]
  },
  {
   "cell_type": "markdown",
   "id": "4230cc05",
   "metadata": {},
   "source": [
    "## Sentence Tokens Length Comparsion"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 28,
   "id": "fd3fdacc",
   "metadata": {},
   "outputs": [],
   "source": [
    "import matplotlib.pyplot as plt"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 31,
   "id": "51807d8b",
   "metadata": {},
   "outputs": [
    {
     "data": {
      "image/png": "iVBORw0KGgoAAAANSUhEUgAAAYoAAAEGCAYAAAB7DNKzAAAAOXRFWHRTb2Z0d2FyZQBNYXRwbG90bGliIHZlcnNpb24zLjQuMywgaHR0cHM6Ly9tYXRwbG90bGliLm9yZy/MnkTPAAAACXBIWXMAAAsTAAALEwEAmpwYAAAgkklEQVR4nO3de7xVdZ3/8dfbgwKGknKOPhAcD/NTmUQdEwIdTDEUHC95yQLL2/ycMH+EY41NYmloMWE/Z2rEZKLgB1p5GWyUUfNGommEHoRERJMU6QQpeEEgQQ5+fn+s78HNYZ8vnCu39/Px2I+99net73d9176991pr7+9WRGBmZtaY3bZ1B8zMbPvmoDAzsywHhZmZZTkozMwsy0FhZmZZHbZ1B1pbZWVlVFdXb+tumJntUObMmbMiIqrKzdvpgqK6upqamppt3Q0zsx2KpNcam+dDT2ZmluWgMDOzLAeFmZll7XTnKMx2RevXr6e2tpa1a9du667Ydq5Tp0707NmT3XfffavrOCjMdgK1tbXstddeVFdXI2lbd8e2UxHBm2++SW1tLb169drqej70ZLYTWLt2Ld26dXNIWJYkunXr1uQ9TweF2U7CIWFboznPEweFmZll+RyF2U6o+qr7W7W9xeNO2+IyY8eO5ec//zkVFRXstttu/OhHP2LAgAFNXte8efNYunQpp556anO62mzz58/nggsuAGDJkiV07dqVrl27UllZyaOPPrrZ8l26dGH16tVt3q+LL76Y008/nXPPPbfN19UYB0XS2i+sLdmaF55ZvS09P3/86e6sr32nfTpTxqxZs7jvvvt49tln6dixIytWrOD9999vVlvz5s2jpqam3YPiiCOOYN68ecD28ea8PXFQ2E7BQb9tLVu2jMrKSjp27AhAZWXlxnlz5szhq1/9KqtXr6ayspIpU6bQvXt3Bg0axIABA3jsscd45513mDRpEgMGDODaa6/lvffe48knn2T06NGcfvrpjBo1ivnz51NXV8eYMWM488wzmTJlCtOnT+cvf/kLf/jDHzj77LP53ve+B8CDDz7I1VdfzYYNG6isrGTGjBmsWbOmbDtbcvvtt/Ov//qvRASnnXYaN9xwwybzV6xYwRlnnME3v/lN+vfvz5e+9CWWLFkCwA9+8AMGDhzImDFjWLJkCa+88gpLlizhiiuu4PLLL2fNmjV87nOfo7a2lg0bNnDNNdcwbNiwLfZpw4YNXHXVVcycOZN169YxcuRILr30UmbOnMmYMWOorKzk+eefp2/fvvz0pz9t8fkrB4WZtdiQIUO4/vrrOfTQQznppJMYNmwYJ5xwAuvXr2fUqFHce++9VFVVceedd/KNb3yDyZMnA1BXV8fTTz/NAw88wHXXXcejjz7K9ddfT01NDTfffDMAV199NZ/61KeYPHky77zzDv379+ekk04Cir2PuXPn0rFjR3r37s2oUaPo1KkTX/ziF3niiSfo1asXb731FlAcGivXzkc+8pFGt2vp0qV8/etfZ86cOeyzzz4MGTKEe+65h7POOguA119/nU9/+tN85zvf4eSTT+bzn/88X/nKVzjuuONYsmQJQ4cOZeHChQC8+OKLPPbYY6xatYrevXtz2WWX8eCDD3LAAQdw//3FB52VK1du1f09adIkunbtyjPPPMO6desYOHAgQ4YMAWDu3LksWLCAAw44gIEDB/LUU09x3HHHNfER3ZSDwsxarEuXLsyZM4df//rXPPbYYwwbNoxx48bRr18/nn/+eU4++WSg+CTcvXv3jfXOOeccAPr27cvixYvLtv3www8zffp0brzxRqD4KnD9J/bBgwfTtWtXAA477DBee+013n77bY4//viNvxPYd999s+187GMfa3S7nnnmGQYNGkRVVTGo6he+8AWeeOIJzjrrLNavX8/gwYP54Q9/yAknnADAo48+ygsvvLCx/rvvvsuqVasAOO200+jYsSMdO3Zkv/324/XXX+eII47gyiuv5Otf/zqnn346n/zkJ7fq/n744Yd57rnnmDZtGlAEzMsvv8wee+xB//796dmzJwBHHXUUixcvdlCY2fahoqKCQYMGMWjQII444gimTp1K37596dOnD7NmzSpbp/5QVUVFBXV1dWWXiQjuvvtuevfuvUn57NmzN9YvbSMiyh5qaaydnIhodF6HDh3o27cvDz300Mag+OCDD5g1axadO3febPlyfT300EOZM2cODzzwAKNHj2bIkCFce+21W9Wv8ePHM3To0E3KZ86cWXY9LeWvx5pZi7300ku8/PLLG2/PmzePgw46iN69e7N8+fKNQbF+/XoWLFiQbWuvvfba+CkcYOjQoYwfP37jm/bcuXOz9Y899lgef/xxXn31VYCNh56a2g7AgAEDePzxx1mxYgUbNmzg9ttv3xgKkpg8eTIvvvgi48aNA4pDcPWHzOrvh5ylS5ey5557cv7553PllVfy7LPPbrFP9dsyYcIE1q9fD8Dvf/971qxZs1V1m8N7FGY7oelfHtii+kf2/GiTll+9ejWjRo3inXfeoUOHDhx88MFMnDiRPfbYg2nTpnH55ZezcuVK6urquOKKK+jTp0+jbZ144omMGzeOo446itGjR3PNNddwxRVXcOSRRxIRVFdXc9999zVav6qqiokTJ3LOOefwwQcfsN9++/HII480uR2A7t27893vfpcTTzyRiODUU0/d5AR4RUUFd9xxB2eccQZ77703N910EyNHjuTII4+krq6O448/nv/8z/9stP358+fzta99jd12243dd9+dCRMmlF3u0ksv5YorrgDgwAMP5KmnnmLx4sUcffTRRARVVVXcc8892W1pCeV2rXZE/fr1i+b8cZG/NbNj29kfv635euz+f/XXrba+pgaF7VgWLly42bkZSXMiol+55X3oyczMshwUZmaW5aAw2wkEkf2Gjlm95jxPfDJ7F7GzH8Pf1b32znq6dXuXDnvu7VFkrVH1/0fRqVOnJtVzUJjtBMbPfptRwEEfXYFoeVAsXLX57wBs51D/D3dNscWgkDQZOB14IyIOT2X7AncC1cBi4HMR8XaaNxq4BNgAXB4RD6XyvsAUoDPwAPBPERGSOgK3An2BN4FhEbE41bkI+GbqynciYmqTts5sF/Huug8Y+8Sbrdae9wit1Naco5gCnNKg7CpgRkQcAsxIt5F0GDAc6JPq3CKpItWZAIwADkmX+jYvAd6OiIOB7wM3pLb2Bb4FDAD6A9+StE/TN9HMzFpii0EREU8AbzUoPhOo/3Q/FTirpPyOiFgXEa8Ci4D+kroDe0fErCjOpNzaoE59W9OAwSoOsg4FHomIt9LeyiNsHlhmZtbGmvutp/0jYhlAut4vlfcA/liyXG0q65GmG5ZvUici6oCVQLdMW5uRNEJSjaSa5cuXN3OTzMysnNb+emy5s2iRKW9unU0LIyZGRL+I6Fc/yqOZmbWO5gbF6+lwEun6jVReCxxYslxPYGkq71mmfJM6kjoAXSkOdTXWlpmZtaPmBsV04KI0fRFwb0n5cEkdJfWiOGn9dDo8tUrSMen8w4UN6tS3dS7wq3Qe4yFgiKR90knsIanMzMza0dZ8PfZ2YBBQKamW4ptI44C7JF0CLAE+CxARCyTdBbwA1AEjI2JDauoyPvx67C/TBWAScJukRRR7EsNTW29J+jbwTFru+ohoeFLdzMza2BaDIiLOa2TW4EaWHwuMLVNeAxxepnwtKWjKzJsMTN5SH83MrO14rCczM8tyUJiZWZaDwszMshwUZmaW5aAwM7MsB4WZmWU5KMzMLMtBYWZmWQ4KMzPLclCYmVmWg8LMzLIcFGZmluWgMDOzLAeFmZllOSjMzCzLQWFmZlkOCjMzy3JQmJlZloPCzMyyHBRmZpbloDAzsywHhZmZZTkozMwsy0FhZmZZDgozM8tyUJiZWZaDwszMshwUZmaW5aAwM7OsFgWFpK9IWiDpeUm3S+okaV9Jj0h6OV3vU7L8aEmLJL0kaWhJeV9J89O8myQplXeUdGcqny2puiX9NTOzpmt2UEjqAVwO9IuIw4EKYDhwFTAjIg4BZqTbSDosze8DnALcIqkiNTcBGAEcki6npPJLgLcj4mDg+8ANze2vmZk1T0sPPXUAOkvqAOwJLAXOBKam+VOBs9L0mcAdEbEuIl4FFgH9JXUH9o6IWRERwK0N6tS3NQ0YXL+3YWZm7aPZQRERfwJuBJYAy4CVEfEwsH9ELEvLLAP2S1V6AH8saaI2lfVI0w3LN6kTEXXASqBbw75IGiGpRlLN8uXLm7tJZmZWRksOPe1D8Ym/F3AA8BFJ5+eqlCmLTHmuzqYFERMjol9E9Kuqqsp33MzMmqRDC+qeBLwaEcsBJP0C+DvgdUndI2JZOqz0Rlq+FjiwpH5PikNVtWm6YXlpndp0eKsr8FYL+mxm26Hqq+5v1/UtHndau65vR9eScxRLgGMk7ZnOGwwGFgLTgYvSMhcB96bp6cDw9E2mXhQnrZ9Oh6dWSTomtXNhgzr1bZ0L/CqdxzAzs3bS7D2KiJgtaRrwLFAHzAUmAl2AuyRdQhEmn03LL5B0F/BCWn5kRGxIzV0GTAE6A79MF4BJwG2SFlHsSQxvbn/NzKx5WnLoiYj4FvCtBsXrKPYuyi0/FhhbprwGOLxM+VpS0JiZ2bbhX2abmVmWg8LMzLIcFGZmluWgMDOzLAeFmZllOSjMzCzLQWFmZlkOCjMzy3JQmJlZloPCzMyyHBRmZpbloDAzsywHhZmZZTkozMwsy0FhZmZZDgozM8tyUJiZWZaDwszMshwUZmaW5aAwM7MsB4WZmWU5KMzMLMtBYWZmWQ4KMzPLclCYmVmWg8LMzLIcFGZmluWgMDOzLAeFmZlltSgoJH1U0jRJL0paKOlYSftKekTSy+l6n5LlR0taJOklSUNLyvtKmp/m3SRJqbyjpDtT+WxJ1S3pr5mZNV1L9yj+A3gwIv4G+FtgIXAVMCMiDgFmpNtIOgwYDvQBTgFukVSR2pkAjAAOSZdTUvklwNsRcTDwfeCGFvbXzMyaqNlBIWlv4HhgEkBEvB8R7wBnAlPTYlOBs9L0mcAdEbEuIl4FFgH9JXUH9o6IWRERwK0N6tS3NQ0YXL+3YWZm7aMlexR/DSwH/p+kuZJ+IukjwP4RsQwgXe+Xlu8B/LGkfm0q65GmG5ZvUici6oCVQLeGHZE0QlKNpJrly5e3YJPMzKyhlgRFB+BoYEJEfBxYQzrM1IhyewKRKc/V2bQgYmJE9IuIflVVVflem5lZk7QkKGqB2oiYnW5PowiO19PhJNL1GyXLH1hSvyewNJX3LFO+SR1JHYCuwFst6LOZmTVRs4MiIv4M/FFS71Q0GHgBmA5clMouAu5N09OB4embTL0oTlo/nQ5PrZJ0TDr/cGGDOvVtnQv8Kp3HMDOzdtKhhfVHAT+TtAfwCvAPFOFzl6RLgCXAZwEiYoGkuyjCpA4YGREbUjuXAVOAzsAv0wWKE+W3SVpEsScxvIX9NTOzJmpRUETEPKBfmVmDG1l+LDC2THkNcHiZ8rWkoDEzs23Dv8w2M7MsB4WZmWU5KMzMLMtBYWZmWQ4KMzPLclCYmVmWg8LMzLIcFGZmluWgMDOzLAeFmZllOSjMzCzLQWFmZlkOCjMzy3JQmJlZloPCzMyyHBRmZpbloDAzsywHhZmZZTkozMwsy0FhZmZZDgozM8tyUJiZWZaDwszMshwUZmaW5aAwM7MsB4WZmWU5KMzMLMtBYWZmWQ4KMzPLanFQSKqQNFfSfen2vpIekfRyut6nZNnRkhZJeknS0JLyvpLmp3k3SVIq7yjpzlQ+W1J1S/trZmZN0xp7FP8ELCy5fRUwIyIOAWak20g6DBgO9AFOAW6RVJHqTABGAIekyymp/BLg7Yg4GPg+cEMr9NfMzJqgRUEhqSdwGvCTkuIzgalpeipwVkn5HRGxLiJeBRYB/SV1B/aOiFkREcCtDerUtzUNGFy/t2FmZu2jpXsUPwD+BfigpGz/iFgGkK73S+U9gD+WLFebynqk6Yblm9SJiDpgJdCtYSckjZBUI6lm+fLlLdwkMzMr1eygkHQ68EZEzNnaKmXKIlOeq7NpQcTEiOgXEf2qqqq2sjtmZrY1OrSg7kDg05JOBToBe0v6KfC6pO4RsSwdVnojLV8LHFhSvyewNJX3LFNeWqdWUgegK/BWC/psZmZN1Ow9iogYHRE9I6Ka4iT1ryLifGA6cFFa7CLg3jQ9HRievsnUi+Kk9dPp8NQqScek8w8XNqhT39a5aR2b7VGYmVnbackeRWPGAXdJugRYAnwWICIWSLoLeAGoA0ZGxIZU5zJgCtAZ+GW6AEwCbpO0iGJPYngb9NfMzDJaJSgiYiYwM02/CQxuZLmxwNgy5TXA4WXK15KCxszMtg3/MtvMzLIcFGZmluWgMDOzLAeFmZllOSjMzCzLQWFmZlkOCjMzy3JQmJlZloPCzMyyHBRmZpbloDAzsywHhZmZZTkozMwsy0FhZmZZDgozM8tyUJiZWZaDwszMshwUZmaW5aAwM7OsVvnPbDMzK6/6qvvbdX2Lx53W6m16j8LMzLIcFGZmluWgMDOzLAeFmZllOSjMzCzLQWFmZlkOCjMzy3JQmJlZloPCzMyymh0Ukg6U9JikhZIWSPqnVL6vpEckvZyu9ympM1rSIkkvSRpaUt5X0vw07yZJSuUdJd2ZymdLqm7BtpqZWTO0ZI+iDvjniPgYcAwwUtJhwFXAjIg4BJiRbpPmDQf6AKcAt0iqSG1NAEYAh6TLKan8EuDtiDgY+D5wQwv6a2ZmzdDsoIiIZRHxbJpeBSwEegBnAlPTYlOBs9L0mcAdEbEuIl4FFgH9JXUH9o6IWRERwK0N6tS3NQ0YXL+3YWZm7aNVzlGkQ0IfB2YD+0fEMijCBNgvLdYD+GNJtdpU1iNNNyzfpE5E1AErgW6t0WczM9s6LQ4KSV2Au4ErIuLd3KJlyiJTnqvTsA8jJNVIqlm+fPmWumxmZk3QoqCQtDtFSPwsIn6Ril9Ph5NI12+k8lrgwJLqPYGlqbxnmfJN6kjqAHQF3mrYj4iYGBH9IqJfVVVVSzbJzMwaaMm3ngRMAhZGxL+XzJoOXJSmLwLuLSkfnr7J1IvipPXT6fDUKknHpDYvbFCnvq1zgV+l8xhmZtZOWvLHRQOBC4D5kualsquBccBdki4BlgCfBYiIBZLuAl6g+MbUyIjYkOpdBkwBOgO/TBcogug2SYso9iSGt6C/ZmbWDM0Oioh4kvLnEAAGN1JnLDC2THkNcHiZ8rWkoDEzs23Dv8w2M7MsB4WZmWU5KMzMLMtBYWZmWQ4KMzPLclCYmVmWg8LMzLIcFGZmluWgMDOzLAeFmZllOSjMzCzLQWFmZlkOCjMzy3JQmJlZloPCzMyyHBRmZpbloDAzsywHhZmZZTkozMwsy0FhZmZZDgozM8tyUJiZWZaDwszMshwUZmaW5aAwM7MsB4WZmWU5KMzMLMtBYWZmWQ4KMzPLclCYmVnWDhEUkk6R9JKkRZKu2tb9MTPblWz3QSGpAvgh8PfAYcB5kg7btr0yM9t1bPdBAfQHFkXEKxHxPnAHcOY27pOZ2S5DEbGt+5Al6VzglIj4x3T7AmBARHy5ZJkRwIh0szfwUjt2sRJY0Y7ra2/evh2bt2/H1d7bdlBEVJWb0aEdO9FcKlO2SbpFxERgYvt0Z1OSaiKi37ZYd3vw9u3YvH07ru1p23aEQ0+1wIElt3sCS7dRX8zMdjk7QlA8AxwiqZekPYDhwPRt3Cczs13Gdn/oKSLqJH0ZeAioACZHxIJt3K1S2+SQVzvy9u3YvH07ru1m27b7k9lmZrZt7QiHnszMbBtyUJiZWdYuHRSSNkiaV3Jp9vAgklan6wMkTcssVy3p+eaupwn9+YakBZKeS9s2oJXb/80W5q9uzfVl1tPwMaxuo/UMknRfW7RdZl312/S8pP+R9NFWbPsn2+PIBpLOlhSS/qYZdcs+1yRdL+mklvdui+sPSbeV3O4gaXlrPV8kXSzp5ibW6SfpptZYP+wAJ7Pb2HsRcVRrNhgRS4FzW7PNppJ0LHA6cHRErJNUCezRmuuIiL9rzfZaoNHHUJIozsN90L5darGN2yRpKjASGNsaDdf/cHU7dB7wJMW3GseUzpBUEREbmtpgRFzbOl3bojXA4ZI6R8R7wMnAn9pp3ZuR1CEiaoCa1mpzl96jaIykxZKuk/SspPn1n3IkVUl6JJX/SNJr6U24tO7GPQZJfSQ9nT4dPifpkLRYhaQfp0/8D0vq3Mqb0B1YERHrACJiRUQsTdt1Q+rT05IOTv08Q9JsSXMlPSpp/1Q+RtJkSTMlvSLp8pLtrN+D6i7piZJPwJ8sWWaspN9J+m19m20t3f8LJd0CPAscKOlrkp5Jj8F1DZbb7HGQdHC6H36XHuv/lZrvImmapBcl/SwFUVubBfRI/ZopqV+arpS0OE1v9jyT9BFJ96dteF7SsDJtTJBUk7b/upL7sOzzv61I6gIMBC6hCIr6PbjHJP0cmJ/K7pE0J/V3RIM2/i31d4akqlQ2RcXIDkj6hKTfpPvjaUl7tfJm/BI4LU2fB9xe0rf+ad1z03XvVH6xpF9IelDSy5K+V1LnHyT9XtLj6b6pL8+9VidKehi4VSV7wLnX8VaLiF32AmwA5pVchqXyxcCoNP1/gJ+k6ZuB0Wn6FIpfiFem26vTdTXwfJoeD3whTe8BdE7z64CjUvldwPmtvF1d0vb8HrgFOKFku76Rpi8E7kvT+/DhN+D+Efi3ND0G+A3QkWI4gTeB3Rts7z+XtFkB7JWmAzgjTX8P+GY7PIb/ne7fD4Bj0vwhFF8zFMUHo/uA43OPAzAbODtNdwL2BAYBKyl+8LkbxRv4cW20TfX3bQXwXxRD2ADMBPql6UpgceZ59hngxyVtdi3Txr4l65kJHJl7/rfh6/B8YFKa/g1wdLq/1wC9Spar729n4HmgW8lzrX77rwVuTtNTKPbu9wBeAT6RyvcGOrTm4wUcCUxLz5d5qf/3NVwfcBJwd5q+OPWra6r3GsWPi7sDS4Cq1PenSrYp91qdA3ROt0vXP4ZGXsdbe/Ghp8YPPf0iXc8BzknTxwFnA0TEg5Le3kL7s4BvSOoJ/CIiXk4fQl+NiHkl7Vc3q/eNiIjVkvoCnwROBO7Uh+dfbi+5/n6a7pmW6U7xxHy1pLn7o9gzWSfpDWB/il/L13sGmCxpd+Ceku16n+JNGYptPLm1tq+BTR5DFecoXouI36aiIekyN93uAhxC8ULc7HFInzR7RMR/A0TE2tQuwNMRUZtuz6N43J5sg23qXNL+HOCRLSxf7nk2H7hR0g0Ubxi/LlPvc+mTeQeKN6fDgOfSvHLP/7ZyHvCDNH1Hun0/xf1d+ly8XNLZafpAisfxTYoPBnem8p/yYd/r9QaWRcQzABHxbmtvQEQ8l5575wEPNJjdFZiq4ohCALuXzJsRESsBJL0AHETxZj4zIpan8juBQ9Pyudfq9CgOfZWzpddxlg89NW5dut7Ah+dymnSoISJ+DnwaeA94SNKnGrTdsP1WExEbImJmRHwL+DLFJ0zYdJys+unxFJ9YjgAupfh0Uy/b14h4guIT+p+A2yRdmGatj/Rxply9NramZFrAdyPiqHQ5OCImpXnlti33GLf545bUh99BFG8GI1N5HR++Zjc+RuWeZxHxe6AvxWGb70ra5Hi9pF7AlcDgiDiS4o253OPepo+dpG7Ap4CfpENpXwOGUTwOa0qWG0TxafzYiPhbiuDvRHkNfxymMmVtYTpwIyWHnZJvA49FxOHAGWzd66ux/uZeq2vKV8muZ6s4KJrmSeBzAJKGUOwGNkrSXwOvRMRNFE+iI9u8h8V6e+vD8yEAR1Hs1kLxIqy/npWmu/LhybeLmriug4A3IuLHwCSKwwbbk4eA/52OgyOph6T9Gls4fdqslXRWWr6jpD3bpaeb92UlcDlwZdpjW0zx5g8lX5go9zyTdADwl4j4KcWbV8PHZW+KN5aV6Tj337fltmScC9waEQdFRHVEHEjxKfm4Bst1Bd6OiL+kcybHlMzbjQ/vj8+z+V7ei8ABkj4BIGkvSW0RfpOB6yNifoPy0tfXxVvRzmxgkKRu6XH/bCNtNem12hK7+qGn+l38eg9GRO4rstcBt6cTg48Dy4BVmeWHAedLWg/8Gbie4gXa1roA41V8rbIOWEQxDPvpQEdJsyleXOel5ccA/yXpT8BvgV5NWNcg4GtpG1dTnPvYbkTEw5I+BsxKh49WUxwTz32L5gLgR5KuB9az6Qu1XUXEXEm/ozjJeyNwl4qh9n9Vsli559kngP8r6QOKbbisQbu/kzQXWEBxnPypNt+Y8s4DxjUou5uiv38oKXsQ+JKk5yj+RuC3JfPWAH0kzaE4jzSsZB4R8X56zY5X8YWF9yj2Tlr1K9zpsOR/lJn1PYpDT19l08etsXaWSRpD8UFuGcWXMirS7DE0/7XabB7CowkkdQQ2RDH+1LHAhMw5ju1O2rXvFxE76/j9ZtYGdvU9iqb6K4pPdLtRnKz94jbuj5lZm/MehZmZZflktpmZZTkozMwsy0FhZmZZDgqzEmqnUW/NdiQOCjMzy3JQmG3BFkbsbGx03WtUjDL7iKTbJV2ZyhsbAbZa0q9VjID6rKS/S+W7SbpFxYip90l6QB+OiNpX0uMqRlR9KI3/Y9bqHBRmW/YkxWi0H6cYtO5fSub9DTAU6A98S9LuKQg+A3ycYkC9fluxjjeAkyPiaIpfFtf/6cw5FIMDHkExWuixAGloh/HAuRHRl2L4iFb5zwqzhvyDO7Mta+rouscB99aP5Cnpf7ZiHbsDN0s6imJ4kfrRQo8D/iuKP1/6s6THUnlv4HDgkTQ0SQXFcA9mrc5BYbZl44F/j4jpaRTTMSXzmjoCbdkRYIGvAK8Df5vmr03ljbUlYEFEHLvl7pu1jA89mW1ZU0fsfBI4Q1KnNGrtaSXzFlNmBNi0jmVpz+ECPhwE7kngM+lcxf4UgzBCMTBeVRpzjHTIq0+TtspsKzkozDa1p6TakstX+XDEzl8DWxxQMf1BznTgdxR/olNDMaopFCPAXibpNxR/UFPvFuAiSb+lOOxU/98Cd1P8wczzwI8ohqBeGRHvUwTNDWl02XnA9vI/5raT8VhPZm1AUpf0T4N7Ak8AIyLi2Ra21Q14GhgYEX9uzf6a5fgchVnbmCjpMIrzEFObGxLJfem/RfYAvu2QsPbmPQozM8vyOQozM8tyUJiZWZaDwszMshwUZmaW5aAwM7Os/w/ZhAnFsFOr6wAAAABJRU5ErkJggg==\n",
      "text/plain": [
       "<Figure size 432x288 with 1 Axes>"
      ]
     },
     "metadata": {
      "needs_background": "light"
     },
     "output_type": "display_data"
    }
   ],
   "source": [
    "sixway_df.plot.bar(x='Language', y='Sentence_Tokens_Len', rot=0)\n",
    "plt.savefig(\"Sentence_Tokens_Comparison.png\")"
   ]
  },
  {
   "cell_type": "markdown",
   "id": "449d8213",
   "metadata": {},
   "source": [
    "## Average Sentence Length Comparsion"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 30,
   "id": "c505bb3c",
   "metadata": {},
   "outputs": [
    {
     "data": {
      "image/png": "iVBORw0KGgoAAAANSUhEUgAAAXAAAAEGCAYAAAB8Ys7jAAAAOXRFWHRTb2Z0d2FyZQBNYXRwbG90bGliIHZlcnNpb24zLjQuMywgaHR0cHM6Ly9tYXRwbG90bGliLm9yZy/MnkTPAAAACXBIWXMAAAsTAAALEwEAmpwYAAAd2klEQVR4nO3deZRU5b3u8e/DJBgQp1YxqJDE2diIDaKCB5OgqHgdkCDGAW9y8cYYjQbvIscMaHQdY5wSkhhFWHASxQFnTQxIRAVRG7BBFI+oaROEAHKihxaV6Xf/2LvboumhuruqYcPzWatW7drDu9+3q+rpXW/t/ZYiAjMzy542W7sCZmbWPA5wM7OMcoCbmWWUA9zMLKMc4GZmGdWuNXe25557Ro8ePVpzl2ZmmTdv3rwPIqKk9vxWDfAePXowd+7c1tylmVnmSXqvrvnuQjEzy6hGA1xSR0mvSFog6XVJ16bzx0p6X1JFeju1+NU1M7Nq+XShfAZ8LSKqJLUHZkn6c7rstoi4uXjVMzOz+jQa4JFca1+VPmyf3nz9vW0X1q9fz9KlS/n000+3dlXM6NixI927d6d9+/Z5rZ/Xl5iS2gLzgK8Av42IlyWdAlwm6UJgLvDDiPhXHduOAkYB7L///vm1wqyVLF26lC5dutCjRw8kbe3q2A4sIli9ejVLly6lZ8+eeW2T15eYEbExInoB3YG+ko4A7gC+DPQClgO31LPtXRFRFhFlJSVbnAVjtlV9+umn7LHHHg5v2+okscceezTp02CTzkKJiA+BmcDgiFiRBvsmYDzQtyllmW0rHN62rWjqazGfs1BKJO2aTncCvgG8KalbzmpnAYuatGczM2uRfPrAuwGT037wNsADEfGkpD9I6kXyhWYlcEnRamnWSnqMeaqg5VXeeFpe6z3yyCOcffbZLF68mEMOOaSgdSi0iRMncttttyGJTZs2ccMNN3DGGWc0uZzKykpefPFFzjvvvCLUsnGdO3emqqqq8RWb6fbbb2fUqFHsvPPORdtfPmehLASOqmP+BQWtyQ6s0KHRmHxDxVrPlClT6N+/P/fddx9jx45tcXkbN26kbdu2La9YLUuXLuWGG25g/vz5dO3alaqqKlatWtWssiorK7n33nu3WoAX2+233875559fE+DF4CsxzbayqqoqZs+ezYQJE7jvvvsA+POf/8w3v/nNmnVmzpzJ6aefDsC0adM49thj6d27N8OGDas5quvRowfXXXcd/fv358EHH2T8+PH06dOH0tJShg4dytq1awF455136NevH3369OGnP/0pnTt3rtnPL3/5S/r06cORRx7Jz372sy3qunLlSrp06VKzTefOnWvOmHjnnXcYPHgwRx99NAMGDODNN98EYOTIkVx++eUcd9xxfOlLX2Lq1KkAjBkzhhdeeIFevXpx2223sXHjRq6++uqa/d955501bR84cCDnnHMOhxxyCN/61reo/iWx8vJyjjvuOEpLS+nbty9r1qypt5x8NLUNmzZt4tJLL+Xwww9nyJAhnHrqqUydOpVf//rXLFu2jBNPPJETTzyxpvxrrrmG0tJS+vXrx4oVK/KuV30c4GZb2aOPPsrgwYM56KCD2H333Zk/fz6DBg3ipZde4uOPPwbg/vvvZ/jw4XzwwQdcf/31PPPMM8yfP5+ysjJuvfXWmrI6duzIrFmzOPfcczn77LMpLy9nwYIFHHrooUyYMAGAK664giuuuILy8nL23Xffmm2nTZvGkiVLeOWVV6ioqGDevHk8//zzm9W1tLSUvffem549e3LxxRfzxBNP1CwbNWoU48aNY968edx8881ceumlNcuWL1/OrFmzePLJJxkzZgwAN954IwMGDKCiooIrr7ySCRMm0LVrV8rLyykvL2f8+PH87W9/A+DVV1/l9ttv54033uDdd99l9uzZrFu3juHDh/OrX/2KBQsW8Mwzz9CpU6cGy2lMU9vw8MMPU1lZyWuvvcbdd9/NnDlzALj88svZd999efbZZ3n22WcB+Pjjj+nXrx8LFizghBNOYPz48XnVqSGtOpiVmW1pypQp/OAHPwDg3HPPZcqUKfTu3ZvBgwfzxBNPcM455/DUU09x00038dxzz/HGG29w/PHHA7Bu3TqOPfbYmrKGDx9eM71o0SJ+/OMf8+GHH1JVVcXJJ58MwJw5c3j00UcBOO+88xg9ejSQBPi0adM46qikx7SqqoolS5Zwwgkn1JTZtm1bnn76acrLy5kxYwZXXnkl8+bNY/To0bz44osMGzasZt3PPvusZvrMM8+kTZs2HHbYYfUeeU6bNo2FCxfWHN1+9NFHLFmyhA4dOtC3b1+6d+8OQK9evaisrKRr165069aNPn36ALDLLrs0WE5j51ZXVVU1uQ2zZs1i2LBhtGnThn322Wezo+3aOnTowJAhQwA4+uijmT59eoP1yYcD3GwrWr16NX/9619ZtGgRkti4cSOSuOmmmxg+fDi//e1v2X333enTpw9dunQhIhg0aBBTpkyps7wvfOELNdMjR47k0UcfpbS0lEmTJjFz5swG6xIR/OhHP+KSSxo+H0ESffv2pW/fvgwaNIiLL76Yq666il133ZWKioo6t9lpp5022099+x83blzNP5pqM2fO3Gz7tm3bsmHDBiKiztPu6iunMZs2bWpyG5ryo/Dt27evqW91G1rKXShmW9HUqVO58MILee+996isrOQf//gHPXv2ZNasWQwcOJD58+czfvz4miPrfv36MXv2bN5++20A1q5dy1tvvVVn2WvWrKFbt26sX7+ee+65p2Z+v379eOihhwBq+twBTj75ZCZOnFjTp/7++++zcuXKzcpctmwZ8+fPr3lcUVHBAQccwC677ELPnj158MEHgSTYFixY0GDbu3Tpwpo1azbb/x133MH69esBeOutt2q6kOpyyCGHsGzZMsrLy2vau2HDhiaXU605bejfvz8PPfQQmzZtYsWKFZv9k6zdvmLwEbhZjtY+Q2fKlCk1/anVhg4dyr333suAAQMYMmQIkyZNYvLkyQCUlJQwadIkRowYUfPx/vrrr+eggw7aouyf//znHHPMMRxwwAF89atfrQmT6rMjbrnlFk477TS6du0KwEknncTixYtrumQ6d+7MH//4R/baa6+aMtevX8/o0aNZtmwZHTt2pKSkhN///vcA3HPPPXz3u9/l+uuvZ/369Zx77rmUlpbW2/YjjzySdu3aUVpaysiRI7niiiuorKykd+/eRAQlJSU1XT116dChA/fffz/f//73+eSTT+jUqRPPPPMM3/nOd/IqZ+3atTXdMgBXXXVVk9swdOhQZsyYwRFHHMFBBx3EMcccU/P3HDVqFKeccgrdunWr6QcvNDXlI0BLlZWVhX/QYUs+jXDrWbx4MYceeujWrkarWrt2LZ06dUIS9913H1OmTOGxxx7b2tXKrKqqKjp37szq1avp27cvs2fPZp999ml2eXW9JiXNi4iy2uv6CNxsBzNv3jwuu+wyIoJdd92ViRMnbu0qZdqQIUP48MMPWbduHT/5yU9aFN5N5QA328EMGDCg0b5dy19jXw4XkwPcdnj1nc1gsHDph626vyO779qq+9vWNLVL22eh2A6tY8eOrF69uslvHLNCqx4PvGPHjnlv4yNw26F1796dpUuXNns8j+3din990qr7W7ymU6vub1tT/Ys8+XKAW9G15lk2TT3Dpn379nn/+smO6BSfIbVNcxeKmVlGOcDNzDLKAW5mllEOcDOzjHKAm5lllAPczCyjHOBmZhnlADczy6hGA1xSR0mvSFog6XVJ16bzd5c0XdKS9H634lfXzMyq5XME/hnwtYgoBXoBgyX1A8YAMyLiQGBG+tjMzFpJowEeiar0Yfv0FsAZwOR0/mTgzGJU0MzM6pZXH7iktpIqgJXA9Ih4Gdg7IpYDpPd71bPtKElzJc31gEFmZoWTV4BHxMaI6AV0B/pKOiLfHUTEXRFRFhFlJSUlzaymmZnV1qSzUCLiQ2AmMBhYIakbQHq/sv4tzcys0PI5C6VE0q7pdCfgG8CbwOPARelqFwH+VVQzs1aUz3jg3YDJktqSBP4DEfGkpDnAA5K+DfwdGFbEepqZWS2NBnhELASOqmP+auDrxaiUmZk1zldimplllAPczCyjMvGbmK35m4rg3+Uzs2zwEbiZWUY5wM3MMsoBbmaWUQ5wM7OMcoCbmWWUA9zMLKMc4GZmGeUANzPLKAe4mVlGOcDNzDLKAW5mllEOcDOzjHKAm5llVCZGIzTbVnmkTNuafARuZpZRDnAzs4xygJuZZZQD3MwsoxoNcEn7SXpW0mJJr0u6Ip0/VtL7kirS26nFr66ZmVXL5yyUDcAPI2K+pC7APEnT02W3RcTNxauemZnVp9EAj4jlwPJ0eo2kxcAXi10xMzNrWJP6wCX1AI4CXk5nXSZpoaSJknYrdOXMzKx+eQe4pM7AQ8APIuJ/gDuALwO9SI7Qb6lnu1GS5kqau2rVqpbX2MzMgDwDXFJ7kvC+JyIeBoiIFRGxMSI2AeOBvnVtGxF3RURZRJSVlJQUqt5mZju8fM5CETABWBwRt+bM75az2lnAosJXz8zM6pPPWSjHAxcAr0mqSOf9OzBCUi8ggErgkiLUz8zM6pHPWSizANWx6E+Fr46ZmeXLV2KamWWUA9zMLKMc4GZmGeUANzPLKAe4mVlGOcDNzDLKAW5mllEOcDOzjHKAm5lllAPczCyjHOBmZhnlADczyygHuJlZRjnAzcwyygFuZpZRDnAzs4xygJuZZZQD3MwsoxzgZmYZ5QA3M8soB7iZWUY5wM3MMqrRAJe0n6RnJS2W9LqkK9L5u0uaLmlJer9b8atrZmbV8jkC3wD8MCIOBfoB35N0GDAGmBERBwIz0sdmZtZKGg3wiFgeEfPT6TXAYuCLwBnA5HS1ycCZRaqjmZnVoUl94JJ6AEcBLwN7R8RySEIe2KuebUZJmitp7qpVq1pYXTMzq5Z3gEvqDDwE/CAi/iff7SLirogoi4iykpKS5tTRzMzqkFeAS2pPEt73RMTD6ewVkrqly7sBK4tTRTMzq0s+Z6EImAAsjohbcxY9DlyUTl8EPFb46pmZWX3a5bHO8cAFwGuSKtJ5/w7cCDwg6dvA34FhRamhmZnVqdEAj4hZgOpZ/PXCVsfMzPLlKzHNzDLKAW5mllEOcDOzjHKAm5lllAPczCyjHOBmZhnlADczyygHuJlZRjnAzcwyygFuZpZRDnAzs4xygJuZZZQD3MwsoxzgZmYZ5QA3M8soB7iZWUY5wM3MMsoBbmaWUQ5wM7OMcoCbmWWUA9zMLKMaDXBJEyWtlLQoZ95YSe9Lqkhvpxa3mmZmVls+R+CTgMF1zL8tInqltz8VtlpmZtaYRgM8Ip4H/rsV6mJmZk3Qkj7wyyQtTLtYdqtvJUmjJM2VNHfVqlUt2J2ZmeVqboDfAXwZ6AUsB26pb8WIuCsiyiKirKSkpJm7MzOz2poV4BGxIiI2RsQmYDzQt7DVMjOzxjQrwCV1y3l4FrCovnXNzKw42jW2gqQpwEBgT0lLgZ8BAyX1AgKoBC4pXhXNzKwujQZ4RIyoY/aEItTFzMyawFdimplllAPczCyjGu1CMTPbXvUY81Sr7q/yxtMKWp6PwM3MMsoBbmaWUQ5wM7OMcoCbmWWUA9zMLKMc4GZmGeUANzPLKAe4mVlGOcDNzDLKAW5mllEOcDOzjHKAm5lllAPczCyjHOBmZhnlADczyygHuJlZRjnAzcwyygFuZpZRjQa4pImSVkpalDNvd0nTJS1J73crbjXNzKy2fI7AJwGDa80bA8yIiAOBGeljMzNrRY0GeEQ8D/x3rdlnAJPT6cnAmYWtlpmZNaa5feB7R8RygPR+r/pWlDRK0lxJc1etWtXM3ZmZWW1F/xIzIu6KiLKIKCspKSn27szMdhjNDfAVkroBpPcrC1clMzPLR3MD/HHgonT6IuCxwlTHzMzylc9phFOAOcDBkpZK+jZwIzBI0hJgUPrYzMxaUbvGVoiIEfUs+nqB62JmZk3gKzHNzDLKAW5mllEOcDOzjHKAm5lllAPczCyjHOBmZhnlADczyygHuJlZRjnAzcwyygFuZpZRDnAzs4xygJuZZZQD3MwsoxzgZmYZ5QA3M8soB7iZWUY5wM3MMsoBbmaWUQ5wM7OMcoCbmWWUA9zMLKMa/VX6hkiqBNYAG4ENEVFWiEqZmVnjWhTgqRMj4oMClGNmZk3gLhQzs4xqaYAHME3SPEmjClEhMzPLT0u7UI6PiGWS9gKmS3ozIp7PXSEN9lEA+++/fwt3Z2Zm1Vp0BB4Ry9L7lcAjQN861rkrIsoioqykpKQluzMzsxzNDnBJX5DUpXoaOAlYVKiKmZlZw1rShbI38Iik6nLujYinC1IrMzNrVLMDPCLeBUoLWBczM2sCn0ZoZpZRDnAzs4xygJuZZZQD3MwsoxzgZmYZ5QA3M8soB7iZWUY5wM3MMsoBbmaWUQ5wM7OMcoCbmWWUA9zMLKMc4GZmGeUANzPLKAe4mVlGOcDNzDLKAW5mllEOcDOzjHKAm5lllAPczCyjHOBmZhnlADczy6gWBbikwZL+S9LbksYUqlJmZta4Zge4pLbAb4FTgMOAEZIOK1TFzMysYS05Au8LvB0R70bEOuA+4IzCVMvMzBqjiGjehtI5wOCI+E76+ALgmIi4rNZ6o4BR6cODgf9qfnWbbE/gg1bcX2vbntu3PbcN3L6sa+32HRARJbVntmtBgapj3hb/DSLiLuCuFuyn2STNjYiyrbHv1rA9t297bhu4fVm3rbSvJV0oS4H9ch53B5a1rDpmZpavlgR4OXCgpJ6SOgDnAo8XplpmZtaYZnehRMQGSZcBfwHaAhMj4vWC1awwtkrXTSvantu3PbcN3L6s2yba1+wvMc3MbOvylZhmZhnlADczy6htNsAlbZRUkXNr9qX6kqrS+30lTW1gvR6SFjV3P02ozzWSXpe0MG3bMQUu/8VGllcVcn8N7Kf2c9ijSPsZKOnJYpRdx76q27RI0hOSdi1g2Xdva1czSzpLUkg6pBnb1vk6k3SdpG+0vHaN7j8k/SHncTtJqwr1WpE0UtJvmrhNmaRfF2L/0LLzwIvtk4joVcgCI2IZcE4hy2wqSccCQ4DeEfGZpD2BDoXcR0QcV8jyWqDe51CSSL6D2dS6VWqxmjZJmgx8D7ihEAVXXxS3jRkBzCI5y2xs7gJJbSNiY1MLjIifFqZqjfoYOEJSp4j4BBgEvN9K+96CpHYRMReYW6gyt9kj8PpIqpR0raT5kl6rPjKQVCJpejr/TknvpeGYu23NEbakwyW9kh5NLZR0YLpaW0nj0yPkaZI6FbgJ3YAPIuIzgIj4ICKWpe36RVqnVyR9Ja3n6ZJelvSqpGck7Z3OHytpoqSZkt6VdHlOO6s/cXST9HzOEeOAnHVukLRA0kvVZRZb+vdfLOl3wHxgP0lXSypPn4Nra623xfMg6Svp32FB+lx/OS2+s6Spkt6UdE/6D6LY5gBfTOs1U1JZOr2npMp0eovXmaQvSHoqbcMiScPrKOMOSXPT9l+b8zes8/VfDJI6A8cD3yYJ8OpPO89Kuhd4LZ33qKR5aV1H1SrjlrSuMySVpPMmKbmSG0l9JL2Y/i1ekdSlwM34M3BaOj0CmJJTt77pvl9N7w9O54+U9LCkpyUtkXRTzjYXS3pL0nPp36Z6fkPv07skTQP+UzmfFht6D+ctIrbJG7ARqMi5DU/nVwLfT6cvBe5Op38D/CidHkxyVeie6eOq9L4HsCidHgd8K53uAHRKl28AeqXzHwDOL3C7OqfteQv4HfBvOe26Jp2+EHgynd6Nz88W+g5wSzo9FngR2Inkst7VQPta7f1hTpltgS7pdACnp9M3AT9uhefwkfTvuwnoly4/ieR0LJEcTDwJnNDQ8wC8DJyVTncEdgYGAh+RXEzWhiRY+xepTdV/27bAgyTDSQDMBMrS6T2BygZeZ0OB8Tlldq2jjN1z9jMTOLKh13+R2no+MCGdfhHonf6tPwZ65qxXXddOwCJgj5zXWXXbfwr8Jp2eRPJJuAPwLtAnnb8L0K6QzxVwJDA1fa1UpPV/svb+gG8AD6XTI9N6dU23e4/kosVuwN+BkrTus3Pa1ND7dB7QKX2cu/+x1PMezveW1S6Uh9P7ecDZ6XR/4CyAiHha0r8aKX8OcI2k7sDDEbEkPWj7W0RU5JTfo1m1r0dEVEk6GhgAnAjcr8/796fk3N+WTndP1+lG8qL5W05xT0VyJP+ZpJXA3iRXyFYrByZKag88mtOudSRhCUkbBxWqfbVs9hwq6QN/LyJeSmedlN5eTR93Bg4keZNs8TykR2dfjIhHACLi07RcgFciYmn6uILkeZtVhDZ1yil/HjC9kfXrep29Btws6Rckb+YX6tjum+nRbDuS4DgMWJguq+v1XwwjgNvT6fvSx0+R/K1zX4eXSzornd6P5DlcTfLP+v50/h/5vN7VDgaWR0Q5QET8T6EbEBEL09fdCOBPtRZ3BSYr+fQdQPucZTMi4iMASW8AB5CE7MyIWJXOvx84KF2/offp45F04dSlsfdwgzLXhZL6LL3fyOf9+E36yBwR9wL/C/gE+Iukr9Uqu3b5BRMRGyNiZkT8DLiM5IgMNh9Lpnp6HMl/+a8Cl5AcEVRrsK4R8TzJEe37wB8kXZguWh/pIUBd2xXZxznTAv4jInqlt69ExIR0WV1ta+g5Lvrzlqr+p3QAyRv1e+n8DXz+fqp5jup6nUXEW8DRJF0Q/yFpsz5hST2B0cDXI+JIktCs63kvWjsl7QF8Dbg77Q66GhhO8hx8nLPeQJKj12MjopTkn3FH6lb7ohPVMa8YHgduJqf7JPVz4NmIOAI4nfzeW/XVt6H36cd1b9LgfvKS1QCvyyzgmwCSTiL5SFMvSV8C3o2IX5M8wUcWvYbJfg/W5/3tAL1IPqJB8gapvp+TTnfl8y9eLmrivg4AVkbEeGACyUfgbclfgP+d9rUi6YuS9qpv5fQIbamkM9P1d5K0c6vUdMu6fARcDoxOP+FUkoQy5HxRXtfrTNK+wNqI+CNJsNR+XnYhedN/lPalnlLMttTjHOA/I+KAiOgREfuRHFX2r7VeV+BfEbE27Y/vl7OsDZ//Lc5jy09EbwL7SuoDIKmLpGL8Q5oIXBcRr9Wan/veGplHOS8DAyXtkT7nw+opq0nv05bYlrtQqj+qVns6Iho6lfBaYEr6hdBzwHJgTQPrDwfOl7Qe+CdwHckbp9g6A+OUnH62AXibZLjdIcBOkl4meeGPSNcfCzwo6X3gJaBnE/Y1ELg6bWMVSd/6NiMipkk6FJiTdoNUkfS7NnRmwwXAnZKuA9az+ZuoVUXEq5IWkHzBdzPwgJJhlf+as1pdr7M+wC8lbSJpw3drlbtA0qvA6yR9sbOL3pgtjQBurDXvIZK6vpMz72ng/0paSDJU9Es5yz4GDpc0j+Q7iuE5y4iIden7dZySL6k/ITmaL+hprmnX2q/qWHQTSRfKVWz+nNVXznJJY0kOrpaTfBHfNl08lua/T5ttu7mUXtJOwMZIxmg5FrijgT70bU76MbUsIrbnMZTNrIC25SPwptqf5AioDcmXdP9nK9fHzKyotpsjcDOzHc329CWmmdkOxQFuZpZRDnAzs4xygFtmqJVGUTTLCge4mVlGOcAt0xoZBa6+0Rp/omTUwumSpkganc6vb0TBHpJeUDKq3nxJx6Xz20j6nZJR+J6U9Cd9Psre0ZKeUzJK31/SMTLMCsoBblk3i2R0w6NIBlz6fznLDgFOBvoCP5PUPg3oocBRJANBleWxj5XAoIjoTXI1YfWA/GeTDGr1VZIR6I4FSC+zHgecExFHk1zKXZAxw81ybU8X8tiOqamjNfYHHqseHU7SE3nsoz3wG0m9SC7zrx6Brj/wYCQ/SvFPSc+m8w8GjgCmp0MEtCW59NqsoBzglnXjgFsj4vF0ZLyxOcuaOqJhnSMKAlcCK4DSdPmn6fz6yhLwekQc23j1zZrPXSiWdU0dBW4WcLqkjukoiKflLKukjhEF030sT4+0L+DzAYxmAUPTvvC9SQYPg2RQp5J0TB7SrpvDm9Qqszw4wC1Ldpa0NOd2FZ+PAvcC0OhAYOmPBzwOLCD5gYG5JCPlQTKi4HeV/Ch07s/x/Q64SNJLJN0n1eM7P0Qy+P4i4E6S4UY/ioh1JP8AfpGOVlgBbCu/U2rbEY+FYjscSZ3TX0baGXgeGBUR81tY1h7AK8DxEfHPQtbXrD7uA7cd0V2SDiPp557c3PBOPZmO7d4B+LnD21qTj8DNzDLKfeBmZhnlADczyygHuJlZRjnAzcwyygFuZpZR/x9AythaBTlyGQAAAABJRU5ErkJggg==\n",
      "text/plain": [
       "<Figure size 432x288 with 1 Axes>"
      ]
     },
     "metadata": {
      "needs_background": "light"
     },
     "output_type": "display_data"
    }
   ],
   "source": [
    "sixway_df.plot.bar(x='Language', y='Average_Sentence_Length', rot=0)\n",
    "plt.savefig(\"Average_Sentence_Len_Comparison.png\")"
   ]
  },
  {
   "cell_type": "markdown",
   "id": "78db3bfb",
   "metadata": {},
   "source": [
    "## TTR Comparison"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 40,
   "id": "7ed8cdd1",
   "metadata": {},
   "outputs": [
    {
     "data": {
      "image/png": "iVBORw0KGgoAAAANSUhEUgAAAXQAAAEGCAYAAAB1iW6ZAAAAOXRFWHRTb2Z0d2FyZQBNYXRwbG90bGliIHZlcnNpb24zLjQuMywgaHR0cHM6Ly9tYXRwbG90bGliLm9yZy/MnkTPAAAACXBIWXMAAAsTAAALEwEAmpwYAAAV7klEQVR4nO3df7RlZX3f8ffHy+BgwEk7M7XIoDNViqIi0ZFqgo2JwULUApEGiT9CEktJQ1xtig1dtsmoK0uwpk3FH4jAwiSNiD87ASJa6w8IgjPogGDFjDjGC0wdpy5S0HEAv/1j78scLmfuPXfm3Ll3nnm/1rrr7h/P3fvZZ5/9uc95ztnPSVUhSdr/PW6hKyBJGg8DXZIaYaBLUiMMdElqhIEuSY04aKF2vGLFilq9evVC7V6S9ku33HLL96tq5bB1Cxboq1evZuPGjQu1e0naLyX5zu7W2eUiSY0w0CWpEQa6JDViwfrQh3nwwQeZnJxkx44dC12VsVq6dCmrVq1iyZIlC10VSQ1bVIE+OTnJYYcdxurVq0my0NUZi6pi+/btTE5OsmbNmoWujqSGLaoulx07drB8+fJmwhwgCcuXL2/uVYekxWdRBTrQVJhPafGYJC0+iy7QJUl7ZlH1oU+3+vxrxrq9LRe8fMb127dv56UvfSkAW7duZWJigq1bt/Kc5zyHnTt3snXrVpYtW8ayZctYsWIFl156Kc985jM5+uij2blzJ2vXruWyyy7zzU9JC2JRB/q+tnz5cjZt2gTAunXrOPTQQznvvPMeWX/WWWfxile8gtNPPx2ALVu28LSnPY1Nmzbx8MMPc+KJJ3LVVVfxmte8ZiGqL2kvjLsBOZvZGph7wi6XMZmYmOD444/n7rvvXuiqSDpAGehjsmPHDm6++WZOOumkha6KpAOUgb6XvvWtb3HcccexfPlynvKUp3DssccudJUkHaAM9L001Ye+efNmbrrpJtavX7/QVZJ0gDLQx+Twww/nggsu4O1vf/tCV0XSAWpRf8plPt4Fnk+nnnoq69at4/rrr+fFL37xQldH0gFmUQf6Qlq3bt1jll1xxRWPml+9ejW33377I/NJuPXWW+e5ZpI0nF0uktQIA12SGrHoAr2qFroKY9fiMUlafBZVoC9dupTt27c3FYBT46EvXbp0oasiqXGL6k3RVatWMTk5ybZt2xa6KmM19Y1FkjSfFlWgL1myxG/1kaQ9tKi6XCRJe85Al6RGGOiS1AgDXZIaYaBLUiMMdElqxEiBnuSkJHcm2Zzk/BnKvSDJw0lOH18VJUmjmDXQk0wA7wFOBo4BzkxyzG7KXQhcN+5KSpJmN0oL/Xhgc1XdVVU7gSuBU4aU+13gY8D3xlg/SdKIRgn0I4DvDsxP9ssekeQI4DTg4pk2lOTsJBuTbGzt9n5JWmijBHqGLJs+etafAL9fVQ/PtKGquqSq1lbV2pUrV45YRUnSKEYZy2USOHJgfhVwz7Qya4ErkwCsAH45yUNV9clxVFKSNLtRAn0DcFSSNcDdwKuBXxssUFWPjKiV5ArgasNckvatWQO9qh5Kci7dp1cmgMur6o4k5/TrZ+w3lyTtGyMNn1tV1wLXTls2NMir6qy9r5Ykaa68U1SSGmGgS1IjDHRJaoSBLkmNMNAlqREGuiQ1wkCXpEYY6JLUCANdkhphoEtSIwx0SWqEgS5JjTDQJakRBrokNcJAl6RGGOiS1AgDXZIaYaBLUiMMdElqhIEuSY0w0CWpEQa6JDXCQJekRhjoktQIA12SGmGgS1IjDHRJaoSBLkmNMNAlqREGuiQ1wkCXpEYY6JLUCANdkhphoEtSIwx0SWqEgS5JjTDQJakRIwV6kpOS3Jlkc5Lzh6w/JcltSTYl2ZjkhPFXVZI0k4NmK5BkAngPcCIwCWxIsr6qvj5Q7LPA+qqqJMcCVwHPmI8KS5KGG6WFfjywuaruqqqdwJXAKYMFqur+qqp+9qeAQpK0T40S6EcA3x2Yn+yXPUqS05J8A7gG+M1hG0pydt8ls3Hbtm17Ul9J0m6MEugZsuwxLfCq+kRVPQM4FXjbsA1V1SVVtbaq1q5cuXJOFZUkzWyUQJ8EjhyYXwXcs7vCVfVF4GlJVuxl3SRJczBKoG8AjkqyJsnBwKuB9YMFkjw9Sfrp5wEHA9vHXVlJ0u7N+imXqnooybnAdcAEcHlV3ZHknH79xcCrgNcneRD4EXDGwJukkqR9YNZAB6iqa4Frpy27eGD6QuDC8VZNkjQX3ikqSY0w0CWpEQa6JDXCQJekRhjoktQIA12SGmGgS1IjDHRJaoSBLkmNMNAlqREGuiQ1wkCXpEYY6JLUCANdkhphoEtSIwx0SWqEgS5JjTDQJakRBrokNcJAl6RGGOiS1AgDXZIaYaBLUiMMdElqhIEuSY0w0CWpEQa6JDXCQJekRhjoktQIA12SGmGgS1IjDHRJaoSBLkmNMNAlqREGuiQ1wkCXpEYY6JLUiJECPclJSe5MsjnJ+UPWvybJbf3PjUmeO/6qSpJmMmugJ5kA3gOcDBwDnJnkmGnFvg38fFUdC7wNuGTcFZUkzWyUFvrxwOaququqdgJXAqcMFqiqG6vqB/3sTcCq8VZTkjSbUQL9COC7A/OT/bLd+S3gr4atSHJ2ko1JNm7btm30WkqSZjVKoGfIshpaMPkFukD//WHrq+qSqlpbVWtXrlw5ei0lSbM6aIQyk8CRA/OrgHumF0pyLHApcHJVbR9P9SRJoxqlhb4BOCrJmiQHA68G1g8WSPIU4OPA66rqm+OvpiRpNrO20KvqoSTnAtcBE8DlVXVHknP69RcDfwAsB96bBOChqlo7f9WWJE03SpcLVXUtcO20ZRcPTL8BeMN4qyZJmgvvFJWkRhjoktQIA12SGmGgS1IjDHRJaoSBLkmNMNAlqREGuiQ1wkCXpEYY6JLUCANdkhphoEtSIwx0SWqEgS5JjTDQJakRBrokNcJAl6RGGOiS1AgDXZIaYaBLUiMMdElqhIEuSY0w0CWpEQa6JDXCQJekRhjoktQIA12SGmGgS1IjDHRJaoSBLkmNMNAlqREGuiQ1wkCXpEYY6JLUCANdkhphoEtSIwx0SWrESIGe5KQkdybZnOT8IeufkeRLSX6c5LzxV1OSNJuDZiuQZAJ4D3AiMAlsSLK+qr4+UOz/Am8ETp2PSkqSZjdKC/14YHNV3VVVO4ErgVMGC1TV96pqA/DgPNRRkjSCUQL9COC7A/OT/bI5S3J2ko1JNm7btm1PNiFJ2o1RAj1DltWe7KyqLqmqtVW1duXKlXuyCUnSbowS6JPAkQPzq4B75qc6kqQ9NUqgbwCOSrImycHAq4H181stSdJczfopl6p6KMm5wHXABHB5Vd2R5Jx+/cVJ/iGwEXgi8JMk/wY4pqr+bv6qLkkaNGugA1TVtcC105ZdPDC9la4rRpK0QLxTVJIaYaBLUiMMdElqhIEuSY0w0CWpEQa6JDXCQJekRhjoktQIA12SGmGgS1IjDHRJaoSBLkmNMNAlqREGuiQ1wkCXpEYY6JLUCANdkhox0jcWSdLq86/Zp/vbcsHL9+n+WmALXZIaYaBLUiMMdElqhIEuSY0w0CWpEQa6JDXCjy1KY7QvP9rnx/o0nS10SWqEgS5JjTDQJakRBrokNcJAl6RGGOiS1Ag/trjIOKKdpD213wW6gbd/8/xJ88cuF0lqhIEuSY0w0CWpEQa6JDVipEBPclKSO5NsTnL+kPVJ8q5+/W1Jnjf+qkqSZjJroCeZAN4DnAwcA5yZ5JhpxU4Gjup/zgbeN+Z6SpJmMUoL/Xhgc1XdVVU7gSuBU6aVOQX40+rcBPx0ksPHXFdJ0gxG+Rz6EcB3B+YngX8yQpkjgHsHCyU5m64FD3B/kjvnVNu9swL4/lz/KBfOQ03mh8c3xH5yfC0fG3h8Q+3F8T11dytGCfQMWVZ7UIaqugS4ZIR9jl2SjVW1diH2vS94fPuvlo8NPL59aZQul0ngyIH5VcA9e1BGkjSPRgn0DcBRSdYkORh4NbB+Wpn1wOv7T7u8ELivqu6dviFJ0vyZtculqh5Kci5wHTABXF5VdyQ5p19/MXAt8MvAZuCHwG/MX5X32IJ09exDHt/+q+VjA49vn0nVY7q6JUn7Ie8UlaRGGOiS1Ij9ItCTPJxk08DPY4YfmMO27u9/PznJR2cotzrJ7Xu6nznU581J7uiHTNiUZPpn/Pd2+zfOsv7+ce5vhv1MP4er52k/L0ly9Xxse8i+po7p9iR/meSnx7jtS4fckb3gkpyWpJI8Yw/+duhzLclbk/zS3tdu1v1Xkj8bmD8oybZxPV+SnJXk3XP8m7VJ3jWO/cP+8wUXP6qq48a5waq6Bzh9nNucqyQvAl4BPK+qfpxkBXDwOPdRVT87zu3thd2ewyShez/nJ/u2SnvtkWNK8kHgd4A/GseGq+oN49jOPDgTuIHu027rBlckmaiqh+e6war6g/FUbVYPAM9OckhV/Qg4Ebh7H+37MZIcVFUbgY3j2uZ+0ULfnSRbkrwlyVeSfG2q1ZBkZZLP9Mvfn+Q7fVgO/u0jLfAkz0ry5b61dVuSo/piE0k+0LegP53kkDEfwuHA96vqxwBV9f2quqc/rgv7On05ydP7er4yyc1JvprkfyZ5Ur98XZLLk3w+yV1J3jhwnFOvSA5P8sWBFuWLB8r8UZJbk9w0tc351j/+/zvJe4GvAEcmeVOSDf05eMu0co85D0me3j8Ot/bn+mn95g9N8tEk30jy3/t/GPPtS3R3R9Ofh7X99IokW/rpxzzPkvxUkmv6Y7g9yRlDtvG+JBv743/LwGM49Pk/X5IcCvwc8Ft0gT71iuhzSf4C+Fq/7JNJbunre/a0bfxxX9/PJlnZL7siyen99AuS3Ng/Hl9OctiYD+OvgKmvsToT+NBA3Y7v9/3V/vfR/fKzknw8yaeS/E2Sdwz8zW8k+WaSL/SPzdTyma7VS5J8GvjTDLyinOk6HllVLfof4GFg08DPGf3yLcDv9tP/Gri0n3438B/66ZPo7lpd0c/f3/9eDdzeT18EvKafPhg4pF//EHBcv/wq4LVjPq5D++P5JvBe4OcHjuvN/fTrgav76b/Hrk8mvQH44356HXAj8Hi625C3A0umHe+/G9jmBHBYP13AK/vpdwD/cR+cw0/0j+9PgBf2619G9/Gv0DU0rgb+6UznAbgZOK2fXgo8AXgJcB/dzW2PowvaE+bpmKYe2wngI8BJ/fzngbX99ApgywzPs1cBHxjY5rIh2/j7A/v5PHDsTM//ebwOXwtc1k/fCDyvf7wfANYMlJuq7yHA7cDygefa1PH/AfDufvoKulfLBwN3AS/olz8ROGic5ws4Fvho/3zZ1Nf/6un7A34J+Fg/fVZfr2X9332H7kbKw4G/BVb2df/rgWOa6Vq9BTiknx/c/zp2cx2P+tNCl8vH+9+3AL/ST58AnAZQVZ9K8oNZtv8l4M1JVgEfr6q/6Rt1366qTQPbX71Htd+Nqro/yfOBFwO/AHw4u94f+NDA7//aT6/qyxxO9wT69sDmrqmupf/jJN8DnkR3B++UDcDlSZYAnxw4rp104QndMZ44ruOb5lHnMF0f+neqG8wNukB/GfDVfv5QutE7/5Yh56FvuR1RVZ8AqKod/XYBvlxVk/38JrrzdsM8HNMhA9u/BfjMLOWHPc++BrwzyYV0F/b1Q/7uV/uW7kF0IXIMcFu/btjzf76cCfxJP31lP38N3eM9+Fx8Y5LT+ukj6c7jdrp/4B/ul/85u+o+5Wjg3qraAFBVfzfuA6iq2/rn3pl0988MWgZ8MN0r9AKWDKz7bFXdB5Dk63TjqawAPl9V2/rlHwb+cV9+pmt1fXVdPsPMdh3PaL/ucun9uP/9MLveE5jTS+yq+gvgnwM/Aq5L8ovTtj19+2NTVQ9X1eer6g+Bc+labPDosXCmpi+iawE8B/hXdK2FKTPWtaq+SNfivRv4sySv71c9WH3zYNjfzbMHBqYDvL2qjut/nl5Vl/Xrhh3bTOd43s9bb+qf1FPpLtrf6Zc/xK5r65FzNOx5VlXfBJ5P113x9iSP6k9OsgY4D3hpVR1LF6DDzvu8nrsky4FfBC7tu5DeBJxBdx4eGCj3ErrW7Yuq6rl0/6CXMtywMaH2xY0x64F3MtDd0nsb8LmqejbwSka7vnZX35mu1QeG/8mM+xlJC4E+zA3ArwIkeRndy5/dSvKPgLuq6l10J/vYea9ht9+js6u/HuA4updz0F0sU7+/1E8vY9ebOL8+x309FfheVX0AuIzu5fJich3wm30/LUmOSPIPdle4b71NJjm1L//4JE/YJzV9bF3uA94InNe/AtpCF9Iw8Mb7sOdZkicDP6yqP6cLmenn5Yl0AXBf3w978nweywxOpxsi+6lVtbqqjqRrdZ4wrdwy4AdV9cO+T/+FA+sex67H49d47KumbwBPTvICgCSHJZmPf1KXA2+tqq9NWz54fZ01wnZuBl6SZHl/3v/FbrY1p2t1b+wvXS5TL22nfKqqZvro4luAD/VvMH2Bbhjf/zdD+TOA1yZ5ENgKvJXuQppvhwIXpfu420N0QyecTffJl8cnuZnuIjizL78O+EiSu4GbgDVz2NdLgDf1x3g/Xd/8olFVn07yTOBLfbfJ/XR9tjN9auJ1wPuTvBV4kEdfUPtUVX01ya10bxa+E7gqyeuA/zVQbNjz7AXAf07yE7pj+O1p2701yVeBO+j6cf963g9muDOBC6Yt+xhdfb81sOxTwDlJbgPupHueTnkAeFaSW+je5zhjYB1VtbO/Zi9K98b3j+ha+2P9aG3fHfffhqx6B12Xy+/x6PO2u+3cm2QdXYPrXro39yf61evY82t1jzV563+SxwMPVzcOzYuA983QB7/o9C9p11bVnMdYlnTg2l9a6HP1FLoW0uPo3vT7lwtcH0mad0220CXpQNTqm6KSdMAx0CWpEQa6JDXCQNd+KftolEhpf2KgS1IjDHQ1Y5YR7nY3GuV/Sjcq42eSfCjJef3y3Y2YuDrJ9elGDPxKkp/tlz8uyXvTjTB4dZJrs2sEwecn+UK6EQiv68f3kMbOQFdLbqAbvfFn6AaP+vcD654B/DPgeOAPkyzpA/tVwM/QDWy1doR9fA84saqeR3en49SXE/wK3SBdz6EbXe9FAP0t4RcBp1fV8+luOx/LmOnSdK3eWKQD01xHozwB+B9TI98l+csR9rEEeHeS4+iGJZgaXe8E4CPVfUnH1iSf65cfDTwb+Ew/pMEE3W3i0tgZ6GrJRcB/qar1/ah/6wbWzXXExqEjJgL/Fvg/wHP79Tv65bvbVoA7qupFs1df2jt2uaglcx3h7gbglUmW9qM8vnxg3RaGjJjY7+PeviX+OnYNxnQD8Kq+L/1JdIOhQTdA1cp+TCH6rp5nzemopBEZ6NpfPSHJ5MDP77FrhLvrgVkHNuu/SGE9cCvdly1spBsFELoRE3873ZdsD3594XuBX09yE113y9TY1h+j+yKC24H30w2tel9V7aT7h3BhPxrjJmCxfM+rGuNYLjqgJTm0/+aoJwBfBM6uqq/s5baWA18Gfq6qto6zvtJM7EPXge6SJMfQ9ZN/cE/DvHd1P7b9wcDbDHPta7bQJakR9qFLUiMMdElqhIEuSY0w0CWpEQa6JDXi/wNMGFVf1al+9gAAAABJRU5ErkJggg==\n",
      "text/plain": [
       "<Figure size 432x288 with 1 Axes>"
      ]
     },
     "metadata": {
      "needs_background": "light"
     },
     "output_type": "display_data"
    }
   ],
   "source": [
    "sixway_df.plot.bar(x='Language', y='TTR', rot=0)\n",
    "plt.savefig(\"TTR_Comparison.png\")"
   ]
  },
  {
   "cell_type": "markdown",
   "id": "51f0d6e3",
   "metadata": {},
   "source": [
    "## Sixway Analysis \n",
    " - **Mandarin** has the least amount of Word Tokens at **375,501** where as **English, Spanish, and French** have over **3 million**. Not sure what the reason for this is at the moment. Same phenmomena occurs for Sentence Tokens Length. \n",
    " - **Mandarin** Has the lowest counts for Word Tokens Length, Sentence Tokens Length, and Average Sentence Length."
   ]
  }
 ],
 "metadata": {
  "kernelspec": {
   "display_name": "Python 3 (ipykernel)",
   "language": "python",
   "name": "python3"
  },
  "language_info": {
   "codemirror_mode": {
    "name": "ipython",
    "version": 3
   },
   "file_extension": ".py",
   "mimetype": "text/x-python",
   "name": "python",
   "nbconvert_exporter": "python",
   "pygments_lexer": "ipython3",
   "version": "3.9.7"
  }
 },
 "nbformat": 4,
 "nbformat_minor": 5
}