Language Detection in 7 Minutes.ipynb

{
 "cells": [
  {
   "cell_type": "code",
   "execution_count": null,
   "metadata": {
    "slideshow": {
     "slide_type": "skip"
    }
   },
   "outputs": [],
   "source": [
    "!pip install requests\n",
    "!pip install matplotlib\n",
    "from typing import List"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {
    "slideshow": {
     "slide_type": "slide"
    }
   },
   "source": [
    "# Language detection in 7 minutes\n",
    "## Paweł Kubiak\n"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {
    "slideshow": {
     "slide_type": "subslide"
    }
   },
   "source": [
    "## Language Detection. Why?\n",
    "Before we start any analysis of the text we need to know its language. We need to know it to decide which dictionary to download, which language model to load or which algorithm to use. Language is **must-have**."
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {
    "slideshow": {
     "slide_type": "slide"
    }
   },
   "source": [
    "## Corpus collecting\n",
    "In order to train our model, we will need a lot of text in different languages. For example, we can use:\n",
    "\n",
    "- [European Parliament Proceedings Parallel Corpus](https://www.statmt.org/europarl/) (21 langs)\n",
    "- [Universal Declaration of Human Rights]() (524 langs)\n",
    "- [Books / documents from WikiBooks project](https://en.wikibooks.org/wiki/Main_Page) (76 langs)\n",
    "- [Wikipedia articles](https://en.wikipedia.org/wiki/Special:Statistics) (314 langs)"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {
    "slideshow": {
     "slide_type": "subslide"
    }
   },
   "source": [
    "## Collect Wikipedia articles\n",
    "We use MediaWiki API <sup>[1]</sup> to extract article plaintext:"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 2,
   "metadata": {
    "slideshow": {
     "slide_type": "-"
    }
   },
   "outputs": [],
   "source": [
    "import requests\n",
    "\n",
    "def get_wikipedia(lang: str, title: str) -> str:\n",
    "    \"\"\"Extract wikipedia article as plaintext using API.\"\"\"\n",
    "    req = requests.get(\n",
    "        f\"https://{lang}.wikipedia.org/w/api.php\",\n",
    "        params=dict(\n",
    "            action=\"query\", format=\"json\", prop=\"extracts\",\n",
    "            titles=title, explaintext=True, redirects=True\n",
    "    )).json()\n",
    "    \n",
    "    return list(req['query']['pages'].values())[0]['extract']"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {
    "slideshow": {
     "slide_type": "subslide"
    }
   },
   "source": [
    "## Sample Wikipedia article"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 3,
   "metadata": {
    "scrolled": false
   },
   "outputs": [
    {
     "name": "stdout",
     "output_type": "stream",
     "text": [
      "Kapibara (Hydrochoerus) – rodzaj ssaka z podrodziny kapibar (Hydrochoerinae) w rodzinie kawiowatych (Caviidae).\n",
      "\n",
      "\n",
      "== Zasięg występowania ==\n",
      "Rodzaj obejmuje gatunki występujące w Ameryce Środkowej (Panama) i Południowej (Kolumbia, Wenezuela, Gujana, Brazylia, Ekwador, Peru, Boliwia, Paragwaj, Argentyna i Urugwaj).\n",
      "\n",
      "\n",
      "== Morfologia ==\n",
      "Długość ciała (bez ogona) 1025–1340 mm, długość ogona 10–20 mm, dł\n"
     ]
    }
   ],
   "source": [
    "kapibara = get_wikipedia('pl', 'kapibara')\n",
    "print(kapibara[:400])"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {
    "slideshow": {
     "slide_type": "subslide"
    }
   },
   "source": [
    "## Text cleaning"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 4,
   "metadata": {},
   "outputs": [
    {
     "name": "stdout",
     "output_type": "stream",
     "text": [
      "kapibara hydrochoerus rodzaj ssaka z podrodziny kapibar hydrochoerinae w rodzinie kawiowatych caviidae zasięg występowania rodzaj obejmuje gatunki występujące w ameryce środkowej panama i południowej kolumbia wenezuela gujana brazylia ekwador peru boliwia paragwaj argenty\n"
     ]
    }
   ],
   "source": [
    "import re\n",
    "\n",
    "def clean(text):\n",
    "    text = re.sub('[_\\W\\d]', ' ', text)  # non-alphanumeric and digits\n",
    "    text = re.sub('\\s+', ' ', text.strip())  # normalize whitespaces\n",
    "    text = text.lower()  # convert to lowercase\n",
    "\n",
    "    return text\n",
    "\n",
    "czysta_kapibara = clean(kapibara[:300])\n",
    "print(czysta_kapibara)"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {
    "slideshow": {
     "slide_type": "slide"
    }
   },
   "source": [
    "## What are N-grams? ##\n",
    "\n",
    "> An N-gram is a contiguous sequence of n items from a given sample of the text. The items can be phonemes, syllables, letters or words according to the application.<br/><small>Wikipedia, The Free Encyclopedia<sup>[2]</sup></small>"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {
    "slideshow": {
     "slide_type": "subslide"
    }
   },
   "source": [
    "## What are N-grams? ##\n",
    "\n",
    "- (*unigram*) &LongRightArrow; p, y, c, o, n \n",
    "- (*bigram*) &LongRightArrow; py, yc, co, on \n",
    "- (*trigram*) &LongRightArrow; pyc, yco, con "
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {
    "slideshow": {
     "slide_type": "subslide"
    }
   },
   "source": [
    "### N-grams in Python"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 5,
   "metadata": {},
   "outputs": [
    {
     "data": {
      "text/plain": [
       "['pyc', 'yco', 'con', 'on2', 'n20', '202', '020']"
      ]
     },
     "execution_count": 5,
     "metadata": {},
     "output_type": "execute_result"
    }
   ],
   "source": [
    "def make_ngrams(text: str, n: int) -> List[str]:\n",
    "    \"\"\"Generate all n-grams of length `n` from `text`.\"\"\"\n",
    "    for i in range(n, len(text)+1):\n",
    "        yield text[i-n:i]\n",
    "\n",
    "list(make_ngrams(\"pycon2020\", 3))"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {
    "slideshow": {
     "slide_type": "subslide"
    }
   },
   "source": [
    "## Top N-grams\n",
    "\n",
    "> The top 300 or so N-grams are almost\n",
    "always highly correlated to the language. ... Starting around rank 300 or so, an N-gram frequency profile begins to show N-grams that are more specific to the subject of the document.<br/>\n",
    "<small>N-Gram-Based Text Categorization<sup>[3]</sup></small>\n"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {
    "slideshow": {
     "slide_type": "subslide"
    }
   },
   "source": [
    "## N-grams ranking in Python"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 6,
   "metadata": {},
   "outputs": [],
   "source": [
    "from collections import Counter\n",
    "\n",
    "def ranking(text: str, topn: int) -> List[str]:\n",
    "    \"\"\"\n",
    "    Collect N-grams of given lengths from `text`.\n",
    "    Return `topn` most common N-grams.\n",
    "    \"\"\"\n",
    "    counts = Counter()\n",
    "    words = text.split()\n",
    "    for length in [1, 2, 3]:\n",
    "        for word in words:\n",
    "            counts.update(make_ngrams(f\"_{word}_\", length))\n",
    "        \n",
    "    return [key for key, _ in counts.most_common(topn)]"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {
    "slideshow": {
     "slide_type": "subslide"
    }
   },
   "source": [
    "## Sample N-grams rankings for PL"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 7,
   "metadata": {},
   "outputs": [
    {
     "name": "stdout",
     "output_type": "stream",
     "text": [
      "_, a, e, i, o, n, r, s, t, w, z, y, c, u, m, k, p, d, l, j, e_, a_, ie, ni, st, g, _p, in, _s, na, i_, ow, _w, li, er, _n, b, h, y_, te, ch, _z, o_, ż, ro, ra, _l, ó, ą, po, an, h_, _o, _i, _d, wa, ch_, ko, _t, nie, w_, _na, m_, nu, _li, ie_, ł, f, ny, ac, _m, pr, _k, lin, inu, ę, wi, za, cz, je, ne, yc, rz, ze, _j, _po, ys, em, x, z_, do, _g, u_, yst, ia, ych, mi, na_, _pr, ów, al, ej, t_, zy, or, od, ą_, ś, _u, _a, on, owa, op, to, ta, ic, en, x_, ar, os, cj, _i_, ux, _r, sz, ki, wan, _je, nux, wy, le, ty, om, _ko, _w_, ym, es, gr, aj, mo, _za, sy, ją, ani, go, ste, eg, j_, tem, wn, ak, si, ów_, ter, ć, ks, dz, ć_, _z_, la, ux_, _sy, sys, rze, pro, ne_, tr, so, we, ię, ka, rac, _do, em_, ob, ed, wo, ró, ę_, uż, est, go_, ej_, dy, sa, ma, ego, sta, fi, me, st_, am, ec, ry, is, nyc, ym_, sk, _op, era\n"
     ]
    }
   ],
   "source": [
    "text = clean(get_wikipedia('pl', 'Linux'))\n",
    "\n",
    "rank = ranking(text, 200)\n",
    "print(', '.join(rank))"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {
    "slideshow": {
     "slide_type": "subslide"
    }
   },
   "source": [
    "## Hypothesis\n",
    "\n",
    "> Texts in the same language generate similar N-gram rankings, while for different languages the rankings will differ strongly."
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 8,
   "metadata": {
    "slideshow": {
     "slide_type": "skip"
    }
   },
   "outputs": [],
   "source": [
    "import matplotlib.pyplot as plt\n",
    "\n",
    "def rank_wiki(name, n=200):\n",
    "    t = clean(get_wikipedia(*name.split(':', 1)))\n",
    "    return ranking(t,n)\n",
    "    \n",
    "def vizualize_rankings(ax, n1, n2, n=100):\n",
    "    r1 = rank_wiki(n1, n)\n",
    "    r2 = rank_wiki(n2, n)\n",
    "    m = [\n",
    "        [i == j if (j in r1 and i in r2) else 0.1 for j in r2]\n",
    "        for i in r1\n",
    "    ]\n",
    "    \n",
    "    ax.imshow(m)\n",
    "    ax.set_xticks([])\n",
    "    ax.set_xlabel(n1, fontsize=18)\n",
    "    ax.set_yticks([])\n",
    "    ax.set_ylabel(n2, fontsize=18)"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {
    "slideshow": {
     "slide_type": "subslide"
    }
   },
   "source": [
    "## Rankings correlation"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 9,
   "metadata": {},
   "outputs": [
    {
     "data": {
      "image/png": "iVBORw0KGgoAAAANSUhEUgAAA6EAAAG6CAYAAAD51UBbAAAAOXRFWHRTb2Z0d2FyZQBNYXRwbG90bGliIHZlcnNpb24zLjMuMywgaHR0cHM6Ly9tYXRwbG90bGliLm9yZy/Il7ecAAAACXBIWXMAAAsTAAALEwEAmpwYAAAp/klEQVR4nO3dfbhlWV0f+O+ir1XdxOGlm5dOp9BrMaiMM2MLBJSnIJjGaWOhtJoHHBXsFqOicYxRQaJEgw6IdtTo0NH4wosJJHkS03HSSTct722eoDa2mAHsSFlApkFtGESkqaJgzR/3ltx76tQ9Z5+9z9rnnvP5PE89dffZZ6/12/ucc3f9ap3fWqXWGgAAAGjhfmMHAAAAwOaQhAIAANCMJBQAAIBmJKEAAAA0IwkFAACgGUkoAAAAzWyN0elDLr+kbj/iM/5y++6333+MMFgz5eiRfdv1zNnmfU5qEUNrXa9z39dl1jWeZRVeg77XbFLfc5oVT9f90yxyTJ/jP3Lmj++ttT60Uyfss4n35qF/Pw19/LT3fd/fH33b6/rZHPr31TyGvkZd+xv6vjXG79xJQ9wXhrSM/jftGk4z9GfnoHvzKEno9iM+I7912yP+cvvaq64eIwzWzNax7X3b506dbt7npBYxtNb1Ovd9XWZd41lW4TXoe80m9T2nWfF03T/NIsf0Of7Wd9/4nk4dcIFNvDcP/ftp6OOnve/7/v7o217Xz+bQv6/mMfQ16trf0PetMX7nThrivjCkZfS/addwmqE/Owfdm30dFwAAgGYkoQAAADQzytdx7377/fd9zee2e+7at38TvgIEAKtk8t48NPd6AM4zEgoAAEAzklAAAACakYQCAADQzCg1oZNm1YWoIwGAtoa+97p3A3CekVAAAACakYQCAADQjCQUAACAZlaiJnSSGlAAGJf5GgBYFiOhAAAANCMJBQAAoBlJKAAAAM2sZE2oOhQAWG3u1QAsykgoAAAAzUhCAQAAaEYSCgAAQDMrWRM6S9+6ksk6lSHaBIBNpgYUgHkZCQUAAKAZSSgAAADNSEIBAABo5lDWhPY1rU5FLQsALM58DQDMy0goAAAAzUhCAQAAaEYSCgAAQDNrWRO6SH2nuhMAaMdcDACby0goAAAAzUhCAQAAaEYSCgAAQDNrWRM6RF2JWhUA+LSh74vuqwCby0goAAAAzUhCAQAAaEYSCgAAQDOSUAAAAJpZy4mJJi0ymYIJEwDg09wXWYSJHoFpjIQCAADQjCQUAACAZiShAAAANLMWNaGz6g3UHwDAwcrRI9k6tj12GKPaOr698sf37aNve12ff/LEdRPHdzp8IUNfo9b9LdJe6/du62vcov9Nu4bTtIzJSCgAAADNSEIBAABoRhIKAABAM2tRE7qKNZ/WxQLgMKlnzubcqdNjh9HUZP1T1/Nf9vHT6rNm9dG1puuWO27etz3575VFYjzo+V0tUqM29DXq2l/f9hdpr28MXa/Z0Ofc1TL637RrOE3Lz46RUAAAAJqRhAIAANCMJBQAAIBm1qImdAzWJgWA1WZ+htlcE2AMRkIBAABoRhIKAABAM5JQAAAAmlETuqChayjUrQBAN+ZnADicjIQCAADQjCQUAACAZiShAAAANKMmdEXMqltRMwoA+5mfAeBwMhIKAABAM5JQAAAAmpGEAgAA0Iya0DmNXSeiLgUAlsu9FhjKZO5w8sT2KHGsKiOhAAAANCMJBQAAoBlJKAAAAM2oCZ2TdTwBYLO4twOLmvx9sXV8nDhWlZFQAAAAmpGEAgAA0IwkFAAAgGbUhA5EzSgALFfre6l7NcByGAkFAACgGUkoAAAAzUhCAQAAaEZNaCND15WoMQVg3bnXAawnI6EAAAA0IwkFAACgGUkoAAAAzagJPaQm62LUzQDQRzl6JFvHtscOY5+TJ7b3bW8dX25/W8e3Zz5n7OP79tG3vWU/fwit+xz7NRkihsPwui67/027htO0jMlIKAAAAM1IQgEAAGhGEgoAAEAzakLXRNcaUDWkAOxVz5zNuVOnxw6jl673tsn6p67nv+zjp9Vnzeqja01X1/bmibFL+7MsUqM29DXq2l/f9hdpr28My34fDG0Z/W/aNZym5WfHSCgAAADNSEIBAABoRhIKAABAM2pCRzJ2TaYaUAD2WsV1Qrvqu66odULHf/4QrBO6/D7Hrl+0TuhyWCcUAACAtSQJBQAAoBlJKAAAAM2oCR3JrJrMsWtGAQAAlsFIKAAAAM1IQgEAAGhGEgoAAEAzakJX1GGoAVW3CgAAdGUkFAAAgGYkoQAAADQjCQUAAKAZNaEs7LDXgKppBfi0euZszp06PXYYTW0d39633fX8l3385P55+ph2zEG6tjdPjF3an6Xr+STJLXfcvG978v6+SJsH6XpNltFe3xiW/T4Y2jL637RrOM3Qv18OYiQUAACAZiShAAAANCMJBQAAoBk1oSS5sD4yWf8ayXU/PwBWy+S99uSJ7VHiWHfu77D6jIQCAADQjCQUAACAZiShAAAANCMJBQAAoBkTE62pyckPZhXpK+IHgOWavNduHR8nDpjU9d+N0JeRUAAAAJqRhAIAANCMJBQAAIBmRqkJLUePZOvY9hhdb4zJBbA3se5k6/j22CGsRAzL1vUcW1+TVXwN+sY09DnNam+R/lbtHGEIk3Vzk/daOKzUgNKakVAAAACakYQCAADQjCQUAACAZkapCa1nzubcqdNjdE1Drdecmqwha/Eem1W3to7v867Xue/r0rc2cBVeg77XbFLfc5oVT9f90yxyzJDHwzKswjqf1nME1oGRUAAAAJqRhAIAANCMJBQAAIBmRqkJZTNM1qmoYwFYXdbwHn9923mOP3niuoljenW59LWex6jnPuzrUR+GtZnHrtNfRv+bdg2naRmTkVAAAACakYQCAADQjCQUAACAZtSEclFD13CqAQUAAIyEAgAA0IwkFAAAgGYkoQAAADSjJpSLUsM5PGulAgCw6YyEAgAA0IwkFAAAgGYkoQAAADSjJnRDqEVcDa47sKrqmbM5d+r02GE0tXV8e9921/Nf9vGT++fpY9oxB+na3jwxdml/lq7nM0+fi7TZpb++7S/SXt8Ylv0+GNotd9y8b/vkiet6t7lp13Calp8dI6EAAAA0IwkFAACgGUkoAAAAzagJ3RBqEWF51FwDDGPy9+nJE9ujxMFqm7zPbh0fJw4WZyQUAACAZiShAAAANCMJBQAAoJm5a0JLKU9Ocm+t9R0X2f/QJI+utb55qOCA9bOO9ZPrcA7A+NRDqvWDTdFlJPSNSX6vlPJdF9n/vyV5Q++IAAAAWFtdv477/iQ/U0r5uVKKr/ICAADQSdclWl6Q5AuTfF+SR5ZSnlFr/ejwYQEAALCOuiahtdb6vFLK3UluSvKbpZSn1Vrft4TY4NBbx/rHvlwDWE3l6JFsHdseO4xRbR3fHvX4kyeuW3offdtb9vOH0LrPsV+TIWI4DK/rsvvftGs4TcuYFvpKba31l5J8RZLPSvLWUspfHzQqAAAA1tLCdZ211t9I8sQkH8/OpEVPHygmAAAA1lSvyYVqre9M8oQkb0/ytweJCAAAgLXVpSb0hiT/efLBWuufllKekuRFSR42TFjQ3yrUY6p/BA6LeuZszp06PXYYTU3WP3U9/2UfP60+a/I5F64tel2nGGbFvEiMXdqfZZEata7n1FfXa7KM9vrGsOz3wdCW0f+mXcNpWn525k5Ca62vPGDfmSTPHyQiAAAA1tbcX8ctpVxRSnn0xGOfs7tm6L8opVw7fHgAAACsky5fx/0nST43yeOTpJTymUnekuSq3f3PLKX8zVrrm4cNEQAAgHXRJQn9kiS/umf7mdlJQL8iyV1Jbk/yvCSSUJaia42nekwA9rqwfnJ7lDiGNHmv2zo+ThwAXXSZHffhSd63Z/tvJfmdWuuttdYPJHlFki8aMDYAAADWTJck9BNJLtuz/TeSvGnP9oeTXDFATAAAAKypLkno3Um+tuz4qiSXJ3ndnv2PSPKhIYMDAABgvXSpCX1Zdr5y+/8luX+SU9mfhD4pye8PFhlMmKx7WcfaHgCWR/0kh4F/37AJuqwT+qpSSk1yXZI/S/LiWusnkp3lW5I8KMlNS4gRAACANdFlJDS11l/N/hlyzz/+wSSPHSooAAAA1lOXmlAAAADoZe6R0FLKr8zxtFprfU6PeGBuantooev6tABcnN+ps/n3DZugy9dxr5/jOTWJJBQAAICp5v46bq31fpN/knxGks9L8otJ/kuSBy8pTgAAANZAr5rQWusna63/rdb6bUk+mOSlw4QFAADAOuo0O+4Mtyb54STPHbBNoCG1OhdyDdgU5eiRbB3bHjuMUW0d31754/v20be9vs+fXPNyGfWOQ1+j1v0t0l7r927ra9yi/027htO0jGnI2XEvT/KZA7YHAADAmuk9ElpKeVCSpyb5niR39m0PAACA9dVliZZPZWf226m7k3woyd8fIigAAADWU5eR0FflwiS0Zif5vDvJa2qtfz5UYKA+sT3XGACAZZs7Ca21Xr/EOACABZRSHlZr/ZMZz/nrtdbfbhUTABxkyImJAID2fq+U8mUX21lK+YEkdzSMBwAO1CkJLaX8D6WUf1hKuaOU8t9KKV+y+/hDdh///OWECQBcxEeS/KdSyktLKZecf7CU8vBSymuTvDjJa0eLDgAmzJ2EllIemuR3krwwyRVJjie5LElqrfcm+aYk37qEGAGAi3tMklcn+f4kv1lKOV5K+VtJ3p7kyUm+p9b6lWMGCAB7dZmY6MeSXJnkCUnem2Sy/uTfJ7lmoLg4hIaeSMgkOQCz1Vr/IsmzSym3J3lZdpLPy7IzaeC1tda7RgwPAC7Q5eu4T0tyU631bZm+VMupJI8YJCoAoKs7snMvvv/u9q9JQAFYRV2S0Ick+cMD9n8qyaX9wgEAuiqlPCPJ25JsJ/k7Sf5DkheUUm4vpVw5ZmwAMKlLEvqBJI88YP8XZedrugBAI6WUX0zymuz8R/Fjaq2/XGt9epLvTnIiye+WUq4dM0YA2KtLTeh/TPKcUsrPJTm7d0cp5QlJnp3kZ4YLjcNGDSeshqHrs1l535zkp5O8oNb6ifMP1lp/rpTy5iT/Mskt6XbPB4Cl6TIS+o+SnEvyu0lekp260G8qpbwmyZuT3JPkpYNHCAAc5Ctrrd+3NwE9r9b6e0kem+SV7cMCgOnmTkJrrR9I8sVJ3pqd/3UtSZ6V5BnZWX/sSbXWDy0jSABgulrrf5yx/2O11ue0igcAZun01Zxa6/uSPL2U8oAkn5edRPQPJZ8AAADMY6H6kFrrR5L89sCxABtI/eLwXMPNUko5NcfTaq31oMkFU8+czblTp4cJ6pDYOr69b7vr+S/7+Mn98/Qx7ZiDdG1vnhi7tD9L1/OZp89F2uzSX9/2F2mvbwzLfh8MbRn933LHzfu2T564rtPxh+0aTtPys3PRJLSU8lmLNFhrNUMuALTz3ly4fvdWks9JclV2Zs39f1sHBQAXc9BI6OlceFObxyWLhQIAdFVrfcrF9pVS/vck/zjJtzcLCABmOCgJfVEuTEK/KsnVSW5P8o7dx74gyTVJ7kryfw8bHgCwqFrra0opT8pOIvq0seMBgOSAJLTW+iN7t0spX5+dr/Y8ttZ618S+xyR5XZK7hw8RWGeHrX5xsoY1OXznwMa5K8k3jh0EwCqbvJdvHR8njlUy+W+ekye2B2u7yzqhP5Dk/5pMQJOk1vq2JC9L8oKB4gIAhnF1kk+NHQQAnNdldtxHJfmTA/b/8e5zAIBGSilPvsiuy5M8NcnfSfJr7SICgIN1SULfn+RrSikvq7XuqxUtpdwvydcm+cCQwQEAM70x0ycSLLt//0aS72oWDQDM0CUJ/cUk/2eS20opP5XkD3Yf//wkfz/Jk5L80DwNlaNHsnVsu0PX0N0qrLe0CjEsW9dzbH1Nhu5vWj1E17qRvjENfU6z2lukv1U7xzV3w5THapIPJbm71mq+BjbaMuvaYJ0ts062SxL640kenp3/Tb1myv6X1VpfMkhUAMBcaq2vHDsGAOhi7iR09yu4f6+UclOSpyc5nwufSvLrtdY/uOjBAAAAkG4joUmS3a/1/OQSYgEAFlBK+StJvj47EwRekU/Xg55Xa63PaR4YAExRJuYYauIB5fL6hDLtG72wuMkasnOnTjfvc1KLGA4yWQczxHqWXa9z39elb23g2K9B0v+aTep7TrPi6bp/mkWO6XP8re++8c5a6+M6dbImSimPT/IfkjzkgKfVWuslB7XzwEuvrE889qxBYwNgcx10b+40ElpKeXCS5yR5QpIH58J1RmutVXYJAO38VJIjSZ6R5PW11g+NHA8AHGjuJLSU8tlJfjPJVUn+LMkDsjPz3vlk9N4kf7GEGAGAi3tskhfXWv/N2IEAwDwmRzIP8mNJHpSdmXEflZ16k2dmJxl9SZI/z84yLQBAOx9J8sGxgwCAeXX5Ou41SX6x1vqGUsoVu4+VWuvHkvxgKeULkrw0yTcMHSQwn2XUgAIr79eSXJvkprEDAYB5dBkJvSLJf939+RO7f1+2Z//tSb5siKAAgLk9P8nDSik/V0p5ZCllcmZcAFgpXUZC/zTJ5bs//3mSjyfZ3rP/SPYnpQDA8n04SU3y+CTfkSRT8tBaa+28LBsALMOBN6RSypOTvLPW+qdJ/p8kX5js3MlKKb+V5DtKKb+enRHVb03yriXHCwDs96rsJKEAcCjM+l/RNyR5VpJXJ/n3Sb63lHJZrfW+JC9KcluSP9p9bk3yNcsKlPWnnrE/1ww2T631+rFjAIAuZiWhf/l9nlrrTdkz6UGt9fWllC9J8vVJPpnk39Va//NSogQAAGAt9KoPqbX+TpLfGSgWAAAA1pxJCgDgECmlfCrJp5Lcv9Z6dnd7Vk2oiYkAWBnz3JC+ppTyP87ZXq21/mifgNhcy65nVHMKrInzExF9cmK7l3rmbM6dOt23mUNl6/j2vu2u57/s4yf3z9PHtGMO0rW9W+64ed/2yRPX9Wp/lq7nM0+fi7TZpb++7S/SXt8Yul6zoc+5q2X0v2nXcJqWn515ktCvzvwTDtUkklAAWJLJiYhMTATAYTNPEvriJL+x7EAAgOGVUr44yZNrrT8xdiwAkMyXhL6z1vqmpUcCACzDNdlZVk0SCsBKKLVevIxkd7KDb6y1vnrITh946ZX1iceeNWSTAGywW99945211seNHccqKqX8YJIX1VovOeh57s0ADOmge/P9WgcDAADA5pqVhL4nyUdbBAIAAMD6O7AmtNb6Oa0CAQAAYP2NsnD1Jq5FxvL1Xa9tiD4nreP7vOt1Hnodva4m17QbY33Yvtds0tDr7s1aq2wV1qxbhfXTVkkp5fIOT7//PE/axHuzdUIvbG9yTe3JdUC7fjatE9qmvU1b49I6ocuxauuEAgCr5d7srM09j9LhuQCwdJJQADh8XhWJJQCHlCQUAA6ZWuv1Y8cAAIuShAJrrWsN6GQ91Bg1pABjmfydt3V8uf35nQubyTqhAAAANNNpJLSU8iVJ/m6SRyW5IjuTHexVa62PHCg2AGCGUsoVSb4zO/fgH513HwCMZe4ktJTy7CQvT/KJJHcnee+yggIA5vaQJD+SnYmKJhPNg/YBwCi6jIT+YJI/SPLUWus9S4oH2HAXrlG33bR/9UgcQu9PcsMC+2B0fucOY+x7J3TVJQn97CTfLwEFgNVRa/1Ikld23QcAY+kyMdF/T3J0WYEAAACw/rokoT+f5BtKKZcsKxgAAADWW5ev496Z5GuT/FYp5WVJ/ijJJyefVGt980CxARvIGnXQXSnl65J8Vz49e/2kWms98J5fjh7J1rHtJUR3eGwd31754/v20be9ZT9/CK37HPs1SZKTJ65r2ucYr+uy+2/9+R/7Gk7TMqYuSejr9vz8S9mZaW+vsvuYkVIAaKSU8v1JfjzJB5P8l92/AWBldUlCza4HAKvnO5O8Nck1tdb7xg4GAGaZOwmttZpdDwBWz5VJfkICCsBh0WUkdDDqTmhhFb5rvwoxLNuq10DM6m9yLbVl16Du9LE98znLPL5re4v0t2rnuOb+MMmDxg4CAObVOQktpTw8yeOSPDhTZtettb5qgLgAgPn84yQ/VEr52VrrR8cOBgBmmTsJLaXcL8nLknxLDl7aRRIKAO18MsmfJHlXKeVXcvHZ692fAVgJXUZCvy/JtyX550lem51k8/lJ/jzJ30vyZ0leMHB8AMDBXrHn5x+6yHNq/CcxACuiSxL6TUlurbU+u5Ryfg2yO2utry+l/GqStyd5bJLXz2qonjmbc6dOdw4WDjJZQ9biPTarbm0d3+ddr3Pf16VvbeAqvAZ9r9mkvuc02f4td9y8b3tyvbnJ/uZ5TRY5ZsjjN8yXjh0AAHTRJQk9nuQXdn/+1O7fn5Ektda/KKW8PDtf1f3J4cIDAA5Sa33T2DEAQBcH1XZOui/JJ3Z//mh2vtrzsD37P5DkEQPFBQB0VEo5Wkr5a6WUI2PHAgAX0yUJfU+SRyZJrfUT2ZkS/sv37H9qkj8eLjQAYB6llMeUUl6fnXka3pvkxO7jDyulvK6U8tRRAwSAPbp8Hff1Sb46OxMUJcmvJnlRKeWqJCXJk5LcOGx4rLPb7rlr3/a1V109Shyw6SY/ey3WSmU4pZSrk7wlyb3ZmXzohvP7aq1/Ukq5LDvzOvzGQe1s4nwNQ9esT9ZXz7qvzep/Wi1035ryvu11rdceuoZ9HkNfo679DT2XwRh1+JP6vg+Gtoz+N+0aTtPys9MlCb0xyWtLKUdrrWeSvCQ7X8f9xuxMBf/PkvzwYJEBAPN4UZJ7knxRkkuTfPPE/tcleUbroADgYuZOQmut70/y/j3bn0zyf+z+AQDG8aQkL6m1frSUcnTK/vcmuapxTABwUTNrQkspX1dK+f1Syn2llPeVUn6slNKllhQAWJ5Ls7NW98U8oFUgADCPA0dCSyknk7x6d/ODSf5qkhckOZLkecsNDQCYw7uzs073xfzNJO9oFAsAzDTr67jfnZ3k89pa69tKKQ9JcnOS55ZSXrhbGwoLMRHR6pucPCpJTp7Ybh4HcKBXJ3lhKeVfJ/nd3cdqkpRSvjc7M9l/96xGytEj2Tq2vawYD4W+k26cPHHdRHvD9z/05CVd21v284fQus+xX5MhYjgMr+uy+9+0azhNy5hmJaGPTfJPa61vS5Ja672llH+Q5A1JHp3kruWGBwDMcGOSL0tyW5J3ZScB/elSykOTXJnk9iQ3jRceAOw3q7bzQUnunnjsD7KzJMuDlhAPANBBrfVsdpLQ70tyX5KPJ/nc7CzZ8rwkT6u1fmq8CAFgv1kjoSXJuYnHzm+bnAgAVkCt9VySn979AwArbZ4lWrZLKY/Zs/3A3b8fVUr58OSTz391Fw6bafWPm163Ou38u9Y4AQDAXvMkoT+6+2fSxepLLlk8HAAAANbZrCT0HzWJAgAAgI1wYBJaa5WEAgAAMJh5vo4LG2HT6z83xWTtr9cdAKAtM9wCAADQzNxJaCnlylLKr5RSfrnLPgAAADivy0joA5Ncv/unyz4AAABI0q0m9L1JvnSBfQArYx1qQNW1AgCH2dxJaK31viRv6roPAAAAzjMxEQAAAM1IQgEAAGim1Fqn7yjl1ALt1VrrI2c96YGXXlmfeOxZCzQPABe69d033llrfdzYcRxmDyiX1yeUa8YOo6mt49v7ts+dOr1Sx0/un6ePacccpGt788TYpf1Zup7PPH0u0maX/vq2v0h7fWPoes1uuePmfdsnT1zXqb++hr7mQ7S57M9SC0N/dg66Nx9UE/reJNMzVAAAAFjARZPQWutTGsYBAADABlATCgAAQDNd1glNkpRSjiZ5SpLjuw+dSvKmWuvHB4wLAAAYweR61CdPbO/bnlyfeut4oJNOSWgp5dlJfirJg5OU3Ydrkg+XUr631vqKYcMDAABgncydhJZSnpnkFdmZsOjGJO/Y3fUFSb49yS+XUu6rtf6roYMEAABgPXQZCf0HSd6V5ItrrR/Z8/ivl1JuSvLW3edIQgEAAJjqouuEXvDEUj6e5IW11p+8yP7nJ/mRWutls9raxLXIWL6+67UN0eekFjF0MVnjMVnTMY+u13nodfS6WoXXoO81mzT0unuz1ipbhTXrZh1vndD+rOENwJAOujd3mR33AzP21yR/3KE9AAAANkyXJPQVSW4opXzm5I5SygOS3JDk5QPFBQAAwBrqUhP6liRPS/L7uzWg79p9/NFJnpvk3iRvKaU8ee9BtdY3DxEoAAAAh1+XJPT2PT+/NDtfv00+vVTLZ088p+w+55KFo2OlDFFPyLi8ZgAAjK1LEnrD0qIAAABgI8ydhNZaX7nMQAAAAFh/XSYmAgAAgF7mXid0SNYiA2BI1gntbxPX8B56HePJ42fNpbDImrx91xnu217XNXyHXtd4HkNfo679Db2+9RhrM08aYv3oIS2j/027htMM/dkZap1QAAAA6EUSCgAAQDOSUAAAAJpREwrAoacmtD/3ZgCGpCYUAACAlSAJBQAAoBlJKAAAAM1sjdFpPXO297pRMKnvem9D9DlpHd/nXa/z0OvwdbUKr0Hfazap61qEXeMZYi2zoddbW4X10wCAYRgJBQAAoBlJKAAAAM1IQgEAAGhmlJpQAIYzqwZ0smb05IntpcXC4bWJ8zUMXbM+9PHTaqH71pT3ba9rvXbf99Qi9eBDX6Ou/Q09l8EYdfiThpgrYEjL6L9vm7fccfO+7cl786pdw2lafnaMhAIAANCMJBQAAIBmJKEAAAA0oyYUYGCrVoN5YV3KOHGw2srRI9k6tj12GKPqW+/U4vih68a6trfs5w+hdZ9jvyZDxHAYXtdl99+3zZMnrptob7n9LUPLmIyEAgAA0IwkFAAAgGYkoQAAADSjJhQOscnaw1nrRdKGGkwAgIszEgoAAEAzklAAAACakYQCAADQTKm1Nu/0gZdeWZ947FnN+wVgPd367hvvrLU+buw4DrMHlMvrE8o1Y4fR1OSaeOdOnV6p46et2Terj67r/HVtb54Yu7Q/yyLrFg59jbr217f9RdrrG8Oy3wdDW0b/ra/hLXfcvG97cp3RMQz92Tno3mwkFAAAgGYkoQAAADQjCQUAAKAZ64QCAAA0tIprirdcf95IKAAAAM1IQgEAAGhGEgoAAEAzo6wTuolrkbF8fddrG6LPSbNiaPnd+6F0vc5Dr6PXVYv3wSx9r9mkodfdm7U22iqsWTfreOuE9mcNbwCGZJ1QAAAAVoIkFAAAgGYkoQAAADQjCQUAAKCZrbEDgE02ORHRYZyoCAAAujASCgAAQDOSUAAAAJqRhAIAANCMmlA4QOsaTTWgAACsOyOhAAAANCMJBQAAoBlJKAAAAM2oCYUDzFrH8+SJ7WaxAADAOjASCgAAQDOSUAAAAJqRhAIAANDMKDWh5eiRbB3bHqNrNsjW8e3B2+xaA7qMGFZN13NsfU1W8TXoG9PQ5zSrvUX6W7VzBABWh5FQAAAAmpGEAgAA0IwkFAAAgGYkoQAAADQjCQUAAKAZSSgAAADNSEIBAABoRhIKAABAM5JQAAAAmpGEAgAA0IwkFAAAgGa2xui0njmbc6dOj9E1a2zr+Pa+7Rbvsck+J/WN4bZ77tq3fe1VV/dqbwhdr3Pf12XWNZ5lFX7X9L1mk/qe06x4uu6fZpFjhjweAFhdRkIBAABoRhIKAABAM5JQAAAAmhmlJhSYz2QN6CrWiAIAQBdGQgEAAGhGEgoAAEAzklAAAACaGaUmtBw9kq1j22N0zQZZhXUFh47h5In97W0dH7T5hXQ9x9avyyq8Dyb1jWnoc5rV3iL9rdo5AgCrw0goAAAAzUhCAQAAaEYSCgAAQDOj1ITWM2dz7tTpMbpmjU3WkLV4j82qW1v19/ki6452vc59X5e+tYGr8Br0vWaT+p7TrHi67p9mkWOGPB4AWF1GQgEAAGhGEgoAAEAzklAAAACasU4oa2sVashWIYaDDLHuqHVCu1u1NTStEwoAtGQkFAAAgGYkoQAAADQjCQUAAKAZSSgAAADNSEIBAABoRhIKAABAM5JQAAAAmhllnVAAYLXUM2dz7tTpscNoanI92q7nv+zjp62XO6uPrmvsdm1vnhi7tD/LImsGD32NZrnljpv3bZ88cV2v9rpe40WPOej4SX3fB0NbRv+bdg2nafnZMRIKAABAM5JQAAAAmpGEAgAA0EyptTbv9AHl8vqEck3zfllvfWtzhuhz0jrWV02e82QtzLVXXX3g8/vWTHW1Cq9B12vQusZqVl3KKtQnzTr+1nffeGet9XGdOmGfB156ZX3isWeNHQYAa+Kge7ORUAAAAJqRhAIAANCMJBQAAIBmrBMK9DJZAwpJcts9d+3bPnlie5Q4AIDVYyQUAACAZiShAAAANCMJBQAAoJlRakLL0SPZOrY9RtdskL7rS65LDMvW9RxbX5NVfA36xjT0Oc1qb5H+Tp64bqFY+vQJABwORkIBAABoRhIKAABAM5JQAAAAmpGEAgAA0MwoExPVM2dz7tTpMbpmjU1OZNLiPTZr8pR1fJ/Pus633XPXvu3JCWq6XpO+E9SswmvQ9b257PfVrHi67p9mkWOGPB4AWF1GQgEAAGhGEgoAAEAzklAAAACaGaUmtBw9kq1j22N0zQZZhRqyVYhh2SbP8eSJ7anPu9jzl20VX4O+MQ19TrPaW6S/VTtHAGB1GAkFAACgGUkoAAAAzUhCAQAAaMY6oawN64S20XfNS+uEJrfccfO+7WuvuvrA50+yTigAcJgZCQUAAKAZSSgAAADNSEIBAABoxjqhrK1VqCFbhRiWres5Wic0OXniun3bW8e7HW+dUADgMDMSCgAAQDOSUAAAAJqRhAIAANCMdUJZG9YJbcM6od31vWaTNmGd0Mm1VCfraAGAw8tIKAAAAM1IQgEAAGhGEgoAAEAzo9SEAvBpt91z177tkye2R4ljlVx71dX7truupQoArC4joQAAADQjCQUAAKAZSSgAAADNqAkFGNmq1z+qWQUAhmQkFAAAgGYkoQAAADQjCQUAAKCZUWpCy9Ej2Tq2PUbXbJCt49tjh7ASMSxb13NsfU1W8TXoG9PQ5zSrvZMnrhu8zWUfDwCsLiOhAAAANCMJBQAAoBlJKAAAAM2UWmv7Tkv50yTvad4xAOvqs2utDx07iMPMvRmAgV303jxKEgoAAMBm8nVcAAAAmpGEAgAA0IwkFHoqpbyxlHJ64DZ/pJRSSynbQ7YLAJvAvRlWmyQUGtpzA3vc2LEAAO7NMAZJKKymH0tyWcxUCQCrwr0ZBrI1dgDAhWqt55KcGzsOAGCHezMMx0goTFFKuX73qzlP3f2azntKKWdKKW8vpXzdHMffv5Ty+aWUv7pg/xfUnex57PNKKS8upfz33Zh+r5TyFRPHP2X3uddPafsVpZS6Z/uLSikfL6XcXkopex6/pJTy5lLKx0op//Mi5wEAQ3Fvdm9mfUhC4WAvTfJ1SW5K8g+THEnymmk3kAmPT/LOJC9ZQkyvTPKkJDcmeWGShya5edGJEmqtv5vkeUmemuQH9uz64d1+vqfW+l/7BAwAA3Jvdm/mkPN1XDjYQ5L8r7XWP0uSUsrPJ3l7kp8qpfyrWut9I8R0b5KvrLXW3ZjekOS3knxbkhcs0mCt9WdLKdckedFue5cm+cEk/7bW+gvDhA0Ag3Bvdm/mkDMSCgf7p+dvckmy+/PPJ3lwkqdc7KBa6xtrraXWev0SYvon529yu339dpKPJnlUz3ZvSPKBJK9J8s+TvC/Jt/RsEwCG5t4Mh5yRUDjYO6c89o7dv4+3DGSPU1Me+2CSK/o0Wmv9UCnlhiS37z50otb64T5tAsASuDfDIWckFA6fT17k8bLn53qR5yQH/+fTV+35+ep5AwKADefeDB1IQuFgj57y2P+0+/e0//VcFR/a/fvyKfum/i9xKeWrknxXkpcneUuSG0sp/8tywgOAhbk3wyEnCYWDPbeU8sDzG7s/f3uSDyd508UO6jsN/AD+KDtrmT1174OllCcm+eLJJ5dS/lqSX8nOV5z+bpJvSHJfkn9ZSrls6dECwPzcm92bOeTUhMLB7k3y1lLKy3e3b0jyWUm+pdb6sQOOe3ySN2Rnyvbrp+z/5lLKl095/M5a63/qEW+SpNb60VLKK5J8SynlNUnemJ3JEW7IzgyCX3j+uaWU+yX5F0n+SpJn7p7Xx0opz0nya0l+Jjuz+wHAKnBvdm/mkJOEwsGen531uL4zycOT3J3kG2qtr+7Z7nMv8vgvJOl9o9v1PdmpRfnqJE9PcmeSr0zyrdlzo0vyQ0n+RpLvqLX+/vkHa63/rpRyU5LvKKW8ttb6bweKCwD6cG92b+aQK3tmkwZ27S54/fIkX1prfeO40QAA7s2wPtSEAgAA0IwkFAAAgGYkoQAAADSjJhQAAIBmjIQCAADQjCQUAACAZiShAAAANCMJBQAAoBlJKAAAAM1IQgEAAGjm/wd06cXTyzDhxQAAAABJRU5ErkJggg==\n",
      "text/plain": [
       "<Figure size 1152x576 with 2 Axes>"
      ]
     },
     "metadata": {},
     "output_type": "display_data"
    }
   ],
   "source": [
    "_, ax =plt.subplots(1, 2, figsize=(16,8))\n",
    "vizualize_rankings(ax[0], 'pl:Linux', 'pl:Pan Tadeusz')\n",
    "vizualize_rankings(ax[1], 'pl:Linux', 'en:Linux')"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {
    "slideshow": {
     "slide_type": "slide"
    }
   },
   "source": [
    "## Rankings Distance Measure\n",
    "> ![N-gram ranking distance](https://www.researchgate.net/profile/William_Cavnar/publication/2375544/figure/fig2/AS:669559593898000@1536646855685/Calculating-The-Out-Of-Place-Measure-Between-Two-Profiles.png)\n",
    "<small>N-Gram-Based Text Categorization<sup>[3]</sup></small>\n"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {
    "slideshow": {
     "slide_type": "subslide"
    }
   },
   "source": [
    "## Rankings Distance Measure in Python"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 10,
   "metadata": {},
   "outputs": [],
   "source": [
    "def distance(rank1: List[str], rank2: List[str]) -> int:\n",
    "    \"\"\"Compute distane between two N-grams rankings.\"\"\"\n",
    "    score = 0\n",
    "    for i, key in enumerate(rank1):\n",
    "        if key in rank2:\n",
    "            score += abs(rank2.index(key) - i)\n",
    "        else:\n",
    "            score += len(rank2) # MAX penalty\n",
    "    return score"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 11,
   "metadata": {},
   "outputs": [
    {
     "data": {
      "text/plain": [
       "(14956, 23623)"
      ]
     },
     "execution_count": 11,
     "metadata": {},
     "output_type": "execute_result"
    }
   ],
   "source": [
    "(\n",
    "    distance(rank_wiki('pl:Linux'), rank_wiki('pl:Pan Tadeusz')),\n",
    "    distance(rank_wiki('pl:Linux'), rank_wiki('en:Linux'))\n",
    ")"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {
    "slideshow": {
     "slide_type": "slide"
    }
   },
   "source": [
    "## N-grams Language Detector"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 12,
   "metadata": {},
   "outputs": [],
   "source": [
    "class LinuxLanguageDetector:\n",
    "    \"\"\"Detect text language based on N-grams model.\"\"\"\n",
    "    def __init__(self, langs: List[str], n: int = 200):\n",
    "        self.n = n\n",
    "        self.ranks = {\n",
    "            lang: rank_wiki(f\"{lang}:Linux\", n)\n",
    "            for lang in langs\n",
    "        }\n",
    "        \n",
    "    def detect(self, text: str) -> str:\n",
    "        rank = ranking(clean(text), self.n)\n",
    "        return min(\n",
    "            self.ranks,\n",
    "            key=lambda lang: distance(rank, self.ranks[lang])\n",
    "        )"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {
    "slideshow": {
     "slide_type": "subslide"
    }
   },
   "source": [
    "## Testing"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 13,
   "metadata": {},
   "outputs": [],
   "source": [
    "langdet = LinuxLanguageDetector(['pl', 'en', 'de', 'fr', 'it'])"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 14,
   "metadata": {},
   "outputs": [
    {
     "data": {
      "text/plain": [
       "'pl'"
      ]
     },
     "execution_count": 14,
     "metadata": {},
     "output_type": "execute_result"
    }
   ],
   "source": [
    "langdet.detect(\"PyCon PL 2020 to trzynasta edycja \"\n",
    "    \"ogólnopolskiej konferencji z grupy PyCon.\")"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 15,
   "metadata": {},
   "outputs": [
    {
     "data": {
      "text/plain": [
       "'it'"
      ]
     },
     "execution_count": 15,
     "metadata": {},
     "output_type": "execute_result"
    }
   ],
   "source": [
    "langdet.detect(\"Python è un linguaggio di programmazione di più \"\n",
    "    \"alto livello rispetto alla maggior parte degli altri linguaggi.\")"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 16,
   "metadata": {},
   "outputs": [
    {
     "data": {
      "text/plain": [
       "'en'"
      ]
     },
     "execution_count": 16,
     "metadata": {},
     "output_type": "execute_result"
    }
   ],
   "source": [
    "langdet.detect(\"Capybaras are highly social, living in groups of up \"\n",
    "    \"to 100 and communicating through a variety of vocalizations.\")"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {
    "slideshow": {
     "slide_type": "subslide"
    }
   },
   "source": [
    "## Evaluation on Sample Articles"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 17,
   "metadata": {
    "slideshow": {
     "slide_type": "skip"
    }
   },
   "outputs": [
    {
     "data": {
      "text/plain": [
       "['Ołeksij Bahniuk',\n",
       " 'Iisaku',\n",
       " 'Dayot Upamecano',\n",
       " 'Babin Średni',\n",
       " 'Papierówka (jabłoń)',\n",
       " 'Emil Michałowski',\n",
       " 'Eschenburg',\n",
       " 'Ja, Najwyższy',\n",
       " 'Antoni Słociński',\n",
       " 'Douglas Bay']"
      ]
     },
     "execution_count": 17,
     "metadata": {},
     "output_type": "execute_result"
    }
   ],
   "source": [
    "def wiki_randomize(lang: str, limit: int) -> List[str]:\n",
    "    \"\"\"Return random wikipedia articles' titles.\"\"\"\n",
    "    data = requests.get(\n",
    "        f\"https://{lang}.wikipedia.org/w/api.php\",\n",
    "        params=dict(action=\"query\", format=\"json\", generator=\"random\",\n",
    "                    grnnamespace=0, grnlimit=limit, prop=\"info\"\n",
    "    )).json()\n",
    "    return [page['title'] for page in data['query']['pages'].values()]\n",
    "\n",
    "wiki_randomize('pl', 10)"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 18,
   "metadata": {
    "slideshow": {
     "slide_type": "skip"
    }
   },
   "outputs": [],
   "source": [
    "def evaluate(model, n=10, cutoff=5):\n",
    "    \"\"\"Evaluate model on random wikipedia articles.\"\"\"\n",
    "    langs = list(model.ranks.keys())\n",
    "    counts, counts_cut = Counter(), Counter()\n",
    "    for lang in langs:\n",
    "        for title in wiki_randomize(lang, n):\n",
    "            text = get_wikipedia(lang, title)\n",
    "            counts[(lang, model.detect(text))] += 1\n",
    "            \n",
    "            text = ' '.join(clean(text).split()[:cutoff])\n",
    "            counts_cut[(lang, model.detect(text))] += 1\n",
    "\n",
    "    return langs, counts, counts_cut\n",
    "\n",
    "\n",
    "def show_matrix(ax, langs, data):\n",
    "    data = [\n",
    "        [data[(expected, detected)] for detected in langs]\n",
    "        for expected in langs\n",
    "    ]\n",
    "    ax.imshow(data)\n",
    "    ax.set_xticks(range(len(langs)))\n",
    "    ax.set_xticklabels(langs)\n",
    "    ax.set_xlabel(\"detected\", fontsize=18)\n",
    "    ax.set_yticks(range(len(langs)))\n",
    "    ax.set_yticklabels(langs)\n",
    "    ax.set_ylabel(\"expected\", fontsize=18)\n",
    "    for i in range(len(langs)):\n",
    "        for j in range(len(langs)):\n",
    "            val = data[i][j]\n",
    "            text = ax.text(j, i, val, ha=\"center\", va=\"center\", color=\"k\" if val else \"w\", fontsize=24)\n",
    "    \n",
    "\n",
    "def show_evaluation(model, n):\n",
    "    _, ax = plt.subplots(1, 2, figsize=(16, 8))\n",
    "    langs, full_text, with_cutoff = evaluate(model, n)\n",
    "\n",
    "    N = sum(full_text.values())\n",
    "    acc = 100 * sum(full_text[i,i] for i in langs) / N\n",
    "    ax[0].set_title(\"Full Text (acc: %.2f%%, n: %d)\" % (acc,N), fontsize=22)\n",
    "    show_matrix(ax[0], langs, full_text)\n",
    "\n",
    "    acc = 100 * sum(with_cutoff[i,i] for i in langs) / N\n",
    "    ax[1].set_title(\"First 5 words (acc: %.2f%%, n: %d)\" % (acc, N), fontsize=22)\n",
    "    show_matrix(ax[1], langs, with_cutoff)"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 23,
   "metadata": {},
   "outputs": [
    {
     "data": {
      "image/png": "iVBORw0KGgoAAAANSUhEUgAAA7QAAAHhCAYAAABeG/LDAAAAOXRFWHRTb2Z0d2FyZQBNYXRwbG90bGliIHZlcnNpb24zLjMuMywgaHR0cHM6Ly9tYXRwbG90bGliLm9yZy/Il7ecAAAACXBIWXMAAAsTAAALEwEAmpwYAABxqklEQVR4nO3deXxU9b3/8fcne0jYkSUoIAiKK2WptmqrItXWulBrr1rbura2t1V71Vqvt7a/eu0tta21i9dat7oUrVbBpUgFtS63Lqi4AIKgogiIsiRkJWS+vz++Z8gQZpJMZjJnTvJ6Ph7zCJxl5jPf+c58zucs32POOQEAAAAAEDUFYQcAAAAAAEBXUNACAAAAACKJghYAAAAAEEkUtAAAAACASKKgBQAAAABEEgUtAAAAACCSKGi7iZm9a2aug8dJGb7GT4Ln+Umb6UcE05/sxHOM6UScyR4/6ei584WZ/d3MtprZbmHH0p3MbIiZnW1m/2tmL5pZU/BZ/b6T6x9sZg+Y2QYzazSzt8zsF2bWv4P19jazO81sbfCaq4MYRmTwXqqC51gdPOdaM7vDzCZ0sF7/IOa3gvewIXhPn2xnnc+b2cvB66wxs/82s6IUy/Y1s/fN7Fkzs66+v3xhZmd24rs+vJ31jzWzf5jZJjOrN7M3zOwKMytNsfzc4LvY5b6B3imdnJqw7Jhwo25fJ/PvqWHHmUsJv0m35eC1Rge/W3d392vlAzObaGa3mNk7Qb6rN7MlZvZLMxvaznoFZvbvZrbIzGrNrNrMnjaz0zKM5/TgeaqD510UvE67tUkX8k5fM/ujmX1kZg1m9qSZTWnn+S8K+uAxmby/fNGJ384b2lk3rW0qM5tsZjEz+1X3vaPkkm60IavmS1qfYt57uQwkhVpJf04yfS9Jh0r6UNKjSeYv7saYJElm5iTJOdflwsHMvijp85J+6pz7KFux5anDJN3clRWDxHSHpEJJz0r6QNIhki6VNNPMDnXObUiy3mclzZNULullSU9JOkjS+ZJONrPDnHMr0oxloqSnJQ2W9KakByRNkHSGpC+Z2eecc88mWW94EPtYSaslzZU0UtJJko43s9Occ/e2WWeypIckNQbvYz9JV0gaJOk7ScL7b0nDJH3e9aybeK+S9EyKeQ3JJprZDyTNktQi6UlJmyV9Vr6Nvmhm051z9W1W+5Gk4yX9TNJZmYeNXiivcmo28pSkOkn3pZj3TgbPi/b9SlKx/O9SjxYUZ3MllUpaKZ/3yuTz/MWSzjCzz7TN12ZWKOl+SSdIqpH0j+A5pkv6i5kd4py7sAvx/EE+xzZKWiipOXjO30uabmZfds7FkqzXlbxzu/x2wKuS1kg6RtKTZjbJObeqzfPvIekqSbOdc/PTfV957m/y2/xt/SvZwl3ZpnLOvWxm90v6rpnd4Jx7K4vxt885x6MbHpLeleQkHdGNr/GT4DV+0mb6EcH0JzN47jMzfY4svD/nu2iX1y+QL4i2SuoXdp/IQXt9StL1ks6RNEn+B95J+n0H6+0uqV4+QZyYML1I0t3BczyQZL0KSeuC+d9tM++XwfSXJFman9mrwbrXtJn3vWD6B5L6JFn3oWD+bElFCdNPDN5bnaSqNuvMDdY5LPh/uaQlkrZLGtFm2SnB9KvD/qyz2Gfi3/Pb0lxvqqRY0KYHJ0yvlPTP4DmvTbHu34J1Dwz7/fOIziOdnCppnKR9JBXnIK4u5ylJY4L13w27ffPl0dXfpC68zqHB69we9nvOQZsWSHo/eL9XJubkII/PC+bNTbLuxcG8JZKGJUwfL79jySVuN3QynpOD9dZJGp8wfZikpcG8C5Osl3bekfSJYPp8SQXBtG8E0/6Q5DXmSNqU+F6j/kj47RyT5nppb1MF8w8I1rsvl++TU47Rk31e0t6S7nXO1YQdTHdzzv3LOfcd59zNzrnF8sVXZ1wkX8j92Tk3N+H5tkv6pvxe2ZPMbN82650labikJ5xzbU9rvkz+qN9k+c+hs74g6UD5Pcg/TJzhnPud/B7ZKvmNnh3MbH9JXwxi/WYQe3y9ufJ7aPsE7zXRZEkrnHPPBMs2SLpL/kj1tITnL5R0o/zRkqvSeD891Q8lmaRZzrnn4xOdc7Xy/SIm6TtmNiDJurcE634vB3GiF3LOrXLOvemcaw47FuSti4K/N4UZRI6MV+uO66tdUHVIknOuTtJPg/9+KnGlIO/9IPjvt51zHyas95Z8npf8WU3puDz4e5lLOIIXPP+3g//+MMmpx13JO5ODv7e51iO+d8gfGW77fmfKF2s/SHyvvVEG21Ryzr0uaZH8duOonAQsrqHNC8H5/M7Mjkgx/7Zg/pk5DawdZraHmV1nZsuDaxJqzF9XeKZZ67WF5s0L4v9TkucpMLOFlnCtpwXXBicss9P5/mmEGT9lNNkp1fFrRq8Jrtv40My2mb9W8z4zO6SD93+wmd1lrdd4fhw8z/8zs8FJlp9oZjea2cqgvTab2Wvmr10ZncZ76g4nBX/vajsj2BHwUJvlOrNei/zR3WTrdSaWu4PnaOuuNsu1Xe9B59zWNNYbLH/KUqJNwd+yhGkXyCfGbzvnGpM8f8Ys4dp3Mys2f03QmwnXrdyZy+TQTpwlat1Jkeyzf1v+FKYS+R0UbT0qfynD6SkKXiAjluIa2sRca2afMbNHgt/umLVef1tmZj80f119bfD7vs7M/mX++vqyYLls5amsMbPfBq//7TbTLXifzszuSbLeX4N5X2ozvdjMvmtmzwc5vsHMlpnZz1Pkufg1we+aWZGZXWJmr5pZnZltaRPPOUEbNwSxzTGzAzt4f6ea2ePmr51sDtZ73cz+YGbj0minKvlc8I785S1t5xeb2dfMbLb5bZyt5q/VXGpms8xsUDvPXWxm3zSzJ4I4m8zsPTN72My+mmR5M7OvmN9O2mB+O+QD89tF2drp19TJ5T5u8/9PSRoqaY1z7qkky98rf6rwNDMb2ZkXMLPd5c922hasvxPn3D/lz8IaLn86dHy9ruadeD/dnLBsTFK1EnK8mfWV9Fv5/tClS7c6o81v0BQze9DMNgZ5/lUzO6e7XjtNJwV/092mivuz/IGBb2U3rNQoaJE2MztS0uvyG/kF8huoz8sfWbtVCQVksCfwa/I/UOfaroMIXCnpKEmvyJ/aIvnrcxOL0D+3eXQmxj6SZshf//d/KRa7WtL35a+heUHSg5I2yp8O84yZnZLiuS+X/+E8Xf505gfk33//4P0c0Gb5rwfv6Tz5vYsPyZ8eUxC85yPbLB/fgXFbZ95rJsysn/zpeZL0YorF4tM/0Wb6J9rM7+x67enqc3Z2vb3MrDJh+ruSxtnOg0DtE/x9R9pxTc1PJd3hnFuQOvSsKZY/BeyH8keq58nvef6qfL8c0HaFLPSZvYKN9RvN72Q5vU07Jdpbfs/sJtfm+qMEKT/7YEfFk8FzHN3FeIFMnCLpCUmjJD2m4Bo+80eEHpH0P/LXjf1T/hT5pZL2kD8SNSB4jsXKME8lqDCzy80PXPNbM/tOsOGfroXB37bfq0lq3bA/ymznnc7yOSgm3ybx6WXy10v+TtL+8uMjPCT//i+T9JKZjU0Rh8m329WSNsjn1iUJ8/8gf2T0QPn8/Jh83nxeUqrBZn4if+rjYZJeky+GXpDfaP6OEs6o6YTj5C+pWZh4tDLBMPkjUMfIbxP8Xb4v7CZ/xPJFMxuSJMaBwXJ/lC8GX5G//vQd+VOcr26zfIn8Ka73yG+vrJC/lvpN+Tb/bZLXiO80OSKN97s6eM4+kq5o8/lXyG+3SFLbgw7t5lXnr1WNf66TOhlL/DmXBGdEJZMsf3Q177wb/I3ndQU7JHbTztenXy1fvH8rRZ/ItmPltyP3lP+evST/fbjJzC5uu7DtPIDcmC6+5tnBzp/rze+0a2/brKvbVHHxbaUTuxJol+T6XO7e8lB61/s82d6ykm4L5p/ZZvpPlONraCWNkD+CtV3+OoTEazH2kP8BTxbr4cE6NQqumZAvZFuCaXsliSGTa5NmBOs/1c4yxyrJdRLyg9Zsk09kfdrMmxk871ZJxydZd5qk3dv8vzl47+cktlcwf6KkiSk+79sy7IPx/pHyGlr5H1AnaXM7y8Tf86KEaf3in4+k/inWi1+78nEaMW8K1jkoxfyBCa9bmTD9ZXVwLY/8Hlknaf+EabOCaT+X3yFxZLDcu5JKg2Xmyu+5HpLJ59GJ935Ewnt7UdLQhHn95ROek3RFknW71GfU+j1P9tgk6ctJ1jkhmP9KO8/7fbVzDY2kC4P513dnm/LoOQ+ll1Pjy45pM/3JhP79zSTrfUat1/5XtJln8kVJ25yQSZ4a0873b5v8RnY6YxD0l881mxRcLxhMvyR4zteCv59ImDcp/pvT5rl+EUxfJmlkwvRy+aLLSfpXO+9ntZLn9eOD+dWSPpkwvVC+gIuvf1vCvFL502W3SpqQ5DnHS9ozjXa6K3iNs1PM7xvEWdxmern8JRNO0v8mWW9OMO//tOt4DWXygwkmTvtNsPxySfu0mVco6YQkrxFvnw6/B23Wm6rWa17fCj7Dh+Rz22ZJ/9m2r0n6tdoZDyFYJj4OxXc7GccFSjEuR8Iy1wXL/DJhWpfyjnzhulV+wLhPyO/YuSNY7lsJbdMi6f+l06ZdeWjn36Cz28w7I+G70fZ3JvG7NSbN13xXO/+2JD7mSBqYZJ0ubVMlzDO1bs8NTSferj44Qtv9njBLOkz2bWEH1kUXyRcVv3LO/dkFPVeSnHPvyx+FlNpcH+ece1p+L2BfSfeYP3XyLvmjlOc551ZmOc5Jwd9lqRZwzj3qklwn4Zx7SH7v7yC1OXoq6cfB30uD5dqu+6Jzbk3CpCvk9wT/0vlrW12b5Zc559rGuE4+wa1LFXsWxfes1bWzTHxUvL5J1mtv3WTrZRpP4gh9yeJJ9338XP4o6GWStkh6XH4v8Pecc03mT8E7QdIlzrkdp2OZPy2xu34/44lux6jSzrlq+eJb8iNBttXVPrNOfvCwT0oaIn/05VPyZx0MlP+utr11QVfbOtHS4G86R+8BKTs59THn3I1Jpg8L/j7t/LWFOzjvWbfrCKqZaJK/Nn+G/OihfeSPVMZ3tP2n0rhmP/idWCT/3Z2cMGu6/I7VnwT/P7rNPKn16K7MrFyt1zJe4Jz7IOE1GuRHsa+VdIiZHZoinMtT5PWLgr/XOudeSHjeFvlR9dcmWaeffDG5yiUZNd8595Zz7p0UcSQzKfibdPvAObfVOfeQa3MNdvDevyu/0+DkxHlmNkn+aNRW+SJgbZt1G51z8xKWHyrfxjFJX3LOvdlm+Rbn3INJwlsePNLqh865RfK/7S/K38XiZPlrJAfL78D5V9vtE2Xnt76tTLc50lrP+btb/Kf8AZeX5Qv4M+RH773JWsfHWCk/+r6kHZfDlbf7TjLzN+fcLYkTnHN3yvfJfvJFdqJmtX726Y4N8LCkU+XPxiuX//y/JX/2xImS5ibZnsnosw/6Uvz7lZM8z217ul+qWwykukVGvotfm7DLtQ+Bl+Q7+SQzK3M7X2/4P/J7wI+R31PcX9IfnXO7XNOTBfF7qm1sb6HgtKEvyp/eM0Ct34n9g78T5E9Biw9hfpD8j0mHp5QFP5Qzgv92euAJ59zlah00Ad3IObfZ/K17zpa0r3x/ucs5tyThmponnHO3SVJwDdRP5U9HbDKzhyT9u0tyS6MMvOf8oAptxTd4qpK8jy71GedvS9D21gTPyd8e6VeS/kP+1hbZvn1B/DrlYe0uBewqGzn1/hTTX5Y/UnOOma2Q3+jstsFhnHPrtOs1Zm/ID4jzrPypuj8ws+vbFkjtWCjpYPlCdZGZFcufIfWc/OVBTfIF7TXB8vGCNvFyiinyG7RrnXOPJYn74+C37zT5M0t2uY2a/E6xnQSXdsQL4DuTPG+Tmd0rfwZH4vSPzOxdSQcFv0t/alsApqmz2wefkG+fMfKjAcdP1d0maTczG+ic2xxMOzb4+6Dr3C0Cj5K/3vNZ59ySjhaOc87t0/FSuzJ/jfgd8gXRdPlttXL5bZRrJC0ws3Piua4ncc79zsxelvQl+c/xBflLiFrM7D/kC64jg/43VP7WQcdLKjOzdyT9yDm3y3W7GXo4xfQ35c/c2ynPBzuVuvTZO+e+22bSKkmrzOxR+btKHC7fNqluHdZVOc3zFLTd7+fOuSfDDiKL4tfMvJhwGUYqg+WvnZXk99iY2dfkr1voL3+U5qJuiFHB80v+dOakzOxb8qfU9Gnnefol/Ht08Pc9l/raj0RDgufe3g1HoLMlvoetop1l4nvqEgcGSDxSWiF/6kln1utMPAPbiSfxyHCyeNJ9H3J+wIPrkiz/M/nP8ChpxwiId8qfMnSx/F7OqySNNbNpLsk987oo1b004325LMX8bPtv+Q3L/cxslHMuHleX2zpB/L0MyChC9EbZyKmrk010zq0ys+/L33bsD5L+YGZvy59COlf+NMlkg9VlnXPuITN7RX5j+2j5azo7Y6H8Eamj5Y/0HiL/XV3gnKs3s39JOszMSuWPDn5GvshNLErjA/y0d9Tz7TbLJtqQIkcOkT99OKYUn4Far3ls6+vyG9z/Iek/zOwj+SJ9vqQ7g6PTndXu9kFwTeBd8mfntKefWgcbim8fdLbQTnf5Lguudb5bPtYZCUV4taQ7gj7+tKTfmNkc59yWYH42fuvbynSbo0uxOH/v+p12vARnCv5U0q3OuSeDgxCPyheUV8qfmn2BpDvNrNYl3AUiC0LP886598zsVvlTtb+gnQvayOV5TjmOhnz6nAqDv/do10Ew2j6Sjax3klq/ILsreTLMhi3B337JZprZNEn/Kz8Az6Xye74q5a87MvmjyVLrHlnJnwKWjnSXD0N8o2JAMEBUMnsEf9+NT3B+9OO2ibzD9TohvmxHz7nR+aH6O7Ve8N7i7y/VhlTi8tPkBxq5OuEUt8vkf+RnOufmOOd+KX/0crJaj8RnQ7YK44wEGz3xI8+J39N3g7/tjbjc0Wcf/yw2p5gPdKeUOySdvz3YaPnTQeO38DpD/qykRe38TnaHeLGTTp58Vv79HRoUrW1PKV4ov6P10/JHcisk/V+KArSrOawzO3zTEly2tKekf5PP3Wvlz676vaSVHQxw09aW4G+qz/J/5IvZpcHfKkklzjkLtg/il3ZEZfvgNPkdCX9PKGZbA/HF3mr5Qj9xcK13g7+p8rGUfp7v6nPG/51J3mnr9/Knbl8a/H+G/A6kXznnrnHOzZE/clmnNrcRzIK8yPNK/RvzbvA3k22qnOb5fCqUerNtwd9Uo4q298XPtfeDv1c5587s4LHTEPDm72t1nfz7vUu+s98TjPSXbfEN8V1uLRA4WT4Z/dY590vn3HLnXF3CNSR7JVknvkdtj05eW7FR/seyyNK4pUAuBXu14yMGpholMj7q5Cttpr/cxfXa09Xn7Ox6K13yIeh3SLimZrlar1uV/OnmbybsvZb8KIXxeT1K0A7xIxmJOw/elN9gHdROv+7os49/L7N5qjaQFc659c65G5xzZzjnxshfc/l68DfbG7btiX9PattdKoFzLn60tVz+9N7p8kdQ4vftjJ9afLSSn24stZ5ZtWc7LxU/W+uDdpZp62P5Hd0FSl2YjEm1snOu3jn3V+fvtz5JfiP8Hvkjv39II46Otg/idzj4t+Ba2nXx62mDUYGHJ1knvn2wdydjSHf5TMTbur2j2FuCv4m3JGo3r5q/m0T88qzO5vn4cvu1sx01rc2yUnbyzg5mdrL8acX/4ZyLn3oez+PxvK4g3y9TD8zxgVS/MdnYpsppnqegzQ/xhLDL+fFmNkw7D+4QtvigBklvaZNKkAT+Kp9kL5M/fegJ+Wt1rkmxWjyBdOXU+PiXcd8U8+M/2u+3nWFmuynJETfn3Hr5a39L5ONvV3BqWnxD4dyOlg9R/DSaZPfI6yf/oy/tek1Ue+sVyg9CkGy9zsRyavAcbcVfK1UsxwfXvnZ2vWQukk9e33TObUuYXqddT0+Pn20QhaPx6fqi/PvdqoTT4oI2if8OJPvsx8oPPrJNwfXnScS/ly+nmA/kDefcq2q9LKHthm0meSqlYMyGw4P/prp1Rirxo7Enyh+Ffco5tz3huWq0c0G7cOfVd4yFMdLMdhmEzvw9aON54cnOBhXEEL+NXrLfjhJJX07j+dbJD7wopVdwdHn7QP52fcmut4qPM3Bislv6JPG4fN/5tJlN7MTymYhff31IspnB7YZ2ul1d4F+SPpK0u5l9Jsmqp8if5faiSxg4rD3ODx76svx21C7bkWb2Wfmz99Zr58IyG3knvmw/+fExFgQDMcXFB0BKlud7XI4Pbt8U/761/Y3JaJsqeO54n0rnoEbXuRwMpdwbH0rvFgPxobrfkTQiYfog+fP548Nrn9lmvZ8E03/SZvoRwfQnM4j/zGTPIX9KR7X8j8a/SypKsu5+8qP2JU67LXi+uQnTRkj6UCmGBg/aw0k6oAvxl0lqlN+jV5xk/n+o9fYMibd/6Ss/lH28zdu27ZeC6TWSvpDkeadq59v2fFJ+RMTmtp9fMH8f7Tpc///IFxD/k2EfjPePlLftSfhM6+UHQzkhYXqR/H3/nJIMsS9/RsG6YP6/t5l3TTD9Ze16K4CRwft7Uwm3gwjmFcgPUuAk/aLNvO8G0z9QmyHtg/kPB/NnJ/ZL+Y26FvlkVdVBW4yW35C7Mcm8x+VPEzokIdb4ax6dor/flsbndYTa+d6qddj+d5PMS7vPyCftbyf2/4R5x8nvVXXJnlN+r20saNPEW29UqvW2BNe289r3BMvsclsgHjySPZTd2/YkfQ756+W/oDZ5Tf60478F617fZl4meeq8tr+BwfR95QeucfKnA6f7vFODdRuCvxe1mT9XPi9tkz8yV5jkOeK/4W9o5+2SMvmd006pb9vzbjuxnRQss0XS1ITpBZKuVWvuvS1h3mj5ncL9kjzfRcHyS9Non7OCdW5KMT9+e6PLk7Trxwkxtu1fDwbTn05ss4R2a3vbnvhtipaqze2Igj6X7NaA8dz5yY7eZ8I6B8j/Xjv5MwwSb+nUX74ocfLXjBa2WTd+y6cl2vlWcuPVmv9PTPKaKXOSfBHlgvX3Spg+NHgdJ+nCJOtllHcSlv+9/HdjXJvp8dt2PRxvI/mdADFJz6To67v0gw5eOx5nqt+g25R8ez/ldlMHr3eipMlJpg+Wvy7fyW/T73JrHWWwTSV/HbKT9EZnY830kZMX6Y0PpZd8S9R6z6dN8gXV/ODfbyT82JzZZr2fKMcFbTDvSPnTaZ38nr/H5E8hfkT+NBon6e6E5b8RTFstaVCb5/pc8GOxSdLoNvPi90DbID+gwU1KkYBSvIf4/dF2+QzkBx56L+H57w/aeaP8j+zNydo2WPfKhB+y14Iv+8PyyWCX15MfQbc5mLdSfmNgTvDZJvtcb1OaxVDCus8lPNYEz7O+zfRkP26nyW/gxCQ9FbR3vA+/pRT3EZP0Wfli2MnfLmK2fHJ28nt2906yzhi1kwjkN+biGw1Lg+dcFPy/XtJhKWIZLn/6tAtiv1t+wyIWvLevdKL9HgraK9l92Y4Inrs26CuLg///S7sW7fFE8ac0Prv48yf93qr9gjbtPiM/UIOTT+zPBu11v/zpVfHP529KskMoWP8HwTLb5W8M/1e17qB6Tkl2OgTrFQXL1UkakG4f59E7H8pNQXuRWoutx+Xz2gPyeS6+AT66zTpdzlPBb0hMfifefcH6L6o1XyxTwg7SNNqqQK33gHRqU2yr9V6gTtKcFM9RJn8WVfw370H5HVHxtlgtaWybdVL+RrVZ7o/Bcs3yZzH9RT43Nki6vu1vmVrvldskf+r03cHvTTyHbpP0xTTaZ1iwzjtKcp9ftRZcLviMZkv6p/xG/F3t9K9Bat0R0SC/bfSXoN9tbtsu8te1PpLQFk8Fyy9Q8FuaJLZ4XB1+D9qs918J666S/62fp9ZtuWpJBydZr1CthXp1sN5Dat1Z8tsUr3db28+xzfz459wQPN/9ar2v6QNKspMlWK9LeSdh/U8Gn+PlKeY/HjzXq0EctfLf0S+0WW5sQnumU2A+2d7np9QF7ZiE1xuTxuv9JljnnaCd7wr6crytt0ianmLdLm9Tyd+608mPQ5LW71dXHzl5kd74UBrJN1h+UPAF/0D+h3a1/N7Kfu108J8ohII2mD9c/qbvi+VPSWwM3vOT8nsAxwXL7RP8IDRL+nSK5/oftRYGiXuByuUH3Xk7aBOnJD/w7byHY4J1bk0xf6h8Yn1bPlG+L+lP8keOk7ZtwrqHyf+Qrg1i+0g+kf1YbYr2YPkDJN0afK5N8hsbi+VvXj+qzbLxz/u2LnxurhOPpH1S/tS0OcF7aZLfwPiFpP4dvObe8j+S64P13pN0g9rsoU5Yfow6+GGWH4TjhuC5muQ3JO9Um73YSdYbIH9kYWWw3kfBe+pwb7ZaN2JObWeZL8mfPtMkX3TfpCRFmVo3tA5J47M7QrktaEvkR2meL//drQ368gfyO4O+1InnOFZ+o22z/IbJEvlTAEvbWeeLSrPY58FDuSlox8n/9j8e/PY0Bt/zl+V3ZO6WZJ0u5ylJ58gXssuD71CzfIHxT/kRxsszaK/7g1jWJ5m3b8Jv8PfaeY5i+Q3TF9Sa59+UH1tgcJLlU/5GtVnO5I9OvxI8Z3xH/iS1bnvclrB8X/mdDXPkd7DWyp8ltUw+h+/bhfaJn3302RTzjwj6wcbg9V4J2qIgVf8K1iuVP5voWfmioVE+7z+oJLkleL4z5E/73hT0oTXyv6vfSbJ8u3m8g/d8ZNDf1gSvUx+04XVqsx2SJMbvyp/RVhe0/TOSTm9nndvafo5Jljk9aKea4Hlfkj/zr6CD95F23gnWKwo+x9eVekftIPkDGvHtoJeV/Ah0fHvh0TQ/gyfb+/yU/YL2CEm3yG9vbpD/jdkatMPP1UExri5uU8kfhNjeXr/K9sOCFwZ6nOAc/iXyp9SOdH5kXqBbmVmVfFH4gHPuS2HHk2/M7H750w4nOedeCzkcAL2QmR0ivxP9Dudch2NiAInM7I/yO2UmO+cWhxxOXjGzA+WPcP/NOdfpa+IzxaBQ6LGc31tzqfz1FReHHA56j6PlT2n6z7ADyTdBojtJ0p8pZgGExTn3nPzRytPMbHzY8SByZkj6C8VsUlfKHwm+PJcvyhFa9Hhm9nf50SLHOuc+CjseoLcys7nyA+9McH6EUgAIhZmNlj/l9kHn3KkdLQ+gfWY2Wf5042udczk9kERBCwAAAACIJE45BgAAAABEEgUtAAAAACCSisIOIBuGDCp0Y/YoDjuMSFvxWp+wQwCAvLFVmz92zu0WdhxR1n9QkRs6ktyciQ/fKA87hMiz0tKwQ4g819QUdgg9An0xczVN65Pm5h5R0I7Zo1gvzN8j7DAi7ZiqSWGHAAB5Y4G7b3XYMUTd0JHFum7uuLDDiLRr95oYdgiRVziGPpiplhWrwg6hR6AvZm7+8llJczOnHAMAAAAAIomCFgAAAAAQSRS0AAAAAIBIoqAFAAAAAEQSBS0AAAAAIJIoaAEAAAAAkURBCwAAAACIJApaAAAAAEAkUdACAAAAACKJghYAAAAAEEkUtAAAAACASKKgBQAAAABEEgUtAAAAACCSKGgBAAAAAJFEQQsAAAAAiCQKWgAAAABAJFHQAgAAAAAiiYIWAAAAABBJFLQAAAAAgEiioAUAAAAARBIFLQAAAAAgkihoAQAAAACRREELAAAAAIgkCloAAAAAQCRR0AIAAAAAIomCFgAAAAAQSRS0AAAAAIBIoqAFAAAAAERSUdgBRNHW2pieeLZBixY3atGrTVq0uFEbN8ckSUueGqV9xpe0u34s5nTTXTX68z01WvZWs1panPbas1inzeyrC84doJISy8XbyHsDhw3QaZfP1MHHTdaQkYNUV12vN19YqQeue0SvPP5G2OFFBu2YOdowc7QhwhaLOS28f4uemFutd95sVF1Ni8rKCzRybKkOnt5XJ3xjkPpUFoYdZt7ju5yZ7bEmbap/T9WN61TTuF7VjevV3NIgSTp0zLmqLB0ccoTRQV/sup7WD805F3YMGZt6UJl7Yf4eOXu9OfNqdfLZ65PO66igbW52mnnWOs1bWC9JKimRCgtMDY3+c5g2qVQL7hupyorcHjw/pmpSTl+vI3seMErXLPyx+g/pJ0mqq65XWWWZCgsLFIvFdMsVs3XPrDnhBhkBtGPmaMPMRbENF7j7XnLOTQ07jigbf0C5u27uuLDDkCQ1NsT00/Pe06v/qtsxraJvgeprY4pvBg0dWayf3TlGI0a1v1M6l67da2LYIewkit/lwgn50QfjPty6QovXPpB0Xr4WEi0rVoUdwi7oi5mJYj+UpPnLZyXNzZxy3EVDhxTq89P76MqLB+qGa3br9Ho/mrVR8xbWq6zMdMt1Q1X79jhtfXus5t4+QoMGFujFxU06/9IN3Rh5/ispK9FP516m/kP66a2X39a5+39fJw38hr406Ezd+6sHVVBQoLOvPk1TZhwYdqh5jXbMHG2YOdoQ+eDu33+kV/9VJzPpG5cO1V8X76O/Lp6oB5ZO1A9+s7sq+hVowwfN+u3la8MONW/xXc6eksI+GlIxVuMGH6p9hx0TdjiRQ1/Mjp7UDylou+D4z1Vo3et76uE7q/TjSwZrxmf7dGq99Ru267c3VUuSfn7FYH3jK/1UWGgyM31xRoVu+vVQSdLdc2r12tKmbos/3x33raM1fMxQ1W9t0I9OmKXVS9dIkuq3NujGS+/QMw+84H+sfvbVkCPNb7Rj5mjDzNGGyAdPPuhz79FfHqCvnL+bKvr6U4uLSwr02eP767wrhkuSXnuuTlurW0KLM5/xXc6OoZV76ci9vqcpu5+ivYYcpiEVe4YdUuTQFzPX0/ohBW0XFBZ27RrX+x+pVVOTU/9+BTrvjH67zD/x2EpNGFcs56TZD2zNNMzImn764ZKkx2c/o41rN+0y/95fzpUkTZgyVrtPqMppbFFCO2aONswcbYh8sOXj7ZKkcfuWJZ0//oDyHf9uaojlJKao4bucHWZsemeKvpi5ntYPe9a7yXNPPOsvtj78kDKVlSVv+vjR3ieeachZXPmkvLJM46eMlSS9NH9x0mWWPfeWarf466A+MX3/XIUWKbRj5mjDzNGGyBfDdi+WJK1a2ph0/luv+5w7YEiRhgwvzllcUcF3GfmCvohkKGhzaNmKbZKk/SakHnBiYjBv2Vvb1BMG7ErXqIm7q6DAd8t3l6xJuoxzTu8v99c5jd43d4OBRQntmDnaMHO0IfLFMf82UJK04L4t+usNH6luqz+tuHlbTE89XK0/Xb1eZtI5lw8LM8y8xXcZ+YK+iGTy+rY9ZvakpEucc4vCjiUb1m3wCbRqeOpmrxrmr+uprXOqrXPqW9m7buEzaMSAHf9OdhpJ67zNfvnhA1Iu05vRjpmjDTNHG/ZMUczNJ541WB+uadbDd2zSn6/ZoD9fs0EVfQvUUBdTLCbt84ly/dt3dtMnj+obdqh5ie8y8gV9EclwhDaH6ur9dTllZamL1D7lrR9JbV3vu46nrKL1+qamhm0pl2uq94NmlVcmvx6qt6MdM0cbZo42RL4oLDR987+G69z/HKbCYJ9y3VZfzEpSQ21M1Zu2hxdgnuO7jHxBX0QyeXGE1szGSHpU0kuSJktaIunrYcYEAEBv1pNy86aPmnXVt97XilcbNP1LAzTznMEaMapEmzY065lHa3T37z7Sby5bqw/e2aYzL+W0YwCIknw6Qru3pOudcxMl1Uj6TsjxZF1FH9/cjY2pr42tTxhdsbIinz6e3Gisax2wo7Q89bXGpX1KJUkNtckH+OjtaMfM0YaZow17hB6Rm399yQda8WqDPveVAfqPa0Zqz33KVNanQFVjSvWV83fTd//bj4T6txs/1uoV9MO2+C4jX9AXkUw+VUzvO+eeDf59p6TD2lvYzL5pZovMbNFHG6Nxz7iqYf6A+Nr1qU9rWvuhfy+VFaa+lfn08eRG/JoHSRpcNSjlcoOr/AAfm9Zv6e6QIol2zBxtmDnasEfocm6u3pQfufm9txr1yjN+xNOTzhqcdJmjZg5Qv4GFisWk5x/vvbfNS4XvMvIFfRHJ5FPF1PawZbtD/DrnbnTOTXXOTd1tcGE3hpU9Eyf4WwEsWZH6nP/4SMgTx6fe69STvf/mB4oFFzWN2W/3pMuYmfbY2+9NX730/ZzFFiW0Y+Zow8zRhj1Cl3Nz/0H5kZvfX9Wac4ftkTq3Dg/mbVjT3O0xRQ3fZeQL+iKSyaeCdpSZfSr49+mSngkzmO5w5KH+HrPPPN+oxsbkAz4t+Ge9JOmow8qTzu/pGmobtWLR25KkyTMOSrrMPgePV+WACknSKwvfyFlsUUI7Zo42zBxt2CNEPjdbwjiMH61NXaxu+MAXvuW98HKfjvBdRr6gLyKZfPrVXi7p381smaSBkv435HiybuYXKlRaatpSHdPNf6nZZf5D/6jT8lXNMpNOndl7bx3w+OynJUlHnX5Y0uHWT7n4eEnSikWrtGbF2lyGFim0Y+Zow8zRhpEX+dy858TWUU7n37M56TLPL9yqLcHlS3tP6p07lDvCdxn5gr6ItvKpoN3unDvDOTfROXeyc67eOXdEvt7n7uONLTsem7e0Hm3dUhPbaV4s1np21vChRbrg3P6SpMuu2qg77q1RS4uf//eFdTrnog8lSaeeVKkD9y3N4bvJL4/8cYHWv7tBFf366KqHLteoif6UkvLKMp076wwdfvIhkqRbrvhLmGHmPdoxc7Rh5mjDyItUbk5mxKgSTT7cH62Ze+tG3XbNh9rysR/LoqGuRY/dt1nX/uADSdKw3Yt18PTeu0O5PXyXs2fb9vodj+aW1kGLtscad5rnXLtn+Pda9MXs6En90PIhyODWAA875/bvyvpTDypzL8zfI7tBdaBwxMpOLbfqhdEas0fxjv83NzvNPGud5i30pxaXlpoKC6T6Bv85TJtUqsfuHZnzAaGOqZqU09fryNgDR+sXC65U/yH9JEl11fUqqyxTYWGBYrGYbrlitu6ZNSfcICOAdswcbZi5KLbhAnffS865qWHHEaZMc/P4A8rddXPHZTeoLtq0oVn/+bXVen9l045p5ZUFaqht3SE9YEiRfnrLKI3bL3+O0F6718SwQ9hJFL/LhRPyow8mmr98VqeW+8zY81Ve3L+bo+lYy4pVYYewC/pi5qLWDyVp/vJZSXNzXtyH1jn3rqQuJcyoKS42PXj7CP3pzhrd/tcaLV3RrJYWp0n7l+jUk/rqwvMGqKTEOn6iHu7t11brvAMu1mmXz9TBx03WkJGDVLNxq5a/sFL3/+ZhvfI410R0Bu2YOdowc7RhNPWk3DxoaLGumztWj87erP/7R41Wr2hS3dYW9aks0IjRJZp2ZF+d8PVB6j84LzaL8hbfZeQL+iIS5cUR2kyFcYS2p8m3I7QAECaO0GYun47QRlW+HaGNonw7KhZF+XiENoroi5lLdYQ2n66hBQAAAACg0yhoAQAAAACRREELAAAAAIgkCloAAAAAQCRR0AIAAAAAIomCFgAAAAAQSRS0AAAAAIBIoqAFAAAAAEQSBS0AAAAAIJIoaAEAAAAAkURBCwAAAACIJApaAAAAAEAkUdACAAAAACKJghYAAAAAEEkUtAAAAACASKKgBQAAAABEEgUtAAAAACCSKGgBAAAAAJFEQQsAAAAAiCQKWgAAAABAJFHQAgAAAAAiiYIWAAAAABBJFLQAAAAAgEiioAUAAAAARBIFLQAAAAAgkihoAQAAAACRREELAAAAAIgkCloAAAAAQCRR0AIAAAAAIomCFgAAAAAQSRS0AAAAAIBIoqAFAAAAAERSUdgBZMOK1/romKpJYYcRafPXLg47hMijDwJAqw/fKNe1e00MO4xIIzdn7gtHjAs7BECS1LJiVdgh9FgcoQUAAAAARBIFLQAAAAAgkihoAQAAAACRREELAAAAAIgkCloAAAAAQCRR0AIAAAAAIomCFgAAAAAQSRS0AAAAAIBIoqAFAAAAAEQSBS0AAAAAIJIoaAEAAAAAkURBCwAAAACIJApaAAAAAEAkUdACAAAAACKJghYAAAAAEEkUtAAAAACASKKgBQAAAABEEgUtAAAAACCSKGgBAAAAAJFEQQsAAAAAiCQKWgAAAABAJFHQAgAAAAAiiYIWAAAAABBJFLQAAAAAgEiioAUAAAAARBIFLQAAAAAgkihoAQAAAACRREELAAAAAIgkCloAAAAAQCQVhR1ATzVw2ACddvlMHXzcZA0ZOUh11fV684WVeuC6R/TK42+EHV7ottbG9MSzDVq0uFGLXm3SosWN2rg5Jkla8tQo7TO+pN31YzGnm+6q0Z/vqdGyt5rV0uK0157FOm1mX11w7gCVlFgu3kYk0BczRxtmjjZEPqAfZsY5p7vn1OrWu2v0yutNqq2LacTQIk3/TB9d+p0BmjCu/dwNaXusSZvq31N14zrVNK5XdeN6Nbc0SJIOHXOuKksHhxxhdPB9zlxPaUNzzoUdQ8b62SB3sE0PO4wd9jxglK5Z+GP1H9JPklRXXa+yyjIVFhYoFovplitm655Zc8INso35axfn9PXmzKvVyWevTzqvo4K2udlp5lnrNG9hvSSppEQqLDA1NPq+PG1SqRbcN1KVFbk9AeGYqkk5fb3OiGJfzDe0Yeai2IYL3H0vOeemhh1HlJGbM5fr3NyebducTv3Wes19tE6SVFQk9a0s0OYtfmd0eZnpLzcM1wnHVIQZ5i6+cMTJYYewkw+3rtDitQ8knZevBW3LilVhh7CLKH6f800U2zBVbuaU4ywrKSvRT+depv5D+umtl9/Wuft/XycN/Ia+NOhM3furB1VQUKCzrz5NU2YcGHaooRs6pFCfn95HV148UDdcs1un1/vRrI2at7BeZWWmW64bqtq3x2nr22M19/YRGjSwQC8ubtL5l27oxsijgb6YOdowc7Qh8gH9MHOXX/2x5j5ap6Ii6TdXDdGWFWP18bKxWv3SGH3puAo1NDqdfv56rXq3OexQ815JYR8NqRircYMP1b7Djgk7nMjh+5y5ntaGFLRZdty3jtbwMUNVv7VBPzphllYvXSNJqt/aoBsvvUPPPPCC7yQ/+2rIkYbr+M9VaN3re+rhO6v040sGa8Zn+3RqvfUbtuu3N1VLkn5+xWB94yv9VFhoMjN9cUaFbvr1UEnS3XNq9drSpm6LPwroi5mjDTNHGyIf0A8zs+Hj7br+Np97L/n2QH3v3AEqL/ebkLtXFWn2DcM1YVyxGhqdfvyLjWGGmveGVu6lI/f6nqbsfor2GnKYhlTsGXZIkcP3OXM9rQ0paLNs+umHS5Ien/2MNq7dtMv8e385V5I0YcpY7T6hKqex5ZPCwq5d43r/I7VqanLq369A553Rb5f5Jx5bqQnjiuWcNPuBrZmGGWn0xczRhpmjDZEP6IeZefyZBm3b5v994Tf77zK/qMj03bP99Afm1am2LpbL8CLFjE3vTPF9zlxPa0O+VVlUXlmm8VPGSpJemr846TLLnntLtVv89SefmL5/rkLrMZ541g+ccPghZSorS95940d7n3imIWdx5Rv6YuZow8zRhsgH9MPMvbdmuyRpQP8CDR2SfDzRvffyY180Njo983zvzb/oXnyfM9cT25CCNotGTdxdBQW+Sd9dsibpMs45vb98rSRp9L575Cy2nmLZCr+LeL8JqQeNmhjMW/bWNvWEQc+6gr6YOdowc7Qh8gH9MHMWnFTV0pI6p27f3jpvaZCrgWzj+5y5ntiGFLRZNGjEgB3/Tnb4vnXeZr/88AEpl0Fy6za0SJKqhqe+41TVsEJJUm2dU21d7yxo6YuZow0zRxsiH9APMzdqd59zt9Y6rVm7PekyyxKK2HUftuQkLvQ+fJ8z1xPbkII2i8oqynb8u6kh9d7Jpno/WFF5ZVnKZZBcXb2/LqesLPU1uH3KW7t1b72Oh76YOdowc7Qh8gH9MHNHfLpcxcX+37+8fvMu8xsbY/rdzdU7/r+1tnfmXnQ/vs+Z64ltGGpBa2ZnmNkLZrbYzP5oZoVmVmtmV5vZq2b2nJkNCzNGAAB6E3Iz2hq2W5G++TU/6NPvb6nWlbM26oN129Xc7PTya406/mvrtHrNdhUFJ08VcLgEQA6F9pNjZhMl/ZukQ51zkyS1SPqqpApJzznnDpL0lKTzwooxXY11jTv+XVqe+hrP0j6lkqSG2saUyyC5ij6+yzY2pj6VuL6hdc9wZUXvzKr0xczRhpmjDaOH3Ew/TOUXPxqsY4/qI+ekq3+zWaMmv6uyUas07Zg1evyZBv30B4M0sL+/5GdAv96Ze9H9+D5nrie2YZi/ONMlTZH0opktDv4/VtI2SQ8Hy7wkaUyylc3sm2a2yMwWNSs/7jcaP9dckgZXDUq53OCqgZKkTeu3dHdIPU7VML/7d+365NfwSNLa4NqdygpT38remVTpi5mjDTNHG0YSuZl+mFRZWYEeumOE7rx+mI6b0UfjxhRr3JhiHX9MhebNrtIPvjtQW2p8/t1rbOqNZCATfJ8z1xPbMMytfZP0Z+fcpOCxt3PuJ5KaXevQtC2Sko7+45y70Tk31Tk3tVilOQq5fe+/+YFiMX90cMx+uyddxsy0x97+fk6rl76fs9h6iokT/EU8S9oZQTE+MMXE8b03odIXM0cbZo42jCRyM/0wpYIC02kz++rB26u04l+jteJfozXnthH63BF99PqyJjU3++U+NSX/r7lDNPF9zlxPbMMwC9qFkr5sZkMlycwGmdnoEOPJWENto1YseluSNHnGQUmX2efg8aocUCFJemXhGzmLrac48lB/j9lnnm9UY2PyQScW/LNeknTUYeU5iyvf0BczRxtmjjaMJHIz/bBL7p5TK0matH/JjtvnAdnG9zlzPbENQytonXNLJf2XpH+Y2WuSHpM0Iqx4suXx2U9Lko46/bCkw1yfcvHxkqQVi1ZpzYq1uQytR5j5hQqVlpq2VMd0819qdpn/0D/qtHxVs8ykU2f2DSHC/EFfzBxtmDnaMFrIzfTDrnh1SZP+cKsf5fiy7w0MORr0dHyfM9fT2jDUCwydc/cEpzQd6Jyb4px7zjlXmTD/PufcmSGGmLZH/rhA69/doIp+fXTVQ5dr1ER/KL+8skznzjpDh598iCTpliv+EmaYeeHjjS07Hpu3tB5t3VIT22leLNY6ANTwoUW64Fw/0uJlV23UHffW7LjR+98X1umciz6UJJ16UqUO3Dc/TncLC30xc7Rh5mjD6CE3I5knnq3Xr2/YrJXvbNuRd6trWvTH26t19CkfqLHR6ZQTKvWVE3r3zuTO2La9fsejuaV1wJ3tscad5rWe5Y9EfJ8z19Pa0HrCl6WfDXIH2/Sww9hh7IGj9YsFV6r/kH6SpLrqepVVlqmwsECxWEy3XDFb98yaE26Qbcxfuzjnr1k4YmWnllv1wmiN2aN4x/+bm51mnrVO8xb6U4tLS02FBVJ9g+/L0yaV6rF7R+Z8QKhjqibl9PU6I4p9Md/QhpmLYhsucPe95JybGnYcUUZuzlwYuTmV2+6p0TkXbZAkFRVJfSsLtKU6pvhm5KkzK3XbdcNUXJz6PvFh+MIRJ4cdwi7mL5/VqeU+M/Z8lRf37+ZoOtayYlXYIewiit/nfBPFNkyVmylou8nAYQN02uUzdfBxkzVk5CDV1TRo+Qsrdf9vHtYrj+ffuehRKmglKRZz+tOdNbr9rzVauqJZLS1O48cW69ST+urC8waopCT3CTUfC1open0xH9GGmYtaG1LQZo7cnLl8KmhXvrNN199araefa9TqNc3aWhfT0CFFOmRKmc4+ra+OObIi7BCToqDNXD4WtFL0vs/5KGptSEGLduVT0oyqfC1oAaSPgjZz5ObMkZszl48FbdTka0GL3idVbu6dN+kEAAAAAEQeBS0AAAAAIJIoaAEAAAAAkURBCwAAAACIJApaAAAAAEAkUdACAAAAACKJghYAAAAAEEkUtAAAAACASKKgBQAAAABEEgUtAAAAACCSKGgBAAAAAJFEQQsAAAAAiCQKWgAAAABAJFHQAgAAAAAiiYIWAAAAABBJFLQAAAAAgEiioAUAAAAARBIFLQAAAAAgkihoAQAAAACRREELAAAAAIgkCloAAAAAQCRR0AIAAAAAIomCFgAAAAAQSRS0AAAAAIBIoqAFAAAAAEQSBS0AAAAAIJIoaAEAAAAAkURBCwAAAACIJApaAAAAAEAkUdACAAAAACKJghYAAAAAEEkUtAAAAACASCoKOwDkh2OqJoUdQuTNX7s47BAij34IAK2+cMTJYYcQeRNmrw47hMhbNiXsCHqGbcdOCzuE6Jt3X9LJHKEFAAAAAEQSBS0AAAAAIJLaPeXYzB7vwnM659z0LsYDAADaQW4GAKBVR9fQjpXk2kyrkDQk+PeW4O+A4O/HkmqzERgAAEiK3AwAQKDdU46dc2Occ3vGH5KmS2qQdJ2kKufcIOfcIElVkn4rqT5YBgAAdANyMwAArdK9hvZaSf/nnPu+c259fKJzbr1z7iJJzwXLAACA3CA3AwB6rXQL2iMk/bOd+U8GywAAgNw4QuRmAEAvlW5B6yRNbGf+fhnEAgAA0kduBgD0WukWtP+Q9G0z+7qZWXyied+Q9K1gGQAAkBvkZgBAr9XRKMdt/YekaZJulfRzM3srmD5e0jBJ7wfLAACA3CA3AwB6rbSO0Drn1kiaJGmWpM2SPhk8NgfTJgXLAACAHCA3AwB6s3SP0Mo5Vy3pP4MHAAAIGbkZANBbpXsNLQAAAAAAeSHtgtbM9jCzW8xsjZltM7Ojgum7BdOnZT9MAACQCrkZANBbpVXQmtmekhZJOlnSEkmF8XnOuY8kTZV0bjYDBAAAqZGbAQC9WbrX0F4tKSZpf0kNkja0mf93ScdnIS4AANA55GYAQK+V7inHR0u63jn3vvyN3NtaLWn3jKMCAACdRW4GAPRa6Ra0/SSta2d+ibowcjIAAOgycjMAoNdKt6B9X9J+7cw/RNLKrocDAADSRG4GAPRa6Ra090s628z2T5jmJMnMTpZ0iqS/Zik2AADQMXIzAKDXSregvVrSGknPS7pTPmH+0Mz+JZ8sX5X0q6xGCAAA2kNuBgD0WmkVtM65GkmfknST/G0ATNIMSXtLul7Skc65xmwHCQAAkiM3AwB6s7QHiQgS54WSLjSz3eQT50fOuWQjKwIAgG5GbgYA9FZpHaE1sysTr9Fxzn3knNsQT5hmtp+ZXZntIAEAQHLkZgBAb5buNbQ/kXRgO/P3l/TjLkcDAADS9RORmwEAvVS6BW1HyiRtz/JzAgCAriM3AwB6rA6voTWzfpIGJEwabGajkiw6SNJX5e+HBwAAugm5GQAArzODQn1fUvzaGyfpN8EjGZP0g4yjAgAA7SE3AwCgzhW0TwZ/TT55PiDptTbLOEm1kp5zzv1f1qIDAADJPBn8JTcDAHq1Dgta59w/Jf1TksxstKQbnHPPd3dgUTdw2ACddvlMHXzcZA0ZOUh11fV684WVeuC6R/TK42+EHV4k0Ibt21ob0xPPNmjR4kYterVJixY3auPmmCRpyVOjtM/4knbXj8WcbrqrRn++p0bL3mpWS4vTXnsW67SZfXXBuQNUUmK5eBuRQF/MHG2YXeTmrqEfdt32WJM21b+n6sZ1qmlcr+rG9WpuaZAkHTrmXFWWDg45wvz3u8l3d3rZL/3pKI2cMrQbo4k+vs9d19iwWR+vf0NbNq5S3da12tZUq4KCQpX1GaxBu+2tqtGHqrSsX9hhdpr1hFvU9bNB7mCbHnYYO+x5wChds/DH6j/Ed4S66nqVVZapsLBAsVhMt1wxW/fMmhNukHkuim04f+3inL7enHm1Ovns9UnndVTQNjc7zTxrneYtrJcklZRIhQWmhkb/ezBtUqkW3DdSlRXZHjeufcdUTcrp63VGFPtivoliGy5w973knJsadhxRRm7OXOGEcWGHsMOHW1do8doHks7L54J2wuzVYYeww80z5rQ7f1tds7Y3tqiguEBnzz9R5QNKcxNYB5ZNyb8x5aL4fd527LSwQ5AkNTZs0QtP/lz+JB6vsKhULS3NkvMHRoqKy7XvJ76mAYPz5zdIkp6ad1nS3NyZU453MLN/lzTTOXd0ivn/kPQ359wfuxZm9JWUleincy9T/yH99NbLb2vW13+n1UvXqE/fcp1x5Zd1ysUn6OyrT9PKl9/WS4+1PTsMEm2YjqFDCjXloFJNm1SqquFFOv/Sjzq13o9mbdS8hfUqKzNdP2s3nXFyXxUUSI8sqNdZF36oFxc36fxLN+jO64d38zvIb/TFzNGG3Y/c3DH6YXaUFPZRv7Lh6l82QqVFlVr64fywQ4qUcx47qd35s099VB+v2KI9D6/Km2I2H/F9zlBQtA7abR8NGzlFA4bspeLiPorFtmvLxpVauWSuGhs2acnLf9a0z1yqktK+IQfcsXQPv5wp6a125q+QdHaXo+kBjvvW0Ro+ZqjqtzboRyfM0uqlayRJ9VsbdOOld+iZB15QQUGBzv7ZV0OONH/Rhp1z/OcqtO71PfXwnVX68SWDNeOzfTq13voN2/Xbm6olST+/YrC+8ZV+Kiw0mZm+OKNCN/3an+J095xavba0qdvijwL6YuZow5w4U+TmdtEPMze0ci8dudf3NGX3U7TXkMM0pGLPsEPqUT5avlkfr9giSdrneNq2PXyfM1NUXK7Jh16g/aeepd1GHKjiYr/9WFBQpEG77aP9p56lgoIitWxv0rr3ngs52s5Jt6AdL+n1duYvCZbptaaffrgk6fHZz2jj2k27zL/3l3MlSROmjNXuE6pyGltU0IadU1jYtWtc73+kVk1NTv37Fei8M3a9PuLEYys1YVyxnJNmP7A10zAjjb6YOdowJ8jNHaAfZs4st5eg9DZvPvSOJKl8UKnGHDoi5GjyG9/nzBQVl6uyX+p26VM5VH0H+LvAba35IFdhZSTdX6di+Ru0p1LWwfwerbyyTOOnjJUkvTR/cdJllj33lmq31EmSPjF9/1yFFhm0Yfd74lk/iMfhh5SprCz5T0D8aO8TzzTkLK58Q1/MHG2YM+TmdtAPke9i22Na/qi/1nfvY0eroIidB6nwfc6N+FFbRWSspXS/MSskzWhn/uckrep6ONE2auLuKijwTfrukjVJl3HO6f3layVJo/fdI2exRQVt2P2WrdgmSdpvQupBoyYG85a9tU09YeC4rqAvZo42zBlyczvoh8h3q59dp4ZN/hIfTjduH9/n7udiLarZ4new9KkcFnI0nZNuQTtb0ufM7Coz27E1bGbFZvb/5JPmX7oSiJn9xMwu6cq6+WLQiAE7/p3sFIjWeZv98sMHpFymt6INu9+6DS2SpKrhqceEqxpWKEmqrXOqreudBS19MXO0Yc6Qm9tBP0S+WxacbjxkwgDttvfAkKPJb3yfu9/a9/6lbU1bJZmGjZwSdjidktYox5KulfR5SVdI+raZvRlM30fSIElPS/pV9sKLlrKK1jO6mhq2pVyuqd7vhSuv7LVngKVEG3a/uno/ul1ZWeprcPuUt+7rqq2LqW9l7zv9ib6YOdowZ8jN7aAfIp81Vjfpnaf90cR9vsjR2Y7wfe5etTXr9M7yRyVJVaM/rYq+PfAIrXOuWX5P7w8lrZH0ieDxvqQfSDraOZe6d7VhZleY2Qoze0bS3sG0cWb2qJm9ZGZPm9k+6cQIAEBvQm4GomvFo+8p1hxTQZFp7y+MDjsc9GJNjTVa+vLtisWaVdlvpMbu/fmwQ+q0dI/QxhPnL4JHl5nZFEmnSpoUxPGypJck3SjpfOfcW2Z2sKTrJR2VyWvlSmNd445/l5aXqKG2MelypX38vcVSze/NaMPuV9GnQFuqY2psTH0qcX1DbMe/Kyt639FZib6YDbRh7pCbU6MfIp8te9ifbjz60yPUZxBHEzvC97l7NG+r1+sv3qzGhk0q7zPE37qnsDjssDotzC3VwyU94Jyrd87VSHpQfhTGT0u618wWS/qjpKRjl5vZN81skZktalZ+3Cszfr6+JA2uGpRyucFV/vqITeu3dHdIkUMbdr+qYX4/1tr121Mus/ZDf51tZYX1ytONJfpiNtCGkURuph8iRza9Xa0NS/x1oAwG1Tl8n7Nve3ODXl90s+pr16u0bIAO+OS5KintG3ZYaUl7S9XM9jCzW8xsjZltM7Ojgum7BdOnZRjPFufcpITHxGQLOududM5Ndc5NLVZpBi+ZPe+/+YFiMX9ka8x+uyddxsy0x97+3k+rl76fs9iigjbsfhMn+D1uS1akPgMxPhLyxPGpR0Lu6eiLmaMNc4fcnBr9EPkqPhhUWf8S7fkZ7pfaGXyfs6tl+za9sehW1VavUUlpXx34yfNUVh69gcnSKmjNbE9JiySdLH+j9sL4POfcR5KmSjq3k0/3lKSTzKzczPpKOl5SvaR3zOyU4PXMzA5KJ8YwNdQ2asWityVJk2ckD3ufg8erckCFJOmVhW/kLLaooA2735GH+nuLPfN8oxobY0mXWfDPeknSUYeV5yyufENfzBxtmBvk5vbRD5GPYi0xLf+7vzXK+GNGq7C4sIM1IPF9zqaWlma98dJtqtmyWkXFfXTAtHNVXjEk7LC6JN0jtFdLiknaX9JXJbUdJvXvkg7rzBM5516WdI+kVyXNk/RiMOurks4xs1flE/OJacYYqsdnPy1JOur0w5IOFX7KxcdLklYsWqU1K9bmMrTIoA2718wvVKi01LSlOqab/1Kzy/yH/lGn5auaZSadOjNap5xkG30xc7RhTpCbO0A/RL55//kPVfdRgyRp4vFjwg0mYvg+Zy4W266lL9+u6k2rVFRUrgOmnauKvsPDDqvL0i1oj5Z0vXPufUnJRpRZLSn58f8knHNXO+cmOOcOc86d7pz7pXPuHefcsc65g5xz+zrnfppmjKF65I8LtP7dDaro10dXPXS5Rk30zVFeWaZzZ52hw08+RJJ0yxVduiVgr0Abdt7HG1t2PDZvaT3auqUmttO8WKz16zp8aJEuOLe/JOmyqzbqjntr1NLi5/99YZ3OuehDSdKpJ1XqwH3z45TBsNAXM0cb5gS5uQP0w+zYtr1+x6O5pXWwne2xxp3mOdc771+ejjeDwaAGje2nYfsNDjmaaOH7nBnnYnpz8Wxt/niFCgtLtf/Us9S3/8iww8qIpfOjY2aNkr7rnLvJzAZL+kj+dgCPB/MvlPQz51xFt0SbQj8b5A626bl8yXaNPXC0frHgSvUf0k+SVFddr7LKMhUWFigWi+mWK2brnllzwg0yz0WxDeevXZzz1ywcsbJTy616YbTG7NE6Wl1zs9PMs9Zp3kJ/anFpqamwQKpv8L8H0yaV6rF7R+Z8QKhjqibl9PU6I4p9Md9EsQ0XuPtecs5NDTuOziA3d04U+2HhhHFhh7CT+ctndWq5z4w9X+XF/bs5ms6ZMHt12CHsYltts27+3Bxtb2zRpy84SFPOTHpJet5YNiX1IJJhieL3eduxmQxlkD1bNr2t157/oySpoKBIhUWpR9cuLR+gyZ/+Xq5C69BT8y5LmpvTvW3P+5L2a2f+IZI6t4Xdg7392mqdd8DFOu3ymTr4uMkaMnKQajZu1fIXVur+3zysVx7nfP6O0Ibdq7jY9ODtI/SnO2t0+19rtHRFs1panCbtX6JTT+qrC88boJKStmct9k70xczRht2O3NwJ9EPki7cee0/bG1tkBaa9jxsTdjiRxPc5AwkHM2Ox7Yptq025aFRu3ZPuEdpZks6XdKikdfJ7gac7554ws5Plr7v5sXPu6u4INpV82wuM3imMI7Q9TT4eoUXvFLEjtOTmHirfjtBGUT4eoY2afDxCG0X5coQ2ylIdoe3KoFBrJD0v6U75a3V+aGb/kvRX+UEkfpVhrAAAoPPIzQCAXiutgja4yfqnJN0kfxsAkzRD0t6Srpd0pHOuMfUzAACAbCI3AwB6s3SvoY0nzgslXWhmu8knzo8cQ9oBABAKcjMAoLdKu6BNFNywHQAA5AlyMwCgN+lSQWtmX5E0U9LYYNLbkh5wzv01W4EBAIDOIzcDAHqjtApaM6uQNEfSUfKnM20JZk2T9BUz+5akE5xzdVmMEQAApEBuBgD0Zl0Z5Xi6pN9JqnLODXLODZJUFUw7MlgGAADkBrkZANBrpVvQ/puke51zFznn1scnOufWO+cukvS3YBkAAJAb5GYAQK+VbkHbT9IT7cx/PFgGAADkBrkZANBrpVvQviZpfDvzx0t6vevhAACANJGbAQC9VroF7X9JOs/Mjm87w8xOlHSupP/MRmAAAKBTyM0AgF4r3dv2fFXSO5LmmNlyScuC6RMl7S2/B/gMMzsjYR3nnDsn40gBAEAy5GYAQK+VbkF7ZsK/9wkeiQ4MHomcJJImAADd48yEf5ObAQC9SloFrXMu3VOUAQBANyI3AwB6M5IgAAAAACCS0ipozeyoTizz/a6HAwAA0kFuBgD0ZukeoX3MzP7bzHZZz8yGmNnfJf0yO6EBAIBOIDcDAHqtdAvau+WH/n/KzPaITzSzo+Xvgzdd0mXZCw8AAHSA3AwA6LXSKmidc1+Vv5/dJEmvmtmpZjZL0nxJ9ZIOc86xFxgAgBwhNwMAerN0b9sj59wtZvaspPsk3RVMni3p2865rdkMDgAAdIzcDADorbo6yvF+kkZKikkySUMklWYrKAAAkDZyMwCg10l3lONSM/tfSfdKels+eV4i6Qj505ymZz1CAACQErkZANCbpXuEdpGkb0n6vaRPO+dWOOd+LelQ+et05pvZz7IcIwAASI3cDADotdItaEdIOsE5d6Fzblt8onPuJfnBKGaLkRQBAMglcjMAoNdKd1Cog5xzHySb4Zyrk/Q1M/tH5mEBAIBOIjcDAHqtdG/bsyNhBtfsjDSzkjbL3JGt4AAAQPvIzQCA3iztUY7NbLKZPS5pq6T3JB0WTB9qZguDG7kDAIAcITcDAHqrdEc5niTpaUnjJN2eOM85t0FSuaRvZCs4AADQPnIzAKA3S/cI7U8lrZW/JcAP5e9zl2ihpE9mIS4AANA55GYAQK+VbkF7uKQ/OedqJbkk89+TVJVxVAAAoLPIzQCAXivdgrZMUnU78/tlEAsAAEgfuRkA0GulW9CukjSlnflHSVra9XAAAECayM0AgF4r3fvQ/kXSj8zsr5JeCaY5STKziyUdK+nC7IUHRMcxVZPCDiHy5q9dHHYIPQJ9sdchNwMpLJuyPewQIo/cnB3HcOFHt0m3oP2lpBmS5kt6Uz5hXmtmu0kaLukxSddnNUIAANAecjMAoNdK65Rj59w2+aR5iaQGSY2SJkj6WNIPJH3RORfLdpAAACA5cjMAoDdL9witnHPbJV0bPAAAQMjIzQCA3irdQaEAAAAAAMgLFLQAAAAAgEiioAUAAAAARBIFLQAAAAAgkihoAQAAAACRREELAAAAAIgkCloAAAAAQCRR0AIAAAAAIomCFgAAAAAQSRS0AAAAAIBIoqAFAAAAAEQSBS0AAAAAIJIoaAEAAAAAkURBCwAAAACIJApaAAAAAEAkUdACAAAAACKJghYAAAAAEEkUtAAAAACASKKgBQAAAABEEgUtAAAAACCSKGgBAAAAAJFEQQsAAAAAiCQKWgAAAABAJFHQAgAAAAAiiYIWAAAAABBJFLQAAAAAgEgqCjuAnmrgsAE67fKZOvi4yRoycpDqquv15gsr9cB1j+iVx98IO7xIoA2zg3ZMbWttTE8826BFixu16NUmLVrcqI2bY5KkJU+N0j7jS9pdPxZzuumuGv35nhote6tZLS1Oe+1ZrNNm9tUF5w5QSYnl4m1EAv0Q+YB+2HXbY03aVP+eqhvXqaZxvaob16u5pUGSdOiYc1VZOjjkCKOFvpgZ55zunlOrW++u0SuvN6m2LqYRQ4s0/TN9dOl3BmjCuPbzN7ye0g/NORd2DBnrZ4PcwTY97DB22POAUbpm4Y/Vf0g/SVJddb3KKstUWFigWCymW66YrXtmzQk3yDxHG2ZH1Npx/trFOX29OfNqdfLZ65PO66igbW52mnnWOs1bWC9JKimRCgtMDY3+N3XapFItuG+kKityfyLMMVWTcv6a7YlaP5SkBe6+l5xzU8OOI8rIzZkrnDAu7BB2+HDrCi1e+0DSeflc0LasWBV2CLuIWl/MdW7uyLZtTqd+a73mPlonSSoqkvpWFmjzFr9DurzM9JcbhuuEYyrCDHMX5ObMpcrNnHKcZSVlJfrp3MvUf0g/vfXy2zp3/+/rpIHf0JcGnal7f/WgCgoKdPbVp2nKjAPDDjVv0YbZQTt2ztAhhfr89D668uKBuuGa3Tq93o9mbdS8hfUqKzPdct1Q1b49TlvfHqu5t4/QoIEFenFxk86/dEM3Rh4N9EPkA/phdpQU9tGQirEaN/hQ7TvsmLDDiST6YuYuv/pjzX20TkVF0m+uGqItK8bq42VjtfqlMfrScRVqaHQ6/fz1WvVuc9ih5q2e1g8paLPsuG8dreFjhqp+a4N+dMIsrV66RpJUv7VBN156h5554AXfSX721ZAjzV+0YXbQjh07/nMVWvf6nnr4zir9+JLBmvHZPp1ab/2G7frtTdWSpJ9fMVjf+Eo/FRaazExfnFGhm349VJJ095xavba0qdvijwL6IfIB/TBzQyv30pF7fU9Tdj9Few05TEMq9gw7pEiiL2Zmw8fbdf1tPv9e8u2B+t65A1Re7suZ3auKNPuG4ZowrlgNjU4//sXGMEPNaz2tH1LQZtn00w+XJD0++xltXLtpl/n3/nKuJGnClLHafUJVTmOLCtowO2jHjhUWdu0a1/sfqVVTk1P/fgU674x+u8w/8dhKTRhXLOek2Q9szTTMSKMfIh/QDzNnxiZjNtAXM/P4Mw3ats3/+8Jv9t9lflGR6btn++kPzKtTbV0sl+FFRk/rh/w6ZVF5ZZnGTxkrSXpp/uKkyyx77i3VbvHn/H9i+v65Ci0yaMPsoB271xPP+oFQDj+kTGVlyX9G40d7n3imIWdx5Rv6IfIB/RD5gr6YuffWbJckDehfoKFDko9tu/defvyLxkanZ57vvTk4lZ7YDylos2jUxN1VUOCb9N0la5Iu45zT+8vXSpJG77tHzmKLCtowO2jH7rVshd89vN+E1INGTQzmLXtrm3rC4HtdQT9EPqAfIl/QFzNnwYlVLS2p8+r27a3zlgb5Gq16Yj/Mq4LWzC4ws2VmdlfYsXTFoBEDdvw72eH71nmb/fLDB6RcpreiDbODduxe6za0SJKqhqe+81nVsEJJUm2dU21d7yxo6Yc9A7kZyA76YuZG7e7z7tZapzVrtyddZllCEbvuw5acxBUlPbEf5lVBK+k7kmY453ZcgWxmkblXbllF2Y5/NzWk3iPUVO8HiSmvLEu5TG9FG2YH7di96ur9NTllZamvwe1T3vrz2luv4aEf9hjkZiAL6IuZO+LT5Sou9v/+5fWbd5nf2BjT726u3vH/rbW9M/+2pyf2w7wpaM3sBkljJc0zs2ozu8PMnpV0R8ihAQDQK5GbAeSTYbsV6Ztf84M+/f6Wal05a6M+WLddzc1OL7/WqOO/tk6r12xXUbDLrSBvKh10p7z5mJ1z50taK+lISddK2lfS0c6500INLA2NdY07/l1anvrautI+pZKkhtrGlMv0VrRhdtCO3auij//pbGxMfSpxfUPrXuHKirz5qc0p+mH0kZuB7KEvZscvfjRYxx7VR85JV/9ms0ZNfldlo1Zp2jFr9PgzDfrpDwZpYH9/2c+Afr0z/7anJ/bDfP6UH3TOpRyazMy+aWaLzGxRs/LjPo/xc80laXDVoJTLDa4aKEnatH5Ld4cUObRhdtCO3atqmN/1u3Z98ut3JGltcN1OZYWpb2U+/9R2H/phj0RuBrqIvpgdZWUFeuiOEbrz+mE6bkYfjRtTrHFjinX8MRWaN7tKP/juQG2p8Tl4r7GpC7beqif2w3zeyqprb6Zz7kbn3FTn3NRileYqpna9/+YHisX8UZkx++2edBkz0x57+/s5rV76fs5iiwraMDtox+41cYK/gGdJO6MnxgelmDi+9yZT+mGPRG4Guoi+mD0FBabTZvbVg7dXacW/RmvFv0Zrzm0j9Lkj+uj1ZU1qbvbLfWpK/l//mWs9sR/mc0EbOQ21jVqx6G1J0uQZByVdZp+Dx6tyQIUk6ZWFb+QstqigDbODduxeRx7q7zH7zPONamxMPuDEgn/WS5KOOqw8Z3HlG/oh8gH9EPmCvpgbd8+plSRN2r9kxy300Kon9kMK2ix7fPbTkqSjTj8s6TDXp1x8vCRpxaJVWrNibS5DiwzaMDtox+4z8wsVKi01bamO6ea/1Owy/6F/1Gn5qmaZSafO7BtChPmDfoh8QD9EvqAvdq9XlzTpD7f6UY4v+97AkKPJXz2tH+ZVQeucG+Oc+9g59xPn3C/DjqcrHvnjAq1/d4Mq+vXRVQ9drlET/aH88soynTvrDB1+8iGSpFuu+EuYYeY12jA7aMfO+Xhjy47H5i2tR1u31MR2mheLtQ4ANXxokS4414+yeNlVG3XHvTU7bvL+94V1OueiDyVJp55UqQP3zY/TLsNCP4w+cjPitm2v3/FobmkdKGZ7rHGnec71zntvdwZ9MXNPPFuvX9+wWSvf2bYj91bXtOiPt1fr6FM+UGOj0yknVOorJ/TuHcrt6Wn90HrCj04/G+QOtulhh7HD2ANH6xcLrlT/If0kSXXV9SqrLFNhYYFisZhuuWK27pk1J9wg8xxtmB1Ra8f5axfn/DULR6zs1HKrXhitMXsU7/h/c7PTzLPWad5Cf2pxaampsECqb/C/qdMmleqxe0eGMiDUMVWTcv6a7YlaP5SkBe6+l5xzU8OOI8rIzZkrnDAu7BB2Mn/5rE4t95mx56u8uH83R9M5LStWhR3CLqLWF8PIze257Z4anXPRBklSUZHUt7JAW6pjipc0p86s1G3XDVNxcep7xYeB3Jy5VLmZgrabDBw2QKddPlMHHzdZQ0YOUl1Ng5a/sFL3/+ZhvfJ4/p+Lng9ow+yIUjtGqaCVpFjM6U931uj2v9Zo6YpmtbQ4jR9brFNP6qsLzxugkpJwkmm+JU0pWv1QoqDNBnJz5ihoM5ePBa0Urb6YbwXtyne26fpbq/X0c41avaZZW+tiGjqkSIdMKdPZp/XVMUdWhB1iUuTmzFHQAsh7+ZY0oyofk2bUUNBmjtycuXwraKMoXwvaKCE3Zwe5OXOpcnNeXUMLAAAAAEBnUdACAAAAACKJghYAAAAAEEkUtAAAAACASKKgBQAAAABEEgUtAAAAACCSKGgBAAAAAJFEQQsAAAAAiCQKWgAAAABAJFHQAgAAAAAiiYIWAAAAABBJFLQAAAAAgEiioAUAAAAARBIFLQAAAAAgkihoAQAAAACRREELAAAAAIgkCloAAAAAQCRR0AIAAAAAIomCFgAAAAAQSRS0AAAAAIBIoqAFAAAAAEQSBS0AAAAAIJIoaAEAAAAAkURBCwAAAACIJApaAAAAAEAkUdACAAAAACKJghYAAAAAEEkUtAAAAACASKKgBQAAAABEEgUtAAAAACCSKGgBAAAAAJFEQQsAAAAAiCQKWgAAAABAJBWFHQAAxB1TNSnsEHqE+WsXhx1C5BWOCDsCQGoYOyjsECKvZMWqsEOIvC8ccXLYIfQIE19aHXYIkbdgcvLpHKEFAAAAAEQSBS0AAAAAIJIoaAEAAAAAkURBCwAAAACIJApaAAAAAEAkUdACAAAAACKJghYAAAAAEEkUtAAAAACASKKgBQAAAABEEgUtAAAAACCSKGgBAAAAAJFEQQsAAAAAiCQKWgAAAABAJFHQAgAAAAAiiYIWAAAAABBJFLQAAAAAgEiioAUAAAAARBIFLQAAAAAgkihoAQAAAACRREELAAAAAIgkCloAAAAAQCRR0AIAAAAAIomCFgAAAAAQSRS0AAAAAIBIoqAFAAAAAEQSBS0AAAAAIJIoaAEAAAAAkURBCwAAAACIpKKwA+ipBg4boNMun6mDj5usISMHqa66Xm++sFIPXPeIXnn8jbDDiwTaMDtox8zRhqltrY3piWcbtGhxoxa92qRFixu1cXNMkrTkqVHaZ3xJu+vHYk433VWjP99To2VvNaulxWmvPYt12sy+uuDcASopsVy8DfQSfJcz09iwWR+vf0NbNq5S3da12tZUq4KCQpX1GaxBu+2tqtGHqrSsX9hhRgJ9seu2x5q0qf49VTeuU03jelU3rldzS4Mk6dAx56qydHDIEea3302+u9PLfulPR2nklKHdGE12mHMu7Bgy1s8GuYNtethh7LDnAaN0zcIfq/8Q/6NeV12vssoyFRYWKBaL6ZYrZuueWXPCDTLP0YbZQTtmLoptOH/t4py91px5tTr57PVJ53VU0DY3O808a53mLayXJJWUSIUFpoZGn5emTSrVgvtGqrIi9ycTFY5Y+ZJzbmrOX7gHITdnbtux08IOYYfGhi164cmfS2rdbiwsKlVLS7Pk/E6souJy7fuJr2nA4HEhRbmrkkdfDDuEXUStLxZOyJ/PU5I+3LpCi9c+kHRePhe0E2avDjsESdLNM+a0O39bXbO2N7aooLhAZ88/UeUDSnMTWCf8bvLdSXMzR2izrKSsRD+de5n6D+mnt15+W7O+/jutXrpGffqW64wrv6xTLj5BZ199mla+/LZeeuy1sMPNS7RhdtCOmaMNO2fokEJNOahU0yaVqmp4kc6/9KNOrfejWRs1b2G9yspM18/aTWec3FcFBdIjC+p11oUf6sXFTTr/0g268/rh3fwO0NPxXc6CoGgdtNs+GjZyigYM2UvFxX0Ui23Xlo0rtXLJXDU2bNKSl/+saZ+5VCWlfUMOOD/RF7OjpLCP+pUNV/+yESotqtTSD+eHHVJknPPYSe3On33qo/p4xRbteXhVXhWz7eEa2iw77ltHa/iYoarf2qAfnTBLq5eukSTVb23QjZfeoWceeEEFBQU6+2dfDTnS/EUbZgftmDnasGPHf65C617fUw/fWaUfXzJYMz7bp1Prrd+wXb+9qVqS9PMrBusbX+mnwkKTmemLMyp006/9KU53z6nVa0ubui1+9A58lzNXVFyuyYdeoP2nnqXdRhyo4mL/XS8oKNKg3fbR/lPPUkFBkVq2N2nde8+FHG3+oi9mbmjlXjpyr+9pyu6naK8hh2lIxZ5hh9RjfLR8sz5esUWStM/x0WlXCtosm3764ZKkx2c/o41rN+0y/95fzpUkTZgyVrtPqMppbFFBG2YH7Zg52rBjhYVdu8b1/kdq1dTk1L9fgc47Y9dr7k48tlITxhXLOWn2A1szDRO9HN/lzBUVl6uyX+q26VM5VH0HjJIkba35IFdhRQ59MXNmlC/d5c2H3pEklQ8q1ZhDR4QcTefRI7KovLJM46eMlSS9NH9x0mWWPfeWarfUSZI+MX3/XIUWGbRhdtCOmaMNu9cTz/oBPA4/pExlZclTUfxo7xPPNOQsLvQ8fJdzJ37UVj1gfJbuQF9EPottj2n5o/46372PHa2CouiUidGJNAJGTdxdBQW+Sd9dsibpMs45vb98rSRp9L575Cy2qKANs4N2zBxt2L2WrdgmSdpvQupBoyYG85a9tU09YQBDhIPvcm64WItqtviN4T6Vw0KOJj/RF5HPVj+7Tg2b/CU+UTrdWMrDgtbM/i/4O8bMTg87nnQMGjFgx7+TnUbSOm+zX374gJTL9Fa0YXbQjpmjDbvXug0tkqSq4anHJqwaVihJqq1zqq2joA0TuRkdWfvev7Staask07CRU8IOJy/RF5HPlgWnGw+ZMEC77T0w5GjSk3cFrXPu08E/x0iKVNIsqyjb8e+mhm0pl2uq93s/yivLUi7TW9GG2UE7Zo427F519X7E1LKy1Nfg9ilvTVG1dbFujwmpkZvRntqadXpn+aOSpKrRn1ZFX47QJkNfRL5qrG7SO0/7MwP2+WK0js5KeVjQmllt8M+fSzrczBab2ffDjAkAgN6M3IxUmhprtPTl2xWLNauy30iN3fvzYYcEIE0rHn1PseaYCopMe39hdNjhpC3vCtoEP5T0tHNuknPu2rCD6YzGusYd/y4tT31dWGkff0+nhtrGlMv0VrRhdtCOmaMNu1dFH59+GhtTn0pc39B6VLayIp/TVa9CbsYOzdvq9fqLN6uxYZPK+wzxt+4pLA47rLxFX0S+WvawP9149KdHqM+g6J0ZENktBDP7ppktMrNFzcqPexTGr3mQpMFVg1IuN7jKn5e+af2W7g4pcmjD7KAdM0cbdq+qYf7a2bXrt6dcZu2H/jrbygpT38rIpqtehdzce2xvbtDri25Wfe16lZYN0AGfPFclpX3DDiuv0ReRjza9Xa0NS/w13VEbDCouslsIzrkbnXNTnXNTi1UadjiSpPff/ECxmD+iMGa/3ZMuY2baY29/X7HVS9/PWWxRQRtmB+2YOdqwe02c4I/iLFmR+jqy+EjIE8enPpKB/EJu7h1atm/TG4tuVW31GpWU9tWBnzxPZeXRGkQmDPRF5KP4YFBl/Uu052eiee/jfC5ot0qK1K6+htpGrVj0tiRp8oyDki6zz8HjVTmgQpL0ysI3chZbVNCG2UE7Zo427F5HHurvV/nM841qbEw+4NOCf9ZLko46rDxncaFD5OZerqWlWW+8dJtqtqxWUXEfHTDtXJVXDAk7rEigLyLfxFpiWv53f7ut8ceMVmFxYcgRdU0+F7SvSWoxs1ejNPDE47OfliQddfphSYdbP+Xi4yVJKxat0poVa3MZWmTQhtlBO2aONuw+M79QodJS05bqmG7+S80u8x/6R52Wr2qWmXTqzEjVTz0dubkXi8W2a+nLt6t60yoVFZXrgGnnqqLv8LDDihT6IvLJ+89/qLqPGiRJE48fE24wGci7gtY5Vxn8bXbOHeWcOygqA09I0iN/XKD1725QRb8+uuqhyzVqoj+lpLyyTOfOOkOHn3yIJOmWK/4SZph5jTbMDtoxc7Rh53y8sWXHY/OW1qOtW2piO82LxVoHgBo+tEgXnNtfknTZVRt1x701amnx8/++sE7nXPShJOnUkyp14L75cepqb0ZuhnMxvbl4tjZ/vEKFhaXaf+pZ6tt/ZNhhRQ59MTu2ba/f8WhuaR08a3uscad5znEP8/a8GQwGNWhsPw3bb3DI0XSd9YQPup8Ncgfb9LDD2GHsgaP1iwVXqv+QfpKkuup6lVWWqbCwQLFYTLdcMVv3zJoTbpB5jjbMDtoxc1Fsw/lrF+f09QpHrOzUcqteGK0xe7SOgNrc7DTzrHWat9CfWlxaaioskOobfF6aNqlUj907MpQBoQpHrHzJOTc15y/cg5CbM7ft2Glhh7DDlk1v67Xn/yhJKigoUmFR6pFQS8sHaPKnv5er0NpV8uiLYYewi6j1xcIJ48IOYRfzl8/q1HKfGXu+yov7d3M0nTNh9uqwQ9jJttpm3fy5Odre2KJPX3CQppw5MeyQOvS7yXcnzc1FYQTT07392mqdd8DFOu3ymTr4uMkaMnKQajZu1fIXVur+3zysVx7nmoiO0IbZQTtmjjbsPsXFpgdvH6E/3Vmj2/9ao6UrmtXS4jRp/xKdelJfXXjeAJWUWNhhoofgu5yhhAMgsdh2xbbVplyUW/e0j76IfPDWY+9pe2OLrMC093Fjwg4nIxyhBYAeJtdHaHsijtBmjtycuXw6QhtV+XiENmry8QhtFOXbEdooSnWENu+uoQUAAAAAoDMoaAEAAAAAkURBCwAAAACIJApaAAAAAEAkUdACAAAAACKJghYAAAAAEEkUtAAAAACASKKgBQAAAABEEgUtAAAAACCSKGgBAAAAAJFEQQsAAAAAiCQKWgAAAABAJFHQAgAAAAAiiYIWAAAAABBJFLQAAAAAgEiioAUAAAAARBIFLQAAAAAgkihoAQAAAACRREELAAAAAIgkCloAAAAAQCRR0AIAAAAAIomCFgAAAAAQSRS0AAAAAIBIoqAFAAAAAEQSBS0AAAAAIJIoaAEAAAAAkURBCwAAAACIJApaAAAAAEAkUdACAAAAACKJghYAAAAAEEkUtAAAAACASKKgBQAAAABEEgUtAAAAACCSzDkXdgwZM7OPJK0OO452DJH0cdhBRBxtmB20Y+Zow8xFoQ1HO+d2CzuIKCM39wq0YXbQjpmjDTMXhTZMmpt7REGb78xskXNuathxRBltmB20Y+Zow8zRhsgH9MPM0YbZQTtmjjbMXJTbkFOOAQAAAACRREELAAAAAIgkCtrcuDHsAHoA2jA7aMfM0YaZow2RD+iHmaMNs4N2zBxtmLnItiHX0AIAAAAAIokjtAAAAACASKKgzTEze9LMIjmCGKLLzH5iZpeEHUdUmdkFZrbMzO4KO5YoM7P/C/6OMbPTw44HiCM3Iwzk5syQm7OjJ+RmCloA6Nh3JM1wzn01PsHMikKMJ5Kcc58O/jlGUiSTJgAgb5Cbs6An5GYK2m4S7OV408zuCvYe3WdmfcKOKyrM7Awze8HMFpvZH82s0MxqzexqM3vVzJ4zs2Fhx5nPzOwKM1thZs9I2juYNs7MHjWzl8zsaTPbJ+Qw856Z3SBprKR5ZlZtZneY2bOS7gg5tMgxs9rgnz+XdHjw/f5+mDGhdyE3Z4bcnDlyc3aQm7OnJ+RmCtrutbek651zEyXVyO9JQgfMbKKkf5N0qHNukqQWSV+VVCHpOefcQZKeknReaEHmOTObIulUSZMkfUHStGDWjZK+55ybIukSSdeHEmCEOOfOl7RW0pGSrpW0r6SjnXOnhRpYtP1Q0tPOuUnOuWvDDga9Drm5C8jNmSM3Zw+5uVtENjdzWL57ve+cezb4952SLggzmAiZLmmKpBfNTJLKJW2QtE3Sw8EyL0maEUp00XC4pAecc/WSZGYPSiqT9GlJ9wbtKkml4YQXaQ865xrCDgJAl5Gbu4bcnDlyc/chN/diFLTdq+09kbhHUueYpD875y7faaLZJa71PlMtov+mq0DSlmDPOrquLuwAAGSE3Nw15ObuQW7ODnJzL8Ypx91rlJl9Kvj36ZKeCTOYCFko6ctmNlSSzGyQmY0OOaaoeUrSSWZWbmZ9JR0vqV7SO2Z2iiSZd1CYQaLX2iqpb9hBoNciN3cNuTlz5Gbks8jmZgra7rVc0r+b2TJJAyX9b8jxRIJzbqmk/5L0DzN7TdJjkkaEG1W0OOdelnSPpFclzZP0YjDrq5LOMbNXJS2RdGI4EaKXe01SSzCITKQGnkCPQG7uAnJz5sjNyHORzc3WepYIssnMxkh62Dm3f9ixAAAAcjMA9EQcoQUAAAAARBJHaAEAAAAAkcQRWgAAAABAJFHQAgAAAAAiiYIWAAAAABBJFLRADpnZu2b2ZNhx5AszO8LMnJmdGXYsAIDeidy8M3IzooaCFogQM7solwkm168HAEDUkJuBcFHQAtFykaQze/DrAQAQNReJ3AyEhoIWAAAAABBJFLRANzCzPczsr2ZWbWY1ZvaQmY1rZ/mjzewfZrbFzBrN7DUzO7/NMk7SaEmfDa5tiT/GJCwz1cweMLOPzazJzJab2RVmVpTkNfcys1vNbI2ZbTOztWY218ymdNPrnWhmrwTv730zu0pScdqNCwBAF5Cbyc3omXbp2AAyY2YDJD0laQ9JN0haKumzkp6QVJ5k+W8Gyz0n6WpJdZJmSPpfMxvnnLs0WPRrkq6V9HGwXNxHwfMcJ+l+SSsl/UrSJkmfkvRTSZMknZLwmlMlLZRPWjdLekPSoCDOT0t6KcuvN1PS3yS9G8zfLuksScelaEYAALKG3ExuRg/mnOPBg0cWH5J+JslJOqvN9N8E059MmDZCUqOkvyR5nusktUgamzDt3cT1E6aXSVovn6yL2sz7fvC6RwT/N/kk2SjpwCTPVZDl1yuU9J588h2SsFx/SauDZc8M+3PjwYMHDx4990FuJjfz6LkPTjkGsu8kSR9Kur3N9FlJlv2ypFJJN5vZkMSHpIfkLws4uhOvOUPSMEm3ShrQ5nn+HizzueDvJEn7SbrVOfda2ydyzsWy/HpT5PeI3+qc+zjhdarl934DANDdThK5OfH1yM3oMTjlGMi+sZJedM61JE50zq0zsy1tlp0Y/F3QzvMN68Rrxp/nlk48z/jg7yudeN5svN7Y4O+bSZZZmkEMAAB0Frl559cjN6PHoKAFwmXB369LWpdimbfTeJ5LJS1OsczazoeVd68HAECukJuBCKGgBbLvbUnjzawwcU+wmY2QNKDNsm8Ffz92zrW3JzjOpZgef566TjzPiuDvpBy9Xjzp75Nk3r6diAEAgEyRm3dGbkaPwTW0QPbNlT+l5+ttpl+WZNm/SmqS9P/MLNkoi/3NrDRhUq38iIdtzZe0QdIPzWyX+WZWbmZ9g/++KmmJpLPNbL8ky1rCf7Pxei9JWiPprOA6nvgy/SSd33ZdAAC6AbmZ3IweypxLtZMHQFeY2UD5U31Gyg+ssETSEfLD5pdLWuKcOyJh+bMk3STpfUl3yI8uuJukA+QHsdjXOfdusOyfJJ0jP1T/MkkxSQ855+rM7BhJc+QT3S3yQ/YPkN/7+iVJM51zTwbPM03+1gCFar01wAD5WwM86pz7XZZf78vyGwjvSvqT/K0BzpbUIOkT8qNO3pZWQwMA0EnkZnIzei4KWqAbmNkoSb9W62iC/5R0kXyiejcxaQbLHyrpEkmHyieejyUtl/SwpD845xqD5YZK+l9JRwbLmaQ9E5Lq/pJ+GMzfTdJmSaskzQueZ1PCa+4t6UfyIzUOCl7zBUk/dc693A2v9yVJV8oPWrFB0m3ytxb4h0iaAIBuRm4mN6NnoqAFAAAAAEQS19ACAAAAACKJghYAAAAAEEkUtAAAAACASKKgBQAAAABEEgUtAAAAACCSKGgBAAAAAJFEQQsAAAAAiCQKWgAAAABAJFHQAgAAAAAiiYIWAAAAABBJ/x/Wn52IXdXBiQAAAABJRU5ErkJggg==\n",
      "text/plain": [
       "<Figure size 1152x576 with 2 Axes>"
      ]
     },
     "metadata": {
      "needs_background": "light"
     },
     "output_type": "display_data"
    }
   ],
   "source": [
    "show_evaluation(langdet, n=10)"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {
    "slideshow": {
     "slide_type": "slide"
    }
   },
   "source": [
    "## Pros and Cons\n",
    "\n",
    "<ul style=\"list-style-type: none; padding-left:1em\">\n",
    "    <li>&check; Very simple idea and implementation</li>\n",
    "    <li>&check; Low memory requirement of the trained model (only n-gram rankings)</li>\n",
    "    <li>&check; Fast trainable (no gradients optimization)</li>\n",
    "    <li>&check; High tolerance for errors</li>\n",
    "    <li>&cross; Poorly handles short and mixed texts</li>\n",
    "</ul>"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {
    "slideshow": {
     "slide_type": "slide"
    }
   },
   "source": [
    "## Don't try this at work - Use nltk\n",
    "The same n-gram based method has been implemented in nltk package."
   ]
  },
  {
   "cell_type": "code",
   "execution_count": null,
   "metadata": {
    "scrolled": true
   },
   "outputs": [],
   "source": [
    "!pip install nltk"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "Before the first run we must download additional language resources."
   ]
  },
  {
   "cell_type": "code",
   "execution_count": null,
   "metadata": {},
   "outputs": [],
   "source": [
    "import nltk\n",
    "nltk.download('crubadan');"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {
    "slideshow": {
     "slide_type": "subslide"
    }
   },
   "source": [
    "## Don't try this at work - Use nltk"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 22,
   "metadata": {},
   "outputs": [
    {
     "data": {
      "text/plain": [
       "('pol', 'eng')"
      ]
     },
     "execution_count": 22,
     "metadata": {},
     "output_type": "execute_result"
    }
   ],
   "source": [
    "from nltk.classify.textcat import TextCat\n",
    "textcat = TextCat()\n",
    "\n",
    "pl = textcat.guess_language(\"PyCon PL 2020 to trzynasta edycja \"\n",
    "    \"ogólnopolskiej konferencji z grupy PyCon.\")\n",
    "\n",
    "en = textcat.guess_language(\"Capybaras are highly social, \"\n",
    "    \"living in groups of up to 100 and communicating through \"\n",
    "    \"a variety of vocalizations.\")\n",
    "\n",
    "(pl, en)"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {
    "slideshow": {
     "slide_type": "slide"
    }
   },
   "source": [
    "## References\n",
    "\n",
    "1. [MediaWiki API Help.](https://www.mediawiki.org/w/api.php?action=help&modules=query%2Bextracts)\n",
    "2. [Wikipedia contributors, \"N-gram,\" Wikipedia, The Free Encyclopedia](https://en.wikipedia.org/wiki/N-gram)\n",
    "3. [Cavnar, W. B. and Trenkle, J. M. 1994. N-gram-based text categorization](https://www.researchgate.net/publication/2375544_N-Gram-Based_Text_Categorization)"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {
    "slideshow": {
     "slide_type": "slide"
    }
   },
   "source": [
    "## Thank you for your attention :)\n",
    "\n",
    "https://github.com/pkubiak/pl.pycon.2020"
   ]
  }
 ],
 "metadata": {
  "celltoolbar": "Slideshow",
  "kernelspec": {
   "display_name": "Python 3",
   "language": "python",
   "name": "python3"
  },
  "language_info": {
   "codemirror_mode": {
    "name": "ipython",
    "version": 3
   },
   "file_extension": ".py",
   "mimetype": "text/x-python",
   "name": "python",
   "nbconvert_exporter": "python",
   "pygments_lexer": "ipython3",
   "version": "3.8.6"
  },
  "rise": {
   "backimage": "https://pl.pycon.org/2020/static/PyConPL20.png",
   "footer": "<span class=\"github\"><a href=\"https://github.com/pkubiak/pl.pycon.2020\">pkubiak / pl.pycon.2020</a></span>",
   "header": "Language detection in 7 minutes"
  }
 },
 "nbformat": 4,
 "nbformat_minor": 2
}