4-pmlb.ipynb

{
 "cells": [
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "# Datasets and Benchmark\n",
    "\n",
    "Since we are going to play a bit with data and datasets - ultimately to show some examples of data analysis and machine learning - \n",
    "**in this notebook** we will present a quite new package in the Python ecosystem, i.e. `pmlb` that automatically provides datasets for data analysis and Machine Learning.\n",
    "\n",
    "\n",
    "### Don't worry\n",
    "\n",
    "If you are an absolute beginner, or you simply don't know what Machine Learning is, **nothing to worry about**!\n",
    "We will talk about it in the next section! :)\n",
    "\n",
    "**In this notebook** we just focus on the `pmlb` package from a technical perspective. \n",
    "\n",
    "If you are completely new to concepts like **Supervised Learning**, **Classification**, or **Regression** come back to this notebook later after the \"Introduction to Machine Learning\" section for a quick recap and cross-check of these concepts !-)\n"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "---"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "## Introducting the Penn Machine Learning Benchmarks\n",
    "\n",
    "**Penn Machine Learning Benchmark** (`PMLB`) is a collection of code and data for a large, curated set of benchmark datasets for evaluating and comparing supervised machine learning algorithms. \n",
    "\n",
    "These data sets cover a broad range of applications, and include _binary/multi-class classification_ problems and _regression problems_, as well as combinations of categorical, ordinal, and continuous features. \n",
    "\n",
    "**There are no missing values in these data sets**.\n",
    "\n",
    "PMLB was developed in the [Computational Genetics Lab](http://epistasis.org/) at the [University of Pennsylvania](https://www.upenn.edu/) with funding from the [NIH](http://www.nih.gov/) under grant R01 AI117694. \n",
    "\n",
    "More information here: [github.com/EpistasisLab/penn-ml-benchmarks](https://github.com/EpistasisLab/penn-ml-benchmarks)"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "## Data set format\n",
    "\n",
    "All data sets are stored in a common format:\n",
    "\n",
    "* First row is the column names\n",
    "* Each following row corresponds to one row of the data\n",
    "* The target column is named `target`\n",
    "* All columns are tab (`\\t`) separated\n",
    "* All files are compressed with `gzip` to conserve space"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "## Python wrapper\n",
    "\n",
    "For easy access to the benchmark data sets, we have provided a Python wrapper named `pmlb`. The wrapper can be installed on Python via `pip`:\n",
    "\n",
    "```\n",
    "pip install pmlb\n",
    "```\n",
    "\n",
    "and used in Python scripts as follows:"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 1,
   "metadata": {},
   "outputs": [
    {
     "name": "stdout",
     "output_type": "stream",
     "text": [
      "IMPORT ERROR: Please install the pmlb package: pip install pmlb\n"
     ]
    }
   ],
   "source": [
    "try: \n",
    "    import plmb\n",
    "except ImportError:\n",
    "    print('IMPORT ERROR: Please install the pmlb package: pip install pmlb')"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 2,
   "metadata": {},
   "outputs": [
    {
     "name": "stdout",
     "output_type": "stream",
     "text": [
      "Collecting pmlb\n",
      "  Using cached https://files.pythonhosted.org/packages/0d/38/bb7bf1785add978af1e6c2bd27926405959c71ebaeefa6ff610bf53d8b25/pmlb-0.3.tar.gz\n",
      "Requirement already satisfied: pandas in /Users/valerio/anaconda3/envs/webvalley19/lib/python3.7/site-packages (from pmlb) (0.24.2)\n",
      "Requirement already satisfied: python-dateutil>=2.5.0 in /Users/valerio/anaconda3/envs/webvalley19/lib/python3.7/site-packages (from pandas->pmlb) (2.8.0)\n",
      "Requirement already satisfied: pytz>=2011k in /Users/valerio/anaconda3/envs/webvalley19/lib/python3.7/site-packages (from pandas->pmlb) (2019.1)\n",
      "Requirement already satisfied: numpy>=1.12.0 in /Users/valerio/anaconda3/envs/webvalley19/lib/python3.7/site-packages (from pandas->pmlb) (1.16.4)\n",
      "Requirement already satisfied: six>=1.5 in /Users/valerio/anaconda3/envs/webvalley19/lib/python3.7/site-packages (from python-dateutil>=2.5.0->pandas->pmlb) (1.12.0)\n",
      "Building wheels for collected packages: pmlb\n",
      "  Building wheel for pmlb (setup.py) ... \u001b[?25ldone\n",
      "\u001b[?25h  Stored in directory: /Users/valerio/Library/Caches/pip/wheels/5a/c7/a4/be59e63a2cb56f6c58f068305b95c212e0aac1a930fd77d6b0\n",
      "Successfully built pmlb\n",
      "Installing collected packages: pmlb\n",
      "Successfully installed pmlb-0.3\n"
     ]
    }
   ],
   "source": [
    "# if you got an ImportError in the previous cell:\n",
    "\n",
    "!pip install pmlb"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "#### Dataset List\n",
    "\n",
    "You can list **all** the available data sets in `pmlb` as it follows:\n"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 3,
   "metadata": {
    "scrolled": true
   },
   "outputs": [
    {
     "name": "stdout",
     "output_type": "stream",
     "text": [
      "All Available Datasets: \n",
      "========================\n",
      "1) GAMETES_Epistasis_2-Way_1000atts_0.4H_EDM-1_EDM-1_1\n",
      "2) GAMETES_Epistasis_2-Way_20atts_0.1H_EDM-1_1\n",
      "3) GAMETES_Epistasis_2-Way_20atts_0.4H_EDM-1_1\n",
      "4) GAMETES_Epistasis_3-Way_20atts_0.2H_EDM-1_1\n",
      "5) GAMETES_Heterogeneity_20atts_1600_Het_0.4_0.2_50_EDM-2_001\n",
      "6) GAMETES_Heterogeneity_20atts_1600_Het_0.4_0.2_75_EDM-2_001\n",
      "7) Hill_Valley_with_noise\n",
      "8) Hill_Valley_without_noise\n",
      "9) adult\n",
      "10) agaricus-lepiota\n",
      "11) allbp\n",
      "12) allhyper\n",
      "13) allhypo\n",
      "14) allrep\n",
      "15) analcatdata_aids\n",
      "16) analcatdata_asbestos\n",
      "17) analcatdata_authorship\n",
      "18) analcatdata_bankruptcy\n",
      "19) analcatdata_boxing1\n",
      "20) analcatdata_boxing2\n",
      "21) analcatdata_creditscore\n",
      "22) analcatdata_cyyoung8092\n",
      "23) analcatdata_cyyoung9302\n",
      "24) analcatdata_dmft\n",
      "25) analcatdata_fraud\n",
      "26) analcatdata_germangss\n",
      "27) analcatdata_happiness\n",
      "28) analcatdata_japansolvent\n",
      "29) analcatdata_lawsuit\n",
      "30) ann-thyroid\n",
      "31) appendicitis\n",
      "32) australian\n",
      "33) auto\n",
      "34) backache\n",
      "35) balance-scale\n",
      "36) banana\n",
      "37) biomed\n",
      "38) breast\n",
      "39) breast-cancer\n",
      "40) breast-cancer-wisconsin\n",
      "41) breast-w\n",
      "42) buggyCrx\n",
      "43) bupa\n",
      "44) calendarDOW\n",
      "45) car\n",
      "46) car-evaluation\n",
      "47) cars\n",
      "48) cars1\n",
      "49) chess\n",
      "50) churn\n",
      "51) clean1\n",
      "52) clean2\n",
      "53) cleve\n",
      "54) cleveland\n",
      "55) cleveland-nominal\n",
      "56) cloud\n",
      "57) cmc\n",
      "58) coil2000\n",
      "59) colic\n",
      "60) collins\n",
      "61) confidence\n",
      "62) connect-4\n",
      "63) contraceptive\n",
      "64) corral\n",
      "65) credit-a\n",
      "66) credit-g\n",
      "67) crx\n",
      "68) dermatology\n",
      "69) diabetes\n",
      "70) dis\n",
      "71) dna\n",
      "72) ecoli\n",
      "73) fars\n",
      "74) flags\n",
      "75) flare\n",
      "76) german\n",
      "77) glass\n",
      "78) glass2\n",
      "79) haberman\n",
      "80) hayes-roth\n",
      "81) heart-c\n",
      "82) heart-h\n",
      "83) heart-statlog\n",
      "84) hepatitis\n",
      "85) horse-colic\n",
      "86) house-votes-84\n",
      "87) hungarian\n",
      "88) hypothyroid\n",
      "89) ionosphere\n",
      "90) iris\n",
      "91) irish\n",
      "92) kddcup\n",
      "93) kr-vs-kp\n",
      "94) krkopt\n",
      "95) labor\n",
      "96) led24\n",
      "97) led7\n",
      "98) letter\n",
      "99) liver-disorder\n",
      "100) lupus\n",
      "101) lymphography\n",
      "102) magic\n",
      "103) mfeat-factors\n",
      "104) mfeat-fourier\n",
      "105) mfeat-karhunen\n",
      "106) mfeat-morphological\n",
      "107) mfeat-pixel\n",
      "108) mfeat-zernike\n",
      "109) mnist\n",
      "110) mofn-3-7-10\n",
      "111) molecular-biology_promoters\n",
      "112) monk1\n",
      "113) monk2\n",
      "114) monk3\n",
      "115) movement_libras\n",
      "116) mushroom\n",
      "117) mux6\n",
      "118) new-thyroid\n",
      "119) nursery\n",
      "120) optdigits\n",
      "121) page-blocks\n",
      "122) parity5\n",
      "123) parity5+5\n",
      "124) pendigits\n",
      "125) phoneme\n",
      "126) pima\n",
      "127) poker\n",
      "128) postoperative-patient-data\n",
      "129) prnn_crabs\n",
      "130) prnn_fglass\n",
      "131) prnn_synth\n",
      "132) profb\n",
      "133) promoters\n",
      "134) ring\n",
      "135) saheart\n",
      "136) satimage\n",
      "137) schizo\n",
      "138) segmentation\n",
      "139) shuttle\n",
      "140) sleep\n",
      "141) solar-flare_1\n",
      "142) solar-flare_2\n",
      "143) sonar\n",
      "144) soybean\n",
      "145) spambase\n",
      "146) spect\n",
      "147) spectf\n",
      "148) splice\n",
      "149) tae\n",
      "150) texture\n",
      "151) threeOf9\n",
      "152) tic-tac-toe\n",
      "153) titanic\n",
      "154) tokyo1\n",
      "155) twonorm\n",
      "156) vehicle\n",
      "157) vote\n",
      "158) vowel\n",
      "159) waveform-21\n",
      "160) waveform-40\n",
      "161) wdbc\n",
      "162) wine-quality-red\n",
      "163) wine-quality-white\n",
      "164) wine-recognition\n",
      "165) xd6\n",
      "166) yeast\n",
      "167) 1027_ESL\n",
      "168) 1028_SWD\n",
      "169) 1029_LEV\n",
      "170) 1030_ERA\n",
      "171) 1089_USCrime\n",
      "172) 1096_FacultySalaries\n",
      "173) 1191_BNG_pbc\n",
      "174) 1193_BNG_lowbwt\n",
      "175) 1196_BNG_pharynx\n",
      "176) 1199_BNG_echoMonths\n",
      "177) 1201_BNG_breastTumor\n",
      "178) 1203_BNG_pwLinear\n",
      "179) 1595_poker\n",
      "180) 192_vineyard\n",
      "181) 195_auto_price\n",
      "182) 197_cpu_act\n",
      "183) 201_pol\n",
      "184) 207_autoPrice\n",
      "185) 210_cloud\n",
      "186) 215_2dplanes\n",
      "187) 218_house_8L\n",
      "188) 225_puma8NH\n",
      "189) 227_cpu_small\n",
      "190) 228_elusage\n",
      "191) 229_pwLinear\n",
      "192) 230_machine_cpu\n",
      "193) 294_satellite_image\n",
      "194) 344_mv\n",
      "195) 4544_GeographicalOriginalofMusic\n",
      "196) 485_analcatdata_vehicle\n",
      "197) 503_wind\n",
      "198) 505_tecator\n",
      "199) 519_vinnie\n",
      "200) 522_pm10\n",
      "201) 523_analcatdata_neavote\n",
      "202) 527_analcatdata_election2000\n",
      "203) 529_pollen\n",
      "204) 537_houses\n",
      "205) 542_pollution\n",
      "206) 547_no2\n",
      "207) 556_analcatdata_apnea2\n",
      "208) 557_analcatdata_apnea1\n",
      "209) 560_bodyfat\n",
      "210) 561_cpu\n",
      "211) 562_cpu_small\n",
      "212) 564_fried\n",
      "213) 573_cpu_act\n",
      "214) 574_house_16H\n",
      "215) 579_fri_c0_250_5\n",
      "216) 581_fri_c3_500_25\n",
      "217) 582_fri_c1_500_25\n",
      "218) 583_fri_c1_1000_50\n",
      "219) 584_fri_c4_500_25\n",
      "220) 586_fri_c3_1000_25\n",
      "221) 588_fri_c4_1000_100\n",
      "222) 589_fri_c2_1000_25\n",
      "223) 590_fri_c0_1000_50\n",
      "224) 591_fri_c1_100_10\n",
      "225) 592_fri_c4_1000_25\n",
      "226) 593_fri_c1_1000_10\n",
      "227) 594_fri_c2_100_5\n",
      "228) 595_fri_c0_1000_10\n",
      "229) 596_fri_c2_250_5\n",
      "230) 597_fri_c2_500_5\n",
      "231) 598_fri_c0_1000_25\n",
      "232) 599_fri_c2_1000_5\n",
      "233) 601_fri_c1_250_5\n",
      "234) 602_fri_c3_250_10\n",
      "235) 603_fri_c0_250_50\n",
      "236) 604_fri_c4_500_10\n",
      "237) 605_fri_c2_250_25\n",
      "238) 606_fri_c2_1000_10\n",
      "239) 607_fri_c4_1000_50\n",
      "240) 608_fri_c3_1000_10\n",
      "241) 609_fri_c0_1000_5\n",
      "242) 611_fri_c3_100_5\n",
      "243) 612_fri_c1_1000_5\n",
      "244) 613_fri_c3_250_5\n",
      "245) 615_fri_c4_250_10\n",
      "246) 616_fri_c4_500_50\n",
      "247) 617_fri_c3_500_5\n",
      "248) 618_fri_c3_1000_50\n",
      "249) 620_fri_c1_1000_25\n",
      "250) 621_fri_c0_100_10\n",
      "251) 622_fri_c2_1000_50\n",
      "252) 623_fri_c4_1000_10\n",
      "253) 624_fri_c0_100_5\n",
      "254) 626_fri_c2_500_50\n",
      "255) 627_fri_c2_500_10\n",
      "256) 628_fri_c3_1000_5\n",
      "257) 631_fri_c1_500_5\n",
      "258) 633_fri_c0_500_25\n",
      "259) 634_fri_c2_100_10\n",
      "260) 635_fri_c0_250_10\n",
      "261) 637_fri_c1_500_50\n",
      "262) 641_fri_c1_500_10\n",
      "263) 643_fri_c2_500_25\n",
      "264) 644_fri_c4_250_25\n",
      "265) 645_fri_c3_500_50\n",
      "266) 646_fri_c3_500_10\n",
      "267) 647_fri_c1_250_10\n",
      "268) 648_fri_c1_250_50\n",
      "269) 649_fri_c0_500_5\n",
      "270) 650_fri_c0_500_50\n",
      "271) 651_fri_c0_100_25\n",
      "272) 653_fri_c0_250_25\n",
      "273) 654_fri_c0_500_10\n",
      "274) 656_fri_c1_100_5\n",
      "275) 657_fri_c2_250_10\n",
      "276) 658_fri_c3_250_25\n",
      "277) 659_sleuth_ex1714\n",
      "278) 663_rabe_266\n",
      "279) 665_sleuth_case2002\n",
      "280) 666_rmftsa_ladata\n",
      "281) 678_visualizing_environmental\n",
      "282) 687_sleuth_ex1605\n",
      "283) 690_visualizing_galaxy\n",
      "284) 695_chatfield_4\n",
      "285) 706_sleuth_case1202\n",
      "286) 712_chscase_geyser1\n"
     ]
    }
   ],
   "source": [
    "from pmlb import dataset_names\n",
    "\n",
    "print('All Available Datasets: ')\n",
    "print('========================')\n",
    "for i, name in enumerate(dataset_names):\n",
    "    print('{}) {}'.format(i+1, name))"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "Moreover, if you are particularly interested in getting the list of available datasets for **Classification** or **Regression**:"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 5,
   "metadata": {
    "scrolled": true
   },
   "outputs": [
    {
     "name": "stdout",
     "output_type": "stream",
     "text": [
      "Classification Datasets: \n",
      "=========================\n",
      "1) GAMETES_Epistasis_2-Way_1000atts_0.4H_EDM-1_EDM-1_1\n",
      "2) GAMETES_Epistasis_2-Way_20atts_0.1H_EDM-1_1\n",
      "3) GAMETES_Epistasis_2-Way_20atts_0.4H_EDM-1_1\n",
      "4) GAMETES_Epistasis_3-Way_20atts_0.2H_EDM-1_1\n",
      "5) GAMETES_Heterogeneity_20atts_1600_Het_0.4_0.2_50_EDM-2_001\n",
      "6) GAMETES_Heterogeneity_20atts_1600_Het_0.4_0.2_75_EDM-2_001\n",
      "7) Hill_Valley_with_noise\n",
      "8) Hill_Valley_without_noise\n",
      "9) adult\n",
      "10) agaricus-lepiota\n",
      "11) allbp\n",
      "12) allhyper\n",
      "13) allhypo\n",
      "14) allrep\n",
      "15) analcatdata_aids\n",
      "16) analcatdata_asbestos\n",
      "17) analcatdata_authorship\n",
      "18) analcatdata_bankruptcy\n",
      "19) analcatdata_boxing1\n",
      "20) analcatdata_boxing2\n",
      "21) analcatdata_creditscore\n",
      "22) analcatdata_cyyoung8092\n",
      "23) analcatdata_cyyoung9302\n",
      "24) analcatdata_dmft\n",
      "25) analcatdata_fraud\n",
      "26) analcatdata_germangss\n",
      "27) analcatdata_happiness\n",
      "28) analcatdata_japansolvent\n",
      "29) analcatdata_lawsuit\n",
      "30) ann-thyroid\n",
      "31) appendicitis\n",
      "32) australian\n",
      "33) auto\n",
      "34) backache\n",
      "35) balance-scale\n",
      "36) banana\n",
      "37) biomed\n",
      "38) breast\n",
      "39) breast-cancer\n",
      "40) breast-cancer-wisconsin\n",
      "41) breast-w\n",
      "42) buggyCrx\n",
      "43) bupa\n",
      "44) calendarDOW\n",
      "45) car\n",
      "46) car-evaluation\n",
      "47) cars\n",
      "48) cars1\n",
      "49) chess\n",
      "50) churn\n",
      "51) clean1\n",
      "52) clean2\n",
      "53) cleve\n",
      "54) cleveland\n",
      "55) cleveland-nominal\n",
      "56) cloud\n",
      "57) cmc\n",
      "58) coil2000\n",
      "59) colic\n",
      "60) collins\n",
      "61) confidence\n",
      "62) connect-4\n",
      "63) contraceptive\n",
      "64) corral\n",
      "65) credit-a\n",
      "66) credit-g\n",
      "67) crx\n",
      "68) dermatology\n",
      "69) diabetes\n",
      "70) dis\n",
      "71) dna\n",
      "72) ecoli\n",
      "73) fars\n",
      "74) flags\n",
      "75) flare\n",
      "76) german\n",
      "77) glass\n",
      "78) glass2\n",
      "79) haberman\n",
      "80) hayes-roth\n",
      "81) heart-c\n",
      "82) heart-h\n",
      "83) heart-statlog\n",
      "84) hepatitis\n",
      "85) horse-colic\n",
      "86) house-votes-84\n",
      "87) hungarian\n",
      "88) hypothyroid\n",
      "89) ionosphere\n",
      "90) iris\n",
      "91) irish\n",
      "92) kddcup\n",
      "93) kr-vs-kp\n",
      "94) krkopt\n",
      "95) labor\n",
      "96) led24\n",
      "97) led7\n",
      "98) letter\n",
      "99) liver-disorder\n",
      "100) lupus\n",
      "101) lymphography\n",
      "102) magic\n",
      "103) mfeat-factors\n",
      "104) mfeat-fourier\n",
      "105) mfeat-karhunen\n",
      "106) mfeat-morphological\n",
      "107) mfeat-pixel\n",
      "108) mfeat-zernike\n",
      "109) mnist\n",
      "110) mofn-3-7-10\n",
      "111) molecular-biology_promoters\n",
      "112) monk1\n",
      "113) monk2\n",
      "114) monk3\n",
      "115) movement_libras\n",
      "116) mushroom\n",
      "117) mux6\n",
      "118) new-thyroid\n",
      "119) nursery\n",
      "120) optdigits\n",
      "121) page-blocks\n",
      "122) parity5\n",
      "123) parity5+5\n",
      "124) pendigits\n",
      "125) phoneme\n",
      "126) pima\n",
      "127) poker\n",
      "128) postoperative-patient-data\n",
      "129) prnn_crabs\n",
      "130) prnn_fglass\n",
      "131) prnn_synth\n",
      "132) profb\n",
      "133) promoters\n",
      "134) ring\n",
      "135) saheart\n",
      "136) satimage\n",
      "137) schizo\n",
      "138) segmentation\n",
      "139) shuttle\n",
      "140) sleep\n",
      "141) solar-flare_1\n",
      "142) solar-flare_2\n",
      "143) sonar\n",
      "144) soybean\n",
      "145) spambase\n",
      "146) spect\n",
      "147) spectf\n",
      "148) splice\n",
      "149) tae\n",
      "150) texture\n",
      "151) threeOf9\n",
      "152) tic-tac-toe\n",
      "153) titanic\n",
      "154) tokyo1\n",
      "155) twonorm\n",
      "156) vehicle\n",
      "157) vote\n",
      "158) vowel\n",
      "159) waveform-21\n",
      "160) waveform-40\n",
      "161) wdbc\n",
      "162) wine-quality-red\n",
      "163) wine-quality-white\n",
      "164) wine-recognition\n",
      "165) xd6\n",
      "166) yeast\n"
     ]
    }
   ],
   "source": [
    "from pmlb import classification_dataset_names, regression_dataset_names\n",
    "\n",
    "print('Classification Datasets: ')\n",
    "print('=========================')\n",
    "for i, name in enumerate(classification_dataset_names):\n",
    "    print('{}) {}'.format(i+1, name))\n",
    "    "
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 6,
   "metadata": {},
   "outputs": [
    {
     "name": "stdout",
     "output_type": "stream",
     "text": [
      "Regression Datasets: \n",
      "=========================\n",
      "1) 1027_ESL\n",
      "2) 1028_SWD\n",
      "3) 1029_LEV\n",
      "4) 1030_ERA\n",
      "5) 1089_USCrime\n",
      "6) 1096_FacultySalaries\n",
      "7) 1191_BNG_pbc\n",
      "8) 1193_BNG_lowbwt\n",
      "9) 1196_BNG_pharynx\n",
      "10) 1199_BNG_echoMonths\n",
      "11) 1201_BNG_breastTumor\n",
      "12) 1203_BNG_pwLinear\n",
      "13) 1595_poker\n",
      "14) 192_vineyard\n",
      "15) 195_auto_price\n",
      "16) 197_cpu_act\n",
      "17) 201_pol\n",
      "18) 207_autoPrice\n",
      "19) 210_cloud\n",
      "20) 215_2dplanes\n",
      "21) 218_house_8L\n",
      "22) 225_puma8NH\n",
      "23) 227_cpu_small\n",
      "24) 228_elusage\n",
      "25) 229_pwLinear\n",
      "26) 230_machine_cpu\n",
      "27) 294_satellite_image\n",
      "28) 344_mv\n",
      "29) 4544_GeographicalOriginalofMusic\n",
      "30) 485_analcatdata_vehicle\n",
      "31) 503_wind\n",
      "32) 505_tecator\n",
      "33) 519_vinnie\n",
      "34) 522_pm10\n",
      "35) 523_analcatdata_neavote\n",
      "36) 527_analcatdata_election2000\n",
      "37) 529_pollen\n",
      "38) 537_houses\n",
      "39) 542_pollution\n",
      "40) 547_no2\n",
      "41) 556_analcatdata_apnea2\n",
      "42) 557_analcatdata_apnea1\n",
      "43) 560_bodyfat\n",
      "44) 561_cpu\n",
      "45) 562_cpu_small\n",
      "46) 564_fried\n",
      "47) 573_cpu_act\n",
      "48) 574_house_16H\n",
      "49) 579_fri_c0_250_5\n",
      "50) 581_fri_c3_500_25\n",
      "51) 582_fri_c1_500_25\n",
      "52) 583_fri_c1_1000_50\n",
      "53) 584_fri_c4_500_25\n",
      "54) 586_fri_c3_1000_25\n",
      "55) 588_fri_c4_1000_100\n",
      "56) 589_fri_c2_1000_25\n",
      "57) 590_fri_c0_1000_50\n",
      "58) 591_fri_c1_100_10\n",
      "59) 592_fri_c4_1000_25\n",
      "60) 593_fri_c1_1000_10\n",
      "61) 594_fri_c2_100_5\n",
      "62) 595_fri_c0_1000_10\n",
      "63) 596_fri_c2_250_5\n",
      "64) 597_fri_c2_500_5\n",
      "65) 598_fri_c0_1000_25\n",
      "66) 599_fri_c2_1000_5\n",
      "67) 601_fri_c1_250_5\n",
      "68) 602_fri_c3_250_10\n",
      "69) 603_fri_c0_250_50\n",
      "70) 604_fri_c4_500_10\n",
      "71) 605_fri_c2_250_25\n",
      "72) 606_fri_c2_1000_10\n",
      "73) 607_fri_c4_1000_50\n",
      "74) 608_fri_c3_1000_10\n",
      "75) 609_fri_c0_1000_5\n",
      "76) 611_fri_c3_100_5\n",
      "77) 612_fri_c1_1000_5\n",
      "78) 613_fri_c3_250_5\n",
      "79) 615_fri_c4_250_10\n",
      "80) 616_fri_c4_500_50\n",
      "81) 617_fri_c3_500_5\n",
      "82) 618_fri_c3_1000_50\n",
      "83) 620_fri_c1_1000_25\n",
      "84) 621_fri_c0_100_10\n",
      "85) 622_fri_c2_1000_50\n",
      "86) 623_fri_c4_1000_10\n",
      "87) 624_fri_c0_100_5\n",
      "88) 626_fri_c2_500_50\n",
      "89) 627_fri_c2_500_10\n",
      "90) 628_fri_c3_1000_5\n",
      "91) 631_fri_c1_500_5\n",
      "92) 633_fri_c0_500_25\n",
      "93) 634_fri_c2_100_10\n",
      "94) 635_fri_c0_250_10\n",
      "95) 637_fri_c1_500_50\n",
      "96) 641_fri_c1_500_10\n",
      "97) 643_fri_c2_500_25\n",
      "98) 644_fri_c4_250_25\n",
      "99) 645_fri_c3_500_50\n",
      "100) 646_fri_c3_500_10\n",
      "101) 647_fri_c1_250_10\n",
      "102) 648_fri_c1_250_50\n",
      "103) 649_fri_c0_500_5\n",
      "104) 650_fri_c0_500_50\n",
      "105) 651_fri_c0_100_25\n",
      "106) 653_fri_c0_250_25\n",
      "107) 654_fri_c0_500_10\n",
      "108) 656_fri_c1_100_5\n",
      "109) 657_fri_c2_250_10\n",
      "110) 658_fri_c3_250_25\n",
      "111) 659_sleuth_ex1714\n",
      "112) 663_rabe_266\n",
      "113) 665_sleuth_case2002\n",
      "114) 666_rmftsa_ladata\n",
      "115) 678_visualizing_environmental\n",
      "116) 687_sleuth_ex1605\n",
      "117) 690_visualizing_galaxy\n",
      "118) 695_chatfield_4\n",
      "119) 706_sleuth_case1202\n",
      "120) 712_chscase_geyser1\n"
     ]
    }
   ],
   "source": [
    "print('Regression Datasets: ')\n",
    "print('=========================')\n",
    "for i, name in enumerate(regression_dataset_names):\n",
    "    print('{}) {}'.format(i+1, name))"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "## Fetching a Dataset\n",
    "\n",
    "To fetch data from a dataset in `pmlb` it is just necessary the **name** of the dataset. \n",
    "\n",
    "By default, a `pandas.DataFrame` will be returned as output."
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 8,
   "metadata": {},
   "outputs": [
    {
     "name": "stdout",
     "output_type": "stream",
     "text": [
      "                age     workclass        fnlwgt     education  education-num  \\\n",
      "count  48842.000000  48842.000000  4.884200e+04  48842.000000   48842.000000   \n",
      "mean      38.643585      3.870439  1.896641e+05     10.288420      10.078089   \n",
      "std       13.710510      1.464234  1.056040e+05      3.874492       2.570973   \n",
      "min       17.000000      0.000000  1.228500e+04      0.000000       1.000000   \n",
      "25%       28.000000      4.000000  1.175505e+05      9.000000       9.000000   \n",
      "50%       37.000000      4.000000  1.781445e+05     11.000000      10.000000   \n",
      "75%       48.000000      4.000000  2.376420e+05     12.000000      12.000000   \n",
      "max       90.000000      8.000000  1.490400e+06     15.000000      16.000000   \n",
      "\n",
      "       marital-status    occupation  relationship          race           sex  \\\n",
      "count    48842.000000  48842.000000  48842.000000  48842.000000  48842.000000   \n",
      "mean         2.618750      6.577700      1.443287      3.668052      0.668482   \n",
      "std          1.507703      4.230509      1.602151      0.845986      0.470764   \n",
      "min          0.000000      0.000000      0.000000      0.000000      0.000000   \n",
      "25%          2.000000      3.000000      0.000000      4.000000      0.000000   \n",
      "50%          2.000000      7.000000      1.000000      4.000000      1.000000   \n",
      "75%          4.000000     10.000000      3.000000      4.000000      1.000000   \n",
      "max          6.000000     14.000000      5.000000      4.000000      1.000000   \n",
      "\n",
      "       capital-gain  capital-loss  hours-per-week  native-country  \\\n",
      "count  48842.000000  48842.000000    48842.000000    48842.000000   \n",
      "mean    1079.067626     87.502314       40.422382       36.749355   \n",
      "std     7452.019058    403.004552       12.391444        7.775343   \n",
      "min        0.000000      0.000000        1.000000        0.000000   \n",
      "25%        0.000000      0.000000       40.000000       39.000000   \n",
      "50%        0.000000      0.000000       40.000000       39.000000   \n",
      "75%        0.000000      0.000000       45.000000       39.000000   \n",
      "max    99999.000000   4356.000000       99.000000       41.000000   \n",
      "\n",
      "             target  \n",
      "count  48842.000000  \n",
      "mean       0.760718  \n",
      "std        0.426649  \n",
      "min        0.000000  \n",
      "25%        1.000000  \n",
      "50%        1.000000  \n",
      "75%        1.000000  \n",
      "max        1.000000  \n"
     ]
    }
   ],
   "source": [
    "# Returns a pandas DataFrame\n",
    "from pmlb import fetch_data\n",
    "\n",
    "adult_data = fetch_data('adult')\n",
    "print(adult_data.describe())"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "In the next notebooks, we will be using this **package** to easily download and use datasets for further analysis and manipulations"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "---"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "### Feel free to SKIP \n",
    "\n",
    "Please feel free to **skip** the rest of this notebook and come back to it later once we will introduce **NumPy**, and \n",
    "**Scikit-learn** for Machine Learning."
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "The `fetch_data` function has two additional parameters:\n",
    "\n",
    "* `return_X_y` (True/False): Whether to return the data in scikit-learn format, with the features and labels stored in separate NumPy arrays.\n",
    "\n",
    "* `local_cache_dir` (string): The directory on your local machine to store the data files so you don't have to fetch them over the web again. By default, the wrapper does not use a local cache directory.\n",
    "\n",
    "For example:\n"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 9,
   "metadata": {},
   "outputs": [
    {
     "name": "stdout",
     "output_type": "stream",
     "text": [
      "[[3.90000e+01 7.00000e+00 7.75160e+04 ... 0.00000e+00 4.00000e+01\n",
      "  3.90000e+01]\n",
      " [5.00000e+01 6.00000e+00 8.33110e+04 ... 0.00000e+00 1.30000e+01\n",
      "  3.90000e+01]\n",
      " [3.80000e+01 4.00000e+00 2.15646e+05 ... 0.00000e+00 4.00000e+01\n",
      "  3.90000e+01]\n",
      " ...\n",
      " [3.80000e+01 4.00000e+00 3.74983e+05 ... 0.00000e+00 5.00000e+01\n",
      "  3.90000e+01]\n",
      " [4.40000e+01 4.00000e+00 8.38910e+04 ... 0.00000e+00 4.00000e+01\n",
      "  3.90000e+01]\n",
      " [3.50000e+01 5.00000e+00 1.82148e+05 ... 0.00000e+00 6.00000e+01\n",
      "  3.90000e+01]]\n",
      "[1 1 1 ... 1 1 0]\n"
     ]
    }
   ],
   "source": [
    "from pmlb import fetch_data\n",
    "\n",
    "# Returns NumPy arrays\n",
    "adult_X, adult_y = fetch_data('adult', return_X_y=True, local_cache_dir='./data')\n",
    "print(adult_X)\n",
    "print(adult_y)"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "\n",
    "\n",
    "## Example usage: Compare two classification algorithms with PMLB\n",
    "\n",
    "PMLB is designed to make it easy to benchmark machine learning algorithms against each other. \n",
    "\n",
    "Below is a Python code snippet showing the most basic way to use PMLB to compare two algorithms.\n"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 11,
   "metadata": {},
   "outputs": [],
   "source": [
    "from sklearn.linear_model import LogisticRegression\n",
    "from sklearn.naive_bayes import GaussianNB\n",
    "from sklearn.model_selection import train_test_split\n",
    "\n",
    "import matplotlib.pyplot as plt\n",
    "import seaborn as sb\n",
    "\n",
    "from pmlb import fetch_data, classification_dataset_names"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 13,
   "metadata": {
    "scrolled": false
   },
   "outputs": [
    {
     "name": "stdout",
     "output_type": "stream",
     "text": [
      "1 GAMETES_Epistasis_2-Way_1000atts_0.4H_EDM-1_EDM-1_1 DONE\n",
      "2 GAMETES_Epistasis_2-Way_20atts_0.1H_EDM-1_1 DONE\n",
      "3 GAMETES_Epistasis_2-Way_20atts_0.4H_EDM-1_1 DONE\n",
      "4 GAMETES_Epistasis_3-Way_20atts_0.2H_EDM-1_1 DONE\n",
      "5 GAMETES_Heterogeneity_20atts_1600_Het_0.4_0.2_50_EDM-2_001 DONE\n",
      "6 GAMETES_Heterogeneity_20atts_1600_Het_0.4_0.2_75_EDM-2_001 DONE\n"
     ]
    },
    {
     "name": "stderr",
     "output_type": "stream",
     "text": [
      "/Users/valerio/anaconda3/envs/webvalley19/lib/python3.7/site-packages/sklearn/svm/base.py:929: ConvergenceWarning: Liblinear failed to converge, increase the number of iterations.\n",
      "  \"the number of iterations.\", ConvergenceWarning)\n"
     ]
    },
    {
     "name": "stdout",
     "output_type": "stream",
     "text": [
      "7 Hill_Valley_with_noise DONE\n",
      "8 Hill_Valley_without_noise DONE\n",
      "9 adult DONE\n",
      "10 agaricus-lepiota DONE\n",
      "11 allbp DONE\n",
      "12 allhyper DONE\n",
      "13 allhypo DONE\n",
      "14 allrep DONE\n",
      "15 analcatdata_aids DONE\n",
      "16 analcatdata_asbestos DONE\n",
      "17 analcatdata_authorship DONE\n",
      "18 analcatdata_bankruptcy DONE\n",
      "19 analcatdata_boxing1 DONE\n",
      "20 analcatdata_boxing2 DONE\n",
      "21 analcatdata_creditscore DONE\n"
     ]
    },
    {
     "data": {
      "text/plain": [
       "Text(0, 0.5, 'Test Accuracy')"
      ]
     },
     "execution_count": 13,
     "metadata": {},
     "output_type": "execute_result"
    },
    {
     "data": {
      "image/png": "iVBORw0KGgoAAAANSUhEUgAAAYIAAAD4CAYAAADhNOGaAAAABHNCSVQICAgIfAhkiAAAAAlwSFlzAAALEgAACxIB0t1+/AAAADh0RVh0U29mdHdhcmUAbWF0cGxvdGxpYiB2ZXJzaW9uMy4xLjAsIGh0dHA6Ly9tYXRwbG90bGliLm9yZy+17YcXAAAgAElEQVR4nO3deXxU5dn/8c+VIfsKAoLskFgEtyqidW8R6o76tI/aum+lC4ICidqAO4RFfSQEkMUqz1MVrf6stFbFFZeKJiggyBIWEQSBgITs2/X7Yw40hiQMkDNnJnO9X6+8mDlzZs43cJjr3Ofc931EVTHGGBO5orwOYIwxxltWCIwxJsJZITDGmAhnhcAYYyKcFQJjjIlwbbwOcKjat2+vPXv29DqGMcaElYKCgp2q2qGx18KuEPTs2ZP8/HyvYxhjTFgRkW+aes1ODRljTISzQmCMMRHOCoExxkQ4KwTGGBPhrBAYY0yEc60QiMjTIrJdRL5q4nURkakiUigiy0TkFLeyGGOMaZqbLYJngAubef0iIMP5uQOY4WKW/SorK1m/fn0wNmXChKqyevVq6urqvI5ijCdcG0egqotEpGczqwwF5ql/HuxPRSRNRDqr6la3MgGMGDGSNWvWMHFiDqeddpqbm2pSbm4uhYWFnmy7vi1btgDQpUsXT3Okp6czfPhwz7Y/f/58Zs6cyRVXXMHIkSM9yxEqQmH/DJV9E7zfP4PBy2sEXYBv6z3f7Cw7gIjcISL5IpK/Y8eOI9poVVUVdXW1ZGePZdWqVUf0WeGuvLyc8vJyr2N46q233mLmzJmAf98wocH2zeDycmSxNLKs0bvkqOosYBbAgAEDjvhOOrWJHaisrWBMZhYzpufRtWvXI/3IQxIqRxcjRowA4Mknn/Q4iTc+++wzcnImUpvSmTYVP3gdJ2SEwv4Z6ftmsHnZItgMdKv3vCvwXTA2XBcdT0nGEErKqxg9Zgy7du0KxmZNCFm1ahVjx46jNj6NsvRBID6vIxnjGS8LwWvADU7voTOAPW5fH6hP41IpSR/E99t3kpmVRVlZWbA2bTy2efNmxmRmUSnRlKYPBl+M15GM8ZSb3UefB/4N/ERENovIrSIyTESGOau8DqwHCoHZwB/cytKUuqSOlPY+n8LCQrLHjqW6ujrYEUyQ7dq1i1GjR1NSXkVJxhA0JsHrSMZ4zs1eQ9ce5HUF/ujW9gNVm9aN8p5ns6TgQyZOnMh9991HVJSNs2uNysrKyMzMYvuOIkqO/SUal+p1JGNCgn3jATXtM6jscipvv/02Tz31lNdxjAuqq6vJzs6mcF0hpb3Ppy6po9eRjAkZYXc/gpYQVV2Ob8/mHy2rTWhHXWwK8+fPZ926dfz617/2KF3w7N27F4DFixd7nMRdqsr48eMpLi6mul0vEDng35+6Gm/CGRMCIq4QiAi+0h0krHmryXXy8/Mj6uY3WVlZXkcImuhdG4jeteGA5UfcJ9mYMBZxheC+++6loqKiyderqqrIzMykurqayi6nUJNyTBDTmZYWs+0rondvpG/fvgftH5+WlhakVMaElogrBH369DnoOq+88grDh9/JN98upyS1G3WJRwUhmWlpbYrWEb17I+eddx7jxo3D57OxAsY0xi4WNyI5OZkpUyZzVLu2JBUuRCqKvY5kDpFvzxbiN37ISSedxH333WdFwJhmWCFoQvv27XlsymQSYnz+YlBt856Ei6jSnSSue5eePXry6KOPEhsb63UkY0KaFYJm9OjRg0kTc4iuLSex8G2otQFnoU4qikkqXMhR7doyZfIkkpKSvI5kTMizQnAQ/fv358EHHsBXVkTCunfB5qwPWVJdTlLhQhJi2vDYlMm0b9/e60jGhAUrBAE488wzGTVqFL49W4jb+CGodTYMObXVJK5dSHRtOZMm5tCjRw+vExkTNqwQBOiSSy7hlltuIbpoHTGbI2eMQVioqyVh3bv4ynfx4AMP0L9/f68TGRNWrBAcguuvv56hQ4cSu2050dtWeB3HAKgSt/EjfHu2MHr0aM4880yvExkTdiJuHMGREBHuvPNOdu3axYcffoRGx1NzVG+vY0W0mM35RBet49Zbb+Xiiy/2Oo4xYclaBIfI5/ORnZ3N8SccT/zGRfiKg3IvHdOI6G0riN22nCuuuILrrrvO6zjGhC0rBIchNjaWCePH06N7dxLXvUtUWZHXkSJOm6L1xH27mHPOOYfhw4cj0tidT40xgbBCcJiSk5OZPGkS7dJSSVq7EKnc63WkiOEr/o74jYs44cQTyc7OtlHDxhwhKwRHoGPHjv7Rx9FRJK19y0YfB0FUaRGJ696lR/fujLdRw8a0CCsER6hnz57k5EwguqaMBBt97Cqp3EtS4ULapaUyedIkkpOTvY5kTKtghaAFnHDCCdx///34SneSsP49G33sAqkuJ2ntWyRER/H4Y1Po2NHuMGZMS7FC0ELOPvts7r77bnw/bCZu40c2+rgl1VaTUPg20TVlTLRRw8a0OCsELeiyyy7jpptuIrqokJgtBV7HaR3q6khY9x6+0p088MADHH/88V4nMqbVsQFlLezGG2+kqKiIBQsWoNEJVB/dz+tI4Wv/qOHNjBo9mrPOOsvrRMa0SlYIWpiIMGLECHbt2sXHH3/sH33crpfXscJSzJYCoosKufnmm7n00ku9jmNMq2WnhlzQpk0bxo0bR7/+/UnYsAhf8VavI4Wd6O9XErt1GZdddhk33HCD13GMadWsELgkNjaWnAkT6Nq1K4nr3iGqbJfXkcJGm10biNv0KWeddTYjR460UcPGuMwKgYtSUlJ4bMpk2qYmO6OPS7yOFPJ8xVtJ2LCI/v2PZ9y4sTZq2JggsELgMv/o4ynER+OMPq7wOlLIiiorInHdO3Tt2pUJE8bbqGFjgsQKQRD06tWLnAkTaFNdaqOPmyCVe0la+zbtUv2tqJSUFK8jGRMxrBAEyYknnsj994/DV7qDhPXvg9ro432kuoKktQuJj4YpUybbqGFjgsy6jwbROeecw8iRI3niiSdI/HI+dXGpEOnXQVXxVRYTTQ05jz1Gr17W1daYYLMWQZBdeumlpKSkEFVTjtTZKaKo6nKkupxBgwZx4okneh3HmIhkLYIgUlVyc3MpLi6mottAqjvZdAmoErfhA9544w1OOukkLrroIq8TGRNxrEUQRM899xyvvvoqVUcfb0VgHxEqep5DbeoxTJ48mcWLF3udyJiIY4UgSP71r38xe/Zsqo/qTWW307yOE1qifJT1GURtfDvGjh3H119/7XUiYyKKFYIg+PTTT5k8eTK1qV2o6HkO2EjZA/miKc0YTFVULJlZWXz77bdeJzImYlghcNnKlSsZN+5+auPbUdbnFxBlI2WbotHxlGQMoaSimlGjRlNUVOR1JGMighUCF23atInMrCyqomIpzRgMvmivI4U8jUuhJH0wO4p2MXrMGEpKbFoOY9xmhcAlO3fuZNTo0ZRW1lCSMQSNjvc6UtioS2xPaZ+fs2HDRrLHjqWqqsrrSMa0aq4WAhG5UERWi0ihiNzTyOupIrJARJaKyAoRudnNPMFSUlLCmMxMdhbtpiR9MBpn0yUcqtrUrpT3PJsvv/iCCRMmUGf3gTbGNa4VAhHxAXnARUA/4FoRaXi7rj8CK1X1JOB84DERiXErUzBUVVWRnZ3Nhg0bKe3zc+oS23sdKWzVtE+noutpvPfee+Tl5aF2H2hjXOHmgLKBQKGqrgcQkReAocDKeusokCz+CeeTgF1AjYuZXFVXV8f48eP58ssvKe91LrWpXb2OFPaqOx1PVHUZL7/8Mh06dOCaa67xOpJrli9fzqJFi7yOERK+++47APLy8jxOEjp+//vfExXlzrG7m4WgC1C/D+Bm4PQG60wDXgO+A5KBq1UPnI1NRO4A7gDo3r27K2GPlKqSl5fH+++/T0XX06hpn+51pNZBhMpuA5HqMmbOnEm7du0YMmSI16lcsX79el566SVi20jEX7yrqvW3/v7x//7mcRLvVdcpNXUwbNgw17bhZiForLN8w7b9L4EvgV8AfYCFIvKhqhb/6E2qs4BZAAMGDAjJ8wMvvPACL7/8MlVH97dRwy1NhIpe5xJVU0lOzkTS0tIYOHCg16lcM+WMIlJjQnI3Nx54dUM8r2xIcHUbbh54bAa61XveFf+Rf303A6+oXyGwAejrYiZXvPnmmzz11FNUt+tFZbeBNmDMDVE+ytJ/QW18GmPHjmPVqlVeJzKm1XCzEHwOZIhIL+cC8DX4TwPVtwkYBCAiRwM/Ada7mKnFLV68mIkTJ1GbcgwVvc61IuAmXwyl6YOplGgyM7PYvHmz14mMaRVcKwSqWgP8CXgT+Bp4UVVXiMgwEdl3suth4EwRWQ68A2Sp6k63MrW0VatWOaOG0yhLt1HDwaAxCZRkDGFveZWNPjamhbg6DbWqvg683mDZzHqPvwPC8srf5s2bGZOZRaXEOKOGw7rXa1jRuFRK0i+ANW+QmZVF7tSpJCS4ew7VmNYs0jsnHJaioiJGjRpNSXkVJRmD0Wj7Egq2uqQOlPY+n3Xr1pGdPZbqarvJjzGHywrBISotLSUzK4vtO4soSb8AjUv1OlLEqk3rRnnPs1mypICcnBwbfWzMYbJCcAiqq6vJHjuWdevWUdr7fOqSOngdKeLVtM+gsuupvPPOO8ycOfPgbzDGHMAKQYDq6uqYMGECXyxZQnnPs6lN63bwN5mgqOp0IlUd+/Hiiy8yf/58r+MYE3asEARo5syZvPvuu1R2PZWa9hlexzH1iVDZfSDV7XoyY8YM3n77ba8TGRNWrBAEYP78+bz44otUdexHVacTvY5jGiNRVPQ6l9qUzkyYMIH8/HyvExkTNqwQHMTChQuZMWMG1e16UtndRg2HtKg2lPUZRE1cKn/Ozmb16tVeJzImLFghaMbnn3/OhJwcalM6U9HrPBD76wp5bWIoTR9CJdGMycxiy5YtXicyJuTZN1sTVq9eTfbYsdTGpVLWZ5CNGg4jGpNASfoQ9pZVMGr0aHbv3u11JGNCmhWCRmzZssU/aphoStOHQBsbNRxuNN4/+vj77TvIzMykrKzM60jGhCwrBA3s3r2bUaNHs7esgpL0IWiMjRoOV3VJHSntfT5rCwsZO26cjT42pglWCOopKysjMzOT77fv8I8ajrdRw+GuNq075T3OoiA/n0mTJtnoY2MaYYXAUV1dzdhx41hbWOiMGu7odSTTQmo6HEtll1NYuHAhs2fP9jqOMSHnoIVARBaLyO9EJCUYgbxQV1fHpEmTKMjPp7zHWdSmhebtMM3hq+p8ElUd+/L888/z0ksveR3HmJASSIvgRqA38KWI/J+IDHI5U9DNmjWLhQsXUtnlFGo6HOt1HOMGESq7n0FN257k5eXxzjvveJ3ImJBx0EKgqqtUNQvIAF4G5onIBhEZKyJprid02UsvvcQLL7xAVce+VHU+yes4xk0SRXnvc6lL7sT48eMpKCjwOpExISGgawQi0g/IASYAfweuA6qAd92L5r533nmHvLw8atr2pLL7GTZqOBJEtaE0fRA1sf7Rx2vXrvU6kTGeC+gaATAdWA78VFX/oKofq+pE/DeoD0sFBQWMHz+euuROlPc+10YNR5I2sZRmDKaizsfoMWPYunWr14mM8VQg337Xq+r5qjpPVcvrv6Cql7uUy1Vr167lz9nZ1MSmUpo+CKJcvWOnCUEak0hpxhCKS8sZNWo0P/zwg9eRjPFMQIWg/rUAEWkrIg+6mMlVW7duZfSYMVTU+fz3Gm4T63Uk45G6+DRK+lzA1m3fk5mVZaOPTcQKpBBcqqr7D5dUdTdwmXuR3DUhJ4fi0nJKM4agMYlexzEeq0s+mtLe57NmzRqeeeYZr+MY44lACoFPRPZPtiMicUDYTr5TsreEqsSjqYsP+w5PpoXUtu2OxCRSUlLidRRjPBHIyfEXgIUi8jSgwK3AX11NZYwxJmgOWghUdbyILAcGAQJMUtV/up7MGGNMUATUXUZVFwALXM5ijDHGA4GMIzhNRD4VkT0iUiEilSJSHIxwxhhj3BdIi2A6/pHELwADgZuAbi5mMsYYE0SB9BqKUtXVQBtVrVbV2cAFLucyxhgTJIG0CEqd7qNLRWQ8sBVIcjeWMcaYYAmkRXCTs96fgFr8s5D+ysVMxhhjgqjZFoGI+ID7VfVGoAIYG5RUxhhjgqbZFoGq1gKdRSQ6SHmMMcYEWSDXCNYDH4rI34HSfQtVdaprqYwxxgRNIIVgB7AQSHB+jDHGtCKBTDFh1wWMMaYVO2ghEJGF+Ceb+xFVHeJKImOMMUEVyKmh7HqP44D/AirdiWOMMSbYAjk1tLjBog9E5AOX8hhjjAmyQCadS6n3kyYig4DOgXy4iFwoIqtFpFBE7mlinfNF5EsRWWEFxhgo3GP30DZ+1XWwca/7+0MgW1iB/xqBADXABuD2g73JGYyWBwwGNgOfi8hrqrqy3jpp+Ce1u1BVN4lIx0P/FYxpHQYOHEjvXj2ZunwjV/Yq4/Ke5USJ16mMV4oqopi2IoV1e3xcffXVREUFMhHE4Qnk1NDhzjQ6EChU1fUAIvICMBRYWW+d3wCvqOomZ1vbD3NbxoS9zp07M33GTB5//HFeeestCoujGdZvL0nRB/TVMK3cil1tmL4ylVpfHA8+eC/nnXeeq9sL5NTQMOfIfd/ztiJyRwCf3QX4tt7zzc6y+o4F2orI+yJSICI3NJHhDhHJF5H8HTt2BLBpY8JTXFwc9957L3fddRcrf4jj/vx2bNzr8zqWCZI6hdc2xjNpaSrtOnVj5lOzXC8CENikc8NU9Yd9T1R1N/D7AN7XWKO24aFNG+BU4BLgl8BYETn2gDepzlLVAao6oEOHDgFs2pjwJSIMHTqUqbm5kHgUDxek8cF3sV7HMi4rrRaeXJ7C39Yn8POf/4IZM5+ie/fuQdl2IIXgR4cjIhIFBDL30GZ+fAObrsB3jazzhqqWqupOYBFwUgCfbUyr169fP2bPmcuJJ5/C3FVJzPk6kapar1MZN2za6+P+grYs2x3LnXfeydixY0lICN5EDoEUgoUi8ryInCci5wJ/Bd4O4H2fAxki0su5n8E1wGsN1vk7cI6ItBGRBOB04OtDyG9Mq5aWlsakyZO57rrrWLQ1jke+aMuOcvcuGprg+3BrLA8uSaM2rh1PPjmVq666CpHg9hIIZI8aA3wM3AWMAj4CRh/sTapag/8eBm/i/3J/UVVXONcchjnrfA28ASwDPgPmqOpXh/OLGNNa+Xw+brvtNsaPH8/O2iTGFbRl6U6bEDjcVdfBX1YlMvvrJI4/4WTmzH2a448/3pMsgXQfjQamq+o02H9qKAZ/V9JmqerrwOsNls1s8HwyMDnQwMZEqjPPPJNZs+cwNvvPPLZsA0N7lnFlL+tiGo52lkeRuyKFDcU+rr32Wm699VbatPFu/EggLYL3gMR6zxOBd92JY4xpzjHHHMP0GTO58MIL+fvGBKYsTWFvtVWCcLK8KJpxBW35vjqRhx9+mN/97neeFgEIrBDEq+refU+cxzYdtTEeiY2NJSsri9GjR7O6OI5x+e1YX2xdTENdncKrG+KZsjSFDsf0YNbsOZxzzjlexwICKwRlIrK/J4+InIz/tpXGGI+ICJdeeim50/LwJbXnkSVpvLclFrWxZyGppFp4YlkKr2xI4ILBg5k+YyZdu3b1OtZ+gbRH7gL+n4h84zzvjn9EsDHGY3379mX2nLk88sjD/OXzfNbuacNNPyklxhoIIWPjXh+5X6Wyu6oNI0cOZ+jQoUHvFXQwAc0+KiLHAcfhHyS2ArDezMaEiNTUVHJyJjJv3jyeffZZNpXGMLz/Ho5OqPM6WsT74LtY5q1JIq3dUTw55SH69+/vdaRGBdQhWVUrVfVLIBmYCmxxNZUx5pD4fD5uvvlmcnJy2K3J3F/Qli+si6lnqmph7teJzF2VxAkn/5TZc+aGbBGAwOYaOlVEHhORjcC/8A8U86azqzGmWWeccQazZs/hmB59eGJZCi+ti6fOrhsE1Y7yKB75oi0fbI3juuuuY/LkKaSlpR38jR5qshCIyIMisgp4HFgLnAZsV9W5znQQxpgQ1LlzZ/LypnPxxRez4JsEJi9NobgqtM5Jt1ZLi6IZl9+WHTWJPProo9x22234fKF/waa5FsGfgO3AE8DTqrqDRu5dbIwJPbGxsWRmZjJmzBjW7I3n/oJ2rLMb3rimTuGV9fE8vjSFTt16MWv2HM466yyvYwWsuULQCf+I3/8G1ovIX4B4Z2SxMSYMXHLJJeTlTSc6pSOPfJHK25uti2lL21stPL4shVc3JjDkl78kb/oMunRpOON+aGvyS11Vq1V1gar+Bv99A97EPx/QFhGZF6yAxpgjc+yxxzJr9hwGnHY689Yk8dTKJCqt31+LWF/s4/78dny9J467776be+65h7i4OK9jHbJAew2VqeoLqjoU6AfYvYWNCSMpKSlMmDCBW265hX9vj+WhJW3ZVmaN+8OlCu9tieWRJWlEJbUnd1oel19+eciNDwjUIe8JqrpbVee6EcYY456oqChuuOEGJk6cxA9OF9OCHdbF9FBV1cKcVYn8ZXUSJ59yKrNmz6Fv375exzoidkhgTIQZOHAgs+fMpXuvDJ5cnsL8wgRqbexZQLaXR/HwkrZ8uDVuf1EN9a6hgQhkHMEBXQ0aW2aMCR+dOnViau40LrvsMv65yX+P3D3WxbRZX+z0dw0tqkvaf5otHLqGBiKQFsFnAS4zxoSR2NhYRo0aRVZWFutK4hmX34611sX0AHUKf1sXzxPLUjime29mzZ7Dz372M69jtagm/9VFpCPQGX+X0RP4z83oUwjzaah95buI3fSp1zE859u7DYDa5E4eJ/Ge1lR6HcEzF110Eenp6Ywbm834JXBteimDu1YQptc9W9TeKmHGymS+2hXNRRddxMiRI4mNjfU6VotrrvxfAtyC/6bzefynEOwFxrqcyzWJiQkk+upg70avo3iussI/m3iSzSoOsdGt8j94oDIyMpg1ew733Xcv/7f8Kz74LpbEaBtwsKW0DXurhTvvvJOrrrrK6ziuabIQqOpfgL+IyH+r6otBzOSq3NxcryOEjBEjRgDw5JNPepzEhIKPPvqI1atWE+uD6CgrAgDxberYW+3j9df/yRlnnMExxxzjdSRXBHJCsKOIpKhqsYjMBE4B7lXVd1zOZowJgsrKSqZNm8aCBQvo17aGP/QvJiXGCsE+X+6MZubX67jj9tv4c/bYVnd9AAK7WHyHUwSG4D9N9HtgkruxjDHBsG3bNu4c/icWLFjAZT3KyDx5jxWBBk5uX83DA3ZzlK+Ee++9l6effpra2tY1NDuQQrBvr7gI+IuqFgT4PmNMCPvss8+4/bZb2bRhLSNOKObXfcqJsgvEjeoQX8fYn+7m3M4VzJs3j6ysTH744QevY7WYQE4NLRWR1/HPN/RnEUnCZiE1psUtXryY1157zfXtqCqffPLJ/uddEmv4cGscH24NnTlyvi3x98/vlhSaR975+QVcccUVHHXUUUEbVfzQQw+5Nm4hkEJwM3AqUKiqZSLSHrjVlTQRIjc3l8LCQq9j7M+w76KxV9LT0xk+fLinGULBtm3b+Pjjj+mSVIfPpSPz0mqhqOI/H961yzHExsayx53NHba9JVsB2JPS2eMkB+qTAuvWbwCgqKiIjz/+mG7JdbjVmNpTKa4P9gvknsW1ItIbGAw8CsRjp4Zahfj4eK8jmEbcc/IPpLpwnn7fTdSj2/j40/A7Q3qStHDo0VZcXMyjjzzM4s8+p3tiFTf9pJRYFw7YX90Qzysb3B26ddBCICLTgGjgXPyFoBSYif+OZeYw2NGvCbb6N1Gf+vAjHHfccV5HCnspKSlMyJnIvHnzePbZZ9hUGsOd/fdwdEL4TdwUyJH9mar6O/CPOlLVXUCMq6mMMS2iqhbmODdRP+mnpzB7zlwrAi0oKiqKm266iZyciftndF0ShjO6BlIIqp27kimAiBwFhF/JMybC7HBmyly0NY7rr7+eiZMmt4qZMkPR6aefzqzZc+jaM53/WZ7CS+viqQujLjXN3bx+32mjPOBloIOIPAh8BEwMQjZjzGFa2mCmzFtvvbXVzJQZqjp37kzutDwuvfRSFnyTwKSlqRSHyYyuzbUIPgNQ1XlANjAF2A38WlVfCEI2Y8whqlN4eX08jy1LoXMrnSkzlMXGxjJ69GiysrIo3Ouf0bUwDGZ0bS7h/lKmqiuAFe7HMcYcrvozZV544YXcddddET2Rnpf2z+ia/Wce/QJ+m17CoC6VITuja3OFoIOI3N3Ui6r6uAt5jDGHYX2xj9wVaRRX+xg9+i4uueSSkO0aGikyMjKYNWcujz76CPM+XUzhnmhu7lviShfTI9XcqSEfkAQkN/FjjPFY/Zuot0nusP8ctRWB0JCcnMz48f5rNP/eHsuDBW3ZWhZ6w7CaaxFsVdWHgpbEGHNIKmvh2dWJfLQtjoGnncafs7NJTU31OpZpICoqiuuvv57jjjuOhx58gAcKori9714GdKjyOtp+zZUmO6QwJkR9X+bvGvrx93HceOONTMjJsSIQ4gYMGMDsOXPp3vtYpi5PZn5hArUh0hG/uUIwKGgpjDEB+2JnNPcXtGW3JpOTM5Gbb77ZuoaGiaOPPprc3Glcfvnl/HNTPJOWpro+j1AgmiwEzghiY0yIqFN4ybmJetee6cyaPYfTTz/d61jmEMXExHD33Xdz3333sb7U38V0rcddTEPvqoUx5gDFVcLkpSks+CaBSy65hNxpeXTuHHozc5rADRkyhOkzZhLfthPjl6Ty1rdxqEejkV0tBCJyoYisFpFCEbmnmfVOE5FaEfmVm3mMCUfr9rRhXH471uyNJzMzkzFjxtj4gFaiT58+PDVrNqf/7Ez+b20iM1YkUVET/ByuFQIR8eGfnuIioB9wrYj0a2K9icCbbmUxJhypwtubY3nki1RiUzsyffoMLr74Yq9jmRaWnJzMI488wu23385nO+J4cElbvisN7skaN7c2EP/NbNarahXwAjC0kfWG45/LaLuLWYwJK5W18NTKJOatSeK0gacza85cMjIyvB5y+ycAAAszSURBVI5lXBIVFcVvf/tbJk+ZQmlUKg8UtOWz7cGb5NnNQtAF+Lbe883Osv1EpAtwJf77GzRJRO4QkXwRyd+xY0eLBzUmlGwr8/FQQVv+vT2WW265hfHjJ5CcbGM4I8Gpp57KrDlz6Z3Rl2lfJfP82gRqg3DdwM1C0FifqIa/0v8AWara7I1JVXWWqg5Q1QEdOnRosYDGhKJJS1PZIylMmjSZG264gago69MRSTp27MiTU3O58sor+de38by+yf07CbrZZ2kz0K3e867Adw3WGQC84AyHbw9cLCI1qvqqi7mMCUn7bh3aO/1YHnzwITp16uRxIuOV6OhoRowYQb9+/ZgyZTLxLk8Z4mYh+BzIEJFewBbgGuA39VdQ1V77HovIM8A/rAiYSDVkyBB69epFjx49iImxmwAaGDx4MH379iU1NdXVQYOuFQJVrRGRP+HvDeQDnlbVFSIyzHm92esCxkQiuyBsGurWrdvBVzpCrg5nU9XXgdcbLGu0AKjqTW5mMcYY0zi7CmWMMRHOCoExxkQ4KwTGGBPhrBAYY0yEs0JgjDERzgqBMcZEOCsExhgT4awQGGNMhLNCYIwxEc4KgTHGRDgrBMYYE+GsEBhjTISzQmCMMRHOCoExxkQ4KwTGGBPhrBAYY0yEs0JgjDERzgqBMcZEOCsExhgT4awQGGNMhLNCYIwxEc4KgTHGRDgrBMYYE+GsEBhjTISzQmCMMRHOCoExxkQ4KwTGGBPhrBAYY0yEs0JgjDERzgqBMcZEOCsExhgT4awQGGNMhLNCYIwxEc4KgTHGRDgrBMYYE+GsEBhjTISzQmCMMRHOCoExxkQ4VwuBiFwoIqtFpFBE7mnk9d+KyDLn5xMROcnNPMYYYw7kWiEQER+QB1wE9AOuFZF+DVbbAJynqicCDwOz3MpjjDGmcW62CAYChaq6XlWrgBeAofVXUNVPVHW38/RToKuLeYwxxjTCzULQBfi23vPNzrKm3Ar8q7EXROQOEckXkfwdO3a0YERjjDFuFgJpZJk2uqLIz/EXgqzGXlfVWao6QFUHdOjQoQUjGmOMaePiZ28GutV73hX4ruFKInIiMAe4SFWLXMxjjDGmEW62CD4HMkSkl4jEANcAr9VfQUS6A68A16vqGhezGGOMaYJrLQJVrRGRPwFvAj7gaVVdISLDnNdnAuOAo4DpIgJQo6oD3MpkjDHmQG6eGkJVXwdeb7BsZr3HtwG3uZnBGGNM82xksTHGRDgrBMYYE+GsEBhjTIRz9RqBMSb85ObmUlhY6GmGfdsfMWKEpzkA0tPTGT58uNcxXGWFwBgTcuLj472OEFGsEBhjfqS1H/2aA9k1AmOMiXBWCIwxJsJZITDGmAhnhcAYYyKcFQJjjIlwVgiMMSbCWSEwxpgIZ4XAGGMinKg2evfIkCUiO4BvvM7RirQHdnodwphG2L7ZsnqoaqP3+g27QmBalojk282ATCiyfTN47NSQMcZEOCsExhgT4awQmFleBzCmCbZvBoldIzDGmAhnLQJjjIlwVgiMMSbCWSFoISJS0gKfcYyI/K2Z19NE5A+Bru+s876IrBaRpSLyuYicfKQ5W5KIPCQiF3idwxyciBwtIs+JyHoRKRCRf4vIlS5vc4CITD2C928UkZfrPf+ViDzjPL5JRHaIyJciskJE/iYiCS0QO+xYIQghqvqdqv6qmVXSgD8cwvr7/FZVTwKmA5OPMCYAItIid7dT1XGq+nZLfJZxj4gI8CqwSFV7q+qpwDVAVze3q6r5qnrnEX7MABHp38Rr81X1ZFXtD1QBVx/htsKSFQIXiUgPEXlHRJY5f3Z3lvcRkU+dI/SH9rUmRKSniHzlPO4vIp85RyvLRCQDyAH6OMsmN1jfJyJTRGS5s35j9xv8N9ClXr4hzlHdEhF5SUSSnOUXi8gqEflIRKaKyD+c5Q+IyCwReQuY52xzsvN7LBOR3znrdRaRRU7Or0TkHGfdZ5zny0XkLmfdZ0TkV87jQSLyhfP60yIS6yzfKCIPOjmXi0hfF/65TPN+AVSp6sx9C1T1G1XNdfbDD51/nyUiciaAiJy/b99xnk8TkZucxzkistLZb6Y4y37t7B9LRWRRw88QkYEi8omzj3wiIj9xlt8kIq+IyBsislZEJjXIPgW4r7lfzjmwSQR2H9lfU3iyexa7axowT1WfFZFbgKnAFcCTwJOq+ryIDGvivcOcdf4qIjGAD7gHOF5VTwZ/4ai3/h1AL+CnqlojIu0a+cwL8R/VISLtgWzgAlUtFZEs4G7nP9FTwLmqukFEnm/wGacCZ6tquYjcAexR1dOcL+2PnSJxFfCmqj4qIj4gATgZ6KKqxzvbT6v/oSISBzwDDFLVNSIyD/g98D/OKjtV9RTxnxobDdzWxN+bcUd/YEkTr20HBqtqhXPA8jzQ5IhgZ9+8EuirqlpvXxgH/FJVtzTcPxyr8O+XNeI/nTge+C/ntZOBnwKVwGoRyVXVb53XXgT+ICLpjXzm1SJyNtAZWAMsaCp3a2YtAnf9DHjOefy/wNn1lr/kPH6u4Zsc/wbuc76ge6hq+UG2dQEwU1VrAFR1V73X/ioim4EsINdZdgbQD/+X95fAjUAPoC+wXlU3OOs1LASv1csyBLjBef9i4CggA/gcuFlEHgBOUNW9wHqgt4jkisiFQHGDz/0JsEFV1zjPnwXOrff6K86fBUDPg/xdGJeJSJ5z5P45EA3MFpHl+Pfrfgd5ezFQAcwRkauAMmf5x8AzInI7/gOfhlKBl5xW8BP4i9M+76jqHlWtAFbi35f3qcV/SvTeRj5zvnNg1QlYDow5SPZWyQpBcAU8aENVnwMuB8qBN0XkFwd5izTz+b/F31p4Dsirt/5C5/zoyaraT1VvdZY3p7TBNofX+4xeqvqWqi7C/yW+BfhfEblBVXcDJwHvA38E5jSSvzmVzp+1WEvWCyuAU/Y9UdU/AoOADsBdwPf4/30HADHOajX8+DsmznlvDTAQeBl/C/kNZ/kw/K3UbsCXInJUgwwPA+85rcrL9n2eo7Le48b2kf/Fv092b+yXU/+AqgX8+OAjYlghcNcn+C+ogf/L+CPn8af8p0l7TcM3AYhIb/xH5lOB14ATgb1AchPbegsY5pzr3Nf83k9Vq/H/JztDRI5zMpy1r7ksIgkiciz+5nfveqedmrt49ibwexGJdj7jWBFJFJEewHZVnQ3MBU5xTkVFqerLwFjqfak4VgE96zXfrwc+aGbbJrjeBeJE5Pf1lu3rYZMKbFXVOvz/bvuO5r8B+olIrIik4i8cONeiUlX1dWAk/tM6iEgfVV2squPwzzrarUGGVPwHFwA3HUp4Z/9/wtleU84G1h3K57YWVghaToKIbK73czdwJ/5TJMvw/wcZ4aw7Ev/5+M/wn5vc08jnXQ185Zx26Yv/WkMR/lM5X4lIw94/c4BNwDIRWQr8puEHOqd0HgNGq+oO/P+ZnnfyfYr/nG05/p5Jb4jIR/iP9BrLt2+bK4ElTnP9KfxHYufjP6L7An/BexL/Rer3nd/nGRo0050m/c34m/7LgTpgJiYkOEfMVwDnicgGZ999Fv/pxunAjSLyKXAsTqvROUf/IrAM+CvwhfNxycA/nP3uA/wtCoDJTmeAr4BFwNIGMSYBE0TkYxo/dXQwczmwpXC1OB0y8F9jePgwPjfs2RQTHhB/X+Vy50LZNcC1qjrU61z7iEiSqpaIiOA/lbRWVZ/wOpcxxh12rtUbpwLTnC/aH4BbPM7T0O0iciP+c71f4D/SN8a0UtYiMMaYCGfXCIwxJsJZITDGmAhnhcAYYyKcFQJjjIlwVgiMMSbC/X9gpZA+c/hd0AAAAABJRU5ErkJggg==\n",
      "text/plain": [
       "<Figure size 432x288 with 1 Axes>"
      ]
     },
     "metadata": {
      "needs_background": "light"
     },
     "output_type": "display_data"
    }
   ],
   "source": [
    "logit_test_scores = []\n",
    "gnb_test_scores = []\n",
    "\n",
    "for i, classification_dataset in enumerate(classification_dataset_names):\n",
    "    if i > 20:\n",
    "        break\n",
    "        \n",
    "    X, y = fetch_data(classification_dataset, return_X_y=True)\n",
    "    train_X, test_X, train_y, test_y = train_test_split(X, y)\n",
    "\n",
    "    logit = LogisticRegression(solver='liblinear', multi_class='auto', )\n",
    "    gnb = GaussianNB()\n",
    "\n",
    "    logit.fit(train_X, train_y)\n",
    "    gnb.fit(train_X, train_y)\n",
    "\n",
    "    logit_test_scores.append(logit.score(test_X, test_y))\n",
    "    gnb_test_scores.append(gnb.score(test_X, test_y))\n",
    "    print('{} {} DONE'.format(i+1, classification_dataset))\n",
    "\n",
    "sb.boxplot(data=[logit_test_scores, gnb_test_scores], notch=True)\n",
    "plt.xticks([0, 1], ['LogisticRegression', 'GaussianNB'])\n",
    "plt.ylabel('Test Accuracy')"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": null,
   "metadata": {},
   "outputs": [],
   "source": []
  }
 ],
 "metadata": {
  "kernelspec": {
   "display_name": "Python 3.7 (WebValley2019)",
   "language": "python",
   "name": "webvalley19"
  },
  "language_info": {
   "codemirror_mode": {
    "name": "ipython",
    "version": 3
   },
   "file_extension": ".py",
   "mimetype": "text/x-python",
   "name": "python",
   "nbconvert_exporter": "python",
   "pygments_lexer": "ipython3",
   "version": "3.7.3"
  }
 },
 "nbformat": 4,
 "nbformat_minor": 2
}