Introduction_to_Matplotlib_and_Line_Plots.ipynb

{
    "cells": [
        {
            "cell_type": "markdown",
            "metadata": {
                "deletable": true,
                "editable": true
            },
            "source": "<a href=\"https://cognitiveclass.ai\"><img src = \"https://ibm.box.com/shared/static/9gegpsmnsoo25ikkbl4qzlvlyjbgxs5x.png\" width = 400> </a>\n\n<h1 align=center><font size = 5>Introduction to Matplotlib and Line Plots</font></h1>"
        },
        {
            "cell_type": "markdown",
            "metadata": {
                "deletable": true,
                "editable": true
            },
            "source": "## Introduction\n\nThe aim of these labs is to introduce you to data visualization with Python as concrete and as consistent as possible. \nSpeaking of consistency, because there is no *best* data visualization library avaiblable for Python - up to creating these labs - we have to introduce different libraries and show their benefits when we are discussing new visualization concepts. Doing so, we hope to make students well-rounded with visualization libraries and concepts so that they are able to judge and decide on the best visualitzation technique and tool for a given problem _and_ audience.\n\nPlease make sure that you have completed the prerequisites for this course, namely **Python 101** and **Data Analysis with Python**. We highly recommend that you complete these two courses before embarking on completing this course. Nevertheless, we have included a brief crash course on *pandas*.\n\n------------"
        },
        {
            "cell_type": "markdown",
            "metadata": {
                "deletable": true,
                "editable": true
            },
            "source": "## Table of Contents\n\n<div class=\"alert alert-block alert-info\" style=\"margin-top: 20px\">\n\n1. [Exploring Datasets with *pandas*](#0)<br>\n1.1 [The Dataset: Immigration to Canada from 1980 to 2013](#2)<br>\n1.2 [*pandas* Basics](#4) <br>\n1.3 [*pandas* Intermediate: Indexing and Selection](#6) <br>\n2. [Visualizing Data using Matplotlib](#8) <br>\n2.1 [Matplotlib: Standard Python Visualization Library](#10) <br>\n3. [Line Plots](#12)\n</div>\n<hr>"
        },
        {
            "cell_type": "markdown",
            "metadata": {
                "deletable": true,
                "editable": true
            },
            "source": "# Exploring Datasets with *pandas* <a id=\"0\"></a>\n\n*pandas* is an essential data analysis toolkit for Python. From their [website](http://pandas.pydata.org/):\n>*pandas* is a Python package providing fast, flexible, and expressive data structures designed to make working with \u201crelational\u201d or \u201clabeled\u201d data both easy and intuitive. It aims to be the fundamental high-level building block for doing practical, **real world** data analysis in Python.\n\nThe course heavily relies on *pandas* for data wrangling, analysis, and visualization. We encourage you to spend some time and  familizare yourself with the *pandas* API Reference: http://pandas.pydata.org/pandas-docs/stable/api.html."
        },
        {
            "cell_type": "markdown",
            "metadata": {
                "deletable": true,
                "editable": true
            },
            "source": "## The Dataset: Immigration to Canada from 1980 to 2013 <a id=\"2\"></a>"
        },
        {
            "cell_type": "markdown",
            "metadata": {
                "deletable": true,
                "editable": true
            },
            "source": "Dataset Source: [International migration flows to and from selected countries - The 2015 revision](http://www.un.org/en/development/desa/population/migration/data/empirical2/migrationflows.shtml).\n\nThe dataset contains annual data on the flows of international immigrants as recorded by the countries of destination. The data presents both inflows and outflows according to the place of birth, citizenship or place of previous / next residence both for foreigners and nationals. The current version presents data pertaining to 45 countries.\n\nIn this lab, we will focus on the Canadian immigration data.\n\n<img src = \"https://ibm.box.com/shared/static/mb48k9fiylkd7z3a21cq38xxfy1wni2y.png\" align=\"center\" width=900>\n\nFor sake of simplicity, Canada's immigration data has been extracted and uploaded to one of IBM servers. You can fetch the data from [here](https://ibm.box.com/shared/static/lw190pt9zpy5bd1ptyg2aw15awomz9pu.xlsx).\n\n---"
        },
        {
            "cell_type": "markdown",
            "metadata": {
                "deletable": true,
                "editable": true
            },
            "source": "## *pandas* Basics<a id=\"4\"></a>"
        },
        {
            "cell_type": "markdown",
            "metadata": {
                "deletable": true,
                "editable": true
            },
            "source": "The first thing we'll do is import two key data analysis modules: *pandas* and **Numpy**."
        },
        {
            "cell_type": "code",
            "execution_count": 1,
            "metadata": {
                "collapsed": false,
                "deletable": true,
                "editable": true,
                "jupyter": {
                    "outputs_hidden": false
                }
            },
            "outputs": [],
            "source": "import numpy as np  # useful for many scientific computing in Python\nimport pandas as pd # primary data structure library"
        },
        {
            "cell_type": "code",
            "execution_count": 2,
            "metadata": {},
            "outputs": [],
            "source": "from __future__ import print_function # adds compatibility to python 2"
        },
        {
            "cell_type": "markdown",
            "metadata": {
                "deletable": true,
                "editable": true
            },
            "source": "Let us download and import our primary Canadian Immigration dataset using *pandas* `read_excel()` method. But before we can do that, we need to download a module which *pandas* requires to read in excel files. This module is **xlrd**."
        },
        {
            "cell_type": "code",
            "execution_count": 3,
            "metadata": {
                "collapsed": false,
                "deletable": true,
                "editable": true,
                "jupyter": {
                    "outputs_hidden": false
                }
            },
            "outputs": [
                {
                    "name": "stdout",
                    "output_type": "stream",
                    "text": "Collecting xlrd\n\u001b[?25l  Downloading https://files.pythonhosted.org/packages/b0/16/63576a1a001752e34bf8ea62e367997530dc553b689356b9879339cf45a4/xlrd-1.2.0-py2.py3-none-any.whl (103kB)\n\u001b[K     |\u2588\u2588\u2588\u2588\u2588\u2588\u2588\u2588\u2588\u2588\u2588\u2588\u2588\u2588\u2588\u2588\u2588\u2588\u2588\u2588\u2588\u2588\u2588\u2588\u2588\u2588\u2588\u2588\u2588\u2588\u2588\u2588| 112kB 8.3MB/s eta 0:00:01\n\u001b[?25hInstalling collected packages: xlrd\nSuccessfully installed xlrd-1.2.0\nxlrd installed!\n"
                }
            ],
            "source": "# install xlrd\n!pip install xlrd\nimport xlrd\nprint('xlrd installed!')"
        },
        {
            "cell_type": "markdown",
            "metadata": {
                "deletable": true,
                "editable": true
            },
            "source": "Now we are ready to read in our data."
        },
        {
            "cell_type": "code",
            "execution_count": 4,
            "metadata": {
                "collapsed": false,
                "deletable": true,
                "editable": true,
                "jupyter": {
                    "outputs_hidden": false
                }
            },
            "outputs": [
                {
                    "name": "stdout",
                    "output_type": "stream",
                    "text": "Data read into a pandas dataframe!\n"
                }
            ],
            "source": "df_can = pd.read_excel('https://s3-api.us-geo.objectstorage.softlayer.net/cf-courses-data/CognitiveClass/DV0101EN/labs/Data_Files/Canada.xlsx',\n                       sheet_name='Canada by Citizenship',\n                       skiprows=range(20),\n                       skipfooter=2)\n\n\nprint ('Data read into a pandas dataframe!')"
        },
        {
            "cell_type": "markdown",
            "metadata": {
                "deletable": true,
                "editable": true
            },
            "source": "Let's view the top 5 rows of the dataset using the `head()` function."
        },
        {
            "cell_type": "code",
            "execution_count": 5,
            "metadata": {
                "collapsed": false,
                "deletable": true,
                "editable": true,
                "jupyter": {
                    "outputs_hidden": false
                }
            },
            "outputs": [
                {
                    "data": {
                        "text/html": "<div>\n<style scoped>\n    .dataframe tbody tr th:only-of-type {\n        vertical-align: middle;\n    }\n\n    .dataframe tbody tr th {\n        vertical-align: top;\n    }\n\n    .dataframe thead th {\n        text-align: right;\n    }\n</style>\n<table border=\"1\" class=\"dataframe\">\n  <thead>\n    <tr style=\"text-align: right;\">\n      <th></th>\n      <th>Type</th>\n      <th>Coverage</th>\n      <th>OdName</th>\n      <th>AREA</th>\n      <th>AreaName</th>\n      <th>REG</th>\n      <th>RegName</th>\n      <th>DEV</th>\n      <th>DevName</th>\n      <th>1980</th>\n      <th>...</th>\n      <th>2004</th>\n      <th>2005</th>\n      <th>2006</th>\n      <th>2007</th>\n      <th>2008</th>\n      <th>2009</th>\n      <th>2010</th>\n      <th>2011</th>\n      <th>2012</th>\n      <th>2013</th>\n    </tr>\n  </thead>\n  <tbody>\n    <tr>\n      <th>0</th>\n      <td>Immigrants</td>\n      <td>Foreigners</td>\n      <td>Afghanistan</td>\n      <td>935</td>\n      <td>Asia</td>\n      <td>5501</td>\n      <td>Southern Asia</td>\n      <td>902</td>\n      <td>Developing regions</td>\n      <td>16</td>\n      <td>...</td>\n      <td>2978</td>\n      <td>3436</td>\n      <td>3009</td>\n      <td>2652</td>\n      <td>2111</td>\n      <td>1746</td>\n      <td>1758</td>\n      <td>2203</td>\n      <td>2635</td>\n      <td>2004</td>\n    </tr>\n    <tr>\n      <th>1</th>\n      <td>Immigrants</td>\n      <td>Foreigners</td>\n      <td>Albania</td>\n      <td>908</td>\n      <td>Europe</td>\n      <td>925</td>\n      <td>Southern Europe</td>\n      <td>901</td>\n      <td>Developed regions</td>\n      <td>1</td>\n      <td>...</td>\n      <td>1450</td>\n      <td>1223</td>\n      <td>856</td>\n      <td>702</td>\n      <td>560</td>\n      <td>716</td>\n      <td>561</td>\n      <td>539</td>\n      <td>620</td>\n      <td>603</td>\n    </tr>\n    <tr>\n      <th>2</th>\n      <td>Immigrants</td>\n      <td>Foreigners</td>\n      <td>Algeria</td>\n      <td>903</td>\n      <td>Africa</td>\n      <td>912</td>\n      <td>Northern Africa</td>\n      <td>902</td>\n      <td>Developing regions</td>\n      <td>80</td>\n      <td>...</td>\n      <td>3616</td>\n      <td>3626</td>\n      <td>4807</td>\n      <td>3623</td>\n      <td>4005</td>\n      <td>5393</td>\n      <td>4752</td>\n      <td>4325</td>\n      <td>3774</td>\n      <td>4331</td>\n    </tr>\n    <tr>\n      <th>3</th>\n      <td>Immigrants</td>\n      <td>Foreigners</td>\n      <td>American Samoa</td>\n      <td>909</td>\n      <td>Oceania</td>\n      <td>957</td>\n      <td>Polynesia</td>\n      <td>902</td>\n      <td>Developing regions</td>\n      <td>0</td>\n      <td>...</td>\n      <td>0</td>\n      <td>0</td>\n      <td>1</td>\n      <td>0</td>\n      <td>0</td>\n      <td>0</td>\n      <td>0</td>\n      <td>0</td>\n      <td>0</td>\n      <td>0</td>\n    </tr>\n    <tr>\n      <th>4</th>\n      <td>Immigrants</td>\n      <td>Foreigners</td>\n      <td>Andorra</td>\n      <td>908</td>\n      <td>Europe</td>\n      <td>925</td>\n      <td>Southern Europe</td>\n      <td>901</td>\n      <td>Developed regions</td>\n      <td>0</td>\n      <td>...</td>\n      <td>0</td>\n      <td>0</td>\n      <td>1</td>\n      <td>1</td>\n      <td>0</td>\n      <td>0</td>\n      <td>0</td>\n      <td>0</td>\n      <td>1</td>\n      <td>1</td>\n    </tr>\n  </tbody>\n</table>\n<p>5 rows \u00d7 43 columns</p>\n</div>",
                        "text/plain": "         Type    Coverage          OdName  AREA AreaName   REG  \\\n0  Immigrants  Foreigners     Afghanistan   935     Asia  5501   \n1  Immigrants  Foreigners         Albania   908   Europe   925   \n2  Immigrants  Foreigners         Algeria   903   Africa   912   \n3  Immigrants  Foreigners  American Samoa   909  Oceania   957   \n4  Immigrants  Foreigners         Andorra   908   Europe   925   \n\n           RegName  DEV             DevName  1980  ...  2004  2005  2006  \\\n0    Southern Asia  902  Developing regions    16  ...  2978  3436  3009   \n1  Southern Europe  901   Developed regions     1  ...  1450  1223   856   \n2  Northern Africa  902  Developing regions    80  ...  3616  3626  4807   \n3        Polynesia  902  Developing regions     0  ...     0     0     1   \n4  Southern Europe  901   Developed regions     0  ...     0     0     1   \n\n   2007  2008  2009  2010  2011  2012  2013  \n0  2652  2111  1746  1758  2203  2635  2004  \n1   702   560   716   561   539   620   603  \n2  3623  4005  5393  4752  4325  3774  4331  \n3     0     0     0     0     0     0     0  \n4     1     0     0     0     0     1     1  \n\n[5 rows x 43 columns]"
                    },
                    "execution_count": 5,
                    "metadata": {},
                    "output_type": "execute_result"
                }
            ],
            "source": "df_can.head()\n# tip: You can specify the number of rows you'd like to see as follows: df_can.head(10) "
        },
        {
            "cell_type": "markdown",
            "metadata": {
                "deletable": true,
                "editable": true
            },
            "source": "We can also veiw the bottom 5 rows of the dataset using the `tail()` function."
        },
        {
            "cell_type": "code",
            "execution_count": 6,
            "metadata": {
                "collapsed": false,
                "deletable": true,
                "editable": true,
                "jupyter": {
                    "outputs_hidden": false
                }
            },
            "outputs": [
                {
                    "data": {
                        "text/html": "<div>\n<style scoped>\n    .dataframe tbody tr th:only-of-type {\n        vertical-align: middle;\n    }\n\n    .dataframe tbody tr th {\n        vertical-align: top;\n    }\n\n    .dataframe thead th {\n        text-align: right;\n    }\n</style>\n<table border=\"1\" class=\"dataframe\">\n  <thead>\n    <tr style=\"text-align: right;\">\n      <th></th>\n      <th>Type</th>\n      <th>Coverage</th>\n      <th>OdName</th>\n      <th>AREA</th>\n      <th>AreaName</th>\n      <th>REG</th>\n      <th>RegName</th>\n      <th>DEV</th>\n      <th>DevName</th>\n      <th>1980</th>\n      <th>...</th>\n      <th>2004</th>\n      <th>2005</th>\n      <th>2006</th>\n      <th>2007</th>\n      <th>2008</th>\n      <th>2009</th>\n      <th>2010</th>\n      <th>2011</th>\n      <th>2012</th>\n      <th>2013</th>\n    </tr>\n  </thead>\n  <tbody>\n    <tr>\n      <th>190</th>\n      <td>Immigrants</td>\n      <td>Foreigners</td>\n      <td>Viet Nam</td>\n      <td>935</td>\n      <td>Asia</td>\n      <td>920</td>\n      <td>South-Eastern Asia</td>\n      <td>902</td>\n      <td>Developing regions</td>\n      <td>1191</td>\n      <td>...</td>\n      <td>1816</td>\n      <td>1852</td>\n      <td>3153</td>\n      <td>2574</td>\n      <td>1784</td>\n      <td>2171</td>\n      <td>1942</td>\n      <td>1723</td>\n      <td>1731</td>\n      <td>2112</td>\n    </tr>\n    <tr>\n      <th>191</th>\n      <td>Immigrants</td>\n      <td>Foreigners</td>\n      <td>Western Sahara</td>\n      <td>903</td>\n      <td>Africa</td>\n      <td>912</td>\n      <td>Northern Africa</td>\n      <td>902</td>\n      <td>Developing regions</td>\n      <td>0</td>\n      <td>...</td>\n      <td>0</td>\n      <td>0</td>\n      <td>1</td>\n      <td>0</td>\n      <td>0</td>\n      <td>0</td>\n      <td>0</td>\n      <td>0</td>\n      <td>0</td>\n      <td>0</td>\n    </tr>\n    <tr>\n      <th>192</th>\n      <td>Immigrants</td>\n      <td>Foreigners</td>\n      <td>Yemen</td>\n      <td>935</td>\n      <td>Asia</td>\n      <td>922</td>\n      <td>Western Asia</td>\n      <td>902</td>\n      <td>Developing regions</td>\n      <td>1</td>\n      <td>...</td>\n      <td>124</td>\n      <td>161</td>\n      <td>140</td>\n      <td>122</td>\n      <td>133</td>\n      <td>128</td>\n      <td>211</td>\n      <td>160</td>\n      <td>174</td>\n      <td>217</td>\n    </tr>\n    <tr>\n      <th>193</th>\n      <td>Immigrants</td>\n      <td>Foreigners</td>\n      <td>Zambia</td>\n      <td>903</td>\n      <td>Africa</td>\n      <td>910</td>\n      <td>Eastern Africa</td>\n      <td>902</td>\n      <td>Developing regions</td>\n      <td>11</td>\n      <td>...</td>\n      <td>56</td>\n      <td>91</td>\n      <td>77</td>\n      <td>71</td>\n      <td>64</td>\n      <td>60</td>\n      <td>102</td>\n      <td>69</td>\n      <td>46</td>\n      <td>59</td>\n    </tr>\n    <tr>\n      <th>194</th>\n      <td>Immigrants</td>\n      <td>Foreigners</td>\n      <td>Zimbabwe</td>\n      <td>903</td>\n      <td>Africa</td>\n      <td>910</td>\n      <td>Eastern Africa</td>\n      <td>902</td>\n      <td>Developing regions</td>\n      <td>72</td>\n      <td>...</td>\n      <td>1450</td>\n      <td>615</td>\n      <td>454</td>\n      <td>663</td>\n      <td>611</td>\n      <td>508</td>\n      <td>494</td>\n      <td>434</td>\n      <td>437</td>\n      <td>407</td>\n    </tr>\n  </tbody>\n</table>\n<p>5 rows \u00d7 43 columns</p>\n</div>",
                        "text/plain": "           Type    Coverage          OdName  AREA AreaName  REG  \\\n190  Immigrants  Foreigners        Viet Nam   935     Asia  920   \n191  Immigrants  Foreigners  Western Sahara   903   Africa  912   \n192  Immigrants  Foreigners           Yemen   935     Asia  922   \n193  Immigrants  Foreigners          Zambia   903   Africa  910   \n194  Immigrants  Foreigners        Zimbabwe   903   Africa  910   \n\n                RegName  DEV             DevName  1980  ...  2004  2005  2006  \\\n190  South-Eastern Asia  902  Developing regions  1191  ...  1816  1852  3153   \n191     Northern Africa  902  Developing regions     0  ...     0     0     1   \n192        Western Asia  902  Developing regions     1  ...   124   161   140   \n193      Eastern Africa  902  Developing regions    11  ...    56    91    77   \n194      Eastern Africa  902  Developing regions    72  ...  1450   615   454   \n\n     2007  2008  2009  2010  2011  2012  2013  \n190  2574  1784  2171  1942  1723  1731  2112  \n191     0     0     0     0     0     0     0  \n192   122   133   128   211   160   174   217  \n193    71    64    60   102    69    46    59  \n194   663   611   508   494   434   437   407  \n\n[5 rows x 43 columns]"
                    },
                    "execution_count": 6,
                    "metadata": {},
                    "output_type": "execute_result"
                }
            ],
            "source": "df_can.tail()"
        },
        {
            "cell_type": "markdown",
            "metadata": {
                "deletable": true,
                "editable": true
            },
            "source": "When analyzing a dataset, it's always a good idea to start by getting basic information about your dataframe. We can do this by using the `info()` method."
        },
        {
            "cell_type": "code",
            "execution_count": 7,
            "metadata": {
                "collapsed": false,
                "deletable": true,
                "editable": true,
                "jupyter": {
                    "outputs_hidden": false
                },
                "scrolled": true
            },
            "outputs": [
                {
                    "name": "stdout",
                    "output_type": "stream",
                    "text": "<class 'pandas.core.frame.DataFrame'>\nRangeIndex: 195 entries, 0 to 194\nData columns (total 43 columns):\n #   Column    Non-Null Count  Dtype \n---  ------    --------------  ----- \n 0   Type      195 non-null    object\n 1   Coverage  195 non-null    object\n 2   OdName    195 non-null    object\n 3   AREA      195 non-null    int64 \n 4   AreaName  195 non-null    object\n 5   REG       195 non-null    int64 \n 6   RegName   195 non-null    object\n 7   DEV       195 non-null    int64 \n 8   DevName   195 non-null    object\n 9   1980      195 non-null    int64 \n 10  1981      195 non-null    int64 \n 11  1982      195 non-null    int64 \n 12  1983      195 non-null    int64 \n 13  1984      195 non-null    int64 \n 14  1985      195 non-null    int64 \n 15  1986      195 non-null    int64 \n 16  1987      195 non-null    int64 \n 17  1988      195 non-null    int64 \n 18  1989      195 non-null    int64 \n 19  1990      195 non-null    int64 \n 20  1991      195 non-null    int64 \n 21  1992      195 non-null    int64 \n 22  1993      195 non-null    int64 \n 23  1994      195 non-null    int64 \n 24  1995      195 non-null    int64 \n 25  1996      195 non-null    int64 \n 26  1997      195 non-null    int64 \n 27  1998      195 non-null    int64 \n 28  1999      195 non-null    int64 \n 29  2000      195 non-null    int64 \n 30  2001      195 non-null    int64 \n 31  2002      195 non-null    int64 \n 32  2003      195 non-null    int64 \n 33  2004      195 non-null    int64 \n 34  2005      195 non-null    int64 \n 35  2006      195 non-null    int64 \n 36  2007      195 non-null    int64 \n 37  2008      195 non-null    int64 \n 38  2009      195 non-null    int64 \n 39  2010      195 non-null    int64 \n 40  2011      195 non-null    int64 \n 41  2012      195 non-null    int64 \n 42  2013      195 non-null    int64 \ndtypes: int64(37), object(6)\nmemory usage: 65.6+ KB\n"
                }
            ],
            "source": "df_can.info()"
        },
        {
            "cell_type": "markdown",
            "metadata": {
                "deletable": true,
                "editable": true
            },
            "source": "To get the list of column headers we can call upon the dataframe's `.columns` parameter."
        },
        {
            "cell_type": "code",
            "execution_count": 8,
            "metadata": {
                "collapsed": false,
                "deletable": true,
                "editable": true,
                "jupyter": {
                    "outputs_hidden": false
                }
            },
            "outputs": [
                {
                    "data": {
                        "text/plain": "array(['Type', 'Coverage', 'OdName', 'AREA', 'AreaName', 'REG', 'RegName',\n       'DEV', 'DevName', 1980, 1981, 1982, 1983, 1984, 1985, 1986, 1987,\n       1988, 1989, 1990, 1991, 1992, 1993, 1994, 1995, 1996, 1997, 1998,\n       1999, 2000, 2001, 2002, 2003, 2004, 2005, 2006, 2007, 2008, 2009,\n       2010, 2011, 2012, 2013], dtype=object)"
                    },
                    "execution_count": 8,
                    "metadata": {},
                    "output_type": "execute_result"
                }
            ],
            "source": "df_can.columns.values "
        },
        {
            "cell_type": "markdown",
            "metadata": {
                "deletable": true,
                "editable": true
            },
            "source": "Similarly, to get the list of indicies we use the `.index` parameter."
        },
        {
            "cell_type": "code",
            "execution_count": 9,
            "metadata": {
                "collapsed": false,
                "deletable": true,
                "editable": true,
                "jupyter": {
                    "outputs_hidden": false
                },
                "scrolled": true
            },
            "outputs": [
                {
                    "data": {
                        "text/plain": "array([  0,   1,   2,   3,   4,   5,   6,   7,   8,   9,  10,  11,  12,\n        13,  14,  15,  16,  17,  18,  19,  20,  21,  22,  23,  24,  25,\n        26,  27,  28,  29,  30,  31,  32,  33,  34,  35,  36,  37,  38,\n        39,  40,  41,  42,  43,  44,  45,  46,  47,  48,  49,  50,  51,\n        52,  53,  54,  55,  56,  57,  58,  59,  60,  61,  62,  63,  64,\n        65,  66,  67,  68,  69,  70,  71,  72,  73,  74,  75,  76,  77,\n        78,  79,  80,  81,  82,  83,  84,  85,  86,  87,  88,  89,  90,\n        91,  92,  93,  94,  95,  96,  97,  98,  99, 100, 101, 102, 103,\n       104, 105, 106, 107, 108, 109, 110, 111, 112, 113, 114, 115, 116,\n       117, 118, 119, 120, 121, 122, 123, 124, 125, 126, 127, 128, 129,\n       130, 131, 132, 133, 134, 135, 136, 137, 138, 139, 140, 141, 142,\n       143, 144, 145, 146, 147, 148, 149, 150, 151, 152, 153, 154, 155,\n       156, 157, 158, 159, 160, 161, 162, 163, 164, 165, 166, 167, 168,\n       169, 170, 171, 172, 173, 174, 175, 176, 177, 178, 179, 180, 181,\n       182, 183, 184, 185, 186, 187, 188, 189, 190, 191, 192, 193, 194])"
                    },
                    "execution_count": 9,
                    "metadata": {},
                    "output_type": "execute_result"
                }
            ],
            "source": "df_can.index.values"
        },
        {
            "cell_type": "markdown",
            "metadata": {
                "deletable": true,
                "editable": true
            },
            "source": "Note: The default type of index and columns is NOT list."
        },
        {
            "cell_type": "code",
            "execution_count": 10,
            "metadata": {
                "collapsed": false,
                "deletable": true,
                "editable": true,
                "jupyter": {
                    "outputs_hidden": false
                }
            },
            "outputs": [
                {
                    "name": "stdout",
                    "output_type": "stream",
                    "text": "<class 'pandas.core.indexes.base.Index'>\n<class 'pandas.core.indexes.range.RangeIndex'>\n"
                }
            ],
            "source": "print(type(df_can.columns))\nprint(type(df_can.index))"
        },
        {
            "cell_type": "markdown",
            "metadata": {
                "deletable": true,
                "editable": true
            },
            "source": "To get the index and columns as lists, we can use the `tolist()` method."
        },
        {
            "cell_type": "code",
            "execution_count": 11,
            "metadata": {
                "collapsed": false,
                "deletable": true,
                "editable": true,
                "jupyter": {
                    "outputs_hidden": false
                }
            },
            "outputs": [
                {
                    "name": "stdout",
                    "output_type": "stream",
                    "text": "<class 'list'>\n<class 'list'>\n"
                }
            ],
            "source": "df_can.columns.tolist()\ndf_can.index.tolist()\n\nprint (type(df_can.columns.tolist()))\nprint (type(df_can.index.tolist()))"
        },
        {
            "cell_type": "markdown",
            "metadata": {
                "deletable": true,
                "editable": true
            },
            "source": "To view the dimensions of the dataframe, we use the `.shape` parameter."
        },
        {
            "cell_type": "code",
            "execution_count": 12,
            "metadata": {
                "collapsed": false,
                "deletable": true,
                "editable": true,
                "jupyter": {
                    "outputs_hidden": false
                }
            },
            "outputs": [
                {
                    "data": {
                        "text/plain": "(195, 43)"
                    },
                    "execution_count": 12,
                    "metadata": {},
                    "output_type": "execute_result"
                }
            ],
            "source": "# size of dataframe (rows, columns)\ndf_can.shape    "
        },
        {
            "cell_type": "markdown",
            "metadata": {
                "deletable": true,
                "editable": true
            },
            "source": "Note: The main types stored in *pandas* objects are *float*, *int*, *bool*, *datetime64[ns]* and *datetime64[ns, tz] (in >= 0.17.0)*, *timedelta[ns]*, *category (in >= 0.15.0)*, and *object* (string). In addition these dtypes have item sizes, e.g. int64 and int32. "
        },
        {
            "cell_type": "markdown",
            "metadata": {
                "deletable": true,
                "editable": true
            },
            "source": "Let us clean the data set to remove a few unnecessary columns. We can use *pandas* `drop()` method as follows:"
        },
        {
            "cell_type": "code",
            "execution_count": 13,
            "metadata": {
                "collapsed": false,
                "deletable": true,
                "editable": true,
                "jupyter": {
                    "outputs_hidden": false
                }
            },
            "outputs": [
                {
                    "data": {
                        "text/html": "<div>\n<style scoped>\n    .dataframe tbody tr th:only-of-type {\n        vertical-align: middle;\n    }\n\n    .dataframe tbody tr th {\n        vertical-align: top;\n    }\n\n    .dataframe thead th {\n        text-align: right;\n    }\n</style>\n<table border=\"1\" class=\"dataframe\">\n  <thead>\n    <tr style=\"text-align: right;\">\n      <th></th>\n      <th>OdName</th>\n      <th>AreaName</th>\n      <th>RegName</th>\n      <th>DevName</th>\n      <th>1980</th>\n      <th>1981</th>\n      <th>1982</th>\n      <th>1983</th>\n      <th>1984</th>\n      <th>1985</th>\n      <th>...</th>\n      <th>2004</th>\n      <th>2005</th>\n      <th>2006</th>\n      <th>2007</th>\n      <th>2008</th>\n      <th>2009</th>\n      <th>2010</th>\n      <th>2011</th>\n      <th>2012</th>\n      <th>2013</th>\n    </tr>\n  </thead>\n  <tbody>\n    <tr>\n      <th>0</th>\n      <td>Afghanistan</td>\n      <td>Asia</td>\n      <td>Southern Asia</td>\n      <td>Developing regions</td>\n      <td>16</td>\n      <td>39</td>\n      <td>39</td>\n      <td>47</td>\n      <td>71</td>\n      <td>340</td>\n      <td>...</td>\n      <td>2978</td>\n      <td>3436</td>\n      <td>3009</td>\n      <td>2652</td>\n      <td>2111</td>\n      <td>1746</td>\n      <td>1758</td>\n      <td>2203</td>\n      <td>2635</td>\n      <td>2004</td>\n    </tr>\n    <tr>\n      <th>1</th>\n      <td>Albania</td>\n      <td>Europe</td>\n      <td>Southern Europe</td>\n      <td>Developed regions</td>\n      <td>1</td>\n      <td>0</td>\n      <td>0</td>\n      <td>0</td>\n      <td>0</td>\n      <td>0</td>\n      <td>...</td>\n      <td>1450</td>\n      <td>1223</td>\n      <td>856</td>\n      <td>702</td>\n      <td>560</td>\n      <td>716</td>\n      <td>561</td>\n      <td>539</td>\n      <td>620</td>\n      <td>603</td>\n    </tr>\n  </tbody>\n</table>\n<p>2 rows \u00d7 38 columns</p>\n</div>",
                        "text/plain": "        OdName AreaName          RegName             DevName  1980  1981  \\\n0  Afghanistan     Asia    Southern Asia  Developing regions    16    39   \n1      Albania   Europe  Southern Europe   Developed regions     1     0   \n\n   1982  1983  1984  1985  ...  2004  2005  2006  2007  2008  2009  2010  \\\n0    39    47    71   340  ...  2978  3436  3009  2652  2111  1746  1758   \n1     0     0     0     0  ...  1450  1223   856   702   560   716   561   \n\n   2011  2012  2013  \n0  2203  2635  2004  \n1   539   620   603  \n\n[2 rows x 38 columns]"
                    },
                    "execution_count": 13,
                    "metadata": {},
                    "output_type": "execute_result"
                }
            ],
            "source": "# in pandas axis=0 represents rows (default) and axis=1 represents columns.\ndf_can.drop(['AREA','REG','DEV','Type','Coverage'], axis=1, inplace=True)\ndf_can.head(2)"
        },
        {
            "cell_type": "markdown",
            "metadata": {
                "deletable": true,
                "editable": true
            },
            "source": "Let us rename the columns so that they make sense. We can use `rename()` method by passing in a dictionary of old and new names as follows:"
        },
        {
            "cell_type": "code",
            "execution_count": 14,
            "metadata": {
                "collapsed": false,
                "deletable": true,
                "editable": true,
                "jupyter": {
                    "outputs_hidden": false
                }
            },
            "outputs": [
                {
                    "data": {
                        "text/plain": "Index([  'Country', 'Continent',    'Region',   'DevName',        1980,\n              1981,        1982,        1983,        1984,        1985,\n              1986,        1987,        1988,        1989,        1990,\n              1991,        1992,        1993,        1994,        1995,\n              1996,        1997,        1998,        1999,        2000,\n              2001,        2002,        2003,        2004,        2005,\n              2006,        2007,        2008,        2009,        2010,\n              2011,        2012,        2013],\n      dtype='object')"
                    },
                    "execution_count": 14,
                    "metadata": {},
                    "output_type": "execute_result"
                }
            ],
            "source": "df_can.rename(columns={'OdName':'Country', 'AreaName':'Continent', 'RegName':'Region'}, inplace=True)\ndf_can.columns"
        },
        {
            "cell_type": "markdown",
            "metadata": {
                "deletable": true,
                "editable": true
            },
            "source": "We will also add a 'Total' column that sums up the total immigrants by country over the entire period 1980 - 2013, as follows:"
        },
        {
            "cell_type": "code",
            "execution_count": 15,
            "metadata": {
                "collapsed": false,
                "deletable": true,
                "editable": true,
                "jupyter": {
                    "outputs_hidden": false
                }
            },
            "outputs": [],
            "source": "df_can['Total'] = df_can.sum(axis=1)"
        },
        {
            "cell_type": "markdown",
            "metadata": {
                "deletable": true,
                "editable": true
            },
            "source": "We can check to see how many null objects we have in the dataset as follows:"
        },
        {
            "cell_type": "code",
            "execution_count": 16,
            "metadata": {
                "collapsed": false,
                "deletable": true,
                "editable": true,
                "jupyter": {
                    "outputs_hidden": false
                },
                "scrolled": true
            },
            "outputs": [
                {
                    "data": {
                        "text/plain": "Country      0\nContinent    0\nRegion       0\nDevName      0\n1980         0\n1981         0\n1982         0\n1983         0\n1984         0\n1985         0\n1986         0\n1987         0\n1988         0\n1989         0\n1990         0\n1991         0\n1992         0\n1993         0\n1994         0\n1995         0\n1996         0\n1997         0\n1998         0\n1999         0\n2000         0\n2001         0\n2002         0\n2003         0\n2004         0\n2005         0\n2006         0\n2007         0\n2008         0\n2009         0\n2010         0\n2011         0\n2012         0\n2013         0\nTotal        0\ndtype: int64"
                    },
                    "execution_count": 16,
                    "metadata": {},
                    "output_type": "execute_result"
                }
            ],
            "source": "df_can.isnull().sum()"
        },
        {
            "cell_type": "markdown",
            "metadata": {
                "deletable": true,
                "editable": true
            },
            "source": "Finally, let's view a quick summary of each column in our dataframe using the `describe()` method."
        },
        {
            "cell_type": "code",
            "execution_count": 17,
            "metadata": {
                "collapsed": false,
                "deletable": true,
                "editable": true,
                "jupyter": {
                    "outputs_hidden": false
                }
            },
            "outputs": [
                {
                    "data": {
                        "text/html": "<div>\n<style scoped>\n    .dataframe tbody tr th:only-of-type {\n        vertical-align: middle;\n    }\n\n    .dataframe tbody tr th {\n        vertical-align: top;\n    }\n\n    .dataframe thead th {\n        text-align: right;\n    }\n</style>\n<table border=\"1\" class=\"dataframe\">\n  <thead>\n    <tr style=\"text-align: right;\">\n      <th></th>\n      <th>1980</th>\n      <th>1981</th>\n      <th>1982</th>\n      <th>1983</th>\n      <th>1984</th>\n      <th>1985</th>\n      <th>1986</th>\n      <th>1987</th>\n      <th>1988</th>\n      <th>1989</th>\n      <th>...</th>\n      <th>2005</th>\n      <th>2006</th>\n      <th>2007</th>\n      <th>2008</th>\n      <th>2009</th>\n      <th>2010</th>\n      <th>2011</th>\n      <th>2012</th>\n      <th>2013</th>\n      <th>Total</th>\n    </tr>\n  </thead>\n  <tbody>\n    <tr>\n      <th>count</th>\n      <td>195.000000</td>\n      <td>195.000000</td>\n      <td>195.000000</td>\n      <td>195.000000</td>\n      <td>195.000000</td>\n      <td>195.000000</td>\n      <td>195.000000</td>\n      <td>195.000000</td>\n      <td>195.000000</td>\n      <td>195.000000</td>\n      <td>...</td>\n      <td>195.000000</td>\n      <td>195.000000</td>\n      <td>195.000000</td>\n      <td>195.000000</td>\n      <td>195.000000</td>\n      <td>195.000000</td>\n      <td>195.000000</td>\n      <td>195.000000</td>\n      <td>195.000000</td>\n      <td>195.000000</td>\n    </tr>\n    <tr>\n      <th>mean</th>\n      <td>508.394872</td>\n      <td>566.989744</td>\n      <td>534.723077</td>\n      <td>387.435897</td>\n      <td>376.497436</td>\n      <td>358.861538</td>\n      <td>441.271795</td>\n      <td>691.133333</td>\n      <td>714.389744</td>\n      <td>843.241026</td>\n      <td>...</td>\n      <td>1320.292308</td>\n      <td>1266.958974</td>\n      <td>1191.820513</td>\n      <td>1246.394872</td>\n      <td>1275.733333</td>\n      <td>1420.287179</td>\n      <td>1262.533333</td>\n      <td>1313.958974</td>\n      <td>1320.702564</td>\n      <td>32867.451282</td>\n    </tr>\n    <tr>\n      <th>std</th>\n      <td>1949.588546</td>\n      <td>2152.643752</td>\n      <td>1866.997511</td>\n      <td>1204.333597</td>\n      <td>1198.246371</td>\n      <td>1079.309600</td>\n      <td>1225.576630</td>\n      <td>2109.205607</td>\n      <td>2443.606788</td>\n      <td>2555.048874</td>\n      <td>...</td>\n      <td>4425.957828</td>\n      <td>3926.717747</td>\n      <td>3443.542409</td>\n      <td>3694.573544</td>\n      <td>3829.630424</td>\n      <td>4462.946328</td>\n      <td>4030.084313</td>\n      <td>4247.555161</td>\n      <td>4237.951988</td>\n      <td>91785.498686</td>\n    </tr>\n    <tr>\n      <th>min</th>\n      <td>0.000000</td>\n      <td>0.000000</td>\n      <td>0.000000</td>\n      <td>0.000000</td>\n      <td>0.000000</td>\n      <td>0.000000</td>\n      <td>0.000000</td>\n      <td>0.000000</td>\n      <td>0.000000</td>\n      <td>0.000000</td>\n      <td>...</td>\n      <td>0.000000</td>\n      <td>0.000000</td>\n      <td>0.000000</td>\n      <td>0.000000</td>\n      <td>0.000000</td>\n      <td>0.000000</td>\n      <td>0.000000</td>\n      <td>0.000000</td>\n      <td>0.000000</td>\n      <td>1.000000</td>\n    </tr>\n    <tr>\n      <th>25%</th>\n      <td>0.000000</td>\n      <td>0.000000</td>\n      <td>0.000000</td>\n      <td>0.000000</td>\n      <td>0.000000</td>\n      <td>0.000000</td>\n      <td>0.500000</td>\n      <td>0.500000</td>\n      <td>1.000000</td>\n      <td>1.000000</td>\n      <td>...</td>\n      <td>28.500000</td>\n      <td>25.000000</td>\n      <td>31.000000</td>\n      <td>31.000000</td>\n      <td>36.000000</td>\n      <td>40.500000</td>\n      <td>37.500000</td>\n      <td>42.500000</td>\n      <td>45.000000</td>\n      <td>952.000000</td>\n    </tr>\n    <tr>\n      <th>50%</th>\n      <td>13.000000</td>\n      <td>10.000000</td>\n      <td>11.000000</td>\n      <td>12.000000</td>\n      <td>13.000000</td>\n      <td>17.000000</td>\n      <td>18.000000</td>\n      <td>26.000000</td>\n      <td>34.000000</td>\n      <td>44.000000</td>\n      <td>...</td>\n      <td>210.000000</td>\n      <td>218.000000</td>\n      <td>198.000000</td>\n      <td>205.000000</td>\n      <td>214.000000</td>\n      <td>211.000000</td>\n      <td>179.000000</td>\n      <td>233.000000</td>\n      <td>213.000000</td>\n      <td>5018.000000</td>\n    </tr>\n    <tr>\n      <th>75%</th>\n      <td>251.500000</td>\n      <td>295.500000</td>\n      <td>275.000000</td>\n      <td>173.000000</td>\n      <td>181.000000</td>\n      <td>197.000000</td>\n      <td>254.000000</td>\n      <td>434.000000</td>\n      <td>409.000000</td>\n      <td>508.500000</td>\n      <td>...</td>\n      <td>832.000000</td>\n      <td>842.000000</td>\n      <td>899.000000</td>\n      <td>934.500000</td>\n      <td>888.000000</td>\n      <td>932.000000</td>\n      <td>772.000000</td>\n      <td>783.000000</td>\n      <td>796.000000</td>\n      <td>22239.500000</td>\n    </tr>\n    <tr>\n      <th>max</th>\n      <td>22045.000000</td>\n      <td>24796.000000</td>\n      <td>20620.000000</td>\n      <td>10015.000000</td>\n      <td>10170.000000</td>\n      <td>9564.000000</td>\n      <td>9470.000000</td>\n      <td>21337.000000</td>\n      <td>27359.000000</td>\n      <td>23795.000000</td>\n      <td>...</td>\n      <td>42584.000000</td>\n      <td>33848.000000</td>\n      <td>28742.000000</td>\n      <td>30037.000000</td>\n      <td>29622.000000</td>\n      <td>38617.000000</td>\n      <td>36765.000000</td>\n      <td>34315.000000</td>\n      <td>34129.000000</td>\n      <td>691904.000000</td>\n    </tr>\n  </tbody>\n</table>\n<p>8 rows \u00d7 35 columns</p>\n</div>",
                        "text/plain": "               1980          1981          1982          1983          1984  \\\ncount    195.000000    195.000000    195.000000    195.000000    195.000000   \nmean     508.394872    566.989744    534.723077    387.435897    376.497436   \nstd     1949.588546   2152.643752   1866.997511   1204.333597   1198.246371   \nmin        0.000000      0.000000      0.000000      0.000000      0.000000   \n25%        0.000000      0.000000      0.000000      0.000000      0.000000   \n50%       13.000000     10.000000     11.000000     12.000000     13.000000   \n75%      251.500000    295.500000    275.000000    173.000000    181.000000   \nmax    22045.000000  24796.000000  20620.000000  10015.000000  10170.000000   \n\n              1985         1986          1987          1988          1989  \\\ncount   195.000000   195.000000    195.000000    195.000000    195.000000   \nmean    358.861538   441.271795    691.133333    714.389744    843.241026   \nstd    1079.309600  1225.576630   2109.205607   2443.606788   2555.048874   \nmin       0.000000     0.000000      0.000000      0.000000      0.000000   \n25%       0.000000     0.500000      0.500000      1.000000      1.000000   \n50%      17.000000    18.000000     26.000000     34.000000     44.000000   \n75%     197.000000   254.000000    434.000000    409.000000    508.500000   \nmax    9564.000000  9470.000000  21337.000000  27359.000000  23795.000000   \n\n       ...          2005          2006          2007          2008  \\\ncount  ...    195.000000    195.000000    195.000000    195.000000   \nmean   ...   1320.292308   1266.958974   1191.820513   1246.394872   \nstd    ...   4425.957828   3926.717747   3443.542409   3694.573544   \nmin    ...      0.000000      0.000000      0.000000      0.000000   \n25%    ...     28.500000     25.000000     31.000000     31.000000   \n50%    ...    210.000000    218.000000    198.000000    205.000000   \n75%    ...    832.000000    842.000000    899.000000    934.500000   \nmax    ...  42584.000000  33848.000000  28742.000000  30037.000000   \n\n               2009          2010          2011          2012          2013  \\\ncount    195.000000    195.000000    195.000000    195.000000    195.000000   \nmean    1275.733333   1420.287179   1262.533333   1313.958974   1320.702564   \nstd     3829.630424   4462.946328   4030.084313   4247.555161   4237.951988   \nmin        0.000000      0.000000      0.000000      0.000000      0.000000   \n25%       36.000000     40.500000     37.500000     42.500000     45.000000   \n50%      214.000000    211.000000    179.000000    233.000000    213.000000   \n75%      888.000000    932.000000    772.000000    783.000000    796.000000   \nmax    29622.000000  38617.000000  36765.000000  34315.000000  34129.000000   \n\n               Total  \ncount     195.000000  \nmean    32867.451282  \nstd     91785.498686  \nmin         1.000000  \n25%       952.000000  \n50%      5018.000000  \n75%     22239.500000  \nmax    691904.000000  \n\n[8 rows x 35 columns]"
                    },
                    "execution_count": 17,
                    "metadata": {},
                    "output_type": "execute_result"
                }
            ],
            "source": "df_can.describe()"
        },
        {
            "cell_type": "markdown",
            "metadata": {
                "deletable": true,
                "editable": true
            },
            "source": "---\n## *pandas* Intermediate: Indexing and Selection (slicing)<a id=\"6\"></a>\n"
        },
        {
            "cell_type": "markdown",
            "metadata": {
                "deletable": true,
                "editable": true
            },
            "source": "### Select Column\n**There are two ways to filter on a column name:**\n\nMethod 1: Quick and easy, but only works if the column name does NOT have spaces or special characters.\n```python\n    df.column_name \n        (returns series)\n```\n\nMethod 2: More robust, and can filter on multiple columns.\n\n```python\n    df['column']  \n        (returns series)\n```\n\n```python \n    df[['column 1', 'column 2']] \n        (returns dataframe)\n```\n---"
        },
        {
            "cell_type": "markdown",
            "metadata": {
                "deletable": true,
                "editable": true
            },
            "source": "Example: Let's try filtering on the list of countries ('Country')."
        },
        {
            "cell_type": "code",
            "execution_count": 18,
            "metadata": {
                "collapsed": false,
                "deletable": true,
                "editable": true,
                "jupyter": {
                    "outputs_hidden": false
                },
                "scrolled": true
            },
            "outputs": [
                {
                    "data": {
                        "text/plain": "0         Afghanistan\n1             Albania\n2             Algeria\n3      American Samoa\n4             Andorra\n            ...      \n190          Viet Nam\n191    Western Sahara\n192             Yemen\n193            Zambia\n194          Zimbabwe\nName: Country, Length: 195, dtype: object"
                    },
                    "execution_count": 18,
                    "metadata": {},
                    "output_type": "execute_result"
                }
            ],
            "source": "df_can.Country  # returns a series"
        },
        {
            "cell_type": "markdown",
            "metadata": {
                "deletable": true,
                "editable": true
            },
            "source": "Let's try filtering on the list of countries ('OdName') and the data for years: 1980 - 1985."
        },
        {
            "cell_type": "code",
            "execution_count": 19,
            "metadata": {
                "collapsed": false,
                "deletable": true,
                "editable": true,
                "jupyter": {
                    "outputs_hidden": false
                }
            },
            "outputs": [
                {
                    "data": {
                        "text/html": "<div>\n<style scoped>\n    .dataframe tbody tr th:only-of-type {\n        vertical-align: middle;\n    }\n\n    .dataframe tbody tr th {\n        vertical-align: top;\n    }\n\n    .dataframe thead th {\n        text-align: right;\n    }\n</style>\n<table border=\"1\" class=\"dataframe\">\n  <thead>\n    <tr style=\"text-align: right;\">\n      <th></th>\n      <th>Country</th>\n      <th>1980</th>\n      <th>1981</th>\n      <th>1982</th>\n      <th>1983</th>\n      <th>1984</th>\n      <th>1985</th>\n    </tr>\n  </thead>\n  <tbody>\n    <tr>\n      <th>0</th>\n      <td>Afghanistan</td>\n      <td>16</td>\n      <td>39</td>\n      <td>39</td>\n      <td>47</td>\n      <td>71</td>\n      <td>340</td>\n    </tr>\n    <tr>\n      <th>1</th>\n      <td>Albania</td>\n      <td>1</td>\n      <td>0</td>\n      <td>0</td>\n      <td>0</td>\n      <td>0</td>\n      <td>0</td>\n    </tr>\n    <tr>\n      <th>2</th>\n      <td>Algeria</td>\n      <td>80</td>\n      <td>67</td>\n      <td>71</td>\n      <td>69</td>\n      <td>63</td>\n      <td>44</td>\n    </tr>\n    <tr>\n      <th>3</th>\n      <td>American Samoa</td>\n      <td>0</td>\n      <td>1</td>\n      <td>0</td>\n      <td>0</td>\n      <td>0</td>\n      <td>0</td>\n    </tr>\n    <tr>\n      <th>4</th>\n      <td>Andorra</td>\n      <td>0</td>\n      <td>0</td>\n      <td>0</td>\n      <td>0</td>\n      <td>0</td>\n      <td>0</td>\n    </tr>\n    <tr>\n      <th>...</th>\n      <td>...</td>\n      <td>...</td>\n      <td>...</td>\n      <td>...</td>\n      <td>...</td>\n      <td>...</td>\n      <td>...</td>\n    </tr>\n    <tr>\n      <th>190</th>\n      <td>Viet Nam</td>\n      <td>1191</td>\n      <td>1829</td>\n      <td>2162</td>\n      <td>3404</td>\n      <td>7583</td>\n      <td>5907</td>\n    </tr>\n    <tr>\n      <th>191</th>\n      <td>Western Sahara</td>\n      <td>0</td>\n      <td>0</td>\n      <td>0</td>\n      <td>0</td>\n      <td>0</td>\n      <td>0</td>\n    </tr>\n    <tr>\n      <th>192</th>\n      <td>Yemen</td>\n      <td>1</td>\n      <td>2</td>\n      <td>1</td>\n      <td>6</td>\n      <td>0</td>\n      <td>18</td>\n    </tr>\n    <tr>\n      <th>193</th>\n      <td>Zambia</td>\n      <td>11</td>\n      <td>17</td>\n      <td>11</td>\n      <td>7</td>\n      <td>16</td>\n      <td>9</td>\n    </tr>\n    <tr>\n      <th>194</th>\n      <td>Zimbabwe</td>\n      <td>72</td>\n      <td>114</td>\n      <td>102</td>\n      <td>44</td>\n      <td>32</td>\n      <td>29</td>\n    </tr>\n  </tbody>\n</table>\n<p>195 rows \u00d7 7 columns</p>\n</div>",
                        "text/plain": "            Country  1980  1981  1982  1983  1984  1985\n0       Afghanistan    16    39    39    47    71   340\n1           Albania     1     0     0     0     0     0\n2           Algeria    80    67    71    69    63    44\n3    American Samoa     0     1     0     0     0     0\n4           Andorra     0     0     0     0     0     0\n..              ...   ...   ...   ...   ...   ...   ...\n190        Viet Nam  1191  1829  2162  3404  7583  5907\n191  Western Sahara     0     0     0     0     0     0\n192           Yemen     1     2     1     6     0    18\n193          Zambia    11    17    11     7    16     9\n194        Zimbabwe    72   114   102    44    32    29\n\n[195 rows x 7 columns]"
                    },
                    "execution_count": 19,
                    "metadata": {},
                    "output_type": "execute_result"
                }
            ],
            "source": "df_can[['Country', 1980, 1981, 1982, 1983, 1984, 1985]] #returns a dataframe\n# notice that 'Country' is string, and the years are integers. \n# for the sake of consistency, we will convert all column names to string later on."
        },
        {
            "cell_type": "markdown",
            "metadata": {
                "deletable": true,
                "editable": true
            },
            "source": "### Select Row\n\nThere are main 3 ways to select rows:\n\n```python\n    df.loc[label]        \n        #filters by the labels of the index/column\n    df.iloc[index]       \n        #filters by the positions of the index/column\n    df.ix[label/index]  \n        #filters by labels first (loc) but falls back to positions (iloc) if label is not found \n```"
        },
        {
            "cell_type": "markdown",
            "metadata": {
                "deletable": true,
                "editable": true
            },
            "source": "Before we proceed, notice that the defaul index of the dataset is a numeric range from 0 to 194. This makes it very difficult to do a query by a specific country. For example to search for data on Japan, we need to know the corressponding index value.\n\nThis can be fixed very easily by setting the 'Country' column as the index using `set_index()` method."
        },
        {
            "cell_type": "code",
            "execution_count": 20,
            "metadata": {
                "collapsed": false,
                "deletable": true,
                "editable": true,
                "jupyter": {
                    "outputs_hidden": false
                },
                "scrolled": true
            },
            "outputs": [],
            "source": "df_can.set_index('Country', inplace=True)\n# tip: The opposite of set is reset. So to reset the index, we can use df_can.reset_index()"
        },
        {
            "cell_type": "code",
            "execution_count": 21,
            "metadata": {
                "collapsed": false,
                "deletable": true,
                "editable": true,
                "jupyter": {
                    "outputs_hidden": false
                }
            },
            "outputs": [
                {
                    "data": {
                        "text/html": "<div>\n<style scoped>\n    .dataframe tbody tr th:only-of-type {\n        vertical-align: middle;\n    }\n\n    .dataframe tbody tr th {\n        vertical-align: top;\n    }\n\n    .dataframe thead th {\n        text-align: right;\n    }\n</style>\n<table border=\"1\" class=\"dataframe\">\n  <thead>\n    <tr style=\"text-align: right;\">\n      <th></th>\n      <th>Continent</th>\n      <th>Region</th>\n      <th>DevName</th>\n      <th>1980</th>\n      <th>1981</th>\n      <th>1982</th>\n      <th>1983</th>\n      <th>1984</th>\n      <th>1985</th>\n      <th>1986</th>\n      <th>...</th>\n      <th>2005</th>\n      <th>2006</th>\n      <th>2007</th>\n      <th>2008</th>\n      <th>2009</th>\n      <th>2010</th>\n      <th>2011</th>\n      <th>2012</th>\n      <th>2013</th>\n      <th>Total</th>\n    </tr>\n    <tr>\n      <th>Country</th>\n      <th></th>\n      <th></th>\n      <th></th>\n      <th></th>\n      <th></th>\n      <th></th>\n      <th></th>\n      <th></th>\n      <th></th>\n      <th></th>\n      <th></th>\n      <th></th>\n      <th></th>\n      <th></th>\n      <th></th>\n      <th></th>\n      <th></th>\n      <th></th>\n      <th></th>\n      <th></th>\n      <th></th>\n    </tr>\n  </thead>\n  <tbody>\n    <tr>\n      <th>Afghanistan</th>\n      <td>Asia</td>\n      <td>Southern Asia</td>\n      <td>Developing regions</td>\n      <td>16</td>\n      <td>39</td>\n      <td>39</td>\n      <td>47</td>\n      <td>71</td>\n      <td>340</td>\n      <td>496</td>\n      <td>...</td>\n      <td>3436</td>\n      <td>3009</td>\n      <td>2652</td>\n      <td>2111</td>\n      <td>1746</td>\n      <td>1758</td>\n      <td>2203</td>\n      <td>2635</td>\n      <td>2004</td>\n      <td>58639</td>\n    </tr>\n    <tr>\n      <th>Albania</th>\n      <td>Europe</td>\n      <td>Southern Europe</td>\n      <td>Developed regions</td>\n      <td>1</td>\n      <td>0</td>\n      <td>0</td>\n      <td>0</td>\n      <td>0</td>\n      <td>0</td>\n      <td>1</td>\n      <td>...</td>\n      <td>1223</td>\n      <td>856</td>\n      <td>702</td>\n      <td>560</td>\n      <td>716</td>\n      <td>561</td>\n      <td>539</td>\n      <td>620</td>\n      <td>603</td>\n      <td>15699</td>\n    </tr>\n    <tr>\n      <th>Algeria</th>\n      <td>Africa</td>\n      <td>Northern Africa</td>\n      <td>Developing regions</td>\n      <td>80</td>\n      <td>67</td>\n      <td>71</td>\n      <td>69</td>\n      <td>63</td>\n      <td>44</td>\n      <td>69</td>\n      <td>...</td>\n      <td>3626</td>\n      <td>4807</td>\n      <td>3623</td>\n      <td>4005</td>\n      <td>5393</td>\n      <td>4752</td>\n      <td>4325</td>\n      <td>3774</td>\n      <td>4331</td>\n      <td>69439</td>\n    </tr>\n  </tbody>\n</table>\n<p>3 rows \u00d7 38 columns</p>\n</div>",
                        "text/plain": "            Continent           Region             DevName  1980  1981  1982  \\\nCountry                                                                        \nAfghanistan      Asia    Southern Asia  Developing regions    16    39    39   \nAlbania        Europe  Southern Europe   Developed regions     1     0     0   \nAlgeria        Africa  Northern Africa  Developing regions    80    67    71   \n\n             1983  1984  1985  1986  ...  2005  2006  2007  2008  2009  2010  \\\nCountry                              ...                                       \nAfghanistan    47    71   340   496  ...  3436  3009  2652  2111  1746  1758   \nAlbania         0     0     0     1  ...  1223   856   702   560   716   561   \nAlgeria        69    63    44    69  ...  3626  4807  3623  4005  5393  4752   \n\n             2011  2012  2013  Total  \nCountry                               \nAfghanistan  2203  2635  2004  58639  \nAlbania       539   620   603  15699  \nAlgeria      4325  3774  4331  69439  \n\n[3 rows x 38 columns]"
                    },
                    "execution_count": 21,
                    "metadata": {},
                    "output_type": "execute_result"
                }
            ],
            "source": "df_can.head(3)"
        },
        {
            "cell_type": "code",
            "execution_count": 22,
            "metadata": {
                "deletable": true,
                "editable": true
            },
            "outputs": [],
            "source": "# optional: to remove the name of the index\ndf_can.index.name = None"
        },
        {
            "cell_type": "markdown",
            "metadata": {
                "deletable": true,
                "editable": true
            },
            "source": "Example: Let us view the number of immigrants from Japan (row 87) for the following scenarios:\n    1. The full row data (all columns)\n    2. For year 2013\n    3. For years 1980 to 1985"
        },
        {
            "cell_type": "code",
            "execution_count": 24,
            "metadata": {
                "deletable": true,
                "editable": true,
                "scrolled": true
            },
            "outputs": [
                {
                    "name": "stdout",
                    "output_type": "stream",
                    "text": "Continent                 Asia\nRegion            Eastern Asia\nDevName      Developed regions\n1980                       701\n1981                       756\n1982                       598\n1983                       309\n1984                       246\n1985                       198\n1986                       248\n1987                       422\n1988                       324\n1989                       494\n1990                       379\n1991                       506\n1992                       605\n1993                       907\n1994                       956\n1995                       826\n1996                       994\n1997                       924\n1998                       897\n1999                      1083\n2000                      1010\n2001                      1092\n2002                       806\n2003                       817\n2004                       973\n2005                      1067\n2006                      1212\n2007                      1250\n2008                      1284\n2009                      1194\n2010                      1168\n2011                      1265\n2012                      1214\n2013                       982\nTotal                    27707\nName: Japan, dtype: object\nContinent                 Asia\nRegion            Eastern Asia\nDevName      Developed regions\n1980                       701\n1981                       756\n1982                       598\n1983                       309\n1984                       246\n1985                       198\n1986                       248\n1987                       422\n1988                       324\n1989                       494\n1990                       379\n1991                       506\n1992                       605\n1993                       907\n1994                       956\n1995                       826\n1996                       994\n1997                       924\n1998                       897\n1999                      1083\n2000                      1010\n2001                      1092\n2002                       806\n2003                       817\n2004                       973\n2005                      1067\n2006                      1212\n2007                      1250\n2008                      1284\n2009                      1194\n2010                      1168\n2011                      1265\n2012                      1214\n2013                       982\nTotal                    27707\nName: Japan, dtype: object\nContinent                 Asia\nRegion            Eastern Asia\nDevName      Developed regions\n1980                       701\n1981                       756\n1982                       598\n1983                       309\n1984                       246\n1985                       198\n1986                       248\n1987                       422\n1988                       324\n1989                       494\n1990                       379\n1991                       506\n1992                       605\n1993                       907\n1994                       956\n1995                       826\n1996                       994\n1997                       924\n1998                       897\n1999                      1083\n2000                      1010\n2001                      1092\n2002                       806\n2003                       817\n2004                       973\n2005                      1067\n2006                      1212\n2007                      1250\n2008                      1284\n2009                      1194\n2010                      1168\n2011                      1265\n2012                      1214\n2013                       982\nTotal                    27707\nName: Japan, dtype: object\n"
                }
            ],
            "source": "# 1. the full row data (all columns)\nprint(df_can.loc['Japan'])\n\n# alternate methods\nprint(df_can.iloc[87])\nprint(df_can[df_can.index == 'Japan'].T.squeeze())"
        },
        {
            "cell_type": "code",
            "execution_count": 25,
            "metadata": {
                "collapsed": false,
                "deletable": true,
                "editable": true,
                "jupyter": {
                    "outputs_hidden": false
                },
                "scrolled": true
            },
            "outputs": [
                {
                    "name": "stdout",
                    "output_type": "stream",
                    "text": "982\n982\n"
                }
            ],
            "source": "# 2. for year 2013\nprint(df_can.loc['Japan', 2013])\n\n# alternate method\nprint(df_can.iloc[87, 36]) # year 2013 is the last column, with a positional index of 36"
        },
        {
            "cell_type": "code",
            "execution_count": 26,
            "metadata": {
                "collapsed": false,
                "deletable": true,
                "editable": true,
                "jupyter": {
                    "outputs_hidden": false
                }
            },
            "outputs": [
                {
                    "name": "stdout",
                    "output_type": "stream",
                    "text": "1980    701\n1981    756\n1982    598\n1983    309\n1984    246\n1984    246\nName: Japan, dtype: object\n1980    701\n1981    756\n1982    598\n1983    309\n1984    246\n1985    198\nName: Japan, dtype: object\n"
                }
            ],
            "source": "# 3. for years 1980 to 1985\nprint(df_can.loc['Japan', [1980, 1981, 1982, 1983, 1984, 1984]])\nprint(df_can.iloc[87, [3, 4, 5, 6, 7, 8]])"
        },
        {
            "cell_type": "markdown",
            "metadata": {
                "deletable": true,
                "editable": true
            },
            "source": "Column names that are integers (such as the years) might introduce some confusion. For example, when we are referencing the year 2013, one might confuse that when the 2013th positional index. \n\nTo avoid this ambuigity, let's convert the column names into strings: '1980' to '2013'."
        },
        {
            "cell_type": "code",
            "execution_count": 27,
            "metadata": {
                "collapsed": false,
                "deletable": true,
                "editable": true,
                "jupyter": {
                    "outputs_hidden": false
                }
            },
            "outputs": [
                {
                    "name": "stdout",
                    "output_type": "stream",
                    "text": "<class 'str'>\n<class 'str'>\n<class 'str'>\n<class 'str'>\n<class 'str'>\n<class 'str'>\n<class 'str'>\n<class 'str'>\n<class 'str'>\n<class 'str'>\n<class 'str'>\n<class 'str'>\n<class 'str'>\n<class 'str'>\n<class 'str'>\n<class 'str'>\n<class 'str'>\n<class 'str'>\n<class 'str'>\n<class 'str'>\n<class 'str'>\n<class 'str'>\n<class 'str'>\n<class 'str'>\n<class 'str'>\n<class 'str'>\n<class 'str'>\n<class 'str'>\n<class 'str'>\n<class 'str'>\n<class 'str'>\n<class 'str'>\n<class 'str'>\n<class 'str'>\n<class 'str'>\n<class 'str'>\n<class 'str'>\n<class 'str'>\n"
                },
                {
                    "data": {
                        "text/plain": "[None,\n None,\n None,\n None,\n None,\n None,\n None,\n None,\n None,\n None,\n None,\n None,\n None,\n None,\n None,\n None,\n None,\n None,\n None,\n None,\n None,\n None,\n None,\n None,\n None,\n None,\n None,\n None,\n None,\n None,\n None,\n None,\n None,\n None,\n None,\n None,\n None,\n None]"
                    },
                    "execution_count": 27,
                    "metadata": {},
                    "output_type": "execute_result"
                }
            ],
            "source": "df_can.columns = list(map(str, df_can.columns))\n[print (type(x)) for x in df_can.columns.values] #<-- uncomment to check type of column headers"
        },
        {
            "cell_type": "markdown",
            "metadata": {
                "deletable": true,
                "editable": true
            },
            "source": "Since we converted the years to string, let us declare a variable that will allow us to easily call upon the full range of years:"
        },
        {
            "cell_type": "code",
            "execution_count": 30,
            "metadata": {
                "deletable": true,
                "editable": true
            },
            "outputs": [
                {
                    "data": {
                        "text/plain": "['1980',\n '1981',\n '1982',\n '1983',\n '1984',\n '1985',\n '1986',\n '1987',\n '1988',\n '1989',\n '1990',\n '1991',\n '1992',\n '1993',\n '1994',\n '1995',\n '1996',\n '1997',\n '1998',\n '1999',\n '2000',\n '2001',\n '2002',\n '2003',\n '2004',\n '2005',\n '2006',\n '2007',\n '2008',\n '2009',\n '2010',\n '2011',\n '2012',\n '2013']"
                    },
                    "execution_count": 30,
                    "metadata": {},
                    "output_type": "execute_result"
                }
            ],
            "source": "# useful for plotting later on\nyears = list(map(str, range(1980, 2014)))\nyears"
        },
        {
            "cell_type": "markdown",
            "metadata": {
                "deletable": true,
                "editable": true
            },
            "source": "### Filtering based on a criteria\nTo filter the dataframe based on a condition, we simply pass the condition as a boolean vector. \n\nFor example, Let's filter the dataframe to show the data on Asian countries (AreaName = Asia)."
        },
        {
            "cell_type": "code",
            "execution_count": 31,
            "metadata": {
                "collapsed": false,
                "deletable": true,
                "editable": true,
                "jupyter": {
                    "outputs_hidden": false
                },
                "scrolled": true
            },
            "outputs": [
                {
                    "name": "stdout",
                    "output_type": "stream",
                    "text": "Afghanistan        True\nAlbania           False\nAlgeria           False\nAmerican Samoa    False\nAndorra           False\n                  ...  \nViet Nam           True\nWestern Sahara    False\nYemen              True\nZambia            False\nZimbabwe          False\nName: Continent, Length: 195, dtype: bool\n"
                }
            ],
            "source": "# 1. create the condition boolean series\ncondition = df_can['Continent']=='Asia'\nprint (condition)"
        },
        {
            "cell_type": "code",
            "execution_count": 32,
            "metadata": {
                "collapsed": false,
                "deletable": true,
                "editable": true,
                "jupyter": {
                    "outputs_hidden": false
                }
            },
            "outputs": [
                {
                    "data": {
                        "text/html": "<div>\n<style scoped>\n    .dataframe tbody tr th:only-of-type {\n        vertical-align: middle;\n    }\n\n    .dataframe tbody tr th {\n        vertical-align: top;\n    }\n\n    .dataframe thead th {\n        text-align: right;\n    }\n</style>\n<table border=\"1\" class=\"dataframe\">\n  <thead>\n    <tr style=\"text-align: right;\">\n      <th></th>\n      <th>Continent</th>\n      <th>Region</th>\n      <th>DevName</th>\n      <th>1980</th>\n      <th>1981</th>\n      <th>1982</th>\n      <th>1983</th>\n      <th>1984</th>\n      <th>1985</th>\n      <th>1986</th>\n      <th>...</th>\n      <th>2005</th>\n      <th>2006</th>\n      <th>2007</th>\n      <th>2008</th>\n      <th>2009</th>\n      <th>2010</th>\n      <th>2011</th>\n      <th>2012</th>\n      <th>2013</th>\n      <th>Total</th>\n    </tr>\n  </thead>\n  <tbody>\n    <tr>\n      <th>Afghanistan</th>\n      <td>Asia</td>\n      <td>Southern Asia</td>\n      <td>Developing regions</td>\n      <td>16</td>\n      <td>39</td>\n      <td>39</td>\n      <td>47</td>\n      <td>71</td>\n      <td>340</td>\n      <td>496</td>\n      <td>...</td>\n      <td>3436</td>\n      <td>3009</td>\n      <td>2652</td>\n      <td>2111</td>\n      <td>1746</td>\n      <td>1758</td>\n      <td>2203</td>\n      <td>2635</td>\n      <td>2004</td>\n      <td>58639</td>\n    </tr>\n    <tr>\n      <th>Armenia</th>\n      <td>Asia</td>\n      <td>Western Asia</td>\n      <td>Developing regions</td>\n      <td>0</td>\n      <td>0</td>\n      <td>0</td>\n      <td>0</td>\n      <td>0</td>\n      <td>0</td>\n      <td>0</td>\n      <td>...</td>\n      <td>224</td>\n      <td>218</td>\n      <td>198</td>\n      <td>205</td>\n      <td>267</td>\n      <td>252</td>\n      <td>236</td>\n      <td>258</td>\n      <td>207</td>\n      <td>3310</td>\n    </tr>\n    <tr>\n      <th>Azerbaijan</th>\n      <td>Asia</td>\n      <td>Western Asia</td>\n      <td>Developing regions</td>\n      <td>0</td>\n      <td>0</td>\n      <td>0</td>\n      <td>0</td>\n      <td>0</td>\n      <td>0</td>\n      <td>0</td>\n      <td>...</td>\n      <td>359</td>\n      <td>236</td>\n      <td>203</td>\n      <td>125</td>\n      <td>165</td>\n      <td>209</td>\n      <td>138</td>\n      <td>161</td>\n      <td>57</td>\n      <td>2649</td>\n    </tr>\n    <tr>\n      <th>Bahrain</th>\n      <td>Asia</td>\n      <td>Western Asia</td>\n      <td>Developing regions</td>\n      <td>0</td>\n      <td>2</td>\n      <td>1</td>\n      <td>1</td>\n      <td>1</td>\n      <td>3</td>\n      <td>0</td>\n      <td>...</td>\n      <td>12</td>\n      <td>12</td>\n      <td>22</td>\n      <td>9</td>\n      <td>35</td>\n      <td>28</td>\n      <td>21</td>\n      <td>39</td>\n      <td>32</td>\n      <td>475</td>\n    </tr>\n    <tr>\n      <th>Bangladesh</th>\n      <td>Asia</td>\n      <td>Southern Asia</td>\n      <td>Developing regions</td>\n      <td>83</td>\n      <td>84</td>\n      <td>86</td>\n      <td>81</td>\n      <td>98</td>\n      <td>92</td>\n      <td>486</td>\n      <td>...</td>\n      <td>4171</td>\n      <td>4014</td>\n      <td>2897</td>\n      <td>2939</td>\n      <td>2104</td>\n      <td>4721</td>\n      <td>2694</td>\n      <td>2640</td>\n      <td>3789</td>\n      <td>65568</td>\n    </tr>\n    <tr>\n      <th>Bhutan</th>\n      <td>Asia</td>\n      <td>Southern Asia</td>\n      <td>Developing regions</td>\n      <td>0</td>\n      <td>0</td>\n      <td>0</td>\n      <td>0</td>\n      <td>1</td>\n      <td>0</td>\n      <td>0</td>\n      <td>...</td>\n      <td>5</td>\n      <td>10</td>\n      <td>7</td>\n      <td>36</td>\n      <td>865</td>\n      <td>1464</td>\n      <td>1879</td>\n      <td>1075</td>\n      <td>487</td>\n      <td>5876</td>\n    </tr>\n    <tr>\n      <th>Brunei Darussalam</th>\n      <td>Asia</td>\n      <td>South-Eastern Asia</td>\n      <td>Developing regions</td>\n      <td>79</td>\n      <td>6</td>\n      <td>8</td>\n      <td>2</td>\n      <td>2</td>\n      <td>4</td>\n      <td>12</td>\n      <td>...</td>\n      <td>4</td>\n      <td>5</td>\n      <td>11</td>\n      <td>10</td>\n      <td>5</td>\n      <td>12</td>\n      <td>6</td>\n      <td>3</td>\n      <td>6</td>\n      <td>600</td>\n    </tr>\n    <tr>\n      <th>Cambodia</th>\n      <td>Asia</td>\n      <td>South-Eastern Asia</td>\n      <td>Developing regions</td>\n      <td>12</td>\n      <td>19</td>\n      <td>26</td>\n      <td>33</td>\n      <td>10</td>\n      <td>7</td>\n      <td>8</td>\n      <td>...</td>\n      <td>370</td>\n      <td>529</td>\n      <td>460</td>\n      <td>354</td>\n      <td>203</td>\n      <td>200</td>\n      <td>196</td>\n      <td>233</td>\n      <td>288</td>\n      <td>6538</td>\n    </tr>\n    <tr>\n      <th>China</th>\n      <td>Asia</td>\n      <td>Eastern Asia</td>\n      <td>Developing regions</td>\n      <td>5123</td>\n      <td>6682</td>\n      <td>3308</td>\n      <td>1863</td>\n      <td>1527</td>\n      <td>1816</td>\n      <td>1960</td>\n      <td>...</td>\n      <td>42584</td>\n      <td>33518</td>\n      <td>27642</td>\n      <td>30037</td>\n      <td>29622</td>\n      <td>30391</td>\n      <td>28502</td>\n      <td>33024</td>\n      <td>34129</td>\n      <td>659962</td>\n    </tr>\n    <tr>\n      <th>China, Hong Kong Special Administrative Region</th>\n      <td>Asia</td>\n      <td>Eastern Asia</td>\n      <td>Developing regions</td>\n      <td>0</td>\n      <td>0</td>\n      <td>0</td>\n      <td>0</td>\n      <td>0</td>\n      <td>0</td>\n      <td>0</td>\n      <td>...</td>\n      <td>729</td>\n      <td>712</td>\n      <td>674</td>\n      <td>897</td>\n      <td>657</td>\n      <td>623</td>\n      <td>591</td>\n      <td>728</td>\n      <td>774</td>\n      <td>9327</td>\n    </tr>\n    <tr>\n      <th>China, Macao Special Administrative Region</th>\n      <td>Asia</td>\n      <td>Eastern Asia</td>\n      <td>Developing regions</td>\n      <td>0</td>\n      <td>0</td>\n      <td>0</td>\n      <td>0</td>\n      <td>0</td>\n      <td>0</td>\n      <td>0</td>\n      <td>...</td>\n      <td>21</td>\n      <td>32</td>\n      <td>16</td>\n      <td>12</td>\n      <td>21</td>\n      <td>21</td>\n      <td>13</td>\n      <td>33</td>\n      <td>29</td>\n      <td>284</td>\n    </tr>\n    <tr>\n      <th>Cyprus</th>\n      <td>Asia</td>\n      <td>Western Asia</td>\n      <td>Developing regions</td>\n      <td>132</td>\n      <td>128</td>\n      <td>84</td>\n      <td>46</td>\n      <td>46</td>\n      <td>43</td>\n      <td>48</td>\n      <td>...</td>\n      <td>7</td>\n      <td>9</td>\n      <td>4</td>\n      <td>7</td>\n      <td>6</td>\n      <td>18</td>\n      <td>6</td>\n      <td>12</td>\n      <td>16</td>\n      <td>1126</td>\n    </tr>\n    <tr>\n      <th>Democratic People's Republic of Korea</th>\n      <td>Asia</td>\n      <td>Eastern Asia</td>\n      <td>Developing regions</td>\n      <td>1</td>\n      <td>1</td>\n      <td>3</td>\n      <td>1</td>\n      <td>4</td>\n      <td>3</td>\n      <td>0</td>\n      <td>...</td>\n      <td>14</td>\n      <td>10</td>\n      <td>7</td>\n      <td>19</td>\n      <td>11</td>\n      <td>45</td>\n      <td>97</td>\n      <td>66</td>\n      <td>17</td>\n      <td>388</td>\n    </tr>\n    <tr>\n      <th>Georgia</th>\n      <td>Asia</td>\n      <td>Western Asia</td>\n      <td>Developing regions</td>\n      <td>0</td>\n      <td>0</td>\n      <td>0</td>\n      <td>0</td>\n      <td>0</td>\n      <td>0</td>\n      <td>0</td>\n      <td>...</td>\n      <td>114</td>\n      <td>125</td>\n      <td>132</td>\n      <td>112</td>\n      <td>128</td>\n      <td>126</td>\n      <td>139</td>\n      <td>147</td>\n      <td>125</td>\n      <td>2068</td>\n    </tr>\n    <tr>\n      <th>India</th>\n      <td>Asia</td>\n      <td>Southern Asia</td>\n      <td>Developing regions</td>\n      <td>8880</td>\n      <td>8670</td>\n      <td>8147</td>\n      <td>7338</td>\n      <td>5704</td>\n      <td>4211</td>\n      <td>7150</td>\n      <td>...</td>\n      <td>36210</td>\n      <td>33848</td>\n      <td>28742</td>\n      <td>28261</td>\n      <td>29456</td>\n      <td>34235</td>\n      <td>27509</td>\n      <td>30933</td>\n      <td>33087</td>\n      <td>691904</td>\n    </tr>\n    <tr>\n      <th>Indonesia</th>\n      <td>Asia</td>\n      <td>South-Eastern Asia</td>\n      <td>Developing regions</td>\n      <td>186</td>\n      <td>178</td>\n      <td>252</td>\n      <td>115</td>\n      <td>123</td>\n      <td>100</td>\n      <td>127</td>\n      <td>...</td>\n      <td>632</td>\n      <td>613</td>\n      <td>657</td>\n      <td>661</td>\n      <td>504</td>\n      <td>712</td>\n      <td>390</td>\n      <td>395</td>\n      <td>387</td>\n      <td>13150</td>\n    </tr>\n    <tr>\n      <th>Iran (Islamic Republic of)</th>\n      <td>Asia</td>\n      <td>Southern Asia</td>\n      <td>Developing regions</td>\n      <td>1172</td>\n      <td>1429</td>\n      <td>1822</td>\n      <td>1592</td>\n      <td>1977</td>\n      <td>1648</td>\n      <td>1794</td>\n      <td>...</td>\n      <td>5837</td>\n      <td>7480</td>\n      <td>6974</td>\n      <td>6475</td>\n      <td>6580</td>\n      <td>7477</td>\n      <td>7479</td>\n      <td>7534</td>\n      <td>11291</td>\n      <td>175923</td>\n    </tr>\n    <tr>\n      <th>Iraq</th>\n      <td>Asia</td>\n      <td>Western Asia</td>\n      <td>Developing regions</td>\n      <td>262</td>\n      <td>245</td>\n      <td>260</td>\n      <td>380</td>\n      <td>428</td>\n      <td>231</td>\n      <td>265</td>\n      <td>...</td>\n      <td>2226</td>\n      <td>1788</td>\n      <td>2406</td>\n      <td>3543</td>\n      <td>5450</td>\n      <td>5941</td>\n      <td>6196</td>\n      <td>4041</td>\n      <td>4918</td>\n      <td>69789</td>\n    </tr>\n    <tr>\n      <th>Israel</th>\n      <td>Asia</td>\n      <td>Western Asia</td>\n      <td>Developing regions</td>\n      <td>1403</td>\n      <td>1711</td>\n      <td>1334</td>\n      <td>541</td>\n      <td>446</td>\n      <td>680</td>\n      <td>1212</td>\n      <td>...</td>\n      <td>2446</td>\n      <td>2625</td>\n      <td>2401</td>\n      <td>2562</td>\n      <td>2316</td>\n      <td>2755</td>\n      <td>1970</td>\n      <td>2134</td>\n      <td>1945</td>\n      <td>66508</td>\n    </tr>\n    <tr>\n      <th>Japan</th>\n      <td>Asia</td>\n      <td>Eastern Asia</td>\n      <td>Developed regions</td>\n      <td>701</td>\n      <td>756</td>\n      <td>598</td>\n      <td>309</td>\n      <td>246</td>\n      <td>198</td>\n      <td>248</td>\n      <td>...</td>\n      <td>1067</td>\n      <td>1212</td>\n      <td>1250</td>\n      <td>1284</td>\n      <td>1194</td>\n      <td>1168</td>\n      <td>1265</td>\n      <td>1214</td>\n      <td>982</td>\n      <td>27707</td>\n    </tr>\n    <tr>\n      <th>Jordan</th>\n      <td>Asia</td>\n      <td>Western Asia</td>\n      <td>Developing regions</td>\n      <td>177</td>\n      <td>160</td>\n      <td>155</td>\n      <td>113</td>\n      <td>102</td>\n      <td>179</td>\n      <td>181</td>\n      <td>...</td>\n      <td>1940</td>\n      <td>1827</td>\n      <td>1421</td>\n      <td>1581</td>\n      <td>1235</td>\n      <td>1831</td>\n      <td>1635</td>\n      <td>1206</td>\n      <td>1255</td>\n      <td>35406</td>\n    </tr>\n    <tr>\n      <th>Kazakhstan</th>\n      <td>Asia</td>\n      <td>Central Asia</td>\n      <td>Developing regions</td>\n      <td>0</td>\n      <td>0</td>\n      <td>0</td>\n      <td>0</td>\n      <td>0</td>\n      <td>0</td>\n      <td>0</td>\n      <td>...</td>\n      <td>506</td>\n      <td>408</td>\n      <td>436</td>\n      <td>394</td>\n      <td>431</td>\n      <td>377</td>\n      <td>381</td>\n      <td>462</td>\n      <td>348</td>\n      <td>8490</td>\n    </tr>\n    <tr>\n      <th>Kuwait</th>\n      <td>Asia</td>\n      <td>Western Asia</td>\n      <td>Developing regions</td>\n      <td>1</td>\n      <td>0</td>\n      <td>8</td>\n      <td>2</td>\n      <td>1</td>\n      <td>4</td>\n      <td>4</td>\n      <td>...</td>\n      <td>66</td>\n      <td>35</td>\n      <td>62</td>\n      <td>53</td>\n      <td>68</td>\n      <td>67</td>\n      <td>58</td>\n      <td>73</td>\n      <td>48</td>\n      <td>2025</td>\n    </tr>\n    <tr>\n      <th>Kyrgyzstan</th>\n      <td>Asia</td>\n      <td>Central Asia</td>\n      <td>Developing regions</td>\n      <td>0</td>\n      <td>0</td>\n      <td>0</td>\n      <td>0</td>\n      <td>0</td>\n      <td>0</td>\n      <td>0</td>\n      <td>...</td>\n      <td>173</td>\n      <td>161</td>\n      <td>135</td>\n      <td>168</td>\n      <td>173</td>\n      <td>157</td>\n      <td>159</td>\n      <td>278</td>\n      <td>123</td>\n      <td>2353</td>\n    </tr>\n    <tr>\n      <th>Lao People's Democratic Republic</th>\n      <td>Asia</td>\n      <td>South-Eastern Asia</td>\n      <td>Developing regions</td>\n      <td>11</td>\n      <td>6</td>\n      <td>16</td>\n      <td>16</td>\n      <td>7</td>\n      <td>17</td>\n      <td>21</td>\n      <td>...</td>\n      <td>42</td>\n      <td>74</td>\n      <td>53</td>\n      <td>32</td>\n      <td>39</td>\n      <td>54</td>\n      <td>22</td>\n      <td>25</td>\n      <td>15</td>\n      <td>1089</td>\n    </tr>\n    <tr>\n      <th>Lebanon</th>\n      <td>Asia</td>\n      <td>Western Asia</td>\n      <td>Developing regions</td>\n      <td>1409</td>\n      <td>1119</td>\n      <td>1159</td>\n      <td>789</td>\n      <td>1253</td>\n      <td>1683</td>\n      <td>2576</td>\n      <td>...</td>\n      <td>3709</td>\n      <td>3802</td>\n      <td>3467</td>\n      <td>3566</td>\n      <td>3077</td>\n      <td>3432</td>\n      <td>3072</td>\n      <td>1614</td>\n      <td>2172</td>\n      <td>115359</td>\n    </tr>\n    <tr>\n      <th>Malaysia</th>\n      <td>Asia</td>\n      <td>South-Eastern Asia</td>\n      <td>Developing regions</td>\n      <td>786</td>\n      <td>816</td>\n      <td>813</td>\n      <td>448</td>\n      <td>384</td>\n      <td>374</td>\n      <td>425</td>\n      <td>...</td>\n      <td>593</td>\n      <td>580</td>\n      <td>600</td>\n      <td>658</td>\n      <td>640</td>\n      <td>802</td>\n      <td>409</td>\n      <td>358</td>\n      <td>204</td>\n      <td>24417</td>\n    </tr>\n    <tr>\n      <th>Maldives</th>\n      <td>Asia</td>\n      <td>Southern Asia</td>\n      <td>Developing regions</td>\n      <td>0</td>\n      <td>0</td>\n      <td>0</td>\n      <td>1</td>\n      <td>0</td>\n      <td>0</td>\n      <td>0</td>\n      <td>...</td>\n      <td>0</td>\n      <td>0</td>\n      <td>2</td>\n      <td>1</td>\n      <td>7</td>\n      <td>4</td>\n      <td>3</td>\n      <td>1</td>\n      <td>1</td>\n      <td>30</td>\n    </tr>\n    <tr>\n      <th>Mongolia</th>\n      <td>Asia</td>\n      <td>Eastern Asia</td>\n      <td>Developing regions</td>\n      <td>0</td>\n      <td>0</td>\n      <td>0</td>\n      <td>0</td>\n      <td>0</td>\n      <td>0</td>\n      <td>0</td>\n      <td>...</td>\n      <td>59</td>\n      <td>64</td>\n      <td>82</td>\n      <td>59</td>\n      <td>118</td>\n      <td>169</td>\n      <td>103</td>\n      <td>68</td>\n      <td>99</td>\n      <td>952</td>\n    </tr>\n    <tr>\n      <th>Myanmar</th>\n      <td>Asia</td>\n      <td>South-Eastern Asia</td>\n      <td>Developing regions</td>\n      <td>80</td>\n      <td>62</td>\n      <td>46</td>\n      <td>31</td>\n      <td>41</td>\n      <td>23</td>\n      <td>18</td>\n      <td>...</td>\n      <td>210</td>\n      <td>953</td>\n      <td>1887</td>\n      <td>975</td>\n      <td>1153</td>\n      <td>556</td>\n      <td>368</td>\n      <td>193</td>\n      <td>262</td>\n      <td>9245</td>\n    </tr>\n    <tr>\n      <th>Nepal</th>\n      <td>Asia</td>\n      <td>Southern Asia</td>\n      <td>Developing regions</td>\n      <td>1</td>\n      <td>1</td>\n      <td>6</td>\n      <td>1</td>\n      <td>2</td>\n      <td>4</td>\n      <td>13</td>\n      <td>...</td>\n      <td>607</td>\n      <td>540</td>\n      <td>511</td>\n      <td>581</td>\n      <td>561</td>\n      <td>1392</td>\n      <td>1129</td>\n      <td>1185</td>\n      <td>1308</td>\n      <td>10222</td>\n    </tr>\n    <tr>\n      <th>Oman</th>\n      <td>Asia</td>\n      <td>Western Asia</td>\n      <td>Developing regions</td>\n      <td>0</td>\n      <td>0</td>\n      <td>0</td>\n      <td>8</td>\n      <td>0</td>\n      <td>0</td>\n      <td>0</td>\n      <td>...</td>\n      <td>14</td>\n      <td>18</td>\n      <td>16</td>\n      <td>10</td>\n      <td>7</td>\n      <td>14</td>\n      <td>10</td>\n      <td>13</td>\n      <td>11</td>\n      <td>224</td>\n    </tr>\n    <tr>\n      <th>Pakistan</th>\n      <td>Asia</td>\n      <td>Southern Asia</td>\n      <td>Developing regions</td>\n      <td>978</td>\n      <td>972</td>\n      <td>1201</td>\n      <td>900</td>\n      <td>668</td>\n      <td>514</td>\n      <td>691</td>\n      <td>...</td>\n      <td>14314</td>\n      <td>13127</td>\n      <td>10124</td>\n      <td>8994</td>\n      <td>7217</td>\n      <td>6811</td>\n      <td>7468</td>\n      <td>11227</td>\n      <td>12603</td>\n      <td>241600</td>\n    </tr>\n    <tr>\n      <th>Philippines</th>\n      <td>Asia</td>\n      <td>South-Eastern Asia</td>\n      <td>Developing regions</td>\n      <td>6051</td>\n      <td>5921</td>\n      <td>5249</td>\n      <td>4562</td>\n      <td>3801</td>\n      <td>3150</td>\n      <td>4166</td>\n      <td>...</td>\n      <td>18139</td>\n      <td>18400</td>\n      <td>19837</td>\n      <td>24887</td>\n      <td>28573</td>\n      <td>38617</td>\n      <td>36765</td>\n      <td>34315</td>\n      <td>29544</td>\n      <td>511391</td>\n    </tr>\n    <tr>\n      <th>Qatar</th>\n      <td>Asia</td>\n      <td>Western Asia</td>\n      <td>Developing regions</td>\n      <td>0</td>\n      <td>0</td>\n      <td>0</td>\n      <td>0</td>\n      <td>0</td>\n      <td>0</td>\n      <td>1</td>\n      <td>...</td>\n      <td>11</td>\n      <td>2</td>\n      <td>5</td>\n      <td>9</td>\n      <td>6</td>\n      <td>18</td>\n      <td>3</td>\n      <td>14</td>\n      <td>6</td>\n      <td>157</td>\n    </tr>\n    <tr>\n      <th>Republic of Korea</th>\n      <td>Asia</td>\n      <td>Eastern Asia</td>\n      <td>Developing regions</td>\n      <td>1011</td>\n      <td>1456</td>\n      <td>1572</td>\n      <td>1081</td>\n      <td>847</td>\n      <td>962</td>\n      <td>1208</td>\n      <td>...</td>\n      <td>5832</td>\n      <td>6215</td>\n      <td>5920</td>\n      <td>7294</td>\n      <td>5874</td>\n      <td>5537</td>\n      <td>4588</td>\n      <td>5316</td>\n      <td>4509</td>\n      <td>142581</td>\n    </tr>\n    <tr>\n      <th>Saudi Arabia</th>\n      <td>Asia</td>\n      <td>Western Asia</td>\n      <td>Developing regions</td>\n      <td>0</td>\n      <td>0</td>\n      <td>1</td>\n      <td>4</td>\n      <td>1</td>\n      <td>2</td>\n      <td>5</td>\n      <td>...</td>\n      <td>198</td>\n      <td>252</td>\n      <td>188</td>\n      <td>249</td>\n      <td>246</td>\n      <td>330</td>\n      <td>278</td>\n      <td>286</td>\n      <td>267</td>\n      <td>3425</td>\n    </tr>\n    <tr>\n      <th>Singapore</th>\n      <td>Asia</td>\n      <td>South-Eastern Asia</td>\n      <td>Developing regions</td>\n      <td>241</td>\n      <td>301</td>\n      <td>337</td>\n      <td>169</td>\n      <td>128</td>\n      <td>139</td>\n      <td>205</td>\n      <td>...</td>\n      <td>392</td>\n      <td>298</td>\n      <td>690</td>\n      <td>734</td>\n      <td>366</td>\n      <td>805</td>\n      <td>219</td>\n      <td>146</td>\n      <td>141</td>\n      <td>14579</td>\n    </tr>\n    <tr>\n      <th>Sri Lanka</th>\n      <td>Asia</td>\n      <td>Southern Asia</td>\n      <td>Developing regions</td>\n      <td>185</td>\n      <td>371</td>\n      <td>290</td>\n      <td>197</td>\n      <td>1086</td>\n      <td>845</td>\n      <td>1838</td>\n      <td>...</td>\n      <td>4930</td>\n      <td>4714</td>\n      <td>4123</td>\n      <td>4756</td>\n      <td>4547</td>\n      <td>4422</td>\n      <td>3309</td>\n      <td>3338</td>\n      <td>2394</td>\n      <td>148358</td>\n    </tr>\n    <tr>\n      <th>State of Palestine</th>\n      <td>Asia</td>\n      <td>Western Asia</td>\n      <td>Developing regions</td>\n      <td>0</td>\n      <td>0</td>\n      <td>0</td>\n      <td>0</td>\n      <td>0</td>\n      <td>0</td>\n      <td>0</td>\n      <td>...</td>\n      <td>453</td>\n      <td>627</td>\n      <td>441</td>\n      <td>481</td>\n      <td>400</td>\n      <td>654</td>\n      <td>555</td>\n      <td>533</td>\n      <td>462</td>\n      <td>6512</td>\n    </tr>\n    <tr>\n      <th>Syrian Arab Republic</th>\n      <td>Asia</td>\n      <td>Western Asia</td>\n      <td>Developing regions</td>\n      <td>315</td>\n      <td>419</td>\n      <td>409</td>\n      <td>269</td>\n      <td>264</td>\n      <td>385</td>\n      <td>493</td>\n      <td>...</td>\n      <td>1458</td>\n      <td>1145</td>\n      <td>1056</td>\n      <td>919</td>\n      <td>917</td>\n      <td>1039</td>\n      <td>1005</td>\n      <td>650</td>\n      <td>1009</td>\n      <td>31485</td>\n    </tr>\n    <tr>\n      <th>Tajikistan</th>\n      <td>Asia</td>\n      <td>Central Asia</td>\n      <td>Developing regions</td>\n      <td>0</td>\n      <td>0</td>\n      <td>0</td>\n      <td>0</td>\n      <td>0</td>\n      <td>0</td>\n      <td>0</td>\n      <td>...</td>\n      <td>85</td>\n      <td>46</td>\n      <td>44</td>\n      <td>15</td>\n      <td>50</td>\n      <td>52</td>\n      <td>47</td>\n      <td>34</td>\n      <td>39</td>\n      <td>503</td>\n    </tr>\n    <tr>\n      <th>Thailand</th>\n      <td>Asia</td>\n      <td>South-Eastern Asia</td>\n      <td>Developing regions</td>\n      <td>56</td>\n      <td>53</td>\n      <td>113</td>\n      <td>65</td>\n      <td>82</td>\n      <td>66</td>\n      <td>78</td>\n      <td>...</td>\n      <td>575</td>\n      <td>500</td>\n      <td>487</td>\n      <td>519</td>\n      <td>512</td>\n      <td>499</td>\n      <td>396</td>\n      <td>296</td>\n      <td>400</td>\n      <td>9174</td>\n    </tr>\n    <tr>\n      <th>Turkey</th>\n      <td>Asia</td>\n      <td>Western Asia</td>\n      <td>Developing regions</td>\n      <td>481</td>\n      <td>874</td>\n      <td>706</td>\n      <td>280</td>\n      <td>338</td>\n      <td>202</td>\n      <td>257</td>\n      <td>...</td>\n      <td>2065</td>\n      <td>1638</td>\n      <td>1463</td>\n      <td>1122</td>\n      <td>1238</td>\n      <td>1492</td>\n      <td>1257</td>\n      <td>1068</td>\n      <td>729</td>\n      <td>31781</td>\n    </tr>\n    <tr>\n      <th>Turkmenistan</th>\n      <td>Asia</td>\n      <td>Central Asia</td>\n      <td>Developing regions</td>\n      <td>0</td>\n      <td>0</td>\n      <td>0</td>\n      <td>0</td>\n      <td>0</td>\n      <td>0</td>\n      <td>0</td>\n      <td>...</td>\n      <td>40</td>\n      <td>26</td>\n      <td>37</td>\n      <td>13</td>\n      <td>20</td>\n      <td>30</td>\n      <td>20</td>\n      <td>20</td>\n      <td>14</td>\n      <td>310</td>\n    </tr>\n    <tr>\n      <th>United Arab Emirates</th>\n      <td>Asia</td>\n      <td>Western Asia</td>\n      <td>Developing regions</td>\n      <td>0</td>\n      <td>2</td>\n      <td>2</td>\n      <td>1</td>\n      <td>2</td>\n      <td>0</td>\n      <td>5</td>\n      <td>...</td>\n      <td>31</td>\n      <td>42</td>\n      <td>37</td>\n      <td>33</td>\n      <td>37</td>\n      <td>86</td>\n      <td>60</td>\n      <td>54</td>\n      <td>46</td>\n      <td>836</td>\n    </tr>\n    <tr>\n      <th>Uzbekistan</th>\n      <td>Asia</td>\n      <td>Central Asia</td>\n      <td>Developing regions</td>\n      <td>0</td>\n      <td>0</td>\n      <td>0</td>\n      <td>0</td>\n      <td>0</td>\n      <td>0</td>\n      <td>0</td>\n      <td>...</td>\n      <td>330</td>\n      <td>262</td>\n      <td>284</td>\n      <td>215</td>\n      <td>288</td>\n      <td>289</td>\n      <td>162</td>\n      <td>235</td>\n      <td>167</td>\n      <td>3368</td>\n    </tr>\n    <tr>\n      <th>Viet Nam</th>\n      <td>Asia</td>\n      <td>South-Eastern Asia</td>\n      <td>Developing regions</td>\n      <td>1191</td>\n      <td>1829</td>\n      <td>2162</td>\n      <td>3404</td>\n      <td>7583</td>\n      <td>5907</td>\n      <td>2741</td>\n      <td>...</td>\n      <td>1852</td>\n      <td>3153</td>\n      <td>2574</td>\n      <td>1784</td>\n      <td>2171</td>\n      <td>1942</td>\n      <td>1723</td>\n      <td>1731</td>\n      <td>2112</td>\n      <td>97146</td>\n    </tr>\n    <tr>\n      <th>Yemen</th>\n      <td>Asia</td>\n      <td>Western Asia</td>\n      <td>Developing regions</td>\n      <td>1</td>\n      <td>2</td>\n      <td>1</td>\n      <td>6</td>\n      <td>0</td>\n      <td>18</td>\n      <td>7</td>\n      <td>...</td>\n      <td>161</td>\n      <td>140</td>\n      <td>122</td>\n      <td>133</td>\n      <td>128</td>\n      <td>211</td>\n      <td>160</td>\n      <td>174</td>\n      <td>217</td>\n      <td>2985</td>\n    </tr>\n  </tbody>\n</table>\n<p>49 rows \u00d7 38 columns</p>\n</div>",
                        "text/plain": "                                               Continent              Region  \\\nAfghanistan                                         Asia       Southern Asia   \nArmenia                                             Asia        Western Asia   \nAzerbaijan                                          Asia        Western Asia   \nBahrain                                             Asia        Western Asia   \nBangladesh                                          Asia       Southern Asia   \nBhutan                                              Asia       Southern Asia   \nBrunei Darussalam                                   Asia  South-Eastern Asia   \nCambodia                                            Asia  South-Eastern Asia   \nChina                                               Asia        Eastern Asia   \nChina, Hong Kong Special Administrative Region      Asia        Eastern Asia   \nChina, Macao Special Administrative Region          Asia        Eastern Asia   \nCyprus                                              Asia        Western Asia   \nDemocratic People's Republic of Korea               Asia        Eastern Asia   \nGeorgia                                             Asia        Western Asia   \nIndia                                               Asia       Southern Asia   \nIndonesia                                           Asia  South-Eastern Asia   \nIran (Islamic Republic of)                          Asia       Southern Asia   \nIraq                                                Asia        Western Asia   \nIsrael                                              Asia        Western Asia   \nJapan                                               Asia        Eastern Asia   \nJordan                                              Asia        Western Asia   \nKazakhstan                                          Asia        Central Asia   \nKuwait                                              Asia        Western Asia   \nKyrgyzstan                                          Asia        Central Asia   \nLao People's Democratic Republic                    Asia  South-Eastern Asia   \nLebanon                                             Asia        Western Asia   \nMalaysia                                            Asia  South-Eastern Asia   \nMaldives                                            Asia       Southern Asia   \nMongolia                                            Asia        Eastern Asia   \nMyanmar                                             Asia  South-Eastern Asia   \nNepal                                               Asia       Southern Asia   \nOman                                                Asia        Western Asia   \nPakistan                                            Asia       Southern Asia   \nPhilippines                                         Asia  South-Eastern Asia   \nQatar                                               Asia        Western Asia   \nRepublic of Korea                                   Asia        Eastern Asia   \nSaudi Arabia                                        Asia        Western Asia   \nSingapore                                           Asia  South-Eastern Asia   \nSri Lanka                                           Asia       Southern Asia   \nState of Palestine                                  Asia        Western Asia   \nSyrian Arab Republic                                Asia        Western Asia   \nTajikistan                                          Asia        Central Asia   \nThailand                                            Asia  South-Eastern Asia   \nTurkey                                              Asia        Western Asia   \nTurkmenistan                                        Asia        Central Asia   \nUnited Arab Emirates                                Asia        Western Asia   \nUzbekistan                                          Asia        Central Asia   \nViet Nam                                            Asia  South-Eastern Asia   \nYemen                                               Asia        Western Asia   \n\n                                                           DevName  1980  \\\nAfghanistan                                     Developing regions    16   \nArmenia                                         Developing regions     0   \nAzerbaijan                                      Developing regions     0   \nBahrain                                         Developing regions     0   \nBangladesh                                      Developing regions    83   \nBhutan                                          Developing regions     0   \nBrunei Darussalam                               Developing regions    79   \nCambodia                                        Developing regions    12   \nChina                                           Developing regions  5123   \nChina, Hong Kong Special Administrative Region  Developing regions     0   \nChina, Macao Special Administrative Region      Developing regions     0   \nCyprus                                          Developing regions   132   \nDemocratic People's Republic of Korea           Developing regions     1   \nGeorgia                                         Developing regions     0   \nIndia                                           Developing regions  8880   \nIndonesia                                       Developing regions   186   \nIran (Islamic Republic of)                      Developing regions  1172   \nIraq                                            Developing regions   262   \nIsrael                                          Developing regions  1403   \nJapan                                            Developed regions   701   \nJordan                                          Developing regions   177   \nKazakhstan                                      Developing regions     0   \nKuwait                                          Developing regions     1   \nKyrgyzstan                                      Developing regions     0   \nLao People's Democratic Republic                Developing regions    11   \nLebanon                                         Developing regions  1409   \nMalaysia                                        Developing regions   786   \nMaldives                                        Developing regions     0   \nMongolia                                        Developing regions     0   \nMyanmar                                         Developing regions    80   \nNepal                                           Developing regions     1   \nOman                                            Developing regions     0   \nPakistan                                        Developing regions   978   \nPhilippines                                     Developing regions  6051   \nQatar                                           Developing regions     0   \nRepublic of Korea                               Developing regions  1011   \nSaudi Arabia                                    Developing regions     0   \nSingapore                                       Developing regions   241   \nSri Lanka                                       Developing regions   185   \nState of Palestine                              Developing regions     0   \nSyrian Arab Republic                            Developing regions   315   \nTajikistan                                      Developing regions     0   \nThailand                                        Developing regions    56   \nTurkey                                          Developing regions   481   \nTurkmenistan                                    Developing regions     0   \nUnited Arab Emirates                            Developing regions     0   \nUzbekistan                                      Developing regions     0   \nViet Nam                                        Developing regions  1191   \nYemen                                           Developing regions     1   \n\n                                                1981  1982  1983  1984  1985  \\\nAfghanistan                                       39    39    47    71   340   \nArmenia                                            0     0     0     0     0   \nAzerbaijan                                         0     0     0     0     0   \nBahrain                                            2     1     1     1     3   \nBangladesh                                        84    86    81    98    92   \nBhutan                                             0     0     0     1     0   \nBrunei Darussalam                                  6     8     2     2     4   \nCambodia                                          19    26    33    10     7   \nChina                                           6682  3308  1863  1527  1816   \nChina, Hong Kong Special Administrative Region     0     0     0     0     0   \nChina, Macao Special Administrative Region         0     0     0     0     0   \nCyprus                                           128    84    46    46    43   \nDemocratic People's Republic of Korea              1     3     1     4     3   \nGeorgia                                            0     0     0     0     0   \nIndia                                           8670  8147  7338  5704  4211   \nIndonesia                                        178   252   115   123   100   \nIran (Islamic Republic of)                      1429  1822  1592  1977  1648   \nIraq                                             245   260   380   428   231   \nIsrael                                          1711  1334   541   446   680   \nJapan                                            756   598   309   246   198   \nJordan                                           160   155   113   102   179   \nKazakhstan                                         0     0     0     0     0   \nKuwait                                             0     8     2     1     4   \nKyrgyzstan                                         0     0     0     0     0   \nLao People's Democratic Republic                   6    16    16     7    17   \nLebanon                                         1119  1159   789  1253  1683   \nMalaysia                                         816   813   448   384   374   \nMaldives                                           0     0     1     0     0   \nMongolia                                           0     0     0     0     0   \nMyanmar                                           62    46    31    41    23   \nNepal                                              1     6     1     2     4   \nOman                                               0     0     8     0     0   \nPakistan                                         972  1201   900   668   514   \nPhilippines                                     5921  5249  4562  3801  3150   \nQatar                                              0     0     0     0     0   \nRepublic of Korea                               1456  1572  1081   847   962   \nSaudi Arabia                                       0     1     4     1     2   \nSingapore                                        301   337   169   128   139   \nSri Lanka                                        371   290   197  1086   845   \nState of Palestine                                 0     0     0     0     0   \nSyrian Arab Republic                             419   409   269   264   385   \nTajikistan                                         0     0     0     0     0   \nThailand                                          53   113    65    82    66   \nTurkey                                           874   706   280   338   202   \nTurkmenistan                                       0     0     0     0     0   \nUnited Arab Emirates                               2     2     1     2     0   \nUzbekistan                                         0     0     0     0     0   \nViet Nam                                        1829  2162  3404  7583  5907   \nYemen                                              2     1     6     0    18   \n\n                                                1986  ...   2005   2006  \\\nAfghanistan                                      496  ...   3436   3009   \nArmenia                                            0  ...    224    218   \nAzerbaijan                                         0  ...    359    236   \nBahrain                                            0  ...     12     12   \nBangladesh                                       486  ...   4171   4014   \nBhutan                                             0  ...      5     10   \nBrunei Darussalam                                 12  ...      4      5   \nCambodia                                           8  ...    370    529   \nChina                                           1960  ...  42584  33518   \nChina, Hong Kong Special Administrative Region     0  ...    729    712   \nChina, Macao Special Administrative Region         0  ...     21     32   \nCyprus                                            48  ...      7      9   \nDemocratic People's Republic of Korea              0  ...     14     10   \nGeorgia                                            0  ...    114    125   \nIndia                                           7150  ...  36210  33848   \nIndonesia                                        127  ...    632    613   \nIran (Islamic Republic of)                      1794  ...   5837   7480   \nIraq                                             265  ...   2226   1788   \nIsrael                                          1212  ...   2446   2625   \nJapan                                            248  ...   1067   1212   \nJordan                                           181  ...   1940   1827   \nKazakhstan                                         0  ...    506    408   \nKuwait                                             4  ...     66     35   \nKyrgyzstan                                         0  ...    173    161   \nLao People's Democratic Republic                  21  ...     42     74   \nLebanon                                         2576  ...   3709   3802   \nMalaysia                                         425  ...    593    580   \nMaldives                                           0  ...      0      0   \nMongolia                                           0  ...     59     64   \nMyanmar                                           18  ...    210    953   \nNepal                                             13  ...    607    540   \nOman                                               0  ...     14     18   \nPakistan                                         691  ...  14314  13127   \nPhilippines                                     4166  ...  18139  18400   \nQatar                                              1  ...     11      2   \nRepublic of Korea                               1208  ...   5832   6215   \nSaudi Arabia                                       5  ...    198    252   \nSingapore                                        205  ...    392    298   \nSri Lanka                                       1838  ...   4930   4714   \nState of Palestine                                 0  ...    453    627   \nSyrian Arab Republic                             493  ...   1458   1145   \nTajikistan                                         0  ...     85     46   \nThailand                                          78  ...    575    500   \nTurkey                                           257  ...   2065   1638   \nTurkmenistan                                       0  ...     40     26   \nUnited Arab Emirates                               5  ...     31     42   \nUzbekistan                                         0  ...    330    262   \nViet Nam                                        2741  ...   1852   3153   \nYemen                                              7  ...    161    140   \n\n                                                 2007   2008   2009   2010  \\\nAfghanistan                                      2652   2111   1746   1758   \nArmenia                                           198    205    267    252   \nAzerbaijan                                        203    125    165    209   \nBahrain                                            22      9     35     28   \nBangladesh                                       2897   2939   2104   4721   \nBhutan                                              7     36    865   1464   \nBrunei Darussalam                                  11     10      5     12   \nCambodia                                          460    354    203    200   \nChina                                           27642  30037  29622  30391   \nChina, Hong Kong Special Administrative Region    674    897    657    623   \nChina, Macao Special Administrative Region         16     12     21     21   \nCyprus                                              4      7      6     18   \nDemocratic People's Republic of Korea               7     19     11     45   \nGeorgia                                           132    112    128    126   \nIndia                                           28742  28261  29456  34235   \nIndonesia                                         657    661    504    712   \nIran (Islamic Republic of)                       6974   6475   6580   7477   \nIraq                                             2406   3543   5450   5941   \nIsrael                                           2401   2562   2316   2755   \nJapan                                            1250   1284   1194   1168   \nJordan                                           1421   1581   1235   1831   \nKazakhstan                                        436    394    431    377   \nKuwait                                             62     53     68     67   \nKyrgyzstan                                        135    168    173    157   \nLao People's Democratic Republic                   53     32     39     54   \nLebanon                                          3467   3566   3077   3432   \nMalaysia                                          600    658    640    802   \nMaldives                                            2      1      7      4   \nMongolia                                           82     59    118    169   \nMyanmar                                          1887    975   1153    556   \nNepal                                             511    581    561   1392   \nOman                                               16     10      7     14   \nPakistan                                        10124   8994   7217   6811   \nPhilippines                                     19837  24887  28573  38617   \nQatar                                               5      9      6     18   \nRepublic of Korea                                5920   7294   5874   5537   \nSaudi Arabia                                      188    249    246    330   \nSingapore                                         690    734    366    805   \nSri Lanka                                        4123   4756   4547   4422   \nState of Palestine                                441    481    400    654   \nSyrian Arab Republic                             1056    919    917   1039   \nTajikistan                                         44     15     50     52   \nThailand                                          487    519    512    499   \nTurkey                                           1463   1122   1238   1492   \nTurkmenistan                                       37     13     20     30   \nUnited Arab Emirates                               37     33     37     86   \nUzbekistan                                        284    215    288    289   \nViet Nam                                         2574   1784   2171   1942   \nYemen                                             122    133    128    211   \n\n                                                 2011   2012   2013   Total  \nAfghanistan                                      2203   2635   2004   58639  \nArmenia                                           236    258    207    3310  \nAzerbaijan                                        138    161     57    2649  \nBahrain                                            21     39     32     475  \nBangladesh                                       2694   2640   3789   65568  \nBhutan                                           1879   1075    487    5876  \nBrunei Darussalam                                   6      3      6     600  \nCambodia                                          196    233    288    6538  \nChina                                           28502  33024  34129  659962  \nChina, Hong Kong Special Administrative Region    591    728    774    9327  \nChina, Macao Special Administrative Region         13     33     29     284  \nCyprus                                              6     12     16    1126  \nDemocratic People's Republic of Korea              97     66     17     388  \nGeorgia                                           139    147    125    2068  \nIndia                                           27509  30933  33087  691904  \nIndonesia                                         390    395    387   13150  \nIran (Islamic Republic of)                       7479   7534  11291  175923  \nIraq                                             6196   4041   4918   69789  \nIsrael                                           1970   2134   1945   66508  \nJapan                                            1265   1214    982   27707  \nJordan                                           1635   1206   1255   35406  \nKazakhstan                                        381    462    348    8490  \nKuwait                                             58     73     48    2025  \nKyrgyzstan                                        159    278    123    2353  \nLao People's Democratic Republic                   22     25     15    1089  \nLebanon                                          3072   1614   2172  115359  \nMalaysia                                          409    358    204   24417  \nMaldives                                            3      1      1      30  \nMongolia                                          103     68     99     952  \nMyanmar                                           368    193    262    9245  \nNepal                                            1129   1185   1308   10222  \nOman                                               10     13     11     224  \nPakistan                                         7468  11227  12603  241600  \nPhilippines                                     36765  34315  29544  511391  \nQatar                                               3     14      6     157  \nRepublic of Korea                                4588   5316   4509  142581  \nSaudi Arabia                                      278    286    267    3425  \nSingapore                                         219    146    141   14579  \nSri Lanka                                        3309   3338   2394  148358  \nState of Palestine                                555    533    462    6512  \nSyrian Arab Republic                             1005    650   1009   31485  \nTajikistan                                         47     34     39     503  \nThailand                                          396    296    400    9174  \nTurkey                                           1257   1068    729   31781  \nTurkmenistan                                       20     20     14     310  \nUnited Arab Emirates                               60     54     46     836  \nUzbekistan                                        162    235    167    3368  \nViet Nam                                         1723   1731   2112   97146  \nYemen                                             160    174    217    2985  \n\n[49 rows x 38 columns]"
                    },
                    "execution_count": 32,
                    "metadata": {},
                    "output_type": "execute_result"
                }
            ],
            "source": "# 2. pass this condition into the dataFrame\ndf_can[condition]"
        },
        {
            "cell_type": "code",
            "execution_count": 33,
            "metadata": {
                "collapsed": false,
                "deletable": true,
                "editable": true,
                "jupyter": {
                    "outputs_hidden": false
                }
            },
            "outputs": [
                {
                    "data": {
                        "text/html": "<div>\n<style scoped>\n    .dataframe tbody tr th:only-of-type {\n        vertical-align: middle;\n    }\n\n    .dataframe tbody tr th {\n        vertical-align: top;\n    }\n\n    .dataframe thead th {\n        text-align: right;\n    }\n</style>\n<table border=\"1\" class=\"dataframe\">\n  <thead>\n    <tr style=\"text-align: right;\">\n      <th></th>\n      <th>Continent</th>\n      <th>Region</th>\n      <th>DevName</th>\n      <th>1980</th>\n      <th>1981</th>\n      <th>1982</th>\n      <th>1983</th>\n      <th>1984</th>\n      <th>1985</th>\n      <th>1986</th>\n      <th>...</th>\n      <th>2005</th>\n      <th>2006</th>\n      <th>2007</th>\n      <th>2008</th>\n      <th>2009</th>\n      <th>2010</th>\n      <th>2011</th>\n      <th>2012</th>\n      <th>2013</th>\n      <th>Total</th>\n    </tr>\n  </thead>\n  <tbody>\n    <tr>\n      <th>Afghanistan</th>\n      <td>Asia</td>\n      <td>Southern Asia</td>\n      <td>Developing regions</td>\n      <td>16</td>\n      <td>39</td>\n      <td>39</td>\n      <td>47</td>\n      <td>71</td>\n      <td>340</td>\n      <td>496</td>\n      <td>...</td>\n      <td>3436</td>\n      <td>3009</td>\n      <td>2652</td>\n      <td>2111</td>\n      <td>1746</td>\n      <td>1758</td>\n      <td>2203</td>\n      <td>2635</td>\n      <td>2004</td>\n      <td>58639</td>\n    </tr>\n    <tr>\n      <th>Bangladesh</th>\n      <td>Asia</td>\n      <td>Southern Asia</td>\n      <td>Developing regions</td>\n      <td>83</td>\n      <td>84</td>\n      <td>86</td>\n      <td>81</td>\n      <td>98</td>\n      <td>92</td>\n      <td>486</td>\n      <td>...</td>\n      <td>4171</td>\n      <td>4014</td>\n      <td>2897</td>\n      <td>2939</td>\n      <td>2104</td>\n      <td>4721</td>\n      <td>2694</td>\n      <td>2640</td>\n      <td>3789</td>\n      <td>65568</td>\n    </tr>\n    <tr>\n      <th>Bhutan</th>\n      <td>Asia</td>\n      <td>Southern Asia</td>\n      <td>Developing regions</td>\n      <td>0</td>\n      <td>0</td>\n      <td>0</td>\n      <td>0</td>\n      <td>1</td>\n      <td>0</td>\n      <td>0</td>\n      <td>...</td>\n      <td>5</td>\n      <td>10</td>\n      <td>7</td>\n      <td>36</td>\n      <td>865</td>\n      <td>1464</td>\n      <td>1879</td>\n      <td>1075</td>\n      <td>487</td>\n      <td>5876</td>\n    </tr>\n    <tr>\n      <th>India</th>\n      <td>Asia</td>\n      <td>Southern Asia</td>\n      <td>Developing regions</td>\n      <td>8880</td>\n      <td>8670</td>\n      <td>8147</td>\n      <td>7338</td>\n      <td>5704</td>\n      <td>4211</td>\n      <td>7150</td>\n      <td>...</td>\n      <td>36210</td>\n      <td>33848</td>\n      <td>28742</td>\n      <td>28261</td>\n      <td>29456</td>\n      <td>34235</td>\n      <td>27509</td>\n      <td>30933</td>\n      <td>33087</td>\n      <td>691904</td>\n    </tr>\n    <tr>\n      <th>Iran (Islamic Republic of)</th>\n      <td>Asia</td>\n      <td>Southern Asia</td>\n      <td>Developing regions</td>\n      <td>1172</td>\n      <td>1429</td>\n      <td>1822</td>\n      <td>1592</td>\n      <td>1977</td>\n      <td>1648</td>\n      <td>1794</td>\n      <td>...</td>\n      <td>5837</td>\n      <td>7480</td>\n      <td>6974</td>\n      <td>6475</td>\n      <td>6580</td>\n      <td>7477</td>\n      <td>7479</td>\n      <td>7534</td>\n      <td>11291</td>\n      <td>175923</td>\n    </tr>\n    <tr>\n      <th>Maldives</th>\n      <td>Asia</td>\n      <td>Southern Asia</td>\n      <td>Developing regions</td>\n      <td>0</td>\n      <td>0</td>\n      <td>0</td>\n      <td>1</td>\n      <td>0</td>\n      <td>0</td>\n      <td>0</td>\n      <td>...</td>\n      <td>0</td>\n      <td>0</td>\n      <td>2</td>\n      <td>1</td>\n      <td>7</td>\n      <td>4</td>\n      <td>3</td>\n      <td>1</td>\n      <td>1</td>\n      <td>30</td>\n    </tr>\n    <tr>\n      <th>Nepal</th>\n      <td>Asia</td>\n      <td>Southern Asia</td>\n      <td>Developing regions</td>\n      <td>1</td>\n      <td>1</td>\n      <td>6</td>\n      <td>1</td>\n      <td>2</td>\n      <td>4</td>\n      <td>13</td>\n      <td>...</td>\n      <td>607</td>\n      <td>540</td>\n      <td>511</td>\n      <td>581</td>\n      <td>561</td>\n      <td>1392</td>\n      <td>1129</td>\n      <td>1185</td>\n      <td>1308</td>\n      <td>10222</td>\n    </tr>\n    <tr>\n      <th>Pakistan</th>\n      <td>Asia</td>\n      <td>Southern Asia</td>\n      <td>Developing regions</td>\n      <td>978</td>\n      <td>972</td>\n      <td>1201</td>\n      <td>900</td>\n      <td>668</td>\n      <td>514</td>\n      <td>691</td>\n      <td>...</td>\n      <td>14314</td>\n      <td>13127</td>\n      <td>10124</td>\n      <td>8994</td>\n      <td>7217</td>\n      <td>6811</td>\n      <td>7468</td>\n      <td>11227</td>\n      <td>12603</td>\n      <td>241600</td>\n    </tr>\n    <tr>\n      <th>Sri Lanka</th>\n      <td>Asia</td>\n      <td>Southern Asia</td>\n      <td>Developing regions</td>\n      <td>185</td>\n      <td>371</td>\n      <td>290</td>\n      <td>197</td>\n      <td>1086</td>\n      <td>845</td>\n      <td>1838</td>\n      <td>...</td>\n      <td>4930</td>\n      <td>4714</td>\n      <td>4123</td>\n      <td>4756</td>\n      <td>4547</td>\n      <td>4422</td>\n      <td>3309</td>\n      <td>3338</td>\n      <td>2394</td>\n      <td>148358</td>\n    </tr>\n  </tbody>\n</table>\n<p>9 rows \u00d7 38 columns</p>\n</div>",
                        "text/plain": "                           Continent         Region             DevName  1980  \\\nAfghanistan                     Asia  Southern Asia  Developing regions    16   \nBangladesh                      Asia  Southern Asia  Developing regions    83   \nBhutan                          Asia  Southern Asia  Developing regions     0   \nIndia                           Asia  Southern Asia  Developing regions  8880   \nIran (Islamic Republic of)      Asia  Southern Asia  Developing regions  1172   \nMaldives                        Asia  Southern Asia  Developing regions     0   \nNepal                           Asia  Southern Asia  Developing regions     1   \nPakistan                        Asia  Southern Asia  Developing regions   978   \nSri Lanka                       Asia  Southern Asia  Developing regions   185   \n\n                            1981  1982  1983  1984  1985  1986  ...   2005  \\\nAfghanistan                   39    39    47    71   340   496  ...   3436   \nBangladesh                    84    86    81    98    92   486  ...   4171   \nBhutan                         0     0     0     1     0     0  ...      5   \nIndia                       8670  8147  7338  5704  4211  7150  ...  36210   \nIran (Islamic Republic of)  1429  1822  1592  1977  1648  1794  ...   5837   \nMaldives                       0     0     1     0     0     0  ...      0   \nNepal                          1     6     1     2     4    13  ...    607   \nPakistan                     972  1201   900   668   514   691  ...  14314   \nSri Lanka                    371   290   197  1086   845  1838  ...   4930   \n\n                             2006   2007   2008   2009   2010   2011   2012  \\\nAfghanistan                  3009   2652   2111   1746   1758   2203   2635   \nBangladesh                   4014   2897   2939   2104   4721   2694   2640   \nBhutan                         10      7     36    865   1464   1879   1075   \nIndia                       33848  28742  28261  29456  34235  27509  30933   \nIran (Islamic Republic of)   7480   6974   6475   6580   7477   7479   7534   \nMaldives                        0      2      1      7      4      3      1   \nNepal                         540    511    581    561   1392   1129   1185   \nPakistan                    13127  10124   8994   7217   6811   7468  11227   \nSri Lanka                    4714   4123   4756   4547   4422   3309   3338   \n\n                             2013   Total  \nAfghanistan                  2004   58639  \nBangladesh                   3789   65568  \nBhutan                        487    5876  \nIndia                       33087  691904  \nIran (Islamic Republic of)  11291  175923  \nMaldives                        1      30  \nNepal                        1308   10222  \nPakistan                    12603  241600  \nSri Lanka                    2394  148358  \n\n[9 rows x 38 columns]"
                    },
                    "execution_count": 33,
                    "metadata": {},
                    "output_type": "execute_result"
                }
            ],
            "source": "# we can pass mutliple criteria in the same line. \n# let's filter for AreaNAme = Asia and RegName = Southern Asia\n\ndf_can[(df_can['Continent']=='Asia') & (df_can['Region']=='Southern Asia')]\n\n# note: When using 'and' and 'or' operators, pandas requires we use '&' and '|' instead of 'and' and 'or'\n# don't forget to enclose the two conditions in parentheses"
        },
        {
            "cell_type": "markdown",
            "metadata": {
                "deletable": true,
                "editable": true
            },
            "source": "Before we proceed: let's review the changes we have made to our dataframe."
        },
        {
            "cell_type": "code",
            "execution_count": 34,
            "metadata": {
                "collapsed": false,
                "deletable": true,
                "editable": true,
                "jupyter": {
                    "outputs_hidden": false
                },
                "scrolled": true
            },
            "outputs": [
                {
                    "name": "stdout",
                    "output_type": "stream",
                    "text": "data dimensions: (195, 38)\nIndex(['Continent', 'Region', 'DevName', '1980', '1981', '1982', '1983',\n       '1984', '1985', '1986', '1987', '1988', '1989', '1990', '1991', '1992',\n       '1993', '1994', '1995', '1996', '1997', '1998', '1999', '2000', '2001',\n       '2002', '2003', '2004', '2005', '2006', '2007', '2008', '2009', '2010',\n       '2011', '2012', '2013', 'Total'],\n      dtype='object')\n"
                },
                {
                    "data": {
                        "text/html": "<div>\n<style scoped>\n    .dataframe tbody tr th:only-of-type {\n        vertical-align: middle;\n    }\n\n    .dataframe tbody tr th {\n        vertical-align: top;\n    }\n\n    .dataframe thead th {\n        text-align: right;\n    }\n</style>\n<table border=\"1\" class=\"dataframe\">\n  <thead>\n    <tr style=\"text-align: right;\">\n      <th></th>\n      <th>Continent</th>\n      <th>Region</th>\n      <th>DevName</th>\n      <th>1980</th>\n      <th>1981</th>\n      <th>1982</th>\n      <th>1983</th>\n      <th>1984</th>\n      <th>1985</th>\n      <th>1986</th>\n      <th>...</th>\n      <th>2005</th>\n      <th>2006</th>\n      <th>2007</th>\n      <th>2008</th>\n      <th>2009</th>\n      <th>2010</th>\n      <th>2011</th>\n      <th>2012</th>\n      <th>2013</th>\n      <th>Total</th>\n    </tr>\n  </thead>\n  <tbody>\n    <tr>\n      <th>Afghanistan</th>\n      <td>Asia</td>\n      <td>Southern Asia</td>\n      <td>Developing regions</td>\n      <td>16</td>\n      <td>39</td>\n      <td>39</td>\n      <td>47</td>\n      <td>71</td>\n      <td>340</td>\n      <td>496</td>\n      <td>...</td>\n      <td>3436</td>\n      <td>3009</td>\n      <td>2652</td>\n      <td>2111</td>\n      <td>1746</td>\n      <td>1758</td>\n      <td>2203</td>\n      <td>2635</td>\n      <td>2004</td>\n      <td>58639</td>\n    </tr>\n    <tr>\n      <th>Albania</th>\n      <td>Europe</td>\n      <td>Southern Europe</td>\n      <td>Developed regions</td>\n      <td>1</td>\n      <td>0</td>\n      <td>0</td>\n      <td>0</td>\n      <td>0</td>\n      <td>0</td>\n      <td>1</td>\n      <td>...</td>\n      <td>1223</td>\n      <td>856</td>\n      <td>702</td>\n      <td>560</td>\n      <td>716</td>\n      <td>561</td>\n      <td>539</td>\n      <td>620</td>\n      <td>603</td>\n      <td>15699</td>\n    </tr>\n  </tbody>\n</table>\n<p>2 rows \u00d7 38 columns</p>\n</div>",
                        "text/plain": "            Continent           Region             DevName  1980  1981  1982  \\\nAfghanistan      Asia    Southern Asia  Developing regions    16    39    39   \nAlbania        Europe  Southern Europe   Developed regions     1     0     0   \n\n             1983  1984  1985  1986  ...  2005  2006  2007  2008  2009  2010  \\\nAfghanistan    47    71   340   496  ...  3436  3009  2652  2111  1746  1758   \nAlbania         0     0     0     1  ...  1223   856   702   560   716   561   \n\n             2011  2012  2013  Total  \nAfghanistan  2203  2635  2004  58639  \nAlbania       539   620   603  15699  \n\n[2 rows x 38 columns]"
                    },
                    "execution_count": 34,
                    "metadata": {},
                    "output_type": "execute_result"
                }
            ],
            "source": "print ('data dimensions:', df_can.shape)\nprint(df_can.columns)\ndf_can.head(2)"
        },
        {
            "cell_type": "markdown",
            "metadata": {
                "deletable": true,
                "editable": true
            },
            "source": "---\n# Visualizing Data using Matplotlib<a id=\"8\"></a>"
        },
        {
            "cell_type": "markdown",
            "metadata": {
                "deletable": true,
                "editable": true
            },
            "source": "## Matplotlib: Standard Python Visualization Library<a id=\"10\"></a>\n\nThe primary plotting library we will explore in the course is [Matplotlib](http://matplotlib.org/).  As mentioned on their website: \n>Matplotlib is a Python 2D plotting library which produces publication quality figures in a variety of hardcopy formats and interactive environments across platforms. Matplotlib can be used in Python scripts, the Python and IPython shell, the jupyter notebook, web application servers, and four graphical user interface toolkits.\n\nIf you are aspiring to create impactful visualization with python, Matplotlib is an essential tool to have at your disposal."
        },
        {
            "cell_type": "markdown",
            "metadata": {
                "deletable": true,
                "editable": true
            },
            "source": "### Matplotlib.Pyplot\n\nOne of the core aspects of Matplotlib is `matplotlib.pyplot`. It is Matplotlib's scripting layer which we studied in details in the videos about Matplotlib. Recall that it is a collection of command style functions that make Matplotlib work like MATLAB. Each `pyplot` function makes some change to a figure: e.g., creates a figure, creates a plotting area in a figure, plots some lines in a plotting area, decorates the plot with labels, etc. In this lab, we will work with the scripting layer to learn how to generate line plots. In future labs, we will get to work with the Artist layer as well to experiment first hand how it differs from the scripting layer. \n"
        },
        {
            "cell_type": "markdown",
            "metadata": {
                "deletable": true,
                "editable": true
            },
            "source": "Let us start by importing `Matplotlib` and `Matplotlib.pyplot` as follows:"
        },
        {
            "cell_type": "code",
            "execution_count": 35,
            "metadata": {
                "collapsed": false,
                "deletable": true,
                "editable": true,
                "jupyter": {
                    "outputs_hidden": false
                }
            },
            "outputs": [],
            "source": "# we are using the inline backend\n%matplotlib inline \n\nimport matplotlib as mpl\nimport matplotlib.pyplot as plt"
        },
        {
            "cell_type": "markdown",
            "metadata": {
                "deletable": true,
                "editable": true
            },
            "source": "*optional: check if Matplotlib is loaded."
        },
        {
            "cell_type": "code",
            "execution_count": 36,
            "metadata": {
                "collapsed": false,
                "deletable": true,
                "editable": true,
                "jupyter": {
                    "outputs_hidden": false
                }
            },
            "outputs": [
                {
                    "name": "stdout",
                    "output_type": "stream",
                    "text": "Matplotlib version:  3.1.0\n"
                }
            ],
            "source": "print ('Matplotlib version: ', mpl.__version__ ) # >= 2.0.0"
        },
        {
            "cell_type": "markdown",
            "metadata": {
                "deletable": true,
                "editable": true
            },
            "source": "*optional: apply a style to Matplotlib."
        },
        {
            "cell_type": "code",
            "execution_count": 37,
            "metadata": {
                "collapsed": false,
                "deletable": true,
                "editable": true,
                "jupyter": {
                    "outputs_hidden": false
                }
            },
            "outputs": [
                {
                    "name": "stdout",
                    "output_type": "stream",
                    "text": "['grayscale', 'seaborn-pastel', 'seaborn-ticks', 'seaborn-whitegrid', 'seaborn-poster', 'Solarize_Light2', 'tableau-colorblind10', 'seaborn-deep', 'dark_background', 'seaborn', 'ggplot', 'seaborn-dark-palette', 'seaborn-bright', '_classic_test', 'bmh', 'seaborn-paper', 'seaborn-talk', 'fast', 'seaborn-darkgrid', 'seaborn-notebook', 'seaborn-white', 'fivethirtyeight', 'seaborn-muted', 'classic', 'seaborn-dark', 'seaborn-colorblind']\n"
                }
            ],
            "source": "print(plt.style.available)\nmpl.style.use(['ggplot']) # optional: for ggplot-like style"
        },
        {
            "cell_type": "markdown",
            "metadata": {
                "deletable": true,
                "editable": true
            },
            "source": "### Plotting in *pandas*\n\nFortunately, pandas has a built-in implementation of Matplotlib that we can use. Plotting in *pandas* is as simple as appending a `.plot()` method to a series or dataframe.\n\nDocumentation:\n- [Plotting with Series](http://pandas.pydata.org/pandas-docs/stable/api.html#plotting)<br>\n- [Plotting with Dataframes](http://pandas.pydata.org/pandas-docs/stable/api.html#api-dataframe-plotting)"
        },
        {
            "cell_type": "markdown",
            "metadata": {
                "deletable": true,
                "editable": true
            },
            "source": "# Line Pots (Series/Dataframe) <a id=\"12\"></a>"
        },
        {
            "cell_type": "markdown",
            "metadata": {
                "deletable": true,
                "editable": true
            },
            "source": "**What is a line plot and why use it?**\n\nA line chart or line plot is a type of plot which displays information as a series of data points called 'markers' connected by straight line segments. It is a basic type of chart common in many fields.\nUse line plot when you have a continuous data set. These are best suited for trend-based visualizations of data over a period of time."
        },
        {
            "cell_type": "markdown",
            "metadata": {
                "deletable": true,
                "editable": true
            },
            "source": "**Let's start with a case study:**\n\nIn 2010, Haiti suffered a catastrophic magnitude 7.0 earthquake. The quake caused widespread devastation and loss of life and aout three million people were affected by this natural disaster. As part of Canada's humanitarian effort, the Government of Canada stepped up its effort in accepting refugees from Haiti. We can quickly visualize this effort using a `Line` plot:\n\n**Question:** Plot a line graph of immigration from Haiti using `df.plot()`.\n"
        },
        {
            "cell_type": "markdown",
            "metadata": {
                "deletable": true,
                "editable": true
            },
            "source": "First, we will extract the data series for Haiti."
        },
        {
            "cell_type": "code",
            "execution_count": 38,
            "metadata": {
                "collapsed": false,
                "deletable": true,
                "editable": true,
                "jupyter": {
                    "outputs_hidden": false
                }
            },
            "outputs": [
                {
                    "data": {
                        "text/plain": "1980    1666\n1981    3692\n1982    3498\n1983    2860\n1984    1418\nName: Haiti, dtype: object"
                    },
                    "execution_count": 38,
                    "metadata": {},
                    "output_type": "execute_result"
                }
            ],
            "source": "haiti = df_can.loc['Haiti', years] # Passing in years 1980 - 2013 to exclude the 'total' column\nhaiti.head()"
        },
        {
            "cell_type": "markdown",
            "metadata": {
                "deletable": true,
                "editable": true
            },
            "source": "Next, we will plot a line plot by appending `.plot()` to the `haiti` dataframe."
        },
        {
            "cell_type": "code",
            "execution_count": null,
            "metadata": {
                "collapsed": false,
                "deletable": true,
                "editable": true,
                "jupyter": {
                    "outputs_hidden": false
                }
            },
            "outputs": [],
            "source": "haiti.plot()"
        },
        {
            "cell_type": "markdown",
            "metadata": {
                "deletable": true,
                "editable": true
            },
            "source": "*pandas* automatically populated the x-axis with the index values (years), and the y-axis with the column values (population).\n\nLet us label the x and y axis using `plt.title()`, `plt.ylabel()`, and `plt.xlabel()` as follows:"
        },
        {
            "cell_type": "code",
            "execution_count": null,
            "metadata": {
                "collapsed": false,
                "deletable": true,
                "editable": true,
                "jupyter": {
                    "outputs_hidden": false
                },
                "scrolled": true
            },
            "outputs": [],
            "source": "haiti.plot(kind='line')\n\nplt.title('Immigration from Haiti')\nplt.ylabel('Number of immigrants')\nplt.xlabel('Years')\n\nplt.show() # Need this line to show the updates made to the figure"
        },
        {
            "cell_type": "markdown",
            "metadata": {
                "deletable": true,
                "editable": true
            },
            "source": "We can clearly notice how number of immigrants from Haiti spiked up from 2010 as Canada stepped up its efforts to accept refugees from Haiti. Let us annotate this spike in the plot by using the `plt.text()` method."
        },
        {
            "cell_type": "code",
            "execution_count": null,
            "metadata": {
                "collapsed": false,
                "deletable": true,
                "editable": true,
                "jupyter": {
                    "outputs_hidden": false
                }
            },
            "outputs": [],
            "source": "haiti.plot(kind='line')\n\nplt.title('Immigration from Haiti')\nplt.ylabel('Number of Immigrants')\nplt.xlabel('Years')\n\n# annotate the 2010 Earthquake. \n# syntax: plt.text(x, y, label)\nplt.text(20, 6000, '2010 Earthquake') # see note below\n\nplt.show() "
        },
        {
            "cell_type": "markdown",
            "metadata": {
                "deletable": true,
                "editable": true
            },
            "source": "With just a few lines of code, you were able to quickly identify and visualize the spike in immigration!\n\nQuick note on x and y values in `plt.text(x, y, label)`:\n\n    Since the x-axis (years) is type 'string', we need to specify the years in terms of its index position. Eg 20th index is year 2000. The y axis (number of Immigrants) is type 'integer', so we can just specify the value y = 6000.\n```python\n    plt.text(20, 6000, '2010 Earthquake') # years stored as type str\n```\n    If the years were stored as type 'integer' or 'float', we would have specified x = 2000 instead.\n```python\n    plt.text(2000, 6000, '2010 Earthquake') # years stored as type int\n```\n    We will cover advanced annotation methods in later modules."
        },
        {
            "cell_type": "markdown",
            "metadata": {
                "deletable": true,
                "editable": true
            },
            "source": "We can easily add more countries to line plot to make meaningful comparisons immigration from different countries. \n\n**Question:** Let us compare the number of immigrants from India and China from 1980 to 2013.\n"
        },
        {
            "cell_type": "markdown",
            "metadata": {
                "deletable": true,
                "editable": true
            },
            "source": "Step 1: Get the data set for China and India."
        },
        {
            "cell_type": "code",
            "execution_count": null,
            "metadata": {
                "collapsed": false,
                "deletable": true,
                "editable": true,
                "jupyter": {
                    "outputs_hidden": false
                }
            },
            "outputs": [],
            "source": "df_CI = df_can.loc[['India', 'China'], years]\ndf_CI.head()"
        },
        {
            "cell_type": "markdown",
            "metadata": {
                "deletable": true,
                "editable": true
            },
            "source": "Step 2: Plot graph. We will explicitly specify line plot by passing in `kind` parameter to `plot()`."
        },
        {
            "cell_type": "code",
            "execution_count": null,
            "metadata": {
                "collapsed": false,
                "deletable": true,
                "editable": true,
                "jupyter": {
                    "outputs_hidden": false
                },
                "scrolled": true
            },
            "outputs": [],
            "source": "df_CI.plot(kind='line')"
        },
        {
            "cell_type": "markdown",
            "metadata": {
                "deletable": true,
                "editable": true
            },
            "source": "That doesn't look right...\n\nRecall that *pandas* plots the indices on the x-axis and the columns as individual lines on the y-axis. Since `df_CI` is a dataframe with the `country` as the index and `years` as the columns, we must first transpose the dataframe using `transpose()` method to swap the row and columns."
        },
        {
            "cell_type": "code",
            "execution_count": null,
            "metadata": {
                "collapsed": false,
                "deletable": true,
                "editable": true,
                "jupyter": {
                    "outputs_hidden": false
                }
            },
            "outputs": [],
            "source": "df_CI = df_CI.transpose()\ndf_CI.head()"
        },
        {
            "cell_type": "markdown",
            "metadata": {
                "deletable": true,
                "editable": true
            },
            "source": "*pandas* will auomatically graph the two countries on the same graph. Note that we can alternatively pass a `title` parameter to the `plot()` method as an alternate approach to add the title."
        },
        {
            "cell_type": "code",
            "execution_count": null,
            "metadata": {
                "collapsed": false,
                "deletable": true,
                "editable": true,
                "jupyter": {
                    "outputs_hidden": false
                }
            },
            "outputs": [],
            "source": "df_CI.plot(kind='line')\n\nplt.title('Immigrants from China and India')\nplt.ylabel('Number of Immigrants')\nplt.xlabel('Years')\n\nplt.show() "
        },
        {
            "cell_type": "markdown",
            "metadata": {
                "deletable": true,
                "editable": true
            },
            "source": "From the above plot, we can observe that the China and India have very similar immigration trends through the years. "
        },
        {
            "cell_type": "markdown",
            "metadata": {
                "deletable": true,
                "editable": true
            },
            "source": "*Note*: How come we didn't need to transpose Haiti's dataframe before plotting (like we did for df_CI)?\n\nThat's because `haiti` is a series as opposed to a dataframe, and has the years as its indices as shown below. \n```python\nprint(type(haiti))\nprint(haiti.head(5))\n```\n>class 'pandas.core.series.Series' <br>\n>1980    1666 <br>\n>1981    3692 <br>\n>1982    3498 <br>\n>1983    2860 <br>\n>1984    1418 <br>\n>Name: Haiti, dtype: int64 <br>"
        },
        {
            "cell_type": "markdown",
            "metadata": {
                "deletable": true,
                "editable": true
            },
            "source": "Line plot is a handy tool to display several dependent variables against one independent variable. However, it is recommended that no more than 5-10 lines on a single graph; any more than that and it becomes difficult to interpret."
        },
        {
            "cell_type": "markdown",
            "metadata": {
                "deletable": true,
                "editable": true
            },
            "source": "**Question:** Compare the trend of top 5 countries that contributed the most to immigration to Canada."
        },
        {
            "cell_type": "markdown",
            "metadata": {
                "deletable": true,
                "editable": true
            },
            "source": "Step 1: Get the dataset. Recall that we created a `Total` column that calculates the cumulative immigration by country. We will sort on this column to get our top 5 countries using pandas `sort_values()` method."
        },
        {
            "cell_type": "code",
            "execution_count": null,
            "metadata": {
                "collapsed": false,
                "deletable": true,
                "editable": true,
                "jupyter": {
                    "outputs_hidden": false
                },
                "scrolled": true
            },
            "outputs": [],
            "source": "# inplace = True paramemter saves the changes to the original df_can dataframe\ndf_can.sort_values(by='Total', ascending=False, axis=0, inplace=True)\n\n# get the top 5 entries\ndf_top5 = df_can.head(5)\n\n# transpose the dataframe\ndf_top5 = df_top5[years].transpose() \n\ndf_top5"
        },
        {
            "cell_type": "markdown",
            "metadata": {
                "deletable": true,
                "editable": true
            },
            "source": "Step 2: Plot the dataframe. To make the plot more readeable, we will change the size using the `figsize` parameter."
        },
        {
            "cell_type": "code",
            "execution_count": null,
            "metadata": {
                "collapsed": false,
                "jupyter": {
                    "outputs_hidden": false
                }
            },
            "outputs": [],
            "source": "df_top5.plot(kind='line', figsize=(14, 8)) # pass a tuple (x, y) size\n\nplt.title('Immigration Trend of Top 5 Countries')\nplt.ylabel('Number of Immigrants')\nplt.xlabel('Years')\n\nplt.show()"
        },
        {
            "cell_type": "markdown",
            "metadata": {
                "deletable": true,
                "editable": true
            },
            "source": "### Other Plots\n\nCongratulations! you have learned how to wrangle data with python and create a line plot with Matplotlib. There are many other plotting styles available other than the default Line plot, all of which can be accessed by passing `kind` keyword to `plot()`. The full list of available plots are as follows:\n\n* `bar` for vertical bar plots\n* `barh` for horizontal bar plots\n* `hist` for histogram\n* `box` for boxplot\n* `kde` or `density` for density plots\n* `area` for area plots\n* `pie` for pie plots\n* `scatter` for scatter plots\n* `hexbin` for hexbin plot"
        },
        {
            "cell_type": "markdown",
            "metadata": {
                "deletable": true,
                "editable": true
            },
            "source": "### Thank you for completing this lab!\n\nThis notebook was created by [Jay Rajasekharan](https://www.linkedin.com/in/jayrajasekharan) with contributions from [Ehsan M. Kermani](https://www.linkedin.com/in/ehsanmkermani), and [Slobodan Markovic](https://www.linkedin.com/in/slobodan-markovic), and revised by [Alex Aklson](https://www.linkedin.com/in/aklson/). I hope you found this lab session interesting. Feel free to contact me if you have any questions!"
        },
        {
            "cell_type": "markdown",
            "metadata": {
                "deletable": true,
                "editable": true
            },
            "source": "This notebook is part of the free course on **Cognitive Class** called *Data Visualization with Python*. If you accessed this notebook outside the course, you can take this free self-paced course, online by clicking [here](https://cocl.us/DV0101EN_Lab1)."
        },
        {
            "cell_type": "markdown",
            "metadata": {
                "deletable": true,
                "editable": true
            },
            "source": "<hr>\nCopyright &copy; 2017 [Cognitive Class](https://cognitiveclass.ai/?utm_source=bducopyrightlink&utm_medium=dswb&utm_campaign=bdu). This notebook and its source code are released under the terms of the [MIT License](https://bigdatauniversity.com/mit-license/)."
        }
    ],
    "metadata": {
        "kernelspec": {
            "display_name": "Python",
            "language": "python",
            "name": "conda-env-python-py"
        },
        "language_info": {
            "codemirror_mode": {
                "name": "ipython",
                "version": 3
            },
            "file_extension": ".py",
            "mimetype": "text/x-python",
            "name": "python",
            "nbconvert_exporter": "python",
            "pygments_lexer": "ipython3",
            "version": "3.6.7"
        }
    },
    "nbformat": 4,
    "nbformat_minor": 4
}