student tutorial.ipynb

{
 "cells": [
  {
   "cell_type": "markdown",
   "metadata": {
    "slideshow": {
     "slide_type": "slide"
    }
   },
   "source": [
    "# NumPy and Pandas Tutorial\n",
    "## HODP Bootcamp Week 4\n",
    "### October 10, 2018"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {
    "slideshow": {
     "slide_type": "subslide"
    }
   },
   "source": [
    "## Some Python refreshers . . . \n",
    "- datatypes (strings, integers)\n",
    "- functions\n",
    "- data structures like lists and dictionaries"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 2,
   "metadata": {
    "collapsed": true
   },
   "outputs": [],
   "source": [
    "lst = [1, \"Emma\", 5.0, {\"name\": \"Emma\", \"age\": 20}]"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 3,
   "metadata": {},
   "outputs": [
    {
     "data": {
      "text/plain": [
       "1"
      ]
     },
     "execution_count": 3,
     "metadata": {},
     "output_type": "execute_result"
    }
   ],
   "source": [
    "# Get the first element of the list\n",
    "lst[0]"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 4,
   "metadata": {},
   "outputs": [
    {
     "data": {
      "text/plain": [
       "{'name': 'Emma', 'age': 20}"
      ]
     },
     "execution_count": 4,
     "metadata": {},
     "output_type": "execute_result"
    }
   ],
   "source": [
    "# Get the last element of the list\n",
    "lst[-1]"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 5,
   "metadata": {},
   "outputs": [
    {
     "name": "stdout",
     "output_type": "stream",
     "text": [
      "name\n",
      "Emma\n",
      "age\n",
      "20\n"
     ]
    }
   ],
   "source": [
    "# Get all of the keys of the dictionary\n",
    "for key in lst[-1]:\n",
    "    print(key)\n",
    "    print(lst[-1][key])"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": null,
   "metadata": {
    "collapsed": true
   },
   "outputs": [],
   "source": [
    "# Get all of the values of the dictionary"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {
    "slideshow": {
     "slide_type": "subslide"
    }
   },
   "source": [
    "## This week:\n",
    "* Learn how to use Python libraries numpy and pandas to make data analysis easy and efficient\n",
    "* Understand key differences between Python, NumPy, Pandas, and more traditional tools like Google Sheets\n",
    "* Practice your new data science skills!"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {
    "slideshow": {
     "slide_type": "slide"
    }
   },
   "source": [
    "## Getting Started"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 6,
   "metadata": {
    "collapsed": true,
    "slideshow": {
     "slide_type": "subslide"
    }
   },
   "outputs": [],
   "source": [
    "import numpy as np\n",
    "import pandas as pd"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {
    "slideshow": {
     "slide_type": "slide"
    }
   },
   "source": [
    "## Python vs. NumPy\n",
    "* Python lists are flexible, but bugs can be tough to find and for-loops to manipulate data can be slow\n",
    "* NumPy arrays have fixed types and functions can be __vectorized__ and operations can be __broadcast__ across arrays"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 7,
   "metadata": {
    "slideshow": {
     "slide_type": "subslide"
    }
   },
   "outputs": [
    {
     "data": {
      "text/plain": [
       "['Emma', 'Jeffrey', 1, 2]"
      ]
     },
     "execution_count": 7,
     "metadata": {},
     "output_type": "execute_result"
    }
   ],
   "source": [
    "lst = [\"Emma\", \"Jeffrey\", 1, 2] # This is a valid Python list\n",
    "lst"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 8,
   "metadata": {
    "slideshow": {
     "slide_type": "subslide"
    }
   },
   "outputs": [
    {
     "data": {
      "text/plain": [
       "array(['Emma', 'Jeffrey', '1', '2'],\n",
       "      dtype='<U7')"
      ]
     },
     "execution_count": 8,
     "metadata": {},
     "output_type": "execute_result"
    }
   ],
   "source": [
    "np_lst = np.array(lst) # Numpy forces them all to be strings\n",
    "np_lst"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 10,
   "metadata": {
    "slideshow": {
     "slide_type": "subslide"
    }
   },
   "outputs": [
    {
     "name": "stdout",
     "output_type": "stream",
     "text": [
      "Emma 4\n",
      "Jeffrey 4\n"
     ]
    },
    {
     "ename": "TypeError",
     "evalue": "unsupported operand type(s) for +: 'int' and 'str'",
     "output_type": "error",
     "traceback": [
      "\u001b[0;31m---------------------------------------------------------------------------\u001b[0m",
      "\u001b[0;31mTypeError\u001b[0m                                 Traceback (most recent call last)",
      "\u001b[0;32m<ipython-input-10-22c694a6c4ca>\u001b[0m in \u001b[0;36m<module>\u001b[0;34m\u001b[0m\n\u001b[1;32m      1\u001b[0m \u001b[0;32mfor\u001b[0m \u001b[0melt\u001b[0m \u001b[0;32min\u001b[0m \u001b[0mlst\u001b[0m\u001b[0;34m:\u001b[0m\u001b[0;34m\u001b[0m\u001b[0m\n\u001b[0;32m----> 2\u001b[0;31m     \u001b[0mprint\u001b[0m\u001b[0;34m(\u001b[0m\u001b[0melt\u001b[0m \u001b[0;34m+\u001b[0m \u001b[0;34m\" 4\"\u001b[0m\u001b[0;34m)\u001b[0m\u001b[0;34m\u001b[0m\u001b[0m\n\u001b[0m",
      "\u001b[0;31mTypeError\u001b[0m: unsupported operand type(s) for +: 'int' and 'str'"
     ]
    }
   ],
   "source": [
    "for elt in lst:\n",
    "    print(elt + \" 4\")"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 11,
   "metadata": {
    "slideshow": {
     "slide_type": "subslide"
    }
   },
   "outputs": [
    {
     "name": "stdout",
     "output_type": "stream",
     "text": [
      "Emma is fun\n",
      "Jeffrey is fun\n",
      "1 is fun\n",
      "2 is fun\n"
     ]
    }
   ],
   "source": [
    "for elt in np_lst:\n",
    "    print(elt + \" is fun\")"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {
    "slideshow": {
     "slide_type": "slide"
    }
   },
   "source": [
    "## Creating NumPy arrays"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {
    "slideshow": {
     "slide_type": "fragment"
    }
   },
   "source": [
    "First, we can use ``np.array`` to create arrays from Python lists:"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 12,
   "metadata": {
    "slideshow": {
     "slide_type": "fragment"
    }
   },
   "outputs": [
    {
     "data": {
      "text/plain": [
       "array([1, 4, 2, 5, 3])"
      ]
     },
     "execution_count": 12,
     "metadata": {},
     "output_type": "execute_result"
    }
   ],
   "source": [
    "# integer array:\n",
    "np.array([1, 4, 2, 5, 3])"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {
    "slideshow": {
     "slide_type": "subslide"
    }
   },
   "source": [
    "Remember that unlike Python lists, NumPy is constrained to arrays that all contain the same type.\n",
    "If types do not match, NumPy will upcast if possible (here, integers are up-cast to floating point):"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 13,
   "metadata": {
    "slideshow": {
     "slide_type": "fragment"
    }
   },
   "outputs": [
    {
     "data": {
      "text/plain": [
       "array([ 3.14,  4.  ,  2.  ,  3.  ])"
      ]
     },
     "execution_count": 13,
     "metadata": {},
     "output_type": "execute_result"
    }
   ],
   "source": [
    "np.array([3.14, 4, 2, 3]) # Notice how the elements in the resulting array are all floats"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 14,
   "metadata": {
    "slideshow": {
     "slide_type": "fragment"
    }
   },
   "outputs": [
    {
     "data": {
      "text/plain": [
       "array([ 1.,  2.,  3.,  4.], dtype=float32)"
      ]
     },
     "execution_count": 14,
     "metadata": {},
     "output_type": "execute_result"
    }
   ],
   "source": [
    "np.array([1, 2, 3, 4], dtype='float32') # You can explicitly set the type with the dtype keyword"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {
    "slideshow": {
     "slide_type": "subslide"
    }
   },
   "source": [
    "Numpy has a bunch of handy built-in functions to generate arrays:"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 16,
   "metadata": {
    "slideshow": {
     "slide_type": "fragment"
    }
   },
   "outputs": [
    {
     "data": {
      "text/plain": [
       "array([0, 0, 0, 0, 0, 0, 0, 0, 0, 0])"
      ]
     },
     "execution_count": 16,
     "metadata": {},
     "output_type": "execute_result"
    }
   ],
   "source": [
    "# Create a length-10 integer array filled with zeros\n",
    "np.zeros(10, dtype=int)"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 17,
   "metadata": {
    "slideshow": {
     "slide_type": "fragment"
    }
   },
   "outputs": [
    {
     "data": {
      "text/plain": [
       "array([[ 1.,  1.,  1.,  1.,  1.],\n",
       "       [ 1.,  1.,  1.,  1.,  1.],\n",
       "       [ 1.,  1.,  1.,  1.,  1.]])"
      ]
     },
     "execution_count": 17,
     "metadata": {},
     "output_type": "execute_result"
    }
   ],
   "source": [
    "# Create a 3x5 floating-point array filled with ones\n",
    "np.ones((3, 5), dtype=float)"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 18,
   "metadata": {
    "slideshow": {
     "slide_type": "subslide"
    }
   },
   "outputs": [
    {
     "data": {
      "text/plain": [
       "array([ 0.  ,  0.25,  0.5 ,  0.75,  1.  ])"
      ]
     },
     "execution_count": 18,
     "metadata": {},
     "output_type": "execute_result"
    }
   ],
   "source": [
    "# Create an array of five values evenly spaced between 0 and 1\n",
    "np.linspace(0, 1, 5)"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 20,
   "metadata": {
    "slideshow": {
     "slide_type": "fragment"
    }
   },
   "outputs": [
    {
     "data": {
      "text/plain": [
       "array([[0, 1, 2],\n",
       "       [3, 4, 5],\n",
       "       [6, 7, 8]])"
      ]
     },
     "execution_count": 20,
     "metadata": {},
     "output_type": "execute_result"
    }
   ],
   "source": [
    "array = np.arange(9).reshape(3,3)\n",
    "array"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {
    "slideshow": {
     "slide_type": "subslide"
    }
   },
   "source": [
    "We can slice NumPy arrays and index into them using bracket notation:"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 21,
   "metadata": {
    "slideshow": {
     "slide_type": "fragment"
    }
   },
   "outputs": [
    {
     "data": {
      "text/plain": [
       "array([[0, 1, 2],\n",
       "       [3, 4, 5],\n",
       "       [6, 7, 8]])"
      ]
     },
     "execution_count": 21,
     "metadata": {},
     "output_type": "execute_result"
    }
   ],
   "source": [
    "array"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 23,
   "metadata": {
    "slideshow": {
     "slide_type": "fragment"
    }
   },
   "outputs": [
    {
     "data": {
      "text/plain": [
       "1"
      ]
     },
     "execution_count": 23,
     "metadata": {},
     "output_type": "execute_result"
    }
   ],
   "source": [
    "array[0, 1]"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 28,
   "metadata": {
    "slideshow": {
     "slide_type": "fragment"
    }
   },
   "outputs": [
    {
     "data": {
      "text/plain": [
       "array([2, 5, 8])"
      ]
     },
     "execution_count": 28,
     "metadata": {},
     "output_type": "execute_result"
    }
   ],
   "source": [
    "array[:, 2]"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 29,
   "metadata": {
    "slideshow": {
     "slide_type": "fragment"
    }
   },
   "outputs": [
    {
     "data": {
      "text/plain": [
       "array([3, 4, 5])"
      ]
     },
     "execution_count": 29,
     "metadata": {},
     "output_type": "execute_result"
    }
   ],
   "source": [
    "array[1, :]"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {
    "slideshow": {
     "slide_type": "slide"
    }
   },
   "source": [
    "## Rule of Thumb: Don't reinvent the wheel\n",
    "Google if a function already exists that does what you want"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {
    "slideshow": {
     "slide_type": "slide"
    }
   },
   "source": [
    "## So, how is this useful for data analysis?"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {
    "slideshow": {
     "slide_type": "fragment"
    }
   },
   "source": [
    "Often when faced with a large amount of data, a first step is to compute summary statistics for the data in question.\n",
    "Perhaps the most common summary statistics are the __mean__ and __standard deviation__, which allow you to summarize the \"typical\" values in a dataset, but other aggregates are useful as well (the sum, product, median, minimum and maximum, quantiles, etc.)."
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {
    "slideshow": {
     "slide_type": "subslide"
    }
   },
   "source": [
    "NumPy has fast built-in aggregation functions for working on arrays; we'll discuss and demonstrate some of them here."
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 31,
   "metadata": {
    "slideshow": {
     "slide_type": "fragment"
    }
   },
   "outputs": [
    {
     "name": "stdout",
     "output_type": "stream",
     "text": [
      "97.5 ms ± 8.06 ms per loop (mean ± std. dev. of 7 runs, 10 loops each)\n",
      "591 µs ± 57.7 µs per loop (mean ± std. dev. of 7 runs, 10 loops each)\n"
     ]
    }
   ],
   "source": [
    "big_array = np.random.rand(1000000)\n",
    "%timeit -n 10 sum(big_array)\n",
    "%timeit -n 10 np.sum(big_array)"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {
    "slideshow": {
     "slide_type": "subslide"
    }
   },
   "source": [
    "### Some more handy features of NumPy:"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {
    "slideshow": {
     "slide_type": "subslide"
    }
   },
   "source": [
    "One common type of aggregation operation is an aggregate along a row or column."
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {
    "slideshow": {
     "slide_type": "fragment"
    }
   },
   "source": [
    "Say you have some data stored in a two-dimensional array:"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 32,
   "metadata": {
    "slideshow": {
     "slide_type": "fragment"
    }
   },
   "outputs": [
    {
     "name": "stdout",
     "output_type": "stream",
     "text": [
      "[[ 0.60645511  0.29349287  0.5432832   0.0800523 ]\n",
      " [ 0.34077667  0.08264608  0.06142478  0.50616481]\n",
      " [ 0.7539802   0.14892548  0.0816191   0.52675839]]\n"
     ]
    }
   ],
   "source": [
    "M = np.random.random((3, 4))\n",
    "print(M)"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {
    "slideshow": {
     "slide_type": "subslide"
    }
   },
   "source": [
    "By default, each NumPy aggregation function will return the aggregate over the entire array:"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 33,
   "metadata": {
    "slideshow": {
     "slide_type": "fragment"
    }
   },
   "outputs": [
    {
     "data": {
      "text/plain": [
       "0.06142478036099186"
      ]
     },
     "execution_count": 33,
     "metadata": {},
     "output_type": "execute_result"
    }
   ],
   "source": [
    "M.min()"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {
    "slideshow": {
     "slide_type": "fragment"
    }
   },
   "source": [
    "But what if you want the min for each row or each column?"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 35,
   "metadata": {
    "slideshow": {
     "slide_type": "fragment"
    }
   },
   "outputs": [
    {
     "data": {
      "text/plain": [
       "array([ 0.0800523 ,  0.06142478,  0.0816191 ])"
      ]
     },
     "execution_count": 35,
     "metadata": {},
     "output_type": "execute_result"
    }
   ],
   "source": [
    "# Find the min of each row\n",
    "M.min(axis=1)"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 36,
   "metadata": {},
   "outputs": [
    {
     "data": {
      "text/plain": [
       "array([ 0.34077667,  0.08264608,  0.06142478,  0.0800523 ])"
      ]
     },
     "execution_count": 36,
     "metadata": {},
     "output_type": "execute_result"
    }
   ],
   "source": [
    "# Find the min of each column\n",
    "M.min(axis=0)"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {
    "slideshow": {
     "slide_type": "slide"
    }
   },
   "source": [
    "### Other aggregation functions"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {
    "slideshow": {
     "slide_type": "fragment"
    }
   },
   "source": [
    "Most aggregates have a ``NaN``-safe counterpart that computes the result while ignoring missing values, which are marked by the special floating-point ``NaN`` value."
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {
    "slideshow": {
     "slide_type": "fragment"
    }
   },
   "source": [
    "The following table provides a list of useful aggregation functions available in NumPy:\n",
    "\n",
    "|Function Name      |   NaN-safe Version  | Description                                   |\n",
    "|-------------------|---------------------|-----------------------------------------------|\n",
    "| ``np.sum``        | ``np.nansum``       | Compute sum of elements                       |\n",
    "| ``np.prod``       | ``np.nanprod``      | Compute product of elements                   |\n",
    "| ``np.mean``       | ``np.nanmean``      | Compute mean of elements                      |\n",
    "| ``np.std``        | ``np.nanstd``       | Compute standard deviation                    |\n",
    "| ``np.var``        | ``np.nanvar``       | Compute variance                              |\n",
    "| ``np.min``        | ``np.nanmin``       | Find minimum value                            |\n",
    "| ``np.max``        | ``np.nanmax``       | Find maximum value                            |\n",
    "| ``np.argmin``     | ``np.nanargmin``    | Find index of minimum value                   |\n",
    "| ``np.argmax``     | ``np.nanargmax``    | Find index of maximum value                   |\n",
    "| ``np.median``     | ``np.nanmedian``    | Compute median of elements                    |\n",
    "| ``np.percentile`` | ``np.nanpercentile``| Compute rank-based statistics of elements     |\n",
    "| ``np.any``        | N/A                 | Evaluate whether any elements are true        |\n",
    "| ``np.all``        | N/A                 | Evaluate whether all elements are true        |"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {
    "slideshow": {
     "slide_type": "slide"
    }
   },
   "source": [
    "## Pandas"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {
    "slideshow": {
     "slide_type": "subslide"
    }
   },
   "source": [
    "* Pandas is another useful library for data analysis.\n",
    "* While NumPy is really useful for math, it relies on __arrays__ of specific datatypes (ints, floats, etc).\n",
    "* Pandas uses two data structures: `Series` and `DataFrame` that are designed to package lots of different types of data similar to a spreadsheet.\n",
    "* It combines the functionality of Python and NumPy with the ease of use of Google Sheets."
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {
    "slideshow": {
     "slide_type": "slide"
    }
   },
   "source": [
    "## Example: House Rankings"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {
    "slideshow": {
     "slide_type": "fragment"
    }
   },
   "source": [
    "We will:\n",
    "1. Read in the data\n",
    "2. Manipulate the data into a more useable form\n",
    "3. Analyze the data\n",
    "4. Plot our results"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {
    "slideshow": {
     "slide_type": "slide"
    }
   },
   "source": [
    "### Reading in the data"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {
    "slideshow": {
     "slide_type": "subslide"
    }
   },
   "source": [
    "It's super easy to use Pandas to read in data from csv files:"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 37,
   "metadata": {
    "slideshow": {
     "slide_type": "fragment"
    }
   },
   "outputs": [
    {
     "data": {
      "text/html": [
       "<div>\n",
       "<style>\n",
       "    .dataframe thead tr:only-child th {\n",
       "        text-align: right;\n",
       "    }\n",
       "\n",
       "    .dataframe thead th {\n",
       "        text-align: left;\n",
       "    }\n",
       "\n",
       "    .dataframe tbody tr th {\n",
       "        vertical-align: top;\n",
       "    }\n",
       "</style>\n",
       "<table border=\"1\" class=\"dataframe\">\n",
       "  <thead>\n",
       "    <tr style=\"text-align: right;\">\n",
       "      <th></th>\n",
       "      <th>House</th>\n",
       "      <th>1</th>\n",
       "      <th>2</th>\n",
       "      <th>3</th>\n",
       "      <th>4</th>\n",
       "      <th>5</th>\n",
       "      <th>6</th>\n",
       "      <th>7</th>\n",
       "      <th>8</th>\n",
       "      <th>9</th>\n",
       "      <th>10</th>\n",
       "      <th>11</th>\n",
       "      <th>12</th>\n",
       "    </tr>\n",
       "  </thead>\n",
       "  <tbody>\n",
       "    <tr>\n",
       "      <th>0</th>\n",
       "      <td>Adams</td>\n",
       "      <td>20</td>\n",
       "      <td>15</td>\n",
       "      <td>24</td>\n",
       "      <td>38</td>\n",
       "      <td>37</td>\n",
       "      <td>44</td>\n",
       "      <td>67</td>\n",
       "      <td>75</td>\n",
       "      <td>74</td>\n",
       "      <td>28</td>\n",
       "      <td>32</td>\n",
       "      <td>80</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>1</th>\n",
       "      <td>Cabot</td>\n",
       "      <td>5</td>\n",
       "      <td>13</td>\n",
       "      <td>16</td>\n",
       "      <td>17</td>\n",
       "      <td>7</td>\n",
       "      <td>20</td>\n",
       "      <td>16</td>\n",
       "      <td>31</td>\n",
       "      <td>49</td>\n",
       "      <td>118</td>\n",
       "      <td>148</td>\n",
       "      <td>94</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>2</th>\n",
       "      <td>Kirkland</td>\n",
       "      <td>19</td>\n",
       "      <td>19</td>\n",
       "      <td>35</td>\n",
       "      <td>50</td>\n",
       "      <td>71</td>\n",
       "      <td>63</td>\n",
       "      <td>72</td>\n",
       "      <td>70</td>\n",
       "      <td>56</td>\n",
       "      <td>24</td>\n",
       "      <td>24</td>\n",
       "      <td>31</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>3</th>\n",
       "      <td>Mather</td>\n",
       "      <td>17</td>\n",
       "      <td>15</td>\n",
       "      <td>19</td>\n",
       "      <td>25</td>\n",
       "      <td>27</td>\n",
       "      <td>40</td>\n",
       "      <td>44</td>\n",
       "      <td>67</td>\n",
       "      <td>112</td>\n",
       "      <td>37</td>\n",
       "      <td>55</td>\n",
       "      <td>76</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>4</th>\n",
       "      <td>Quincy</td>\n",
       "      <td>28</td>\n",
       "      <td>43</td>\n",
       "      <td>55</td>\n",
       "      <td>90</td>\n",
       "      <td>71</td>\n",
       "      <td>82</td>\n",
       "      <td>65</td>\n",
       "      <td>44</td>\n",
       "      <td>21</td>\n",
       "      <td>17</td>\n",
       "      <td>14</td>\n",
       "      <td>4</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>5</th>\n",
       "      <td>Leverett</td>\n",
       "      <td>11</td>\n",
       "      <td>22</td>\n",
       "      <td>40</td>\n",
       "      <td>73</td>\n",
       "      <td>76</td>\n",
       "      <td>81</td>\n",
       "      <td>94</td>\n",
       "      <td>66</td>\n",
       "      <td>36</td>\n",
       "      <td>18</td>\n",
       "      <td>11</td>\n",
       "      <td>6</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>6</th>\n",
       "      <td>Dunster</td>\n",
       "      <td>45</td>\n",
       "      <td>67</td>\n",
       "      <td>113</td>\n",
       "      <td>56</td>\n",
       "      <td>70</td>\n",
       "      <td>42</td>\n",
       "      <td>44</td>\n",
       "      <td>52</td>\n",
       "      <td>19</td>\n",
       "      <td>10</td>\n",
       "      <td>11</td>\n",
       "      <td>5</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>7</th>\n",
       "      <td>Currier</td>\n",
       "      <td>14</td>\n",
       "      <td>10</td>\n",
       "      <td>16</td>\n",
       "      <td>15</td>\n",
       "      <td>18</td>\n",
       "      <td>19</td>\n",
       "      <td>20</td>\n",
       "      <td>23</td>\n",
       "      <td>43</td>\n",
       "      <td>92</td>\n",
       "      <td>114</td>\n",
       "      <td>150</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>8</th>\n",
       "      <td>Eliot</td>\n",
       "      <td>37</td>\n",
       "      <td>57</td>\n",
       "      <td>60</td>\n",
       "      <td>67</td>\n",
       "      <td>57</td>\n",
       "      <td>76</td>\n",
       "      <td>49</td>\n",
       "      <td>40</td>\n",
       "      <td>38</td>\n",
       "      <td>23</td>\n",
       "      <td>16</td>\n",
       "      <td>14</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>9</th>\n",
       "      <td>Lowell</td>\n",
       "      <td>152</td>\n",
       "      <td>106</td>\n",
       "      <td>63</td>\n",
       "      <td>51</td>\n",
       "      <td>45</td>\n",
       "      <td>35</td>\n",
       "      <td>22</td>\n",
       "      <td>24</td>\n",
       "      <td>14</td>\n",
       "      <td>5</td>\n",
       "      <td>7</td>\n",
       "      <td>10</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>10</th>\n",
       "      <td>Pforzheimer</td>\n",
       "      <td>10</td>\n",
       "      <td>21</td>\n",
       "      <td>15</td>\n",
       "      <td>6</td>\n",
       "      <td>16</td>\n",
       "      <td>19</td>\n",
       "      <td>29</td>\n",
       "      <td>33</td>\n",
       "      <td>66</td>\n",
       "      <td>158</td>\n",
       "      <td>98</td>\n",
       "      <td>63</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>11</th>\n",
       "      <td>Winthrop</td>\n",
       "      <td>176</td>\n",
       "      <td>146</td>\n",
       "      <td>78</td>\n",
       "      <td>46</td>\n",
       "      <td>39</td>\n",
       "      <td>13</td>\n",
       "      <td>12</td>\n",
       "      <td>9</td>\n",
       "      <td>6</td>\n",
       "      <td>4</td>\n",
       "      <td>4</td>\n",
       "      <td>1</td>\n",
       "    </tr>\n",
       "  </tbody>\n",
       "</table>\n",
       "</div>"
      ],
      "text/plain": [
       "          House    1    2    3   4   5   6   7   8    9   10   11   12\n",
       "0         Adams   20   15   24  38  37  44  67  75   74   28   32   80\n",
       "1         Cabot    5   13   16  17   7  20  16  31   49  118  148   94\n",
       "2      Kirkland   19   19   35  50  71  63  72  70   56   24   24   31\n",
       "3        Mather   17   15   19  25  27  40  44  67  112   37   55   76\n",
       "4        Quincy   28   43   55  90  71  82  65  44   21   17   14    4\n",
       "5      Leverett   11   22   40  73  76  81  94  66   36   18   11    6\n",
       "6       Dunster   45   67  113  56  70  42  44  52   19   10   11    5\n",
       "7       Currier   14   10   16  15  18  19  20  23   43   92  114  150\n",
       "8         Eliot   37   57   60  67  57  76  49  40   38   23   16   14\n",
       "9        Lowell  152  106   63  51  45  35  22  24   14    5    7   10\n",
       "10  Pforzheimer   10   21   15   6  16  19  29  33   66  158   98   63\n",
       "11     Winthrop  176  146   78  46  39  13  12   9    6    4    4    1"
      ]
     },
     "execution_count": 37,
     "metadata": {},
     "output_type": "execute_result"
    }
   ],
   "source": [
    "rankings = pd.read_csv(\"house_rankings_2018.csv\")\n",
    "rankings"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {
    "slideshow": {
     "slide_type": "fragment"
    }
   },
   "source": [
    "And it looks beautiful:"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 38,
   "metadata": {
    "slideshow": {
     "slide_type": "fragment"
    }
   },
   "outputs": [
    {
     "data": {
      "text/html": [
       "<div>\n",
       "<style>\n",
       "    .dataframe thead tr:only-child th {\n",
       "        text-align: right;\n",
       "    }\n",
       "\n",
       "    .dataframe thead th {\n",
       "        text-align: left;\n",
       "    }\n",
       "\n",
       "    .dataframe tbody tr th {\n",
       "        vertical-align: top;\n",
       "    }\n",
       "</style>\n",
       "<table border=\"1\" class=\"dataframe\">\n",
       "  <thead>\n",
       "    <tr style=\"text-align: right;\">\n",
       "      <th></th>\n",
       "      <th>1</th>\n",
       "      <th>2</th>\n",
       "      <th>3</th>\n",
       "      <th>4</th>\n",
       "      <th>5</th>\n",
       "      <th>6</th>\n",
       "      <th>7</th>\n",
       "      <th>8</th>\n",
       "      <th>9</th>\n",
       "      <th>10</th>\n",
       "      <th>11</th>\n",
       "      <th>12</th>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>House</th>\n",
       "      <th></th>\n",
       "      <th></th>\n",
       "      <th></th>\n",
       "      <th></th>\n",
       "      <th></th>\n",
       "      <th></th>\n",
       "      <th></th>\n",
       "      <th></th>\n",
       "      <th></th>\n",
       "      <th></th>\n",
       "      <th></th>\n",
       "      <th></th>\n",
       "    </tr>\n",
       "  </thead>\n",
       "  <tbody>\n",
       "    <tr>\n",
       "      <th>Adams</th>\n",
       "      <td>20</td>\n",
       "      <td>15</td>\n",
       "      <td>24</td>\n",
       "      <td>38</td>\n",
       "      <td>37</td>\n",
       "      <td>44</td>\n",
       "      <td>67</td>\n",
       "      <td>75</td>\n",
       "      <td>74</td>\n",
       "      <td>28</td>\n",
       "      <td>32</td>\n",
       "      <td>80</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>Cabot</th>\n",
       "      <td>5</td>\n",
       "      <td>13</td>\n",
       "      <td>16</td>\n",
       "      <td>17</td>\n",
       "      <td>7</td>\n",
       "      <td>20</td>\n",
       "      <td>16</td>\n",
       "      <td>31</td>\n",
       "      <td>49</td>\n",
       "      <td>118</td>\n",
       "      <td>148</td>\n",
       "      <td>94</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>Kirkland</th>\n",
       "      <td>19</td>\n",
       "      <td>19</td>\n",
       "      <td>35</td>\n",
       "      <td>50</td>\n",
       "      <td>71</td>\n",
       "      <td>63</td>\n",
       "      <td>72</td>\n",
       "      <td>70</td>\n",
       "      <td>56</td>\n",
       "      <td>24</td>\n",
       "      <td>24</td>\n",
       "      <td>31</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>Mather</th>\n",
       "      <td>17</td>\n",
       "      <td>15</td>\n",
       "      <td>19</td>\n",
       "      <td>25</td>\n",
       "      <td>27</td>\n",
       "      <td>40</td>\n",
       "      <td>44</td>\n",
       "      <td>67</td>\n",
       "      <td>112</td>\n",
       "      <td>37</td>\n",
       "      <td>55</td>\n",
       "      <td>76</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>Quincy</th>\n",
       "      <td>28</td>\n",
       "      <td>43</td>\n",
       "      <td>55</td>\n",
       "      <td>90</td>\n",
       "      <td>71</td>\n",
       "      <td>82</td>\n",
       "      <td>65</td>\n",
       "      <td>44</td>\n",
       "      <td>21</td>\n",
       "      <td>17</td>\n",
       "      <td>14</td>\n",
       "      <td>4</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>Leverett</th>\n",
       "      <td>11</td>\n",
       "      <td>22</td>\n",
       "      <td>40</td>\n",
       "      <td>73</td>\n",
       "      <td>76</td>\n",
       "      <td>81</td>\n",
       "      <td>94</td>\n",
       "      <td>66</td>\n",
       "      <td>36</td>\n",
       "      <td>18</td>\n",
       "      <td>11</td>\n",
       "      <td>6</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>Dunster</th>\n",
       "      <td>45</td>\n",
       "      <td>67</td>\n",
       "      <td>113</td>\n",
       "      <td>56</td>\n",
       "      <td>70</td>\n",
       "      <td>42</td>\n",
       "      <td>44</td>\n",
       "      <td>52</td>\n",
       "      <td>19</td>\n",
       "      <td>10</td>\n",
       "      <td>11</td>\n",
       "      <td>5</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>Currier</th>\n",
       "      <td>14</td>\n",
       "      <td>10</td>\n",
       "      <td>16</td>\n",
       "      <td>15</td>\n",
       "      <td>18</td>\n",
       "      <td>19</td>\n",
       "      <td>20</td>\n",
       "      <td>23</td>\n",
       "      <td>43</td>\n",
       "      <td>92</td>\n",
       "      <td>114</td>\n",
       "      <td>150</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>Eliot</th>\n",
       "      <td>37</td>\n",
       "      <td>57</td>\n",
       "      <td>60</td>\n",
       "      <td>67</td>\n",
       "      <td>57</td>\n",
       "      <td>76</td>\n",
       "      <td>49</td>\n",
       "      <td>40</td>\n",
       "      <td>38</td>\n",
       "      <td>23</td>\n",
       "      <td>16</td>\n",
       "      <td>14</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>Lowell</th>\n",
       "      <td>152</td>\n",
       "      <td>106</td>\n",
       "      <td>63</td>\n",
       "      <td>51</td>\n",
       "      <td>45</td>\n",
       "      <td>35</td>\n",
       "      <td>22</td>\n",
       "      <td>24</td>\n",
       "      <td>14</td>\n",
       "      <td>5</td>\n",
       "      <td>7</td>\n",
       "      <td>10</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>Pforzheimer</th>\n",
       "      <td>10</td>\n",
       "      <td>21</td>\n",
       "      <td>15</td>\n",
       "      <td>6</td>\n",
       "      <td>16</td>\n",
       "      <td>19</td>\n",
       "      <td>29</td>\n",
       "      <td>33</td>\n",
       "      <td>66</td>\n",
       "      <td>158</td>\n",
       "      <td>98</td>\n",
       "      <td>63</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>Winthrop</th>\n",
       "      <td>176</td>\n",
       "      <td>146</td>\n",
       "      <td>78</td>\n",
       "      <td>46</td>\n",
       "      <td>39</td>\n",
       "      <td>13</td>\n",
       "      <td>12</td>\n",
       "      <td>9</td>\n",
       "      <td>6</td>\n",
       "      <td>4</td>\n",
       "      <td>4</td>\n",
       "      <td>1</td>\n",
       "    </tr>\n",
       "  </tbody>\n",
       "</table>\n",
       "</div>"
      ],
      "text/plain": [
       "               1    2    3   4   5   6   7   8    9   10   11   12\n",
       "House                                                             \n",
       "Adams         20   15   24  38  37  44  67  75   74   28   32   80\n",
       "Cabot          5   13   16  17   7  20  16  31   49  118  148   94\n",
       "Kirkland      19   19   35  50  71  63  72  70   56   24   24   31\n",
       "Mather        17   15   19  25  27  40  44  67  112   37   55   76\n",
       "Quincy        28   43   55  90  71  82  65  44   21   17   14    4\n",
       "Leverett      11   22   40  73  76  81  94  66   36   18   11    6\n",
       "Dunster       45   67  113  56  70  42  44  52   19   10   11    5\n",
       "Currier       14   10   16  15  18  19  20  23   43   92  114  150\n",
       "Eliot         37   57   60  67  57  76  49  40   38   23   16   14\n",
       "Lowell       152  106   63  51  45  35  22  24   14    5    7   10\n",
       "Pforzheimer   10   21   15   6  16  19  29  33   66  158   98   63\n",
       "Winthrop     176  146   78  46  39  13  12   9    6    4    4    1"
      ]
     },
     "execution_count": 38,
     "metadata": {},
     "output_type": "execute_result"
    }
   ],
   "source": [
    "rankings.set_index(\"House\", inplace=True)\n",
    "rankings"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {
    "slideshow": {
     "slide_type": "slide"
    }
   },
   "source": [
    "### Manipulating the data"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {
    "slideshow": {
     "slide_type": "subslide"
    }
   },
   "source": [
    "It may be useful to also have this data in a NumPy array so we can use some of the NumPy aggregate functions to analyze our data (although Pandas also has its own version of these functions).  It's easy to convert between types:"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 39,
   "metadata": {
    "slideshow": {
     "slide_type": "fragment"
    }
   },
   "outputs": [
    {
     "data": {
      "text/plain": [
       "array([[ 20,  15,  24,  38,  37,  44,  67,  75,  74,  28,  32,  80],\n",
       "       [  5,  13,  16,  17,   7,  20,  16,  31,  49, 118, 148,  94],\n",
       "       [ 19,  19,  35,  50,  71,  63,  72,  70,  56,  24,  24,  31],\n",
       "       [ 17,  15,  19,  25,  27,  40,  44,  67, 112,  37,  55,  76],\n",
       "       [ 28,  43,  55,  90,  71,  82,  65,  44,  21,  17,  14,   4],\n",
       "       [ 11,  22,  40,  73,  76,  81,  94,  66,  36,  18,  11,   6],\n",
       "       [ 45,  67, 113,  56,  70,  42,  44,  52,  19,  10,  11,   5],\n",
       "       [ 14,  10,  16,  15,  18,  19,  20,  23,  43,  92, 114, 150],\n",
       "       [ 37,  57,  60,  67,  57,  76,  49,  40,  38,  23,  16,  14],\n",
       "       [152, 106,  63,  51,  45,  35,  22,  24,  14,   5,   7,  10],\n",
       "       [ 10,  21,  15,   6,  16,  19,  29,  33,  66, 158,  98,  63],\n",
       "       [176, 146,  78,  46,  39,  13,  12,   9,   6,   4,   4,   1]])"
      ]
     },
     "execution_count": 39,
     "metadata": {},
     "output_type": "execute_result"
    }
   ],
   "source": [
    "rankings.values"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {
    "slideshow": {
     "slide_type": "subslide"
    }
   },
   "source": [
    "We can also splice this array to just get the values for the first column or row:"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 43,
   "metadata": {
    "slideshow": {
     "slide_type": "fragment"
    }
   },
   "outputs": [
    {
     "data": {
      "text/plain": [
       "array([ 20,   5,  19,  17,  28,  11,  45,  14,  37, 152,  10, 176])"
      ]
     },
     "execution_count": 43,
     "metadata": {},
     "output_type": "execute_result"
    }
   ],
   "source": [
    "# The first column\n",
    "rankings.values[:, 0]"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 44,
   "metadata": {
    "slideshow": {
     "slide_type": "fragment"
    }
   },
   "outputs": [
    {
     "data": {
      "text/plain": [
       "array([20, 15, 24, 38, 37, 44, 67, 75, 74, 28, 32, 80])"
      ]
     },
     "execution_count": 44,
     "metadata": {},
     "output_type": "execute_result"
    }
   ],
   "source": [
    "# The first row\n",
    "rankings.values[0, :]"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {
    "slideshow": {
     "slide_type": "slide"
    }
   },
   "source": [
    "### Analyzing the data"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {
    "slideshow": {
     "slide_type": "subslide"
    }
   },
   "source": [
    "#### First, how many students filled out the survey?"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 72,
   "metadata": {
    "slideshow": {
     "slide_type": "fragment"
    }
   },
   "outputs": [],
   "source": [
    "# TODO\n",
    "n = rankings.sum(axis=1)[0]\n"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {
    "slideshow": {
     "slide_type": "subslide"
    }
   },
   "source": [
    "#### Which house was the most popular? The least popular?"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 50,
   "metadata": {
    "slideshow": {
     "slide_type": "fragment"
    }
   },
   "outputs": [
    {
     "data": {
      "text/plain": [
       "'Winthrop'"
      ]
     },
     "execution_count": 50,
     "metadata": {},
     "output_type": "execute_result"
    }
   ],
   "source": [
    "# Most popular -- TODO\n",
    "rankings.iloc[:, 0].argmax()"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 53,
   "metadata": {
    "slideshow": {
     "slide_type": "fragment"
    }
   },
   "outputs": [
    {
     "name": "stdout",
     "output_type": "stream",
     "text": [
      "Currier\n",
      "150\n"
     ]
    }
   ],
   "source": [
    "# Least popular -- TODO\n",
    "rankings.iloc[:, 11].argmax()\n",
    "print(rankings.iloc[:, 11].idxmax())\n",
    "print(rankings.iloc[:, 11].max())"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {
    "slideshow": {
     "slide_type": "subslide"
    }
   },
   "source": [
    "#### Make a `DataFrame` with the percentage of first place rankings for each house."
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 60,
   "metadata": {
    "scrolled": true,
    "slideshow": {
     "slide_type": "fragment"
    }
   },
   "outputs": [
    {
     "name": "stdout",
     "output_type": "stream",
     "text": [
      "House\n",
      "Adams           3.745318\n",
      "Cabot           0.936330\n",
      "Kirkland        3.558052\n",
      "Mather          3.183521\n",
      "Quincy          5.243446\n",
      "Leverett        2.059925\n",
      "Dunster         8.426966\n",
      "Currier         2.621723\n",
      "Eliot           6.928839\n",
      "Lowell         28.464419\n",
      "Pforzheimer     1.872659\n",
      "Winthrop       32.958801\n",
      "Name: 1, dtype: float64\n"
     ]
    }
   ],
   "source": [
    "# TODO\n",
    "newdf = w_rankings.iloc[:, 0] / n * 100\n",
    "\n",
    "print(newdf)"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {
    "slideshow": {
     "slide_type": "subslide"
    }
   },
   "source": [
    "#### Make a `DataFrame` with the average ranking for each house."
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {
    "slideshow": {
     "slide_type": "fragment"
    }
   },
   "source": [
    "Hint: You could use a `for` loop"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 55,
   "metadata": {
    "collapsed": true,
    "slideshow": {
     "slide_type": "fragment"
    }
   },
   "outputs": [],
   "source": [
    "# For loop approach -- TODO\n",
    "w_rankings = rankings.copy()"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 62,
   "metadata": {
    "slideshow": {
     "slide_type": "subslide"
    }
   },
   "outputs": [
    {
     "name": "stdout",
     "output_type": "stream",
     "text": [
      "               1    2     3     4     5     6     7     8     9     10     11  \\\n",
      "House                                                                           \n",
      "Adams         20   60   216   608   925  1584  3283  4800  5994   2800   3872   \n",
      "Cabot          5   52   144   272   175   720   784  1984  3969  11800  17908   \n",
      "Kirkland      19   76   315   800  1775  2268  3528  4480  4536   2400   2904   \n",
      "Mather        17   60   171   400   675  1440  2156  4288  9072   3700   6655   \n",
      "Quincy        28  172   495  1440  1775  2952  3185  2816  1701   1700   1694   \n",
      "Leverett      11   88   360  1168  1900  2916  4606  4224  2916   1800   1331   \n",
      "Dunster       45  268  1017   896  1750  1512  2156  3328  1539   1000   1331   \n",
      "Currier       14   40   144   240   450   684   980  1472  3483   9200  13794   \n",
      "Eliot         37  228   540  1072  1425  2736  2401  2560  3078   2300   1936   \n",
      "Lowell       152  424   567   816  1125  1260  1078  1536  1134    500    847   \n",
      "Pforzheimer   10   84   135    96   400   684  1421  2112  5346  15800  11858   \n",
      "Winthrop     176  584   702   736   975   468   588   576   486    400    484   \n",
      "\n",
      "                12  \n",
      "House               \n",
      "Adams        11520  \n",
      "Cabot        13536  \n",
      "Kirkland      4464  \n",
      "Mather       10944  \n",
      "Quincy         576  \n",
      "Leverett       864  \n",
      "Dunster        720  \n",
      "Currier      21600  \n",
      "Eliot         2016  \n",
      "Lowell        1440  \n",
      "Pforzheimer   9072  \n",
      "Winthrop       144  \n"
     ]
    }
   ],
   "source": [
    "w_rankings\n",
    "\n",
    "for i in range(12):\n",
    "    w_rankings.iloc[:, i] = w_rankings.iloc[:, i] * (i + 1)\n",
    "\n",
    "print(w_rankings)"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {
    "slideshow": {
     "slide_type": "subslide"
    }
   },
   "source": [
    "Or you could use Pandas `pd.DataFrame.apply()` to apply a function to your `DataFrame`."
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 64,
   "metadata": {
    "collapsed": true,
    "slideshow": {
     "slide_type": "fragment"
    }
   },
   "outputs": [],
   "source": [
    "def f(row):\n",
    "    for i in range(12):\n",
    "        row[i] *= (i + 1)\n",
    "    return row"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 67,
   "metadata": {
    "slideshow": {
     "slide_type": "fragment"
    }
   },
   "outputs": [],
   "source": [
    "weighted_rankings = rankings.apply(f, axis=1)"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 68,
   "metadata": {
    "slideshow": {
     "slide_type": "subslide"
    }
   },
   "outputs": [
    {
     "data": {
      "text/html": [
       "<div>\n",
       "<style>\n",
       "    .dataframe thead tr:only-child th {\n",
       "        text-align: right;\n",
       "    }\n",
       "\n",
       "    .dataframe thead th {\n",
       "        text-align: left;\n",
       "    }\n",
       "\n",
       "    .dataframe tbody tr th {\n",
       "        vertical-align: top;\n",
       "    }\n",
       "</style>\n",
       "<table border=\"1\" class=\"dataframe\">\n",
       "  <thead>\n",
       "    <tr style=\"text-align: right;\">\n",
       "      <th></th>\n",
       "      <th>1</th>\n",
       "      <th>2</th>\n",
       "      <th>3</th>\n",
       "      <th>4</th>\n",
       "      <th>5</th>\n",
       "      <th>6</th>\n",
       "      <th>7</th>\n",
       "      <th>8</th>\n",
       "      <th>9</th>\n",
       "      <th>10</th>\n",
       "      <th>11</th>\n",
       "      <th>12</th>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>House</th>\n",
       "      <th></th>\n",
       "      <th></th>\n",
       "      <th></th>\n",
       "      <th></th>\n",
       "      <th></th>\n",
       "      <th></th>\n",
       "      <th></th>\n",
       "      <th></th>\n",
       "      <th></th>\n",
       "      <th></th>\n",
       "      <th></th>\n",
       "      <th></th>\n",
       "    </tr>\n",
       "  </thead>\n",
       "  <tbody>\n",
       "    <tr>\n",
       "      <th>Adams</th>\n",
       "      <td>20</td>\n",
       "      <td>60</td>\n",
       "      <td>216</td>\n",
       "      <td>608</td>\n",
       "      <td>925</td>\n",
       "      <td>1584</td>\n",
       "      <td>3283</td>\n",
       "      <td>4800</td>\n",
       "      <td>5994</td>\n",
       "      <td>2800</td>\n",
       "      <td>3872</td>\n",
       "      <td>11520</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>Cabot</th>\n",
       "      <td>5</td>\n",
       "      <td>52</td>\n",
       "      <td>144</td>\n",
       "      <td>272</td>\n",
       "      <td>175</td>\n",
       "      <td>720</td>\n",
       "      <td>784</td>\n",
       "      <td>1984</td>\n",
       "      <td>3969</td>\n",
       "      <td>11800</td>\n",
       "      <td>17908</td>\n",
       "      <td>13536</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>Kirkland</th>\n",
       "      <td>19</td>\n",
       "      <td>76</td>\n",
       "      <td>315</td>\n",
       "      <td>800</td>\n",
       "      <td>1775</td>\n",
       "      <td>2268</td>\n",
       "      <td>3528</td>\n",
       "      <td>4480</td>\n",
       "      <td>4536</td>\n",
       "      <td>2400</td>\n",
       "      <td>2904</td>\n",
       "      <td>4464</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>Mather</th>\n",
       "      <td>17</td>\n",
       "      <td>60</td>\n",
       "      <td>171</td>\n",
       "      <td>400</td>\n",
       "      <td>675</td>\n",
       "      <td>1440</td>\n",
       "      <td>2156</td>\n",
       "      <td>4288</td>\n",
       "      <td>9072</td>\n",
       "      <td>3700</td>\n",
       "      <td>6655</td>\n",
       "      <td>10944</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>Quincy</th>\n",
       "      <td>28</td>\n",
       "      <td>172</td>\n",
       "      <td>495</td>\n",
       "      <td>1440</td>\n",
       "      <td>1775</td>\n",
       "      <td>2952</td>\n",
       "      <td>3185</td>\n",
       "      <td>2816</td>\n",
       "      <td>1701</td>\n",
       "      <td>1700</td>\n",
       "      <td>1694</td>\n",
       "      <td>576</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>Leverett</th>\n",
       "      <td>11</td>\n",
       "      <td>88</td>\n",
       "      <td>360</td>\n",
       "      <td>1168</td>\n",
       "      <td>1900</td>\n",
       "      <td>2916</td>\n",
       "      <td>4606</td>\n",
       "      <td>4224</td>\n",
       "      <td>2916</td>\n",
       "      <td>1800</td>\n",
       "      <td>1331</td>\n",
       "      <td>864</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>Dunster</th>\n",
       "      <td>45</td>\n",
       "      <td>268</td>\n",
       "      <td>1017</td>\n",
       "      <td>896</td>\n",
       "      <td>1750</td>\n",
       "      <td>1512</td>\n",
       "      <td>2156</td>\n",
       "      <td>3328</td>\n",
       "      <td>1539</td>\n",
       "      <td>1000</td>\n",
       "      <td>1331</td>\n",
       "      <td>720</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>Currier</th>\n",
       "      <td>14</td>\n",
       "      <td>40</td>\n",
       "      <td>144</td>\n",
       "      <td>240</td>\n",
       "      <td>450</td>\n",
       "      <td>684</td>\n",
       "      <td>980</td>\n",
       "      <td>1472</td>\n",
       "      <td>3483</td>\n",
       "      <td>9200</td>\n",
       "      <td>13794</td>\n",
       "      <td>21600</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>Eliot</th>\n",
       "      <td>37</td>\n",
       "      <td>228</td>\n",
       "      <td>540</td>\n",
       "      <td>1072</td>\n",
       "      <td>1425</td>\n",
       "      <td>2736</td>\n",
       "      <td>2401</td>\n",
       "      <td>2560</td>\n",
       "      <td>3078</td>\n",
       "      <td>2300</td>\n",
       "      <td>1936</td>\n",
       "      <td>2016</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>Lowell</th>\n",
       "      <td>152</td>\n",
       "      <td>424</td>\n",
       "      <td>567</td>\n",
       "      <td>816</td>\n",
       "      <td>1125</td>\n",
       "      <td>1260</td>\n",
       "      <td>1078</td>\n",
       "      <td>1536</td>\n",
       "      <td>1134</td>\n",
       "      <td>500</td>\n",
       "      <td>847</td>\n",
       "      <td>1440</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>Pforzheimer</th>\n",
       "      <td>10</td>\n",
       "      <td>84</td>\n",
       "      <td>135</td>\n",
       "      <td>96</td>\n",
       "      <td>400</td>\n",
       "      <td>684</td>\n",
       "      <td>1421</td>\n",
       "      <td>2112</td>\n",
       "      <td>5346</td>\n",
       "      <td>15800</td>\n",
       "      <td>11858</td>\n",
       "      <td>9072</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>Winthrop</th>\n",
       "      <td>176</td>\n",
       "      <td>584</td>\n",
       "      <td>702</td>\n",
       "      <td>736</td>\n",
       "      <td>975</td>\n",
       "      <td>468</td>\n",
       "      <td>588</td>\n",
       "      <td>576</td>\n",
       "      <td>486</td>\n",
       "      <td>400</td>\n",
       "      <td>484</td>\n",
       "      <td>144</td>\n",
       "    </tr>\n",
       "  </tbody>\n",
       "</table>\n",
       "</div>"
      ],
      "text/plain": [
       "               1    2     3     4     5     6     7     8     9     10     11  \\\n",
       "House                                                                           \n",
       "Adams         20   60   216   608   925  1584  3283  4800  5994   2800   3872   \n",
       "Cabot          5   52   144   272   175   720   784  1984  3969  11800  17908   \n",
       "Kirkland      19   76   315   800  1775  2268  3528  4480  4536   2400   2904   \n",
       "Mather        17   60   171   400   675  1440  2156  4288  9072   3700   6655   \n",
       "Quincy        28  172   495  1440  1775  2952  3185  2816  1701   1700   1694   \n",
       "Leverett      11   88   360  1168  1900  2916  4606  4224  2916   1800   1331   \n",
       "Dunster       45  268  1017   896  1750  1512  2156  3328  1539   1000   1331   \n",
       "Currier       14   40   144   240   450   684   980  1472  3483   9200  13794   \n",
       "Eliot         37  228   540  1072  1425  2736  2401  2560  3078   2300   1936   \n",
       "Lowell       152  424   567   816  1125  1260  1078  1536  1134    500    847   \n",
       "Pforzheimer   10   84   135    96   400   684  1421  2112  5346  15800  11858   \n",
       "Winthrop     176  584   702   736   975   468   588   576   486    400    484   \n",
       "\n",
       "                12  \n",
       "House               \n",
       "Adams        11520  \n",
       "Cabot        13536  \n",
       "Kirkland      4464  \n",
       "Mather       10944  \n",
       "Quincy         576  \n",
       "Leverett       864  \n",
       "Dunster        720  \n",
       "Currier      21600  \n",
       "Eliot         2016  \n",
       "Lowell        1440  \n",
       "Pforzheimer   9072  \n",
       "Winthrop       144  "
      ]
     },
     "execution_count": 68,
     "metadata": {},
     "output_type": "execute_result"
    }
   ],
   "source": [
    "weighted_rankings"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 69,
   "metadata": {
    "slideshow": {
     "slide_type": "subslide"
    }
   },
   "outputs": [
    {
     "name": "stdout",
     "output_type": "stream",
     "text": [
      "House\n",
      "Adams          66.820225\n",
      "Cabot          96.159176\n",
      "Kirkland       51.619850\n",
      "Mather         74.116105\n",
      "Quincy         34.707865\n",
      "Leverett       41.543071\n",
      "Dunster        29.142322\n",
      "Currier        97.567416\n",
      "Eliot          38.069288\n",
      "Lowell         20.372659\n",
      "Pforzheimer    88.048689\n",
      "Winthrop       11.833333\n",
      "dtype: float64\n"
     ]
    }
   ],
   "source": [
    "mean_rankings = weighted_rankings.sum(axis=1) / n\n",
    "print(mean_rankings)"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 70,
   "metadata": {
    "slideshow": {
     "slide_type": "subslide"
    }
   },
   "outputs": [
    {
     "data": {
      "text/plain": [
       "House\n",
       "Winthrop       11.833333\n",
       "Lowell         20.372659\n",
       "Dunster        29.142322\n",
       "Quincy         34.707865\n",
       "Eliot          38.069288\n",
       "Leverett       41.543071\n",
       "Kirkland       51.619850\n",
       "Adams          66.820225\n",
       "Mather         74.116105\n",
       "Pforzheimer    88.048689\n",
       "Cabot          96.159176\n",
       "Currier        97.567416\n",
       "dtype: float64"
      ]
     },
     "execution_count": 70,
     "metadata": {},
     "output_type": "execute_result"
    }
   ],
   "source": [
    "mean_rankings.sort_values()"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {
    "slideshow": {
     "slide_type": "slide"
    }
   },
   "source": [
    "## Congrats! You're on your way to becoming a data science expert!\n",
    "### Next week we'll tackle making visualizations of our findings using matplotlib and d3"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": null,
   "metadata": {
    "collapsed": true
   },
   "outputs": [],
   "source": []
  }
 ],
 "metadata": {
  "celltoolbar": "Raw Cell Format",
  "kernelspec": {
   "display_name": "Python 3",
   "language": "python",
   "name": "python3"
  },
  "language_info": {
   "codemirror_mode": {
    "name": "ipython",
    "version": 3
   },
   "file_extension": ".py",
   "mimetype": "text/x-python",
   "name": "python",
   "nbconvert_exporter": "python",
   "pygments_lexer": "ipython3",
   "version": "3.6.1"
  }
 },
 "nbformat": 4,
 "nbformat_minor": 2
}