Tutorial_050_Pandas_Series.ipynb

{
 "cells": [
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "# Tutorial 05, Part 0: Pandas, series, data frames \n",
    "* data structures and data analysis tools\n",
    "\n",
    "[The official project homepage](https://pandas.pydata.org)"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "## Basic data structures - start with Series then build up to DataFrames\n",
    "\n",
    "[Pandas quick start guide for Series](https://pandas.pydata.org/pandas-docs/stable/dsintro.html#series)\n",
    "\n",
    "* A **Series** is a 1D array that can hold any type of data (numeric types, non-numeric, Python objects and so forth).\n",
    "    * Unlike a 1D numpy array, each entry is **labeled** with an index that is used to keep track of what each entry is, and can be used to lookup the value corresponding to each index during analysis.\n",
    "    * These labels are fixed - they will always index the same value unless you explicitly break that link.\n",
    "    * The list of labels that forms the index can either be declared upon series creation or, by default, it will range from 0 to len(data)-1.\n",
    "        * If you're going to use Pandas to organize your data, specifying usable and informative labels is a good idea because that's one of the main advantages of organizing your data in this manner - if you just want to fly blind then NumPy is usually fine on its own\n",
    "        \n",
    "<div class=\"alert alert-warning\">\n",
    "Pandas will allow you to specify non-unique labels. This can be ok for operations that don't rely on indexing by label. However, operations that do rely on unique labels for indexing may throw an unexpected error so in general its good practice to use unique labels!\n",
    "</div>"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "## Import libs"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 1,
   "metadata": {},
   "outputs": [],
   "source": [
    "# standard numpy module\n",
    "import numpy as np\n",
    "\n",
    "# import a generic pandas object and also a few specific functions that we'll use\n",
    "import pandas as pd \n",
    "\n",
    "# new - get and store current file path for file i/o later on in tutorial\n",
    "import os\n",
    "cwd = os.getcwd()"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "## Create a series from an numpy ndarray"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 2,
   "metadata": {},
   "outputs": [
    {
     "name": "stdout",
     "output_type": "stream",
     "text": [
      "Index labels:  ['Sub0', 'Sub1', 'Sub2', 'Sub3', 'Sub4', 'Sub5', 'Sub6', 'Sub7', 'Sub8', 'Sub9', 'Sub10', 'Sub11'] \n",
      "\n",
      "Sub0     0.791550\n",
      "Sub1     1.130879\n",
      "Sub2     2.110631\n",
      "Sub3     1.632258\n",
      "Sub4     0.865049\n",
      "Sub5     1.681111\n",
      "Sub6     1.418027\n",
      "Sub7     1.164641\n",
      "Sub8     0.251004\n",
      "Sub9     0.672859\n",
      "Sub10    1.345909\n",
      "Sub11    0.300852\n",
      "dtype: float64\n"
     ]
    }
   ],
   "source": [
    "# make some data and then use pd.Series\n",
    "\n",
    "# random seed so we get the same thing each time \n",
    "np.random.RandomState(0)\n",
    "\n",
    "# For this simulation, lets have 20 subjects, and some data\n",
    "# generated from a Rayleigh distribution \n",
    "# (no particular motivation for selecting this distribution, just for something different)\n",
    "# Rayleigh is the distribution of vector magnitudes generated by two independent components (e.g. wind speed)\n",
    "N = 12\n",
    "data = np.random.rayleigh(scale=1, size=N)\n",
    "\n",
    "# make a list of subject names for use as index labels\n",
    "label_prefix = 'Sub'\n",
    "index=[]\n",
    "for n in np.arange(N):\n",
    "    index.append(label_prefix+str(n))\n",
    "\n",
    "# print our list of index labels\n",
    "print('Index labels: ', index, '\\n')\n",
    "\n",
    "# then make our pandas Series by passing in our data array and our index labels\n",
    "s = pd.Series(data, index=index)\n",
    "print(s)"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "## Note that each subject is now a field in the series and can be used to retrieve the corresponding value...there are a few ways to do this"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 6,
   "metadata": {},
   "outputs": [
    {
     "name": "stdout",
     "output_type": "stream",
     "text": [
      "0.3008515129731635\n",
      "0.3008515129731635\n"
     ]
    }
   ],
   "source": [
    "# access by field\n",
    "print(s.Sub11)\n",
    "\n",
    "# access by index label\n",
    "print(s['Sub11'])\n",
    "\n",
    "# will cover more advanced slicing below"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "## Can also use labels to check for membership or to index over labels"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 10,
   "metadata": {},
   "outputs": [
    {
     "name": "stdout",
     "output_type": "stream",
     "text": [
      "True\n",
      "False\n",
      "\n",
      "\n",
      "Sub0\n",
      "Sub1\n",
      "Sub2\n",
      "Sub3\n",
      "Sub4\n",
      "Sub5\n",
      "Sub6\n",
      "Sub7\n",
      "Sub8\n",
      "Sub9\n",
      "Sub10\n",
      "Sub11\n",
      "\n",
      "\n",
      "0.7915499064925858\n",
      "1.1308793254337082\n",
      "2.1106313898167905\n",
      "1.6322582476765632\n",
      "0.8650487774278873\n",
      "1.6811111822086306\n",
      "1.4180268527665374\n",
      "1.164641009382217\n",
      "0.2510037666754611\n",
      "0.6728594944862711\n",
      "1.3459093213349125\n",
      "0.3008515129731635\n"
     ]
    }
   ],
   "source": [
    "# check for membership\n",
    "print('Sub11' in s)\n",
    "print('Sub13' in s)\n",
    "print('\\n')\n",
    "\n",
    "# iterate over index labels, with l==index name\n",
    "for l in s.index:\n",
    "    print(l)\n",
    "\n",
    "print('\\n')\n",
    "\n",
    "# iterate over values...\n",
    "for l in s.values:\n",
    "    print(l)    "
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 11,
   "metadata": {},
   "outputs": [
    {
     "name": "stdout",
     "output_type": "stream",
     "text": [
      "0.7915499064925858\n",
      "1.1308793254337082\n",
      "2.1106313898167905\n",
      "1.6322582476765632\n",
      "0.8650487774278873\n",
      "1.6811111822086306\n",
      "1.4180268527665374\n",
      "1.164641009382217\n",
      "0.2510037666754611\n",
      "0.6728594944862711\n",
      "1.3459093213349125\n",
      "0.3008515129731635\n"
     ]
    }
   ],
   "source": [
    "# can also get to the values more directly like this:\n",
    "for d in s:\n",
    "    print(d)"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "## Before moving on, there are a few other optional (but important) parameters of the pd.Series call\n",
    "* dtype - default is to infer the data type (int32, float64, str, etc) based on the values in data\n",
    "    * However, can also explicitly declare the data\n",
    "    * This can be good if you want to, for example, re-cast the data to save space or to make types compatible\n",
    "    * But this may also have important negative consequences if not done thoughtfully! \n",
    "* copy - if not specified then the default behavior is set to False and the new series will have a 'view' of the data.\n",
    "    * This can save space, but can sometimes lead to confusion as any change to the values in s will also change the values in the original 'data' array\n",
    "    * Setting copy=False will make a new copy of the data in 's' that is independent of the input 'data' array\n"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "### Explicitly declare a different dtype to see where things can go wrong"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 12,
   "metadata": {},
   "outputs": [
    {
     "name": "stdout",
     "output_type": "stream",
     "text": [
      "[0.79154991 1.13087933 2.11063139 1.63225825]\n",
      "\n",
      " Sub0    0\n",
      "Sub1    1\n",
      "Sub2    2\n",
      "Sub3    1\n",
      "dtype: int32\n"
     ]
    }
   ],
   "source": [
    "# make a series with the data array from above, but make it int32 instead of the inferred (and correct) float64 type\n",
    "s = pd.Series(data, index=index, dtype='int32')\n",
    "\n",
    "# first 4 values in our original data array\n",
    "print(data[:4])\n",
    "\n",
    "# first 4 values in our series of type int32...might not be what you want!\n",
    "print('\\n', s[:4])"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "### Another example: declaring dtype can be handy if you want to, for example, do str manipulations with the data array later or if you want to merge with another series of type str"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 13,
   "metadata": {},
   "outputs": [
    {
     "name": "stdout",
     "output_type": "stream",
     "text": [
      "[0.79154991 1.13087933 2.11063139 1.63225825]\n",
      "\n",
      " Sub0    0.79155\n",
      "Sub1    1.13088\n",
      "Sub2    2.11063\n",
      "Sub3    1.63226\n",
      "dtype: object\n"
     ]
    }
   ],
   "source": [
    "# make a series with the data array from above, but this time make it a str\n",
    "# instead of the inferred float64 type\n",
    "s = pd.Series(data, index=index, dtype='str')\n",
    "\n",
    "# first 4 values in our original data array\n",
    "print(data[:4])\n",
    "\n",
    "# first 4 values in our series of type str...preserves info and we're now\n",
    "# all set to do a bunch of str operation without having to deal with \n",
    "# recasting each time we interact with the values in s\n",
    "print('\\n', s[:4])"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "<div class=\"alert alert-info\">\n",
    "Note that the dtype of series 's' is now an 'object'. This is the Pandas version of a Python 'str'\n",
    "</div>"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "### Now explicitly ask for a 'copy' of the data instead of the default view"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 14,
   "metadata": {},
   "outputs": [
    {
     "name": "stdout",
     "output_type": "stream",
     "text": [
      "Original data:  [0 1 2 3] \n",
      "\n",
      "Original values in series\n",
      "d1    0\n",
      "d2    1\n",
      "d3    2\n",
      "d4    3\n",
      "dtype: int32\n",
      "\n",
      "New values in series\n",
      "d1    100\n",
      "d2      1\n",
      "d3      2\n",
      "d4      3\n",
      "dtype: int32\n",
      "\n",
      "New data: [100   1   2   3] \n",
      "data[0] changed too!\n"
     ]
    }
   ],
   "source": [
    "# Same as before - create a series based on a short data array (0:4 in this case for simplicity)\n",
    "# let Pandas figure out the dtype, and use the default copy behavior (i.e. copy=False)\n",
    "\n",
    "N = 4                # number of data points\n",
    "\n",
    "# make data\n",
    "data = np.arange(N)\n",
    "\n",
    "# make index labels\n",
    "index = ['d1','d2','d3','d4']\n",
    "\n",
    "# print out the original data array for reference\n",
    "print('Original data: ', data, '\\n')\n",
    "\n",
    "# make a series with the default behavior of copy=False\n",
    "s = pd.Series(data, index=index, copy=False) #default: create a view of data\n",
    "\n",
    "# print out the new series\n",
    "print('Original values in series')\n",
    "print(s)\n",
    "\n",
    "# now change the value of the first entry in the series\n",
    "s['d1'] = 100\n",
    "\n",
    "# new values in series 's'\n",
    "print('\\nNew values in series')\n",
    "print(s)\n",
    "\n",
    "# and then print the corresponding entry in the data array\n",
    "print('\\nNew data:', data, '\\ndata[0] changed too!')\n",
    "\n",
    "# Note that data[0] changed because the values in s are a view of data...\n",
    "# both are referencing the same chunk of memory"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "<div class=\"alert alert-danger\">\n",
    "Note that this works in the other direction too, which can be more insidious...if you create a Series based on the values in 'data', and then do more work with 'data', then every time you change a value in the original data array, you will also change the corresponding value in s!!!\n",
    "</div>"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 15,
   "metadata": {},
   "outputs": [
    {
     "name": "stdout",
     "output_type": "stream",
     "text": [
      "Original data:  [0 1 2 3] \n",
      "\n",
      "Original values in series\n",
      "d1    0\n",
      "d2    1\n",
      "d3    2\n",
      "d4    3\n",
      "dtype: int32\n",
      "\n",
      "New values in series\n",
      "d1    100\n",
      "d2      1\n",
      "d3      2\n",
      "d4      3\n",
      "dtype: int32\n",
      "\n",
      "New data is the same as the old data: [0 1 2 3]\n",
      "data[0] did not change because it is independent from values in s\n"
     ]
    }
   ],
   "source": [
    "# now do the same thing but this time lets explicitly ask for a copy of the data\n",
    "N = 4                # number of data points\n",
    "\n",
    "# make data\n",
    "data = np.arange(N)\n",
    "\n",
    "# make index labels\n",
    "index = ['d1','d2','d3','d4']\n",
    "\n",
    "# print out the original data array for reference\n",
    "print('Original data: ', data, '\\n')\n",
    "\n",
    "# make a series, but change the default behavior of copy to copy=True\n",
    "s = pd.Series(data, index=index, copy=True)\n",
    "\n",
    "# print out the new series\n",
    "print('Original values in series')\n",
    "print(s)\n",
    "\n",
    "# now change the value of the first entry in the series\n",
    "s['d1'] = 100\n",
    "\n",
    "# new values in series 's'\n",
    "print('\\nNew values in series')\n",
    "print(s)\n",
    "\n",
    "# and then print the corresponding entry in the data array\n",
    "print('\\nNew data is the same as the old data:', data)\n",
    "print('data[0] did not change because it is independent from values in s')"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "## After creating a pandas series, you can do many common operations and access the functionality of other modules \n",
    "* A pd Series behaves similar to a NumPy ndarray, and can be passed to many NumPy functions\n",
    "* Slicing also works like a ndarray - note that index is also sliced\n",
    "* Lots of built in methods as well that emulate NumPy functionality"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "### Can pass pd.Series to most NumPy functions... "
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 16,
   "metadata": {},
   "outputs": [
    {
     "name": "stdout",
     "output_type": "stream",
     "text": [
      "\n",
      "Mean:  0.8916435041937402 Max:  2.4795362373569714\n"
     ]
    }
   ],
   "source": [
    "# make a new series...\n",
    "N = 8\n",
    "data = np.random.exponential(size=N)\n",
    "\n",
    "# make some labels\n",
    "label_prefix = 'Exp'\n",
    "index=[]\n",
    "for n in np.arange(N):\n",
    "    index.append(label_prefix+str(n))\n",
    "    \n",
    "# make the series\n",
    "s = pd.Series(data, index=index)\n",
    "\n",
    "# can pass s to common np operations...\n",
    "print('\\nMean: ', np.mean(s), 'Max: ', np.max(s)) #will disregard the labels, only do for values"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "### Note that the index labels come along for the ride "
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 17,
   "metadata": {},
   "outputs": [
    {
     "name": "stdout",
     "output_type": "stream",
     "text": [
      "Exp0    0.762940\n",
      "Exp1    1.457009\n",
      "Exp2    0.413972\n",
      "Exp3    0.353556\n",
      "Exp4    0.955947\n",
      "Exp5    0.638113\n",
      "Exp6    2.479536\n",
      "Exp7    0.072075\n",
      "dtype: float64\n",
      "\n",
      "Cumproduct\n",
      "\n",
      "Exp0    0.762940\n",
      "Exp1    1.111609\n",
      "Exp2    0.460175\n",
      "Exp3    0.162698\n",
      "Exp4    0.155531\n",
      "Exp5    0.099246\n",
      "Exp6    0.246084\n",
      "Exp7    0.017737\n",
      "dtype: float64\n",
      "\n",
      "Index by label\n",
      "0.24608430355483737\n",
      "0.24608430355483737\n"
     ]
    }
   ],
   "source": [
    "# print our series - set of index labels along with data values\n",
    "print(s)\n",
    "\n",
    "# then apply the NumPy cumulative product operation (multiply N with N+1, then that result by N+2, etc)\n",
    "cp = np.cumprod(s)\n",
    "\n",
    "print('\\nCumproduct\\n')\n",
    "print(cp)\n",
    "\n",
    "# cool part: note that the output also contains the label info, which is handy to keep track of things,\n",
    "# e.g. you can index into cp using thes labels\n",
    "print('\\nIndex by label')\n",
    "print(cp['Exp6'])\n",
    "print(cp.Exp6)"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "### Series objects have many built in operations, much like NumPy \n",
    "[list of attributes and methods](https://pandas.pydata.org/pandas-docs/version/0.23.4/generated/pandas.Series.html)"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 18,
   "metadata": {},
   "outputs": [
    {
     "name": "stdout",
     "output_type": "stream",
     "text": [
      "Data Type:  float64\n",
      "Mean:  0.8916435041937402  Std: 0.7667574319182172 Max:  2.4795362373569714\n",
      "Diff:  Exp0         NaN\n",
      "Exp1    0.694069\n",
      "Exp2   -1.043036\n",
      "Exp3   -0.060416\n",
      "Exp4    0.602391\n",
      "Exp5   -0.317834\n",
      "Exp6    1.841423\n",
      "Exp7   -2.407461\n",
      "dtype: float64\n"
     ]
    }
   ],
   "source": [
    "# attributes\n",
    "print('Data Type: ', s.dtype)\n",
    "\n",
    "# basic methods\n",
    "print('Mean: ', s.mean(), ' Std:', s.std(), 'Max: ', s.max())\n",
    "\n",
    "# numerical derivative\n",
    "print('Diff: ', s.diff())"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "### Slicing also works like NumPy"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 19,
   "metadata": {},
   "outputs": [
    {
     "name": "stdout",
     "output_type": "stream",
     "text": [
      "Exp0    0.762940\n",
      "Exp1    1.457009\n",
      "Exp2    0.413972\n",
      "Exp3    0.353556\n",
      "Exp4    0.955947\n",
      "Exp5    0.638113\n",
      "Exp6    2.479536\n",
      "Exp7    0.072075\n",
      "dtype: float64\n",
      "\n",
      "\n",
      "First 3 entries\n",
      "Exp0    0.762940\n",
      "Exp1    1.457009\n",
      "Exp2    0.413972\n",
      "dtype: float64\n",
      "\n",
      "\n",
      "S * 22\n",
      "Exp0    16.784669\n",
      "Exp1    32.054188\n",
      "Exp2     9.107385\n",
      "Exp3     7.778240\n",
      "Exp4    21.030842\n",
      "Exp5    14.038487\n",
      "Exp6    54.549797\n",
      "Exp7     1.585650\n",
      "dtype: float64\n"
     ]
    }
   ],
   "source": [
    "# print the series\n",
    "print(s)\n",
    "print('\\n')\n",
    "\n",
    "# first 3 values\n",
    "print('First 3 entries')\n",
    "print(s[:3])\n",
    "print('\\n')\n",
    "\n",
    "# Unary operations\n",
    "print('S * 22')\n",
    "print(s * 22)"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "### Note that after slicing, labels stay attached to data..."
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 20,
   "metadata": {},
   "outputs": [
    {
     "name": "stdout",
     "output_type": "stream",
     "text": [
      "Values > .9\n",
      "Exp1    1.457009\n",
      "Exp4    0.955947\n",
      "Exp6    2.479536\n",
      "dtype: float64\n",
      "\n",
      "\n",
      "Index(['Exp1', 'Exp4', 'Exp6'], dtype='object')\n"
     ]
    }
   ],
   "source": [
    "# Example using conditional indexing: find all entries where data > .9\n",
    "print('Values > .9')\n",
    "new_s = s[s>.9]\n",
    "print(new_s)\n",
    "print('\\n')\n",
    "\n",
    "# have access to the index labels that met your condition\n",
    "print(new_s.index)"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 21,
   "metadata": {},
   "outputs": [
    {
     "name": "stdout",
     "output_type": "stream",
     "text": [
      "Exp3    0.353556\n",
      "Exp4    0.955947\n",
      "Exp5    0.638113\n",
      "Exp6    2.479536\n",
      "dtype: float64\n"
     ]
    }
   ],
   "source": [
    "# another example using more advanced slicing...\n",
    "another_slice = s[3:-1]    #4th entry to len(s)-1\n",
    "print(another_slice)"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "<div class=\"alert alert-info\">\n",
    "The fact that labels stay attached to the corresponding values is often useful beacuse you don't have to compute and store a separate index for the new data set like you would in Matlab if you wanted to keep track of where the values > .9 were in the original array.\n",
    "</div>"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "## Although series can be treated much like NumPy arrays, there is one key difference (and often a big advantage)\n",
    "* When you do an operation on a NumPy array, the operation is performed in an element-by-element manner\n",
    "* However, when you do an operation on two pandas series, the operation will be applied to like-labeled values\n",
    "* This can save a lot of trouble in terms of lining up corresponding entries in two data arrays when the data sets are initialized in different orders!"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 22,
   "metadata": {},
   "outputs": [
    {
     "name": "stdout",
     "output_type": "stream",
     "text": [
      "[0 1 2 3 4]\n",
      "[10 11 12 13 14]\n",
      "[10 12 14 16 18]\n"
     ]
    }
   ],
   "source": [
    "# first a quick demo in NumPy just to get re-familiarized with how it works\n",
    "# make two arrays, and add them\n",
    "N=5\n",
    "x = np.arange(N)\n",
    "y = np.linspace(0,N-1,N, dtype='int32')+10\n",
    "print(x)\n",
    "print(y)\n",
    "print(x+y)"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "<div class=\"alert alert-info\">\n",
    "Info alert - the next part is neat and really really useful in many real world applications where data sets are messy...Series operations are performed based on matching labels, not on matching positions in an array!\n",
    "</div>"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "### Following on the NumPy example in the last cell...Now suppose that you ran a set of subjects in two experiments, but the data from each subject were entered in a different order in each study\n",
    "* Even though the data were entered in different orders, you want an easy way to perform operations on specific subjects across experiments \n",
    "* Using NumPy - or Matlab - you'd probably now try to sort your second data set so that the labels from the second study were in the same order as in the first study.\n",
    "* Then you would save an index indicating the sort order, and you'd use that index to rearange the data values from the second data set so that everything lined up with the first data set.\n",
    "* A series can make life much easier here because operations are done on a union of the labels involved!"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 23,
   "metadata": {},
   "outputs": [
    {
     "name": "stdout",
     "output_type": "stream",
     "text": [
      "s0    0\n",
      "s1    1\n",
      "s2    2\n",
      "s3    3\n",
      "s4    4\n",
      "dtype: int32\n",
      "\n",
      "Data from the second experiment - same subjects, but different order\n",
      "\n",
      "s3     7\n",
      "s2     8\n",
      "s4     9\n",
      "s1    10\n",
      "s0    11\n",
      "dtype: int32\n"
     ]
    }
   ],
   "source": [
    "# set up two series - as if we have two data sets from the same set of 5 participants\n",
    "N=5\n",
    "data0 = np.arange(N)\n",
    "index0 = ['s0','s1','s2','s3','s4']\n",
    "s0 = pd.Series(data0, index=index0)\n",
    "\n",
    "# now do our second 'experiment' but this time the subjects were run in a different order\n",
    "data1 = np.arange(N)+7\n",
    "index1 = ['s3','s2','s4','s1','s0']\n",
    "s1 = pd.Series(data1, index=index1)\n",
    "\n",
    "# print out our data series\n",
    "print(s0)\n",
    "print('\\nData from the second experiment - same subjects, but different order\\n')\n",
    "print(s1)"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 24,
   "metadata": {},
   "outputs": [
    {
     "name": "stdout",
     "output_type": "stream",
     "text": [
      "s0    11\n",
      "s1    11\n",
      "s2    10\n",
      "s3    10\n",
      "s4    13\n",
      "dtype: int32\n"
     ]
    }
   ],
   "source": [
    "# Do a simple unary operation like addition across data sets\n",
    "sum_data = s0+s1\n",
    "print(sum_data)\n",
    "# Even though the numerical position of each subject differs across experiments, Pandas figured out how \n",
    "# to properly perform the operation by aligning based on index labels!"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "## Last notes on creation of series...\n",
    "* Thus far we've been initializing series with ndarrays\n",
    "* Can also make series from scalars (assign all indices same value) or from dicts"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "### Suppose you want a series with all the same values...you can do this using np.repeat"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 25,
   "metadata": {},
   "outputs": [
    {
     "data": {
      "text/plain": [
       "0    14\n",
       "1    14\n",
       "2    14\n",
       "3    14\n",
       "dtype: int32"
      ]
     },
     "execution_count": 25,
     "metadata": {},
     "output_type": "execute_result"
    }
   ],
   "source": [
    "N=4\n",
    "data = np.repeat(14, N)\n",
    "index = np.arange(N) \n",
    "\n",
    "# make the series\n",
    "s = pd.Series(data, index=index)\n",
    "\n",
    "# all entries will have the same value\n",
    "s"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "### OR you can achieve the same thing in a more straightforward manner"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 26,
   "metadata": {},
   "outputs": [
    {
     "data": {
      "text/plain": [
       "0    14\n",
       "1    14\n",
       "2    14\n",
       "3    14\n",
       "dtype: int64"
      ]
     },
     "execution_count": 26,
     "metadata": {},
     "output_type": "execute_result"
    }
   ],
   "source": [
    "# series from scalars\n",
    "N=4\n",
    "\n",
    "# don't need repeat cause its a single scalar linked to each index\n",
    "data = 14\n",
    "index = np.arange(N) \n",
    "\n",
    "# make the series\n",
    "s = pd.Series(data, index=index)\n",
    "\n",
    "# all entries will have the same value\n",
    "s"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "### Can also initialize with a dict\n",
    "* dict keys become index labels\n",
    "* data become values"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 27,
   "metadata": {},
   "outputs": [
    {
     "name": "stdout",
     "output_type": "stream",
     "text": [
      "Bob     20.0\n",
      "Ella    17.0\n",
      "Jack    25.3\n",
      "Sam     23.0\n",
      "dtype: float64\n"
     ]
    }
   ],
   "source": [
    "data = {'Bob' : 20, 'Ella' : 17, 'Sam' : 23, 'Jack' : 25.3}\n",
    "s = pd.Series(data)\n",
    "print(s)"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "<div class=\"alert alert-info\">\n",
    "Note that data type is upcast to highest precision entry when you create a Series with mixed numerical data types\n",
    "</div>"
   ]
  }
 ],
 "metadata": {
  "kernelspec": {
   "display_name": "Python 3",
   "language": "python",
   "name": "python3"
  },
  "language_info": {
   "codemirror_mode": {
    "name": "ipython",
    "version": 3
   },
   "file_extension": ".py",
   "mimetype": "text/x-python",
   "name": "python",
   "nbconvert_exporter": "python",
   "pygments_lexer": "ipython3",
   "version": "3.6.4"
  }
 },
 "nbformat": 4,
 "nbformat_minor": 2
}