intro-to-python/index.ipynb

{
 "cells": [
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "# Introduction to Python\n",
    "\n",
    "This tutorial was originally drawn from [Scipy Lecture Notes](http://www.scipy-lectures.org/) by [this list of contributors](http://www.scipy-lectures.org/preface.html#authors). I've continued to modify it as I use it.\n",
    "\n",
    "This work is CC-BY. Author: Aaron L. Brenner\n",
    "\n",
    "\n",
    "Python is a **programming language**, as are C, Fortran, BASIC, PHP,\n",
    "  etc. Some specific features of Python are as follows:\n",
    "\n",
    "* an *interpreted* (as opposed to *compiled*) language. Contrary to e.g. C or Fortran, one does not compile Python code before executing it. In addition, Python can be used **interactively**: many Python interpreters are available, from which commands and scripts can be executed.\n",
    "* a free software released under an **open-source** license: Python can be used and distributed free of charge, even for building commercial software.\n",
    "* **multi-platform**: Python is available for all major operating systems, Windows, Linux/Unix, MacOS X, most likely your mobile phone OS, etc.\n",
    "* a very readable language with clear non-verbose syntax* a language for which a large variety of high-quality packages are\n",
    "    available for various applications, from web frameworks to scientific\n",
    "    computing.\n",
    "* a language very easy to interface with other languages, in particular C and C++.\n",
    "* Some other features of the language are illustrated just below. For example, Python is an object-oriented language, with dynamic typing (the same variable can contain objects of different types during the course of a program).\n",
    "\n",
    "See https://www.python.org/about/ for more information about distinguishing features of Python."
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "## Some Key Learning and Reference Resources\n",
    "\n",
    "If you are interested in moving forward with learning Python, it is worth your time to get acquainted with all of these resources. The tutorial will step you through more Python, and you should be familiar with the basics of the Python language and its standard library.\n",
    "\n",
    "Python documentation home\n",
    "https://docs.python.org/3/\n",
    "\n",
    "Tutorial\n",
    "https://docs.python.org/3/tutorial/index.html\n",
    "\n",
    "Python Language Reference\n",
    "https://docs.python.org/3/reference/index.html#reference-index\n",
    "\n",
    "The Python Standard Library\n",
    "https://docs.python.org/3/library/index.html#library-index"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "## Additional Learning Resources\n",
    "The Python Cookbook, 3rd Edition - This is one of many various 'cookbooks'. These can be very useful not only for seeing solutions to common problems, but also as a way to read brief examples of ideomatic code. Reading code snippets in this way can be a great compliment to language reference documentation and traditional tutorials.\n",
    "http://chimera.labs.oreilly.com/books/1230000000393/\n",
    "\n",
    "Also, don't be embarrased to Google your questions! Try some variation of `python [thing] example`"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "## Let's dive in* with an example that does something (kind of) useful:\n",
    "\n",
    "\\* hat tip to [Mark Pilgrim](http://www.diveintopython.net/)"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "This is a script that inspects a CSV data file and reports on some summary characteristics. Take a minute to read over the code before running it. Don't worry if you don't understand all of what's happening. We'll step through some of this code in more detail as we learn the basics of python. For now, just try to get a feel for what a complete script looks like. After you've read it over, go ahead and execute it."
   ]
  },
  {
   "cell_type": "code",
   "execution_count": null,
   "metadata": {
    "collapsed": false
   },
   "outputs": [],
   "source": [
    "# open the source CSV file\n",
    "csv = open(\"cars.csv\")\n",
    "\n",
    "# create a list with the column names. we assume the first row contiains them.\n",
    "# we strip the carriage return (if there is one) from the line, then split values on the commas.\n",
    "# Note: this uses a nifty python feature called 'list comprehension' to do it in one line\n",
    "column_names = [i for i in csv.readline().strip().split(',')]\n",
    "\n",
    "# read the rest of the file into a matrix (a list of lists). Use the same strip and split methods.\n",
    "data = [line.strip().split(',') for line in csv.readlines()]\n",
    "\n",
    "# now, try to infer the data types of each column from the values in the first row.\n",
    "# the testing here shows some string methods, like isspace(), isalpha(), isdigit().\n",
    "# we'll save these data type assumptions because we'll use them later in a report.\n",
    "column_datatypes = []\n",
    "for value in data[0]:\n",
    "    if len(value) < 1 or value.isspace():\n",
    "        column_datatypes.append('string')\n",
    "    elif value.isalpha():\n",
    "        column_datatypes.append('string')\n",
    "    elif '.' in value or value.isdigit():\n",
    "        column_datatypes.append('numeric')\n",
    "    else:\n",
    "        column_datatypes.append('string')\n",
    "        \n",
    "# now let's do some basic reporting on the csv\n",
    "# overall stats of the file:\n",
    "print(\"this csv file has \" + str(len(column_names)) + \" columns and \" + str(len(data)) + \" rows.\")\n",
    "\n",
    "# loop over each column name, do some different things depending on whether we've inferred \n",
    "# it contains string or numeric values. we declare certain variables with 'False' so even if\n",
    "# we can't fill them we can test them without an error.\n",
    "for i, value in enumerate(column_names):\n",
    "    average_value = False\n",
    "    highest_value = False\n",
    "    lowest_value = False\n",
    "    \n",
    "    # if it's a numeric column, we'll get all the values for this column out of our data matrix,\n",
    "    # convert them to float (remember they are all strings by default), and then get the average,\n",
    "    # high, and low values. If there's an error doing this, just get the values as strings\n",
    "    if column_datatypes[i] == 'numeric':\n",
    "        try:\n",
    "            column_values = [float(data[j][i]) for j in range(len(data))]\n",
    "            average_value = sum(column_values)/len(column_values)\n",
    "            highest_value = sorted(column_values)[-1]\n",
    "            lowest_value = sorted(column_values)[0]\n",
    "        except ValueError:\n",
    "            column_values = [data[j][i] for j in range(len(data))]\n",
    "    else:\n",
    "        column_values = [data[j][i] for j in range(len(data))]\n",
    "    \n",
    "    # the set function removes duplicates from a list, so taking its length is equivilent\n",
    "    # to the number of unique values\n",
    "    unique_value_count = len(set(column_values))\n",
    "    \n",
    "    # now we start printing. First just the field name. The simple way of formatting a string\n",
    "    # is with the + operator. Note: we add one to the index because we don't want our list\n",
    "    # to start with zero.\n",
    "    print(str(i+1) + \". \\\"\" + value + \"\\\"\")\n",
    "    \n",
    "    # next the type we think it is, and the number of unique values\n",
    "    # Note: using the + style of string formatting all non-string values have to be cast to strings\n",
    "    print(\"\\t{0} ({1} of {2} unique)\".format(column_datatypes[i], unique_value_count, len(data)))\n",
    "    \n",
    "    # now different details if it's numeric and successfully converted to float, if it's\n",
    "    # numeric, and didnt', and otherwise we assume it's a string.\n",
    "    # Note: also showing a different, more powerful string formatting method here\n",
    "    if column_datatypes[i] == 'numeric':\n",
    "        if average_value:\n",
    "            print(\"\\taverage value: {0:g}\".format(average_value))\n",
    "            print(\"\\tlowest value: {0:g}\".format(lowest_value))\n",
    "            print(\"\\thighest value: {0:g}\".format(highest_value))\n",
    "        else:\n",
    "            print(\"\\tNOTE: problems converting values to float!\")\n",
    "    else: \n",
    "            print(\"\\tfirst value: {0:s}\".format(column_values[0]))\n",
    "            print(\"\\tlast value: {0:s}\".format(column_values[-1]))"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "After you've run this as-is, change the name of the CSV file from `cars.csv` to `cities.csv`. Run it again."
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "## First Steps"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "Follow along with the instructor in typing instructions:"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": null,
   "metadata": {
    "collapsed": false
   },
   "outputs": [],
   "source": [
    "   "
   ]
  },
  {
   "cell_type": "code",
   "execution_count": null,
   "metadata": {
    "collapsed": false
   },
   "outputs": [],
   "source": []
  },
  {
   "cell_type": "code",
   "execution_count": null,
   "metadata": {
    "collapsed": false
   },
   "outputs": [],
   "source": []
  },
  {
   "cell_type": "code",
   "execution_count": null,
   "metadata": {
    "collapsed": false
   },
   "outputs": [],
   "source": []
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "Two variables `a` and `b` have been defined above. Note that one does not declare the type of an variable before assigning its value.\n",
    "\n",
    "In addition, the type of a variable may change, in the sense that at one point in time it can be equal to a value of a certain type, and a second point in time, it can be equal to a value of a different type. b was first equal to an integer, but it became equal to a string when it was assigned the value ’hello’. But you can see that type often matters, as when we try to print an integer in the midst of a string."
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "## Basic Types"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "### Numerical Types\n",
    "\n",
    "Python supports the following numerical, scalar types."
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "#### Integer"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": null,
   "metadata": {
    "collapsed": false
   },
   "outputs": [],
   "source": [
    "1 + 1"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "Remember how we saw integers in our CSV script example? The number of columns and number of rows are integers"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": null,
   "metadata": {
    "collapsed": false
   },
   "outputs": [],
   "source": [
    "a = len(column_names)\n",
    "type(a)"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "#### Floats\n",
    "\n",
    "Note: most decimal fractions cannot be represented exactly as binary fractions, and certain operations using floats may lead to surprising results. For more details, [start here](https://docs.python.org/3/tutorial/floatingpoint.html)."
   ]
  },
  {
   "cell_type": "code",
   "execution_count": null,
   "metadata": {
    "collapsed": false
   },
   "outputs": [],
   "source": [
    "c = 2.1\n",
    "type(c)"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": null,
   "metadata": {
    "collapsed": false
   },
   "outputs": [],
   "source": [
    "type(average_value)"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "#### Booleans"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": null,
   "metadata": {
    "collapsed": false
   },
   "outputs": [],
   "source": [
    "3 > 4"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": null,
   "metadata": {
    "collapsed": false
   },
   "outputs": [],
   "source": [
    "test = (3 > 4)\n",
    "test"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": null,
   "metadata": {
    "collapsed": false
   },
   "outputs": [],
   "source": [
    "type(test)"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "A Python shell can therefore replace your pocket calculator, with the basic arithmetic operations `+, -, *, /, %` (modulo) natively implemented.\n",
    "\n",
    "Try some things here or follow along with the instructor's examples:"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": null,
   "metadata": {
    "collapsed": false
   },
   "outputs": [],
   "source": []
  },
  {
   "cell_type": "code",
   "execution_count": null,
   "metadata": {
    "collapsed": false
   },
   "outputs": [],
   "source": []
  },
  {
   "cell_type": "code",
   "execution_count": null,
   "metadata": {
    "collapsed": false
   },
   "outputs": [],
   "source": []
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "Type conversion (casting):"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": null,
   "metadata": {
    "collapsed": false
   },
   "outputs": [],
   "source": [
    "float(1)"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "### Comments\n",
    "Commenting code is good practice and is extremely helpful to help others understand your code. And, often, to help **you** understand code that you've written earlier.\n",
    "\n",
    "In python, everything following the hash/pound sign `#` is a comment. Comments can either be their own line(s), or in-line. Use in-line comments sparingly."
   ]
  },
  {
   "cell_type": "code",
   "execution_count": null,
   "metadata": {
    "collapsed": true
   },
   "outputs": [],
   "source": [
    "# this is a comment. We might say, for example, that we're setting the value of Pi:\n",
    "pi = 3.14\n",
    "\n",
    "pie = 'pumpkin' # and this is an in-line comment. Setting the value of pie."
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "### Exercise\n",
    "Add three of the same single digit integers (e.g. 1 + 1 + 1). Is the result what you expected?\n",
    "\n",
    "Next, add three of the same tenths digit, which are floats (e.g. .1 + .1 + .1). Is the result what you expected?"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": null,
   "metadata": {
    "collapsed": false
   },
   "outputs": [],
   "source": []
  },
  {
   "cell_type": "code",
   "execution_count": null,
   "metadata": {
    "collapsed": false
   },
   "outputs": [],
   "source": []
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "What happened with those floats? How could we avoid this? There is an explanation and some suggestions in [this python documentaion on floating points](https://docs.python.org/3/tutorial/floatingpoint.html)."
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "### Containers\n",
    "\n",
    "Python provides many efficient types of containers, in which collections of data can be stored.\n",
    "\n",
    "#### Lists\n",
    "\n",
    "A list is an ordered collection of objects, that may have different types. For example:"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": null,
   "metadata": {
    "collapsed": false
   },
   "outputs": [],
   "source": [
    "l = ['red', 'blue', 'green', 'black', 'white']\n",
    "type(l)"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "And remember in our CSV script example, we used lots of lists! There was a list to store the column names, which we assumed were in the first row of data:"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": null,
   "metadata": {
    "collapsed": false
   },
   "outputs": [],
   "source": [
    "column_names"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "And then each row of the CSV was itself a list, and all the rows were another list. So we used a list of lists, or a matrix."
   ]
  },
  {
   "cell_type": "code",
   "execution_count": null,
   "metadata": {
    "collapsed": false
   },
   "outputs": [],
   "source": [
    "data"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "**Indexing**: accessing individual objects contained in the list:"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": null,
   "metadata": {
    "collapsed": false
   },
   "outputs": [],
   "source": [
    "column_names[0]"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": null,
   "metadata": {
    "collapsed": false
   },
   "outputs": [],
   "source": [
    "column_names[-1]"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "*Indexing starts at 0, not 1!*\n",
    "\n",
    "Slicing: obtaining sublists of regularly-spaced elements:"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": null,
   "metadata": {
    "collapsed": false
   },
   "outputs": [],
   "source": [
    "column_names[1:3]"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "*Note that `l[start:stop]` contains the elements with indices `i` such as `start<= i < stop` (`i` ranging from start to stop-1). Therefore, `l[start:stop]` has `(stop - start)` elements.*\n",
    "\n",
    "**Slicing syntax**: `l[start:stop:stride]`"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "Lists are *mutable* objects and can be modified:"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": null,
   "metadata": {
    "collapsed": false
   },
   "outputs": [],
   "source": [
    "column_names[0] = 'LOOK AT ME!'\n",
    "column_names"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "The elements of a list may have different types:"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": null,
   "metadata": {
    "collapsed": false
   },
   "outputs": [],
   "source": [
    "l = [3, -200, 'hello']\n",
    "l"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "Python offers a large panel of functions to modify lists, or query them. Here are a few examples; for more details, see https://docs.python.org/tutorial/datastructures.html#more-on-lists\n",
    "\n",
    "\n",
    "Add and remove elements:"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": null,
   "metadata": {
    "collapsed": false
   },
   "outputs": [],
   "source": [
    "L = ['red', 'blue', 'green', 'black', 'white']\n",
    "L.append('pink')\n",
    "L"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": null,
   "metadata": {
    "collapsed": false
   },
   "outputs": [],
   "source": [
    "L.pop() # removes and returns the last item"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": null,
   "metadata": {
    "collapsed": false
   },
   "outputs": [],
   "source": [
    "L"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "Add a list to the end of a list with extend()"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": null,
   "metadata": {
    "collapsed": true
   },
   "outputs": [],
   "source": [
    "L.extend(['pink', 'purple']) # extend L, in-place"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": null,
   "metadata": {
    "collapsed": false
   },
   "outputs": [],
   "source": [
    "L"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": null,
   "metadata": {
    "collapsed": false
   },
   "outputs": [],
   "source": [
    "L = L[:-2]\n",
    "L"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "Two ways to reverse a list:"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": null,
   "metadata": {
    "collapsed": false
   },
   "outputs": [],
   "source": [
    "r = L[::-1]\n",
    "r"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": null,
   "metadata": {
    "collapsed": false
   },
   "outputs": [],
   "source": [
    "r.reverse() # in-place\n",
    "r"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "Concatenate lists:"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": null,
   "metadata": {
    "collapsed": false
   },
   "outputs": [],
   "source": [
    "r + L"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "Sort:"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": null,
   "metadata": {
    "collapsed": false
   },
   "outputs": [],
   "source": [
    "sorted(r) # new object"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": null,
   "metadata": {
    "collapsed": false
   },
   "outputs": [],
   "source": [
    "r"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": null,
   "metadata": {
    "collapsed": false
   },
   "outputs": [],
   "source": [
    "r.sort() #in-place\n",
    "r"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "### Exercise\n",
    "We used `sorted()` in our CSV example a few times. That, along with list indexes, helped us get the lowest and highest value. Here's what it looked like:\n",
    "\n",
    "`            highest_value = sorted(column_values)[-1]`\n",
    "`            lowest_value = sorted(column_values)[0]`\n",
    "\n",
    "Try creating your own list of unsorted items. Can you replicate the highest and lowest value expressions above? What happens if your list is not made up of numbers?"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": null,
   "metadata": {
    "collapsed": true
   },
   "outputs": [],
   "source": []
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "**Methods**\n",
    "The notation `r.method()` (e.g. `r.append(3)` and `L.pop()`) is our first example of object-oriented programming (OOP). Being a list, the object r has a method function that is called using the notation `.methodname()`. We will talk about functions later in this tutorial. \n",
    "\n",
    "When you're using jupyter, to see all the different methods available to a variable, type a period after the variable name and hit the tab key."
   ]
  },
  {
   "cell_type": "code",
   "execution_count": null,
   "metadata": {
    "collapsed": false
   },
   "outputs": [],
   "source": []
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "#### Strings\n",
    "\n",
    "We've already seen strings a few times. Python supports many different string syntaxes (single, double or triple quotes):"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": null,
   "metadata": {
    "collapsed": true
   },
   "outputs": [],
   "source": [
    "s = 'Hello, how are you?'\n",
    "s = \"Hi, what's up\"\n",
    "s = '''Hello, # tripling the quotes allows the\n",
    "how are you''' # the string to span more than one line\n",
    "s = \"\"\"Hi,\n",
    "what's up?\"\"\""
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "Double quotes are crucial when you have a quote in the string:"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": null,
   "metadata": {
    "collapsed": false
   },
   "outputs": [],
   "source": [
    "s = 'Hi, what's up?'"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "The newline character is \\n, and the tab character is \\t.\n",
    "\n",
    "Strings are collections like lists. Hence they can be indexed and sliced, using the same syntax and rules.\n",
    "\n",
    "**Indexing strings**:"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": null,
   "metadata": {
    "collapsed": false
   },
   "outputs": [],
   "source": [
    "a = \"hello\"\n",
    "a[0]"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": null,
   "metadata": {
    "collapsed": false
   },
   "outputs": [],
   "source": [
    "a[1]"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": null,
   "metadata": {
    "collapsed": false
   },
   "outputs": [],
   "source": [
    "a[-1]"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "Accents and special characters can also be handled in strings because since Python 3, the string type handles unicode (UTF-8) by default.(For a lot more on Unicode, character encoding, and how it relates to python, see\n",
    "https://docs.python.org/3/howto/unicode.html).\n",
    "\n",
    "A string is an ***immutable object*** and it is not possible to modify its contents. If you want to modify a string, you'll create a new string from the original one (or use a method that returns a new string)."
   ]
  },
  {
   "cell_type": "code",
   "execution_count": null,
   "metadata": {
    "collapsed": false
   },
   "outputs": [],
   "source": [
    "a = \"hello, world!\"\n",
    "a[2] = 'z'"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": null,
   "metadata": {
    "collapsed": false
   },
   "outputs": [],
   "source": [
    "a.replace('l', 'z', 1)"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": null,
   "metadata": {
    "collapsed": false
   },
   "outputs": [],
   "source": [
    "a.replace('l', 'z')"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "Strings have many useful methods, such as `a.replace` as seen above. Remember the `a.` object-oriented notation and use tab completion or `help(str)` to search for new methods.\n",
    "\n",
    "### Exercise\n",
    "\n",
    "We used a few string methods in the CSV example at the start. See them in this chunk of code?\n",
    "\n",
    "```    if len(value) < 1 or value.isspace():\n",
    "        column_datatypes.append('string')\n",
    "    elif value.isalpha():\n",
    "        column_datatypes.append('string')\n",
    "    elif '.' in value or value.isdigit():```\n",
    "\n",
    "Now you try. Create a new variable and assign it with a string. Then, try a few of [python's string methods](https://docs.python.org/2/library/stdtypes.html#string-methods) to see how you can return different versions of your string, or test whether it has certain characteristics.\n"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": null,
   "metadata": {
    "collapsed": true
   },
   "outputs": [],
   "source": []
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "**String formatting:**\n",
    "\n",
    "We also saw string formatting in the CSV example, when we printed some of the reporting:\n",
    "\n",
    "`print(\"\\t{0} ({1} of {2} unique)\".format(column_datatypes[i], unique_value_count, len(data)))`"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": null,
   "metadata": {
    "collapsed": false
   },
   "outputs": [],
   "source": [
    "'An integer: {0} ; a float: {1} ; another string: {2} '.format(1, 0.1, 'string')"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": null,
   "metadata": {
    "collapsed": false
   },
   "outputs": [],
   "source": [
    "i = 102\n",
    "filename = 'processing_of_dataset_{0}.txt'.format(i)\n",
    "filename"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "#### Dictionaries\n",
    "\n",
    "A dictionary is basically an efficient table that maps keys to values. It is an unordered container:"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": null,
   "metadata": {
    "collapsed": false
   },
   "outputs": [],
   "source": [
    "tel = {'emmanuelle': 5752, 'sebastian': 5578}\n",
    "tel['francis'] = 5915\n",
    "tel"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": null,
   "metadata": {
    "collapsed": false
   },
   "outputs": [],
   "source": [
    "tel['sebastian']"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": null,
   "metadata": {
    "collapsed": false
   },
   "outputs": [],
   "source": [
    "tel.keys()"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": null,
   "metadata": {
    "collapsed": false
   },
   "outputs": [],
   "source": [
    "tel.values()"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": null,
   "metadata": {
    "collapsed": false
   },
   "outputs": [],
   "source": [
    "'francis' in tel"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "It can be used to conveniently store and retrieve values associated with a name (a string for a date, a name, etc.). See https://docs.python.org/tutorial/datastructures.html#dictionaries for more information.\n",
    "\n",
    "A dictionary can have keys (resp. values) with different types:"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": null,
   "metadata": {
    "collapsed": false
   },
   "outputs": [],
   "source": [
    "d = {'a':1, 'b':2, 3:'hello'}\n",
    "d"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "### The assignment operator\n",
    "\n",
    "[Python library reference](https://docs.python.org/reference/simple_stmts.html#assignment-statements) says:\n",
    "\n",
    ">Assignment statements are used to (re)bind names to values and to modify attributes or items of\n",
    "mutable objects.\n",
    "\n",
    "In short, it works as follows (simple assignment):\n",
    "1. an expression on the right hand side is evaluated, the corresponding object is created/obtained\n",
    "2. a name on the left hand side is assigned, or bound, to the r.h.s. object\n",
    "\n",
    "Things to note:\n",
    "* a single object can have several names bound to it:"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": null,
   "metadata": {
    "collapsed": false
   },
   "outputs": [],
   "source": [
    "a = [1, 2, 3]\n",
    "b = a\n",
    "a"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": null,
   "metadata": {
    "collapsed": false
   },
   "outputs": [],
   "source": [
    "b"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": null,
   "metadata": {
    "collapsed": false
   },
   "outputs": [],
   "source": [
    "a is b"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": null,
   "metadata": {
    "collapsed": false
   },
   "outputs": [],
   "source": [
    "b[1] = \"hi!\"\n",
    "a"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "### Control Flow\n",
    "\n",
    "Controls the order in which the code is executed."
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "### Conditional Expressions\n",
    "\n",
    "```\n",
    "if <THING>\n",
    "    Evaluates to false:\n",
    "        * any number equal to zero (0,0.0)\n",
    "        * an empty container (list, dictionary)\n",
    "        * False, None\n",
    "    Evaluates to True:\n",
    "        * everything else\n",
    "```\n",
    "`a == b` Tests equality:"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "#### if/elif/else"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": null,
   "metadata": {
    "collapsed": false
   },
   "outputs": [],
   "source": [
    "if 2**2 == 4:\n",
    "    print('Obviously')"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "`a in b` For any collection `b` check to see if `b` contains `a`:"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": null,
   "metadata": {
    "collapsed": false
   },
   "outputs": [],
   "source": [
    "b = [1,2,3]\n",
    "2 in b"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": null,
   "metadata": {
    "collapsed": false
   },
   "outputs": [],
   "source": [
    "5 in b"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "**Blocks are delimited by indentation**\n",
    "\n",
    "Type the following lines in your Python interpreter, and be careful to **respect the indentation depth**. The Jupyter Notebook automatically increases the indentation depth after a coln `:` sign; to decrease the indentation depth, go four spaces to the left with the Backspace key. Press the Enter key twice to leave the logical block."
   ]
  },
  {
   "cell_type": "code",
   "execution_count": null,
   "metadata": {
    "collapsed": false
   },
   "outputs": [],
   "source": [
    "a = 10\n",
    "\n",
    "if a == 1:\n",
    "    print(1)\n",
    "elif a == 2:\n",
    "    print(2)\n",
    "else:\n",
    "    print(\"A lot\")"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "#### for/range\n",
    "\n",
    "Iterating with an index:"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": null,
   "metadata": {
    "collapsed": false
   },
   "outputs": [],
   "source": [
    "for i in range(4):\n",
    "    print(i)"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "But most often, it is more readable to iterate over values:"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": null,
   "metadata": {
    "collapsed": false
   },
   "outputs": [],
   "source": [
    "for word in ('cool', 'powerful', 'readable'):\n",
    "    print('Python is %s ' % word)"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "#### Advanced iteration\n",
    "\n",
    "Iterate over any **sequence**\n",
    "\n",
    "You can iterate over any sequence (string, list, keys in a dictionary, lines in a file, ...):"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": null,
   "metadata": {
    "collapsed": false
   },
   "outputs": [],
   "source": [
    "vowels = 'aeiouy'\n",
    "\n",
    "for i in 'powerful':\n",
    "    if i in vowels:\n",
    "        print(i)"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": null,
   "metadata": {
    "collapsed": false
   },
   "outputs": [],
   "source": [
    "message = \"Hello how are you?\"\n",
    "message.split() # returns a list"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": null,
   "metadata": {
    "collapsed": false
   },
   "outputs": [],
   "source": [
    "for word in message.split():\n",
    "    print(word)"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "Few languages (in particular, languages for scientific computing) allow to loop over anything but integers/indices.\n",
    "\n",
    "With Python it is possible to loop exactly over the objects of interest without bothering with\n",
    "indices you often don’t care about. This feature can often be used to make code more readable.\n",
    "\n",
    "*It is **not** safe to modify the sequence you are iterating over.*\n",
    "\n",
    "**Keeping track of enumeration number**\n",
    "\n",
    "Common task is to iterate over a sequence while keeping track of the item number.\n",
    "* Could use while loop with a counter as above. Or a for loop:"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": null,
   "metadata": {
    "collapsed": false
   },
   "outputs": [],
   "source": [
    "words = ['cool', 'powerful', 'readable']\n",
    "for i in range(0, len(words)):\n",
    "    print(i, words[i])"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "* But, Python provides a built-in function - enumerate - for this:"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": null,
   "metadata": {
    "collapsed": false
   },
   "outputs": [],
   "source": [
    "for index, item in enumerate(words):\n",
    "    print(index, item)"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "When looping over a dictionary use `.items()`:"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": null,
   "metadata": {
    "collapsed": false
   },
   "outputs": [],
   "source": [
    "d = {'a': 1, 'b':1.2, 'c':\"hi\"}\n",
    "\n",
    "for key, val in sorted(d.items()):\n",
    "    print('Key: %s has value: %s ' % (key, val))"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "The ordering of a dictionary in random, thus we use `sorted()` which will sort on the keys."
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "## Exercise\n",
    "### Countdown to blast off\n",
    "Write code that uses a loop to print a count down from 10 to 1, followed by printing the string \"blast off!\". There's more than one way to do this, so figure out what works for you, drawing on things we've already learned."
   ]
  },
  {
   "cell_type": "code",
   "execution_count": null,
   "metadata": {
    "collapsed": true
   },
   "outputs": [],
   "source": []
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "#### List Comprehensions"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": null,
   "metadata": {
    "collapsed": false
   },
   "outputs": [],
   "source": [
    "[i**2 for i in range(4)]"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "Same as:"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": null,
   "metadata": {
    "collapsed": false
   },
   "outputs": [],
   "source": [
    "l = []\n",
    "for i in range(4):\n",
    "    l.append(i)\n",
    "l"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "Now that you've done the countdown exercise above, consider how you could have used a list comprehension in a solution:"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": null,
   "metadata": {
    "collapsed": false
   },
   "outputs": [],
   "source": [
    "[10 - i for i in range(10)]"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {
    "collapsed": true
   },
   "source": [
    "### Defining Functions\n",
    "\n",
    "#### Function definition"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": null,
   "metadata": {
    "collapsed": false
   },
   "outputs": [],
   "source": [
    "def test():\n",
    "    print('in test function')\n",
    "    \n",
    "test()"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "*Function blocks must be indented as other control flow blocks*\n",
    "\n",
    "#### Return statement\n",
    "\n",
    "Functions can *optionally* return values:"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": null,
   "metadata": {
    "collapsed": false
   },
   "outputs": [],
   "source": [
    "def disk_area(radius):\n",
    "    return 3.14 * radius * radius\n",
    "\n",
    "disk_area(1.5)"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "By default, functions return `None`.\n",
    "\n",
    "Note the syntax to define a function:\n",
    "* the def keyword;\n",
    "* is followed by the function’s **name**, then\n",
    "* the arguments of the function are given between parentheses followed by a colon.\n",
    "* the function body;\n",
    "* and return `object` for optionally returning values.\n",
    "\n",
    "#### Parameters\n",
    "\n",
    "Mandatory parameters (positional arguments):"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": null,
   "metadata": {
    "collapsed": false
   },
   "outputs": [],
   "source": [
    "def double_it(x):\n",
    "    return x * 2\n",
    "\n",
    "double_it(3)"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": null,
   "metadata": {
    "collapsed": false
   },
   "outputs": [],
   "source": [
    "double_it()"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "Optional parameters (keyword or named arguments)"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": null,
   "metadata": {
    "collapsed": false
   },
   "outputs": [],
   "source": [
    "def double_it(x=2):\n",
    "    return x * 2\n",
    "\n",
    "double_it()"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": null,
   "metadata": {
    "collapsed": false
   },
   "outputs": [],
   "source": [
    "double_it(3)"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "Keyword arguments allow you to specify *default values.*\n",
    "\n",
    "*Default values are evaluated when the function is defined, not when it is called. This can be problematic when using mutable types (e.g. dictionary or list) and modifying them in the function body, since the modifications will be persistent across invocations of the function.*\n",
    "\n"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "#### Global variables\n",
    "\n",
    "Variables declared outside the function can be referenced within the function:"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": null,
   "metadata": {
    "collapsed": false
   },
   "outputs": [],
   "source": [
    "# We're defining a global variable for pi, and it's actually a special kind of global \n",
    "# because we intend it to be constant (i.e. it's value doesn't change). There's a convention\n",
    "# of using uppercase in naming constants. See https://www.python.org/dev/peps/pep-0008/#constants\n",
    "PI = 3.14159\n",
    "\n",
    "def disk_area(radius):\n",
    "    return PI * radius * radius\n",
    "\n",
    "disk_area(1.5)"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "#### Docstrings\n",
    "\n",
    "Documentation about what the function does and its parameters. General convention:"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": null,
   "metadata": {
    "collapsed": false
   },
   "outputs": [],
   "source": [
    "def funcname(params):\n",
    "    \"\"\"Concise one-line sentence describing the function.\n",
    "\n",
    "    Extended summary which can contain multiple paragraphs.\n",
    "    \"\"\"\n",
    "    # function body\n",
    "    pass"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "There's a great help feature build into Jupyter: type a question mark after any object or function to get quick access to its docstring. Try it:"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": null,
   "metadata": {
    "collapsed": true
   },
   "outputs": [],
   "source": [
    "funcname?"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "**Docstring guidelines**\n",
    "For the sake of standardization, the [Docstring Conventions](https://www.python.org/dev/peps/pep-0257) webpage documents the semantics and conventions associated with Python docstrings.\n",
    "\n",
    "Also, the Numpy and Scipy modules have defined a precise standard for documenting scientific functions, that you may want to follow for your own functions, with a Parameters section, an `Examples` section, etc. See http://projects.scipy.org/numpy/wiki/CodingStyleGuidelines#docstring-standard and\n",
    "http://projects.scipy.org/numpy/browser/trunk/doc/example.py#L37"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "### Create your own function\n",
    "Use any features of Python that we've already worked on, or play with something new from the documentation. Write a function, and include a docstring explaining the function's purpose. Test your function by executing it. If your function uses parameters, try calling the function a few times with different parameters.\n",
    "\n",
    "Define the function here:"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": null,
   "metadata": {
    "collapsed": true
   },
   "outputs": [],
   "source": []
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "Call the function here:"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": null,
   "metadata": {
    "collapsed": true
   },
   "outputs": [],
   "source": []
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "### Importing modules"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "#### Importing objects from modules\n",
    "\n",
    "Modules let you use code that doesn't reside within the notebook and might be part of the *standard library*. (more on this later)"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": null,
   "metadata": {
    "collapsed": false
   },
   "outputs": [],
   "source": [
    "import os\n",
    "\n",
    "os"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "#### Methods\n",
    "\n",
    "Methods are functions attached to objects. You’ve seen these in our examples on lists, dictionaries, strings, etc..."
   ]
  },
  {
   "cell_type": "code",
   "execution_count": null,
   "metadata": {
    "collapsed": false
   },
   "outputs": [],
   "source": [
    "os.listdir('.')"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "And also:"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": null,
   "metadata": {
    "collapsed": false
   },
   "outputs": [],
   "source": [
    "from os import listdir\n",
    "\n",
    "listdir('.')"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "Using alias:"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": null,
   "metadata": {
    "collapsed": true
   },
   "outputs": [],
   "source": [
    "import pandas as pd"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "Modules are thus a good way to organize code in a hierarchical way. Actually, all the data science tools we are going to use are modules:"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": null,
   "metadata": {
    "collapsed": false
   },
   "outputs": [],
   "source": [
    "import pandas as pd\n",
    "\n",
    "pd.Series([0,1,2,3,4,5,6,7,8,9])"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "#### Good Practices\n",
    "\n",
    "* Use **meaningful** object **names**\n",
    "* **Indentation: no choice!**\n",
    "\n",
    "Indenting is compulsory in Python! Every command block following a colon bears an additional indentation level with respect to the previous line with a colon. One must therefore indent after `def f():` or `while:`.\n",
    "At the end of such logical blocks, one decreases the indentation depth (and re-increases it if a new block is entered, etc.)\n",
    "\n",
    "Strict respect of indentation is the price to pay for getting rid of `{` or `;` characters that delineate logical blocks in other languages. Improper indentation leads to errors such as:\n",
    "```\n",
    "------------------------------------------------------------\n",
    "IndentationError: unexpected indent (test.py, line 2)\n",
    "```\n",
    "All this indentation business can be a bit confusing in the beginning. However, with the clear indentation, and in the absence of extra characters, the resulting code is very nice to read compared to other languages.\n",
    "* **Indentation depth**: Inside your text editor, you may choose to indent with any positive number of spaces (1, 2, 3, 4, ...). However, it is considered good practice to **indent with 4 spaces**. You may configure your editor to map the `Tab` key to a 4-space indentation. In Python(x,y), the editor is already configured this way.\n",
    "* **Style guidelines**\n",
    "**Long lines**: you should not write very long lines that span over more than (e.g.) 80 characters. Long lines\n",
    "can be broken with the \\ character"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": null,
   "metadata": {
    "collapsed": true
   },
   "outputs": [],
   "source": [
    "long_line = \"Here is a very very long line \\\n",
    "... that we break in two parts.\""
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "**Spaces**\n",
    "Write well-spaced code: put whitespaces after commas, around arithmetic operators, etc.:"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": null,
   "metadata": {
    "collapsed": true
   },
   "outputs": [],
   "source": [
    "a = 1 # yes\n",
    "a=1 # too cramped"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "A certain number of rules for writing “beautiful” code (and more importantly using the same conventions as anybody else!) are given in the [PEP-8: Style Guide for Python Code](https://www.python.org/dev/peps/pep-0008).\n"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "### Input and Output\n",
    "\n",
    "We write or read **strings** to/from files (other types must be converted to strings). To write in a file:"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": null,
   "metadata": {
    "collapsed": false
   },
   "outputs": [],
   "source": [
    "f = open('workfile.txt', 'w') # opens the workfile file in writing mode\n",
    "type(f)"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": null,
   "metadata": {
    "collapsed": true
   },
   "outputs": [],
   "source": [
    "f.write('This is a test \\nand another test')\n",
    "f.close() # always use close() after opening a file! Very important!"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "To read from a file:"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": null,
   "metadata": {
    "collapsed": false
   },
   "outputs": [],
   "source": [
    "f = open('workfile.txt', 'r')\n",
    "s = f.read()\n",
    "print(s)\n",
    "f.close()"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "For more details: https://docs.python.org/tutorial/inputoutput.html"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "#### Iterating over a file"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": null,
   "metadata": {
    "collapsed": false
   },
   "outputs": [],
   "source": [
    "f = open('workfile.txt', 'r')\n",
    "for line in f:\n",
    "    print(line)\n",
    "f.close()"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "File modes:\n",
    "* r: Read-only\n",
    "* w: Write-only\n",
    "    - Note: This will create a new file or *overwrite* an existing file\n",
    "* a: Append to a file\n",
    "* r+: Read and Write\n",
    "\n",
    "For more information about file modes read the documentation for the `open()` function. https://docs.python.org/3.5/library/functions.html#open"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "### The Standard Library\n",
    "\n",
    "Reference documentation for this section:\n",
    "* The Python Standard Library documentation: https://docs.python.org/library/index.html\n",
    "* Python Essential Reference, David Beazley, Addison-Wesley Professional\n",
    "\n",
    "#### `os` module: operating system functionality\n",
    "\n",
    "> \"A portable way of using operating system dependent functionality.”\n",
    "\n",
    "**Directory and file manipulation\n",
    "\n",
    "Get the current directory:"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": null,
   "metadata": {
    "collapsed": false
   },
   "outputs": [],
   "source": [
    "import os\n",
    "os.getcwd()"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "List a directory:"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": null,
   "metadata": {
    "collapsed": false
   },
   "outputs": [],
   "source": [
    "os.listdir(os.curdir)"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "Make a directory:"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": null,
   "metadata": {
    "collapsed": false
   },
   "outputs": [],
   "source": [
    "os.mkdir('junkdir')\n",
    "\n",
    "'junkdir' in os.listdir(os.curdir)"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "Rename the directory:"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": null,
   "metadata": {
    "collapsed": false
   },
   "outputs": [],
   "source": [
    "os.rename('junkdir', 'foodir')\n",
    "\n",
    "'junkdir' in os.listdir(os.curdir)"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": null,
   "metadata": {
    "collapsed": false
   },
   "outputs": [],
   "source": [
    "'foodir' in os.listdir(os.curdir)"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": null,
   "metadata": {
    "collapsed": false
   },
   "outputs": [],
   "source": [
    "os.rmdir('foodir') #remove directory\n",
    "'foodir' in os.listdir(os.curdir)"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "Delete a file:"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": null,
   "metadata": {
    "collapsed": false
   },
   "outputs": [],
   "source": [
    "fp = open('junk.txt', 'w')\n",
    "fp.close()\n",
    "'junk.txt' in os.listdir(os.curdir)"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": null,
   "metadata": {
    "collapsed": false
   },
   "outputs": [],
   "source": [
    "os.remove('junk.txt')\n",
    "'junk.txt' in os.listdir(os.curdir)"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "#### `glob`: Pattern matching on files\n",
    "\n",
    "The glob module provides convenient file pattern matching.\n",
    "\n",
    "Find all files ending in `.txt`:"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": null,
   "metadata": {
    "collapsed": false
   },
   "outputs": [],
   "source": [
    "import glob\n",
    "\n",
    "glob.glob('*.txt')"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "### Exception handling in Python\n",
    "\n",
    "It is likely that you have raised Exceptions if you have typed all the previous commands of the tutorial. For example, you may have raised an exception if you entered a command with a typo.\n",
    "\n",
    "Exceptions are raised by different kinds of errors arising when executing Python code. In your own code, you may also catch errors, or define custom error types. You may want to look at the descriptions of the [built-in\n",
    "Exceptions](https://docs.python.org/2/library/exceptions.html) when looking for the right exception type."
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "#### Exceptions\n",
    "\n",
    "Exceptions are raised by errors in Python:"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": null,
   "metadata": {
    "collapsed": false
   },
   "outputs": [],
   "source": [
    "1/0"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": null,
   "metadata": {
    "collapsed": false
   },
   "outputs": [],
   "source": [
    "d = {1:1, 2:2}\n",
    "\n",
    "d[3]"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": null,
   "metadata": {
    "collapsed": false
   },
   "outputs": [],
   "source": [
    "l = [1, 2, 3]\n",
    "\n",
    "l[4]"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": null,
   "metadata": {
    "collapsed": false
   },
   "outputs": [],
   "source": [
    "l.foobar"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "#### Catching exceptions\n",
    "\n",
    "**try/except**"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": null,
   "metadata": {
    "collapsed": false
   },
   "outputs": [],
   "source": [
    "while True:\n",
    "    try:\n",
    "        x = int(input('Please enter a number: '))\n",
    "        break\n",
    "    except ValueError:\n",
    "        print('That was no valid number. Try again...')\n",
    "x"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "**try/finally**"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": null,
   "metadata": {
    "collapsed": false
   },
   "outputs": [],
   "source": [
    "try:\n",
    "    x = int(input('Please enter a number: '))\n",
    "finally:\n",
    "    print('Thank you for your input.')"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "#### Raising exceptions\n",
    "\n",
    "* Capturing and reraising an exception:"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": null,
   "metadata": {
    "collapsed": false
   },
   "outputs": [],
   "source": [
    "def filter_name(name):\n",
    "    try:\n",
    "        name = name.encode('ascii')\n",
    "    except UnicodeError as e:\n",
    "        if name == 'Gaël':\n",
    "            print(\"OK, Gaël\")\n",
    "        else:\n",
    "            raise e\n",
    "    return name\n",
    "\n",
    "filter_name(\"Gaël\")"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": null,
   "metadata": {
    "collapsed": false
   },
   "outputs": [],
   "source": [
    "filter_name('Stéfan')"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "* Exceptions to pass messages between parts of the code:"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": null,
   "metadata": {
    "collapsed": false
   },
   "outputs": [],
   "source": [
    "def achilles_arrow(x):\n",
    "    if abs(x - 1) < 1e-3:\n",
    "        raise StopIteration\n",
    "    x = 1 - (1-x)/2.\n",
    "    return x\n",
    "\n",
    "x = 0\n",
    "\n",
    "while True:\n",
    "    try:\n",
    "        x = achilles_arrow(x)\n",
    "    except StopIteration:\n",
    "        break\n",
    "\n",
    "x"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "Use exceptions to notify certain conditions are met (e.g. StopIteration) or not (e.g. custom error raising)"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": null,
   "metadata": {
    "collapsed": true
   },
   "outputs": [],
   "source": []
  }
 ],
 "metadata": {
  "kernelspec": {
   "display_name": "Python 3",
   "language": "python",
   "name": "python3"
  },
  "language_info": {
   "codemirror_mode": {
    "name": "ipython",
    "version": 3
   },
   "file_extension": ".py",
   "mimetype": "text/x-python",
   "name": "python",
   "nbconvert_exporter": "python",
   "pygments_lexer": "ipython3",
   "version": "3.6.0"
  }
 },
 "nbformat": 4,
 "nbformat_minor": 0
}