2-python-basics-recap-and-more.ipynb

{
 "cells": [
  {
   "cell_type": "markdown",
   "metadata": {
    "collapsed": true
   },
   "source": [
    "# More Python Basics\n",
    "\n",
    "---\n",
    "---\n",
    "\n",
    "## Going Further with Python for Text-mining\n",
    "\n",
    "This notebook builds on the [previous notebook](1-intro-to-python-and-text.ipynb) to teach you a bit more Python so you can understand the text-mining examples presented in the following notebooks.\n",
    "\n",
    "These are the fundamentals in working with strings in Python and other basics that every Python user may use every day. However, this is just an introduction and it is not expected that you will be ready and capable of simply diving in to coding straight after completing this course.\n",
    "\n",
    "Rather, these notebooks and the accompanying live teaching sessions are supposed to give you just a taster of what text-mining with Python is about. By the end of the course I hope you will come away with either: an interest to learn more; or equally valid, an informed feeling that coding is not for you.\n",
    "\n",
    "Having said this, there are many approaches to learning programming, and it is often only once you happen upon the right approach for you that you make good progress. It is worth trying different topics, teachers, media and learning styles. I tried to learn programming several times over many years and eventually found the right course that kickstarted my own coding journey.\n",
    "\n",
    "---\n",
    "---\n",
    "\n",
    "## Recap of Strings\n",
    "Welcome back! Here's a quick recap of what we learnt in [1-intro-to-python-and-text](1-intro-to-python-and-text.ipynb). Strings are the way that Python deals with text. \n",
    "\n",
    "Create a *string* and store it with a *name*:"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": null,
   "metadata": {},
   "outputs": [],
   "source": [
    "my_sentence = 'The Moon formed 4.51 billion years ago.'\n",
    "my_sentence"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "_Concatenate_ strings together:"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": null,
   "metadata": {},
   "outputs": [],
   "source": [
    "my_sentence + \" \" + \"It is the fifth-largest satellite in the Solar System.\""
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "_Index_ a string. Remember that indexing in Python starts at 0."
   ]
  },
  {
   "cell_type": "code",
   "execution_count": null,
   "metadata": {},
   "outputs": [],
   "source": [
    "my_sentence[16]"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "_Slice_ a string. Remember that the slice goes from the first index up to but _not_ including the second index."
   ]
  },
  {
   "cell_type": "code",
   "execution_count": null,
   "metadata": {},
   "outputs": [],
   "source": [
    "my_sentence[0:20]"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "Transform a string with _string methods_. Important: the original string `my_sentence` is unchanged. Instead, a string method _returns_ a new string."
   ]
  },
  {
   "cell_type": "code",
   "execution_count": null,
   "metadata": {},
   "outputs": [],
   "source": [
    "my_sentence.swapcase()"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "Test a string with string methods:"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": null,
   "metadata": {},
   "outputs": [],
   "source": [
    "my_sentence.islower()"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "Create a _list_ of strings:"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": null,
   "metadata": {},
   "outputs": [],
   "source": [
    "my_list = ['The Moon formed 4.51 billion years ago',\n",
    "           \"The Moon is Earth's only permanent natural satellite\",\n",
    "          'The Moon was first reached in September 1959']\n",
    "my_list"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "_Slice_ a list. Add a _step_ to jump through a string or list by more than one. Use a step of `-1` to go backwards. "
   ]
  },
  {
   "cell_type": "code",
   "execution_count": null,
   "metadata": {},
   "outputs": [],
   "source": [
    "my_list[0:3:2]"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "---\n",
    "---\n",
    "\n",
    "## Create a List of Strings with List Comprehensions\n",
    "\n",
    "Let's get going on some new material.\n",
    "\n",
    "We can create new lists in a quick and elegant way by using _list comprehensions_. Essentially, a list comprehension _loops_ over each item in a list, one by one, and returns something each time, and creates a new list.\n",
    "\n",
    "For example, here is a list of strings:\n",
    "\n",
    "`['banana', 'apple', 'orange', 'kiwi']`\n",
    "\n",
    "We could use a list comprehension to loop over this list and create a new list with every item made UPPERCASE. The resulting list would look like this:\n",
    "\n",
    "`['BANANA', 'APPLE', 'ORANGE', 'KIWI']`\n",
    "\n",
    "The code for doing this is below:"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": null,
   "metadata": {},
   "outputs": [],
   "source": [
    "fruit = ['banana', 'apple', 'orange', 'kiwi']\n",
    "fruit_u = [item.upper() for item in fruit]\n",
    "fruit_u"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "The pattern is as follows:\n",
    "\n",
    "`[return_something for each_item in list]`\n",
    "\n",
    "First thing to say is the `for` and `in` are *keywords*, that is, they are special reserved words in Python. These must be present exactly in this order in every list comprehension.\n",
    "\n",
    "The other words (`return_something`, `each_item`, `list`) are placeholders for whatever variables (names) you are working with in your case.\n",
    "\n",
    "![List comprehensions diagram](assets/list-comprehension.png)\n",
    "\n",
    "> Let's look at some of the details:\n",
    " * A list comprehension goes inside square brackets (`[]`), which tells Python to create a new list.\n",
    " * `list` is the name of your list. It has to be a list you have already created in a previous step.\n",
    " * The `each_item in list` part is the loop.\n",
    " * `each_item` is the name you assign to each item as it is selected by the loop. The name you choose should be something descriptive that helps you remember what it is.\n",
    " * The `return_something for` part is what happens each time it loops over an item. The `return_something` could just be the original item, or it could be something fairly complicated.\n",
    "\n",
    "The most basic example is just to return exactly the same item each time it loops over and return all items in a list.\n",
    "\n",
    "Here is an example where we have taken our original list `my_list` and created a new list `new_list` with the exact same items unchanged:"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": null,
   "metadata": {},
   "outputs": [],
   "source": [
    "new_list = [item for item in my_list]\n",
    "new_list"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "Why do this? There does not seem much point to creating the same list again. "
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "### Manipulate Lists with String Methods\n",
    "\n",
    "By adding a string method to a list comprehension we have a powerful way to manipulate a list.\n",
    "\n",
    "We have already seen this in the `fruit` example above. Here's another example of the same thing with the 'Moon' list we've been working with. Every time the Python loops over an item it transforms it to uppercase before adding it to the new list:"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": null,
   "metadata": {},
   "outputs": [],
   "source": [
    "new_list_upper = [item.upper() for item in my_list]\n",
    "new_list_upper"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": null,
   "metadata": {},
   "outputs": [],
   "source": [
    "# Write code to transform every item in the list with a string method (of your choice)"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "Hint: see the [full documentation on string methods](https://docs.python.org/3.10/library/stdtypes.html#string-methods)."
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "### Filter Lists with a Condition\n",
    "\n",
    "We can _filter_ a list by adding a _condition_ so that only certain items are included in the new list:"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": null,
   "metadata": {},
   "outputs": [],
   "source": [
    "new_list_p = [item for item in my_list if 'p' in item]\n",
    "new_list_p"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "The pattern is as follows:\n",
    "\n",
    "`[return_something for each_item in list if some_condition]`\n",
    "\n",
    "![List comprehensions with condition diagram](assets/list-comprehension-with-condition.png)\n",
    "\n",
    "Essentially, what we are saying here is that **if** the character \"p\" is **in** the item when Python loops over it, keep it and add it to the new list, otherwise ignore it and throw it away.\n",
    "\n",
    "Thus, the new list has only two of the strings in it. The first string has a \"p\" in \"permanent\"; the second has a \"p\" in \"September\"."
   ]
  },
  {
   "cell_type": "code",
   "execution_count": null,
   "metadata": {},
   "outputs": [],
   "source": [
    "# Write code to filter the list for items that include a number (of your choice)"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "---\n",
    "---\n",
    "## Adding New Capabilities with Imports\n",
    "\n",
    "Python has a lot of amazing capabilities built-in to the language itself, like being able to manipulate strings. However, in any Python project you are likely to want to use Python code written by someone else to go beyond the built-in capabilities. Code 'written by someone else' comes in the form of a file (or files) separate to the one you are currently working on.\n",
    "\n",
    "An external Python file (or sometimes a *package* of files) is called a *module* and in order to use them in your code, you need to *import* it.\n",
    "\n",
    "This is a simple process using the keyword `import` and the name of the module. Just make sure that you `import` something _before_ you want to use it!\n",
    "\n",
    "The pattern is as follows:\n",
    "\n",
    "`import module_name`\n",
    "\n",
    "Here are a series of examples. See if you can guess what each one is doing before running it."
   ]
  },
  {
   "cell_type": "code",
   "execution_count": null,
   "metadata": {},
   "outputs": [],
   "source": [
    "import math\n",
    "math.pi"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": null,
   "metadata": {},
   "outputs": [],
   "source": [
    "import random\n",
    "random.random()"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": null,
   "metadata": {},
   "outputs": [],
   "source": [
    "import locale\n",
    "locale.getlocale()"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "The answers are: the value of the mathematical constant *pi*, a random number (different every time you run it), and the current locale that the computer thinks it is working in."
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "---\n",
    "---\n",
    "## Reusing Code with Functions\n",
    "\n",
    "A function is a _reusable block of code_ that has been wrapped up and given a _name_. The function might have been written by someone else, or it could have been written by you. We don't cover how to write functions in this course; just how to run functions written by someone else.\n",
    "\n",
    "In order to run the code of a function, we use the name followed by parentheses `()`. \n",
    "\n",
    "The pattern is as follows:\n",
    "\n",
    "`name_of_function()`\n",
    "\n",
    "We have already seen this earlier. Here are a selection of functions (or methods) we have run so far:"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": null,
   "metadata": {},
   "outputs": [],
   "source": [
    "# 'lower()' is the function (aka method)\n",
    "my_sentence = 'Butterflies are important as pollinators.'\n",
    "my_sentence.lower()"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": null,
   "metadata": {},
   "outputs": [],
   "source": [
    "# 'isalpha()' is the function (aka method)\n",
    "my_sentence.isalpha()"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": null,
   "metadata": {},
   "outputs": [],
   "source": [
    "# 'random()' is the function\n",
    "random.random()"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "---\n",
    "#### Functions and Methods\n",
    "There is a technical difference between functions and methods. You don't need to worry about the distinction for our course. We will treat all functions and methods as the same.\n",
    "\n",
    "If you are interested in learning more about functions and methods try this [Datacamp Python Functions Tutorial](https://www.datacamp.com/community/tutorials/functions-python-tutorial).\n",
    "\n",
    "---"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "### Functions that Take Arguments\n",
    "If we need to pass particular information to a function, we put that information _in between_ the `()`. Like this:"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": null,
   "metadata": {},
   "outputs": [],
   "source": [
    "math.sqrt(25)"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "The `25` is the value we want to pass to the `sqrt()` function so it can do its work. This value is called an _argument_ to the function. Functions may take any number of arguments, depending on what the function needs.\n",
    "\n",
    "Here is another function with an argument:"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": null,
   "metadata": {},
   "outputs": [],
   "source": [
    "import calendar\n",
    "calendar.isleap(2020)"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "Essentially, you can think of a function as a box. \n",
    "\n",
    "![Function black box diagram](assets/function-black-box.png)\n",
    "\n",
    "You put an input into the box (the input may be nothing), the box does something with the input, and then the box gives you back an output. You generally don't need to worry _how_ the function does what it does (unless you really want to, in which case you can look at its code). You just know that it works.\n",
    "\n",
    "> ***Functions are the basis of how we 'get stuff done' in Python.***\n",
    "\n",
    "For example, we can use the `requests` module to get the text of a Web page:"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": null,
   "metadata": {},
   "outputs": [],
   "source": [
    "import requests\n",
    "response = requests.get('https://www.wikipedia.org/')\n",
    "response.text[136:267]"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "The string `'https://www.wikipedia.org/'` is the argument we pass to the `get()` function for it to open the Web page and read it for us.\n",
    "\n",
    "Why not try your own URL? What happens if you print the whole of `response.text` instead of slicing out some of the characters?"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "---\n",
    "---\n",
    "## Summary\n",
    "\n",
    "Here's what we've covered - how to:\n",
    "\n",
    "* Create and manipulate a new list with a **list comprehension**.\n",
    "* Filter a list with a **condition**.\n",
    "* **import** a **module** to add new capabilities.\n",
    "* Run a **function** with parentheses.\n",
    "* Pass input **arguments** into a function."
   ]
  }
 ],
 "metadata": {
  "kernelspec": {
   "display_name": "Python 3 (ipykernel)",
   "language": "python",
   "name": "python3"
  },
  "language_info": {
   "codemirror_mode": {
    "name": "ipython",
    "version": 3
   },
   "file_extension": ".py",
   "mimetype": "text/x-python",
   "name": "python",
   "nbconvert_exporter": "python",
   "pygments_lexer": "ipython3",
   "version": "3.9.10"
  }
 },
 "nbformat": 4,
 "nbformat_minor": 1
}